DOC-2544: Adding new doctest to support updated VSS article #2886

dwdougherty · 2023-08-09T20:36:06Z

Pull Request check-list

Please make sure to review and check all of these items:

Does $ tox pass with this change (including linting)?
Do the CI tests pass with this change (enable it first in your forked repo and wait for the github action build to finish)?
Is the new or changed code fully tested?
Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
Is there an example added to the examples folder (if applicable)?
Was the change added to CHANGES file?

Description of change

Adding a new doctest file to support an updated VSS article in the RediSearch repo.

`Union` was missing in front of `[List[StreamIdT], Tuple[StreamIdT]]` and VSCode was producing an error because of it. After adding `Union` the type annotation is correctly identified by VSCode.

* trivial typo fix * trivial typo fix

Implement unlink() like delete() to make it work when used in a cluster pipeline.

… mode. (#2568) Co-authored-by: Viktor Ivanov <viktor@infogrid.io>

* Implemented pack command and pack bytes * 1) refactored the command packer construction process 2) now hiredis.pack_bytes is the default choice. Though it's still possible to run redisrs-py (fix the flag in utils.py) or hiredis.pack_command (flag in connection.py) * Switch to hiredis.pack_command * Remove the rust extension module. * 1) Introduce HIREDIS_PACK_AVAILABLE environment variable. 2) Extract serialization functionality out of Connection class. * 1) Fix typo. 2) Add change log entry. 3) Revert the benchmark changes * Ditch the hiredis version check for pack_command. * Fix linter errors * Revert version changes * Fix linter issues * Looks like the current redis-py version is 4.4.1 --------- Co-authored-by: Sergey Prokazov <sergey.prokazov@redis.com>

Co-authored-by: Anuragkillswitch <70265851+Anuragkillswitch@users.noreply.github.com>

…tion.disconnect() (#2557) * A failing unittest * Do not clear the redis-reader's state when we disconnect so that it can finish reading the final message * Test that reading a message of two chunks after a disconnect() works. * Add Changes * fix typos

Co-authored-by: dvora-h <67596500+dvora-h@users.noreply.github.com>

…nd_packer' (#2583) * Fix #2581 UnixDomainSocketConnection' object has no attribute '_command_packer' . Apparently there is no end-to-end tests for Unix sockets so automation didn't catch it. I assume that setting up domain sockets reliably in dockerized environment is not very trivial. Added test for pack_command specifically. * Figuring out why CI fails. Locally: " congratulations :)" * Fix the test. hiredis doesn't treat memoryviews differently.

Right now there is an annoying warning that these methods can't be awaited when using `redis.asyncio`, even tho it does work with no problems.

* update json().arrindex() default values * add unit test * fix falsy checks * more unit tests * add asyncio tests * fix lint line length --------- Co-authored-by: Alex Schmitz <aschmitz@box.com>

* speeding up the protocol parser * linting * changes to ease

Co-authored-by: dvora-h <67596500+dvora-h@users.noreply.github.com>

* update return type of smismember * use Literal instead of int

* Fixed issue #2598 - make Document class subscriptable * Last time added older file, fixed it * retrigger checks * update json().arrindex() default values (#2611) * update json().arrindex() default values * add unit test * fix falsy checks * more unit tests * add asyncio tests * fix lint line length --------- Co-authored-by: Alex Schmitz <aschmitz@box.com> * Speeding up the protocol parsing (#2596) * speeding up the protocol parser * linting * changes to ease * Fixed CredentialsProvider examples (#2587) Co-authored-by: dvora-h <67596500+dvora-h@users.noreply.github.com> * ConnectionPool SSL example (#2605) * [types] update return type of smismember to list[int] (#2617) * update return type of smismember * use Literal instead of int * retrigger checks * Added test for document subscriptable in tests/test_search.py * Fixed linter issue * retrigger checks --------- Co-authored-by: Alex Schmitz <alex.schmitz@gmail.com> Co-authored-by: Alex Schmitz <aschmitz@box.com> Co-authored-by: Chayim <chayim@users.noreply.github.com> Co-authored-by: Bar Shaul <88437685+barshaul@users.noreply.github.com> Co-authored-by: dvora-h <67596500+dvora-h@users.noreply.github.com> Co-authored-by: CrimsonGlory <CrimsonGlory@users.noreply.github.com> Co-authored-by: Raymond Yin <raymond@tryevergreen.com>

async_timeout does not support python 3.11 aio-libs/async-timeout#295 And have two years old annoying bugs: aio-libs/async-timeout#229 #2551 Since asyncio.timeout has been shipped in python 3.11, we should start using it. Partially fixes 2551

* add queue_class to REDIS_ALLOWED_KEYS * fix lint * fix lint --------- Co-authored-by: zach.lee <zach.lee@sendbird.com> Co-authored-by: dvora-h <67596500+dvora-h@users.noreply.github.com>

…all super().__init__ (#2588) Co-authored-by: dvora-h <67596500+dvora-h@users.noreply.github.com>

…2349 (#2582) * Allow data to drain from PythonParser after connection close. * Add Changes

dwdougherty · 2023-08-11T17:44:39Z

Is there any way somebody can go ahead and merge this change? I'm currently playing whack-a-mole with the lint/syntax checker. It will be isolated (for now) on the emb-examples branch. I do need to get it in place so that the new VSS article can be merged on RediSearch.

doctests/search_vss.py

dwdougherty · 2023-08-14T21:07:29Z

@uglide Would you please take a look at this? You can see from the previous two commits that I'm being asked to change an import one way and, after I do, I'm being asked to change it back again. I can't move the RediSearch VSS article commit forward until this Python file is committed to emb-examples. Cc: @adrianoamaral .

chayim · 2023-08-16T06:25:08Z

This is occurring because everything is all sorts of out of date. I recommend:

Sync your emb-examples branch with the latest master
Sync this branch with emb-examples
Locally, run your invoke linters again.

You'll get a concise list of changes. The ones raised by flake8 will need to be made manually. However, running black and isort without their various --check arguments will reformat accordingly.

After doing this all locally - everything checks out.

dwdougherty · 2023-08-16T18:11:51Z

I followed @chayim 's instructions to the letter, including running flake8, isort, and black -l 80 locally before pushing. No joy.

dmaier-redislabs · 2023-08-30T18:05:07Z

@chayim Can you please merge this PR. This lint checker thing is blocking us from publishing the new VSS tutorial. Can we make this check somehow optional (warning, but not blocking the merge).

chayim · 2023-08-31T08:18:56Z

@chayim Can you please merge this PR. This lint checker thing is blocking us from publishing the new VSS tutorial. Can we make this check somehow optional (warning, but not blocking the merge).

I've fixed the PR. There were many missing pieces, as you can see from the multiple commits:

The linter was doing the thing it was programmed to do - at some point someone changed the linter to require 80 character columns (could have been me, but doesn't sound like me - if so sorry!).. something that isn't elsewhere. That was incompatible with the other linter, fixed.
There are various external requirements these examples now have - ones that don't exist in the library. CI now has a dedicated requirements file for this, as a result.
I've updated the README to indicate how one can contribute.

IMHO the example text somewhere else now require updates indicating that to use these examples you need to install external dependencies. I'll leave that with you.

@uglide anything else here?

chayim · 2023-08-31T09:00:39Z

@uglide Got it. Restored. So folks here's the change.

Regarding #1 above, isort has its own character length rules... since apparently we're not leveraging what's already in use by redis-py (because branch). For now, I've updated the action - they do the same thing. Squash + merge.

dwdougherty · 2023-08-31T13:23:08Z

Thank you @uglide and @chayim ! I'm very sorry to have caused so much pain.

ant1fact and others added 30 commits January 22, 2023 20:41

Add missing Union type in method StreamCommands.xclaim() (#2553)

5e258a1

`Union` was missing in front of `[List[StreamIdT], Tuple[StreamIdT]]` and VSCode was producing an error because of it. After adding `Union` the type annotation is correctly identified by VSCode.

Simplify the sync SocketBuffer, add type hints (#2543)

e39c7ba

trivial typo fix (#2566)

42604b6

* trivial typo fix * trivial typo fix

Fix unlink in cluster pipeline (#2562)

9e6a9b5

Implement unlink() like delete() to make it work when used in a cluster pipeline.

Fix issue 2540: Synchronise concurrent command calls to single-client…

428d609

… mode. (#2568) Co-authored-by: Viktor Ivanov <viktor@infogrid.io>

Fix: tuple function cannot be passed more than one argument (#2573)

31a1c0b

Fix issue 2567: NoneType check before raising exception (#2569)

9e00b91

Co-authored-by: Anuragkillswitch <70265851+Anuragkillswitch@users.noreply.github.com>

Add TS.MGET example for OS Redis Cluster (#2507)

fcd8f98

Fix issue with pack_commands returning an empty byte sequence (#2416)

f517287

Co-authored-by: dvora-h <67596500+dvora-h@users.noreply.github.com>

Version 4.5.0 (#2580)

5cb5712

Version 4.5.1 (#2586)

fd7a79d

Fix for lpop and rpop return typing (#2590)

e9ad2a3

Right now there is an annoying warning that these methods can't be awaited when using `redis.asyncio`, even tho it does work with no problems.

Update README to make pip install copy-pastable on zsh (#2584)

6c708c2

update json().arrindex() default values (#2611)

b546a9a

* update json().arrindex() default values * add unit test * fix falsy checks * more unit tests * add asyncio tests * fix lint line length --------- Co-authored-by: Alex Schmitz <aschmitz@box.com>

Speeding up the protocol parsing (#2596)

5588ae0

* speeding up the protocol parser * linting * changes to ease

Fixed CredentialsProvider examples (#2587)

3edd49b

Co-authored-by: dvora-h <67596500+dvora-h@users.noreply.github.com>

ConnectionPool SSL example (#2605)

6d1061f

[types] update return type of smismember to list[int] (#2617)

a372ba4

* update return type of smismember * use Literal instead of int

Remove redundant assignment. (#2620)

91ab12a

Adding supported redis/library details (#2621)

c61eeb2

add queue_class to REDIS_ALLOWED_KEYS (#2577)

d63313b

* add queue_class to REDIS_ALLOWED_KEYS * fix lint * fix lint --------- Co-authored-by: zach.lee <zach.lee@sendbird.com> Co-authored-by: dvora-h <67596500+dvora-h@users.noreply.github.com>

pypy-3.9 CI (#2608)

c871723

introduce AbstractConnection so that UnixDomainSocketConnection can c…

7d474f9

…all super().__init__ (#2588) Co-authored-by: dvora-h <67596500+dvora-h@users.noreply.github.com>

Fix behaviour of async PythonParser to match RedisParser as for issue #…

1b2f408

…2349 (#2582) * Allow data to drain from PythonParser after connection close. * Add Changes

Version 4.5.2 (#2627)

318b114

chayim and others added 3 commits August 9, 2023 15:25

Version 5.0.0 (#2874)

2f67926

DOC-2544: Adding new doctest to support updated VSS article

4e4ff48

Updating all client licenses to clearly be MIT (#2884)

28cc65c

adrianoamaral approved these changes Aug 11, 2023

View reviewed changes

dwdougherty added 2 commits August 11, 2023 07:04

DOC-2554: update import order

e680924

DOC-2544: update formatting

b3a92c4

dmaier-redislabs approved these changes Aug 11, 2023

View reviewed changes

dwdougherty mentioned this pull request Aug 11, 2023

DOC-2544: replace VSS tutorial and move orig. content RediSearch/RediSearch#3760

Merged

2 tasks

dwdougherty commented Aug 14, 2023

View reviewed changes

doctests/search_vss.py Show resolved Hide resolved

DOC-2544: Update import (again)

b42d19a

dwdougherty and others added 2 commits August 16, 2023 10:36

Merge branch 'redis:master' into doc-2544

724807a

DOC-2544: one more attempt using Chayims advice

d23058a

chayim added 6 commits August 31, 2023 10:21

lint fixes

b8372bd

and a reqs file

5f50fdc

another missing requirement

4016a67

and sentence transformers

ce0f076

and the optional, unlisted dependency tabulate. Thanks conda

d5b42af

Updating README for doctests howto

8dde72a

chayim added 2 commits August 31, 2023 11:32

align isort with black

30c1179

typo

894a4b6

chayim added the maintenance Maintenance (CI, Releases, etc) label Aug 31, 2023

chayim merged commit 0e56165 into redis:emb-examples Aug 31, 2023
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DOC-2544: Adding new doctest to support updated VSS article #2886

DOC-2544: Adding new doctest to support updated VSS article #2886

dwdougherty commented Aug 9, 2023

dwdougherty commented Aug 11, 2023

dwdougherty commented Aug 14, 2023 •

edited

Loading

chayim commented Aug 16, 2023

dwdougherty commented Aug 16, 2023

dmaier-redislabs commented Aug 30, 2023

chayim commented Aug 31, 2023 •

edited

Loading

chayim commented Aug 31, 2023

dwdougherty commented Aug 31, 2023

DOC-2544: Adding new doctest to support updated VSS article #2886

DOC-2544: Adding new doctest to support updated VSS article #2886

Conversation

dwdougherty commented Aug 9, 2023

Pull Request check-list

Description of change

dwdougherty commented Aug 11, 2023

dwdougherty commented Aug 14, 2023 • edited Loading

chayim commented Aug 16, 2023

dwdougherty commented Aug 16, 2023

dmaier-redislabs commented Aug 30, 2023

chayim commented Aug 31, 2023 • edited Loading

chayim commented Aug 31, 2023

dwdougherty commented Aug 31, 2023

dwdougherty commented Aug 14, 2023 •

edited

Loading

chayim commented Aug 31, 2023 •

edited

Loading