-
Notifications
You must be signed in to change notification settings - Fork 2.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix issue 2349: Let async HiredisParser finish parsing after a Connection.disconnect() #2557
Conversation
3752994
to
78c81f4
Compare
Codecov ReportBase: 92.25% // Head: 92.28% // Increases project coverage by
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more Additional details and impacted files@@ Coverage Diff @@
## master #2557 +/- ##
==========================================
+ Coverage 92.25% 92.28% +0.02%
==========================================
Files 115 115
Lines 29692 29734 +42
==========================================
+ Hits 27393 27440 +47
+ Misses 2299 2294 -5
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
random test failure is random. |
626b72d
to
ed57208
Compare
@kristjanvalur Thanks for this PR! Can you please resolve the conflicts so I can merge it? |
Co-authored-by: Viktor Ivanov <viktor@infogrid.io> Co-authored-by: Sergey Prokazov <sergey.prokazov@redis.com> Co-authored-by: Anuragkillswitch <70265851+Anuragkillswitch@users.noreply.github.com> Co-authored-by: dvora-h <67596500+dvora-h@users.noreply.github.com> Co-authored-by: Alex Schmitz <aschmitz@box.com> Co-authored-by: Alex Schmitz <alex.schmitz@gmail.com> Co-authored-by: Chayim <chayim@users.noreply.github.com> Co-authored-by: Bar Shaul <88437685+barshaul@users.noreply.github.com> Co-authored-by: CrimsonGlory <CrimsonGlory@users.noreply.github.com> Co-authored-by: Raymond Yin <raymond@tryevergreen.com> Co-authored-by: zach.lee <zach.lee@sendbird.com> Co-authored-by: James R T <jamestiotio@gmail.com> Co-authored-by: dvora-h <dvora.heller@redis.com> Co-authored-by: Marc Schöchlin <marc.schoechlin@flipapp.de> Co-authored-by: Nick Gerow <nick.gerow@enlightedinc.com> Co-authored-by: Igor Malinovskiy <u.glide@gmail.com> Co-authored-by: Chayim I. Kirshen <c@kirshen.com> Co-authored-by: Leibale Eidelman <me@leibale.com> Co-authored-by: Thiago Bellini Ribeiro <hackedbellini@gmail.com> Co-authored-by: woutdenolf <woutdenolf@users.sf.net> Co-authored-by: shacharPash <93581407+shacharPash@users.noreply.github.com> Co-authored-by: Mirek Długosz <miniopl+github@gmail.com> Co-authored-by: Oran Avraham <252748+oranav@users.noreply.github.com> Co-authored-by: mzdehbashi-github <85902780+mzdehbashi-github@users.noreply.github.com> Co-authored-by: Tyler Hutcherson <tyler.hutcherson@redis.com> Co-authored-by: Felipe Machado <462154+felipou@users.noreply.github.com> Co-authored-by: AYMEN Mohammed <53928879+AYMENJD@users.noreply.github.com> Co-authored-by: Marc Schöchlin <ms-github@256bit.org> Co-authored-by: Avasam <samuel.06@hotmail.com> Co-authored-by: Markus Gerstel <2102431+Anthchirp@users.noreply.github.com> Co-authored-by: Kristján Valur Jónsson <sweskman@gmail.com> Co-authored-by: Nick Gerow <Nick.G.123@hotmail.com> Co-authored-by: Cristian Matache <cristianmatache@hotmail.com> Co-authored-by: Anurag Bandyopadhyay <angbpy@gmail.com> Co-authored-by: Seongchuel Ahn <aciddust20@gmail.com> Co-authored-by: Alibi <aliby.bbb@gmail.com> Co-authored-by: Smit Parmar <smitraj333@gmail.com> Co-authored-by: Brad MacPhee <macphee@gmail.com> Co-authored-by: Shahar Lev <shahar_lev@hotmail.com> Co-authored-by: Vladimir Mihailenco <vladimir.webdev@gmail.com> Co-authored-by: Kevin James <KevinJames@thekev.in> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: David Pacsuta <34983281+ant1fact@users.noreply.github.com> Co-authored-by: Rich Bowen <rbowen@rcbowen.com> Co-authored-by: gmbnomis <gmbnomis@users.noreply.github.com> Co-authored-by: Vivanov98 <66319645+Vivanov98@users.noreply.github.com> Co-authored-by: Kosuke <kosuke.zhang@gmail.com> Co-authored-by: Sergey Prokazov <prokazov@users.noreply.github.com> Co-authored-by: jmcbailey <jmcbailey@gmail.com> Co-authored-by: Galtozzy <14139502+Galtozzy@users.noreply.github.com> Co-authored-by: Abhishek Kumar Sinha <aksinha334@gmail.com> Co-authored-by: Eom Taegyung "Iggy <iggy.eom@sendbird.com> Co-authored-by: Mehdi ABAAKOUK <sileht@sileht.net> Co-authored-by: Dongkeun Lee <3315213+zakaf@users.noreply.github.com> Co-authored-by: woutdenolf <wout.de_nolf@esrf.eu> Co-authored-by: Kurt McKee <contactme@kurtmckee.org> Co-authored-by: Juraj Páll <palljuraj1@gmail.com> Co-authored-by: Joan Fontanals <jfontanalsmartinez@gmail.com> Co-authored-by: Stanislav Zmiev <zmievsa@gmail.com> fix (#2566) Fix unlink in cluster pipeline (#2562) Fix issue 2540: Synchronise concurrent command calls to single-client mode. (#2568) Fix: tuple function cannot be passed more than one argument (#2573) Fix issue 2567: NoneType check before raising exception (#2569) Fix issue 2349: Let async HiredisParser finish parsing after a Connection.disconnect() (#2557) Fix issue with `pack_commands` returning an empty byte sequence (#2416) Fix #2581 UnixDomainSocketConnection' object has no attribute '_command_packer' (#2583) Fix #2581 UnixDomainSocketConnection' object has no attribute '_command_packer' . Fix for `lpop` and `rpop` return typing (#2590) Fixed CredentialsProvider examples (#2587) Fixed issue #2598 - make Document class subscriptable fix: replace async_timeout by asyncio.timeout (#2602) Fix behaviour of async PythonParser to match RedisParser as for issue #2349 (#2582) Fix (#2641) fix: do not use asyncio's timeout lib before 3.11.2 (#2659) Fix issue 2660: PytestUnraisableExceptionWarning from asycio client (#2669) Fixing cancelled async futures (#2666) Fix async (#2673) Fix memory leak caused by hiredis (#2693) (#2694) Fix incorrect usage of once flag in async Sentinel (#2718) Fix topk list example. (#2724) Fix `ClusterCommandProtocol` not itself being marked as a protocol (#2729) Fix potential race condition during disconnection (#2719) fix CI (#2748) fix parse_slowlog_get (#2732) fixes for issue #1128 fix create single_connection_client from url (#2752) Fix `xadd` allow non negative maxlen (#2739) Fix JSON.MERGE Summary (#2786) Fixed key error in parse_xinfo_stream (#2788) Fix dead weakref in sentinel connection causing ReferenceError (#2767) (#2771) Fix dead weakref in sentinel conn (#2767) fix redirects and some small cleanups (#2801) Fix type hint for retry_on_error in async cluster (#2804) Fix CI (#2809) Fix async client with resp3 (#2657) Fix `COMMAND` response in resp3 (redis 7+) (#2740) Fix protocol version checking (#2737) Fix parse resp3 dict response: don't use dict comprehension (#2757) Fixing asyncio import (#2759) fix (#2799) fix async tests (#2806) Fix socket garbage collection (#2859) Fixing doc builds (#2869) Fix a duplicate word in `CONTRIBUTING.md` (#2848) Fix timeout retrying on Redis pipeline execution (#2812) Fix type hints in SearchCommands (#2817)
Pull Request check-list
Please make sure to review and check all of these items:
$ tox
pass with this change (including linting)?NOTE: these things are not required to open a PR and can be done
afterwards / while the PR is open.
Description of change
This change addresses an issue raised in issue #2349.
It ensures that an async
Connection.read_response()
call can succeed even if a simultaneousConnection.disconnect()
call is issued. Socket disconnects are graceful in that one closes the send direction and can then wait for any data
in transit to arrive, until an EOF is delivered from the remote side. In
asyncio
parlance this means that to gracefully close a socket, you close thesend
stream and can then read from thereceive
stream until you reach an End Of File.This change makes sure that the internal
BaseParser
state is maintained so that it can continue to request and parse chunks for as long as it can. Previously, the HiredisParser would get its internal state cleared immediately, and this would cause the read, in progress, to fail with an unexpected error. Now an ongoingread_response()
will continue until it succeeds or encounters an error such as EOF.A new
read_response()
issued after adisconnect()
call has been issued (but before it completes) will immediately fail, howerver. This is to maintain previous semantics, even though we could simply allow it to go through and fail later when it attemts to read from a closed connection.