-
-
Notifications
You must be signed in to change notification settings - Fork 839
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
KeyError: HTTPConnection(origin=...) on ReadTimeout in connection_pool #817
Comments
I've made a repo that can usually reproduced the issue: https://github.com/HENNGE/httpx-h2-testing When the client tries to send ~1000 requests the KeyError usually happens. |
Okay, thanks for the report. I've been taking a refresh onto the connection pool handling in httpcore... https://github.com/encode/httpcore/blob/master/httpcore/_async/connection_pool.py Might be that I need to prioritize working on that again, in order to get this, #283, and #514 resolved. |
The example I made indeed also shows #514. I'd expect it to use a single connection (or maybe 1 connection per 100 requests since that's the stream limit). Also with different parameters (eg issue 10k requests) it also exhausts the per-connection stream limit. In our real-world scenario, we don't currently hit the second issue, but the first two (KeyError and lots of connections) do happen and we're also seeing some sort of memory leak (but haven't been able to track this down to httpx or our app yet). Let me know if there's more information you need or if there are other ways I can help. |
I've been poking at @ojii 's test and I think (not fully sure yet) that
If that's correct, then the problem is in cancellation and not on the happy path. |
Having instrumented connection pool / connection store:
I get log:
This is the traceback when
So, basically, the |
Hi here, the |
I'm going to close this on the assumption that its resolved in the 0.13.dev0 pre-release, given that we've totally refreshed the pool handling, and it's a much more careful implementation. |
In some cases, when
httpx._dispatch.connection_pool:ConnectionPool.send
fails, the call tohttpx._dispatch.connection_pool:ConnectionStore.remove
fails because the connection somehow is no longer inConnectionStore.all
.This is in an http2-only context.
Python: 3.8.1
httpx: 0.11.1
The text was updated successfully, but these errors were encountered: