Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

REST Request can be sped up a great deal by keeping an SSL context #631

Closed
serge-rgb opened this issue Mar 20, 2024 · 3 comments
Closed
Labels
bug Something isn't working

Comments

@serge-rgb
Copy link
Contributor

Describe the bug

I am developing an app with Polygon as the market data provider. I need to make a high number of REST requests so I use many threads. I noticed a big performance impact so I did some profiling (see screenshot below)

I used a C/C++ profiler, so the Python part of the callstack is obscured, but you can see in the expanded call stack that the app is spending most of its time reading SSL certificate files.

I found a fix that passes a ssl_context to the PoolManager when creating the base client. This greatly increased the performance of the REST client. I'm opening this issue to submit a pull request.

To Reproduce

Create a python app that makes many REST requests and look at the CPU usage spike

Expected behavior

Better perf

Screenshots
Profiler_aoZobjKaLS

@serge-rgb
Copy link
Contributor Author

Here's a screenshot of the profiler showing the same workload with the fix.

Profiler_YJNqnq793E

@justinpolygon
Copy link
Contributor

This is great @serge-rgb. I'll check it out. Thanks for submitted this with the detailed explanation, plots, and event a patch!

@justinpolygon
Copy link
Contributor

Thanks again for contributing this @serge-rgb. We just rolled out a new release with your change https://github.com/polygon-io/client-python/releases/tag/v1.13.5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants