Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Object Store: Allow custom reqwest client #3127

Closed
freeformstu opened this issue Nov 17, 2022 · 1 comment · Fixed by #3256
Closed

Object Store: Allow custom reqwest client #3127

freeformstu opened this issue Nov 17, 2022 · 1 comment · Fixed by #3256
Labels
enhancement Any new improvement worthy of a entry in the changelog object-store Object Store Interface

Comments

@freeformstu
Copy link

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

I am connecting to our Ceph cluster and getting intermittent connection closed errors from the server. I have tried both multipart and non-multipart uploads and have had the same issues.

Others have run into this issue with the S3 API with reqwest/hyper and have needed to change their client configuration to resolve the issue.
See:

Describe the solution you'd like

I would like to expose the reqwest client for custom configuration.

Describe alternatives you've considered

An alternative would be to expose individual configuration settings in the Reqwest client and continue to shield creation of the client from the end user. This seems like unnecessary complexity for the object store to expose.

Another alternative would be to make the reqwest client public so that it can be modified after the S3Client has been created. Exposing the client in this way is bad API design. Keeping the client private while allowing initial configuration seems to me to be the best route.

Additional context

@freeformstu freeformstu added the enhancement Any new improvement worthy of a entry in the changelog label Nov 17, 2022
tustvold added a commit to tustvold/arrow-rs that referenced this issue Dec 2, 2022
tustvold added a commit to tustvold/arrow-rs that referenced this issue Dec 2, 2022
tustvold added a commit to tustvold/arrow-rs that referenced this issue Dec 2, 2022
alamb pushed a commit that referenced this issue Dec 2, 2022
…#3256)

* Add more ClientConfig Options (#3127)

* Add header support
@alamb alamb added the object-store Object Store Interface label Dec 9, 2022
@alamb
Copy link
Contributor

alamb commented Dec 9, 2022

label_issue.py automatically added labels {'object-store'} from #3252

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Any new improvement worthy of a entry in the changelog object-store Object Store Interface
Projects
None yet
2 participants