Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: due to buffering in the underlying stream reads are limited in size #25

Merged
merged 1 commit into from
Jan 16, 2024
Merged

fix: due to buffering in the underlying stream reads are limited in size #25

merged 1 commit into from
Jan 16, 2024
+357 −113

Commits on Jan 16, 2024

  1. fix: due to buffering in the underlying stream reads are limited in size 

    Currently if you call `Read` and provide a buffer larger than a few k the stream returns only that few k, given the reader abstraction discards the stream between `Read` operations this results in many API calls.

To mitigate this I moved back to retaining the stream from an optimistic get object which is configured when the file is opened, this then is used by subsquent read calls.

This PR adds tests that illustrate the issue.

One thing to note is using `Seek` closes the initial get object stream as this doesn't support these operations, and instead uses `ReadAt`, which results in more API calls.
    @wolfeidau
    wolfeidau committed Jan 16, 2024
    Configuration menu
    Copy the full SHA
    1e05bc6 View commit details
    Browse the repository at this point in the history