Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: add a check for partial response data #435

Merged
merged 4 commits into from
May 31, 2024
Merged

fix: add a check for partial response data #435

merged 4 commits into from
May 31, 2024
+128 −21

Conversation

cojenco
Copy link
Contributor

@cojenco cojenco commented May 14, 2024 •
edited
Loading

Handle corner cases of incomplete downloads with a 200 response code. These are rare cases when a download is prematurely terminated due to network stability or bandwidth throttling etc, but does not result in connection reset errors. The library will

  • (1) compare the # bytes read vs x-goog-stored-content-length header
  • (2) raise a retry-able ConnectionError in the aim to trigger a retry

Add more context to checksum mismatch error message that will help with debugging

  • bytes read from the download request (tracked within the Download object)
  • X-Goog-Stored-Content-Length value from the response header
  • X-Goog-Stored-Content-Encoding value from the response header

Fixes internal b/278011003

@cojenco cojenco requested review from a team as code owners May 14, 2024 21:41
@product-auto-label product-auto-label bot added the size: m Pull request size is medium. label May 14, 2024
@product-auto-label product-auto-label bot added the api: storage Issues related to the googleapis/google-resumable-media-python API. label May 14, 2024
@cojenco cojenco added the owlbot:run Add this label to trigger the Owlbot post processor. label May 14, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 14, 2024
@cojenco cojenco added the owlbot:run Add this label to trigger the Owlbot post processor. label May 15, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 15, 2024
@cojenco cojenco added the owlbot:run Add this label to trigger the Owlbot post processor. label May 15, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 15, 2024
@cojenco cojenco changed the title chore: update checksum mismatch error message with additional context fix: add a check for partial response data May 15, 2024
@cojenco cojenco added the owlbot:run Add this label to trigger the Owlbot post processor. label May 16, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 16, 2024
@cojenco cojenco requested a review from andrewsg May 30, 2024 23:12
andrewsg
andrewsg approved these changes May 30, 2024
View reviewed changes
Copy link
Contributor

@andrewsg andrewsg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@danielduhh
Copy link

Is this good to merge? @cojenco

@cojenco cojenco merged commit aaea392 into googleapis:main May 31, 2024
17 checks passed
@release-please release-please bot mentioned this pull request May 31, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewsg andrewsg andrewsg approved these changes

Assignees

@bajajneha27 bajajneha27

Labels
api: storage Issues related to the googleapis/google-resumable-media-python API. size: m Pull request size is medium.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@cojenco @danielduhh @andrewsg @bajajneha27 @sydney-munro