Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests for async streaming (seems to work!) #301

Merged
merged 6 commits into from
Jun 2, 2023
Merged

Conversation

graebm
Copy link
Contributor

@graebm graebm commented Jun 2, 2023

  • Add basic tests for async streaming of the request body.
  • When async streaming, don't let more than 1 part at a time be in the "prepare" stage.
    • since aws_async_input_stream doesn't allow overlapping read() calls.
  • Stop wrapping synchronous streams in an async stream.
    • Now that we treat async streaming slightly different than synchronous, it's simpler to just have them be separate things.

Future work:

  • more tests
  • benchmarking / optimization

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…gs work more obvious.

The number of prepare_requests() calls to have in flight at a time is affected by whether we're ACTUALLY doing async streaming. Wrapping synchronous streams in an async stream, then having a separate bool to track whether it was ACTUALLY async was really confusing. So just have 2 separate streams, async and synchronous, and the one function that reads from them can just have an obvious if/else way of doing things.
@codecov-commenter
Copy link

codecov-commenter commented Jun 2, 2023

Codecov Report

Merging #301 (08db35f) into main (931d258) will decrease coverage by 0.09%.
The diff coverage is 93.75%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #301      +/-   ##
==========================================
- Coverage   85.97%   85.89%   -0.09%     
==========================================
  Files          17       17              
  Lines        4921     4928       +7     
==========================================
+ Hits         4231     4233       +2     
- Misses        690      695       +5     
Impacted Files Coverage Δ
source/s3_meta_request.c 92.71% <90.90%> (-0.73%) ⬇️
source/s3_request_messages.c 70.85% <95.00%> (-0.14%) ⬇️
source/s3_auto_ranged_put.c 93.25% <100.00%> (+0.15%) ⬆️

... and 2 files with indirect coverage changes

source/s3_meta_request.c Show resolved Hide resolved
tests/s3_tester.c Show resolved Hide resolved
@graebm graebm enabled auto-merge (squash) June 2, 2023 22:58
@graebm graebm merged commit f14d92f into main Jun 2, 2023
@graebm graebm deleted the async-stream-testing branch June 2, 2023 23:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants