Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add support for flexible checksums on streaming payloads #1329

Merged
merged 72 commits into from
Mar 21, 2024

Conversation

dayaffe
Copy link
Contributor

@dayaffe dayaffe commented Feb 7, 2024

Issue #

#1319

Description of changes

  • enables support for checksums on streaming requests
  • enables chunked encoding on clients that support checksums
  • Supported HTTP Clients: URLSession, CRT

Also:

  • Exclude the Transfer-Encoding header from AWS signature calculations, since it is sometimes modified superficially by the HTTP client immediately before sending a HTTP request, causing a signature mismatch.

New/existing dependencies impact assessment, if applicable

No new dependencies were added to this change.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@dayaffe dayaffe had a problem deploying to Codegen-Build-Test February 7, 2024 20:35 — with GitHub Actions Failure
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:35 — with GitHub Actions Inactive
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:35 — with GitHub Actions Inactive
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:35 — with GitHub Actions Inactive
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:35 — with GitHub Actions Inactive
@dayaffe dayaffe had a problem deploying to Codegen-Build-Test February 7, 2024 20:47 — with GitHub Actions Failure
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:47 — with GitHub Actions Inactive
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:47 — with GitHub Actions Inactive
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:47 — with GitHub Actions Inactive
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:47 — with GitHub Actions Inactive
@dayaffe dayaffe had a problem deploying to Codegen-Build-Test February 7, 2024 20:52 — with GitHub Actions Failure
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:52 — with GitHub Actions Inactive
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:52 — with GitHub Actions Inactive
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:52 — with GitHub Actions Inactive
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:52 — with GitHub Actions Inactive
@dayaffe dayaffe temporarily deployed to Codegen-Build-Test February 7, 2024 20:55 — with GitHub Actions Inactive
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:55 — with GitHub Actions Inactive
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:55 — with GitHub Actions Inactive
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:55 — with GitHub Actions Inactive
@dayaffe dayaffe temporarily deployed to Integration-Test February 7, 2024 20:55 — with GitHub Actions Inactive
@dayaffe dayaffe requested review from jbelkins and sichanyoo February 8, 2024 20:41
@dayaffe dayaffe had a problem deploying to Codegen-Build-Test February 8, 2024 21:14 — with GitHub Actions Failure
@jbelkins jbelkins temporarily deployed to Integration-Test March 13, 2024 22:45 — with GitHub Actions Inactive
@jbelkins jbelkins had a problem deploying to Codegen-Build-Test March 13, 2024 22:45 — with GitHub Actions Failure
@jbelkins jbelkins had a problem deploying to Codegen-Build-Test March 13, 2024 22:50 — with GitHub Actions Failure
@jbelkins jbelkins had a problem deploying to Codegen-Build-Test March 13, 2024 23:24 — with GitHub Actions Failure
@jbelkins jbelkins had a problem deploying to Codegen-Build-Test March 14, 2024 23:33 — with GitHub Actions Failure
@jbelkins jbelkins had a problem deploying to Codegen-Build-Test March 14, 2024 23:35 — with GitHub Actions Failure
@jbelkins jbelkins self-requested a review March 15, 2024 20:37
Copy link
Contributor

@jbelkins jbelkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have fixed several issues in this feature, both in this PR and in smithy-lang/smithy-swift#659.

Some refactoring & performance improvements are required, but will be done in a separate PR.

// shouldSignHeaders block was not provided, then include all headers other
// than Transfer-Encoding.
let modifiedShouldSignHeader = { (name: String) in
guard name.lowercased(with: Locale(identifier: "en_US_POSIX")) != "transfer-encoding" else { return false }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the reason for checking en_US_POSIX locale?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you don't specify the locale for the date & string operations that are locale-dependent, then you get the locale that the user sets as default on their device. And that may not result in lowercasing like you expected.

(I have had this happen with datetime stamps in the past, not with lowercasing, but the same risk is present)

@jbelkins jbelkins merged commit 6ecadb2 into main Mar 21, 2024
17 checks passed
@jbelkins jbelkins deleted the day/aws-chunked-encoding branch March 21, 2024 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants