Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update parsing of headers to be infallible (by parsing failed UTF-8 headers as ISO-8859) #3830

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

landonxjames
Copy link
Contributor

@landonxjames landonxjames commented Sep 14, 2024

Motivation and Context

Some users have encountered an issue with where the SDK fails when parsing the content-disposition header from S3's GetObject because the header contain bytes for ISO-8859 encoded strings rather than the UTF-8 we were expecting. This is difficult to handle because the SDK has been designed to assume that all headers are UTF-8 and so they are all represented as Rust String types that only support UTF-8.

This PR implements fallback parsing where bytes that fail to parse as UTF-8 are instead parsed as ISO-8859 in an infallible way.

This is just meant as a jumping off point for discussions of how to handle this problem and should NOT BE MERGED without a group consensus.

Description

Update the TryFrom impls for HeaderMap from http_1x and http_02x to fallback to parsing headers as ISO-8859 when UTF-8 parsing fails. This is done by replacing bytes that are outside the 0-127 ASCII range with a two byte UTF-8 codepoint.

Testing

Added new tests confirming that ISO-8859 headers are correctly parsed to UTF-8.

Checklist

  • For changes to the smithy-rs codegen or runtime crates, I have created a changelog entry Markdown file in the .changelog directory, specifying "client," "server," or both in the applies_to key.
  • For changes to the AWS SDK, generated SDK code, or SDK runtime crates, I have created a changelog entry Markdown file in the .changelog directory, specifying "aws-sdk-rust" in the applies_to key.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Copy link

A new generated diff is ready to view.

A new doc preview is ready to view.

Copy link

A new generated diff is ready to view.

A new doc preview is ready to view.

Copy link

A new generated diff is ready to view.

A new doc preview is ready to view.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant