Update parsing of headers to be infallible (by parsing failed UTF-8 headers as ISO-8859) #3830
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Some users have encountered an issue with where the SDK fails when parsing the
content-disposition
header from S3'sGetObject
because the header contain bytes for ISO-8859 encoded strings rather than the UTF-8 we were expecting. This is difficult to handle because the SDK has been designed to assume that all headers are UTF-8 and so they are all represented as RustString
types that only support UTF-8.This PR implements fallback parsing where bytes that fail to parse as UTF-8 are instead parsed as ISO-8859 in an infallible way.
This is just meant as a jumping off point for discussions of how to handle this problem and should NOT BE MERGED without a group consensus.
Description
Update the
TryFrom
impls forHeaderMap
fromhttp_1x
andhttp_02x
to fallback to parsing headers as ISO-8859 when UTF-8 parsing fails. This is done by replacing bytes that are outside the0-127
ASCII range with a two byte UTF-8 codepoint.Testing
Added new tests confirming that ISO-8859 headers are correctly parsed to UTF-8.
Checklist
.changelog
directory, specifying "client," "server," or both in theapplies_to
key..changelog
directory, specifying "aws-sdk-rust" in theapplies_to
key.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.