You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When writing an HTTP server with hyper, I want to be able to receive and read the chunk extensions that clients send when using HTTP/1.1 chunked encoding.
Currently hyper simply discards these chunk extensions.
Motivating use case
Implementing an S3-compatible server using hyper would require this functionality. Amazon S3 Signature Version 4 includes a "streaming mode" where the SHA256 signature of each chunk is included as a chunk extension:
The server is required to validate the signature of each chunk as it is sent, so it needs to be able to extract the chunk signatures from the HTTP chunk extensions.
Then the Incoming body can yield a Frame::chunk_extensions(..) before each Frame::data(..) (or rather, before the first frame in each chunk?).
This seems like the least invasive option, but might not really solve the problem sufficiently (see Limitations below).
Limitations
Ordering
In order to implement the S3 chunked signature, the server has to be able to associate each set of chunk extensions with the chunk that it precedes.
However, the Body trait provides no guarantee about the order of different types of Frames that are yielded. So programmers would have to trust that any Body is going to yield chunk extensions before each chunk. But theoretically somebody could implement Body in a way that subverts that expectation.
Perhaps this is not truly an issue, but rather an "understood quirk" of the Body trait. I say this because a similar problem exists with the chunked trailers. The Body trait does not document any guarantee that you'll receive at most one Frame::trailers(..), nor that that frame will be the last frame yielded. But I'd wager that users of a Body might make one or more of those assumptions.
Sending
It's not clear how the hyper client code should interpret any Frame::chunk_extensions(..) that are yielded by the request Body 🤔 🤔 🤔 .
Idea 2: Callbacks
Maybe the hyper server code could allow callers to specify some sort of callback that the Dispatch will call whenever it encounters chunk extensions. Haven't thought this idea through very much.
Alternatives considered
I can't think of any. The processing of the chunk extensions is completely hidden away in hyper's guts and cannot be customized by end users.
Any thoughts on other ways to expose this functionality?
The text was updated successfully, but these errors were encountered:
Thanks for the write up! Yea, chunked extensions are a part of HTTP/1.1 that sounded good in the spec, but few implementations bothered to support. And since it's allowed to ignore them, any additional hops (proxies, gateways, load balancers) will likely strip them too.
We could try to figure out how to support them, but I genuinely think that part of S3 should be ignored, and a better supported mechanism should be used to checksum the payload.
Requested functionality
When writing an HTTP server with hyper, I want to be able to receive and read the chunk extensions that clients send when using HTTP/1.1 chunked encoding.
Currently hyper simply discards these chunk extensions.
Motivating use case
Implementing an S3-compatible server using hyper would require this functionality. Amazon S3 Signature Version 4 includes a "streaming mode" where the SHA256 signature of each chunk is included as a chunk extension:
Example Header:
Example chunk from payload:
The server is required to validate the signature of each chunk as it is sent, so it needs to be able to extract the chunk signatures from the HTTP chunk extensions.
Idea 1: New frame type
Define a new
Frame
variant for chunk extensions.Then the
Incoming
body can yield aFrame::chunk_extensions(..)
before eachFrame::data(..)
(or rather, before the first frame in each chunk?).This seems like the least invasive option, but might not really solve the problem sufficiently (see Limitations below).
Limitations
Ordering
In order to implement the S3 chunked signature, the server has to be able to associate each set of chunk extensions with the chunk that it precedes.
However, the
Body
trait provides no guarantee about the order of different types ofFrame
s that are yielded. So programmers would have to trust that anyBody
is going to yield chunk extensions before each chunk. But theoretically somebody could implementBody
in a way that subverts that expectation.Perhaps this is not truly an issue, but rather an "understood quirk" of the
Body
trait. I say this because a similar problem exists with the chunked trailers. TheBody
trait does not document any guarantee that you'll receive at most oneFrame::trailers(..)
, nor that that frame will be the last frame yielded. But I'd wager that users of aBody
might make one or more of those assumptions.Sending
It's not clear how the hyper client code should interpret any
Frame::chunk_extensions(..)
that are yielded by the requestBody
🤔 🤔 🤔 .Idea 2: Callbacks
Maybe the hyper server code could allow callers to specify some sort of callback that the
Dispatch
will call whenever it encounters chunk extensions. Haven't thought this idea through very much.Alternatives considered
I can't think of any. The processing of the chunk extensions is completely hidden away in hyper's guts and cannot be customized by end users.
Any thoughts on other ways to expose this functionality?
The text was updated successfully, but these errors were encountered: