PUSH_PROMISE ordering guarantee

[martinthomson]

HTTP/2 has a pretty strong recommendation to have PUSH_PROMISE precede any text that mentions it. Failing to do so could mean that a client requests a resource that was pushed, which is wasteful.

Splitting the headers and data into two streams means that there is no way to order the PUSH_PROMISE relative to the part of the body that mentions the resource.

I'm sure that @MikeBishop mentioned this once before, but I couldn't find any issues for it. Returning to a model where data and headers share a stream would fix this.

[MikeBishop]

Good point. Don't recall an issue on this specifically -- maybe it came up on the issue about whether to recombine them? Moot point until #176 is sorted out, though.

[krasic]

GCUIC didn't have headers and data in the same stream, it just had all headers on stream 3. It enforced that a application reader receive header bytes before body bytes, and honored the recommendation wrt PUSH_PROMISE ordering. I don't think that should change under two streams per HTTP request.

[martinthomson]

@krasic, GQUIC has exactly the same issue. The body of a response can mention a resource and appear before a PUSH_PROMISE.

Unless, by this:

It enforced that a application reader receive header bytes before body bytes

...you mean that the stack was required to withhold stream data until stream 3 explicitly released it. Yes, we could do the same with split headers and body, but we can't do that generically, you have to have an explicit marker in the headers stream that says "you may read the body now". That's equivalent to just having data on the same stream as headers, with a bunch of extra complexity stacked on.

@MikeBishop, #175 :)

[MikeBishop]

For a simple push application where the server generates HEADERS, PUSH_PROMISE, DATA*, that kind of works -- if the client knows to keep looking for PUSH_PROMISE frames as well. That effectively requires PUSH_PROMISE to be part of the header block.

For a more complex push application where the server generates HEADERS (PUSH_PROMISE? DATA*)*, there's no way for a client to know when it should return to pick up another PUSH_PROMISE. In the best case, it's just processing data as it arrives, and always treats bytes on the headers stream as higher-priority than bytes on the data stream. However, it's then subject to reordering -- or even partial arrival of a PUSH_PROMISE frame, given that QUIC doesn't know app-layer frame boundaries.

My unidirectional2 branch is just as vulnerable here, because it still allocates separate streams for each and keeps PUSH_PROMISE on the parent request's headers stream.

[krasic]

@martinthomson yes, I did mean that the stack withholds stream data until the headers have been consumed.

@MikeBishop Thanks again for a great explanation. I didn't think about promises interleaved with data after the initial headers. I agree with your assessment. FWIW, I don't think either GQUIC nor Google's server-side HTTP/2 actually support generating promises post initial headers, so in that sense they aren't fully HTTP/2 spec compliant. I can envision use cases for streaming pushes, but not without a bunch of other stuff fixed (like cached digests etc.).

[MikeBishop]

The primary case for us (as a server app platform) was a server generating and streaming back partial content as it continues to process. The HTTP/2 requirement is that the PUSH_PROMISE SHOULD come before the DATA or HEADERS frame which contains the reference, which isn't a hard requirement that PUSH_PROMISE precede all the data.

[krasic]

I think by far the most common use of push in the wild is where the pull stream contains rel=preload headers (or similar) in its response headers, and the push promises precede said response headers. At least I know it is status quo for us.

I'd be curious to know about use cases with different patterns. I don't have a clear mapping of how your "server generating and streaming partial content as it continues to process" actually looks wrt server push.

[kazuho]

@martinthomson

Returning to a model where data and headers share a stream would fix this.

To me it seems that sending PUSH_PROMISE as part of the parent stream is problematic regardless of whether or not headers and body are sent as separate streams.

Consider the case where a server is sending multiple responses that all refer to a single resource. This could happen for example when a client requests two HTML files to be shown in different iframes, and when the two HTML files refer to the same CSS file.

In such case, you would need to refer to the pushed stream from both of the responses. However, current design allows you to create a dependency to a pushed response from only one response. Therefore, we have a race condition if the client receives the responses of two or more streams (that both refer to a pushed resource) in a different order than the server sends.

To this end, I think that we should send PUSH_PROMISE using a different stream (possibly by using the stream that will convey the response headers), and within the parent stream only refer to that stream, instead of trying to convey the request headers of the pushed stream in the parent stream.

I understand that I am proposing to create dependency between streams. But I might argue that doing so (rather than trying to send PUSH_PROMISE in the parent stream) would benefit us in the long run, under the assumption that such dependency is expected to show up in application protocols other than HTTP as well.

[MikeBishop]

Interesting idea – or the request headers on their own steam, with unidirectional…. ?? Note, though, that this still doesn’t solve ordering of the promise and the reference. It actually makes it worse, since while you know a push is coming, you don’t know what, and you still need to order that reference with respect to the body fragments.

…

Sent from my Windows 10 phone From: Kazuho Oku<mailto:notifications@github.com> Sent: Thursday, June 8, 2017 7:09 PM To: quicwg/base-drafts<mailto:base-drafts@noreply.github.com> Cc: Mike Bishop<mailto:Michael.Bishop@microsoft.com>; Mention<mailto:mention@noreply.github.com> Subject: Re: [quicwg/base-drafts] PUSH_PROMISE ordering guarantee (#557) @martinthomson<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.luolix.top%2Fmartinthomson&data=02%7C01%7CMichael.Bishop%40microsoft.com%7C741ce3edd32b44556c8e08d4ae9118f5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636325385593951289&sdata=dvNGuieOUvByc7NRDjTXhrbAgEWvNS6IXZ5iy1Cz5OQ%3D&reserved=0> Returning to a model where data and headers share a stream would fix this. To me it seems that sending PUSH_PROMISE as part of the parent stream is problematic regardless of whether or not headers and body are sent as separate streams. Consider the case where a server is sending multiple responses that all refer to a single resource. This could happen for example when a client requests two HTML files to be shown in different iframes, and when the two HTML files refer to the same CSS file. In such case, you would need to refer to the pushed stream from both the responses. However, current design only allows you to create a dependency to a pushed response from one response. Therefore, we have a race condition if the client receives the responses of the two pulled streams in a different order than the server sends. To this end, I think that we should send PUSH_PROMISE using a different stream (possibly the stream that will convey the response headers), and within the parent stream only refer to that stream, instead of trying to convey the request headers of the pushed stream in the parent stream. I understand that I am proposing to create dependency between streams. But I might argue that doing so (rather than trying to send PUSH_PROMISE in the parent stream) would benefit us in the long run, under the assumption that such dependency is expected to show up in application protocols other than HTTP as well. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.luolix.top%2Fquicwg%2Fbase-drafts%2Fissues%2F557%23issuecomment-307167240&data=02%7C01%7CMichael.Bishop%40microsoft.com%7C741ce3edd32b44556c8e08d4ae9118f5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636325385593951289&sdata=0Oj7PimbNpPkLBNyPH%2BkmJ%2FFupC3UuYAPhpEn6%2F2c%2Fw%3D&reserved=0>, or mute the thread<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.luolix.top%2Fnotifications%2Funsubscribe-auth%2FAEE2hWhcm78budIyP44fl_9f4ZcuOsquks5sCCq7gaJpZM4Nl8yw&data=02%7C01%7CMichael.Bishop%40microsoft.com%7C741ce3edd32b44556c8e08d4ae9118f5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636325385593951289&sdata=uagbYjVXjL2YISiuiiUtlfeaH2IrhosHu1xJlmxC6Yw%3D&reserved=0>.

[kazuho]

Note, though, that this still doesn’t solve ordering of the promise and the reference. It actually makes it worse, since while you know a push is coming, you don’t know what, and you still need to order that reference with respect to the body fragments.

Thank you for the response. That is a good point.

I would argue that if we decide to covey the request headers of a pushed response outside the parent stream, we should 1) unify headers and body into single stream, and 2) create a HTTP frame (that is sent as part of the unified stream) that defines a dependency (i.e.tell the client to wait for the headers of a different stream). If we go this route, I think that it would not be "worse" at least.

PS. Or if decide to split headers and body into different streams, then we could define a dependency between the two using the same method as we will do for the pushed streams.

[MikeBishop]

Closed by #692.