Provide opt-in to avoid request body duplication

[jakearchibald]

Once streams land, we want support in fetch. This issues tracks that and is a home for considerations.

[jakearchibald]

From @annevk in #whatwg:

If a redirect is hit, the request gets replayed to the new URL, this means that the request body gets buffered whether it needs to be or not. This may be undesirable/unworkable for large uploads, we may want to consider an option to fail-on-redirect, or prevent-buffering.

[tyoshino]

So, if we want to retry on such a failure on fetch() with fail-on-redirect: true, we need to build another Request object manually? fetch() sets used flag of the given Request.

[annevk]

@tyoshino I don't really see another way of preventing the duplication. You? Files will become very large going forward, or are we not concerned about that?

[tyoshino]

@annevk I'm not objecting to the idea. It makes sense. Just trying to understand manual retry story.

[annevk]

The idea would be that before you do something like this you ensure it's going to work. So network is up, you know the URL will not redirect, and then you start transferring this huge stream of data. If it does go wrong, it probably depends a bit on where the data came from originally how you want to recover.

[tyoshino]

@annevk OK. Thanks.

[tyoshino]

/cc @yutakahirano

[annevk]

So when I added redirect = "error" I forgot about this optimization.

The other avenue for a repeated request is HTTP authentication. Now normal HTTP authentication is not a problem as that is disabled in fetch() (you need to handle that yourself, building Authorization headers etc.). However, there is also HTTP proxy authentication. Opting out of that might be tricky or not desired. So I'd still need some input from network experts whether we can make the optimization and how that would affect HTTP proxy authentication.

[annevk]

(It might be better to move this issue to https://github.com/whatwg/fetch at this point... I'd like to stop using this repository for Fetch issues.)

[tyoshino]

On network with small RTT, it might be ok to:

• ask users who want to do streaming to return a response header asap
• have UAs refrain from forgetting sent data until it receives a response header

Can we use 100 continue or something to carry this kind of signal (It seems the final HTTP2 spec allows 100 continue)?

For large RTT network, it would be big loss if we don't start sending data until we receive a response (we shouldn't assume the underlying connection is TCP. Now we have HTTP2, etc.).

[annevk]

So that would be a protocol addition to enable something similar to this with the server controlling behavior. That might be good to support in due course and probably warrants a distinct ticket (against whatwg/fetch) and coordination with @mnot and cohorts.

However, my open question is about letting the client decide. That is, what should do about the situation where apart from proxy authentication we know no teeing would be needed?

[mnot]

It sounds like you're talking about expect/continue:
http://httpwg.github.io/specs/rfc7231.html#header.expect

[annevk]

Thanks! Indeed that seems sufficient for a negotiated version of this with some kind of opt-in flag passed to fetch(). I'd still be interested in hearing if we can avoid or fail on HTTP proxy authentication (407) so we can have a client-driven solution as well.

[annevk]

Opened two issues. One for the protocol-based version and one for the client decides version:

• Provide opt-in for Expect = "100-continue" whatwg/fetch#41
• Provide opt-out for HTTP proxy authentication? whatwg/fetch#42

Closing this one. Other Fetch stream concerns are tracked by this repository by the way:

• https://github.com/yutakahirano/fetch-with-streams