Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAX_STREAMS and flow control #85

Open
ekinnear opened this issue Oct 29, 2022 · 13 comments · May be fixed by #166
Open

MAX_STREAMS and flow control #85

ekinnear opened this issue Oct 29, 2022 · 13 comments · May be fixed by #166
Assignees
Labels
flow-control Issues blocked on flow control discussions has pull request Issues for which a pull request exists ietf-115 Issues discussed at IETF 115 ietf-116 Issues discussed at IETF 116 ietf-117 Issues discussed at IETF 117 ietf-118 Issues discussed at IETF 118

Comments

@ekinnear
Copy link
Collaborator

From #22 (comment):

figure out MAX_STREAMS, a way to limit the number of streams within a session

We've addressed the rest of the plan except for this part via the capsule DT, but there are still open questions about reordering of closed streams and their interaction with stream flow control limits via MAX_STREAMS.

@ekinnear
Copy link
Collaborator Author

ekinnear commented Feb 7, 2023

If reliable resets happen in QUIC, then it should be possible to model this after how QUIC MAX_STREAMS works, and we can just look at the biggest numbered stream.

@DavidSchinazi
Copy link
Collaborator

Chair: discussed at IETF 116. Consensus in room was to punt this unless someone proposes a solution. More specifically, we'll keep this issue open until IETF 117. If someone proposes a PR (or separate draft) that adds flow control, we will give them agenda time at IETF 117 and discuss whether to include it based on the details of the proposal. If no one has proposed anything by then, we will close the issue with no action.

@DavidSchinazi DavidSchinazi added ietf-116 Issues discussed at IETF 116 and removed ready for PR labels Mar 29, 2023
@DavidSchinazi
Copy link
Collaborator

Chair: discussed in editor's meeting. @ekinnear and @martinthomson wrote up a proposal. Action Item is with @vasilvv to review.

@martinthomson
Copy link
Contributor

https://www.ietf.org/archive/id/draft-thomson-webtrans-session-limit-00.html is the proposal. We might also want to consider the inclusion of the WebTransport-Init header field to govern initial values for those. And then, see ietf-wg-webtrans/draft-ietf-webtrans-http2#78 regarding settings to additionally govern these.

@DavidSchinazi DavidSchinazi added the ietf-117 Issues discussed at IETF 117 label Jul 18, 2023
@DavidSchinazi
Copy link
Collaborator

Discussed at editor's meeting: plan is to discuss at 117. @ekinnear can you make slides please?

@DavidSchinazi
Copy link
Collaborator

Chair: discussed at 117. Lots of discussion in the room, but consensus in room on the following:

  • we can't do pooling without flow control as that would cause starvation
  • we want JavaScript to not get measurable changes when the session is pooled vs not pooled
  • therefore, we can't disable flow control when pooling is disabled
  • this is hard to implement
  • implementers are focusing on other parts of WebTransport first
  • for now, we'll wait on more implementer experience
  • we will not publish the document without flow control

@ekinnear
Copy link
Collaborator Author

ekinnear commented Aug 8, 2023

Discussed at length in editor's meeting. @ekinnear to file an issue with the W3C. Needs more discussion here, too.

@vasilvv
Copy link
Collaborator

vasilvv commented Aug 22, 2023

I've been thinking about this a bit more, and I'm not sure draft-thomson-webtrans-session-limit actually solves the problem fully.

Consider the following situation. Imagine I have a connection where the client can open up to 100 streams (let's exclude actual HTTP requests for now), and it has 10 sessions. We want to avoid the situation where one session uses up all of the resources.

As far as I can tell, the nested flow control does not actually solve this problem. The server can tell the peer that it can accommodate 100 streams on every session, and that would be a perfectly valid thing to do (if, e.g., it's proxying to a backend, and backend's limit is also 100). The problem here is running into the connection-wide limit (that we already know).

Now, one possible scheme here would be to just do proportionate allocation. The browser sees it has 100 streams remaining, it can allocate 10 streams to each session.

Another option would be to separate stream limits for WebTransport sessions and stream limits for HTTP/3 stuff. I wrote a draft on one possible way to do this: https://vasilvv.github.io/draft-vvv-quic-namespaces/draft-vvv-quic-namespaces.html

@DavidSchinazi
Copy link
Collaborator

Chair: discussed in editor's meeting. Folks voiced concerns about various complex solutions to the flow control issue. There's interest in writing up a simple solution for now and allowing future extensions to be more complex if needed. @vasilvv has the action item to write up a minimal proposal that handles just enough flow control

@DavidSchinazi DavidSchinazi added the ietf-118 Issues discussed at IETF 118 label Oct 31, 2023
@vasilvv
Copy link
Collaborator

vasilvv commented Nov 6, 2023

So, the basic problem we are trying to solve here is as follows. Thanks to the QUIC flow control mechanisms, an endpoint has a limit on how many streams it can open. That limit is shared by multiple WebTransport sessions, as well as non-WebTransport streams (such as regular HTTP requests). The endpoint has to limit the number of streams in each session, and also limit number of concurrent HTTP requests so that they don't starve out WebTransport data streams.

We can model this limitation as

max_concurrent_streams = max_streams_per_session * max sessions + max_requests,

where we already know max_sessions from SETTINGS_WEBTRANSPORT_MAX_SESSIONS. We kind of know max_concurrent_streams from the flow control window, though that is not necessarily always the number we want, since the peer can use different strategies for updating window (e.g. start low and ramp it up exponentially).

My current proposal is to add a setting for WEBTRANSPORT_MAX_BIDI_STREAMS_HINT and MAX_TOTAL_BIDI_STREAMS_HINT, that tell us the values of max_streams_per_session and max_concurrent_streams; max_requests can be inferred from the formula above.

Worked example: assume that a server is willing to accept a max of 100 concurrent streams. It sends a MAX_STREAM_ID of 400 to indicate that; it also sends MAX_TOTAL_BIDI_STREAMS_HINT of 100 to indicate that this is what the client should expect going further. The server can set MAX_SESSIONS to 3, and WEBTRANSPORT_MAX_BIDI_STREAMS_HINT to 30, meaning that the three WebTransport sessions can consume a total of 90 streams, leaving a headroom of at least 10 streams to regular HTTP traffic.

Note that those are called "hints", because those are not directly enforced; if the client does not enforce those, some parts of the connection may starve the others, but this is not a concern since (1) all of those share the same origin, and (2) they still have to fit into the peer's overall limit, which is strictly enforced by the protocol.

The above mechanism can be extended to other types of streams, as well as MAX_STREAM_DATA (it's actually easier for some of those, since e.g. the only server-initiated bidi streams in H3 are from WebTransport).

@enygren
Copy link

enygren commented Nov 6, 2023

We should think about this from an adversarial perspective as well. Recent experience with HTTP/2 Rapid Reset (and similar) attacks demonstrate further how servers need to be able to bound their resource usage. We'll want to make sure servers can defend themselves against a range of creative attacks in a way that doesn't impair interoperability.

@DavidSchinazi
Copy link
Collaborator

Chair: discussed in the WG session at IETF 118. No consensus on a path forward at this stage. We heard:

  • we don't need flow control
  • we should do flow control hints
  • we need a reliable flow control solution

All of which happen to be incompatible.

We'll need to keep discussing this. The chairs will consider potentially setting up an interim meeting focused on this topic.

@ekinnear ekinnear added the flow-control Issues blocked on flow control discussions label Jan 23, 2024
@DavidSchinazi DavidSchinazi assigned ekinnear and unassigned vasilvv Mar 5, 2024
@ekinnear ekinnear added has pull request Issues for which a pull request exists and removed ready for PR labels Jul 21, 2024
@DavidSchinazi DavidSchinazi assigned vasilvv and unassigned ekinnear Aug 20, 2024
@DavidSchinazi
Copy link
Collaborator

Assigning to @vasilvv since @ekinnear has written the PR so the next step here is for @vasilvv to review #166.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flow-control Issues blocked on flow control discussions has pull request Issues for which a pull request exists ietf-115 Issues discussed at IETF 115 ietf-116 Issues discussed at IETF 116 ietf-117 Issues discussed at IETF 117 ietf-118 Issues discussed at IETF 118
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants