HTTP/3: Create additional connections when maximum active streams is reached

[JamesNK]

I believe QUIC has a limit on the number of concurrent streams allowed. There should be a setting to allow the client to create multiple connections.

All the reasons specified by #35088 (issue for SocketsHttpHandler.EnableMultipleHttp2Connections) will apply to HTTP/3.

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

I believe QUIC has a limit on the number of concurrent streams allowed. There should be a setting to allow the client to create multiple connections.

All the reasons specified by #35088 (SocketsHttpHandler.EnableMultipleHttp2Connections) will apply to HTTP/3.

Author:	JamesNK
Assignees:	-
Labels:	area-System.Net.Http
Milestone:	-

[scalablecory]

Asking QUIC working group what the feeling is here.

Given that the scalability concerns of HTTP/2 have been addressed, I'm hoping servers will make use of QUIC's new stream allocation strategy to implement higher-by-default limits that are more reactive to current resource utilization. It'd be really interesting to experiment with this in YARP. There's really not a great reason to keep limits low and static anymore.

[geoffkizer]

Yeah, that's a good point. With QUIC and HTTP3 there shouldn't be the same need to create multiple connections.

[JamesNK]

It depends on what the maximum bidirectional QUIC streams is configured to in HttpClient and on the various HTTP/3 enabled servers available.

Do we know what msquic's default stream limits are?

[geoffkizer]

I assume ASP.NET is going to set its own stream limit, right? So msquic default doesn't apply.

[JamesNK]

We'll have a setting to change it, but we'd default it to whatever makes sense for QUIC and HTTP/3. I don't know what that is so we might use whatever msquic recommends.

@nibanks Do you have thoughts here?

[nibanks]

MsQuic defaults to zero. It's entirely up to the app to choose a default value. MsQuic cannot make any assumptions on how many streams are initially acceptable so the app must choose something.

[JamesNK]

Do you know what other QUIC HTTP/3 servers and clients typically set stream limits to? Is there any guidance?

Also, is there a performance impact of a lot of QUIC streams on one connection? With HTTP/2 performance would degrade significantly if there are hundreds of streams on one connection.

Basically, we're trying to figure out whether clients and servers to happily set bidirectional stream limit to int16.maxvalue and not worry about it, or if a client will need to support opening multiple connections when there is a lot of concurrent HTTP/3 requests.

[nibanks]

I do not know what other implementations do. I believe HTTP.sys uses the default in the spec (100 parallel streams). If you want, I can invite you to the QUIC Slack group and you can ask folks there.

As for perf, an increase in the number of streams will reduce performance, but how much really depends, and needs to come down to testing. We use hash tables and don't unduly traverse all the streams, but it doesn't mean it's "free" to have 10,000 parallel streams. Also, as we previously discussed, there are attack surfaces to consider with large limits.

[nibanks]

If you're trying to decide between something like opening 10 connections with 1000 streams each, or 1 connection with 10000 streams, network usage would be most efficient with a single connection, so I would tend to recommend that. But you need to measure perf in your scenarios.

[JamesNK]

If you want, I can invite you to the QUIC Slack group and you can ask folks there.

Yes please.

I believe HTTP.sys uses the default in the spec (100 parallel streams)

If 100 is the default in the spec then HttpClient will need to support creating more QUIC connections when it hits the server limit, as it does with HTTP/2.

[geoffkizer]

Here's what the RFC specifically says:

So as to not unnecessarily limit parallelism, at least 100 request streams SHOULD be permitted at a time.

See https://quicwg.org/base-drafts/draft-ietf-quic-http.html#name-bidirectional-streams

So it's not a default, it's a recommended lower bound. That said, I suspect in practice most servers will probably default to this.

[geoffkizer]

If you're trying to decide between something like opening 10 connections with 1000 streams each, or 1 connection with 10000 streams, network usage would be most efficient with a single connection, so I would tend to recommend that.

The other consideration here is how well the msquic implementation scales to 10000 streams on a single connections, vs 1000 streams on 10 connections (or whatever). Do you have any data re this, or if not, a gut feeling?

[nibanks]

@JamesNK invite sent. @geoffkizer I can run a modified RPS test with different numbers of connections, parallel requests, request size and response size. If you have a specific request, I can run it.

[nibanks]

I ran a couple of runs. All have 10,000 outstanding requests at all times over the number of connections. As you can see from the numbers, RPS goes up as you reduce the number of connections. I used 0 byte requests and 0 byte responses to really make stream frame (and not data) processing the most impactful usage of CPU. So, if you're willing to live with the greater attack surface exposing the larger stream FC give you, then I recommend a larger limit. But as always, please test with your own set up.

[nibanks]

One more set of data:

[geoffkizer]

That's great data, thanks.

[karelz]

Triage: We agree this will be likely needed.
We will need API (pattern of Http2).
Would be nice in 6.0, but can be moved to Future.

[ManickaP]

Did some experiments (prototype in https://github.com/ManickaP/runtime/tree/quic-stream-open-2) and as expected (from our tests with multiple clients) the RPS goes actually down (measured for 10 000 parallel requests):

Single Connection:

| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 36                                    |
| Max Cores usage (%)         | 1 016                                 |
| Max Working Set (MB)        | 5 284                                 |
| Max Private Memory (MB)     | 5 997                                 |
| Build Time (ms)             | 2 642                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 73 335                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.3.24157.5             |
| ASP.NET Core Version        | 9.0.0-preview.3.24156.20+449abac6f1ca |
| .NET Runtime Version        | 9.0.0-preview.3.24156.26+ed1e0abeb64a |
| Processor Count             | 28                                    |
| First request duration (ms) | 481                                   |
| Requests                    | 319 362                               |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 21 390                                |

Multiple Connections:

| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 56                                    |
| Max Cores usage (%)         | 1 578                                 |
| Max Working Set (MB)        | 6 413                                 |
| Max Private Memory (MB)     | 7 145                                 |
| Build Time (ms)             | 2 715                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 73 335                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.3.24157.4             |
| ASP.NET Core Version        | 9.0.0-preview.3.24156.20+449abac6f1ca |
| .NET Runtime Version        | 9.0.0-preview.3.24156.26+ed1e0abeb64a |
| Processor Count             | 28                                    |
| First request duration (ms) | 415                                   |
| Requests                    | 203 915                               |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 15 427                                |

At the moment, it doesn't make sense to implement this. I'll keep this open. Perhaps, when we fix the more glaring perf issues we can resurrect this.

[JamesNK]

A customer has run into this problem - grpc/grpc-dotnet#2404

They have more than 100 concurrent requests and this limitation is preventing new requests from being sent.