Get tests running for HTTP/3 #31898

scalablecory · 2020-02-07T00:28:51Z

This PR gets a minimal set of tests working, but not all of them. It enables all HTTP/3 tests, even the not yet working ones, if you have MsQuic support. Our CI does not have MsQuic support, so the failing tests will not cause CI failures.

Update generic tests to use a Version rather than boolean IsHttp11/IsHttp20, and update some HTTP/2 to work for HTTP/3.
Enable tests for HTTP/3, behind a conditional feature test.
Fix QPackDecoder lzcnt assuming an 8-bit test.
Rename test QPACK classes from QPackEncoder/QPackDecoder -> QPackTestEncoder/QPackTestDecoder to avoid naming confusion with product code classes.
Fix QPackTestDecoder bit flag checks.
Fix a double call to QuicConnection.CloseAsync(). Update to shutdown QuicConnection in a background task.

jkotalik · 2020-02-11T18:36:12Z

src/libraries/Common/src/System/Net/Http/Http3/QPack/QPackDecoder.cs

@@ -220,7 +220,7 @@ private void OnByte(byte b, IHttpHeadersHandler handler)
                    }
                    break;
                case State.CompressedHeaders:
-                    switch (BitOperations.LeadingZeroCount(b))
+                    switch (BitOperations.LeadingZeroCount(b) - 24)


Definitely add a comment for this.

jkotalik · 2020-02-11T18:36:42Z

src/libraries/Common/tests/System/Net/Configuration.Certificates.cs

@@ -69,6 +71,40 @@ static Certificates()
            public static X509Certificate2 GetNoEKUCertificate() => new X509Certificate2(s_noEKUCertificate);
            public static X509Certificate2 GetSelfSignedServerCertificate() => new X509Certificate2(s_selfSignedServerCertificate);
            public static X509Certificate2 GetSelfSignedClientCertificate() => new X509Certificate2(s_selfSignedClientCertificate);
+
+            public static X509Certificate2 GetTls13ServerCertificate()


Ooo, I should try using this in Kestrel as well 😄

jkotalik · 2020-02-11T18:38:28Z

src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/Http3Connection.cs

+                }
+
+                // 0-byte write to force QUIC to allocate a stream ID.
+                await quicStream.WriteAsync(Array.Empty<byte>(), cancellationToken).ConfigureAwait(false);


Here a few thoughts I have:

What if we make quicStream.StreamId nullable and return null if the stream hasn't started yet?

Why do you need to set the Http3RequestStream.StreamId here? Can you just always call into quicStream.StreamId to get the streamId?

Finally, to track goaway, I think we may need to move this to after first write rather than here.

What if we make quicStream.StreamId nullable and return null if the stream hasn't started yet?

Having StreamId populated late is an unfortunate usability issue; I'd rather it just be usable immediately after construction if it's reasonable to do so. I think we need to better understand this from msquic perspective; my feeling is that it is behaving like this (presumably) as a corner-case optimization to avoid allocating unused stream IDs, as QUIC "allocates" gaps in stream IDs it sees and counts them against the maximum. @nibanks do you have any background here?

Why don't you start it in your constructor then? You don't need to delay starting only when you need to send on it.

Unless I remember incorrectly, you said that starting a stream really should be async.

Before the first write or send, there is no async operations in between. We could add a StartAsync method to the stream that's called immediately after creating the stream.

jkotalik

General thoughts, but I don't think there is any reason to block this.

halter73 · 2020-02-12T22:36:19Z

src/libraries/Common/src/System/Net/Http/Http3/QPack/QPackDecoder.cs

@@ -224,7 +224,7 @@ private void OnByte(byte b, IHttpHeadersHandler handler)
                    }
                    break;
                case State.CompressedHeaders:
-                    switch (BitOperations.LeadingZeroCount(b) - 24)
+                    switch (BitOperations.LeadingZeroCount(b) - 24) // byte 'b' is extended to uint, so will have 24 extra 0s.


Suggested change

switch (BitOperations.LeadingZeroCount(b) - 24) // byte 'b' is extended to uint, so will have 24 extra 0s.

switch (BitOperations.LeadingZeroCount(b) - sizeof(uint) + sizeof(byte)) // byte 'b' is extended to uint, so will have 24 extra 0s.

…Http20, and update some HTTP/2 to work for HTTP/3. Enable tests for HTTP/3, behind a conditional feature test. Fix QPackDecoder lzcnt assuming an 8-bit test. Rename test QPACK classes from QPackEncoder/QPackDecoder -> QPackTestEncoder/QPackTestDecoder to avoid naming confusion with product code classes. Fix QPackTestDecoder bit flag checks. Fix a double call to QuicConnection.CloseAsync(). Update to shutdown QuicConnection in a background task. Fix test cert usage.

scalablecory added the area-System.Net.Http label Feb 7, 2020

scalablecory added this to the 5.0 milestone Feb 7, 2020

scalablecory requested review from a team February 7, 2020 00:28

scalablecory self-assigned this Feb 7, 2020

jkotalik reviewed Feb 11, 2020

View reviewed changes

jkotalik approved these changes Feb 11, 2020

View reviewed changes

scalablecory force-pushed the http3-tests branch from a232aa6 to 9a8a941 Compare February 12, 2020 01:44

This was referenced Feb 12, 2020

System.Net.NetworkInformation.Functional.Tests failing CI and PR #32179

Closed

Test failure: System.Net.NetworkInformation.Tests.NetworkInterfaceBasicTest/BasicTest_AccessInstanceProperties_NoExceptions_Linux #18090

Closed

halter73 reviewed Feb 12, 2020

View reviewed changes

scalablecory and others added 2 commits February 13, 2020 14:16

Fix QPack codec file paths

a7dca57

scalablecory force-pushed the http3-tests branch from 4708ff4 to a7dca57 Compare February 13, 2020 22:22

scalablecory merged commit 556fb40 into dotnet:master Feb 14, 2020

scalablecory deleted the http3-tests branch July 29, 2020 16:06

ghost locked as resolved and limited conversation to collaborators Dec 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Get tests running for HTTP/3 #31898

Get tests running for HTTP/3 #31898

scalablecory commented Feb 7, 2020 •

edited

Loading

jkotalik Feb 11, 2020

jkotalik Feb 11, 2020

jkotalik Feb 11, 2020

jkotalik Feb 11, 2020

jkotalik Feb 11, 2020

scalablecory Feb 11, 2020

nibanks Feb 11, 2020

jkotalik Feb 11, 2020

jkotalik left a comment

halter73 Feb 12, 2020

	switch (BitOperations.LeadingZeroCount(b) - 24) // byte 'b' is extended to uint, so will have 24 extra 0s.
	switch (BitOperations.LeadingZeroCount(b) - sizeof(uint) + sizeof(byte)) // byte 'b' is extended to uint, so will have 24 extra 0s.

Get tests running for HTTP/3 #31898

Get tests running for HTTP/3 #31898

Conversation

scalablecory commented Feb 7, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkotalik left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

scalablecory commented Feb 7, 2020 •

edited

Loading