[HTTP/3] Support Single-file and NativeAOT on Windows

[karelz]

HTTP/3 should be supported also in Single-file and NativeAOT scenario on Windows.

Linux is simple as msquic is "part of OS" - external package that is prereq for HTTP/3 to work.
On Windows, we build and include msquic.dll as part of .NET Runtime.

@vitek-karas can point us to what needs to happen to make it work.
cc @jkotas

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

HTTP/3 should be supported also in Single-file scenario on Windows.

Linux is simple as msquic is "part of OS" - external package that is prereq for HTTP/3 to work.
On Windows, we build and include msquic.dll as part of .NET Runtime.

@vitek-karas can point us to what needs to happen to make it work.
cc @jkotas

Author:	karelz
Assignees:	-
Labels:	enhancement, area-System.Net.Http
Milestone:	8.0.0

[karelz]

Triage: We discussed with @vitek-karas that such work should be part of 8.0 -- it is too late for 7.0 and the usage of HTTP/3 is not that much widespread yet.
Moreover, there is workaround (which we can document on demand) to copy msquic.dll next to Single-file (while not perfect, it is at least functional).

[vitek-karas]

@VSadov should know the technical details, but high level:

• msquic needs to be statically linked into the singlefilehost.exe during our build
• The native entry points which msquic exposes and are used by managed code need to be "marked" as internal, so that interop redirects them into the exe (and won't try to load msquic.dll
• The managed code which uses msquic might need changes - the above will work without changes if we use simple PInvokes (DllImport) but if there's other usage we might need to make some tweaks

There's obviously a size aspect to all of this - as this will make the single-file host cca 500KB bigger - for everybody.

[teo-tsirpanis]

I wonder if we could create bespoke single-file hosts that link to custom static libraries at publish-time. The apps that don't use HttpClient won't have to bear the burden of half a megabyte of code they won't use, and these that do use it but don't need HTTP/3 would be able to remove support for it with a feature switch, and other libraries could use it to bundle their own native dependencies. I think something similar is done by Blazor WebAssembly to reduce size.

Of course it would be an optional feature since it would need additional dependencies such as a native linker, and the default would still be a single-file host that links everything we might need.

[vitek-karas]

So far we've been trying to avoid dependency on platform linker for these scenarios. I think we might be able to do this for the NativeAOT version of this problem, since that one already has a dependency on platform linker.

[jkotas]

msquic needs to be statically linked into the singlefilehost.exe during our build

This requires to either (1) msquic repo to produce .lib file that gets statically linked into the singlefilehost in dotnet/runtme or (2) vendor msquic sources into dotnet/runtime and build it there for static linking into the singlefilehost.

Once msquic is statically linked into the singlefilehost, the rest is trivial.

[karelz]

@jkotas we build Windows in our fork https://github.com/dotnet/msquic -- that's how Windows DLLs are built for .NET Runtime product.
I assume that building and publishing .lib should not be a big problem in the same repo. cc @wfurt

[jkotas]

Yes, it is certainly doable. If the .lib is built in one repo and consumed in second repo, there are details you need to be careful about - like making sure that the compiler versions and compiler options are the same or at least compatible between the repos. It may be challenging to manage at times when the compilers are getting upgraded.

[nibanks]

Another, completely orthogonal path could be that we align the Windows model with Linux, and we ship MsQuic via a package manager (winget), and .NET doesn't package msquic at all. .NET already has the code to opportunistically load msquic when available.

The obvious downside with this approach is that you will always require an additional step, even in the single-file scenarios, to install the msquic dependency (if not already). So I'm not quite sure if you'd consider this solving the single-file problem or not. Generally, this seems to be deemed an acceptable model on Linux, but is not commonplace on Windows, and I suspect would receive push back. Just a thought though....

[ManickaP]

Part of Windows platform test is disabled against this issue:

runtime/src/libraries/System.Net.Quic/tests/FunctionalTests/MsQuicPlatformDetectionTests.cs

Lines 23 to 38 in 47071da

	[ConditionalFact(typeof(PlatformDetection), nameof(PlatformDetection.IsWindows), nameof(PlatformDetection.SupportsTls13))]
	public void SupportedWindowsPlatforms_IsSupportedIsTrue()
	{
	if (PlatformDetection.HasAssemblyFiles)
	{
	Assert.True(QuicListener.IsSupported);
	Assert.True(QuicConnection.IsSupported);
	}
	else
	{
	// The above if check can be deleted when https://github.com/dotnet/runtime/issues/73290
	// gets fixed and this test starts failing.
	Assert.False(QuicListener.IsSupported);
	Assert.False(QuicConnection.IsSupported);
	}
	}

See #81581

[karelz]

Pushing to 9.0 (Future at this moment) per offline discussion.

[teo-tsirpanis]

Moving to 9.0.0 milestone now that it exists, per above comment.

[ladeak]

I have a .NET 8 AOT application that uses HTTP/3 with msquic on Windows x64. When I run the application 'standalone' from a console it can use H3. However, when I use the same AOT compiled code from a node application (a VSCode plugin) with (https://github.com/microsoft/node-api-dotnet), I get the following exception:

Request Error System.Net.Http.HttpRequestException: Requesting HTTP version 3.0 with version policy RequestVersionExact while unable to establish HTTP/3 connection.
   at System.Net.Http.HttpConnectionPool.ThrowGetVersionException(HttpRequestMessage, Int32, Exception) + 0xec
   at System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__89.MoveNext() + 0x572
--- End of stack trace from previous location ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x20
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0xb2
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x4b
   at System.Net.Http.RedirectHandler.<SendAsync>d__4.MoveNext() + 0x18b
--- End of stack trace from previous location ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x20
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0xb2
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x4b
   at System.Net.Http.HttpClient.<<SendAsync>g__Core|83_0>d.MoveNext() + 0x3b1
--- End of stack trace from previous location ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x20
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0xb2
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x4b
   at CHttp.HttpMessageSender.<SendRequestAsync>d__7.MoveNext() + 0x189

QuicConnection.IsSupported returns false, that gives me the idea that loading the native library does not work.

When I call QuicConnection.ConnectAsync, I get the following exception revealing the underlying cause:

MsQuicOpenVersion for version 2 returned -2147467262 status code. is the NotSupportedReason. Which seems to be matching the NotSupported error code emitted here.

The other interesting bit is when I manually try to run this code in the same context (the difference is that I am passing 1 for the version, I get back a status of '0' (so no error code):

                IntPtr msQuicHandle;
                var loaded = NativeLibrary.TryLoad("msquic.dll", typeof(System.Net.Quic.QuicConnection).Assembly, DllImportSearchPath.AssemblyDirectory, out msQuicHandle);
                if (!loaded)
                    return Task.FromResult("not loaded");
                var open = (delegate* unmanaged[Cdecl]<uint, QUIC_API_TABLE**, int>)NativeLibrary.GetExport(msQuicHandle, "MsQuicOpenVersion");
                QUIC_API_TABLE* table = null;
                var status = open((uint)1, &table);
                return Task.FromResult($"{status} and {table != null}");

Is there a way to override the default way of loading the library (or just the version of it)? Is the wrong version of the msquic.dll is compiled with publish AOT? Or is this a different error?

[jkotas]

You need to manually copy msquic.dll next to your single-file or native-aot compiled binary as a workaround for this issue.

[ladeak]

Thank you for the suggestion. I have tested it, but unfortunately in my case it does not work as the AOT application is run as a VSCode extension:

When I put the dll next to the distributed files it does not get picked up, because Environment.CurrentDirectory or System.AppContext.BaseDirectory points to a different folder compared to my exe.

The executable is a VSCode plugin is under: C:\Users\[username]\.vscode\extensions\[extension-name]\dist
while System.AppContext.BaseDirectory returns the VSCode installation path C:\Users\[username]\AppData\Local\Programs\Microsoft VS Code

I was considering if I can update/change the way we still load the library with NativeLibrary.SetDllImportResolver, but that did not lead anywhere.

I actually tried copying by hand the msquic.dll into the VSCode installation folder, and then everything works as expected, but unfortunately that is not a viable solution.

[MichalStrehovsky]

@ladeak are you on .NET 7? Native AOT had a bug that it didn't do anything for DllImportSearchPath.AssemblyDirectory that is used to load the msquic library. This was fixed in .NET 8 (#90120).

Is your VS code extension an EXE or a native DLL? If it's a DLL, additional considerations apply (#90131).

[ladeak]

I am using .NET 8 RC2. The plugin is an exe that is packaged as a .node file.

[MichalStrehovsky]

The DLLs loaded with DllImportSearchPath.AssemblyDirectory (which is the case of MSQuic) will be loaded from wherever AppContext.BaseDirectory points to. This should default to the same path as Environment.ProcessPath. What is your reported Environment.ProcessPath?

You can overwrite the base directory with your own value as a last resort with AppContext.SetData but I'm surprised this would be necessary:

runtime/src/libraries/System.Private.CoreLib/src/System/AppContext.cs

Lines 26 to 30 in 7faabde

	public static string BaseDirectory =>
	// The value of APP_CONTEXT_BASE_DIRECTORY key has to be a string and it is not allowed to be any other type.
	// Otherwise the caller will get invalid cast exception
	GetData("APP_CONTEXT_BASE_DIRECTORY") as string ??
	(s_defaultBaseDirectory ??= GetBaseDirectoryCore());

[ladeak]

Environment.ProcessPath points to C:\Users\[username]\AppData\Local\Programs\Microsoft VS Code\Code.exe.

I will check if I can override this in VS Code or use the AppContext as suggested. Thank you for the details.

EDIT: Yes, that resolves the issue: I can package the file along the extension and set APP_CONTEXT_BASE_DIRECTORY to the path of the extension.

Thank you @jkotas and @MichalStrehovsky for the patience support for understanding the context.