NativeAOT fails to resolve PInvoke in System.Data.SqlClient

[vitek-karas]

See #89782 (comment) for details.
The failure is really weird - I can confirm that the native dll is next to the AOT compiled app.
Non-AOT version of the app can load the dll just fine (it fails later on for me, but that's not the issue here).
I also tried just a simple app with a PInvoke (using Libuv package and calling into it) and that works just fine even with NativeAOT.

Repro steps:
https://github.com/dotnet/runtime/files/12243049/sqlClient.zip
Publish as AOT and run on win-x64.

Unhandled Exception: System.DllNotFoundException: Unable to load DLL 'sni.dll' or one of its dependencies: The specified module could not be found.
   at System.Runtime.InteropServices.NativeLibrary.LoadLibErrorTracker.Throw(String) + 0x6f
   at Internal.Runtime.CompilerHelpers.InteropHelpers.FixupModuleCell(InteropHelpers.ModuleFixupCell*) + 0xfd
   at Internal.Runtime.CompilerHelpers.InteropHelpers.ResolvePInvokeSlow(InteropHelpers.MethodFixupCell*) + 0x2f
   at System.Data.SqlClient.SNINativeMethodWrapper.UnmanagedIsTokenRestricted(IntPtr, Boolean&) + 0x31
   at System.Data.Win32NativeMethods.IsTokenRestrictedWrapper(IntPtr) + 0x19
   at System.Data.ProviderBase.DbConnectionPoolIdentity.GetCurrentNative() + 0xb0
   at System.Data.ProviderBase.DbConnectionPoolGroup.GetConnectionPool(DbConnectionFactory) + 0x51
   at System.Data.ProviderBase.DbConnectionFactory.GetConnectionPool(DbConnection, DbConnectionPoolGroup) + 0x5f
   at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection, TaskCompletionSource`1, DbConnectionOptions, DbConnectionInternal, DbConnectionInternal&) + 0xb6
   at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection, DbConnectionFactory, TaskCompletionSource`1, DbConnectionOptions) + 0x118
   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1) + 0x101
   at System.Data.SqlClient.SqlConnection.Open() + 0xf8
   at Program.<Main>$(String[] args) + 0x45
   at sqlClient!<BaseAddress>+0x413074

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

See #89782 (comment) for details.
The failure is really weird - I can confirm that the native dll is next to the AOT compiled app.
Non-AOT version of the app can load the dll just fine (it fails later on for me, but that's not the issue here).
I also tried just a simple app with a PInvoke (using Libuv package and calling into it) and that works just fine even with NativeAOT.

Repro steps:
https://github.com/dotnet/runtime/files/12243049/sqlClient.zip
Publish as AOT and run on win-x64.

Unhandled Exception: System.DllNotFoundException: Unable to load DLL 'sni.dll' or one of its dependencies: The specified module could not be found.
   at System.Runtime.InteropServices.NativeLibrary.LoadLibErrorTracker.Throw(String) + 0x6f
   at Internal.Runtime.CompilerHelpers.InteropHelpers.FixupModuleCell(InteropHelpers.ModuleFixupCell*) + 0xfd
   at Internal.Runtime.CompilerHelpers.InteropHelpers.ResolvePInvokeSlow(InteropHelpers.MethodFixupCell*) + 0x2f
   at System.Data.SqlClient.SNINativeMethodWrapper.UnmanagedIsTokenRestricted(IntPtr, Boolean&) + 0x31
   at System.Data.Win32NativeMethods.IsTokenRestrictedWrapper(IntPtr) + 0x19
   at System.Data.ProviderBase.DbConnectionPoolIdentity.GetCurrentNative() + 0xb0
   at System.Data.ProviderBase.DbConnectionPoolGroup.GetConnectionPool(DbConnectionFactory) + 0x51
   at System.Data.ProviderBase.DbConnectionFactory.GetConnectionPool(DbConnection, DbConnectionPoolGroup) + 0x5f
   at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection, TaskCompletionSource`1, DbConnectionOptions, DbConnectionInternal, DbConnectionInternal&) + 0xb6
   at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection, DbConnectionFactory, TaskCompletionSource`1, DbConnectionOptions) + 0x118
   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1) + 0x101
   at System.Data.SqlClient.SqlConnection.Open() + 0xf8
   at Program.<Main>$(String[] args) + 0x45
   at sqlClient!<BaseAddress>+0x413074

Author:	vitek-karas
Assignees:	-
Labels:	area-NativeAOT-coreclr
Milestone:	-

[vitek-karas]

@MichalStrehovsky any ideas - I can try looking into it, but I don't know where to start really.

[jkotas]

System.Data.SqlClient is annotated with DllImportSearchPath.AssemblyDirectory | DllImportSearchPath.System32

These search flags get passed correctly into NativeLibrary.LoadLibraryByName

runtime/src/coreclr/nativeaot/System.Private.CoreLib/src/System/Runtime/InteropServices/NativeLibrary.NativeAot.cs

Lines 37 to 40 in a743d60

	internal static IntPtr LoadLibraryByName(string libraryName, Assembly assembly, DllImportSearchPath searchPath, bool throwOnError)
	{
	int searchPathFlags = (int)(searchPath & ~DllImportSearchPath.AssemblyDirectory);
	bool searchAssemblyDirectory = (searchPath & DllImportSearchPath.AssemblyDirectory) != 0;

that masks out DllImportSearchPath.AssemblyDirectory and so we are left with just DllImportSearchPath.System32.

The problem happens here:

runtime/src/coreclr/nativeaot/System.Private.CoreLib/src/System/Runtime/InteropServices/NativeLibrary.NativeAot.cs

Lines 97 to 101 in a743d60

	else if ((callingAssembly != null) && searchAssemblyDirectory)
	{
	// Try to load the module alongside the assembly where the PInvoke was declared.
	// This only makes sense in dynamic scenarios (JIT/interpreter), so leaving this out for now.
	}

We effectively ignore DllImportSearchPath.AssemblyDirectory/searchAssemblyDirectory and only pass DllImportSearchPath.System32 into LoadLibraryHelper. There is no sni.dll in system32 directory and the attempt to load the library fails.

[vitek-karas]

That comment reads wrong - NativeAOT already made the decision to put native dependencies next to the exe (instead of the assembly the PInvoke came from), so treating AssemblyDirectory as "app directory" would make sense for NativeAOT (also - there's no other location to consider anyway).

[MichalStrehovsky]

This was written as "better to be conservative than to get an MSRC later". How does CoreCLR single file compute "the location of the assembly", is AssemblyDirectory just bashed to mean ApplicationDirectory? We could do the same, but cc @elinor-fung

[MichalStrehovsky]

This was written as "better to be conservative than to get an MSRC later". How does CoreCLR single file compute "the location of the assembly", is AssemblyDirectory just bashed to mean ApplicationDirectory? We could do the same, but cc @elinor-fung

And how does this distinction look like for the "NativeAOT'd shared library" case?

[elinor-fung]

Single-file actually adds the app directory to NATIVE_DLL_SEARCH_DIRECTORIES: #42876, #42772
AssemblyDirectory is currently meaningless for assemblies bundled into the app executable and the app effectively always has ApplicationDirectory. (I do think making AssemblyDirectory for bundled assemblies mean the directory of the app they are bundled in makes more sense - especially now that single-file apps don't have the runtime native bits separate and next to it)

[vitek-karas]

I think this is something we should try to fix for .NET 8. I'll be out for the next week, so if somebody could pick this up?
Just to make it easier for whoever is going to work on this: @elinor-fung could you please paste a link here to where we have CoreCLR/Single-File tests for this? So that we can enable them for NativeAOT once this is fixed.

[elinor-fung]

The coreclr test cases are:

runtime/src/tests/Interop/NativeLibrary/API/NativeLibraryTests.cs

Lines 176 to 177 in fa78033

	[Fact]
	public void LoadLibrary_AssemblyDirectory()

runtime/src/tests/Interop/DllImportSearchPaths/DllImportSearchPathsTest.cs

Lines 30 to 31 in fa78033

	[ConditionalFact(nameof(CanLoadAssemblyInSubdirectory))]
	public static void AssemblyDirectory_Found()

NativeAOT would need a variation of that though. They both use Assembly.LoadFile to load an assembly that is not next to the app and resolve relative to that assembly (corerun adds the app path to the native search directories, so it had to be in a different directory to actually test the AssemblyDirectory logic).

[AaronRobinsonMSFT]

I'll take a look at this.