Guide for packaging C# library using P/Invoke to per-architecture and/or per-platform C++ native DLLs

[djee-ms]

I am trying to make a NuGet package deploying a C# .NET Standard 2.0 library which does P/Invoke calls into a platform-dependent C++ library, which must therefore also be deployed, but is obviously architecture-dependent (x86, x64, ARM, ARM64), as well as platform-dependent (Desktop (Win32) vs. UWP).

I read most documentations on docs.microsoft.com, issues on this GitHub and others, SO issues, etc. and it is still very unclear how to do this. Information is sparse, sometimes contradictory, and the lack of details on some concepts like TFMs makes the task nearly impossible. This whole thing could really use some detailed documenting and samples.

In no particular order:

Target frameworks

• https://docs.microsoft.com/en-us/nuget/reference/target-frameworks has a long list of supported frameworks, but native is not included, as reported in NuGet/docs.microsoft.com-nuget#1480. However https://devblogs.microsoft.com/nuget/native-support/ clearly states that:

  When targeting native projects, a new target framework name is now recognized: native.

• This page lists netcore as a framework with versions like 5.0. But .NET Core is just releasing its 3.0 this week. So clearly there is no relation between the two, but there is not a word on it.

• This page also casually mentions the TFM win10:

  win10 (not supported by Windows 10 Platform)

  There is no explanation on what win10 is (is this Desktop, as opposed to UWP?) nor why win10 would not be supported on Windows 10 despite the name clearly saying otherwise.

Package structure

• https://docs.microsoft.com/en-us/nuget/create-packages/creating-a-package goes into details about the NuGet package structure, but does not mention anything about native DLLs.

• https://devblogs.microsoft.com/nuget/native-support/ says that the build/ folder should be used to put the native DLLs, but https://docs.microsoft.com/en-us/nuget/create-packages/creating-a-package says it is only for .targets and .props files.

• https://devblogs.microsoft.com/nuget/native-support/ says that the build/ folder respects target frameworks, but does that mean that build/native/x86/my.dll vs. build/native/x64/my.dll works? Or should architecture-specific native DLLs be put in runtimes/?

• Does NuGet support Desktop (Win32) and UWP inside the same package? I could not find any documentation about that. I tried using runtimes\win10-x86\native\my.dll for the Desktop variant, and runtimes\win10-x86\lib\uap10.0\my.dll for the UWP variant of the native DLL, but I get an error error APPX1101: Payload contains two or more files with the same destination path. Why is that? Those are different frameworks, why is NuGet trying to import both files into a UWP project?

P/Invoke

It seems many people have problem with deploying architecture-specific native DLLs. A quick search on nuget.org shows that packages like Microsoft.Net.Native.Compiler have many "runtime" variants starting with e.g. a runtime.win10-x64. prefix, but it doesn't seem there is documentation about this approach.

https://github.com/Mizux/dotnet-native attempts to provide an example using the undocumented runtime.json used by CoreFX, but looking at the example it seems that for each native DLL variant, a specific .NET Standard 2.0 wrapper assembly is needed, instead of using a single one with multiple native DLLs. This sounds very cumbersome, is that the only option?

Related to that, if it is possible to use a single .NET Standard 2.0 assembly, then how to deploy the correct native DLL? With a .NET Core 3.0 sample app, it seems that currently NuGet copies the entire runtimes/ folder inside the bin/ folder of the app, instead of only the appropriate native DLL. This results in multiple copies, and of course the wrong DLL path which prevents DllImport from finding the native DLL.

Other issues

There are many other logged issues that seem partially related:

• csproj: document how to properly pack platform-specific native assemblies #6645 mentions that

  runtimes/{rid}/native does not work with netfx

  but there is no context about where that comes from. And it suggests putting native DLLs in lib/ which is reserved for assemblies, so doesn't seem to be a correct solution.

• Absolutely need example of how to include native libraries #6648 closed as duplicate, although the context is not clear (what kind of app / platform is this about?)

• Support or Documentation of Creating of Packages with Native Applications - PhantomJS #3931 seems to be somewhat related, but uses project.json (?)

• NuGet for UWP apps cannot call C++ libraries  #2350 asks about the P/Invoke and packaging issue, but was closed without answer.

• Feature Request: Allow architecture-specific binaries without requiring client projects to use RuntimeIdentifier(s) #6846 touches on the deploy problem when consuming the package

• Improvement: Handle platform specific assemblies (x86, x64 etc) with msbuild conditionals #8573, Issues using nuget packages with C# libraries compiled for specific architecture. #8435, UAP - platform specific binaries not picked up from runtimes #1221, (UWP) Platform Dependent UWP Library fails due to Compile-Time Reference Assembly Missing #5606, ... I didn't read all of them, there are too many.

• Several issues mention https://stackoverflow.com/questions/49162127/how-to-include-native-assets-in-nuget-on-multiple-platforms-only-64-bit but this seems to be only a subset of the issues, it is not clear how this scales to multiple architectures AND multiple platforms at the same time. It also seems to suggest multiple assemblies are needed.

• The road of the AssemblyLoadContext seems to be a runtime solution to a packaging problem, and really not a path I want to get onto.

[dfields-msft]

The Desktop/UWP flavor dimension can potentially be simplified by deploying only UWP binaries, and adding a dependency on Microsoft.VCRTForwarders.140 to enable those binaries to be used in Desktop apps as well.

[lostmsu]

I am about to package a set of very large native libraries my .NET Standard DLL wraps. Each platform-specific lib is >200MB, so I'd like to know if it is possible to ship them in separate NuGet packages somehow (sounds like that's what NativeCompiler is doing).

So I need guidance on how to achieve that.

[djee-ms]

We currently ship 10 GB (unpacked) of NuGet packages for https://github.com/microsoft/MixedReality-WebRTC. This is more or less working, but we don't use any multi-framework feature from NuGet; instead it's mostly manual setup via .props and .targets files. And because of that users have to download all architectures even if they don't use them, since NuGet has no knowledge about the architecture specificity of each package. This is far from being great.

[davidhunter22]

We were having what I believe is the same, or very similar, problem as the original poster

We have a solution that builds a .Net assembly using C++/CLI. This is a wrapper round some C++ code to expose it to, in our case, C#. This assembly is build in x86 and x64 variants. We then would like to create a single NuGet package containing both variants. We then have a separate C# solution using the new SDK style and that we would like to consume this package. The C# solution has x86 and x64 platforms rather than AnyCPU as the assembly as run time has to have a runtime "bitness" matching the underlying C++/CLI assembly. The machinery behind the would select the correct C++/CLI assembly from the package.

We have not been able to achieve this. As the original poster mentioned there is some information.
Based on this we have tried literally hundreds of variants of nuspec file to build the C++/CLI
package. Some fail when we pack other succeed but then fail when we with a variety of errors. Obviously we have never found an example of this scenario working.

Anyway our current work around which is not to painful is to create tow nuget packages with x86 and x64 in the name using a fairly simple nuspec file like

<package xmlns="http://schemas.microsoft.com/packaging/2013/05/nuspec.xsd">
    <metadata>
    	<id>Foo.x64</id>
	    <version>1.0.0</version>	
        ... usual stuff
    </metadata>
    <files>
         <file src="x64\Debug\net47\Foo.dll" target="lib\net47" />
    </files>
</package>

In the C# project file we then have

    <ItemGroup Condition="'$(Platform)'=='x64'">
        <PackageReference Include="Foo.x64" Version="1.0.0" />
    </ItemGroup>
    <ItemGroup Condition="'$(Platform)'=='x86'">
        <PackageReference Include="Foo.x86" Version="1.0.0" />
    </ItemGroup>

Note in the real world we multi target various framework and core versions and it works fine. We would really love to move to a single unified package. The main reason is not creating the the two packages but the fact that in consuming solutions you can no longer just use the standard NuGet Solution manager. You can get the initial from this but then you have to go in and manually edit the project file to copy it and add the conditions. This is a problem as we don't actually know who may reference this package and other than documenting things can't help

[Mizux]

FYI, my 2 cents

1. Actually, You don't need a specific variant of .Net Standard dll wrapper simply omit the extension and .Net magic will pick the correct native library. So, now your wrapper code become identical for all architectures and you can move/merge all C# managed code from each runtime packages to the "meta" package.
   ref: https://www.mono-project.com/docs/advanced/pinvoke/#library-names

2. Please take at look at my fully working project https://github.com/Mizux/dotnet-native
   It's a Modern CMake, C++ project with auto generated .Net wrappers using SWIG.

3. Notice this project, dotnet-native, was intended to only focus on the .Net cpsroj stuff and use a "fake" already compiled library contrary to cmake-swig which aims to provide a complete working example from C++ source to swig generated .Net Standard wrapper but at the cost of a higher complexity...
   note: Maybe I should revamp the cmake-swig project but without the java/python and with only one library Foo (i.e. also removing Bar and FooBar).

4. BTW my Generic/meta/pure .Net package always pull all runtime packages. µ$oft on the contrary in its project, e.g.:

• the "meta" package: https://www.nuget.org/packages/Microsoft.NETCore.App/ and
• one runtime https://www.nuget.org/packages/runtime.linux-x64.Microsoft.NETCore.App/
  They use a sophisticated targets rules to grab the only required runtime package, everything is
  located in build/<tfm>/* of the meta package microsoft.netcore.app.2.2.8.nupkg` (download and unzip it to reverse it ;))

DISCLAIMER: At first, I was a Linux C/C++ embedded developer, so my knowledge to Windows dev and .Net world is somewhat limited and I exclusively use the command prompt and dotnet-cli on Windows VM so don't ask me where to click/set this properties on VS Studio , I'm only writing all .csproj by hand ;)

[djee-ms]

Note that point 1. from @Mizux is partially wrong; the no-extension DllImport will only work if the DLL filename doesn't contain any dot '.' character in it, due to a bug in LoadLibraryEx() which doesn't add automatically the .dll if there's already another dot . character. So [DllImport("mydll")] will work, but [DllImport("a.b.c")] will dot find a.b.c.dll. We hit that on https://github.com/microsoft/MixedReality-WebRTC and had to rename our native DLL to work around the issue.

[lostmsu]

@djee-ms do you happen to know if this problem is also present on *nix?

[djee-ms]

As far as I know the issue I am referencing (dotnet/runtime#7223) is a specific issue with how LoadLibraryEx() is implemented on Windows, and therefore only affects DllImport on Windows. On Linux I am pretty sure DllImport uses dl_open() which has different rules. I cannot guarantee however that dl_open() doesn't have the same kind of issue, although to the best of my (limited) knowledge it doesn't. But again for our project we need cross-platform so I didn't look too much into it, since we had to rename for Windows anyway. After renaming and removing the dot '.' in the filename I can confirm that the same assembly with the same DllImport can be used on both Windows and Android, you simply need 2 different implementations of the native library.

[Davilink]

Any update on this ?
The documentation is really not clear at all
https://stackoverflow.com/a/40652794

[davidhunter22]

Hi, I just had another go at trying to solve this with no joy.

Maybe if I state my problem another way someone might know that there is a way to achieve this. I am doing the following.

1. Create a C# project and I add x64 and x86 platform to it and remove AnyCPU
2. PackageReference a Nuget package which should contains platform specific .NET assemblies. Note there doesn't need to be any native C++, C+/CLI, PInvoke code involved in this at all.
3. The PackageReference uses a platform specific assembly in the nuget package based on the platform being built.

My problem is not constructing the package I can make a package with any structure. The problem is knowing wether there is any logic underlying PackageReference that understands and uses platforms at all. when I do a build I think another issue related to this mentioned that the following would be and possible package structure

lib/net5.0/x86/Foo.dll
lib/net5.0/x64/Foo.dll

So this would behave similarly to the target framework which the SDK build system obviously does understand when you do a PackageReference, in other words it know to look in lib/net5.0 if you are building net5.0.
Without knowing if the build process has any logic based on platform I have to resort to guessing. There may be none at all in which case I am wasting my time. Note I have tried all the suggestions I have seen in Issues, stack overflow, ....

[Mizux]

If your Pinvoke list a file without extension, at runtime, the ".Net runtime machine" will look at runtimes/<RID>/native/ e.g. runtimes/linux-x64/native/Foo.dll and runtimes/linux-x86/native/Foo.dll

Please take a look at https://github.com/Mizux/dotnet-native

[Mizux]

https://github.com/Mizux/dotnet-native attempts to provide an example using the undocumented runtime.json used by CoreFX, but looking at the example it seems that for each native DLL variant, a specific .NET Standard 2.0 wrapper assembly is needed, instead of using a single one with multiple native DLLs. This sounds very cumbersome, is that the only option?

Actually,

1. I don't use anymore any "runtime.json"...
2. While you need one C++ native .dll per RID, you'll only need ONE .Net Standard 2.0 wrapper i.e. the wrapper will pick the right native library according to the running RID...

[suehshtri]

I ran into this with a .netStandard2.0 library wrapping native code and to be used by an app that will be migrated slowly over time from .Net Framework to .Net 6. I wanted to retain support for 32-bit but lean towards 64-bit. So far I count 3 axes: platform, architecture, .net/framework/core/standard.

[zivkan]

Earlier this year I wrote a docs page on exactly this topic: https://learn.microsoft.com/nuget/create-packages/native-files-in-net-packages