Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a way to list native assets that a project will load from the app directory (list them in deps.json) #11373

Open
safern opened this issue Apr 21, 2020 · 7 comments
Labels
Feature Request native-assets Issues related to how the SDK should deal with Native assets

Comments

@safern
Copy link
Member

safern commented Apr 21, 2020

In dotnet/runtime we're currently enabling a feature for applications to carry their own copy of ICU for globalization purposes instead of using system-wide installed ICU.

In order to make this feature work, we're using NativeLibrary.TryLoad API which uses some probing using NATIVE_DLL_SEARCH_DIRECTORIES property passed by the host to the runtime. However, this property is populated based of the deps.json file, so if we don't have any native assets coming from NuGet packages or we want to use a different folder for our custom built native assets, we wont be able to load from there as NATIVE_DLL_SEARCH_DIRECTORIES will not include this directory. This is of course blocking for framework dependent apps as for self contained apps NATIVE_DLL_SEARCH_DIRECTORIES includes the app directory always.

Currently the only way to have native assets included in deps.json file is via NuGet resolved assets, however, I did manage to do this by adding:

  <ItemGroup>
    <IcuAssemblies Include="icu\*.so*" />
    <RuntimeTargetsCopyLocalItems Include="@(IcuAssemblies)" AssetType="native" CopyLocal="true" DestinationSubDirectory="runtimes/linux-x64/native/" DestinationSubPath="%(FileName)%(Extension)" RuntimeIdentifier="linux-x64" NuGetPackageId="System.Private.Runtime.UnicodeData" />
  </ItemGroup>

This requires me to specify a NuGetPackageId and it has to be an actual package that the app references, so if I have a project that doesn't reference any NuGet packages, then I won't be able to do this.

Open questions

Should assets specify a RID or should the SDK assume it, based on the publish/build rid?
How should they be listed in the deps.json file? Under the built app library section? Under a new loose native assets section?
We need to make sure this works end-to-end on publish, packaging and as transitive dependencies as what if I have a reference to an app that depends on this loose native assets?

Notes

This will be a very important feature for ICU story since we're now providing a default behavior to use ICU on Windows and many customers will want to use the same version or a custom version with custom locales in their app across different OSs.

cc: @dsplaisted @ericstj @jkotas @tarekgh @eerhardt @jeffschwMSFT

@dsplaisted
Copy link
Member

Some possible syntaxes (based on a group discussion):

<ItemGroup>
  <NativeLibrary Include="path/to/icu.so" RuntimeIdentifier="linux-x64" />

  <Content Include="path/to/icu.so" CopyToOutput="PreserveNewest"
     RuntimeIdentifier="linux-x64" />
</ItemGroup>

Ideally this would be transitive. That could require plumbing through NuGet.

Also look at how Xamarin represents native assets for iOS and Android.

@tarekgh
Copy link
Member

tarekgh commented Apr 28, 2020

Is it possible we can plumb NuGet from now?

@ericstj
Copy link
Member

ericstj commented Mar 5, 2021

Might be the same issue as #10575
Related dotnet/runtime#11404
Seems like it's needed for dotnet/core#5699 cc @terrajobst

I hacked this together a bit. The SDK does honor items that the project creates but they must be tied to a package. It won't honor any items that claim to come from the project.

Here's a very ugly hack: https://github.com/ericstj/sample-code/tree/nativeLibSample
I wouldn't recommend shipping this deps file produced with this method, but it might do the job for local testing.

Not sure if NuGet would need to be involved for transitive references, what if these were just flowed via MSBuild items and you could add them to

internal class SingleProjectInfo
?

@dotMorten
Copy link

<NativeLibrary Include="path/to/icu.so" RuntimeIdentifier="linux-x64" />

This is what I'd like to see. I'd also like for the nuget pack command to then correctly pack the library. That way when I do a nuget reference or a project reference, things works the exact same way

@robmen
Copy link
Contributor

robmen commented May 1, 2021

@ericstj and @terrajobst we also need a solution for this in the WiX Toolset to find things like mergemod.dll. For now, we're going to try to go with Eric's "very ugly hack"

@taoyouh
Copy link

taoyouh commented Jun 13, 2021

It seems that the RuntimeTargetsCopyLocalItems solution is not transitive. Another project that references the project with RuntimeTargetsCopyLocalItems will not have the native assets included in the output.

@ericstj
Copy link
Member

ericstj commented Jun 16, 2021

Correct, since it's not part of the assets file it's not transitive. There is no good way to flow additional files transitively through ProjectReferences by default. Your best bet is to handle it with some common targets that can be used in multiple projects in your repository.

kzu added a commit to devlooped/chromium that referenced this issue Jun 3, 2022
It is unfortunately not possible to conditionally reference native assets depending on the installed tool's runtime platform. This means we need to assume the dependency will be restored at the project level and look it up via the project.assets.json, rather than the runtime deps.

See
- How a tool project is installed/restored: https://github.com/dotnet/sdk/blob/main/src/Cli/dotnet/ToolPackage/ToolPackageInstaller.cs
- Native files not being automatically added to the runtime deps.json: dotnet/sdk#11373
- How to do that via custom targets, but this wouldn't be possible on the target machine when restoring the temp project created for global tools: https://github.com/ericstj/sample-code/blob/nativeLibSample/addNative/addNative.csproj
- How runtime.json might work but it's going away: dotnet/runtime#11404
- Really going away: dotnet/runtime#49137
- How it's still not solved for .net6: NuGet/Home#5862
kzu added a commit to devlooped/chromium that referenced this issue Jun 3, 2022
It is unfortunately not possible to conditionally reference native assets depending on the installed tool's runtime platform. This means we need to assume the dependency will be restored at the project level and look it up via the project.assets.json, rather than the runtime deps.

See
- How a tool project is installed/restored: https://github.com/dotnet/sdk/blob/main/src/Cli/dotnet/ToolPackage/ToolPackageInstaller.cs
- Native files not being automatically added to the runtime deps.json: dotnet/sdk#11373
- How to do that via custom targets, but this wouldn't be possible on the target machine when restoring the temp project created for global tools: https://github.com/ericstj/sample-code/blob/nativeLibSample/addNative/addNative.csproj
- How runtime.json might work but it's going away: dotnet/runtime#11404
- Really going away: dotnet/runtime#49137
- How it's still not solved for .net6: NuGet/Home#5862
@baronfel baronfel added the native-assets Issues related to how the SDK should deal with Native assets label Jan 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature Request native-assets Issues related to how the SDK should deal with Native assets
Projects
None yet
Development

No branches or pull requests

8 participants