Dogfooding instructions for .NET 8 NativeAOT on macOS x64 via Rosetta #79253
Comments
|
Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas Issue DetailsWe're extremely happy that NativeAOT on Mac x64/arm64 should now work in .NET 8 daily builds. However, I'm running into issues with using the .NET 8 daily builds of the SDK with ILC. Here's the repro: Steps:
Expected result:
Actual result: What's weird is that it claims it can't do cross-compilation, which I'm not actually doing. This is the x64 SDK running in an x64 terminal trying to build an x64 release. Furthermore, it requests the arm64 PackageReference. Adding that reference changes nothing, it reports the same error. It also complains about an explicit package reference that I'm not seeing. I tried following the instructions on dotnet/installer to add the Nuget.config, but that ends up in yet other errors. I shared the output of Is there anything I should do differently to consume and test .NET 8 NativeAOT for macOS x64? `dotnet --info` output:cc @jkotas
|
The native AOT compiler defaults to running natively: You should be able to override this by setting |
|
I believe the warning you're seeing is this bug: #78647. It's an artifact of how we're downloading the package now. The warning shouldn't be appearing. |
|
Thanks, both. @jkotas I can confirm setting The assumption we have is that if the shell is x64, every process under it should run in x64 via Rosetta. That's how all of our other build tools do it (node, yarn, electron, Makefiles, CMake, clang, etc.) and it makes it very easy to target multiple archs on an Apple Silicon machine and never worry about cross compilation because every process just inherits the arch from its parent. |
|
I agree with you that it would make more sense to use the SDK architecture as the host architecture for the IL compiler. PublishReadyToRun compilation works like that. There is no reason for PublishAot to be different. |
|
Hmm, personally it sounds like the process architecture is the right default, no? I believe the x64 terminal config in MacOS works by changing the process arch, but not the machine arch. If the publish takes the current process arch then you could use the -arm64 SDK to easily compile for x64 by just running it in the x64 process. |
Note that this file's name is case sensitive on Unix. Either all lower: I simply added an entry in the existing global config located on macOS at: <add key="dotnet8" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet8/nuget/v3/index.json" />(after the existing line With that in place, the cross-compilation works as well: % dotnet8 new console -n helloworldapp
# workaround for https://github.com/dotnet/installer/pull/14991
# (templates are currently generating net7.0 projects by default until that PR is merged)
% sed -i '' 's/net7/net8/' helloworldapp/helloworldapp.csproj
% dotnet8 publish helloworldapp -a x64 -o helloworldapp/dist -p:PublishAot=true
% file helloworldapp/dist/helloworldapp
helloworldapp/dist/helloworldapp: Mach-O 64-bit executable x86_64
alias dotnet8=~/.dotnet8/dotnetThis is a refresh script #!/usr/bin/env bash
SECONDS=0
DEST="$HOME/.dotnet8"
# recreate destination directory
rm -rf "$DEST"
mkdir "$DEST"
# download and install
curl -sSL https://aka.ms/dotnet/8.0.1xx/daily/dotnet-sdk-osx-arm64.tar.gz | tar xzf - -C "$DEST"
# refresh caches
"$DEST"/dotnet nuget locals all --clear
"$DEST"/dotnet --info
duration=$SECONDS
echo "$(($duration / 60)) minutes and $(($duration % 60)) seconds elapsed." |
I think we should run everything with the same architecture as the SDK which started it, unless it's an explicit choice to do otherwise. Having a random mix is just confusing, and honestly unexpected. Not counting that there's no guarantee that there will be an arm64 runtime available on the machine, so this could even fail to run. |
|
Thanks so much, everybody. @am11 these instructions on consuming daily builds are extremely valuable, thank you for sharing. So much better not to pollute the regular dotnet installation with these daily builds but have more control. The nuget package clearing is neat too. I got the x64 build working as cross compilation now with @am11's instructions now using the arm64 SDK, and it works just fine using either x64 or arm64 terminal. I can also target arm64 using the arm64 SDK using those instructions, when in an arm64 terminal. Curiously, if I am in a Rosetta terminal and invoke the arm64 SDK, instead of getting a nice error saying I shouldn't do that, it fails with linker errors. To repro that, follow @am11's instructions to install dotnet8.
dotnet8 publish output |
I think @christianscheuer summed up my understanding of the Apple approach to these things, and I agree this is the way it seems to work:
In this case we would target the arch of the process. For Native AOT at least, the concept of not having an x64 runtime doesn't matter -- the binary will have the correct architecture by definition. |
|
Actually, when thinking about it I'm not sure everything works the way just described. In particular, does running in an x64 environment always run the x64 version of those tools, or do those tools just assume the architecture of the machine by the process they're running in? This is something to investigate. Is clang running as an x64 process or is clang targeting x64? If it's running as an x64 process that does simplify things -- this would basically be a native targeting scenario, not a cross-targeting scenario. That said, there is some inefficiency here, as in theory there could be performance improvements to actually doing cross-compilation using the arm64 SDK. |
|
Thank you - that's exactly the point I was trying to make, @agocke. Just tested - I can confirm clang runs as x64 process when invoked from x64 shell. It's a fat binary: And yes, I agree it would be possible to get perf optimizations by ignoring that and always run arm64 if available and make cross compilation work transparently to the end user "as if" it had been x64 all along. I would separate that out though as a potential performance optimization and get it to work in the usual Apple way first (everything inherits process arch). Another point would be to consider adding a RID |
Yeah, only problem is that this is somewhat of an advanced scenario in .NET. We don't have fat binaries for the runtime so you have to install each architecture manually, and the entrypoints for each are different, so if you decide you want to execute the x64 version of the SDK you have to pick the x64 dotnet file manually. |
|
Absolutely, I didn't want to overcomplicate things. We can easily combine the binaries to a fat binary ourselves. Just wanted to be sure people were aware of the use case in the design choices. The issue here was though that although I was using the x64 SDK, it would still run ILC in arm64 mode, which was what really caused the issue. If ILC just follows the process arch or SDK arch instead of the machine's native arch, then there shouldn't be any issues. |
|
We got our ~444K LoC app building and running successfully now on .NET 8 previews, through just about a half day's worth of troubleshooting and a few rd.xml patches – hooray! Great job to everybody on making this work. In terms of libraries maintained by Microsoft that don't support trimming yet, I had to add a few rd.xml entries. Does it make sense to report issues in those libraries (AzureAD org) so that this can be tracked? I'm aware rd.xml is not a supported format, so would also love to know if there are other ways it's possible in code to ensure all metadata and methods are produced for a specific type (including specialisations). I tried various tricks like constructing an instance of a type, but this didn't seem to be enough to ensure all methods were created. Rd.xml is fine for now, but I noticed @MichalStrehovsky's PR to make it clear this is not a stable or supported format. |
|
Yup it would be great to know what problems you’re running into |
ghost
added
the
in-pr
The host architecture for the ilc compiler package should default to .NET SDK architecture that the build is running under. Fixes dotnet#79253
The host architecture for the ilc compiler package should default to .NET SDK architecture that the build is running under. Fixes #79253
ghost
removed
the
in-pr
ghost
locked as resolved and limited conversation to collaborators
We're extremely happy that NativeAOT on Mac x64/arm64 should now work in .NET 8 daily builds.
As one of the early proponents of CoreRT, we're looking forward to help beta test this in a large real world application.
However, I'm running into issues with using the .NET 8 daily builds of the SDK with ILC.
We need to build macOS universal binaries (eventually) so all of our builds of x64 are done via Rosetta 2 on an Apple Silicon device.
Here's the repro:
https://github.com/christianscheuer/nativeaot-net8
Steps:
./build.shwhich essentially just ensures we're in Rosetta, then runsdotnet publish -c Release -r osx-x64Expected result:
Actual result:
What's weird is that it claims it can't do cross-compilation, which I'm not actually doing. This is the x64 SDK running in an x64 terminal trying to build an x64 release. Furthermore, it requests the arm64 PackageReference. Adding that reference changes nothing, it reports the same error. It also complains about an explicit package reference that I'm not seeing.
I tried following the instructions on dotnet/installer to add the Nuget.config, but that ends up in yet other errors.
I shared the output of
dotnet --infobelow.Is there anything I should do differently to consume and test .NET 8 NativeAOT for macOS x64?
Are there dogfooding instructions for this scenario somewhere that I haven't located?
`dotnet --info` output:
cc @jkotas
The text was updated successfully, but these errors were encountered: