[NativeAOT] Initial osx-arm64 bring up #75264

filipnavara · 2022-09-08T12:53:17Z

Depends on dotnet/llvm-project#185.
Fixes #67232
Fixes #64337

Thanks to @am11 @MichalStrehovsky and @jkotas for the help with pushing this forward and analysing all the issues that came up.

src/coreclr/nativeaot/Runtime/unix/UnwindHelpers.cpp

am11 · 2022-09-08T13:06:14Z

src/coreclr/gc/unix/gcenv.unix.cpp

@@ -633,7 +633,7 @@ void GCToOSInterface::YieldThread(uint32_t switchCount)
 static void* VirtualReserveInner(size_t size, size_t alignment, uint32_t flags, uint32_t hugePagesFlag = 0)
 {
    assert(!(flags & VirtualReserveFlags::WriteWatch) && "WriteWatch not supported on Unix");
-    if (alignment == 0)
+    if (alignment < OS_PAGE_SIZE)


Wrap under #if defined(HOST_ARM64) && defined(TARGET_OSX) && defined(FEATURE_NATIVEAOT) to limit the scope of this change?

I would still like to know why CoreCLR doesn't hit this but I assume that it's because of some GC feature switch (such as FEATURE_MANUALLY_MANAGED_CARD_BUNDLES).

While I can add the #if guards the code would still misbehave fatally when alignment < OS_PAGE_SIZE so I think the change is quite safe.

I would still like to know why CoreCLR doesn't hit this

AFAIK gcenv.unix.cpp is only used by CoreCLR when standalone GC is used (i.e. COMPlus_GCName is set to point to a standalone GC instead of the one in coreclr.dll/.so). @mangod9 @dotnet/gc can you have a look at this change?

Yes, the regular CI pipelines do not use this code. We should trigger gc-standalone pipeline https://dev.azure.com/dnceng/public/_build?definitionId=1017 against this PR to get realistic results (windows arm64 checked failure in that pipeline is preexisting, the rest should be green). AFAIK, there is no azp run trigger for that pipeline, someone with access will need to trigger it manually.

AFAIK, there is no azp run trigger for that pipeline, someone with access will need to trigger it manually.

I tried to trigger it but it's complaining about the pool. It doesn't look like the gc-standalone pipeline ran for a week or so. It coincides with the pool changes. Cc @dotnet/gc.

The interesting thing about the gc-standalone pipeline is that it doesn't test mac at all. So if there was a arm64 macOS issue with standalone GC, we wouldn't even know it.

if there was a arm64 macOS issue with standalone GC, we wouldn't even know it.

This change as written is not specific to arm64 macOS, other platforms can break. My original commit was am11@45f3a48, but it was recreated without the guard which has potential to break existing code.

The specific call with the "invalid" alignment parameter is macOS arm64 specific though. It happens because of 16Kb page sizes and GC_PAGE_SIZE != OS_PAGE_SIZE.

src/coreclr/nativeaot/Runtime/CommonMacros.h

src/coreclr/nativeaot/Runtime/unix/unixasmmacrosarm64.inc

src/native/external/llvm-libunwind/src/DwarfParser.hpp

src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp

src/native/external/llvm-libunwind/src/CompactUnwinder.hpp

filipnavara · 2022-09-09T10:25:02Z

The last two commits unblock the library tests. It seems to work quite well, so we may try to get a full run on the CI once this is merged.

filipnavara · 2022-09-09T12:07:16Z

src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/DependencyAnalysis/FatFunctionPointerNode.cs

-            builder.RequireInitialAlignment(factory.Target.MinimumFunctionAlignment);
+            // These need to be aligned the same as method pointers because they show up in same contexts
+            // (macOS ARM64 has even stricter alignment requirement for the linker, so round up to pointer size)
+            builder.RequireInitialAlignment(factory.Target.PointerSize);


This one is actually hell to diagnose because it produces no error but causes corruption when linking. If absolute relocation of a symbols is 4-byte aligned and spans between two pages in the executable then only the lower part of the relocation is written correctly into the final executable.

This was the root cause of the intermittent problem I've hit earlier.

filipnavara · 2022-09-09T18:00:25Z

I ran the library test locally and the results look promising:

/Users/filipnavara/Projects/runtime/eng/testing/tests.targets(181,5): error : One or more tests failed while running tests from 'System.Console.Tests'. [/Users/filipnavara/Projects/runtime/src/libraries/System.Console/tests/System.Console.Tests.csproj]
/Users/filipnavara/Projects/runtime/eng/testing/tests.targets(181,5): error : One or more tests failed while running tests from 'System.Diagnostics.Process.Tests'. [/Users/filipnavara/Projects/runtime/src/libraries/System.Diagnostics.Process/tests/System.Diagnostics.Process.Tests.csproj]
/Users/filipnavara/Projects/runtime/eng/testing/tests.targets(181,5): error : One or more tests failed while running tests from 'System.Security.Cryptography.OpenSsl.Tests'. [/Users/filipnavara/Projects/runtime/src/libraries/System.Security.Cryptography.OpenSsl/tests/System.Security.Cryptography.OpenSsl.Tests.csproj]
/Users/filipnavara/Projects/runtime/eng/testing/tests.targets(181,5): error : One or more tests failed while running tests from 'System.Runtime.Extensions.Tests'. [/Users/filipnavara/Projects/runtime/src/libraries/System.Runtime.Extensions/tests/System.Runtime.Extensions.Tests.csproj]

Specifically, System.Runtime.Extensions.Tests has single failure related to macOS Ventura (#75371):

  [FAIL] System.Tests.EnvironmentTests.OSVersion_ValidVersion_OSX
  Assert.Contains() Failure
  Not found: 13
  In value:  osx-arm64
     at System.Tests.EnvironmentTests.OSVersion_ValidVersion_OSX() + 0x68
     at System.Runtime.Extensions!<BaseAddress>+0x1256b74
     at System.Runtime.RawCalliHelper.Call(IntPtr, Void*, Byte&, Byte&, Void*) + 0x40
     at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x27c

System.Security.Cryptography.OpenSsl.Tests fails because of missing OpenSSL library (expected):

  Running assembly:System.Security.Cryptography.OpenSsl.Tests, Version=7.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
  No usable version of libssl was found
  /Users/filipnavara/Projects/runtime/artifacts/bin/System.Security.Cryptography.OpenSsl.Tests/Debug/net7.0-unix/publish/RunTests.sh: line 168: 63790 Abort trap: 6           ./System.Security.Cryptography.OpenSsl.Tests -notrait category=OuterLoop -notrait category=failing -xml testResults.xml $RSP_FILE
  ~/Projects/runtime/src/libraries/System.Security.Cryptography.OpenSsl/tests
  ----- end Fri Sep 9 19:49:15 CEST 2022 ----- exit code 134 ----------------------------------------------------------
  exit code 134 means SIGABRT Abort. Managed or native assert, or runtime check such as heap corruption, caused call to abort(). Core dumped.
  ulimit -c value: 0
/Users/filipnavara/Projects/runtime/eng/testing/tests.targets(181,5): error : One or more tests failed while running tests from 'System.Security.Cryptography.OpenSsl.Tests'. [/Users/filipnavara/Projects/runtime/src/libraries/System.Security.Cryptography.OpenSsl/tests/System.Security.Cryptography.OpenSsl.Tests.csproj]

System.Diagnostics.Process.Tests has one failure (flaky):

  [FAIL] System.Diagnostics.Tests.ProcessTests.GetProcessesByName_ProcessName_ReturnsExpected
  Assert.All() Failure: 1 out of 2 items in the collection did not pass.
  [0]: Item: System.Diagnostics.Process (System.Diagnost)
       Xunit.Sdk.EqualException: Assert.Equal() Failure
       Expected: 2022-09-09T19:49:07.9700180+02:00
       Actual:   2022-09-09T19:49:07.9916470+02:00
          at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) + 0x16c
          at Xunit.Assert.Equal[T](T expected, T actual) + 0x40
          at System.Diagnostics.Tests.ProcessTests.<>c__DisplayClass90_0.<GetProcessesByName_ProcessName_ReturnsExpected>b__3(Process process) + 0x40
          at Xunit.Assert.<>c__DisplayClass11_0`1.<All>b__0(T item, Int32 index) + 0x3c
          at Xunit.Assert.All[T](IEnumerable`1 collection, Action`2 action) + 0xf0
     at System.Diagnostics.Tests.ProcessTests.GetProcessesByName_ProcessName_ReturnsExpected() + 0x350
     at System.Diagnostics.Process!<BaseAddress>+0x11234f4
     at System.Runtime.RawCalliHelper.Call(IntPtr, Void*, Byte&, Byte&, Void*) + 0x40
     at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x27c

System.Console.Tests has couple of failures related to Remote Executor (missing RemoteExecutor guard):

[FAIL] Color.RedirectedOutput_EnvVarSet_EmitsAnsiCodes(envVar: null)
  System.PlatformNotSupportedException : RemoteExecutor is not supported on this platform.
     at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(MethodInfo method, String[] args, RemoteInvokeOptions options, Boolean pasteArguments) + 0xb8
     at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(Action`1 method, String arg, RemoteInvokeOptions options) + 0x6c
     at Color.RedirectedOutput_EnvVarSet_EmitsAnsiCodes(String envVar) + 0x174
     at System.Console!<BaseAddress>+0x10c0ad8
     at System.Runtime.RawCalliHelper.Call(IntPtr, Void*, Byte&, Byte&, Void*) + 0x40
     at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x358
  [FAIL] Color.RedirectedOutput_EnvVarSet_EmitsAnsiCodes(envVar: "1")
  System.PlatformNotSupportedException : RemoteExecutor is not supported on this platform.
     at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(MethodInfo method, String[] args, RemoteInvokeOptions options, Boolean pasteArguments) + 0xb8
     at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(Action`1 method, String arg, RemoteInvokeOptions options) + 0x6c
     at Color.RedirectedOutput_EnvVarSet_EmitsAnsiCodes(String envVar) + 0x174
     at System.Console!<BaseAddress>+0x10c0ad8
     at System.Runtime.RawCalliHelper.Call(IntPtr, Void*, Byte&, Byte&, Void*) + 0x40
     at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x358
  [FAIL] Color.RedirectedOutput_EnvVarSet_EmitsAnsiCodes(envVar: "true")
  System.PlatformNotSupportedException : RemoteExecutor is not supported on this platform.
     at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(MethodInfo method, String[] args, RemoteInvokeOptions options, Boolean pasteArguments) + 0xb8
     at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(Action`1 method, String arg, RemoteInvokeOptions options) + 0x6c
     at Color.RedirectedOutput_EnvVarSet_EmitsAnsiCodes(String envVar) + 0x174
     at System.Console!<BaseAddress>+0x10c0ad8
     at System.Runtime.RawCalliHelper.Call(IntPtr, Void*, Byte&, Byte&, Void*) + 0x40
     at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x358
  [FAIL] Color.RedirectedOutput_EnvVarSet_EmitsAnsiCodes(envVar: "tRuE")
  System.PlatformNotSupportedException : RemoteExecutor is not supported on this platform.
     at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(MethodInfo method, String[] args, RemoteInvokeOptions options, Boolean pasteArguments) + 0xb8
     at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(Action`1 method, String arg, RemoteInvokeOptions options) + 0x6c
     at Color.RedirectedOutput_EnvVarSet_EmitsAnsiCodes(String envVar) + 0x174
     at System.Console!<BaseAddress>+0x10c0ad8
     at System.Runtime.RawCalliHelper.Call(IntPtr, Void*, Byte&, Byte&, Void*) + 0x40
     at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x358
  [FAIL] Color.RedirectedOutput_EnvVarSet_EmitsAnsiCodes(envVar: "0")
  System.PlatformNotSupportedException : RemoteExecutor is not supported on this platform.
     at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(MethodInfo method, String[] args, RemoteInvokeOptions options, Boolean pasteArguments) + 0xb8
     at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(Action`1 method, String arg, RemoteInvokeOptions options) + 0x6c
     at Color.RedirectedOutput_EnvVarSet_EmitsAnsiCodes(String envVar) + 0x174
     at System.Console!<BaseAddress>+0x10c0ad8
     at System.Runtime.RawCalliHelper.Call(IntPtr, Void*, Byte&, Byte&, Void*) + 0x40
     at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x358
  [FAIL] Color.RedirectedOutput_EnvVarSet_EmitsAnsiCodes(envVar: "false")
  System.PlatformNotSupportedException : RemoteExecutor is not supported on this platform.
     at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(MethodInfo method, String[] args, RemoteInvokeOptions options, Boolean pasteArguments) + 0xb8
     at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(Action`1 method, String arg, RemoteInvokeOptions options) + 0x6c
     at Color.RedirectedOutput_EnvVarSet_EmitsAnsiCodes(String envVar) + 0x174
     at System.Console!<BaseAddress>+0x10c0ad8
     at System.Runtime.RawCalliHelper.Call(IntPtr, Void*, Byte&, Byte&, Void*) + 0x40
     at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x358

am11 · 2022-09-10T01:21:05Z

We can enable CI testing in the next PR (after this and #75055 are merged).

Thanks a lot @filipnavara for pushing the boundaries! 🐎

src/coreclr/nativeaot/Runtime/ThunksMapping.cpp

src/native/external/llvm-libunwind-version.txt

…[_W] definitions

Co-authored-by: Adeel Mujahid <3840695+am11@users.noreply.github.com>

…te symbol relocations to be aligned to native pointer size.

…ain absolute relocations. ld64 seem to silently corrupt them if they are not aligned to pointer size.

filipnavara · 2022-09-11T07:04:09Z

Rebased after #75393 merge.

src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp

jkotas · 2022-09-11T18:59:10Z

@filipnavara Thank you!

tgjones · 2022-09-12T09:19:05Z

@filipnavara thank you for this!

A general question: is there an intention to backport this to 7.0, or is it strictly an 8.0 thing?

filipnavara · 2022-09-12T09:22:55Z

A general question: is there an intention to backport this to 7.0, or is it strictly an 8.0 thing?

It missed the 7.0 deadline by a lot. However, it is possible to consume the ILCompiler package from NuGet so I think the realistic goal is to make it possible to use that with .NET 7 SDK.

jkotas · 2022-09-12T15:06:40Z

The instructions for using daily build packages are at https://github.com/dotnet/runtime/blob/main/src/coreclr/nativeaot/docs/compiling.md#using-daily-builds

The alignment changed in #75264, but it's important this is always at least MinimumFunctionAlignment no matter the platform because that's how we distinguish fat pointers from real pointers. This assert is obviously true on every platform we support right now.

dotnet-issue-labeler bot added the area-NativeAOT-coreclr label Sep 8, 2022

ghost added the community-contribution Indicates that the PR has been added by a community member label Sep 8, 2022

filipnavara changed the title ~~[NativeAOT] osx-arm64 bring up~~ [NativeAOT] Initial osx-arm64 bring up Sep 8, 2022

am11 approved these changes Sep 8, 2022

View reviewed changes

filipnavara commented Sep 8, 2022

View reviewed changes

src/native/external/llvm-libunwind/src/DwarfParser.hpp Show resolved Hide resolved

am11 mentioned this pull request Sep 8, 2022

Reloc failures with NativeAOT on Apple Silicon #67232

Closed

filipnavara marked this pull request as ready for review September 8, 2022 19:02

filipnavara requested a review from MichalStrehovsky as a code owner September 8, 2022 19:02

VSadov reviewed Sep 8, 2022

View reviewed changes

src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp Show resolved Hide resolved

VSadov mentioned this pull request Sep 8, 2022

[NativeAOT] Revisit process/thread termination scenario #74711

Closed

am11 reviewed Sep 9, 2022

View reviewed changes

src/native/external/llvm-libunwind/src/CompactUnwinder.hpp Outdated Show resolved Hide resolved

filipnavara force-pushed the nativeaot-m1 branch from 1c05b75 to 3d54d63 Compare September 9, 2022 07:21

filipnavara commented Sep 9, 2022

View reviewed changes

filipnavara mentioned this pull request Sep 9, 2022

Intermittent crash in DynamicGenerics test on OSX x64 #73299

Closed

jkotas approved these changes Sep 9, 2022

View reviewed changes

filipnavara mentioned this pull request Sep 10, 2022

Flush instruction cache after thunk pool allocation #75393

Merged

jkotas reviewed Sep 10, 2022

View reviewed changes

src/coreclr/nativeaot/Runtime/ThunksMapping.cpp Show resolved Hide resolved

jkotas reviewed Sep 10, 2022

View reviewed changes

src/native/external/llvm-libunwind-version.txt Show resolved Hide resolved

filipnavara added 9 commits September 11, 2022 08:48

Enable NativeAOT osx-arm64 builds

ca95d4b

Restrict alignment in VirtualReserveInner

35a40f0

Fix generation and handling of compact unwinding info on osx-arm64

387b31b

Fix PC check in findFDE in DWARF CFI unwinding

50eb6f6

Handle MAP_JIT for P/Invoke on osx-arm64

cb81686

Handle P/Invoke with MAP_JIT on osx-arm64

69fd921

Fix incorrect OS_PAGE_SIZE definition

b05ba08

Fix TLS register trashing

8742268

Fix memory trashing caused by incorrect PREPARE_EXTERNAL_VAR_INDIRECT…

92a7164

…[_W] definitions

filipnavara and others added 9 commits September 11, 2022 08:48

Ignore ESRCH in PalHijack (thread is already gone)

6325ba7

Make CompactUnwinder_* parametrized with registry class

125bed7

Remove custom CompactUnwinder

817b10c

Update src/coreclr/nativeaot/Runtime/CommonMacros.h

f44843f

Co-authored-by: Adeel Mujahid <3840695+am11@users.noreply.github.com>

Update llvm-libunwind-version.txt

3308493

Fix initial alignment for __module_initializer. ARM64 requires absolu…

8b9d0dc

…te symbol relocations to be aligned to native pointer size.

Allow publishing with the osx-arm64 RID

9d73bfa

Use pointer sized alignment for fat function pointers since they cont…

5244049

…ain absolute relocations. ld64 seem to silently corrupt them if they are not aligned to pointer size.

Remove __builtin___clear_cache, it was already fixed

59b79b3

filipnavara force-pushed the nativeaot-m1 branch from c12a7be to 59b79b3 Compare September 11, 2022 07:01

jkotas mentioned this pull request Sep 11, 2022

System.Diagnostics.Tests.InstanceDataTests.InstanceDataCollectionCollection tests failing in CI #75417

Open

jkotas reviewed Sep 11, 2022

View reviewed changes

src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp Outdated Show resolved Hide resolved

Update src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp

d8c7ee0

jkotas merged commit cb6fb60 into dotnet:main Sep 11, 2022

filipnavara deleted the nativeaot-m1 branch September 11, 2022 19:05

teo-tsirpanis added this to the 8.0.0 milestone Sep 11, 2022

MichalStrehovsky mentioned this pull request Sep 13, 2022

Assert fat pointers aligned to MinimumFunctionAlignment #75574

Merged

ghost locked as resolved and limited conversation to collaborators Oct 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[NativeAOT] Initial osx-arm64 bring up #75264

[NativeAOT] Initial osx-arm64 bring up #75264

filipnavara commented Sep 8, 2022 •

edited

Loading

am11 Sep 8, 2022

filipnavara Sep 8, 2022

MichalStrehovsky Sep 9, 2022

am11 Sep 9, 2022 •

edited

Loading

MichalStrehovsky Sep 9, 2022 •

edited

Loading

am11 Sep 9, 2022

filipnavara Sep 9, 2022

filipnavara commented Sep 9, 2022

filipnavara Sep 9, 2022

filipnavara commented Sep 9, 2022 •

edited

Loading

am11 commented Sep 10, 2022

filipnavara commented Sep 11, 2022

jkotas commented Sep 11, 2022

tgjones commented Sep 12, 2022

filipnavara commented Sep 12, 2022

jkotas commented Sep 12, 2022

[NativeAOT] Initial osx-arm64 bring up #75264

[NativeAOT] Initial osx-arm64 bring up #75264

Conversation

filipnavara commented Sep 8, 2022 • edited Loading

am11 Sep 8, 2022

Choose a reason for hiding this comment

filipnavara Sep 8, 2022

Choose a reason for hiding this comment

MichalStrehovsky Sep 9, 2022

Choose a reason for hiding this comment

am11 Sep 9, 2022 • edited Loading

Choose a reason for hiding this comment

MichalStrehovsky Sep 9, 2022 • edited Loading

Choose a reason for hiding this comment

am11 Sep 9, 2022

Choose a reason for hiding this comment

filipnavara Sep 9, 2022

Choose a reason for hiding this comment

filipnavara commented Sep 9, 2022

filipnavara Sep 9, 2022

Choose a reason for hiding this comment

filipnavara commented Sep 9, 2022 • edited Loading

am11 commented Sep 10, 2022

filipnavara commented Sep 11, 2022

jkotas commented Sep 11, 2022

tgjones commented Sep 12, 2022

filipnavara commented Sep 12, 2022

jkotas commented Sep 12, 2022

filipnavara commented Sep 8, 2022 •

edited

Loading

am11 Sep 9, 2022 •

edited

Loading

MichalStrehovsky Sep 9, 2022 •

edited

Loading

filipnavara commented Sep 9, 2022 •

edited

Loading