Xcode 16-beta compilation error: non-private labels cannot appear between .cfi_startproc / .cfi_endproc pairs

[vcsjones]

I was doing a little bit of scouting on macOS 16 / Xcode 16 and ran in to a compiler error on Xcode 16.

The gist of the error is

  <instantiation>:5:1: error: non-private labels cannot appear between .cfi_startproc / .cfi_endproc pairs

Repeated many times. The full build log is available at https://gist.github.com/vcsjones/1649f2fb8b19ab86428582b6884229c0

Note this does not require a beta of macOS, only the developer tools.

[dotnet-policy-service]

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

[filipnavara]

I may take a look. Was it Xcode 16 beta 1 or 2?

[vcsjones]

I may take a look. Was it Xcode 16 beta 1 or 2?

Beta 2. 16A5171r.

[filipnavara]

Beta 2. 16A5171r.

That explains it. I run it earlier with Beta 1 and that still builds on my machine (cannot completely rule out that Cmake cache still forced some other Xcode version)... I'll switch to Beta 2 soon.

[filipnavara]

I can reproduce. At first sight, it seems that using .alt_entry should be possible (llvm/llvm-project#82268):

--- a/src/coreclr/nativeaot/Runtime/unix/unixasmmacrosarm64.inc
+++ b/src/coreclr/nativeaot/Runtime/unix/unixasmmacrosarm64.inc
@@ -24,9 +24,11 @@ C_FUNC(\Name):
 .endm
 
 .macro ALTERNATE_ENTRY Name
-        .global C_FUNC(\Name)
 #if !defined(__APPLE__)
+        .global C_FUNC(\Name)
         .hidden C_FUNC(\Name)
+#else
+        .alt_entry C_FUNC(\Name)
 #endif
 C_FUNC(\Name):
 .endm

It still fails with the following errors:

  [ 37%] Building ASM object nativeaot/Runtime/Full/CMakeFiles/Runtime.ServerGC.dir/__/arm64/PInvoke.S.o
  <instantiation>:2:9: error: invalid CFI advance_loc expression
          .cfi_restore fp
          ^
  <instantiation>:2:9: error: invalid CFI advance_loc expression
          .cfi_restore fp
          ^
  <instantiation>:2:9: error: invalid CFI advance_loc expression
          .cfi_adjust_cfa_offset -0x60
          ^
  <instantiation>:2:9: error: invalid CFI advance_loc expression
          .cfi_adjust_cfa_offset -0x40
          ^
  make[3]: *** [nativeaot/Runtime/Full/CMakeFiles/Runtime.WorkstationGC.dir/__/arm64/UniversalTransition.S.o] Error 1
  make[3]: *** Waiting for unfinished jobs....
  make[3]: *** [nativeaot/Runtime/Full/CMakeFiles/Runtime.WorkstationGC.dir/__/arm64/ExceptionHandling.S.o] Error 1
  <instantiation>:2:9: error: invalid CFI advance_loc expression
          .cfi_adjust_cfa_offset -0x60
          ^
  <instantiation>:2:9: error: invalid CFI advance_loc expression
          .cfi_adjust_cfa_offset -0x40
          ^
  [ 37%] Building ASM object nativeaot/Runtime/Full/CMakeFiles/Runtime.ServerGC.dir/__/arm64/InteropThunksHelpers.S.o
  [ 37%] Building ASM object nativeaot/Runtime/Full/CMakeFiles/Runtime.ServerGC.dir/__/arm64/UniversalTransition.S.o
  make[3]: *** [nativeaot/Runtime/Full/CMakeFiles/Runtime.ServerGC.dir/__/arm64/ExceptionHandling.S.o] Error 1
  make[3]: *** Waiting for unfinished jobs....
  [ 37%] Building ASM object nativeaot/Runtime/Full/CMakeFiles/Runtime.ServerGC.dir/__/arm64/StubDispatch.S.o
  [ 37%] Building ASM object nativeaot/Runtime/Full/CMakeFiles/Runtime.ServerGC.dir/__/arm64/WriteBarriers.S.o
  [ 37%] Building ASM object nativeaot/Runtime/Full/CMakeFiles/Runtime.ServerGC.dir/__/arm64/ThunkPoolThunks.S.o
  make[2]: *** [nativeaot/Runtime/Full/CMakeFiles/Runtime.WorkstationGC.dir/all] Error 2
  make[2]: *** Waiting for unfinished jobs....
  <instantiation>:2:9: error: invalid CFI advance_loc expression
          .cfi_restore fp
          ^
  <instantiation>:2:9: error: invalid CFI advance_loc expression
          .cfi_restore fp
          ^

Hopefully it will be relatively easy to figure it out and the linker won't complain about the .alt_entry.

[filipnavara]

So, on the upside... the .alt_entry seems to do the trick for the first error, and links. On the downside, the "invalid CFI advance_loc expression" error could be a compiler bug from a change that was supposed to detect the same condition as the other error (llvm/llvm-project@0b06727).

This doesn't compile either:

.section __TEXT,__text
.globl _foo
_foo:
  .cfi_startproc
  sub sp, sp, 8
  .cfi_adjust_cfa_offset 8
  .alt_entry _bar
_bar:
  add sp, sp, 8
  .cfi_adjust_cfa_offset -8
  ret
  .cfi_endproc

with the error

src/test.S:10:3: error: invalid CFI advance_loc expression
  .cfi_adjust_cfa_offset -8

I'll have to sleep on it, and read more about the .alt_entry semantics.

[filipnavara]

Filed as LLVM issue: llvm/llvm-project#97116
We will still need to switch to using .alt_entry, and we may need to do that in ObjWriter as well (FYI @ivanpovazan).

[filipnavara]

The advance_loc bug has been fixed in upstream LLVM. I filed Apple Feedback FB14191360 to make sure the fix gets included in Xcode 16.

We have two options on how to proceed now:

1. Wait and see for beta 3 or 4 if it makes it, then submit a PR to use .alt_entry
2. Submit the .alt_entry patch now and add code to not emit DWARF epilogs for the broken Xcode versions. This should only slightly affect the debuggability of the runtime code itself. We can then update the code once fixed Xcode is released.

Thoughts?

As for ObjWriter, we should modify the symbol emitting code to track symbol names with non-zero offsets and emit the N_ALT_ENTRY flag for them. This will tell the linker it's not a separate "atom" but a label inside an existing "atom". Additionally, this may also require some changes for the (de)hydrated data to emit it as single atom. I plan to do the work on this at some point.

[filipnavara]

The LLVM fix was still not backported as of Xcode 16 beta 4.

[agocke]

Confirmed that this is still a problem. I say we wait for Beta 5, and then if it's still not there, we make the change as suggested.

[filipnavara]

Xcode 16 beta 5 was released today. The bug is still present. I'll look into submitting a PR with .alt_entry and relevant workarounds later this week.

[filipnavara]

Small update. There are currently two blocking bugs in the Apple Clang compiler, both were already fixed upstream but they are still present in Xcode 16 beta 5:

• Mach-O check for improperly nested .cfi_* regions doesn't take .alt_entry into account llvm/llvm-project#82261
• Mach-O check for improperly nested .cfi_* regions doesn't take .alt_entry into account llvm/llvm-project#97116 (also filed as Apple feedback FB14191360)

Additionally, some preliminary testing with new linker uncovered one new bug:

• Use .alt_entry/.private_extern on Apple platforms #106224 (comment) (also filed as Apple feedback FB14743667)

The linker bug is likely non-blocking. We have a workaround in place that passes the smoke tests (N_NO_DEAD_STRIP flag added in #103039). I'll attempt to make a better workaround later.

[filipnavara]

Xcode 16.1 beta was released. It ships Apple clang version 16.0.0 (clang-1600.0.24.1) which is the same version as Xcode 16 beta 5, and it does NOT resolve the issue.

[filipnavara]

Xcode 16 beta 6 seems to ship a fix for the compiler bugs 🎉