[NativeAOT][8.0] Use ld_classic in ILC build and in build integration #97856

VSadov · 2024-02-02T07:33:56Z

Customer Impact

Programs produced in an environment running XCode version 15 or higher will experience various crashes upon starting.

Also, if a build machine is upgraded to xcode 15+ it can't complete the build successfully as ILC, being a NativeAOT component, will not be runnable, thus stages such as building tests would fail.

Analysis

XCode 15 has introduced a new linker that contains a number of incompatibilities. Some of those incompatibilities may be unintentional and may eventually be fixed. It seems possible that point releases (i.e 15.0 vs. 15.2) have different set of issues.
For the time being there is a way to force the use of the old (aka "classic") linker and get around incompatibilities.

While we can chase incompatibilities and adapt to them in the active development branch, in 8.0 branch it makes more sense to opt for "classic" behavior.

Testing

Manual testing locally with XCode 15.2 and regular lab run with XCode 14 that is still used by the lab.

Risk

Low. This change conditionally reintroduces preexisting linker behavior.

There is a risk that at some point "classic" linker will no longer be supported.
It will likely be far enough in the future that the "prime" linker becomes more stable and the runtime/compiler could be adapted to work with it at that time.

ghost · 2024-02-02T07:34:08Z

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes: #97745

Author:	VSadov
Assignees:	-
Labels:	`area-NativeAOT-coreclr`
Milestone:	-

filipnavara · 2024-02-05T07:23:43Z

I think we should do it. Xamarin workloads in .NET 8 already force the old linker, and the new linker appears quite buggy. We can try to workaround/fix things for .NET 9 and backport them if Apple decides to ditch the old linker.

jkotas

The OSX builds are failing with ld: library not found for -ld_classic.

MichalPetryka · 2024-02-05T15:52:45Z

The OSX builds are failing with ld: library not found for -ld_classic.

Probably the code needs to test if XCode is 15 or later?

filipnavara · 2024-02-05T15:57:05Z

Probably the code needs to test if XCode is 15 or later?

Possibly, and use -Wl, prefix.

src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets

am11 · 2024-02-05T16:34:17Z

src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj

@@ -79,6 +79,7 @@

  <ItemGroup Condition="'$(NativeAotSupported)' == 'true'">
    <CustomLinkerArg Condition="'$(CrossBuild)' == 'true' and '$(_hostArchitecture)' == '$(_targetArchitecture)' and '$(_hostOS)' != 'windows' and '$(_IsApplePlatform)' != 'true'" Include="--gcc-toolchain=$(ROOTFS_DIR)/usr" />
+    <CustomLinkerArg Condition="'$(_IsApplePlatform)' == 'true'" Include="-ld_classic" />


This shouldn't be needed?

IIRC it uses the released package, not the one we just built, so it is necessary.

VSadov · 2024-02-05T16:54:21Z

Probably the code needs to test if XCode is 15 or later?

Is there a way to check for that?
Or perhaps we should use -ld64 and ignore the warning that -ld-classic is preferred.

am11 · 2024-02-05T17:47:30Z

Is there a way to check for that?

We added major version detection for lld (for linker script):

runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets

Lines 202 to 208 in 034d9f1

    
           <Exec Command="&quot;$(CppLinker)&quot; -fuse-ld=lld -Wl,--version" Condition="'$(LinkerFlavor)' == 'lld'" IgnoreExitCode="true" StandardOutputImportance="Low" ConsoleToMSBuild="true"> 
        
             <Output TaskParameter="ExitCode" PropertyName="_LinkerVersionStringExitCode" /> 
        
             <Output TaskParameter="ConsoleOutput" PropertyName="_LinkerVersionString" /> 
        
           </Exec> 
        
           <PropertyGroup Condition="'$(_LinkerVersionStringExitCode)' == '0' and '$(_LinkerVersionString)' != ''"> 
        
             <_LinkerVersion>$([System.Text.RegularExpressions.Regex]::Match($(_LinkerVersionString), '[1-9]\d*'))</_LinkerVersion>

exact same regex returns 15 with xcodebuild -version command on my machine ™️.

src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets

jkotas

LGTM modulo feedback.

@VSadov Could you please get it approved for servicing once ready?

Also, we will want to apply it to main as well: #98117 (comment)

carlossanlop · 2024-02-08T22:43:10Z

Friendly reminder that Monday February 12th is the Code Complete deadline for the March Release. Please make sure to address the feedback and then send the email to Tactics requesting approval.

…e.Unix.targets PR feedback Co-authored-by: Jan Kotas <jkotas@microsoft.com>

VSadov · 2024-02-09T06:33:26Z

I've re-validated that the fix still works after the PR feedback changes and that UseLdClassicXCodeLinker also works - right now export UseLdClassicXCodeLinker=false results in test build failures, as expected

jeffschwMSFT

approved. we could consider this a tell mode change to accommodate a coming xcode change

VSadov · 2024-02-12T17:49:34Z

Thanks!!

…dotnet#97856) * use ld_classic in ILC build and in build integration * PR feedback * Maybe fix the build for non-apple * Update src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets PR feedback Co-authored-by: Jan Kotas <jkotas@microsoft.com> --------- Co-authored-by: Jan Kotas <jkotas@microsoft.com>

) (#98726) Forward port of #97856 Workaround for #97745

This pool is only available internally for now, so let's use it! * Add workaround from #97856 to use classic linker on macos

use ld_classic in ILC build and in build integration

034d9f1

dotnet-issue-labeler bot added the area-NativeAOT-coreclr label Feb 2, 2024

ghost assigned VSadov Feb 2, 2024

filipnavara approved these changes Feb 5, 2024

View reviewed changes

jkotas approved these changes Feb 5, 2024

View reviewed changes

jkotas requested changes Feb 5, 2024

View reviewed changes

ghost added the needs-author-action An issue or pull request that requires more info or actions from the author. label Feb 5, 2024

am11 reviewed Feb 5, 2024

View reviewed changes

src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets Outdated Show resolved Hide resolved

am11 reviewed Feb 5, 2024

View reviewed changes

ghost removed the needs-author-action An issue or pull request that requires more info or actions from the author. label Feb 5, 2024

VSadov added 2 commits February 5, 2024 19:56

PR feedback

2b3ab63

Maybe fix the build for non-apple

d790091

VSadov marked this pull request as ready for review February 7, 2024 06:41

VSadov requested a review from MichalStrehovsky as a code owner February 7, 2024 06:41

filipnavara mentioned this pull request Feb 8, 2024

Use new MacOS arm64 pool for official builds #98117

Merged

jkotas reviewed Feb 8, 2024

View reviewed changes

src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets Outdated Show resolved Hide resolved

jkotas approved these changes Feb 8, 2024

View reviewed changes

Update src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Nativ…

a90e308

…e.Unix.targets PR feedback Co-authored-by: Jan Kotas <jkotas@microsoft.com>

VSadov added the Servicing-consider Issue for next servicing release review label Feb 9, 2024

build-analysis bot mentioned this pull request Feb 9, 2024

Tracking issue for CI build timeouts #76454

Closed

jeffschwMSFT approved these changes Feb 9, 2024

View reviewed changes

VSadov added this to the 8.0.x milestone Feb 10, 2024

steveisok pushed a commit to steveisok/runtime that referenced this pull request Feb 12, 2024

Add workaround from dotnet#97856 to use classic linker on macos

432eb1f

jeffschwMSFT added Servicing-approved Approved for servicing release and removed Servicing-consider Issue for next servicing release review labels Feb 12, 2024

jeffschwMSFT modified the milestones: 8.0.x, 8.0.3 Feb 12, 2024

jeffschwMSFT merged commit bb540a8 into dotnet:release/8.0-staging Feb 12, 2024
110 of 114 checks passed

VSadov deleted the 80stage branch February 12, 2024 17:49

jkotas mentioned this pull request Feb 20, 2024

[NativeAOT] Use ld_classic in ILC build and in build integration #98726

Merged

jkotas added a commit that referenced this pull request Feb 21, 2024

[NativeAOT] Use ld_classic in ILC build and in build integration (#97856

11bdd1b

) (#98726) Forward port of #97856 Workaround for #97745

steveisok added a commit that referenced this pull request Feb 29, 2024

Use new MacOS arm64 pool for official builds (#98117)

89f7ad3

This pool is only available internally for now, so let's use it! * Add workaround from #97856 to use classic linker on macos

github-actions bot locked and limited conversation to collaborators Mar 14, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[NativeAOT][8.0] Use ld_classic in ILC build and in build integration #97856

[NativeAOT][8.0] Use ld_classic in ILC build and in build integration #97856

VSadov commented Feb 2, 2024 •

edited

Loading

ghost commented Feb 2, 2024

filipnavara commented Feb 5, 2024 •

edited

Loading

jkotas left a comment

MichalPetryka commented Feb 5, 2024

filipnavara commented Feb 5, 2024

am11 Feb 5, 2024

filipnavara Feb 5, 2024

VSadov commented Feb 5, 2024

am11 commented Feb 5, 2024 •

edited

Loading

jkotas left a comment

carlossanlop commented Feb 8, 2024

VSadov commented Feb 9, 2024

jeffschwMSFT left a comment

VSadov commented Feb 12, 2024

[NativeAOT][8.0] Use ld_classic in ILC build and in build integration #97856

[NativeAOT][8.0] Use ld_classic in ILC build and in build integration #97856

Conversation

VSadov commented Feb 2, 2024 • edited Loading

Customer Impact

Analysis

Testing

Risk

ghost commented Feb 2, 2024

filipnavara commented Feb 5, 2024 • edited Loading

jkotas left a comment

Choose a reason for hiding this comment

MichalPetryka commented Feb 5, 2024

filipnavara commented Feb 5, 2024

am11 Feb 5, 2024

Choose a reason for hiding this comment

filipnavara Feb 5, 2024

Choose a reason for hiding this comment

VSadov commented Feb 5, 2024

am11 commented Feb 5, 2024 • edited Loading

jkotas left a comment

Choose a reason for hiding this comment

carlossanlop commented Feb 8, 2024

VSadov commented Feb 9, 2024

jeffschwMSFT left a comment

Choose a reason for hiding this comment

VSadov commented Feb 12, 2024

VSadov commented Feb 2, 2024 •

edited

Loading

filipnavara commented Feb 5, 2024 •

edited

Loading

am11 commented Feb 5, 2024 •

edited

Loading