Apple: Re-implement SDK discovery instead of using xcrun

[madsmtm]

Shelling out to a global command line tool like this makes it difficult for rustc to properly implement change-tracking of the involved variables and paths (e.g. we could tell Cargo to relink after the user runs xcode-select --switch or installs/updates Xcode) as required for #118204, and makes error handling more difficult. So instead, this PR re-implements the SDK discovery logic that xcrun does, to avoid depending on that.

The logic is now roughly:

let sdkroot = if let Some(sdkroot) = env::var("SDKROOT") && sdk_matches(sdkroot, target_os) {
    sdkroot
} else if let Some(dir) = env::var("DEVELOPER_DIR") {
    search_for_platform_and_sdk(dir, sdk_name)
} else if let Some(path) = fs::read_link("/var/db/xcode_select_link")? { // Special path that `xcode-select --switch` configures, undocumented
    search_for_platform_and_sdk(path, sdk_name)
} else if fs::exists("/Applications/Xcode.app")? { // Hard-coded fall back
    search_for_platform_and_sdk("/Applications/Xcode.app/Contents/Developer", sdk_name)
} else if fs::exists("/Library/Developer/CommandLineTools")? { // Hard-coded fall back
    search_for_sdk("/Library/Developer/CommandLineTools/SDKs", sdk_name)
} else {
    panic!("tell the user that they need to install Xcode")
};

I have tested this in a newly created virtual machine, and verified that:

• When neither Xcode nor the command line tools are installed, an error message is shown which directs the user towards xcode-select --install, which fixes Detect linkage error when xcode-select --install hasn't been run #84534.
• When using a newly installed Xcode that hasn't yet been launched, linking fails and the error message shows a readable note about agreeing to the license with sudo xcodebuild -license, which fixes For *-apple-ios targets, if license agreement is not agreed to (yet), better behaviour is necessary #56829.

Part of #129432.

I'm not sure, but I don't think release notes are desirable here? The behaviour should be mostly identical apart from better error messages, and besides, this was very under-documented before.

CC @BlackHoleFox @thomcc.

[rustbot]

r? @TaKO8Ki

rustbot has assigned @TaKO8Ki.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

These commits modify compiler targets.
(See the Target Tier Policy.)

[workingjubilee]

not saying we shouldn't add this, but how long do you think is left until we get fed up with shoving this kind of code into cg_ssa and split out rustc_linker? ...or would it be rustc_archiver?

[madsmtm]

not saying we shouldn't add this, but how long do you think is left until we get fed up with shoving this kind of code into cg_ssa and split out rustc_linker? ...or would it be rustc_archiver?

No idea honestly, I'm not really familiar enough with the overall crate structure to say.

[BlackHoleFox]

Short initial review since I was tagged, but not an actual compiler reviewer etc etc.

[rustbot]

Some changes occurred in src/doc/rustc/src/platform-support

cc @Noratrieb

[workingjubilee]

r? compiler

[BlackHoleFox]

macOS changes here look fine to me.

[bors]

☔ The latest upstream changes (presumably #132497) made this pull request unmergeable. Please resolve the merge conflicts.

[pnkfelix]

I'm not comfortable approving this without more team discussion. it is adding an extra amount of maintenance labor, for gains that are ... unclear at best to me. (And I say this as someone who develops on a Mac ... though I freely admit that I also bias towards Linux for a lot of development.)

Maybe an MCP is warranted?

@rustbot label I-compiler-nominated

[pnkfelix]

@rustbot label: +S-waiting-on-team -S-waiting-on-review

[madsmtm]

it is adding an extra amount of maintenance labor, for gains that are ... unclear at best to me.

Yeah, that's a fair assessment, I guess an alternative is to parse the error messages from xcrun (they're just very bad, so it's kinda difficult to really glean what's going on), and I think we could still somewhat support change-tracking anyhow.

I'll admit that my main motivation here is that I don't like xcrun, though I do still think the reasons I've noted in a code comment has some merit:

The reason why we implement this logic ourselves is:

• Reading these directly is faster than spawning a new process.
• xcrun can be fairly slow to start up after a reboot.
• In the future, we will be able to integrate this better with the compiler's change tracking mechanisms, allowing rebuilds when the involved env vars and paths here change. See Rebuild if deployment target changes #118204.
• It's easier for us to emit better error messages.

[oli-obk]

r? compiler

[apiraino]

Discussed in T-compiler triage.

We are probably in favor of opening an MCP about this proposal, as Felix suggests. We should be trying to get a wider buyout from T-compiler since there are some concerns about mantainability and bus-factor (how many people are familiar with these changes?).

Thanks!

@rustbot label -I-compiler-nominated