Using a shebang-less script as a linker doesn't work anymore on Linux

[glandium]

STR:

• cargo new foo
• cd foo
• echo exit 42 > linker
• chmod +x linker
• RUSTFLAGS="-Clinker=$PWD/linker" cargo +nightly build

Actual result:

error: could not exec the linker `/tmp/foo/linker`
  |
  = note: Exec format error (os error 8)

With 1.63:

error: linking with `/tmp/foo/linker` failed: exit status: 42

This is a regression from #95026. What's going on is that before that merge, the linker was executed via posix_spawnp@GLIBC_2.2.5, and after, it's executed via posix_spawnp@GLIBC_2.15. The difference between the two is that the former retries executing through the shell when the execution first failed with ENOEXEC. This applies to anything the compiler or cargo would execute via std::process::Command. The change in posix_spawnp behavior in glibc comes from https://sourceware.org/bugzilla/show_bug.cgi?id=13134. It's probably for the better, but it should probably be highlighted in the release notes.

[Mark-Simulacrum]

This would happen with anything using Command, right? Not just rustc/cargo.

[Mark-Simulacrum]

Nominating for libs-api to consider whether we want to workaround this by implementing the re-execution ourselves; my feeling is no but it is a regression so I think worth asking.

[glandium]

It only happens when using libstd-*.so from the newer rustc builds, or when building with any rustc against an old glibc.

[glandium]

That is, if you build some random crate that uses Command with any version of rustc, the behavior depends on whether the libc used for the crate compile is < 2.15 or not, not whether rustc is 1.64 or not.

[glandium]

rustc 1.64 itself is affected by the change because it's built with a newer glibc, while older versions of rustc were built with an older glibc.

[glandium]

Note that it might be worth reviewing what other functions from glibc used by rustc/cargo have changed and how, because there might be subtle changes like this hidden somewhere.

[Mark-Simulacrum]

Ah, that makes sense. It seems likely that folks might bump their build environments to a newer glibc since we only support newer ones now, so I think I'll still leave this nominated, but it's good if it affects few users in practice.

T-compiler may want to do that checking as you suggest and consider a (temporary, perhaps with a future compat warning?) fallback, separately from any decisions by std.

[m-ou-se]

Nominating for libs-api to consider whether we want to workaround this by implementing the re-execution ourselves; my feeling is no but it is a regression so I think worth asking.

I also think the answer should be "no", but agree it's worth calling out in release notes or similar.

[cuviper]

Especially given #101511 (comment), that users who link against newer glibc were already getting the new behavior, I don't think we should change to preserve the old.

[m-ou-se]

The consensus in the libs meeting was to not fall back to /bin/sh.

[m-ou-se]

Reassigning to the compiler team. We're not planning to change std::process::Command, but they might want to consider adding special handling in the compiler for this.

[joshtriplett]

I don't think the compiler team should have a special case for this, unless there's some real-world use case that's affecting projects. @glandium, is there some case that's hitting the specific use case of a linker in practice, or is this a more theoretical concern or a concern about the behavior of Command in general?

[glandium]

I don't remember on what I hit this specifically, but I did hit it on some existing code that was using a linker wrapper without a shebang. I can tell for sure it was not Firefox. At the very least, the behavior change should be called out.

[apiraino]

WG-prioritization assigning priority (Zulip discussion).

Tagging for release notes, I feel the suggestion in the first comment makes sense.

@rustbot label -I-prioritize +P-low +release-notes