Distribute libc.a as a separate library instead of bundling it into liblibc.rlib on musl and wasi

[petrochenkov]

Right now the libc crate links native libc with #[link(kind = "static")] for two targets - musl and wasi.

kind = "static" means that object files from native libc.a are bundled into liblibc.rlib and distributed with rustc toolchain in this form.
Then if user builds something with +crt-static then the bundled object files are used, and if user builds with -crt-static then they are "unbundled" and dynamic linking with libc.so available at user site happens.

Experience with windows-gnu targets (#67429) and porting rust to musl-native targets (#40113) showed that it's strongly preferable to use the native toolchain available at user site to using anything bundled with rustc toolchain, and that includes libc.a as well.

At the same time rustc historically provides more "self-contained" experience for selected targets (windows-gnu, musl, wasm), so we must continue shipping libc.a in some form in case the user doesn't have a native toolchain available.

That we means we need to fall back from using the native toolchain if it's available to using bundled libc.a (and other libraries) if it's not.
Such scheme is already used relatively successfully for windows-gnu, but not for musl and wasi.

So, the plan of the work should be roughly like this:

• Link libc as kind = "static-nobundle" instead of kind = "static" in the libc crate (Don't bundle static libc. libc#1327).
• Modify rustbuild to distribute libc.a for musl (and whatever is its equivalent for wasi) similarly to how we distribute crt1.o objects for the same targets.
• Organize library search directories during linking in such way that they find the native toolchain first and then fall back to the rust toolchain's sysroot where the bundled libc.a resides (should be similar to windows-gnu: prefer system crt libraries if they are available #67429).
• Eliminate the "unbundling" logic from the compiler.

Once this is done #65760 should become fixed as well.

UPD: libunwind (https://github.com/rust-lang/rust/tree/master/src/libunwind) on musl also bundles libunwind.a, it should probably have the same treatment.

cc @hvenev @gnzlbg @alexcrichton @mati865 @smaeul

[retep998]

If only we could also eliminate kind = "static" with fire...

[alexcrichton]

FWIW It's hard for me personally at least to read comments like "Eliminate the unbundling logic from the compiler (with fire)." and "If only we could also eliminate kind = static with fire..." as semi-attacks on me personally since I originally implemented all of this. I really do feel that much ire about these features is directed personally at me since I've tried to review patches and provide feedback on other designs, and often times things don't work out. I did the best I could with the resources and knowledge I had at the time. I'm sorry if things didn't work out.

The general idea here is "things just work". I don't think it really matters how they work, so long as they work. The "thing" here for targets like musl and wasi is "you don't need to install a native toolchain". I think anyone has much of a preference how things works so long as it works.

I'm gonna unsubscribe from this issue because I don't think I have much to offer and I'm likely to just get stressed out staying on this. Feel free to cc me if there's a specific question (e.g. about wasi), however.

[petrochenkov]

@alexcrichton
Apologies if some of this sounded like an attack.
This post is written with 5 years of hindsight available, but when this was originally implemented nobody could predict the future.

[retep998]

I'd also like to apologize for my comment. You've done a lot of fantastic work on Rust over the years, and I don't want any of my comments on specific features to feel like they're directed at you!
We've all done stuff in Rust that later turned out to not be the greatest (myself included), but such is hindsight.

kind=static has some unfortunate consequences such as LLVM mangling certain sections it doesn't understand (like Microsoft's LTCG stuff) and not being able to link to static libraries provided by the system. Unless they need to distribute rlibs with C libraries bundled in (which is rare outside of the rust toolchain) there's very little reason for users to not use static-nobundle over static. That it has taken the easy to remember name of static while static-nobundle has been left to languish in unstable land for years is somewhat frustrating, and I wish the situation could improve.

[petrochenkov]

while static-nobundle has been left to languish in unstable land for years

FWIW, I want to write an RFC for general syntax for linking modifiers for native libraries that would cover things like --whole-archive and --as-needed, and could cover bundling as well.
Perhaps it could be less controversial to stabilize it as a modifier to static rather than a separate linking kind.
(This will hopefully happen this or next month, otherwise I think static-nobundle can be stabilized as is.)

UPD: Done in rust-lang/rfcs#2951.

[mati865]

That would also fix #58163

[petrochenkov]

List of libraries that need to be shipped with Rust rather than bundled in rlibs:

• libc.a on musl (linked from libc crate on crates.io)
• libunwind.a on musl (linked from libunwind crate in tree)
• libc.a on wasi (linked from libc crate on crates.io)

Additionally:

• libunwind.a is also unconditionally bundled (regardless of the crt-static flag) on fuchsia and linux when llvm-libunwind feature of libstd is enabled, see src\libunwind\libunwind.rs.

[joshtriplett]

Would this make it easier to decide at build time whether to statically or dynamically link libc, for targets that currently always dynamically link libc?

[petrochenkov]

@joshtriplett
I don't think so?
This issue applies to two specific groups of targets, and both of them can currently link libc statically.

[joshtriplett]

@petrochenkov I realize that. I'm more wondering, if we can defer linking against libc until the final link, could we potentially decide at the final link whether to link libc.a or libc.so, even on glibc?

[petrochenkov]

@joshtriplett
Ah, yes, sure.
Whether libc is linked statically or dynamically is already decided during the final link (on targets that give this choice).

glibc-based targets don't give it right now, but changing that is a matter of switching a flag in the target spec + adding a couple of cfg lines into libc crate.
(I wanted to do this for bionic libc for months, but still didn't because other tasks were more important and contributing to libc is annoying.)

[smaeul]

@joshtriplett To clarify the functional impact of this issue:

For the listed targets, in scenarios where we have already decided to link statically using libc.a (and only for those scenarios), this allows choosing where libc.a comes from at the time of the final link. Currently, libc.a is always the version that was in musl-root or wasi-root when liblibc/rustc was built. With this change, we would be able to dynamically decide between that libc.a and the libc.a provided by the native toolchain.

The expectation is that the native toolchain's libc.a may be newer (security fixes) and will match the native headers.

[joshtriplett]

@petrochenkov

glibc-based targets don't give it right now, but changing that is a matter of switching a flag in the target spec + adding a couple of cfg lines into libc crate.

That'd be incredible. I'd be happy to submit the necessary changes, if you can outline what needs doing. I have a need for static glibc-based Rust binaries.

[mingweishih]

Is there any follow-up on this issue?

[petrochenkov]

@mingweishih
This is still waiting on someone to implement it.
With item 3 done this should actually be a pretty small task at this point.

[petrochenkov]

The only thing required for implementing the first item now is bumping libc version used by this repo.

[petrochenkov]

libc has been updated in #90681.

[petrochenkov]

The last item ('Eliminate the "unbundling" logic from the compiler') was implemented in #105601, closing.