Process wrappers should be statically linked against musl

[aaronmondal]

Description of the feature request:

At the moment the process wrappers are built against glibc as any other regular package. This wasn't a problem with traditional ubuntu-based remote execution images, but it complicates test execution and makes it harder to run Bazel against e.g. alpine and nix-based images which often don't contain glibc or have it in unexpected locations.

Workarounds exist, e.g. using patchelf to patch the process wrappers, but that's a fairly complicated thing to do and it means that off-the-shelf Bazel executables don't work in such setups.

Which category does this issue belong to?

Remote Execution

What underlying problem are you trying to solve with this feature?

This is similar to #22551 but potentially smaller in scope. It would be ideal to have a fully musl-based Bazel, but just the process wrappers would already be a big improvement over the current UX.

The biggest issue IMO is that it's not possible to use the glibc-based wrappers during remote test execution. If the executable was built against a newer version of glibc you can't change the LD_LIBRARY_PATH to an older version as that would unbreak the processwrapper but break the tested executable.

So AFAIK the only way around this is to build executables against a compatible glibc (meaning same or lower version than what Bazel uses). On e.g. nix-based systems (and importantly containers) that's not possible and I'd expect the same to be the case for any other base distros that use rolling releases with very recent glibc versions such as Gentoo, Arch etc.

This also makes it harder to run Bazel on musl-based systems like Alpine. For remote execution it's not necessary to have Bazel in the remote image. So you could use Bazel on Ubuntu invoking into an Alpine image. Except that then you'd hit the process wrapper issue again. If the wrappers were fully self-contained it would make it possible to use Alpine-based images as test runners.

Which operating system are you running Bazel on?

Gentoo, NixOS, Alpine, random custom minimal OCI images

What is the output of bazel info release?

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

What's the output of git remote get-url origin; git rev-parse HEAD ?

-

Have you found anything relevant by searching the web?

#5891
bazelbuild/rules_rust#2656
#12579

Any other information, logs, or outputs that you want to share?

No response

[meisterT]

How much larger are the resulting binaries if you link them statically?

[fmeum]

@aaronmondal Could you explain how processwrapper is involved in remote execution? My understanding is that it is only used during local execution.