Build Linux binaries with musl

[igor-hnizdo]

When I tried to build with "normal" glibc, I got weird errors on old/new glibc.

Instead of building with glibc, which limits us with compatibility and weird issues, let's build with musl, which will always produce 100% static library.

(Musl is a libc alternative. The good thing is musl is statically linked and can cross-compile relatively easily.)

The actual hard work is building all these musl cross-compilers; luckily, Rust people already did that for Rust :) we can steal that and reuse.

Note that this all works on both AMD and ARM hosts, as both the musl cross-compilers images and the go images are published for both.

[igor-hnizdo]

Converting it to draft. I think building static library with glibc is also possible, and at least we will depend only on the elastic repo, and not on the musl repo.

But let me try first. This still does work.

[prusnak]

I am fine with building the static binaries with musl. I think it's better than trying to use static with glibc.

[igor-hnizdo]

Yeah I tried to use static and libc and there were some weird issues. OK, putting it back to in progress.

[tsusanka]

Btw @igor-hnizdo why don't we use golang-crossbuild again for linux? We can't do that?

Oh or that's what you are writing in the original post? That it uses glibc which causes troubles?

[igor-hnizdo]

Yes. I tried that and I got some incompatibilities with the old glibc (I am not sure why, I got some random error when building); and the newer glibc might be incompatible with older Linux distros... maybe?

I did not want to debug this and "just" put there musl, which always builds all statically.

Unfortunately golang-crossbuild does not support musl (I have opened a issue there now)

elastic/golang-crossbuild#230

[igor-hnizdo]

Well, to do some investigation.

• golang-crossbuild with base images debian 7 and 8 does not work, because they have too old gcc.
• golang-crossbuild with base image debian 9 does work, but that uses glibc 2.24
• if we see the list of https://gist.github.com/wagenet/35adca1a032cec2999d47b6c40aa45b1 , there are lot of TLS distos with older glibc
• it probably worked with xgo before, as that install gcc-5 manually?
• we can install gcc 5 manually in the golang-crossbuild ourselves, but then we are already doing some patching... let's just use musl, which will always work :) and will save us distro-compatibility troubles long term, IMO

[tsusanka]

The docker runs fine, I can't test the binary as I have only macOS and NixOS available, but I think we can merge.

@mroz22 if you would like to feel free to test the build on Linux:

cd release/linux/
make clean all
build/trezord-linux-386

and see if it launches okay.

[mroz22]

make clean all produced

+ CGO_ENABLED=1 CC=/usr/local/musl/bin/aarch64-unknown-linux-musl-gcc GOARCH=arm64 go build --ldflags -linkmode external -extldflags "-static" -extld /usr/local/musl/bin/aarch64-unknown-linux-musl-gcc -o release/linux/build/trezord-linux-arm64
go: downloading gopkg.in/natefinch/lumberjack.v2 v2.0.0
go: downloading github.com/gorilla/handlers v1.5.1
go: downloading github.com/gorilla/mux v1.8.0
go: downloading github.com/gorilla/csrf v1.7.0
go: downloading github.com/felixge/httpsnoop v1.0.1
go: downloading github.com/pkg/errors v0.9.1
go: downloading github.com/gorilla/securecookie v1.1.1
error obtaining VCS status: exit status 128
	Use -buildvcs=false to disable VCS stamping.
make: *** [Makefile:18: .binary] Error 1

and it did not finish the build

[igor-hnizdo]

huh that's weird. I will look at that.

[igor-hnizdo]

It seems to be something with the git directory ownership and the way go does the magic stamping. (And how Docker on linux does users.)

golang/go#51253

I will try this on Linux computer to test and see

[igor-hnizdo]

Yeah. New git (released this april) adds a security measure, that dis-allows git traversal if the user is a different user.

It's a mitigation against CVE-2022-24765

https://nvd.nist.gov/vuln/detail/cve-2022-24765

This vulnerability affects users working on multi-user machines, where untrusted parties have write access to the same hard disk. Those untrusted parties could create the folder C:\.git, which would be picked up by Git operations run supposedly outside a repository while searching for a Git directory. Git would then respect any config in said Git directory.

And the user inside the docker is root, a different user than who owns the outside repo. And go then calls git which then errors.

We don't see this in the crossbuild repos, as they use old git. And we don't see it on macOS, as on macOS, docker pretends all shared files are from root, or something like that. (unlike on Linux, where it's just a layer over cgroups or some other linux magic, on macos it has all the virtual machine stuff added.)

The easiest solution is to add safe.directory to the build.sh, which I will now do.

[igor-hnizdo]

The fix here

#267