New cross images seem to require fairly new glibc versions (2.29)

[jszwedko]

Checklist

• I've looked through the issues and pull requests for similar reports

Describe your issue

I noticed that the new cross images seem to require a fairly new glibc version (2.29) for the built binaries for the x86_64-unknown-linux-gnu target.

We are using ghcr.io/cross-rs/x86_64-unknown-linux-gnu:main for the build image. I'll be trying to walk back and tag to an older image. I'll update when I find the most recent one that works.

An image SHA that seems to require 2.29 is sha256:ff64ffcf0960931b890b849c0e74946d49b10f09464d292bf39ebba35fd03309. Previously the image with SHA sha256:98dfbdc3e988dd9a7b359537a79ada1f6f1086fdbcffd953320c43949880a813 built binaries that required glibc 2.18 as determined by https://gist.github.com/fasterthanlime/17e002a8f5e0f189861c

What target(s) are you cross-compiling for?

aarch64-unknown-linux-gnu, aarch64-unknown-linux-musl, arm-unknown-linux-gnueabihf, x86_64-unknown-linux-gnu, x86_64-unknown-linux-musl

Which operating system is the host (e.g computer cross is on) running?

• macOS
• Windows
• Linux / BSD
• other OS (specify in description)

What architecture is the host?

• x86_64 / AMD64
• arm32
• arm64 (including Mac M1)

What container engine is cross using?

• docker
• podman
• other container engine (specify in description)

cross version

cross 0.2.1

Example

git clone https://github.com/vectordotdev/vector
make package-deb-x86_64-unknown-linux-gnu

Additional information / notes

I'll try a newer version of cross to see if that changes anything.

[jszwedko]

The ghcr.io/cross-rs/x86_64-unknown-linux-gnu:0.2.4 image seems to work in that binaries built using it do not require the 2.29 glibc versions. Seemingly this is a change in main since then.

[Alexhuszagh]

The ghcr.io/cross-rs/x86_64-unknown-linux-gnu:0.2.4 image seems to work in that binaries built using it do not require the 2.29 glibc versions. Seemingly this is a change in main since then.

This is by design and part of breaking changes to 0.3.0 where we updated all the Ubuntu versions to 20.04: see #591. This causes backwards incompatibility, which is why this is a breaking change. This was part of the goal in #417, since we needed to support older glibc versions (IE, keep the v0.2.4 images at 16.04) but also allow people in v0.3.0 to use newer toolsets since Ubuntu 16.04 is missing many modern packages.

[jszwedko]

The ghcr.io/cross-rs/x86_64-unknown-linux-gnu:0.2.4 image seems to work in that binaries built using it do not require the 2.29 glibc versions. Seemingly this is a change in main since then.

This is by design and part of breaking changes to 0.3.0 where we updated all the Ubuntu versions to 20.04: see #591. This causes backwards incompatibility, which is why this is a breaking change.

Thanks for the note! That would definitely do it. Will there be a recommendation to use cross to build binaries linked against older versions of libc? We try to support all non-EOL Debian versions and the built binary won't run on Debian 10 (and probably also other old OSes we try to support). Will we need to maintain our own cross base images?

[Alexhuszagh]

You shouldn't, since the v0.2.4 will still be deployed and you can build against them forever (if GIthub registry ever goes down, we can checkout the tag and re-build the images and host them to a new registry and deploy v0.2.5). However, we also have Zig image that uses zig cc to allow you to link against most glibc versions for a subset of targets. This will let you use much older glibc versions on newer Ubuntu versions.

This is documented on the wiki, for example:

[target.aarch64-unknown-linux-gnu]
zig = "2.17"

And this will link against glibc 2.17. If you really need newer Ubuntu versions and older Debian packages for the cross-compiled architecture, you can maintain your own Docker images and use the following logic for adding code from Debian repositories for the cross-compilers.

Note you would have to maintain your own images (although this a lot easier with pre-build hooks now, so they are auto-built when invoking cross), but at least we provide logic for how you would install older libraries/cross-compilers.

I hope that answers everything? Unless I'm misunderstanding something.

[jszwedko]

Thanks for the thorough response @Alexhuszagh !

Yeah, I realize we can keep using the 0.2.4 images. My concern was being left behind additional improvements. I think you addressed that, though, with the rest of your response. Once 0.3.0 is out, we'll try to upgrade and see how it goes. We have looked a little at zig but ran into issues cross-compiling some of our C/C++ dependencies. We'll dig into that a bit more when we try to upgrade.

[Alexhuszagh]

If you have any specific issues with maintaining backwards compatibility, we have a Matrix room and also the discussions forum: I'd be more than happy to help. Also, if there's something that seems it would be relevant to others, it's a great opportunity to improve our wiki documentation.

[Alexhuszagh]

If there's any major bug fixes, I'll see if I can backport them, since we do have a tag and can always create a new branch with bug fixes from there.