Add a `zig cc`-based image. #880

Alexhuszagh · 2022-06-29T07:59:03Z

Uses cargo-zigbuild as a backend, and adds configuration options for zig under [build.zig] and [target.(...).zig]. If enabled, and an image override is not provided, cross will always use the zig image.

It supports custom glibc versions by parsing the libc portion of the target, and extracting a the libc version if present. The target, if built-in, is then the triple/libc pair, otherwise, it's just the triple.

The image does not provide runners, bindgen Clang args, or pkg-config paths, since zig cc does not provide the dynamic library loader (ld-linux*.so) required, meaning none of the binaries can be run. For bindgen, zig cc has an unusual directory structure, so there is no traditional sysroot with usr, lib, and include subdirectories. Finally, since we don't have system packages we can work with, exporting a pkg-config path makes little sense.

Closes #860.

Alexhuszagh · 2022-06-29T08:00:47Z

I still need to figure out the best way to add it to CI (it's currently not built as part of the matrix), since it can target numerous other architectures. It might be good to just change the exported triple from zig to a well-known target like aarch64-unknown-linux-gnu, which supports C++, etc.

docker/Dockerfile.zig

src/lib.rs

Alexhuszagh · 2022-06-30T16:57:46Z

I've changed the format entirely.

cross ... --target now only accepts valid Rust triples, and the config format is as follows:

[target.x86_64-unknown-linux-gnu.zig]
enable = true       # enable use of the zig image
version = "2.17"    # glibc version to use
image = "ghcr.io/cross-rs/zig:local"    # custom image to use

If provided as a bool, it will use the default glibc version:

[target.x86_64-unknown-linux-gnu]
# equivalent to { enable = true }
zig = true

If provided as a string, zig will be automatically enabled:

[target.x86_64-unknown-linux-gnu]
# equivalent to { enable = true, version = "2.17" }
zig = "2.17"

A custom zig image is so users can provide their own, general-purpose zig image if needed. These values can also be provided via environment variables, such as CROSS_BUILD_ZIG, CROSS_BUILD_ZIG_VERSION, and CROSS_BUILD_ZIG_IMAGE, or similar values for specific targets.

src/lib.rs

docker/zig.sh

src/docker/shared.rs

src/lib.rs

CHANGELOG.md

ci/test-zig-image.sh

Alexhuszagh · 2022-06-30T18:40:31Z

Actually we should provide an ARCH here. I'll mark this as a draft since we need to download zig from a binary anyway.

Alexhuszagh · 2022-06-30T19:46:40Z

bors try --target zig

Ok I've changed the build in zig.sh to fetch tarballs when possible, or build from source when we can't. This way, at least we should support numerous different target platforms There's a good chance the bors command fails (due to a permissions error when creating the cache), but weirdly it works on my machine so we'll see....

Alexhuszagh · 2022-06-30T19:58:02Z

Oh this is why it's failing:

$ /usr/bin/docker run --userns host -e 'PKG_CONFIG_ALLOW_CROSS=1' -e 'XARGO_HOME=/xargo' -e 'CARGO_HOME=/cargo' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e CARGO_HTTP_CHECK_REVOKE -e RUSTFLAGS -e CARGO_NET_RETRY -e CARGO_INCREMENTAL -e 'USER=runner' --rm --user 1001:121 -v /home/runner/.xargo:/xargo:Z -v /home/runner/.cargo:/cargo:Z -v /cargo/bin -v /tmp/tmp.xdLqWsDyv2:/project:Z -v /home/runner/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:Z,ro -v /tmp/tmp.xdLqWsDyv2/target:/target:Z -w /project ghcr.io/cross-rs/zig:trying sh -c 'PATH=$PATH:/rust/bin cargo-zigbuild build --target aarch64-unknown-linux-gnu --verbose'

Specifically --rm --user 1001:121, and we have RUN adduser --uid 1000 --disabled-password --gecos "" cross in the Dockerfile.

Alexhuszagh · 2022-06-30T20:23:56Z

Oh this is why it's failing:

$ /usr/bin/docker run --userns host -e 'PKG_CONFIG_ALLOW_CROSS=1' -e 'XARGO_HOME=/xargo' -e 'CARGO_HOME=/cargo' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e CARGO_HTTP_CHECK_REVOKE -e RUSTFLAGS -e CARGO_NET_RETRY -e CARGO_INCREMENTAL -e 'USER=runner' --rm --user 1001:121 -v /home/runner/.xargo:/xargo:Z -v /home/runner/.cargo:/cargo:Z -v /cargo/bin -v /tmp/tmp.xdLqWsDyv2:/project:Z -v /home/runner/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:Z,ro -v /tmp/tmp.xdLqWsDyv2/target:/target:Z -w /project ghcr.io/cross-rs/zig:trying sh -c 'PATH=$PATH:/rust/bin cargo-zigbuild build --target aarch64-unknown-linux-gnu --verbose'

Specifically --rm --user 1001:121, and we have RUN adduser --uid 1000 --disabled-password --gecos "" cross in the Dockerfile.

Ok this has been fixed by not creating a user, but by setting $XDG_CACHE_HOME which will work as long as we're on Linux, which will write it to the target directory (under /target/.zig-cache, to avoid any naming conflicts). If we ever support non-Linux hosts, this may need to be revisited.

Alexhuszagh · 2022-07-17T04:41:54Z

This won't work but the log will be helpful.
bors try --target zig

Alexhuszagh · 2022-07-17T05:11:22Z

bors try --target zig

.github/workflows/ci.yml

bors · 2022-07-17T05:25:35Z

try

Build succeeded:

conclusion

src/lib.rs

src/docker/custom.rs

Uses cargo-zigbuild as a backend, and adds configuration options for zig under `[build.zig]` and `[target.(...).zig]`. If enabled, and an image override is not provided, `cross` will always use the `zig` image. The feature can be enabled by providing `zig` as a table, bool, or string. It supports custom glibc versions by passing the `zig.version` key, and `zig` can be separately enabled or disabled by providing `zig.enable`. ``` [target.x86_64-unknown-linux-gnu.zig] enable = true # enable use of the zig image version = "2.17" # glibc version to use image = "ghcr.io/cross-rs/zig:local" # custom image to use ``` If provided as a bool, it will use the default glibc version: ``` [target.x86_64-unknown-linux-gnu] \# equivalent to { enable = true } zig = true ``` If provided as a string, `zig` will be automatically enabled: ``` [target.x86_64-unknown-linux-gnu] \# equivalent to { enable = true, version = "2.17" } zig = "2.17" ``` The image does not provide runners, `bindgen` Clang args, or `pkg-config` paths, since `zig cc` does not provide the dynamic library loader (`ld-linux*.so`) required, meaning none of the binaries can be run. For `bindgen`, `zig cc` has an unusual directory structure, so there is no traditional sysroot with `usr`, `lib`, and `include` subdirectories. Finally, since we don't have system packages we can work with, exporting a `pkg-config` path makes little sense.

src/lib.rs

Emilgardis

awesome!

Alexhuszagh · 2022-07-17T21:05:28Z

bors r=Emilgardis

bors · 2022-07-17T21:34:48Z

Build succeeded:

conclusion

Alexhuszagh added enhancement container-images labels Jun 29, 2022

Alexhuszagh added this to the v0.2.3 milestone Jun 29, 2022

Emilgardis reviewed Jun 29, 2022

View reviewed changes

docker/Dockerfile.zig Outdated Show resolved Hide resolved

src/lib.rs Outdated Show resolved Hide resolved

Alexhuszagh force-pushed the zigcc branch 2 times, most recently from e22afde to b6a5611 Compare June 29, 2022 17:00