cmake: LLVM needs to link against zlib #8932

andrewrk · 2021-05-29T03:53:36Z

Without this, LLVM is crippled. This makes building Zig on Windows more
involved and is probably going to lead to more issues filed.

For more details see
ziglang/zig-bootstrap#57

daurnimator · 2021-05-29T03:56:04Z

Is this a static-linking-only dependency? (i.e. the dynamic LLVM library would already depend on libz?)

andrewrk · 2021-05-29T04:13:27Z

Yes

daurnimator · 2021-05-29T04:25:44Z

CMakeLists.txt

@@ -713,11 +713,14 @@ set_target_properties(zigcpp PROPERTIES
    COMPILE_FLAGS ${EXE_CFLAGS}
 )

+find_library(ZLIB NAMES libz.a z zlib libz)


Why is libz.a first? shouldn't the following z find it anyway?

I copied this line from another place in the CMakeLists.txt in order to quickly get this working to test the zig-bootstrap changes. This could probably be cleaned up (I'm no cmake expert) and should probably be put behind some ZIG_USE_ZLIB flag which is automatically enabled if ZIG_STATIC is true.

kubkon · 2021-05-29T06:47:15Z

One thing to keep in mind here: if this forces linking of -lz on macOS and LLVM links against symbols from libz, then macOS binary will be malformed. This will be until I add ability to parse "tbd"s in zld. Until then, might it make sense to feature-gate libz so that it is not pulled into macOS?

kubkon · 2021-05-29T12:31:12Z

One thing to keep in mind here: if this forces linking of -lz on macOS and LLVM links against symbols from libz, then macOS binary will be malformed. This will be until I add ability to parse "tbd"s in zld. Until then, might it make sense to feature-gate libz so that it is not pulled into macOS?

I just read ziglang/zig-bootstrap#58 and since we are building libz ourselves and linking it in statically, I think we should be good on macOS without any special feature-gating or whatnot. It would only be a problem if libz was a system lib on a current macOS and only a tbd was available. Then, we would not correctly link libz since zig ld currently doesn't understand text-based definition files.

nektro · 2021-05-29T14:58:55Z

this is also presumably only a temporary measure until #8726 is complete

edit: that issue was placed a lot farther out than i remembered

For more details on why this dependency is needed, see ziglang/zig-bootstrap#57

andrewrk · 2021-05-30T00:43:20Z

I really don't understand how this is causing a segfault on freebsd while making aarch64-linux print "zig0: not found"

andrewrk · 2021-05-30T00:45:26Z

(lldb) run src/stage1.zig -target x86_64-freebsd-gnu -mcpu=baseline --name zig1 --override-lib-dir /usr/home/build/zig/lib -femit-bin=/usr/home/build/zig/build/zig1.o -OReleaseFast --strip -lc --pkg-begin build_options /usr/home/build/zig/build/config.zig --pkg-end --pkg-begin compiler_rt /usr/home/build/zig/lib/std/special/compiler_rt.zig --pkg-end
Process 1585 launching
Process 1585 launched: '/usr/home/build/zig/build/zig0' (x86_64)
Process 1585 stopped
* thread #1, name = 'zig0', stop reason = signal SIGSEGV: invalid address (fault address: 0x0)
    frame #0: 0x0000000000000000
error: Bad address
(lldb) up
error: Already at the top of the stack.

🤷‍♂️

andrewrk · 2021-05-30T00:46:42Z

I'm guessing that we're incorrectly finding zlib on these systems but it's invalid for whatever reason and if we provide our own build it will work fine. Just need to update the CI tarballs to test this.

to llvm 12.0.1-rc1, and -DLLVM_ENABLE_ZLIB=FORCE_ON

andrewrk · 2021-05-30T03:29:24Z

I have no idea why drone CI is failing. It works fine with the bootstrap repository.

mikdusan · 2021-05-30T05:19:48Z

I really don't understand how this is causing a segfault on freebsd while making aarch64-linux print "zig0: not found"

examine zig0 binary with this PR on local freebsd 13.0: debug+static build of zig:

$ file zig0
zig0: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked, for FreeBSD 13.0 (1300139), FreeBSD-style, with debug_info, not stripped
$ ldd zig0
zig0:
zig0: signal 11

notable is that it is a dynamic exe
here are the interesting last args to the c++ -static ... command to produce zig0:

-lrt -lexecinfo -lpthread -lm -Wl,-Bstatic -lz -Wl,-Bdynamic -lpthread -lm -Wl,-Bstatic -lz -Wl,-Bdynamic

here is what those same args look like on master:

-lrt  -lexecinfo  -lpthread  -lm  -lz  -lpthread  -lm  -lz

and after applying the same args to this PR's build (i.e. getting rid of that -Wl,-Bstatic and -Bdynamic nonsense) some sane results appear:

$ file zig0
zig0: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), statically linked, for FreeBSD 13.0 (1300139), FreeBSD-style, with debug_info, not stripped
$ ldd zig0
ldd: zig0: not a dynamic ELF executable

That first zig0 executable is not a proper dynamic executable yet it tries to be: c++ -static means no ability to use dynamic libraries. Subsequent -Wl,-Bstatic are probably harmless, but -Wl,-Bdynamic is likely the culprit.

edit: it stands to reason that the zig0: not found error is also the same issue, instead of SEGV that elf platform gets further but cannot find/load a .so because bogus dynamic exe

to llvm 12.0.1-rc1, and -DLLVM_ENABLE_ZLIB=FORCE_ON

kubkon · 2021-05-30T06:49:55Z

FYI, I've updated macos tarballs to the latest zig-bootstrap, pushed them to our s3 instance and updated the CI scripts. Fingers crossed everything is going to work out fine!

andrewrk · 2021-05-30T18:46:12Z

Thanks @kubkon!

Based on @mikdusan's comment I expect FreeBSD to be fixed based solely on an updated tarball, which I will do today. And then it's just the Drone CI mystery left to solve. The nuclear option would be to stop trying to get alpine linux to build our tarballs, but instead only use it to test, and then use the bootstrap repository to provide the aarch64-linux tarballs. This is probably a good idea; arguably all the Zig tarballs we provide on the download page should be via zig-bootstrap.

kubkon · 2021-05-30T18:52:31Z

Thanks @kubkon!

Based on @mikdusan's comment I expect FreeBSD to be fixed based solely on an updated tarball, which I will do today. And then it's just the Drone CI mystery left to solve. The nuclear option would be to stop trying to get alpine linux to build our tarballs, but instead only use it to test, and then use the bootstrap repository to provide the aarch64-linux tarballs. This is probably a good idea; arguably all the Zig tarballs we provide on the download page should be via zig-bootstrap.

I definitely agree with the last bit: all the Zig tarballs we provide on the download page should be via zig-bootstrap.

to llvm 12.0.1-rc1, and -DLLVM_ENABLE_ZLIB=FORCE_ON

andrewrk · 2021-06-02T06:28:16Z

FreeBSD tarball is updated; let's see if it passes. I also created the aarch64-musl tarball (zig+llvm+lld+clang-aarch64-linux-musl-0.8.0-dev.2703+c12704a33.tar.xz) but need to rework the docker image to use it which will have to wait until my mind is fresh tomorrow.

andrewrk · 2021-06-03T00:19:57Z

whew, I think this may finally be in a passing state.

daurnimator reviewed May 29, 2021

View reviewed changes

andrewrk mentioned this pull request May 29, 2021

Cross compile and link zlib ziglang/zig-bootstrap#58

Merged

andrewrk force-pushed the llvm-needs-zlib branch from 75e2b57 to b3c0b1a Compare May 29, 2021 21:28

cmake: LLVM needs to link against zlib

f8bc529

For more details on why this dependency is needed, see ziglang/zig-bootstrap#57

andrewrk force-pushed the llvm-needs-zlib branch from b3c0b1a to f8bc529 Compare May 30, 2021 00:22

ci: update x86_64-linux, aarch64-linux, and windows tarballs

e46c55e

to llvm 12.0.1-rc1, and -DLLVM_ENABLE_ZLIB=FORCE_ON

ci: update x86_64-macos and aarch64-macos tarballs

0ad534b

to llvm 12.0.1-rc1, and -DLLVM_ENABLE_ZLIB=FORCE_ON

vrischmann mentioned this pull request May 30, 2021

failing build on alpine edge vrischmann/zig-sqlite#32

Closed

ci: update x86_64-freebsd tarball

16c5cba

to llvm 12.0.1-rc1, and -DLLVM_ENABLE_ZLIB=FORCE_ON

andrewrk force-pushed the llvm-needs-zlib branch 5 times, most recently from bcc464d to 98afc04 Compare June 2, 2021 23:54

ci: drone: use zig-bootstrap tarball instead of system toolchain

05cd893

andrewrk force-pushed the llvm-needs-zlib branch from 98afc04 to 05cd893 Compare June 3, 2021 00:03

andrewrk merged commit 87dae0c into master Jun 3, 2021

andrewrk deleted the llvm-needs-zlib branch June 3, 2021 06:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmake: LLVM needs to link against zlib #8932

cmake: LLVM needs to link against zlib #8932

andrewrk commented May 29, 2021

daurnimator commented May 29, 2021

andrewrk commented May 29, 2021

daurnimator May 29, 2021

ifreund May 29, 2021

kubkon commented May 29, 2021

kubkon commented May 29, 2021

nektro commented May 29, 2021 •

edited

Loading

andrewrk commented May 30, 2021

andrewrk commented May 30, 2021

andrewrk commented May 30, 2021

andrewrk commented May 30, 2021

mikdusan commented May 30, 2021 •

edited

Loading

kubkon commented May 30, 2021

andrewrk commented May 30, 2021

kubkon commented May 30, 2021

andrewrk commented Jun 2, 2021

andrewrk commented Jun 3, 2021

cmake: LLVM needs to link against zlib #8932

cmake: LLVM needs to link against zlib #8932

Conversation

andrewrk commented May 29, 2021

daurnimator commented May 29, 2021

andrewrk commented May 29, 2021

daurnimator May 29, 2021

Choose a reason for hiding this comment

ifreund May 29, 2021

Choose a reason for hiding this comment

kubkon commented May 29, 2021

kubkon commented May 29, 2021

nektro commented May 29, 2021 • edited Loading

andrewrk commented May 30, 2021

andrewrk commented May 30, 2021

andrewrk commented May 30, 2021

andrewrk commented May 30, 2021

mikdusan commented May 30, 2021 • edited Loading

kubkon commented May 30, 2021

andrewrk commented May 30, 2021

kubkon commented May 30, 2021

andrewrk commented Jun 2, 2021

andrewrk commented Jun 3, 2021

nektro commented May 29, 2021 •

edited

Loading

mikdusan commented May 30, 2021 •

edited

Loading