using extern crate syntax produces a binary dynamically linked to libstd.so that fails with "cannot open shared object file" when executed

[japaric]

STR

$ rustup default nightly

$ cat foo.rs
#![feature(rustc_private)]

extern crate syntax;

fn main() {}

$ rustc foo.rs

$ ldd foo
        linux-vdso.so.1 =>  (0x00007ffdfafd2000)
        libstd-cb705824.so => not found
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff4359e1000)
        /lib64/ld-linux-x86-64.so.2 (0x0000559219c45000)

$ ./foo
./foo: error while loading shared libraries: libstd-cb705824.so: cannot open shared object file: No such file or directory

$ find ~/.multirust -name 'libsyntax*'
/root/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libsyntax-cb705824.so
/root/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libsyntax_ext-cb705824.so
/root/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libsyntax-cb705824.so
/root/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libsyntax_ext-cb705824.so

I was sort of expecting the compilation to fail because there is no libsyntax.rlib in ~/.multirust and rustc produces binaries statically linked to all the Rust bits. But perhaps syntax is special because of syntax extensions?

Meta

$ rustc -Vv
rustc 1.10.0-nightly (2174bd97c 2016-04-14)
binary: rustc
commit-hash: 2174bd97c1458d89a87eb2b614135d7ad68d6f18
commit-date: 2016-04-14
host: x86_64-unknown-linux-gnu
release: 1.10.0-nightly

cc @alexcrichton

[japaric]

Just in case, the output of rustc -Z print-link-args:

$ rustc -Z print-link-args | sed "s/ /\n/g"
"cc"
"-Wl,--as-needed"
"-Wl,-z,noexecstack"
"-m64"
"-L"
"/root/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib"
"foo.0.o"
"-o"
"foo"
"-Wl,--gc-sections"
"-pie"
"-nodefaultlibs"
"-L"
"/root/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib"
"-Wl,-Bstatic"
"-Wl,-Bdynamic"
"-L"
"/root/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib"
"-l"
"syntax-cb705824"
"-L"
"/root/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib"
"-l"
"serialize-cb705824"
"-L"
"/root/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib"
"-l"
"term-cb705824"
"-L"
"/root/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib"
"-l"
"log-cb705824"
"-L"
"/root/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib"
"-l"
"std-cb705824"
"-l"
"dl"
"-l"
"pthread"
"-l"
"gcc_s"
"-l"
"pthread"
"-l"
"c"
"-l"
"m"
"-l"
"rt"
"-l"
"util"
"-l"
"compiler-rt"

[alexcrichton]

Ah yeah this is expected behavior. There's a nice big comment in dependency_format.rs explaining this, but the gist is that if you link to libsyntax then the following happens:

• The compiler determines that libsyntax is only available as a dylib
• The compiler determines that this dylib is dynamically linked to libstd
• The compiler thinks your program wants to link to libstd
• The compiler deduces the only way to succeed is to dynamically link both

The logic changes subtly if libsyntax.rlib is available (as the compiler prefers static linking), but we don't ship rlibs of the host libraries as it'd just inflate our distribution size.

Closing as intended, however, but I don't mind answering any questions about this as well!

[japaric]

Thanks for the explanation. I have two further questions:

• Why running the binary fails with

$ ./foo
./foo: error while loading shared libraries: libstd-cb705824.so: cannot open shared object file: No such file or directory

? when libstd.so is in the .multirust directory:

$ find ~/.multirust -name 'libstd-*'
~/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libstd-cb705824.so
~/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-cb705824.rlib
~/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-cb705824.so

• Why I get 'error: can't find crate for syntax' when building the same file on host=ARM or host=FreeBSD (basically Unable to use syntax crate on FreeBSD with nightly #32984) when libsyntax.so is available in ~/.multirust:

# host=ARM
$ find ~/.multirust -name 'libsyntax-*'
~/.multirust/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/libsyntax-9cd959afa717c4db.so

? But I think the answer is that rustc expects libsyntax.so to be in $sysroot/lib/rustlib/$target/lib, right? At least that's the difference between the host=x86_64 installation and the host=arm installation:

# host=x86_64
$ find ~/.multirust -name 'libsyntax-*'
~/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libsyntax-cb705824.so
~/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libsyntax-cb705824.so

[alexcrichton]

Why running the binary fails

Is that directory in LD_LIBRARY_PATH?

Why I get 'error: can't find crate for syntax' when building

Yeah the compiler only looks for libraries in rustlib. The ones directly in lib are only meant to be there at runtime for the compiler itself, not linked against.

[japaric]

Is that directory in LD_LIBRARY_PATH?

No, that's why it wasn't working. LD_LIBRARY_PATH=$(rustc --print sysroot)/lib ./foo works. Should rustup set that variable in ~/.cargo/env? (cc @brson) Does multirust set that variable? Because I don't recall ever running into problems with LD_LIBRARY_PATH before, but perhaps I never executed Rust binaries that are dynamically linked to Rust crates 😄.

[alexcrichton]

Yeah this is rarely run into precisely b/c we statically link everything. I suspect rustup doesn't want to do too much here because it's so "non standard", but it certainly could!