Link using the linker directly

[brson]

Right now we link through a C compiler, but Rust should not depend on having a C compiler available nor expose details of cc linking semantics.

I know there have been other issues on this but I can't find them.

[alexcrichton]

I really like this idea, this has bit us in the past by unknowningly bringing in dependencies we weren't expecting.

However, I don't think that this will be an easy thing to do. I found out that compilers add a lot of command line parameters to the system linker. This will be a difficult thing to do on all systems, but it may be doable. I'm pasting below a survey of what I've got a VM for. I got all of these via cc -v foo.c and seeing how it invoked the linker. (I didn't wrap things in code blocks so sane word wrap is here)

OSX

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld -demangle -dynamic -arch x86_64 -macosx_version_min 10.9.0 -o a.out /var/folders/9s/r4snvt950_d9_rcl6w50tvw80000gn/T/a-QOBPrL.o -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.0/lib/darwin/libclang_rt.osx.a

Looks like everything here is guessable except for libclang_rt.osx.a, and that should probably get replaced by our own version of libcompiler-rt anyway.

Arch linux

"/usr/bin/ld.gold" --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o a.out /usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../../lib64/crt1.o /usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../../lib64/crti.o /usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/4.8.2/crtbegin.o -L/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/4.8.2 -L/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/4.8.2/../../.. -L/lib -L/usr/lib /tmp/foo-93f224.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/4.8.2/crtend.o /usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../../lib64/crtn.o

There's a lot of things going on here. We would need to find libgcc, libgcc_s, crtn.o, and maybe crtend.o? The libraries seem standard, but the crt files seem sketchy.

Ubuntu

/usr/bin/ld -z relro --hash-style=gnu --as-needed --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o a.out /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/crt1.o /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/4.6/crtbegin.o -L/usr/lib/gcc/x86_64-linux-gnu/4.6 -L/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/4.6/../../.. -L/lib/x86_64-linux-gnu -L/lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib /tmp/foo-fw3tGx.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/4.6/crtend.o /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/crtn.o

This looks fairly similar to arch. There's some weird flags I haven't seen much before, but they're in theory fairly guessable.

FreeBSD

/usr/bin/ld --eh-frame-hdr -dynamic-linker /libexec/ld-elf.so.1 --hash-style=both --enable-new-dtags -o a.out /usr/lib/crt1.o /usr/lib/crti.o /usr/lib/crtbegin.o -L/usr/lib /tmp/foo-zCRcGs.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/crtend.o /usr/lib/crtn.o

Nice an simple! Like ubuntu/arch, we'd have to find things like ld-elf, crtX.o, and the gcc libs

Windows (MinGW)

I'm not sure how to copy/paste out of a windows terminal, but the command line used collect2.exe instead of ld.exe (not sure if that even exists), and it had a whole bunch of libraries I've never even heard of: moldname, mingwew, msvcrt, advapi32, shell32, user32, kernel32, mingw32, gcc, gcc_eh.

Windows also had the usual crtN.o files to link in.

[thestinger]

@alexcrichton: Arch Linux also passes -O1 --sort-common --as-needed -z relro via LDFLAGS when building packages, but doesn't patch the toolchain to force it like Ubuntu.

(I think --hash-style=gnu became the default)

[brson]

I hope if we can figure out how to invoke the linker correctly enough to get through the bots, that will be a great start, and patches for fixing other systems would arrive quickly. We can also leave in a fallback mode for awhile that just uses C for linking.

[brson]

We can also do it incrementally, using C for platforms that haven't been converted.

[alexcrichton]

One thing I recently realized, we can make some temporary progress by passing -nodefaultlibs to gcc. I think that this will still pass through lots of flags to the linker, but we shouldn't be picking up libs like libgcc by default.

[alexcrichton]

An excellent example of the trickery that we'd need to perform is what clang does apparently: https://github.com/llvm-mirror/clang/blob/master/lib/Driver/Tools.cpp

It looks like there's a whole bunch of logic going on in there to find all these extra files. Including libclang would help, but it would indeed be a pretty big dependency to say that rustc is itself built on top of clang.

[vadimcn]

@brson, @alexcrichton: do you think it is OK to target a specific linker (ld probably), or do we want linker driver to be completely scriptable, similar to gcc tool specs?
Though I am afraid that over time, the latter will tend to develop into a Turing-complete language. 😦

[alexcrichton]

I would expect to target the system ld directly. The main impetus for this is to shed dependencies and to get away from a C compiler dependency, it's unclear to me how realistic of a goal it is though (it sure would be nice!)

[steveklabnik]

Triage: we allow you to choose a linker, but still have a gcc dependency. Except with MSVC, which uses link.exe.

[steveklabnik]

Triage: no change. However, see also: #36120

[tamird]

The clang code for doing this has moved slightly: https://github.com/llvm-mirror/clang/blob/release_50/lib/Driver/ToolChains/Darwin.cpp?utf8=%E2%9C%93#L1910

[petrochenkov]

If someone still wants to pursue this (in which I'm not actually sure), then it can be done step-by-step.

The next logical step is move from -nodefaultlibs (already used, makes gcc omit libraries like -lc or -lgcc) to -nostdlib (makes gcc omit CRT objects like crt1.o as well).

The CRT objects used by different toolchains are documented in #71769.
The object files can be shipped with rustc (like it's currently done with musl), or found on the system (that may be harder, but should cause less issues like mingw example shows).

[petrochenkov]

Some negative experience with linking CRT objects manually instead of relying on gcc on musl targets:
#40113 (comment)
#40113 (comment)

So, musl experience repeats the mingw one - using system CRT objects and libraries is strongly preferable to using those shipped with rustc.