Rustc improvements for C++ linking

[hlopko]

Overview

rustc lacks certain features with regards to linking that prohibit us from
using rustc as the linker driver in rules_rust for a tight integration with a large existing C++ codebase.

Problem description

In general rustc supports 2 ways of configuring linking:

• Source code attribute
  #[link]
• Command line argument
  -l:

In Bazel we use the latter, but feature-wise they are equivalent.
To declare a dependency on a static library libfoo.a, we pass -l:static=foo
to rustc. This will currently link libfoo.a as alwayslink, in a
--whole-archive/--no-whole-archive block. As a result, all symbols from
libfoo.a are seen as used by the linker. While there are some
C++ libraries that have to be linked as alwayslink, the vast majority of libraries
don't want that - it increases the binary size, the link time, and can even
uncover missing symbol bugs not encountered in regular C++ builds.

Even the fact that rustc expects static archives is a problem for certain C++
toolchains. Some toolchains don't typically create static archives, instead they
pass object files directly to the linker in a --start-lib/--end-lib block.
This saves some storage (archives pretty much duplicate the disk used by object
files), and saves io between developer machine and remote cache. This is not
supported by rustc.

Some libraries don't match the lib<name>.a format. For example some projects
use lib<name>.lo for alwayslink libraries. Rustc doesn't support linking to
nonstandard (or verbatim) names using -l: flag currently. There is a way to specify flags
that will be passed verbatim to the linker on the rustc command line -
--codegen=link-arg. However rustc will put these flags at the end of the
linker command line, after it passes crate libraries and Rust standard and core
libraries. That can result in backward references. Imagine a Crate A depends
on C++ Native B, which depends on Crate C. Linker would ideally see
-lcrateA -lnativeB -lcrateC -lrust_stdlib -lc++. But because link-arg flags
are moved at the end, it sees -lcrateA -lrust_stdlib -lnativeB -crateC -lc++.
The backwards reference problem is triggered when Crate C depends on a symbol
from the Rust standard library.

Unstable features/accepted RFCs

static-nobundle

• Tracking issue
  Tracking issue for #[link(kind)] connecting libraries on Windows rust-lang/rust#37403

This unstable feature allows us to disable alwayslink linking behavior by
using the -l:static-nobundle=foo.a flag. It doesn't solve the nonstandard
naming and object group problems. In addition, it seems it's made obsolete by
the native-link-modifiers RFC.

native-link-modifiers

• Tracking issue
  Tracking Issue for linking modifiers for native libraries  rust-lang/rust#81490
• RFC
  https://rust-lang.github.io/rfcs/2951-native-link-modifiers.html

This not yet implemented RFC solves all above mentioned problems.

What to do before RFC 2951 is implemented?

Until RFC 2951 is implemented we don't recommend using rust_binary for mixed C++/Rust binaries.

Current workaround for legacy C++ codebases is to use cc_binary to drive the final
transitive linking.

0. make sure your rust_toolchain declares core and std rlibs so cc_binary knows what to link (TODO: hlopko should upload alloc trampolines that durin42 implemented).
1. don't use rust_binary for the binary crate, use rust_library
2. set crate_root in the rust_library to point to the main.rs
3. add a cc_binary that depends on the rust_library

• in main.rs, replace fn main() {...} with

#[no_mangle]
extern "C" fn main() {...}

4. build

We believe we can implement a Starlark rule/macro that will automate the whole process. We'll attempt that in the near future.

[UebelAndre]

(fantastic write-up btw, very much appreciated)

[dfreese]

I'm not sure it's the same issue, per-say, but the workaround worked. I ran into this where I had been pulling in a local systemd library. The rust_binary compilation would fail without the libsystemd-dev package installed. This package provides the /lib/x86_64-linux-gnu/libsystemd.so library. Without the bare .so, rust_binary would fail, but linking it using the suggested cc_library + no_mangle, worked fine without it present.

This was on a bare ubuntu 20.04 LTS install, so there was already /lib/x86_64-linux-gnu/libsystemd.so.0 and /lib/x86_64-linux-gnu/libsystemd.so.0.28.0. The failure was:

= note: /usr/bin/ld.gold: error: cannot find -lsystemd

WORKSPACE

# Updated 2021-03-11.  Main last updated 2021-03-11.
http_archive(
    name = "rules_rust",
    sha256 = "7449b583d76f971f9a7659c1c2d78e8f459ed93113ed8951ee4b006a657735dc",
    strip_prefix = "rules_rust-2109a0a105baae04148bbe68b9e0f01996ea4e3b",
    urls = [
        "https://github.com/bazelbuild/rules_rust/archive/2109a0a105baae04148bbe68b9e0f01996ea4e3b.tar.gz",
    ],
)

# Updated 2021-02-22.  Master last updated 2021-02-05.
http_archive(
    name = "rules_cc",
    sha256 = "56ac9633c13d74cb71e0546f103ce1c58810e4a76aa8325da593ca4277908d72",
    strip_prefix = "rules_cc-40548a2974f1aea06215272d9c2b47a14a24e556",
    urls = [
        "https://github.com/bazelbuild/rules_cc/archive/40548a2974f1aea06215272d9c2b47a14a24e556.zip",
    ],
)

load("@rules_rust//rust:repositories.bzl", "rust_repository_set")

rust_repository_set(
    name = "rust_linux_x86_64",
    edition = "2018",
    exec_triple = "x86_64-unknown-linux-gnu",
    extra_target_triples = [],
    rustfmt_version = RUSTFMT_VERSION,
    version = RUST_VERSION,
)

new_local_repository(
    name = "systemd",
    build_file = "systemd.BUILD",
    path = "/",
)

systemd.BUILD

load("@rules_cc//cc:defs.bzl", "cc_library")

cc_import(
    name = "systemd",
    shared_library = "lib/x86_64-linux-gnu/libsystemd.so.0",
    visibility = ["//visibility:public"],
)

Working BUILD

load("@rules_rust//rust:defs.bzl", "rust_library")
load("@rules_cc//cc:defs.bzl", "cc_binary")

rust_library(
    name = "check_logging_lib",
    srcs = [
        "check_logging.rs",
    ],
    visibility = ["//:__subpackages__"],
    deps = [
        "//third_party/cargo:libc",
        "@systemd",
    ],
)

cc_binary(
    name = "check_logging",
    deps = [
        ":check_logging_lib",
    ],
)

Working check_logging.rs

extern "C" {
    fn sd_journal_sendv(iv: *const libc::iovec, n: libc::c_int) -> libc::c_int;
}

#[no_mangle]
extern "C" fn main() {
    let message = "MESSAGE=Hello World\n";
    let iovecs = [libc::iovec {
        iov_base: message.as_ptr() as *mut core::ffi::c_void,
        iov_len: message.len() as libc::size_t,
    }];
    unsafe {
        sd_journal_sendv(iovecs.as_ptr(), iovecs.len() as libc::c_int);
    }
}

Failing BUILD

load("@rules_rust//rust:defs.bzl", "rust_binary")
load("@rules_cc//cc:defs.bzl", "cc_binary")

rust_binary(
    name = "check_logging",
    srcs = [
        "check_logging.rs",
    ],
    visibility = ["//:__subpackages__"],
    deps = [
        "//third_party/cargo:libc",
        "@systemd",
    ],
)

Failing check_logging.rs

extern "C" {
    fn sd_journal_sendv(iv: *const libc::iovec, n: libc::c_int) -> libc::c_int;
}

fn main() {
    let message = "MESSAGE=Hello World\n";
    let iovecs = [libc::iovec {
        iov_base: message.as_ptr() as *mut core::ffi::c_void,
        iov_len: message.len() as libc::size_t,
    }];
    unsafe {
        sd_journal_sendv(iovecs.as_ptr(), iovecs.len() as libc::c_int);
    }
}