Skip to content

Commit

Permalink
rustc: Support auto-retry linking on a segfault
Browse files Browse the repository at this point in the history
This is a last-ditch attempt to help our pain with dealing with rust-lang#38878 on the
bots. A new environment variable is added to the compiler,
`RUSTC_RETRY_LINKER_ON_SEGFAULT`, which will instruct the compiler to
automatically retry the final linker invocation if it looks like the linker
segfaulted (up to 2 extra times).

Unfortunately there have been no successful attempts to debug rust-lang#38878. The only
information seems to be that the linker (e.g. `ld` on OSX) is segfaulting
somewhere in some thread pool implementation. This appears to be spurious as
failed PRs will later merge.

The hope is that this helps the queue keep moving without clogging and delaying
PRs due to rust-lang#38878.
  • Loading branch information
alexcrichton committed Mar 10, 2017
1 parent 5aaa606 commit 993eae1
Show file tree
Hide file tree
Showing 2 changed files with 52 additions and 1 deletion.
5 changes: 5 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ matrix:
RUST_CHECK_TARGET=check
RUST_CONFIGURE_ARGS=--build=x86_64-apple-darwin
SRC=.
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
os: osx
osx_image: xcode8.2
install: &osx_install_sccache >
Expand All @@ -53,6 +54,7 @@ matrix:
RUST_CHECK_TARGET=check
RUST_CONFIGURE_ARGS=--build=i686-apple-darwin
SRC=.
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
os: osx
osx_image: xcode8.2
install: *osx_install_sccache
Expand All @@ -62,6 +64,7 @@ matrix:
RUST_CONFIGURE_ARGS="--build=i686-apple-darwin --enable-extended"
SRC=.
DEPLOY=1
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
os: osx
osx_image: xcode8.2
install: >
Expand All @@ -74,6 +77,7 @@ matrix:
RUST_CONFIGURE_ARGS="--target=aarch64-apple-ios,armv7-apple-ios,armv7s-apple-ios,i386-apple-ios,x86_64-apple-ios --enable-extended"
SRC=.
DEPLOY=1
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
os: osx
osx_image: xcode8.2
install: *osx_install_sccache
Expand All @@ -88,6 +92,7 @@ matrix:
RUST_CONFIGURE_ARGS="--enable-extended"
SRC=.
DEPLOY_ALT=1
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
os: osx
osx_image: xcode8.2
install: *osx_install_sccache
Expand Down
48 changes: 47 additions & 1 deletion src/librustc_trans/back/link.rs
Original file line number Diff line number Diff line change
Expand Up @@ -752,8 +752,54 @@ fn link_natively(sess: &Session,
sess.abort_if_errors();

// Invoke the system linker
//
// Note that there's a terribly awful hack that really shouldn't be present
// in any compiler. Here an environment variable is supported to
// automatically retry the linker invocation if the linker looks like it
// segfaulted.
//
// Gee that seems odd, normally segfaults are things we want to know about!
// Unfortunately though in rust-lang/rust#38878 we're experiencing the
// linker segfaulting on Travis quite a bit which is causing quite a bit of
// pain to land PRs when they spuriously fail due to a segfault.
//
// The issue #38878 has some more debugging information on it as well, but
// this unfortunately looks like it's just a race condition in OSX's linker
// with some thread pool working in the background. It seems that no one
// currently knows a fix for this so in the meantime we're left with this...
info!("{:?}", &cmd);
let prog = time(sess.time_passes(), "running linker", || cmd.output());
let retry_on_segfault = env::var("RUSTC_RETRY_LINKER_ON_SEGFAULT").is_ok();
let mut prog;
let mut i = 0;
loop {
i += 1;
prog = time(sess.time_passes(), "running linker", || cmd.output());
if !retry_on_segfault || i > 3 {
break
}
let output = match prog {
Ok(ref output) => output,
Err(_) => break,
};
if output.status.success() {
break
}
let mut out = output.stderr.clone();
out.extend(&output.stdout);
let out = String::from_utf8_lossy(&out);
let msg = "clang: error: unable to execute command: \
Segmentation fault: 11";
if !out.contains(msg) {
break
}

sess.struct_warn("looks like the linker segfaulted when we tried to \
call it, automatically retrying again")
.note(&format!("{:?}", cmd))
.note(&out)
.emit();
}

match prog {
Ok(prog) => {
fn escape_string(s: &[u8]) -> String {
Expand Down

0 comments on commit 993eae1

Please sign in to comment.