Disable LLVM assertions on Nightly, enable them in "alt" builds. #45810

SimonSapin · 2017-11-06T18:00:50Z

Per IRC discussion https://mozilla.logbot.info/rust-infra/20171106#c13812170-c13812204

Background: https://internals.rust-lang.org/t/disabling-llvm-assertions-in-nightly-builds/5388/14

rust-highfive · 2017-11-06T18:01:05Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @nikomatsakis (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

aturon · 2017-11-06T18:02:27Z

@bors r+

bors · 2017-11-06T18:02:28Z

📌 Commit eac34c9 has been approved by aturon

aturon · 2017-11-06T18:03:10Z

cc @rust-lang/infra, just to keep everyone in the loop on this change.

kennytm · 2017-11-07T02:13:31Z

I think we also need to update .travis.yml to point the try branch to the alt build so crater and perf can still test with assertion enabled.

SimonSapin · 2017-11-07T06:32:43Z

@kennytm I’ve pushed another commit. How does it look?

Though I don’t know if perf should have LLVM assertions, since that produces timings different from what everyone will use.

kennytm · 2017-11-07T06:53:31Z

@SimonSapin Unfortunately, unless we have two try branches we are stuck with either sacrificing perf or crater. At least enabling assertions allows the metrics be comparable before and after this PR.

@bors r=aturon

bors · 2017-11-07T06:53:31Z

📌 Commit dfb08b6 has been approved by aturon

michaelwoerister · 2017-11-07T21:21:44Z

@Mark-Simulacrum @alexcrichton Will we back-collect perf.rlo data after this (e.g. by compiling with the former "alt builds" for older nightlies)?

Mark-Simulacrum · 2017-11-07T23:14:16Z

Since this will require changes to perf.rlo anyway, I should be able to make it so that we properly collect non-alt before and alt now. Ideally, we'd add something like @bors perf which would output dedicated artifacts for perf (and then we wouldn't have to share between crater, which does want assertions, and perf, which doesn't). It's also worth thinking about tracking how "bad" assertions are.

michaelwoerister · 2017-11-08T12:54:18Z

Ideally, we'd add something like @bors perf which would output dedicated artifacts for perf

It would be nice if perf.rlo tested with debug assertions disabled too since that is closer to how stable releases perform. Also, I'm quite liberal with debug assertions :)

nikomatsakis · 2017-11-08T17:21:19Z

@Mark-Simulacrum

It's also worth thinking about tracking how "bad" assertions are.

by this do you mean running with assertions both enable and disabled and comparing the results?

Mark-Simulacrum · 2017-11-08T17:27:48Z

Yes, exactly.

shepmaster · 2017-11-08T21:47:53Z

@SimonSapin if you happen to push another commit, can you fix the typo "Nigthly"? It ~~exists~~ existed in the PR title and in your first commit.

oyvindln · 2017-11-10T21:54:57Z

Is it a good idea to merge this as long as #45220 isn't solved, given that it may result in that ending up accessing out-of-bounds memory rather than panicking?

bors · 2017-11-11T12:13:54Z

⌛ Testing commit dfb08b6 with merge 9a9ee1b...

Disable LLVM assertions on Nightly, enable them in "alt" builds. Per IRC discussion https://mozilla.logbot.info/rust-infra/20171106#c13812170-c13812204 Background: https://internals.rust-lang.org/t/disabling-llvm-assertions-in-nightly-builds/5388/14

bors · 2017-11-11T13:30:11Z

💔 Test failed - status-travis

SimonSapin · 2017-11-11T13:51:30Z

[01:12:34] failures:
[01:12:34] thread 'main' panicked at 'Some tests failed', /checkout/src/tools/compiletest/src/main.rs:331:21
[01:12:34] 
[01:12:34] ---- [run-make] run-make/sanitizer-memory stdout ----
[01:12:34]      
[01:12:34] error: make failed
[01:12:34] status: exit code: 2
[01:12:34] command: "make"
[01:12:34] stdout:
[01:12:34] ------------------------------------------
[01:12:34] LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/sanitizer-memory.stage2-x86_64-unknown-linux-gnu:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib:" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/sanitizer-memory.stage2-x86_64-unknown-linux-gnu -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/sanitizer-memory.stage2-x86_64-unknown-linux-gnu  -g -Z sanitizer=memory -Z print-link-args uninit.rs | grep -q librustc_msan
[01:12:34] /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/sanitizer-memory.stage2-x86_64-unknown-linux-gnu/uninit 2>&1 | grep -q use-of-uninitialized-value
[01:12:34] Makefile:6: recipe for target 'all' failed
[01:12:34] 
[01:12:34] ------------------------------------------
[01:12:34] stderr:
[01:12:34] ------------------------------------------
[01:12:34] warning: unused variable: `y`
[01:12:34]   --> uninit.rs:15:9
[01:12:34]    |
[01:12:34] 15 |     let y = xs[0] + xs[1];
[01:12:34]    |         ^
[01:12:34]    |
[01:12:34]    = note: #[warn(unused_variables)] on by default
[01:12:34]    = note: to avoid this warning, consider using `_y` instead
[01:12:34] 
[01:12:34] make: *** [all] Error 1
[01:12:34] 
[01:12:34] ------------------------------------------
[01:12:34] 
[01:12:34] thread '[run-make] run-make/sanitizer-memory' panicked at 'explicit panic', /checkout/src/tools/compiletest/src/runtest.rs:2505:8
[01:12:34] note: Run with `RUST_BACKTRACE=1` for a backtrace.
[01:12:34] 
[01:12:34] 
[01:12:34] failures:
[01:12:34]     [run-make] run-make/sanitizer-memory
[01:12:34] 
[01:12:34] test result: FAILED. 163 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

I’m trying to reproduce locally.

SimonSapin · 2017-11-11T15:06:28Z

:(

$ ./x.py test src/test/run-make --test-args sanitizer-memory
[…]
running 1 test
test [run-make] run-make/sanitizer-memory ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 163 filtered out

SimonSapin · 2017-11-11T17:43:59Z

Ah, this test is entirely inside ifdef SANITIZER_SUPPORT. When building with --enable-sanitizers I get errors below. The relevant part seems to be error: aggregate ‘sigaltstack handler_stack’ has incomplete type and cannot be defined.

[ 90%] Built target RTAsan_dynamic.x86_64
CMakeFiles/Makefile2:1636: recipe for target 'lib/asan/CMakeFiles/asan.dir/rule' failed
Makefile:632: recipe for target 'asan' failed

--- stderr
/home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/ubsan/ubsan_handlers_cxx.cc: In function ‘void __ubsan::HandleCFIBadType(__ubsan::CFICheckFailData*, __ubsan::ValueHandle, bool, __ubsan::ReportOptions)’:
/home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/ubsan/ubsan_handlers_cxx.cc:111:15: warning: ‘CheckKindStr’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   const char *CheckKindStr;
               ^~~~~~~~~~~~
/home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/sanitizer_common/sanitizer_stoptheworld_linux_libcdep.cc: In function ‘int __sanitizer::TracerThread(void*)’:
/home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/sanitizer_common/sanitizer_stoptheworld_linux_libcdep.cc:278:22: error: aggregate ‘sigaltstack handler_stack’ has incomplete type and cannot be defined
   struct sigaltstack handler_stack;
                      ^~~~~~~~~~~~~
make[3]: *** [lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.x86_64.dir/sanitizer_stoptheworld_linux_libcdep.cc.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.x86_64.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....
/home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/sanitizer_common/sanitizer_posix.cc: In function ‘__sanitizer::fd_t __sanitizer::OpenFile(const char*, __sanitizer::FileAccessMode, __sanitizer::error_t*)’:
/home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/sanitizer_common/sanitizer_posix.cc:215:27: warning: ‘flags’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   fd_t res = internal_open(filename, flags, 0660);
              ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/asan/asan_interceptors.cc:265:0:
/home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc: In function ‘__sanitizer::uptr __interceptor_ptrace(int, int, void*, void*)’:
/home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:2810:21: warning: ‘local_iovec.__sanitizer::__sanitizer_iovec::iov_len’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   __sanitizer_iovec local_iovec;
                     ^~~~~~~~~~~
/home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/asan/asan_interceptors.cc:59:10: warning: ‘local_iovec.__sanitizer::__sanitizer_iovec::iov_base’ may be used uninitialized in this function [-Wmaybe-uninitialized]
     uptr __offset = (uptr)(offset);                                     \
          ^~~~~~~~
In file included from /home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/asan/asan_interceptors.cc:265:0:
/home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:2810:21: note: ‘local_iovec.__sanitizer::__sanitizer_iovec::iov_base’ was declared here
   __sanitizer_iovec local_iovec;
                     ^~~~~~~~~~~
In file included from /home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/asan/asan_interceptors.cc:265:0:
/home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc: In function ‘__sanitizer::uptr __interceptor_ptrace(int, int, void*, void*)’:
/home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:2810:21: warning: ‘local_iovec.__sanitizer::__sanitizer_iovec::iov_len’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   __sanitizer_iovec local_iovec;
                     ^~~~~~~~~~~
/home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/asan/asan_interceptors.cc:59:10: warning: ‘local_iovec.__sanitizer::__sanitizer_iovec::iov_base’ may be used uninitialized in this function [-Wmaybe-uninitialized]
     uptr __offset = (uptr)(offset);                                     \
          ^~~~~~~~
In file included from /home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/asan/asan_interceptors.cc:265:0:
/home/simon/rust/src/libcompiler_builtins/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:2810:21: note: ‘local_iovec.__sanitizer::__sanitizer_iovec::iov_base’ was declared here
   __sanitizer_iovec local_iovec;
                     ^~~~~~~~~~~
make[1]: *** [lib/asan/CMakeFiles/asan.dir/rule] Error 2
make: *** [asan] Error 2
thread 'main' panicked at '
command did not execute successfully, got: exit code: 2

build script failed, must exit now', /home/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.26/src/lib.rs:599:4
note: Run with `RUST_BACKTRACE=1` for a backtrace.

thread 'main' panicked at 'command did not execute successfully: "/home/simon/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "x86_64-unknown-linux-gnu" "-j" "8" "--release" "--features" "panic-unwind jemalloc backtrace profiler" "--manifest-path" "/home/simon/rust/src/libstd/Cargo.toml" "--message-format" "json"
expected success, got: exit code: 101', src/bootstrap/compile.rs:882:8

gnzlbg · 2017-11-27T17:26:40Z

The comment in .travis.yml still says that alt-builds don't have assertions enabled, but it is actually the other way around 😆

kennytm · 2017-11-27T19:22:29Z

@gnzlbg Well PRs welcomed 😅

rust-lang#45810 (comment)

SimonSapin · 2017-11-28T13:27:29Z

#46330

michaelwoerister · 2017-11-28T13:58:18Z

@SimonSapin that's a comment you have to dereference twice in order to get to the actual content :D

gnzlbg · 2017-11-28T15:15:31Z

@michaelwoerister dammit michael! I tried dereferencing it twice and now I am trapped in a loop!

Introduced a new src/etc/cat-and-grep.sh script (called in run-make as $(CGREP)), which prints the input and do a grep simultaneously. This is mainly used to debug spurious failures in run-make, such as the sanitizer error in rust-lang#45810, as well as real errors such as rust-lang#46126.

Replace most call to grep in run-make by a script that cat the input. Introduced a new `src/etc/cat-and-grep.sh` script (called in run-make as `$(CGREP)`), which prints the input and do a grep simultaneously. This is mainly used to debug spurious failures in run-make, such as the spurious error in #45810, as well as real errors such as #46126. (cc #40713) Some `grep` still remains, mainly the `grep -c` calls that count the number of matches and print the result to stdout.

Update comment on alternate builds in .travis.yml rust-lang#45810 (comment)

…:nightly); r=nox With rust-lang/rust#45810, normal Nightly now has LLVM assertions disabled. This allows us to entirely stop relying on private/unstable Rust CI artifacts being and remaining available: https://internals.rust-lang.org/t/public-stable-rust-services/6072 Source-Repo: https://github.com/servo/servo Source-Revision: 856dc3c90ab920880e0bdca171720f37e40dd597 UltraBlame original commit: 6406e73a24bca7b3355ff438c283fe0bd20afdaf

rust-highfive assigned nikomatsakis Nov 6, 2017

kennytm added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Nov 7, 2017

frewsxcv mentioned this pull request Nov 7, 2017

Compiling with -Z sanitizer=address aborts #45220

Closed

This was referenced Nov 7, 2017

Promote "alternative" rustc builds to Nightly archives rust-lang/rust-central-station#24

Closed

Add support for a nightly-alt toolchain rust-lang/rustup#1099

Closed

Making alt build available together with Nightly #45334

Closed

SimonSapin changed the title ~~Disable LLVM assertions on Nigthly, enable them in "alt" builds.~~ Disable LLVM assertions on Nightly, enable them in "alt" builds. Nov 8, 2017

kennytm added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Nov 11, 2017

kennytm mentioned this pull request Nov 27, 2017

Spurious failure in run-make/sanitizer-memory #46313

Closed

SimonSapin added a commit to SimonSapin/rust that referenced this pull request Nov 28, 2017

Update comment on alternate builds in .travis.yml

a6d1895

rust-lang#45810 (comment)

SimonSapin mentioned this pull request Nov 28, 2017

Update comment on alternate builds in .travis.yml #46330

Merged

kennytm added a commit to kennytm/rust that referenced this pull request Nov 29, 2017

Rollup merge of rust-lang#46330 - SimonSapin:patch-6, r=kennytm

1e49d4a

Update comment on alternate builds in .travis.yml rust-lang#45810 (comment)

kennytm mentioned this pull request Dec 7, 2017

Prefer the alt build when possible for commits on or after b7ccb0a5a7ee5bf51dbb63490c0d7166cfb993e2 Mark-Simulacrum/bisect-rust#12

Open

aidanhs mentioned this pull request Dec 8, 2017

Crater should do runs against builds with llvm assertions in (by default) rust-lang/crater#165

Closed

lnicola mentioned this pull request Aug 1, 2020

Move from log to tracing #74726

Merged

lnicola mentioned this pull request Oct 24, 2020

Increasingly slow compilation as more levels of async are added in Rust 1.46 #75992

Open

est31 mentioned this pull request Sep 9, 2022

[crater experiment] Enable LLVM assertions #101591

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Disable LLVM assertions on Nightly, enable them in "alt" builds. #45810

Disable LLVM assertions on Nightly, enable them in "alt" builds. #45810

SimonSapin commented Nov 6, 2017

rust-highfive commented Nov 6, 2017

aturon commented Nov 6, 2017

bors commented Nov 6, 2017

aturon commented Nov 6, 2017

kennytm commented Nov 7, 2017

SimonSapin commented Nov 7, 2017

kennytm commented Nov 7, 2017

bors commented Nov 7, 2017

michaelwoerister commented Nov 7, 2017

Mark-Simulacrum commented Nov 7, 2017

michaelwoerister commented Nov 8, 2017 •

edited

Loading

nikomatsakis commented Nov 8, 2017

Mark-Simulacrum commented Nov 8, 2017

shepmaster commented Nov 8, 2017 •

edited

Loading

oyvindln commented Nov 10, 2017

bors commented Nov 11, 2017

bors commented Nov 11, 2017

SimonSapin commented Nov 11, 2017 •

edited by shepmaster

Loading

SimonSapin commented Nov 11, 2017 •

edited by shepmaster

Loading

SimonSapin commented Nov 11, 2017

gnzlbg commented Nov 27, 2017

kennytm commented Nov 27, 2017

SimonSapin commented Nov 28, 2017

michaelwoerister commented Nov 28, 2017

gnzlbg commented Nov 28, 2017

Disable LLVM assertions on Nightly, enable them in "alt" builds. #45810

Disable LLVM assertions on Nightly, enable them in "alt" builds. #45810

Conversation

SimonSapin commented Nov 6, 2017

rust-highfive commented Nov 6, 2017

aturon commented Nov 6, 2017

bors commented Nov 6, 2017

aturon commented Nov 6, 2017

kennytm commented Nov 7, 2017

SimonSapin commented Nov 7, 2017

kennytm commented Nov 7, 2017

bors commented Nov 7, 2017

michaelwoerister commented Nov 7, 2017

Mark-Simulacrum commented Nov 7, 2017

michaelwoerister commented Nov 8, 2017 • edited Loading

nikomatsakis commented Nov 8, 2017

Mark-Simulacrum commented Nov 8, 2017

shepmaster commented Nov 8, 2017 • edited Loading

oyvindln commented Nov 10, 2017

bors commented Nov 11, 2017

bors commented Nov 11, 2017

SimonSapin commented Nov 11, 2017 • edited by shepmaster Loading

SimonSapin commented Nov 11, 2017 • edited by shepmaster Loading

SimonSapin commented Nov 11, 2017

gnzlbg commented Nov 27, 2017

kennytm commented Nov 27, 2017

SimonSapin commented Nov 28, 2017

michaelwoerister commented Nov 28, 2017

gnzlbg commented Nov 28, 2017

michaelwoerister commented Nov 8, 2017 •

edited

Loading

shepmaster commented Nov 8, 2017 •

edited

Loading

SimonSapin commented Nov 11, 2017 •

edited by shepmaster

Loading

SimonSapin commented Nov 11, 2017 •

edited by shepmaster

Loading