Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When a codegen worker has a FatalError, propagate it instead of ICE'ing. #67458

Conversation

pnkfelix
Copy link
Member

Fix #66530

@rust-highfive
Copy link
Collaborator

r? @matthewjasper

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Dec 20, 2019
@pnkfelix
Copy link
Member Author

pnkfelix commented Dec 20, 2019

(I'm a little worried about the test being fragile, in terms of the specifics of its output. I tried making this a compile-fail test instead of a ui test, but when I did that, it seemed like ICE'ing is considered an acceptable outcome for a compile-fail test, as long as the error-pattern matches...

... (which sounds like a bug, if you ask me. We may want to look into changing that, to avoid having hidden ICE's in the compile-fail test suite.)

I guess I could/should look into trying to do some custom normalization of the test output to side-step the fragility regarding the details of the error output.

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-12-20T18:09:20.5572037Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-12-20T18:09:20.5777544Z ##[command]git config gc.auto 0
2019-12-20T18:09:20.5829027Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-12-20T18:09:20.5881126Z ##[command]git config --get-all http.proxy
2019-12-20T18:09:20.6020199Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/67458/merge:refs/remotes/pull/67458/merge
---
2019-12-20T19:03:12.4409734Z .................................................................................................... 1600/9382
2019-12-20T19:03:16.4670125Z .................................................................................................... 1700/9382
2019-12-20T19:03:27.1934798Z ....................................................................i............................... 1800/9382
2019-12-20T19:03:33.6182767Z .................................................................................................... 1900/9382
2019-12-20T19:03:47.3336799Z .....................................................iiiii.......................................... 2000/9382
2019-12-20T19:03:56.4835268Z .................................................................................................... 2200/9382
2019-12-20T19:03:58.5935961Z .................................................................................................... 2300/9382
2019-12-20T19:04:01.3957484Z .................................................................................................... 2400/9382
2019-12-20T19:04:21.4050634Z .................................................................................................... 2500/9382
---
2019-12-20T19:06:42.7613580Z ..............................................................i...............i..................... 4800/9382
2019-12-20T19:06:49.4614888Z .................................................................................................... 4900/9382
2019-12-20T19:06:57.0813381Z .................................................................................................... 5000/9382
2019-12-20T19:07:01.9634089Z ......i............................................................................................. 5100/9382
2019-12-20T19:07:11.4801280Z ........................................................................ii.ii...........i........... 5200/9382
2019-12-20T19:07:19.4564608Z ........i........................................................................................... 5400/9382
2019-12-20T19:07:28.6312136Z .................................................................................................... 5500/9382
2019-12-20T19:07:34.5887848Z ......................................................i............................................. 5600/9382
2019-12-20T19:07:41.0549770Z .................................................................................................... 5700/9382
2019-12-20T19:07:41.0549770Z .................................................................................................... 5700/9382
2019-12-20T19:07:50.2955584Z .................................................................................................... 5800/9382
2019-12-20T19:07:56.6602289Z ..........................................ii...i..ii...........i.................................... 5900/9382
2019-12-20T19:08:17.0884313Z .................................................................................................... 6100/9382
2019-12-20T19:08:24.5525655Z .................................................................................................... 6200/9382
2019-12-20T19:08:24.5525655Z .................................................................................................... 6200/9382
2019-12-20T19:08:30.9741455Z .................................................................F..i..ii........................... 6300/9382
2019-12-20T19:08:57.4027719Z .................................................................................................... 6500/9382
2019-12-20T19:08:59.2652701Z ........................................i........................................................... 6600/9382
2019-12-20T19:09:01.3239197Z .................................................................................................... 6700/9382
2019-12-20T19:09:03.5248523Z ................................i................................................................... 6800/9382
---
2019-12-20T19:10:34.2801266Z .................................................................................................... 7400/9382
2019-12-20T19:10:38.5169224Z .................................................................................................... 7500/9382
2019-12-20T19:10:43.2516210Z .................................................................................................... 7600/9382
2019-12-20T19:10:52.2067741Z .................................................................................................... 7700/9382
2019-12-20T19:11:00.6222916Z .....................................................iiii........................................... 7800/9382
2019-12-20T19:11:14.3081860Z .................................................................................................... 8000/9382
2019-12-20T19:11:19.8842964Z .................................................................................................... 8100/9382
2019-12-20T19:11:35.0848479Z .................................................................................................... 8200/9382
2019-12-20T19:11:42.3666812Z .................................................................................................... 8300/9382
---
2019-12-20T19:13:33.9103595Z 
2019-12-20T19:13:33.9104442Z ---- [ui] ui/non-ice-error-on-worker-io-fail.rs stdout ----
2019-12-20T19:13:33.9105047Z diff of stderr:
2019-12-20T19:13:33.9105610Z 
2019-12-20T19:13:33.9106338Z 1 warning: ignoring --out-dir flag due to -o flag
2019-12-20T19:13:33.9106490Z 2 
2019-12-20T19:13:33.9106869Z - error: failed to write bytecode to /dev/null.non_ice_error_on_worker_io_fail.3a1fbbbh-cgu.0.rcgu.bc.z: Operation not permitted (os error 1)
2019-12-20T19:13:33.9107248Z + error: failed to write bytecode to /dev/null.non_ice_error_on_worker_io_fail.3a1fbbbh-cgu.0.rcgu.bc.z: Permission denied (os error 13)
2019-12-20T19:13:33.9107535Z 5 error: aborting due to previous error
2019-12-20T19:13:33.9107640Z 6 
2019-12-20T19:13:33.9107748Z 
2019-12-20T19:13:33.9107841Z 
2019-12-20T19:13:33.9107841Z 
2019-12-20T19:13:33.9107950Z The actual stderr differed from the expected stderr.
2019-12-20T19:13:33.9108297Z Actual stderr saved to /checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/non-ice-error-on-worker-io-fail/non-ice-error-on-worker-io-fail.stderr
2019-12-20T19:13:33.9108811Z To update references, rerun the tests and pass the `--bless` flag
2019-12-20T19:13:33.9109370Z To only update this specific test, also pass `--test-args non-ice-error-on-worker-io-fail.rs`
2019-12-20T19:13:33.9109650Z error: 1 errors occurred comparing output.
2019-12-20T19:13:33.9109768Z status: exit code: 1
2019-12-20T19:13:33.9109768Z status: exit code: 1
2019-12-20T19:13:33.9110604Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/non-ice-error-on-worker-io-fail.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/non-ice-error-on-worker-io-fail" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-o" "/dev/null" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/non-ice-error-on-worker-io-fail/auxiliary" "-A" "unused"
2019-12-20T19:13:33.9111281Z ------------------------------------------
2019-12-20T19:13:33.9111956Z 
2019-12-20T19:13:33.9112379Z ------------------------------------------
2019-12-20T19:13:33.9112586Z stderr:
2019-12-20T19:13:33.9112586Z stderr:
2019-12-20T19:13:33.9112931Z ------------------------------------------
2019-12-20T19:13:33.9113332Z warning: ignoring --out-dir flag due to -o flag
2019-12-20T19:13:33.9113491Z 
2019-12-20T19:13:33.9113936Z error: failed to write bytecode to /dev/null.non_ice_error_on_worker_io_fail.3a1fbbbh-cgu.0.rcgu.bc.z: Permission denied (os error 13)
2019-12-20T19:13:33.9114289Z error: aborting due to previous error
2019-12-20T19:13:33.9114416Z 
2019-12-20T19:13:33.9114541Z 
2019-12-20T19:13:33.9114914Z ------------------------------------------
---
2019-12-20T19:13:33.9133772Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:537:22
2019-12-20T19:13:33.9134111Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2019-12-20T19:13:33.9151408Z 
2019-12-20T19:13:33.9152188Z 
2019-12-20T19:13:33.9154437Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-7/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "7.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2019-12-20T19:13:33.9155622Z 
2019-12-20T19:13:33.9155733Z 
2019-12-20T19:13:33.9161285Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2019-12-20T19:13:33.9162265Z Build completed unsuccessfully in 0:58:19
2019-12-20T19:13:33.9162265Z Build completed unsuccessfully in 0:58:19
2019-12-20T19:13:33.9215746Z == clock drift check ==
2019-12-20T19:13:33.9231300Z   local time: Fri Dec 20 19:13:33 UTC 2019
2019-12-20T19:13:34.0170566Z   network time: Fri, 20 Dec 2019 19:13:34 GMT
2019-12-20T19:13:34.0170869Z == end clock drift check ==
2019-12-20T19:13:35.0367640Z 
2019-12-20T19:13:35.0469572Z ##[error]Bash exited with code '1'.
2019-12-20T19:13:35.0508560Z ##[section]Starting: Checkout
2019-12-20T19:13:35.0510157Z ==============================================================================
2019-12-20T19:13:35.0510221Z Task         : Get sources
2019-12-20T19:13:35.0510259Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@matthewjasper
Copy link
Contributor

I guess I could/should look into trying to do some custom normalization of the test output to side-step the fragility regarding the details of the error output.

You could also use // dont-check-compiler-stderr

@pnkfelix
Copy link
Member Author

You could also use // dont-check-compiler-stderr

No, that is too broad (I explicitly double-checked just now).

Specifically, it will miss the ICE itself.

(Note that in this particular example, we are always getting the desired diagnostic from the worker thread. The only problem is that the coordinating thread is ICE'ing, because that code was written assuming that an error from the worker must correspond to a panic.)

But I think // normalize-stderr-test will serve my purposes just fine.

@pnkfelix pnkfelix force-pushed the fix-66530-by-propagating-fatal-error-from-worker branch from 9f0210e to eda1411 Compare December 21, 2019 02:15
@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-12-21T02:15:44.9119622Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-12-21T02:15:44.9215621Z ##[command]git config gc.auto 0
2019-12-21T02:15:44.9305873Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-12-21T02:15:44.9359910Z ##[command]git config --get-all http.proxy
2019-12-21T02:15:44.9514213Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/67458/merge:refs/remotes/pull/67458/merge
---
2019-12-21T02:21:38.7568900Z    Compiling serde_json v1.0.40
2019-12-21T02:21:40.3557294Z    Compiling tidy v0.1.0 (/checkout/src/tools/tidy)
2019-12-21T02:21:50.8567525Z     Finished release [optimized] target(s) in 1m 23s
2019-12-21T02:21:50.8661813Z tidy check
2019-12-21T02:21:51.3613210Z tidy error: /checkout/src/test/ui/non-ice-error-on-worker-io-fail.rs:10: line longer than 100 chars
2019-12-21T02:21:53.5238054Z some tidy checks failed
2019-12-21T02:21:53.5241230Z Found 485 error codes
2019-12-21T02:21:53.5241541Z Found 0 error codes with no tests
2019-12-21T02:21:53.5241602Z Done!
2019-12-21T02:21:53.5241602Z Done!
2019-12-21T02:21:53.5241657Z 
2019-12-21T02:21:53.5241788Z 
2019-12-21T02:21:53.5242699Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor"
2019-12-21T02:21:53.5242831Z 
2019-12-21T02:21:53.5242856Z 
2019-12-21T02:21:53.5242905Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
2019-12-21T02:21:53.5242972Z Build completed unsuccessfully in 0:01:27
2019-12-21T02:21:53.5242972Z Build completed unsuccessfully in 0:01:27
2019-12-21T02:21:53.5287376Z == clock drift check ==
2019-12-21T02:21:53.5297480Z   local time: Sat Dec 21 02:21:53 UTC 2019
2019-12-21T02:21:53.6047137Z   network time: Sat, 21 Dec 2019 02:21:53 GMT
2019-12-21T02:21:53.6047240Z == end clock drift check ==
2019-12-21T02:21:54.8993562Z 
2019-12-21T02:21:54.9094409Z ##[error]Bash exited with code '1'.
2019-12-21T02:21:54.9122276Z ##[section]Starting: Checkout
2019-12-21T02:21:54.9123979Z ==============================================================================
2019-12-21T02:21:54.9124034Z Task         : Get sources
2019-12-21T02:21:54.9124085Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors
Copy link
Contributor

bors commented Dec 23, 2019

☔ The latest upstream changes (presumably #67540) made this pull request unmergeable. Please resolve the merge conflicts.

@pnkfelix pnkfelix force-pushed the fix-66530-by-propagating-fatal-error-from-worker branch from 1f33ed9 to 77803af Compare December 23, 2019 17:47
@rust-highfive
Copy link
Collaborator

The job mingw-check of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-12-23T18:17:25.5989238Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-12-23T18:17:25.6215213Z ##[command]git config gc.auto 0
2019-12-23T18:17:26.6255204Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-12-23T18:17:26.6258304Z ##[command]git config --get-all http.proxy
2019-12-23T18:17:26.6261231Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/67458/merge:refs/remotes/pull/67458/merge
---
2019-12-23T18:26:15.6413806Z     Checking rustc_plugin_impl v0.0.0 (/checkout/src/librustc_plugin_impl)
2019-12-23T18:26:15.8289819Z     Checking rustc_privacy v0.0.0 (/checkout/src/librustc_privacy)
2019-12-23T18:26:16.3169875Z     Checking rustc_codegen_ssa v0.0.0 (/checkout/src/librustc_codegen_ssa)
2019-12-23T18:26:17.6531425Z error[E0308]: mismatched types
2019-12-23T18:26:17.6533220Z     --> src/librustc_codegen_ssa/back/write.rs:1566:62
2019-12-23T18:26:17.6534019Z      |
2019-12-23T18:26:17.6534486Z 1566 |                     None => Message::Done::<B> { result: Err(()), worker_id },
2019-12-23T18:26:17.6534988Z      |                                                              ^^ expected enum `std::option::Option`, found `()`
2019-12-23T18:26:17.6535406Z      |
2019-12-23T18:26:17.6535844Z      = note:   expected enum `std::option::Option<back::write::WorkerFatalError>`
2019-12-23T18:26:17.6536429Z 
2019-12-23T18:26:18.7797464Z error: aborting due to previous error
2019-12-23T18:26:18.7797735Z 
2019-12-23T18:26:18.7798365Z For more information about this error, try `rustc --explain E0308`.
2019-12-23T18:26:18.7798365Z For more information about this error, try `rustc --explain E0308`.
2019-12-23T18:26:18.7798636Z error: could not compile `rustc_codegen_ssa`.
2019-12-23T18:26:18.7798902Z warning: build failed, waiting for other jobs to finish...
2019-12-23T18:26:20.5943322Z error: build failed
2019-12-23T18:26:20.5966999Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "check" "-Zconfig-profile" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "2" "--release" "--color" "always" "--features" " llvm" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "--message-format" "json-render-diagnostics"
2019-12-23T18:26:20.5979596Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap check
2019-12-23T18:26:20.5979853Z Build completed unsuccessfully in 0:05:40
2019-12-23T18:26:20.6033147Z == clock drift check ==
2019-12-23T18:26:20.6049524Z   local time: Mon Dec 23 18:26:20 UTC 2019
2019-12-23T18:26:20.6049524Z   local time: Mon Dec 23 18:26:20 UTC 2019
2019-12-23T18:26:21.7810364Z   network time: Mon, 23 Dec 2019 18:26:20 GMT
2019-12-23T18:26:21.7810891Z == end clock drift check ==
2019-12-23T18:26:21.9723612Z 
2019-12-23T18:26:21.9824996Z ##[error]Bash exited with code '1'.
2019-12-23T18:26:21.9859653Z ##[section]Starting: Checkout
2019-12-23T18:26:21.9861485Z ==============================================================================
2019-12-23T18:26:21.9861534Z Task         : Get sources
2019-12-23T18:26:21.9861593Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@pnkfelix pnkfelix force-pushed the fix-66530-by-propagating-fatal-error-from-worker branch 2 times, most recently from 3ec4463 to 18750e0 Compare December 26, 2019 22:43
@matthewjasper
Copy link
Contributor

@bors r+

@bors
Copy link
Contributor

bors commented Dec 27, 2019

📌 Commit 18750e0 has been approved by matthewjasper

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 27, 2019
@bors
Copy link
Contributor

bors commented Dec 28, 2019

⌛ Testing commit 18750e0 with merge 7d17d79335ee63ad9d7f8010ccbb23e88b88e396...

@rust-highfive
Copy link
Collaborator

The job wasm32 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-12-28T05:46:07.9896014Z ---- [ui] ui/non-ice-error-on-worker-io-fail.rs stdout ----
2019-12-28T05:46:07.9896097Z 
2019-12-28T05:46:07.9896160Z error: ui test compiled successfully!
2019-12-28T05:46:07.9896238Z status: exit code: 0
2019-12-28T05:46:07.9897165Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/non-ice-error-on-worker-io-fail.rs" "-Zthreads=1" "--target=wasm32-unknown-emscripten" "--error-format" "json" "-Zui-testing" "--emit" "metadata" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/non-ice-error-on-worker-io-fail" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/wasm32-unknown-emscripten/native/rust-test-helpers" "-o" "/dev/null" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/non-ice-error-on-worker-io-fail/auxiliary" "-A" "unused"
2019-12-28T05:46:07.9897689Z ------------------------------------------
2019-12-28T05:46:07.9897735Z 
2019-12-28T05:46:07.9897967Z ------------------------------------------
2019-12-28T05:46:07.9898031Z stderr:
2019-12-28T05:46:07.9898031Z stderr:
2019-12-28T05:46:07.9898254Z ------------------------------------------
2019-12-28T05:46:07.9898474Z warning: ignoring --out-dir flag due to -o flag
2019-12-28T05:46:07.9898564Z 
2019-12-28T05:46:07.9898786Z ------------------------------------------
2019-12-28T05:46:07.9898827Z 
2019-12-28T05:46:07.9898857Z 
---
2019-12-28T05:46:07.9932379Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:385:22
2019-12-28T05:46:07.9932692Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2019-12-28T05:46:07.9952306Z 
2019-12-28T05:46:07.9953923Z 
2019-12-28T05:46:07.9957396Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/wasm32-unknown-emscripten/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-wasm32-unknown-emscripten" "--mode" "ui" "--target" "wasm32-unknown-emscripten" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/checkout/obj/build/x86_64-unknown-linux-gnu/llvm/build/bin/FileCheck" "--nodejs" "/emsdk-portable/node/12.9.1_64bit/bin/node" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/wasm32-unknown-emscripten/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--llvm-version" "9.0.0-rust-1.42.0-nightly\n" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2019-12-28T05:46:07.9961958Z 
2019-12-28T05:46:07.9962071Z 
2019-12-28T05:46:07.9968942Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test --target wasm32-unknown-emscripten --exclude src/libcore --exclude src/liballoc --exclude src/libproc_macro --exclude src/libstd --exclude src/libterm --exclude src/libtest
2019-12-28T05:46:07.9969126Z Build completed unsuccessfully in 2:02:00
2019-12-28T05:46:07.9969126Z Build completed unsuccessfully in 2:02:00
2019-12-28T05:46:08.0044504Z == clock drift check ==
2019-12-28T05:46:08.0058257Z   local time: Sat Dec 28 05:46:08 UTC 2019
2019-12-28T05:46:08.1551779Z   network time: Sat, 28 Dec 2019 05:46:08 GMT
2019-12-28T05:46:08.1552713Z == end clock drift check ==
2019-12-28T05:46:08.8208994Z 
2019-12-28T05:46:08.8305261Z ##[error]Bash exited with code '1'.
2019-12-28T05:46:08.8357854Z ##[section]Starting: Checkout
2019-12-28T05:46:08.8359719Z ==============================================================================
2019-12-28T05:46:08.8359820Z Task         : Get sources
2019-12-28T05:46:08.8359907Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors
Copy link
Contributor

bors commented Dec 28, 2019

💔 Test failed - checks-azure

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Dec 28, 2019
@bors
Copy link
Contributor

bors commented Dec 29, 2019

⌛ Testing commit 18750e0 with merge bee1bb4...

bors added a commit that referenced this pull request Dec 29, 2019
…from-worker, r=matthewjasper

When a codegen worker has a FatalError, propagate it instead of ICE'ing.

Fix #66530
@bors
Copy link
Contributor

bors commented Dec 29, 2019

💔 Test failed - checks-azure

@rust-highfive
Copy link
Collaborator

The job wasm32 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-12-29T09:31:28.8102806Z ---- [ui] ui/non-ice-error-on-worker-io-fail.rs stdout ----
2019-12-29T09:31:28.8102958Z 
2019-12-29T09:31:28.8103085Z error: ui test compiled successfully!
2019-12-29T09:31:28.8103246Z status: exit code: 0
2019-12-29T09:31:28.8105218Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/non-ice-error-on-worker-io-fail.rs" "-Zthreads=1" "--target=wasm32-unknown-emscripten" "--error-format" "json" "-Zui-testing" "--emit" "metadata" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/non-ice-error-on-worker-io-fail" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/wasm32-unknown-emscripten/native/rust-test-helpers" "-o" "/dev/null" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/non-ice-error-on-worker-io-fail/auxiliary" "-A" "unused"
2019-12-29T09:31:28.8106233Z ------------------------------------------
2019-12-29T09:31:28.8106346Z 
2019-12-29T09:31:28.8107043Z ------------------------------------------
2019-12-29T09:31:28.8107208Z stderr:
2019-12-29T09:31:28.8107208Z stderr:
2019-12-29T09:31:28.8107650Z ------------------------------------------
2019-12-29T09:31:28.8108130Z warning: ignoring --out-dir flag due to -o flag
2019-12-29T09:31:28.8108312Z 
2019-12-29T09:31:28.8108754Z ------------------------------------------
2019-12-29T09:31:28.8108842Z 
2019-12-29T09:31:28.8108907Z 
---
2019-12-29T09:31:28.8140485Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:385:22
2019-12-29T09:31:28.8140619Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2019-12-29T09:31:28.8163475Z 
2019-12-29T09:31:28.8163591Z 
2019-12-29T09:31:28.8165774Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/wasm32-unknown-emscripten/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-wasm32-unknown-emscripten" "--mode" "ui" "--target" "wasm32-unknown-emscripten" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/checkout/obj/build/x86_64-unknown-linux-gnu/llvm/build/bin/FileCheck" "--nodejs" "/emsdk-portable/node/12.9.1_64bit/bin/node" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/wasm32-unknown-emscripten/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--llvm-version" "9.0.0-rust-1.42.0-nightly\n" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2019-12-29T09:31:28.8166599Z 
2019-12-29T09:31:28.8166637Z 
2019-12-29T09:31:28.8169635Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test --target wasm32-unknown-emscripten --exclude src/libcore --exclude src/liballoc --exclude src/libproc_macro --exclude src/libstd --exclude src/libterm --exclude src/libtest
2019-12-29T09:31:28.8169786Z Build completed unsuccessfully in 1:48:34
2019-12-29T09:31:28.8169786Z Build completed unsuccessfully in 1:48:34
2019-12-29T09:31:28.8245150Z == clock drift check ==
2019-12-29T09:31:28.8255638Z   local time: Sun Dec 29 09:31:28 UTC 2019
2019-12-29T09:31:29.1188125Z   network time: Sun, 29 Dec 2019 09:31:29 GMT
2019-12-29T09:31:29.1189056Z == end clock drift check ==
2019-12-29T09:31:30.0395448Z 
2019-12-29T09:31:30.0454342Z ##[error]Bash exited with code '1'.
2019-12-29T09:31:30.0491695Z ##[section]Starting: Checkout
2019-12-29T09:31:30.0493581Z ==============================================================================
2019-12-29T09:31:30.0493667Z Task         : Get sources
2019-12-29T09:31:30.0493761Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@pnkfelix
Copy link
Member Author

pnkfelix commented Dec 30, 2019

sigh, gotta ignore wasm and emscripten for some reason ...

@pnkfelix pnkfelix force-pushed the fix-66530-by-propagating-fatal-error-from-worker branch from 18750e0 to c8a09fe Compare December 30, 2019 17:38
@rust-highfive
Copy link
Collaborator

The job mingw-check of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-12-30T17:39:10.8589086Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-12-30T17:39:11.8403410Z ##[command]git config gc.auto 0
2019-12-30T17:39:11.8407674Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-12-30T17:39:11.8411778Z ##[command]git config --get-all http.proxy
2019-12-30T17:39:11.8414442Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/67458/merge:refs/remotes/pull/67458/merge
---
2019-12-30T17:48:21.9889471Z     Checking rustc_plugin_impl v0.0.0 (/checkout/src/librustc_plugin_impl)
2019-12-30T17:48:22.1818790Z     Checking rustc_privacy v0.0.0 (/checkout/src/librustc_privacy)
2019-12-30T17:48:22.6630523Z     Checking rustc_codegen_ssa v0.0.0 (/checkout/src/librustc_codegen_ssa)
2019-12-30T17:48:24.0581937Z error[E0308]: mismatched types
2019-12-30T17:48:24.0582330Z     --> src/librustc_codegen_ssa/back/write.rs:1566:62
2019-12-30T17:48:24.0582603Z      |
2019-12-30T17:48:24.0582945Z 1566 |                     None => Message::Done::<B> { result: Err(()), worker_id },
2019-12-30T17:48:24.0583570Z      |                                                              ^^ expected enum `std::option::Option`, found `()`
2019-12-30T17:48:24.0583793Z      |
2019-12-30T17:48:24.0584084Z      = note:   expected enum `std::option::Option<back::write::WorkerFatalError>`
2019-12-30T17:48:24.0584374Z 
2019-12-30T17:48:25.1687642Z error: aborting due to previous error
2019-12-30T17:48:25.1687808Z 
2019-12-30T17:48:25.1696282Z For more information about this error, try `rustc --explain E0308`.
2019-12-30T17:48:25.1696282Z For more information about this error, try `rustc --explain E0308`.
2019-12-30T17:48:25.1775373Z error: could not compile `rustc_codegen_ssa`.
2019-12-30T17:48:25.1790661Z warning: build failed, waiting for other jobs to finish...
2019-12-30T17:48:29.5072170Z error: build failed
2019-12-30T17:48:29.5098063Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "check" "-Zconfig-profile" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "2" "--release" "--color" "always" "--features" " llvm" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "--message-format" "json-render-diagnostics"
2019-12-30T17:48:29.5118910Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap check
2019-12-30T17:48:29.5119567Z Build completed unsuccessfully in 0:05:49
2019-12-30T17:48:29.5182336Z == clock drift check ==
2019-12-30T17:48:29.5194380Z   local time: Mon Dec 30 17:48:29 UTC 2019
2019-12-30T17:48:29.5194380Z   local time: Mon Dec 30 17:48:29 UTC 2019
2019-12-30T17:48:29.8068786Z   network time: Mon, 30 Dec 2019 17:48:29 GMT
2019-12-30T17:48:29.8078195Z == end clock drift check ==
2019-12-30T17:48:30.8787592Z 
2019-12-30T17:48:30.8894981Z ##[error]Bash exited with code '1'.
2019-12-30T17:48:30.8928475Z ##[section]Starting: Checkout
2019-12-30T17:48:30.8930587Z ==============================================================================
2019-12-30T17:48:30.8930662Z Task         : Get sources
2019-12-30T17:48:30.8930713Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@pnkfelix
Copy link
Member Author

what ...? But I thought I fetched ... hmm. Well, okay, that teaches me to force-push without reviewing more carefully.

@pnkfelix pnkfelix force-pushed the fix-66530-by-propagating-fatal-error-from-worker branch from c8a09fe to e60a406 Compare December 30, 2019 18:40
@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-12-30T18:41:13.7202770Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-12-30T18:41:14.4717377Z ##[command]git config gc.auto 0
2019-12-30T18:41:14.4726257Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-12-30T18:41:14.4733338Z ##[command]git config --get-all http.proxy
2019-12-30T18:41:14.4739899Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/67458/merge:refs/remotes/pull/67458/merge
---
2019-12-30T19:36:51.1277238Z .................................................................................................... 1500/9465
2019-12-30T19:36:56.4409758Z .................................................................................................... 1600/9465
2019-12-30T19:37:00.8409410Z .................................................................................................... 1700/9465
2019-12-30T19:37:09.4740349Z .................................................................................................... 1800/9465
2019-12-30T19:37:16.8278489Z i................................................................................................... 1900/9465
2019-12-30T19:37:22.8705276Z ......................................................................................iiiii......... 2000/9465
2019-12-30T19:37:42.9398707Z .................................................................................................... 2200/9465
2019-12-30T19:37:45.1066768Z .................................................................................................... 2300/9465
2019-12-30T19:37:47.3829634Z .................................................................................................... 2400/9465
2019-12-30T19:37:52.9833653Z .................................................................................................... 2500/9465
---
2019-12-30T19:40:35.3710409Z .................i...............i.................................................................. 4900/9465
2019-12-30T19:40:44.3565002Z .................................................................................................... 5000/9465
2019-12-30T19:40:49.5447383Z ..............................................................i..................................... 5100/9465
2019-12-30T19:40:56.9743616Z .................................................................................................... 5200/9465
2019-12-30T19:41:03.8395038Z .............................ii.ii...........i...................................................... 5300/9465
2019-12-30T19:41:12.4145627Z .................................................................................................... 5500/9465
2019-12-30T19:41:21.5880599Z .................................................................................................... 5600/9465
2019-12-30T19:41:27.9186306Z ............i....................................................................................... 5700/9465
2019-12-30T19:41:33.6254977Z .................................................................................................... 5800/9465
2019-12-30T19:41:33.6254977Z .................................................................................................... 5800/9465
2019-12-30T19:41:43.5350699Z .................................................................................................... 5900/9465
2019-12-30T19:41:54.3576774Z ii...i..ii...........i.............................................................................. 6000/9465
2019-12-30T19:42:10.4476743Z .................................................................................................... 6200/9465
2019-12-30T19:42:16.9306782Z .................................................................................................... 6300/9465
2019-12-30T19:42:16.9306782Z .................................................................................................... 6300/9465
2019-12-30T19:42:32.6324107Z .........................F..i..ii................................................................... 6400/9465
2019-12-30T19:42:50.8467234Z .................................................................................................... 6600/9465
2019-12-30T19:42:52.7768034Z ...i................................................................................................ 6700/9465
2019-12-30T19:42:54.9069994Z .................................................................................................... 6800/9465
2019-12-30T19:42:57.2096163Z ...i................................................................................................ 6900/9465
---
2019-12-30T19:44:26.2720660Z .................................................................................................... 7500/9465
2019-12-30T19:44:30.7230013Z .................................................................................................... 7600/9465
2019-12-30T19:44:35.6970416Z .................................................................................................... 7700/9465
2019-12-30T19:44:44.8067887Z .................................................................................................... 7800/9465
2019-12-30T19:44:51.7072106Z ..................................iiii.............................................................. 7900/9465
2019-12-30T19:45:05.0532283Z .................................................................................................... 8100/9465
2019-12-30T19:45:12.9119358Z .................................................................................................... 8200/9465
2019-12-30T19:45:25.9653512Z .................................................................................................... 8300/9465
2019-12-30T19:45:32.9689648Z .................................................................................................... 8400/9465
---
2019-12-30T19:47:17.4042513Z 
2019-12-30T19:47:17.4043252Z ---- [ui] ui/non-ice-error-on-worker-io-fail.rs stdout ----
2019-12-30T19:47:17.4043377Z diff of stderr:
2019-12-30T19:47:17.4043408Z 
2019-12-30T19:47:17.4043672Z 1 warning: ignoring --out-dir flag due to -o flag
2019-12-30T19:47:17.4043915Z - error: failed to write bytecode to FILE
2019-12-30T19:47:17.4043915Z - error: failed to write bytecode to FILE
2019-12-30T19:47:17.4044335Z + error: couldn't create a temp dir: Permission denied (os error 13) at path "/dev/rmetaiExk9x"
2019-12-30T19:47:17.4044463Z 5 error: aborting due to previous error
2019-12-30T19:47:17.4044498Z 6 
2019-12-30T19:47:17.4044537Z 
2019-12-30T19:47:17.4044560Z 
2019-12-30T19:47:17.4044560Z 
2019-12-30T19:47:17.4044599Z The actual stderr differed from the expected stderr.
2019-12-30T19:47:17.4044934Z Actual stderr saved to /checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/non-ice-error-on-worker-io-fail/non-ice-error-on-worker-io-fail.stderr
2019-12-30T19:47:17.4048014Z To update references, rerun the tests and pass the `--bless` flag
2019-12-30T19:47:17.4048391Z To only update this specific test, also pass `--test-args non-ice-error-on-worker-io-fail.rs`
2019-12-30T19:47:17.4048507Z error: 1 errors occurred comparing output.
2019-12-30T19:47:17.4048554Z status: exit code: 1
2019-12-30T19:47:17.4048554Z status: exit code: 1
2019-12-30T19:47:17.4049512Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/non-ice-error-on-worker-io-fail.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "--emit" "metadata" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/non-ice-error-on-worker-io-fail" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-o" "/dev/null" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/non-ice-error-on-worker-io-fail/auxiliary" "-A" "unused"
2019-12-30T19:47:17.4049966Z ------------------------------------------
2019-12-30T19:47:17.4049999Z 
2019-12-30T19:47:17.4050206Z ------------------------------------------
2019-12-30T19:47:17.4050250Z stderr:
2019-12-30T19:47:17.4050250Z stderr:
2019-12-30T19:47:17.4050466Z ------------------------------------------
2019-12-30T19:47:17.4050676Z warning: ignoring --out-dir flag due to -o flag
2019-12-30T19:47:17.4050708Z 
2019-12-30T19:47:17.4050962Z error: couldn't create a temp dir: Permission denied (os error 13) at path "/dev/rmetaiExk9x"
2019-12-30T19:47:17.4051056Z error: aborting due to previous error
2019-12-30T19:47:17.4051084Z 
2019-12-30T19:47:17.4051107Z 
2019-12-30T19:47:17.4051329Z ------------------------------------------
---
2019-12-30T19:47:17.4085279Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:385:22
2019-12-30T19:47:17.4086687Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2019-12-30T19:47:17.4125052Z 
2019-12-30T19:47:17.4125140Z 
2019-12-30T19:47:17.4127502Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-7/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "7.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2019-12-30T19:47:17.4127897Z 
2019-12-30T19:47:17.4127928Z 
2019-12-30T19:47:17.4133739Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2019-12-30T19:47:17.4133976Z Build completed unsuccessfully in 0:59:38
2019-12-30T19:47:17.4133976Z Build completed unsuccessfully in 0:59:38
2019-12-30T19:47:17.4190990Z == clock drift check ==
2019-12-30T19:47:17.4204749Z   local time: Mon Dec 30 19:47:17 UTC 2019
2019-12-30T19:47:17.7118702Z   network time: Mon, 30 Dec 2019 19:47:17 GMT
2019-12-30T19:47:17.7124064Z == end clock drift check ==
2019-12-30T19:47:18.7463613Z 
2019-12-30T19:47:18.7609719Z ##[error]Bash exited with code '1'.
2019-12-30T19:47:18.7654555Z ##[section]Starting: Checkout
2019-12-30T19:47:18.7657143Z ==============================================================================
2019-12-30T19:47:18.7657208Z Task         : Get sources
2019-12-30T19:47:18.7657281Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

it uses normalize-stderr-test because not all targets hit the same OS error number nor message ...

... and ignores tidy since I dont know how to make the normalize line shorter ...

and has effectively a no-op for its error-pattern because the targets' error
messages are so wildly different (and the error-pattern check occurs *before*
stderr normalization.)
@pnkfelix pnkfelix force-pushed the fix-66530-by-propagating-fatal-error-from-worker branch from e60a406 to 7c5cff7 Compare December 31, 2019 05:05
@pnkfelix
Copy link
Member Author

I go around claiming I'm going to be careful about force-pushing, but next thing I know, I've blown away the version of the file that actually properly "worked" on Mac OS X and Linux ...

Sigh.

@pnkfelix
Copy link
Member Author

@bors r=matthewjasper

@bors
Copy link
Contributor

bors commented Dec 31, 2019

📌 Commit 7c5cff7 has been approved by matthewjasper

@Centril
Copy link
Contributor

Centril commented Dec 31, 2019

@bors rollup=never

@bors
Copy link
Contributor

bors commented Jan 1, 2020

⌛ Testing commit 7c5cff7 with merge 890d8c9e775898be5872e7e446dcebaa54b29ced...

@rust-highfive
Copy link
Collaborator

The job armhf-gnu of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-01-01T04:34:55.2973978Z ---- [ui] ui/non-ice-error-on-worker-io-fail.rs stdout ----
2020-01-01T04:34:55.2974041Z 
2020-01-01T04:34:55.2974123Z error: ui test compiled successfully!
2020-01-01T04:34:55.2974409Z status: exit code: 0
2020-01-01T04:34:55.2976083Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/non-ice-error-on-worker-io-fail.rs" "-Zthreads=1" "--target=arm-unknown-linux-gnueabihf" "--error-format" "json" "-Zui-testing" "--emit" "metadata" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/non-ice-error-on-worker-io-fail" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/arm-unknown-linux-gnueabihf/native/rust-test-helpers" "-Clinker=arm-linux-gnueabihf-gcc" "-o" "/dev/null" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/non-ice-error-on-worker-io-fail/auxiliary" "-A" "unused"
2020-01-01T04:34:55.2976687Z ------------------------------------------
2020-01-01T04:34:55.2976739Z 
2020-01-01T04:34:55.2976985Z ------------------------------------------
2020-01-01T04:34:55.2977079Z stderr:
2020-01-01T04:34:55.2977079Z stderr:
2020-01-01T04:34:55.2977329Z ------------------------------------------
2020-01-01T04:34:55.2977591Z warning: ignoring --out-dir flag due to -o flag
2020-01-01T04:34:55.2977816Z 
2020-01-01T04:34:55.2978082Z ------------------------------------------
2020-01-01T04:34:55.2978128Z 
2020-01-01T04:34:55.2978175Z 
---
2020-01-01T04:34:55.3027010Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:385:22
2020-01-01T04:34:55.3027691Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2020-01-01T04:34:55.3052105Z 
2020-01-01T04:34:55.3052194Z 
2020-01-01T04:34:55.3058186Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/arm-unknown-linux-gnueabihf/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-arm-unknown-linux-gnueabihf" "--mode" "ui" "--target" "arm-unknown-linux-gnueabihf" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/checkout/obj/build/x86_64-unknown-linux-gnu/llvm/build/bin/FileCheck" "--linker" "arm-linux-gnueabihf-gcc" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/arm-unknown-linux-gnueabihf/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--llvm-version" "9.0.0-rust-1.42.0-nightly\n" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--remote-test-client" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/remote-test-client" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2020-01-01T04:34:55.3059304Z 
2020-01-01T04:34:55.3059386Z 
2020-01-01T04:34:55.3064316Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test --target arm-unknown-linux-gnueabihf
2020-01-01T04:34:55.3064460Z Build completed unsuccessfully in 2:21:58
2020-01-01T04:34:55.3064460Z Build completed unsuccessfully in 2:21:58
2020-01-01T04:34:55.3224654Z == clock drift check ==
2020-01-01T04:34:55.3241183Z   local time: Wed Jan  1 04:34:55 UTC 2020
2020-01-01T04:34:55.8589871Z   network time: Wed, 01 Jan 2020 04:34:55 GMT
2020-01-01T04:34:55.8591007Z == end clock drift check ==
2020-01-01T04:34:56.6852296Z 
2020-01-01T04:34:56.6961182Z ##[error]Bash exited with code '1'.
2020-01-01T04:34:56.7015938Z ##[section]Starting: Checkout
2020-01-01T04:34:56.7018919Z ==============================================================================
2020-01-01T04:34:56.7019024Z Task         : Get sources
2020-01-01T04:34:56.7019103Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors
Copy link
Contributor

bors commented Jan 1, 2020

💔 Test failed - checks-azure

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jan 1, 2020
@matthewjasper matthewjasper added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 8, 2020
If this happens again, I'm going to change tack and make this
an `// only-linux` test or something along those lines.
@pnkfelix
Copy link
Member Author

pnkfelix commented Jan 9, 2020

@bors r=matthewjasper

@bors
Copy link
Contributor

bors commented Jan 9, 2020

📌 Commit 3193836 has been approved by matthewjasper

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 9, 2020
@bors
Copy link
Contributor

bors commented Jan 11, 2020

⌛ Testing commit 3193836 with merge f363745...

bors added a commit that referenced this pull request Jan 11, 2020
…from-worker, r=matthewjasper

When a codegen worker has a FatalError, propagate it instead of ICE'ing.

Fix #66530
@bors
Copy link
Contributor

bors commented Jan 12, 2020

☀️ Test successful - checks-azure
Approved by: matthewjasper
Pushing f363745 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Jan 12, 2020
@bors bors merged commit 3193836 into rust-lang:master Jan 12, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ICE when rustc asked to -o /dev/null on OSX
5 participants