Automatically open an issue when a tool breaks

[oli-obk]

cc @nikomatsakis

fixes rust-lang-nursery/rust-toolstate#6

documentation about issue opening via the github api: https://developer.github.com/v3/issues/#create-an-issue

[rust-highfive]

r? @nikomatsakis

(rust_highfive has picked a reviewer for you, use r? to override)

[nikomatsakis]

r? @kennytm

[nikomatsakis]

This is awesome— can we make it be automatically T-compiler and I-nominated? (I'm assuming T-compiler will always be the appropriate team)

[kennytm]

I'm not sure if this is a good idea as RLS is known to sometimes fail spuriously (recent example: #56161 (comment) → "fixed" by #56461 (comment)).

[phansch]

This probably would mention the wrong person/PR when a rollup caused breakage, right?

[kennytm]

For rollups it would mention the person who created the rollup (same as today), but since the rollupers are also Rust team members they could just edit the issue to point to the actual cause.

What I worried is blaming new contributors over something not related to their code.

[oli-obk]

What I worried is blaming new contributors over something not related to their code.

due to the spurious case? We could just do this for rustfmt and clippy for now until rls isn't spurious anymore.

[kennytm]

@oli-obk Yes due to spurious cases. What if we submit the issue only if the state is degraded to build-fail, which won't be spurious?

[oli-obk]

I pushed some changes, not sure if they actually do what they're supposed to. Are we really using < and > to compare the strings build-fail, test-fail, test-pass and that just happens to work out because of the lexical ordering of fail and pass and the lexical ordering of build and test?

[kennytm]

@oli-obk

Are we really using < and > to compare the strings build-fail, test-fail, test-pass and that just happens to work out because of the lexical ordering of fail and pass and the lexical ordering of build and test?

Yes 😛

[nikomatsakis]

I think that in terms of "blaming" contributors, we should just be careful with our wording, and make sure to cc enough folks from the team.

Is the plan to ignore test failures and just focus on build failures? That seems "ok" to start.

I would like to see the template be something a bit more friendly and perhaps include a bit more information. Maybe something like this?

---

issue title: {TOOL} no longer builds after PR #{PR}

Hello, this is your friendly neighborhood mergebot. After merging PR #{PR}, I observed that the tool {TOOL} no longer builds. A follow-up PR to the repository {TOOL_REPOSITORY} is needed to fix the fallout.

cc @{PR_AUTHOR}, do you think you would have time to do the follow-up work? If so, that would be great!

cc @{PR_REVIEWER}, the PR reviewer, and @rust-lang/compiler -- nominating for prioritization.

[rust-highfive]

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:2bc5a7ee:start=1545237978429117636,finish=1545238035323829083,duration=56894711447
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export IMAGE=x86_64-gnu-llvm-6.0
---

[00:03:02] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:03:02] tidy error: /checkout/src/tools/publish_toolstate.py:78: line longer than 100 chars
[00:03:02] tidy error: /checkout/src/tools/publish_toolstate.py:147: line longer than 100 chars
[00:03:02] tidy error: /checkout/src/tools/publish_toolstate.py:171: line longer than 100 chars
[00:03:03] some tidy checks failed
[00:03:03] 
[00:03:03] 
[00:03:03] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:03:03] 
[00:03:03] 
[00:03:03] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:03:03] Build completed unsuccessfully in 0:00:43
[00:03:03] Build completed unsuccessfully in 0:00:43
[00:03:03] Makefile:79: recipe for target 'tidy' failed
[00:03:03] make: *** [tidy] Error 1
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:22644c00
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Wed Dec 19 16:50:27 UTC 2018
---
travis_time:end:189241f0:start=1545238228319777533,finish=1545238228324256399,duration=4478866
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0ba2a63f
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:00182946
travis_time:start:00182946
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:0277539b
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[kennytm]

Rest LGTM, but I'd like to first go through an FCP after the holiday (next week).

[oli-obk]

@bors r=kennytm

[bors]

📌 Commit 6ed4401 has been approved by kennytm

[bors]

⌛ Testing commit 6ed4401 with merge 39817c1...

[bors]

💔 Test failed - status-appveyor

[mati865]

Command exited with code 259

#58160

[kennytm]

@bors retry

[bors]

⌛ Testing commit 6ed4401 with merge a7902ab0ad5d6cd149d02b5651dd387f4b9507c5...

[bors]

💔 Test failed - checks-travis

[rust-highfive]

The job dist-i686-apple of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

Building stage2 tool cargo (i686-apple-darwin)
[01:07:59]  Downloading crates ...
[01:07:59] warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates.io)
[01:07:59] warning: spurious network error (1 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates.io)
[01:07:59] error: failed to download from `https://crates.io/api/v1/crates/openssl-src/111.1.0+1.1.1a/download`
[01:07:59] Caused by:
[01:07:59]   [6] Couldn't resolve host name (Could not resolve host: crates.io)
[01:07:59] command did not execute successfully: "/Users/travis/build/rust-lang/rust/build/i686-apple-darwin/stage0/bin/cargo" "build" "--target" "i686-apple-darwin" "-j" "4" "--release" "--locked" "--color" "always" "--manifest-path" "/Users/travis/build/rust-lang/rust/src/tools/cargo/Cargo.toml" "--features" "rustc-workspace-hack/all-static" "--message-format" "json"
[01:07:59] expected success, got: exit code: 101
[01:07:59] expected success, got: exit code: 101
[01:07:59] failed to run: /Users/travis/build/rust-lang/rust/build/bootstrap/debug/bootstrap build
[01:07:59] Build completed unsuccessfully in 1:03:39
[01:07:59] make: *** [all] Error 1
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:351e2e2e
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Tue Feb 12 23:28:18 GMT 2019
---
travis_fold:start:after_failure.2
travis_time:start:004f399c
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
total 0
drwx------+ 15 travis  staff  510 Jan 25  2018 ..
drwx------   2 travis  staff   68 Dec  6  2017 .
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:06e92be4
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
travis_time:end:06e92be4:start=1550014104119842000,finish=1550014104148783000,duration=28941000
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0ed34a82
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:205b9c25
travis_time:start:205b9c25
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:027aca82
$ dmesg | grep -i kill
$ dmesg | grep -i kill
Unable to obtain kernel buffer: Operation not permitted
usage: sudo dmesg
travis_fold:end:after_failure.6

Done. Your build exited with 1.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[kennytm]

@bors retry 🤔

[RalfJung]

So far, at least for Miri, every single case of "test-fail" was a real failure.

[Centril]

You might want to special-case miri in a follow-up PR?

[bors]

⌛ Testing commit 6ed4401 with merge e544947...

[bors]

☀️ Test successful - checks-travis, status-appveyor
Approved by: kennytm
Pushing e544947 to master...