Wedged sccached timing out CI builds #39003
Labels
A-spurious
Area: Spurious failures in builds (spuriously == for no apparent reason)
Comments
alexcrichton
added
the
A-spurious
This was referenced Jan 11, 2017
This was referenced Jan 20, 2017
This was referenced Jan 24, 2017
This was referenced Jan 31, 2017
This was referenced Feb 7, 2017
Merged
This was referenced Feb 15, 2017
This was referenced Feb 22, 2017
alexcrichton
added a commit
to alexcrichton/rust
that referenced
this issue
Feb 24, 2017
Currently CI builds can fail spuriously during the LLVM build (rust-lang#39003). I believe this is due to sccache, and I believe that in turn was due to the fact that the sccache server used to just be a raw mio server. Historically raw mio servers are quite complicated to get right, but this is why we built Tokio! The sccache server has been migrated to Tokio which I suspect would fix any latent issues. I have no confirmation of this (never been able to reproduce the deadlock locally), but my hunch is that updating sccache to the master branch will fix the timeouts during the LLVM build. The binaries previously came from Gecko's infrastructure, but I've built new ones by hand for Win/Mac/Linux and uploaded them to our CI bucket.
bors
added a commit
that referenced
this issue
Feb 24, 2017
Update sccache binaries on CI Currently CI builds can fail spuriously during the LLVM build (#39003). I believe this is due to sccache, and I believe that in turn was due to the fact that the sccache server used to just be a raw mio server. Historically raw mio servers are quite complicated to get right, but this is why we built Tokio! The sccache server has been migrated to Tokio which I suspect would fix any latent issues. I have no confirmation of this (never been able to reproduce the deadlock locally), but my hunch is that updating sccache to the master branch will fix the timeouts during the LLVM build. The binaries previously came from Gecko's infrastructure, but I've built new ones by hand for Win/Mac/Linux and uploaded them to our CI bucket.
bors
added a commit
that referenced
this issue
Feb 25, 2017
Update sccache binaries on CI Currently CI builds can fail spuriously during the LLVM build (#39003). I believe this is due to sccache, and I believe that in turn was due to the fact that the sccache server used to just be a raw mio server. Historically raw mio servers are quite complicated to get right, but this is why we built Tokio! The sccache server has been migrated to Tokio which I suspect would fix any latent issues. I have no confirmation of this (never been able to reproduce the deadlock locally), but my hunch is that updating sccache to the master branch will fix the timeouts during the LLVM build. The binaries previously came from Gecko's infrastructure, but I've built new ones by hand for Win/Mac/Linux and uploaded them to our CI bucket.
This was referenced Feb 25, 2017
Merged
|
Hopefully closed by #40084 |
This was referenced Mar 1, 2017
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Some builds have gotten stuck when compiling LLVM which I suspect is due to sccache getting wedged. Some example logs:
Unfortunately I don't have a lot more information other than that :(
cc @luser
The text was updated successfully, but these errors were encountered: