PGO bustage in 1.56

[glandium]

Quickly filing this before I have more information to put it on someone's radar. A short description is that on Linux, Firefox builds with profile-generate enabled crash with all betas of 1.56 (which doesn't happen with 1.55 and earlier versions). I'll update the issue when I figure out more. It doesn't happen on Windows (and I can't say anything about mac because it fails for other reasons there).

LTO is not involved (Firefox does both cross-language LTO and PGO).

[Mark-Simulacrum]

We are (and have been) using PGO for rustc itself (on x86_64-unknown-linux-gnu), so it seems likely that things are not "terribly wrong" with the impl, fwiw. One thing that has changed is that LLVM is also compiled with PGO in our official builds as of 1.56, but I would not expect this to cause downstream issues with regards to correctness.

Would be good to get more information, and if possible, a bisection across rustc commits.

[glandium]

nightly-2021-08-21 is ok
nightly-2021-08-22 is broken
There isn't much between these two, the main thing is LLVM 13. I'll try with bors builds.

[glandium]

bors build from db002a0 is broken.
bors build from e7f7fe4 is ok.

db002a0 is the LLVM 13 merge.

[glandium]

nightly-2021-09-23 is also affected.

[glandium]

At this point, I'm rather convinced it has nothing to do with rust. I mean, obviously, the upgrade to LLVM 13 in rust is involved, but seeing how a) it doesn't reproduce with a local build of Firefox b) it doesn't reproduce locally using the same binaries used on CI in the same docker image used on CI c) it doesn't reproduce on CI under some circumstances the exact details of which are yet to be determined, I'm inclined to call this a pre-existing race condition in Firefox that rust code built with LLVM 13 makes more likely to happen under some circumstances.

[glandium]

For the curious: this was due to a race condition in cubeb-pulse-rs: mozilla/cubeb-pulse-rs#72