Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compiletest should be built with the stage2 compiler not the stage0 compiler #59264

Closed
infinity0 opened this issue Mar 17, 2019 · 9 comments · Fixed by #68019
Closed

compiletest should be built with the stage2 compiler not the stage0 compiler #59264

infinity0 opened this issue Mar 17, 2019 · 9 comments · Fixed by #68019
Labels
T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Comments

@infinity0
Copy link
Contributor

See #57709. Unfortunately the "fix" to that bug breaks the build when building using 1.31.0 and you have to unapply the patch to make it work against 1.31.0.

e.g. from https://buildd.debian.org/status/fetch.php?pkg=rustc&arch=amd64&ver=1.32.0%2Bdfsg1-1&stamp=1548672224&raw=0 :

rustc command: "LD_LIBRARY_PATH"="/usr/lib:/<<BUILDDIR>>/rustc-1.32.0+dfsg1/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/release/deps:/usr/lib" "/usr/bin/rustc" "--crate-name" "compiletest" "

The problem here is that if stage0 is 1.31.0 then the patch MUST be unapplied, if stage0 is 1.32.0 then the patch MUST be applied. Obviously we can't do both at once.

However in both Debian and Fedora we occasionally use either the current or the previous version of rustc to build a given version of rustc, and both of these versions should work.

cc @cuviper

@infinity0
Copy link
Contributor Author

Doing things this way (using stage2) would also have ensured that #57709 was detected early, and never made it into a release.

@Mark-Simulacrum
Copy link
Member

We would prefer to build tools such as compiletest with the bootstrap compiler because stage1, stage2 compilers take a long time to build.

I think largely we just can't at this point support building with multiple compilers -- i.e., the bootstrap compiler should be the same one we use in tree. We just don't have the CI budget and resources to test and support multiple versions of the compiler for crates using unstable features.

As such, I think the only resolution here will be to close this...

cc @rust-lang/infra and @alexcrichton though since this is somewhat of a policy type decision (and we don't have a firm policy AFAIK).

@cuviper
Copy link
Member

cuviper commented Mar 17, 2019

As a matter of policy, we do need such rebuild-bootstrapping to continue to be supported to some extent, even if it's only a "tier-2" use case. That is, fixing these sort of issues reactively is still better than nothing.

@infinity0
Copy link
Contributor Author

Is conditional compilation based on the compiler version possible in Rust today? That could be an alternative solution, instead of always using stage2. The language here is a bit technical and I couldn't figure it out.

@infinity0
Copy link
Contributor Author

I guess for now, I could hack that sort of logic into the Debian build script manually in an ad-hoc way - applying the patch vs not, based on the stage0 compiler version.

@jonas-schievink jonas-schievink added the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label Mar 18, 2019
@alexcrichton
Copy link
Member

I think if we wanted to solve this issue the best way to do it would be to:

  • Require compiletest compiles on stable, except for the dependency on the test crate
  • Have compiletest depend on std/test in-tree binaries, so we always build them (with the stage0 compiler)

That way compiletest works with rebuild-bootstraping and normal use cases because it's not using anything unstable except libtest, which is built from source to link against and already solves this problem otherwise.

@Mark-Simulacrum
Copy link
Member

Have compiletest depend on std/test in-tree binaries, so we always build them (with the stage0 compiler)

I think in general a policy of unstable tools/dependencies having the requirement of being in-tree deps instead of beta is reasonable, yeah.

I can try to make a patch that makes compiletest (and if there are any other such tools) do this sometime this week.

@infinity0
Copy link
Contributor Author

Same issue with 1.40.0 again, when using itself as stage0:

   Compiling compiletest v0.0.0 (/<<PKGBUILDDIR>>/src/tools/compiletest)
     Running `CARGO=/usr/bin/cargo CARGO_MANIFEST_DIR=/<<PKGBUILDDIR>>/src/tools/compiletest CARGO_PKG_VERSION=0.0.0 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_MINOR=0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_NAME=compiletest CARGO_PKG_DESCRIPTION= CARGO_PKG_REPOSITORY= CARGO_PKG_AUTHORS='The Rust Project Developers' CARGO_PKG_HOMEPAGE= CARGO_PKG_VERSION_PRE= LD_LIBRARY_PATH='/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/release/deps:/usr/lib' /<<PKGBUILDDIR>>/build/bootstrap/debug/rustc --edition=2018 --crate-name compiletest src/tools/compiletest/src/main.rs --error-format json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=2 -C debuginfo=0 -C metadata=3bff92eb77ed034a -C extra-filename=-3bff92eb77ed034a --out-dir /<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps --target powerpc64le-unknown-linux-gnu -C linker=powerpc64le-linux-gnu-gcc -L dependency=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps -L dependency=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/release/deps --extern diff=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libdiff-f7c70ead82d3bb7b.rlib --extern env_logger=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libenv_logger-42bd3e818814c443.rlib --extern getopts=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libgetopts-9ad175f6b0afc4b5.rlib --extern lazy_static=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblazy_static-0d6c5f722411bcab.rlib --extern libc=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblibc-0f34c56984de760b.rlib --extern log=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblog-1d69c45aedc3973c.rlib --extern regex=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libregex-877563e5fb3e2b70.rlib --extern rustfix=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/librustfix-52c76b39e6b4bcbd.rlib --extern serde=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libserde-6b72e6c8366be6dd.rlib --extern serde_json=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libserde_json-0db31fa1a064a5dd.rlib --extern walkdir=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libwalkdir-243dfb4a0f7f76f8.rlib -C link-args=-Wl,-z,relro --cap-lints warn --remap-path-prefix=/<<PKGBUILDDIR>>=/usr/src/rustc-1.40.0 -Zexternal-macro-backtrace -Zbinary-dep-depinfo -L native=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/build/backtrace-sys-6a5e41a89dbb2581/out`
rustc command: "LD_LIBRARY_PATH"="/usr/lib:/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/release/deps:/usr/lib" "/usr/bin/rustc" "--edition=2018" "--crate-name" "compiletest" "src/tools/compiletest/src/main.rs" "--error-format" "json" "--json=diagnostic-rendered-ansi" "--crate-type" "bin" "--emit=dep-info,link" "-C" "opt-level=2" "-C" "debuginfo=0" "-C" "metadata=3bff92eb77ed034a" "-C" "extra-filename=-3bff92eb77ed034a" "--out-dir" "/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps" "--target" "powerpc64le-unknown-linux-gnu" "-C" "linker=powerpc64le-linux-gnu-gcc" "-L" "dependency=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps" "-L" "dependency=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/release/deps" "--extern" "diff=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libdiff-f7c70ead82d3bb7b.rlib" "--extern" "env_logger=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libenv_logger-42bd3e818814c443.rlib" "--extern" "getopts=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libgetopts-9ad175f6b0afc4b5.rlib" "--extern" "lazy_static=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblazy_static-0d6c5f722411bcab.rlib" "--extern" "libc=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblibc-0f34c56984de760b.rlib" "--extern" "log=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblog-1d69c45aedc3973c.rlib" "--extern" "regex=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libregex-877563e5fb3e2b70.rlib" "--extern" "rustfix=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/librustfix-52c76b39e6b4bcbd.rlib" "--extern" "serde=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libserde-6b72e6c8366be6dd.rlib" "--extern" "serde_json=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libserde_json-0db31fa1a064a5dd.rlib" "--extern" "walkdir=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libwalkdir-243dfb4a0f7f76f8.rlib" "-C" "link-args=-Wl,-z,relro" "--cap-lints" "warn" "--remap-path-prefix=/<<PKGBUILDDIR>>=/usr/src/rustc-1.40.0" "-Zexternal-macro-backtrace" "-Zbinary-dep-depinfo" "-L" "native=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/build/backtrace-sys-6a5e41a89dbb2581/out" "--sysroot" "/usr" "-C" "debug-assertions=n"
sysroot: "/usr"
libdir: "/usr/lib"
error[E0063]: missing field `time_options` in initializer of `test::TestOpts`
   --> src/tools/compiletest/src/main.rs:545:5
    |
545 |     test::TestOpts {
    |     ^^^^^^^^^^^^^^ missing `time_options`

error[E0063]: missing field `test_type` in initializer of `test::TestDesc`
   --> src/tools/compiletest/src/main.rs:701:23
    |
701 |                 desc: test::TestDesc {
    |                       ^^^^^^^^^^^^^^ missing `test_type`

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0063`.

Did not run successfully: Ok(ExitStatus(ExitStatus(256)))
"/usr/bin/rustc" "--edition=2018" "--crate-name" "compiletest" "src/tools/compiletest/src/main.rs" "--error-format" "json" "--json=diagnostic-rendered-ansi" "--crate-type" "bin" "--emit=dep-info,link" "-C" "opt-level=2" "-C" "debuginfo=0" "-C" "metadata=3bff92eb77ed034a" "-C" "extra-filename=-3bff92eb77ed034a" "--out-dir" "/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps" "--target" "powerpc64le-unknown-linux-gnu" "-C" "linker=powerpc64le-linux-gnu-gcc" "-L" "dependency=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps" "-L" "dependency=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/release/deps" "--extern" "diff=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libdiff-f7c70ead82d3bb7b.rlib" "--extern" "env_logger=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libenv_logger-42bd3e818814c443.rlib" "--extern" "getopts=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libgetopts-9ad175f6b0afc4b5.rlib" "--extern" "lazy_static=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblazy_static-0d6c5f722411bcab.rlib" "--extern" "libc=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblibc-0f34c56984de760b.rlib" "--extern" "log=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblog-1d69c45aedc3973c.rlib" "--extern" "regex=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libregex-877563e5fb3e2b70.rlib" "--extern" "rustfix=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/librustfix-52c76b39e6b4bcbd.rlib" "--extern" "serde=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libserde-6b72e6c8366be6dd.rlib" "--extern" "serde_json=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libserde_json-0db31fa1a064a5dd.rlib" "--extern" "walkdir=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libwalkdir-243dfb4a0f7f76f8.rlib" "-C" "link-args=-Wl,-z,relro" "--cap-lints" "warn" "--remap-path-prefix=/<<PKGBUILDDIR>>=/usr/src/rustc-1.40.0" "-Zexternal-macro-backtrace" "-Zbinary-dep-depinfo" "-L" "native=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/build/backtrace-sys-6a5e41a89dbb2581/out" "--sysroot" "/usr" "-C" "debug-assertions=n"

CC @cuviper AFAICT you do not have a patch that fixes this on Fedora either.

@infinity0
Copy link
Contributor Author

For 1.40.0 the relevant commit to conditionally-apply or not is f6832ad.

For the upcoming 1.41.0 (or is it 1.42.0) stable release it seems we will have to do the same thing for 241d2e7.

Centril added a commit to Centril/rust that referenced this issue Jan 9, 2020
…-Simulacrum

Build compiletest with in-tree libtest

This updates compiletest to build in `Mode::ToolStd`, using the locally-built crates for `std` and especially `test`. This way we're immune to unstable differences in the bootstrap compiler crates, whether that's a prior-release stage0 or a current release local rebuild. Fixes rust-lang#59264.

As a minor cleanup, this also removes the unused `llvm_tools` flag.
Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this issue Jan 10, 2020
…-Simulacrum

Build compiletest with in-tree libtest

This updates compiletest to build in `Mode::ToolStd`, using the locally-built crates for `std` and especially `test`. This way we're immune to unstable differences in the bootstrap compiler crates, whether that's a prior-release stage0 or a current release local rebuild. Fixes rust-lang#59264.

As a minor cleanup, this also removes the unused `llvm_tools` flag.
Centril added a commit to Centril/rust that referenced this issue Jan 10, 2020
…-Simulacrum

Build compiletest with in-tree libtest

This updates compiletest to build in `Mode::ToolStd`, using the locally-built crates for `std` and especially `test`. This way we're immune to unstable differences in the bootstrap compiler crates, whether that's a prior-release stage0 or a current release local rebuild. Fixes rust-lang#59264.

As a minor cleanup, this also removes the unused `llvm_tools` flag.
JohnTitor added a commit to JohnTitor/rust that referenced this issue Jan 10, 2020
…-Simulacrum

Build compiletest with in-tree libtest

This updates compiletest to build in `Mode::ToolStd`, using the locally-built crates for `std` and especially `test`. This way we're immune to unstable differences in the bootstrap compiler crates, whether that's a prior-release stage0 or a current release local rebuild. Fixes rust-lang#59264.

As a minor cleanup, this also removes the unused `llvm_tools` flag.
@bors bors closed this as completed in a74c790 Jan 10, 2020
jyn514 added a commit to jyn514/rust that referenced this issue May 22, 2022
This statically prevents issues like rust-lang#59264,
where tools can only be built with the in-tree compiler and not beta.
jyn514 added a commit to jyn514/rust that referenced this issue May 25, 2022
Using language features occasionally causes issues when using nightly to bootstrap, rather than beta.
See rust-lang#59264 for additional context.
bors added a commit to rust-lang-ci/rust that referenced this issue May 26, 2022
…lacrum

Only allow `compiletest` to use `feature(test)`, not any other feature

Using language features occasionally causes issues when using nightly to bootstrap, rather than beta.
See rust-lang#59264 for additional context.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants