Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consistently unset RUSTC_BOOTSTRAP when compiling bootstrap #120001

Merged
merged 1 commit into from
Jan 18, 2024

Conversation

dtolnay
Copy link
Member

@dtolnay dtolnay commented Jan 15, 2024

Since #113906, all x.py invocations performed by rust-analyzer have RUSTC_BOOTSTRAP=1 set. This was to fix #112391 (comment) — rust-analyzer uses some default cargo from the system when fetching workspace layout, and the standard library uses some unstable cargo feature, so x.py would previously fail if the system toolchain wasn't nightly.

cargo-features = ["public-dependency"]

This PR changes x.py to cancel out this RUSTC_BOOTSTRAP=1 when compiling bootstrap. It will only remain set when running the compiled bootstrap executable. This fixes spurious bootstrap rebuilds in the event that a rust-analyzer x.py invocation is alternated with someone running x.py themself on the command line, if any dependency of bootstrap looks at option_env!("RUSTC_BOOTSTRAP"), which is the case since #119654.

Before:

$ RUSTC_BOOTSTRAP=1 ./x.py check library/core
Building bootstrap
   Compiling proc-macro2 v1.0.76
   Compiling quote v1.0.35
   Compiling syn v2.0.48
   Compiling clap_derive v4.4.7
   Compiling serde_derive v1.0.195
   Compiling clap v4.4.13
   Compiling clap_complete v4.4.6
   Compiling build_helper v0.1.0
   Compiling bootstrap v0.0.0
    Finished dev [unoptimized] target(s) in 6.31s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.23s
Build completed successfully in 0:00:07

$ ./x.py check library/core
Building bootstrap
   Compiling proc-macro2 v1.0.76
   Compiling quote v1.0.35
   Compiling syn v2.0.48
   Compiling clap_derive v4.4.7
   Compiling serde_derive v1.0.195
   Compiling clap v4.4.13
   Compiling clap_complete v4.4.6
   Compiling build_helper v0.1.0
   Compiling bootstrap v0.0.0
    Finished dev [unoptimized] target(s) in 5.30s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.25s
Build completed successfully in 0:00:06

After:

$ RUSTC_BOOTSTRAP=1 ./x.py check library/core
Building bootstrap
    Finished dev [unoptimized] target(s) in 0.06s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.14s
Build completed successfully in 0:00:01

$ ./x.py check library/core
Building bootstrap
    Finished dev [unoptimized] target(s) in 0.04s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.13s
Build completed successfully in 0:00:01

@rustbot
Copy link
Collaborator

rustbot commented Jan 15, 2024

r? @clubby789

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Jan 15, 2024
@onur-ozkan
Copy link
Member

@bors r+

@bors
Copy link
Contributor

bors commented Jan 16, 2024

📌 Commit ee370a1 has been approved by onur-ozkan

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 16, 2024
@dtolnay dtolnay assigned onur-ozkan and unassigned clubby789 Jan 16, 2024
@compiler-errors
Copy link
Member

@bors rollup

compiler-errors added a commit to compiler-errors/rust that referenced this pull request Jan 17, 2024
…-ozkan

Consistently unset RUSTC_BOOTSTRAP when compiling bootstrap

Since rust-lang#113906, all x.py invocations performed by rust-analyzer have RUSTC_BOOTSTRAP=1 set. This was to fix rust-lang#112391 (comment) — rust-analyzer uses some default cargo from the system when fetching workspace layout, and the standard library uses some unstable cargo feature, so x.py would previously fail if the system toolchain wasn't nightly.

https://github.com/rust-lang/rust/blob/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/Cargo.toml#L1

This PR changes x.py to cancel out this RUSTC_BOOTSTRAP=1 when compiling bootstrap. It will only remain set when running the compiled bootstrap executable. This fixes spurious bootstrap rebuilds in the event that a rust-analyzer x.py invocation is alternated with someone running x.py themself on the command line, if any dependency of bootstrap looks at `option_env!("RUSTC_BOOTSTRAP")`, which is the case since rust-lang#119654.

**Before:**

```console
$ RUSTC_BOOTSTRAP=1 ./x.py check library/core
Building bootstrap
   Compiling proc-macro2 v1.0.76
   Compiling quote v1.0.35
   Compiling syn v2.0.48
   Compiling clap_derive v4.4.7
   Compiling serde_derive v1.0.195
   Compiling clap v4.4.13
   Compiling clap_complete v4.4.6
   Compiling build_helper v0.1.0
   Compiling bootstrap v0.0.0
    Finished dev [unoptimized] target(s) in 6.31s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.23s
Build completed successfully in 0:00:07

$ ./x.py check library/core
Building bootstrap
   Compiling proc-macro2 v1.0.76
   Compiling quote v1.0.35
   Compiling syn v2.0.48
   Compiling clap_derive v4.4.7
   Compiling serde_derive v1.0.195
   Compiling clap v4.4.13
   Compiling clap_complete v4.4.6
   Compiling build_helper v0.1.0
   Compiling bootstrap v0.0.0
    Finished dev [unoptimized] target(s) in 5.30s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.25s
Build completed successfully in 0:00:06
```

**After:**

```console
$ RUSTC_BOOTSTRAP=1 ./x.py check library/core
Building bootstrap
    Finished dev [unoptimized] target(s) in 0.06s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.14s
Build completed successfully in 0:00:01

$ ./x.py check library/core
Building bootstrap
    Finished dev [unoptimized] target(s) in 0.04s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.13s
Build completed successfully in 0:00:01
```
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 17, 2024
…mpiler-errors

Rollup of 9 pull requests

Successful merges:

 - rust-lang#115291 (Save liveness results for DestinationPropagation)
 - rust-lang#119651 (proc_macro: Add Literal::c_string constructor)
 - rust-lang#119855 (Enable Static Builds for FreeBSD)
 - rust-lang#119955 (Modify GenericArg and Term structs to use strict provenance rules)
 - rust-lang#119975 (Don't ICE if TAIT-defining fn contains a closure with `_` in return type)
 - rust-lang#119984 (Change return type of unstable `Waker::noop()` from `Waker` to `&Waker`.)
 - rust-lang#120001 (Consistently unset RUSTC_BOOTSTRAP when compiling bootstrap)
 - rust-lang#120020 (Gracefully handle missing typeck information if typeck errored)
 - rust-lang#120032 (Fix `rustc_abi` build on stable)

r? `@ghost`
`@rustbot` modify labels: rollup
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Jan 17, 2024
…-ozkan

Consistently unset RUSTC_BOOTSTRAP when compiling bootstrap

Since rust-lang#113906, all x.py invocations performed by rust-analyzer have RUSTC_BOOTSTRAP=1 set. This was to fix rust-lang#112391 (comment) — rust-analyzer uses some default cargo from the system when fetching workspace layout, and the standard library uses some unstable cargo feature, so x.py would previously fail if the system toolchain wasn't nightly.

https://github.com/rust-lang/rust/blob/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/Cargo.toml#L1

This PR changes x.py to cancel out this RUSTC_BOOTSTRAP=1 when compiling bootstrap. It will only remain set when running the compiled bootstrap executable. This fixes spurious bootstrap rebuilds in the event that a rust-analyzer x.py invocation is alternated with someone running x.py themself on the command line, if any dependency of bootstrap looks at `option_env!("RUSTC_BOOTSTRAP")`, which is the case since rust-lang#119654.

**Before:**

```console
$ RUSTC_BOOTSTRAP=1 ./x.py check library/core
Building bootstrap
   Compiling proc-macro2 v1.0.76
   Compiling quote v1.0.35
   Compiling syn v2.0.48
   Compiling clap_derive v4.4.7
   Compiling serde_derive v1.0.195
   Compiling clap v4.4.13
   Compiling clap_complete v4.4.6
   Compiling build_helper v0.1.0
   Compiling bootstrap v0.0.0
    Finished dev [unoptimized] target(s) in 6.31s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.23s
Build completed successfully in 0:00:07

$ ./x.py check library/core
Building bootstrap
   Compiling proc-macro2 v1.0.76
   Compiling quote v1.0.35
   Compiling syn v2.0.48
   Compiling clap_derive v4.4.7
   Compiling serde_derive v1.0.195
   Compiling clap v4.4.13
   Compiling clap_complete v4.4.6
   Compiling build_helper v0.1.0
   Compiling bootstrap v0.0.0
    Finished dev [unoptimized] target(s) in 5.30s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.25s
Build completed successfully in 0:00:06
```

**After:**

```console
$ RUSTC_BOOTSTRAP=1 ./x.py check library/core
Building bootstrap
    Finished dev [unoptimized] target(s) in 0.06s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.14s
Build completed successfully in 0:00:01

$ ./x.py check library/core
Building bootstrap
    Finished dev [unoptimized] target(s) in 0.04s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.13s
Build completed successfully in 0:00:01
```
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 17, 2024
…iaskrgr

Rollup of 11 pull requests

Successful merges:

 - rust-lang#115291 (Save liveness results for DestinationPropagation)
 - rust-lang#119855 (Enable Static Builds for FreeBSD)
 - rust-lang#119975 (Don't ICE if TAIT-defining fn contains a closure with `_` in return type)
 - rust-lang#119984 (Change return type of unstable `Waker::noop()` from `Waker` to `&Waker`.)
 - rust-lang#120001 (Consistently unset RUSTC_BOOTSTRAP when compiling bootstrap)
 - rust-lang#120020 (Gracefully handle missing typeck information if typeck errored)
 - rust-lang#120031 (Construct closure type eagerly)
 - rust-lang#120032 (Fix `rustc_abi` build on stable)
 - rust-lang#120039 (pat_analysis: Don't rely on contiguous `VariantId`s outside of rustc)
 - rust-lang#120044 (Fix typo in comments (in_place_collect))
 - rust-lang#120056 (Use FnOnceOutput instead of FnOnce where expected)

r? `@ghost`
`@rustbot` modify labels: rollup
compiler-errors added a commit to compiler-errors/rust that referenced this pull request Jan 17, 2024
…-ozkan

Consistently unset RUSTC_BOOTSTRAP when compiling bootstrap

Since rust-lang#113906, all x.py invocations performed by rust-analyzer have RUSTC_BOOTSTRAP=1 set. This was to fix rust-lang#112391 (comment) — rust-analyzer uses some default cargo from the system when fetching workspace layout, and the standard library uses some unstable cargo feature, so x.py would previously fail if the system toolchain wasn't nightly.

https://github.com/rust-lang/rust/blob/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/Cargo.toml#L1

This PR changes x.py to cancel out this RUSTC_BOOTSTRAP=1 when compiling bootstrap. It will only remain set when running the compiled bootstrap executable. This fixes spurious bootstrap rebuilds in the event that a rust-analyzer x.py invocation is alternated with someone running x.py themself on the command line, if any dependency of bootstrap looks at `option_env!("RUSTC_BOOTSTRAP")`, which is the case since rust-lang#119654.

**Before:**

```console
$ RUSTC_BOOTSTRAP=1 ./x.py check library/core
Building bootstrap
   Compiling proc-macro2 v1.0.76
   Compiling quote v1.0.35
   Compiling syn v2.0.48
   Compiling clap_derive v4.4.7
   Compiling serde_derive v1.0.195
   Compiling clap v4.4.13
   Compiling clap_complete v4.4.6
   Compiling build_helper v0.1.0
   Compiling bootstrap v0.0.0
    Finished dev [unoptimized] target(s) in 6.31s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.23s
Build completed successfully in 0:00:07

$ ./x.py check library/core
Building bootstrap
   Compiling proc-macro2 v1.0.76
   Compiling quote v1.0.35
   Compiling syn v2.0.48
   Compiling clap_derive v4.4.7
   Compiling serde_derive v1.0.195
   Compiling clap v4.4.13
   Compiling clap_complete v4.4.6
   Compiling build_helper v0.1.0
   Compiling bootstrap v0.0.0
    Finished dev [unoptimized] target(s) in 5.30s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.25s
Build completed successfully in 0:00:06
```

**After:**

```console
$ RUSTC_BOOTSTRAP=1 ./x.py check library/core
Building bootstrap
    Finished dev [unoptimized] target(s) in 0.06s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.14s
Build completed successfully in 0:00:01

$ ./x.py check library/core
Building bootstrap
    Finished dev [unoptimized] target(s) in 0.04s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.13s
Build completed successfully in 0:00:01
```
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 17, 2024
…iaskrgr

Rollup of 10 pull requests

Successful merges:

 - rust-lang#115291 (Save liveness results for DestinationPropagation)
 - rust-lang#119855 (Enable Static Builds for FreeBSD)
 - rust-lang#119975 (Don't ICE if TAIT-defining fn contains a closure with `_` in return type)
 - rust-lang#120001 (Consistently unset RUSTC_BOOTSTRAP when compiling bootstrap)
 - rust-lang#120020 (Gracefully handle missing typeck information if typeck errored)
 - rust-lang#120031 (Construct closure type eagerly)
 - rust-lang#120032 (Fix `rustc_abi` build on stable)
 - rust-lang#120039 (pat_analysis: Don't rely on contiguous `VariantId`s outside of rustc)
 - rust-lang#120044 (Fix typo in comments (in_place_collect))
 - rust-lang#120056 (Use FnOnceOutput instead of FnOnce where expected)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 18, 2024
…iaskrgr

Rollup of 10 pull requests

Successful merges:

 - rust-lang#115291 (Save liveness results for DestinationPropagation)
 - rust-lang#119855 (Enable Static Builds for FreeBSD)
 - rust-lang#119975 (Don't ICE if TAIT-defining fn contains a closure with `_` in return type)
 - rust-lang#120001 (Consistently unset RUSTC_BOOTSTRAP when compiling bootstrap)
 - rust-lang#120020 (Gracefully handle missing typeck information if typeck errored)
 - rust-lang#120031 (Construct closure type eagerly)
 - rust-lang#120032 (Fix `rustc_abi` build on stable)
 - rust-lang#120039 (pat_analysis: Don't rely on contiguous `VariantId`s outside of rustc)
 - rust-lang#120044 (Fix typo in comments (in_place_collect))
 - rust-lang#120056 (Use FnOnceOutput instead of FnOnce where expected)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit cc9a4a2 into rust-lang:master Jan 18, 2024
11 checks passed
@rustbot rustbot added this to the 1.77.0 milestone Jan 18, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Jan 18, 2024
Rollup merge of rust-lang#120001 - dtolnay:bootstrapbootstrap, r=onur-ozkan

Consistently unset RUSTC_BOOTSTRAP when compiling bootstrap

Since rust-lang#113906, all x.py invocations performed by rust-analyzer have RUSTC_BOOTSTRAP=1 set. This was to fix rust-lang#112391 (comment) — rust-analyzer uses some default cargo from the system when fetching workspace layout, and the standard library uses some unstable cargo feature, so x.py would previously fail if the system toolchain wasn't nightly.

https://github.com/rust-lang/rust/blob/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/Cargo.toml#L1

This PR changes x.py to cancel out this RUSTC_BOOTSTRAP=1 when compiling bootstrap. It will only remain set when running the compiled bootstrap executable. This fixes spurious bootstrap rebuilds in the event that a rust-analyzer x.py invocation is alternated with someone running x.py themself on the command line, if any dependency of bootstrap looks at `option_env!("RUSTC_BOOTSTRAP")`, which is the case since rust-lang#119654.

**Before:**

```console
$ RUSTC_BOOTSTRAP=1 ./x.py check library/core
Building bootstrap
   Compiling proc-macro2 v1.0.76
   Compiling quote v1.0.35
   Compiling syn v2.0.48
   Compiling clap_derive v4.4.7
   Compiling serde_derive v1.0.195
   Compiling clap v4.4.13
   Compiling clap_complete v4.4.6
   Compiling build_helper v0.1.0
   Compiling bootstrap v0.0.0
    Finished dev [unoptimized] target(s) in 6.31s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.23s
Build completed successfully in 0:00:07

$ ./x.py check library/core
Building bootstrap
   Compiling proc-macro2 v1.0.76
   Compiling quote v1.0.35
   Compiling syn v2.0.48
   Compiling clap_derive v4.4.7
   Compiling serde_derive v1.0.195
   Compiling clap v4.4.13
   Compiling clap_complete v4.4.6
   Compiling build_helper v0.1.0
   Compiling bootstrap v0.0.0
    Finished dev [unoptimized] target(s) in 5.30s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.25s
Build completed successfully in 0:00:06
```

**After:**

```console
$ RUSTC_BOOTSTRAP=1 ./x.py check library/core
Building bootstrap
    Finished dev [unoptimized] target(s) in 0.06s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.14s
Build completed successfully in 0:00:01

$ ./x.py check library/core
Building bootstrap
    Finished dev [unoptimized] target(s) in 0.04s
Checking stage0 library artifacts {core} (x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.13s
Build completed successfully in 0:00:01
```
@dtolnay dtolnay deleted the bootstrapbootstrap branch January 18, 2024 07:28
@petrochenkov
Copy link
Contributor

I think this just broke building everything with -Zthreads=8 - https://internals.rust-lang.org/t/rust-compiler-with-parallel-build/20099
If RUSTC_BOOTSTRAP was set externally, then it should not be removed.

@onur-ozkan
Copy link
Member

onur-ozkan commented Jan 18, 2024

I think this just broke building everything with -Zthreads=8 - https://internals.rust-lang.org/t/rust-compiler-with-parallel-build/20099

Correct.

If RUSTC_BOOTSTRAP was set externally, then it should not be removed.

I think setting it consistently is much better approach (or if it wasn't manually set from user). What do you think ? @dtolnay

@dtolnay
Copy link
Member Author

dtolnay commented Jan 18, 2024

I think setting it consistently is much better approach (or if it wasn't manually set from user). What do you think ? @dtolnay

I agree with this, but responded in #120096 (review) with one modification I would suggest.

matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Jan 18, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Jan 19, 2024
Rollup merge of rust-lang#120096 - onur-ozkan:rustc_bootstrap, r=dtolnay

Set RUSTC_BOOTSTRAP=1 consistently

Fixes https://internals.rust-lang.org/t/rust-compiler-with-parallel-build/20099 which is a regression from rust-lang#120001

cc `@dtolnay` `@petrochenkov`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Vendoring broken with stable rustc
7 participants