Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a way for the test suite to test the output of a stable rustc #123404

Open
estebank opened this issue Apr 3, 2024 · 6 comments
Open

Add a way for the test suite to test the output of a stable rustc #123404

estebank opened this issue Apr 3, 2024 · 6 comments
Labels
A-test-infra Area: test infrastructure (may span bootstrap/compiletest/more) A-testsuite Area: The testsuite used to check the correctness of rustc C-feature-request Category: A feature request, i.e: not implemented / a PR. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@estebank
Copy link
Contributor

estebank commented Apr 3, 2024

Right now ui tests only target the output of the currently built rustc. As far as I can tell, there's no -Z flag to tell rustc "act as if you were a stable release". This means that when we customize the diagnostic output based on release channel, we cannot test what stable users will actually see. An example of this is this (as of now unmerged) change that on stable refers to unsized locals and unsized fn arguments with the same note telling the user that's not allowed, while on nightly it tells the user which feature flag is gating that being allowed, without the note stating it's not allowed. That kind of behavior is replicated in multiple diagnostics involving nightly features already. It's the same "problem" we'd have with editions if we didn't already have a stable flag to specify them.

@estebank estebank added A-testsuite Area: The testsuite used to check the correctness of rustc T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) C-feature-request Category: A feature request, i.e: not implemented / a PR. labels Apr 3, 2024
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Apr 3, 2024
@onur-ozkan onur-ozkan removed the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Apr 3, 2024
@matthiaskrgr
Copy link
Member

Another usecase would be to use something like
RUSTC_STABLE=1 rustc ice.rs to check if an ICE would happen on $todays nightly in several months after promotion to beta/stable has happened already. 🙂

@Mark-Simulacrum
Copy link
Member

RUSTC_BOOTSTRAP=0 feels like a reasonable mechanism/extension to me.

@estebank
Copy link
Contributor Author

estebank commented Apr 4, 2024

I'd be concerned with people having set RUSTC_BOOTSTRAP=0 globally in their machines and suddenly having issues with nightly, but at the same time "setting that flag has no stability guarantees".

@jieyouxu
Copy link
Member

jieyouxu commented Nov 13, 2024

FWIW I went with RUSTC_BOOTSTRAP=-1 to mean "force-stable" in #132993 instead of RUSTC_BOOTSTRAP=0 which can seem like "not bootstrap" which is not quite the same I feel like, and probably confusing.

Implementation steps

UI tests can use something like

//@ rustc-env:RUSTC_BOOTSTRAP=-1
//@ only-nightly

to fool convince the unstable compiler under test to think it's very stable. We can of course also add a //@ pretend-stable-rustc but yeah.

@bjorn3
Copy link
Member

bjorn3 commented Nov 13, 2024

There is -Zallow-features= to disable support for all #![feature], but this doesn't affect commandline flags and doesn't work on stable either.

jhpratt added a commit to jhpratt/rust that referenced this issue Nov 18, 2024
…ukang

Make rustc consider itself a stable compiler when `RUSTC_BOOTSTRAP=-1`

Addresses rust-lang#123404 to allow test writers to specify `//@ rustc-env:RUSTC_BOOTSTRAP=-1` to have a given rustc consider itself a stable rustc. This is only intended for testing usages.

I did not use `RUSTC_BOOTSTRAP=0` because that can be confusing, i.e. one might think that means "not bootstrapping", but "forcing a given rustc to consider itself a stable compiler" is a different use case.

I also added a specific test to check `RUSTC_BOOTSTRAP`'s various values and how that interacts with rustc's stability story w.r.t. features and cli flags.

Noticed when trying to write a test for enabling ICE file dumping on stable.

Dunno if this needs a compiler FCP or MCP, but I can file an MCP or ask someone to start an FCP if needed. Note that `RUSTC_BOOTSTRAP` is a perma-unstable env var and has no stability guarantees (heh) whatsoever. This does not affect bootstrapping because bootstrap never sets `RUSTC_BOOTSTRAP=-1`. If someone does set that when bootstrapping, it is considered PEBKAC.

Accompanying dev-guide PR: rust-lang/rustc-dev-guide#2136

cc `@estebank` and `@rust-lang/wg-diagnostics` for FYI
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Nov 18, 2024
Rollup merge of rust-lang#132993 - jieyouxu:i_am_very_stable, r=chenyukang

Make rustc consider itself a stable compiler when `RUSTC_BOOTSTRAP=-1`

Addresses rust-lang#123404 to allow test writers to specify `//@ rustc-env:RUSTC_BOOTSTRAP=-1` to have a given rustc consider itself a stable rustc. This is only intended for testing usages.

I did not use `RUSTC_BOOTSTRAP=0` because that can be confusing, i.e. one might think that means "not bootstrapping", but "forcing a given rustc to consider itself a stable compiler" is a different use case.

I also added a specific test to check `RUSTC_BOOTSTRAP`'s various values and how that interacts with rustc's stability story w.r.t. features and cli flags.

Noticed when trying to write a test for enabling ICE file dumping on stable.

Dunno if this needs a compiler FCP or MCP, but I can file an MCP or ask someone to start an FCP if needed. Note that `RUSTC_BOOTSTRAP` is a perma-unstable env var and has no stability guarantees (heh) whatsoever. This does not affect bootstrapping because bootstrap never sets `RUSTC_BOOTSTRAP=-1`. If someone does set that when bootstrapping, it is considered PEBKAC.

Accompanying dev-guide PR: rust-lang/rustc-dev-guide#2136

cc `@estebank` and `@rust-lang/wg-diagnostics` for FYI
@jieyouxu jieyouxu added the A-test-infra Area: test infrastructure (may span bootstrap/compiletest/more) label Nov 18, 2024
@jieyouxu
Copy link
Member

Update: #132993 has now merged to support RUSTC_BOOTSTRAP=-1 to mean "pretend I am a stable compiler".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-test-infra Area: test infrastructure (may span bootstrap/compiletest/more) A-testsuite Area: The testsuite used to check the correctness of rustc C-feature-request Category: A feature request, i.e: not implemented / a PR. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

7 participants