Harden the pre-tyctxt query system against accidental recomputation #105603

oli-obk · 2022-12-12T10:50:03Z

While the current compiler has no issues where we take and then compute the query again, in #105462 I accidentally introduced such a case.

I also took the opportunity to remove peek_mut, which is only ever used for global_tcx to then invoke enter. I added an enter method directly on the query.

rustbot · 2022-12-12T10:50:11Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @petrochenkov (or someone else) soon.

Please see the contribution instructions for more information.

rustbot · 2022-12-12T10:50:14Z

The Miri subtree was changed

cc @rust-lang/miri

oli-obk · 2022-12-12T12:31:59Z

@bors try @rust-timer queue

bors · 2022-12-12T12:32:09Z

⌛ Trying commit d3de0c4 with merge 8148db11639606c756aa0a0fce2b93954d27c19e...

compiler/rustc_interface/src/queries.rs

bors · 2022-12-12T15:12:28Z

☀️ Try build successful - checks-actions
Build commit: 8148db11639606c756aa0a0fce2b93954d27c19e (8148db11639606c756aa0a0fce2b93954d27c19e)

rust-timer · 2022-12-12T16:30:59Z

Finished benchmarking commit (8148db11639606c756aa0a0fce2b93954d27c19e): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.4%	[-2.4%, -2.4%]	1
All ❌✅ (primary)	-	-	0

Cycles

This benchmark run did not return any relevant results for this metric.

compiler/rustc_interface/src/queries.rs

oli-obk · 2023-01-09T15:39:57Z

@rustbot ready

jyn514

This PR looks like an improvement over the status quo, but Queries is pretty hairy in general ... the whole take() and peek_mut() stuff has never made much sense to me; and why don't we compute the queries on NotComputedYet instead of panicking?

oli-obk · 2023-01-11T10:31:44Z

This PR looks like an improvement over the status quo, but Queries is pretty hairy in general ... the whole take() and peek_mut() stuff has never made much sense to me; and why don't we compute the queries on NotComputedYet instead of panicking?

The main issue is that in order to compute the queries we need access to the Queries datastructure, not just the Query itself (in order to be able to call other queries). This requires some fiddling, but then we end up in a situation where the queries could accidentally call each other directly. A bit hard to explain, so maybe look at the necessary change? (90d8197). It's rather verbose at the call sites now (we could avoid that by adding some macros, ... but that will just make it all harder to read).

I am fairly confident that all of this will get simpler with #105462 as most (all?) of these should become real TyCtxt queries

oli-obk · 2023-01-11T10:37:28Z

@bors try @rust-timer queue

bors · 2023-01-11T13:40:49Z

☀️ Try build successful - checks-actions
Build commit: 74edcc98681086c093ce31fc904515c465558ee3 (74edcc98681086c093ce31fc904515c465558ee3)

rust-timer · 2023-01-11T15:00:07Z

Finished benchmarking commit (74edcc98681086c093ce31fc904515c465558ee3): comparison URL.

Overall result: ❌ regressions - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.0%	[1.0%, 1.1%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-3.3%	[-3.4%, -3.1%]	2
Improvements ✅ (secondary)	-2.6%	[-4.3%, -0.7%]	25
All ❌✅ (primary)	-3.3%	[-3.4%, -3.1%]	2

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.6%	[-2.6%, -2.6%]	1
All ❌✅ (primary)	-	-	0

petrochenkov · 2023-01-11T22:03:33Z

r=me after squashing commits.
@rustbot author

oli-obk · 2023-01-12T09:26:42Z

@bors r=petrochenkov

bors · 2023-01-12T09:26:45Z

📌 Commit 58782a8 has been approved by petrochenkov

It is now in the queue for this repository.

bors · 2023-01-12T13:59:33Z

⌛ Testing commit 58782a8 with merge 222d1ff...

bors · 2023-01-12T17:10:56Z

☀️ Test successful - checks-actions
Approved by: petrochenkov
Pushing 222d1ff to master...

rust-timer · 2023-01-12T19:17:35Z

Finished benchmarking commit (222d1ff): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	5.0%	[3.7%, 7.2%]	3
Regressions ❌ (secondary)	2.6%	[0.7%, 4.6%]	29
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	5.0%	[3.7%, 7.2%]	3

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.0%	[1.0%, 1.0%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.0%	[1.0%, 1.0%]	1

This commit fixes compilation errors but not runtime ones. Related changes: rust-lang/rust#104986 rust-lang/rust#105657 rust-lang/rust#105603 rust-lang/rust#105613

…rochenkov give the resolver access to TyCtxt The resolver is now created after TyCtxt is created. Then macro expansion and name resolution are run and the results fed into queries just like before this PR. Since the resolver had (before this PR) mutable access to the `CStore` and the source span table, these two datastructures are now behind a `RwLock`. To ensure that these are not mutated anymore after the resolver is done, a read lock to them is leaked right after the resolver finishes. ### PRs split out of this one and leading up to it: * rust-lang#105423 * rust-lang#105357 * rust-lang#105603 * rust-lang#106776 * rust-lang#106810 * rust-lang#106812 * rust-lang#108032

Upgrade our toolchain to `nightly-2023-01-23`. The changes here are related to the following changes: - rust-lang/rust#104986 - rust-lang/rust#105657 - rust-lang/rust#105603 - rust-lang/rust#105613 - rust-lang/rust#105977 - rust-lang/rust#104645

give the resolver access to TyCtxt The resolver is now created after TyCtxt is created. Then macro expansion and name resolution are run and the results fed into queries just like before this PR. Since the resolver had (before this PR) mutable access to the `CStore` and the source span table, these two datastructures are now behind a `RwLock`. To ensure that these are not mutated anymore after the resolver is done, a read lock to them is leaked right after the resolver finishes. ### PRs split out of this one and leading up to it: * rust-lang/rust#105423 * rust-lang/rust#105357 * rust-lang/rust#105603 * rust-lang/rust#106776 * rust-lang/rust#106810 * rust-lang/rust#106812 * rust-lang/rust#108032

rustbot assigned petrochenkov Dec 12, 2022

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Dec 12, 2022

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Dec 12, 2022

RalfJung reviewed Dec 12, 2022

View reviewed changes

compiler/rustc_interface/src/queries.rs Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Dec 12, 2022

petrochenkov reviewed Dec 21, 2022

View reviewed changes

compiler/rustc_interface/src/queries.rs Outdated Show resolved Hide resolved

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 21, 2022

cjgillot assigned cjgillot and unassigned cjgillot Dec 25, 2022

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 9, 2023

oli-obk mentioned this pull request Jan 10, 2023

give the resolver access to TyCtxt #105462

Merged

jyn514 approved these changes Jan 10, 2023

View reviewed changes

oli-obk force-pushed the non_repeatable_queries branch from fbce585 to 48a581a Compare January 11, 2023 10:36

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jan 11, 2023

oli-obk force-pushed the non_repeatable_queries branch from 48a581a to cf3b18b Compare January 11, 2023 15:43

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 11, 2023

Harden the pre-tyctxt query system against accidental recomputation

58782a8

oli-obk force-pushed the non_repeatable_queries branch from cf3b18b to 58782a8 Compare January 12, 2023 09:26

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 12, 2023

bors added the merged-by-bors This PR was explicitly merged by bors. label Jan 12, 2023

bors merged commit 222d1ff into rust-lang:master Jan 12, 2023

rustbot added this to the 1.68.0 milestone Jan 12, 2023

oli-obk deleted the non_repeatable_queries branch January 12, 2023 17:12

celinval mentioned this pull request Jan 24, 2023

Upgrade toolchain to nightly-2023-01-23 model-checking/kani#2149

Merged

4 tasks

matthiaskrgr mentioned this pull request Sep 13, 2023

ice: Failed to normalize Alias(Projection, AliasTy { #115808

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Harden the pre-tyctxt query system against accidental recomputation #105603

Harden the pre-tyctxt query system against accidental recomputation #105603

oli-obk commented Dec 12, 2022

rustbot commented Dec 12, 2022

rustbot commented Dec 12, 2022

oli-obk commented Dec 12, 2022

This comment has been minimized.

bors commented Dec 12, 2022

bors commented Dec 12, 2022

This comment has been minimized.

rust-timer commented Dec 12, 2022

oli-obk commented Jan 9, 2023

jyn514 left a comment

oli-obk commented Jan 11, 2023

oli-obk commented Jan 11, 2023

This comment has been minimized.

bors commented Jan 11, 2023

This comment has been minimized.

rust-timer commented Jan 11, 2023

petrochenkov commented Jan 11, 2023

oli-obk commented Jan 12, 2023

bors commented Jan 12, 2023

bors commented Jan 12, 2023

bors commented Jan 12, 2023

rust-timer commented Jan 12, 2023

Harden the pre-tyctxt query system against accidental recomputation #105603

Harden the pre-tyctxt query system against accidental recomputation #105603

Conversation

oli-obk commented Dec 12, 2022

rustbot commented Dec 12, 2022

rustbot commented Dec 12, 2022

oli-obk commented Dec 12, 2022

This comment has been minimized.

bors commented Dec 12, 2022

bors commented Dec 12, 2022

This comment has been minimized.

rust-timer commented Dec 12, 2022

Overall result: no relevant changes - no action needed

oli-obk commented Jan 9, 2023

jyn514 left a comment

Choose a reason for hiding this comment

oli-obk commented Jan 11, 2023

oli-obk commented Jan 11, 2023

This comment has been minimized.

bors commented Jan 11, 2023

This comment has been minimized.

rust-timer commented Jan 11, 2023

Overall result: ❌ regressions - no action needed

petrochenkov commented Jan 11, 2023

oli-obk commented Jan 12, 2023

bors commented Jan 12, 2023

bors commented Jan 12, 2023

bors commented Jan 12, 2023

rust-timer commented Jan 12, 2023

Overall result: no relevant changes - no action needed