Fix dependency tracking for debugger visualizers #111641

michaelwoerister · 2023-05-16T11:54:10Z

This PR fixes dependency tracking for debugger visualizer files by changing the debugger_visualizers query to an eval_always query that scans the AST while it is still available. This way the set of visualizer files is already available when dep-info is emitted. Since the query is turned into an eval_always query, dependency tracking will now reliably detect changes to the visualizer script files themselves.

TODO:

perf.rlo
Needs a bit more documentation in some places
Needs regression test for the incr. comp. case

Fixes #111226
Fixes #111227
Fixes #111295

r? @wesleywiser
cc @gibbyfree

michaelwoerister · 2023-05-16T12:36:10Z

@bors try @rust-timer queue

bors · 2023-05-16T12:36:19Z

⌛ Trying commit 554f8e4a67a414bcabaa33283bdcc6ee4dc2f923 with merge d75ea5f73435c497a3cb2de433e97109163f94a1...

compiler/rustc_passes/src/debugger_visualizer.rs

compiler/rustc_span/src/lib.rs

compiler/rustc_passes/src/check_attr.rs

tests/run-make/debugger-visualizer-dep-info/Makefile

bors · 2023-05-16T14:46:05Z

☀️ Try build successful - checks-actions
Build commit: d75ea5f73435c497a3cb2de433e97109163f94a1 (d75ea5f73435c497a3cb2de433e97109163f94a1)

compiler/rustc_middle/src/hir/map/mod.rs

compiler/rustc_interface/src/passes.rs

compiler/rustc_span/src/lib.rs

rust-timer · 2023-05-16T16:05:47Z

Finished benchmarking commit (d75ea5f73435c497a3cb2de433e97109163f94a1): comparison URL.

Overall result: ❌ regressions - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.4%	[0.4%, 0.4%]	1
Regressions ❌ (secondary)	0.4%	[0.4%, 0.4%]	2
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.4%	[0.4%, 0.4%]	1

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.1%	[2.2%, 6.6%]	6
Improvements ✅ (primary)	-0.9%	[-0.9%, -0.9%]	1
Improvements ✅ (secondary)	-1.5%	[-1.5%, -1.5%]	1
All ❌✅ (primary)	-0.9%	[-0.9%, -0.9%]	1

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.9%	[1.9%, 5.0%]	16
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 641.487s -> 643.487s (0.31%)

michaelwoerister · 2023-05-16T19:08:24Z

Performance looks fine, I'd say. The only regressions come from AST-heavy incr-unchanged tests where it makes a difference that we are now doing the change-detection work that the compiler invalidly skipped before.

michaelwoerister · 2023-05-17T10:20:39Z

@rustbot ready

compiler/rustc_passes/src/debugger_visualizer.rs

cjgillot · 2023-05-17T12:30:49Z

compiler/rustc_passes/src/debugger_visualizer.rs

+/// Traverses and collects the debugger visualizers for a specific crate.
+fn debugger_visualizers(tcx: TyCtxt<'_>, _: LocalCrate) -> Vec<DebuggerVisualizerFile> {
+    let resolver_and_krate = tcx.resolver_for_lowering(()).borrow();
+    let krate = &*resolver_and_krate.1;


Why can't this be done on HIR, like it used to?

So that emitting only dep-info does not depend on HIR lowering. See this test for example.

…in deterministic order.

cjgillot · 2023-05-18T07:01:03Z

@bors r+

bors · 2023-05-18T07:01:06Z

📌 Commit 927e1ef has been approved by cjgillot

It is now in the queue for this repository.

bors · 2023-05-18T15:49:21Z

⌛ Testing commit 927e1ef with merge 2e9660f1bb00ee53638f73fe48cc8f8842daaf60...

bors · 2023-05-18T18:31:23Z

💔 Test failed - checks-actions

michaelwoerister · 2023-05-19T09:33:06Z

Let's see if that fixed the test for macOS.
@bors r=cjgillot

bors · 2023-05-19T09:33:08Z

📌 Commit 987655a has been approved by cjgillot

It is now in the queue for this repository.

bors · 2023-05-19T11:30:47Z

⌛ Testing commit 987655a with merge 17a6810...

bors · 2023-05-19T14:24:58Z

☀️ Test successful - checks-actions
Approved by: cjgillot
Pushing 17a6810 to master...

rust-timer · 2023-05-19T16:23:12Z

Finished benchmarking commit (17a6810): comparison URL.

Overall result: ❌ regressions - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.2%	[0.2%, 0.2%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.1%	[1.2%, 3.5%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.5%	[-1.7%, -1.3%]	2
All ❌✅ (primary)	-	-	0

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 640.861s -> 640.817s (-0.01%)

rustbot assigned wesleywiser May 16, 2023

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels May 16, 2023

michaelwoerister added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 16, 2023

This comment has been minimized.

Sign in to view

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels May 16, 2023

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 16, 2023

wesleywiser reviewed May 16, 2023

View reviewed changes

This comment has been minimized.

Sign in to view

cjgillot reviewed May 16, 2023

View reviewed changes

compiler/rustc_middle/src/hir/map/mod.rs Show resolved Hide resolved

compiler/rustc_interface/src/passes.rs Outdated Show resolved Hide resolved

compiler/rustc_span/src/lib.rs Outdated Show resolved Hide resolved

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 16, 2023

michaelwoerister added 2 commits May 16, 2023 18:50

Fix dependency tracking for debugger visualizers

7f01893

Turn debugger_visualizers from feed- into regular query.

a7d4880

michaelwoerister force-pushed the debugger-visualizer-fixes branch from 554f8e4 to a7d4880 Compare May 16, 2023 16:53

Move DebuggerVisualizerFile types from rustc_span to rustc_middle

d623668

michaelwoerister changed the title ~~(wip) Fix dependency tracking for debugger visualizers~~ Fix dependency tracking for debugger visualizers May 17, 2023

Add/improve tests for debugger_visualizer change detection.

cfca5b0

This comment has been minimized.

Sign in to view

michaelwoerister force-pushed the debugger-visualizer-fixes branch from 96a1854 to cfca5b0 Compare May 17, 2023 09:39

michaelwoerister marked this pull request as ready for review May 17, 2023 10:20

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 17, 2023

cjgillot reviewed May 17, 2023

View reviewed changes

compiler/rustc_passes/src/debugger_visualizer.rs Outdated Show resolved Hide resolved

cjgillot reviewed May 17, 2023

View reviewed changes

Don't sort output of debugger_visualizer query because it already is …

927e1ef

…in deterministic order.

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 18, 2023

apiraino mentioned this pull request May 18, 2023

Tracking Issue for debugger_visualizer #95939

Closed

5 tasks

This comment has been minimized.

Sign in to view

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels May 18, 2023

Fix run-make/inrcemental-debugger-visualizer test for macOS.

987655a

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 19, 2023

bors added the merged-by-bors This PR was explicitly merged by bors. label May 19, 2023

bors merged commit 17a6810 into rust-lang:master May 19, 2023

rustbot added this to the 1.71.0 milestone May 19, 2023

bors mentioned this pull request May 19, 2023

Write to stdout if - is given as output file #111626

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix dependency tracking for debugger visualizers #111641

Fix dependency tracking for debugger visualizers #111641

michaelwoerister commented May 16, 2023 •

edited

Loading

This comment has been minimized.

michaelwoerister commented May 16, 2023

This comment has been minimized.

bors commented May 16, 2023

bors commented May 16, 2023

This comment has been minimized.

rust-timer commented May 16, 2023

michaelwoerister commented May 16, 2023

This comment has been minimized.

michaelwoerister commented May 17, 2023

cjgillot May 17, 2023

michaelwoerister May 17, 2023

cjgillot commented May 18, 2023

bors commented May 18, 2023

bors commented May 18, 2023

This comment has been minimized.

bors commented May 18, 2023

michaelwoerister commented May 19, 2023

bors commented May 19, 2023

bors commented May 19, 2023

bors commented May 19, 2023

rust-timer commented May 19, 2023

Fix dependency tracking for debugger visualizers #111641

Fix dependency tracking for debugger visualizers #111641

Conversation

michaelwoerister commented May 16, 2023 • edited Loading

This comment has been minimized.

michaelwoerister commented May 16, 2023

This comment has been minimized.

bors commented May 16, 2023

bors commented May 16, 2023

This comment has been minimized.

rust-timer commented May 16, 2023

Overall result: ❌ regressions - no action needed

michaelwoerister commented May 16, 2023

This comment has been minimized.

michaelwoerister commented May 17, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cjgillot commented May 18, 2023

bors commented May 18, 2023

bors commented May 18, 2023

This comment has been minimized.

bors commented May 18, 2023

michaelwoerister commented May 19, 2023

bors commented May 19, 2023

bors commented May 19, 2023

bors commented May 19, 2023

rust-timer commented May 19, 2023

Overall result: ❌ regressions - no action needed

michaelwoerister commented May 16, 2023 •

edited

Loading