coverage: Store BCB counter info externally, not directly in the BCB graph #114354

Zalathar · 2023-08-02T01:45:55Z

When deciding how to instrument the underlying MIR for coverage, the InstrumentCoverage pass builds a simplified “Basic Counter Block” graph, and then allocates coverage counters/expressions to various nodes/edges in the BCB graph as necessary. Those counters/expressions are then injected into the function's MIR.

The awkward thing here is that the code for doing this needs &mut access to the graph, in order to associate coverage info with individual nodes, even though it isn't making any structural changes to the graph itself. That makes it harder to understand and modify the instrumentation code.

In addition, the graph alone can't hold all the information that is needed. There ends up being an extra vector of “intermediate expressions” that needs to be passed around separately anyway.

This PR simplifies things by instead storing all of that temporary coverage information in a number of side-tables inside CoverageCounters.

This makes it easier to see all of the information produced by the make-counters step, and how it is used by the inject-into-mir step.

Looking at the combined changes is possible, but I recommend reviewing the commits individually, because the big changes are mostly independent of each other (despite being conceptually related).

rustbot · 2023-08-02T01:46:00Z

r? @jackh726

(rustbot has picked a reviewer for you, use r? to override)

Zalathar · 2023-08-02T01:46:02Z

This is one step in my larger coverage refactoring ambitions described at rust-lang/compiler-team#645.

rustbot · 2023-08-02T01:46:02Z

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

Zalathar · 2023-08-02T01:46:21Z

@rustbot label +A-code-coverage

compiler/rustc_mir_transform/src/coverage/mod.rs

Zalathar · 2023-08-03T02:01:12Z

compiler/rustc_mir_transform/src/coverage/counters.rs

+    /// Coverage counters/expressions that are associated with the control-flow
+    /// edge between two BCBs.
+    bcb_edge_counters: FxHashMap<(BasicCoverageBlock, BasicCoverageBlock), CoverageKind>,


In the old code, each BCB node had its own hashmap of incoming edge counters, indexed by the node that those edges come from.

I've changed that to a single hashmap whose keys are pairs of BCBs, representing an edge.

For most of the code, that change makes no big difference. But it does mean that it's no longer possible to quickly check whether a BCB node has any incoming edge counters, which is something that some debug assertions were relying on. To solve that, I've also added a separate bit set that tracks which BCBs have at least one incoming edge counter.

Zalathar · 2023-08-03T02:04:17Z

compiler/rustc_mir_transform/src/coverage/counters.rs

+    fn set_bcb_counter(
+        &mut self,
+        bcb: BasicCoverageBlock,
+        counter_kind: CoverageKind,
+    ) -> Result<Operand, Error> {


This method is mostly a direct copy of the old BasicCoverageBlockData::set_counter, which was moved here from graph.rs.

Zalathar · 2023-08-03T02:04:59Z

compiler/rustc_mir_transform/src/coverage/counters.rs

+    fn set_bcb_edge_counter(
+        &mut self,
+        from_bcb: BasicCoverageBlock,
+        to_bcb: BasicCoverageBlock,
+        counter_kind: CoverageKind,
+    ) -> Result<Operand, Error> {


This method is mostly a direct copy of the old BasicCoverageBlockData::set_edge_counter, which was moved here from graph.rs.

Zalathar · 2023-08-03T02:10:58Z

compiler/rustc_mir_transform/src/coverage/mod.rs

+        for ((from_bcb, target_bcb), counter_kind) in
+            self.coverage_counters.drain_bcb_edge_counters()
+        {
+            bcb_counters_without_direct_coverage_spans.push((
+                Some(from_bcb),
+                target_bcb,
+                counter_kind,
+            ));
        }


Because edge counters are now stored in a single hashmap (instead of one per BCB node), iterating over all of them is now its own top-level loop, rather than being nested in the per-BCB loop.

This avoids confusion with data structures that actually hold BCB counter information.

This avoids the need to pass around a separate vector to accumulate into, and avoids the need to create a fake empty vector when failure occurs.

Storing coverage counter information in `CoverageCounters` has a few advantages over storing it directly inside BCB graph nodes: - The graph doesn't need to be mutable when making the counters, making it easier to see that the graph itself is not modified during this step. - All of the counter data is clearly visible in one place. - It becomes possible to use a representation that doesn't correspond 1:1 to graph nodes, e.g. storing all the edge counters in a single hashmap instead of several.

jackh726 · 2023-08-13T16:38:38Z

compiler/rustc_mir_transform/src/coverage/tests.rs

@@ -676,10 +676,10 @@ fn test_make_bcb_counters() {
            }
        }
        let mut coverage_counters = counters::CoverageCounters::new(0);
-        let intermediate_expressions = coverage_counters
+        let () = coverage_counters


nit: this let () = isn't needed

Oh yeah, I remember doing this on purpose to verify that the success type was (), but in hindsight it doesn't really add anything.

I have some more planned changes that touch this file, so I'll get rid of the let () = in one of those.

jackh726 · 2023-08-13T18:26:26Z

@bors r+

bors · 2023-08-13T18:26:28Z

📌 Commit 5ca30c4 has been approved by jackh726

It is now in the queue for this repository.

…llaumeGomez Rollup of 5 pull requests Successful merges: - rust-lang#94667 (Add `Iterator::map_windows`) - rust-lang#114069 (Allow using external builds of the compiler-rt profile lib) - rust-lang#114354 (coverage: Store BCB counter info externally, not directly in the BCB graph) - rust-lang#114625 (CI: use smaller machines in PR runs) - rust-lang#114777 (Migrate GUI colors test to original CSS color format) r? `@ghost` `@rustbot` modify labels: rollup

bors · 2023-08-13T22:10:52Z

⌛ Testing commit 5ca30c4 with merge 1b198b3...

Having to keep passing in a graph reference was a holdover from when the graph was partly mutated during traversal. As of rust-lang#114354 that is no longer necessary, so we can simplify the traversal code by storing a graph reference as a field in `TraverseCoverageGraphWithLoops`.

rustbot assigned jackh726 Aug 2, 2023

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Aug 2, 2023

rustbot added the A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) label Aug 2, 2023

Zalathar changed the title ~~coverage: Accumulate counter info externally, not directly in the BCB graph~~ coverage: Store BCB counter info externally, not directly in the BCB graph Aug 2, 2023

Zalathar mentioned this pull request Aug 2, 2023

Changes to StatementKind::Coverage rust-lang/compiler-team#645

Closed

3 tasks

Swatinem approved these changes Aug 2, 2023

View reviewed changes

compiler/rustc_mir_transform/src/coverage/mod.rs Outdated Show resolved Hide resolved

Zalathar force-pushed the external-counters branch 2 times, most recently from ae30452 to b7488e7 Compare August 3, 2023 01:44

Zalathar commented Aug 3, 2023

View reviewed changes

Zalathar force-pushed the external-counters branch from b7488e7 to cd8955a Compare August 4, 2023 01:03

Zalathar force-pushed the external-counters branch 3 times, most recently from 19ec657 to 96450ed Compare August 12, 2023 02:26

Zalathar added 3 commits August 13, 2023 12:18

Rename helper struct BcbCounters to MakeBcbCounters

c74db79

This avoids confusion with data structures that actually hold BCB counter information.

Accumulate intermediate expressions into CoverageCounters

5302c9d

This avoids the need to pass around a separate vector to accumulate into, and avoids the need to create a fake empty vector when failure occurs.

Zalathar force-pushed the external-counters branch from 96450ed to 5ca30c4 Compare August 13, 2023 02:20

jackh726 reviewed Aug 13, 2023

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 13, 2023

GuillaumeGomez mentioned this pull request Aug 13, 2023

Rollup of 5 pull requests #114786

Merged

bors merged commit 2442c9b into rust-lang:master Aug 13, 2023
11 checks passed

rustbot added this to the 1.73.0 milestone Aug 13, 2023

Zalathar deleted the external-counters branch August 13, 2023 23:39

This was referenced Oct 30, 2023

Update some outdated descriptions of coverage instrumentation rust-lang/rustc-dev-guide#1809

Merged

coverage: Replace impossible coverage::Error with assertions #117421

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

coverage: Store BCB counter info externally, not directly in the BCB graph #114354

coverage: Store BCB counter info externally, not directly in the BCB graph #114354

Zalathar commented Aug 2, 2023 •

edited

Loading

rustbot commented Aug 2, 2023

Zalathar commented Aug 2, 2023

rustbot commented Aug 2, 2023

Zalathar commented Aug 2, 2023

Zalathar Aug 3, 2023

Zalathar Aug 3, 2023

Zalathar Aug 3, 2023

Zalathar Aug 3, 2023

jackh726 Aug 13, 2023

Zalathar Aug 13, 2023

jackh726 commented Aug 13, 2023

bors commented Aug 13, 2023

bors commented Aug 13, 2023

coverage: Store BCB counter info externally, not directly in the BCB graph #114354

coverage: Store BCB counter info externally, not directly in the BCB graph #114354

Conversation

Zalathar commented Aug 2, 2023 • edited Loading

rustbot commented Aug 2, 2023

Zalathar commented Aug 2, 2023

rustbot commented Aug 2, 2023

Zalathar commented Aug 2, 2023

Zalathar Aug 3, 2023

Choose a reason for hiding this comment

Zalathar Aug 3, 2023

Choose a reason for hiding this comment

Zalathar Aug 3, 2023

Choose a reason for hiding this comment

Zalathar Aug 3, 2023

Choose a reason for hiding this comment

jackh726 Aug 13, 2023

Choose a reason for hiding this comment

Zalathar Aug 13, 2023

Choose a reason for hiding this comment

jackh726 commented Aug 13, 2023

bors commented Aug 13, 2023

bors commented Aug 13, 2023

Zalathar commented Aug 2, 2023 •

edited

Loading