Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[move-compiler] Fix panic in coverage for inlined code, add controls to display tags and/or color in coverage source listings, don't lean on compiler v1 #14449

Merged
merged 7 commits into from
Aug 29, 2024

Conversation

brmataptos
Copy link
Contributor

@brmataptos brmataptos commented Aug 28, 2024

Description

Add controls for how aptos move coverage source displays covered source code:

  • --color {none, default, always}, by default keep current behavior
  • --tag {none, explicit, on}, when turned explicit or on, show + and - for covered/uncovered lines

Fix a panic for aptos move coverage source --module features on
aptos-move/framework/move-stdlib, caused by inlined lines being
misattributed to the calling module.

Also fix aptos move test --coverage to not rely on V1 compiler.

Issue #14448 created to track remaining coverage issues.

Fixes #14377
Fixes #14360
Fixes #14379
Fixes #14454

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Other (specify)

How Has This Been Tested?

Manual testing, piped to files and less, etc.

Key Areas to Review

Uncolored output of aptos move coverage source --module features now looks like:

Code coverage per line of code:
  + indicates the line is not executable or is fully covered during execution
  - indicates the line is executable but NOT fully covered during execution
Source code follows:
+ /// Defines feature flags for Aptos. Those are used in Aptos specific implementations of features in
+ /// the Move stdlib, the Aptos stdlib, and the Aptos framework.
+ ///
...
+     const TREAT_FRIEND_AS_PRIVATE: u64 = 2;
+ 
-     public fun treat_friend_as_private(): bool acquires Features {
-         is_enabled(TREAT_FRIEND_AS_PRIVATE)
+     }
+ 
...
+     }
+ }
{
  "Result": "Success"
}

With color, lines are green with uncovered parts in red.

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented Aug 28, 2024

⏱️ 6h 11m total CI duration on this PR
Job Cumulative Duration Recent Runs
execution-performance / single-node-performance 46m 🟩🟩
forge-compat-test / forge 46m 🟥🟩
forge-e2e-test / forge 45m 🟥🟩
forge-framework-upgrade-test / forge 33m 🟩
general-lints 15m 🟩🟩🟩🟩 (+4 more)
rust-move-unit-coverage 15m 🟩
rust-cargo-deny 14m 🟩🟩🟩🟩 (+4 more)
rust-move-tests 9m 🟩
rust-move-tests 9m 🟩
rust-move-tests 9m 🟩
rust-move-tests 9m 🟩
rust-move-tests 9m 🟩
check-dynamic-deps 9m 🟩🟩🟩🟩🟩 (+4 more)
execution-performance / test-target-determinator 9m 🟩🟩
test-target-determinator 9m 🟩🟩
rust-move-tests 8m 🟩
rust-move-tests 8m 🟩
rust-move-tests 7m
check 7m 🟩🟩
rust-move-unit-coverage 7m 🟩
rust-move-unit-coverage 7m 🟩
rust-move-unit-coverage 7m 🟩
rust-doc-tests 6m 🟩
rust-move-unit-coverage 5m 🟥
rust-doc-tests 4m 🟩
semgrep/ci 4m 🟩🟩🟩🟩🟩 (+4 more)
rust-move-unit-coverage 3m 🟥
rust-move-unit-coverage 3m 🟥
rust-move-unit-coverage 3m 🟥
file_change_determinator 2m 🟩🟩🟩🟩🟩 (+4 more)
file_change_determinator 2m 🟩🟩🟩🟩🟩 (+4 more)
permission-check 29s 🟩🟩🟩🟩🟩 (+4 more)
permission-check 25s 🟩🟩🟩🟩🟩 (+4 more)
permission-check 24s 🟩🟩🟩🟩🟩 (+4 more)
permission-check 23s 🟩🟩🟩🟩🟩 (+4 more)
file_change_determinator 22s 🟩🟩
rust-move-tests 14s
Backport PR 9s 🟥
determine-docker-build-metadata 7s 🟩🟩
permission-check 4s 🟩🟩
permission-check 2s 🟩

🚨 1 job on the last run was significantly faster/slower than expected

Job Duration vs 7d avg Delta
forge-framework-upgrade-test / forge 18m 12m +42%

settingsfeedbackdocs ⋅ learn more about trunk.io

@brmataptos brmataptos changed the title Fix panic in coverage for inlined code, add controls to display tags and/or color in coverage source linstings [move-compiler] Fix panic in coverage for inlined code, add controls to display tags and/or color in coverage source listings, don't lean on compiler v1 Aug 29, 2024
Copy link
Contributor

@wrwg wrwg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

if cov.uncovered_locations.is_empty() {
return vec![];
}

let mut covs: Vec<_> = cov
.uncovered_locations
.iter()
.filter(|loc| loc.file_hash() == file_hash)
Copy link
Contributor

@vineethk vineethk Aug 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change introduces a new crash: #14454

It can be fixed by adding a check that if covs.is_empty() { return vec![]; } right after creating covs here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had filed the crash while reviewing, but I did not realize until after filing the bug that the root cause was this change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you have tested the scenario mentioned in the issue and it is fixed, then you can mention in the PR description that it also fixes that issue so that it gets closed.

install_dir: move_options.output_dir.clone(),
skip_fetch_latest_git_deps: move_options.skip_fetch_latest_git_deps,
compiler_config: CompilerConfig {
known_attributes: extended_checks::get_all_attribute_names().clone(),
skip_attribute_checks: false,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should skip_attribute_checks also not be copied from move_options?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

source_coverage
.compute_source_coverage(source_path)
.output_source_coverage(&mut std::io::stdout())
let t1 = source_coverage.compute_source_coverage(source_path);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please undo or use better var names.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

source_cov
.compute_source_coverage(source_path)
.output_source_coverage(&mut coverage_writer)
let t1 = source_cov.compute_source_coverage(source_path);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please undo or use better var names.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}

#[derive(ValueEnum, Clone, Debug, Serialize)]
pub enum TextIndicator {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to have a brief note on what these mean.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link

codecov bot commented Aug 29, 2024

Codecov Report

Attention: Patch coverage is 13.08411% with 93 lines in your changes missing coverage. Please review.

Project coverage is 59.6%. Comparing base (b3ab3d7) to head (394145a).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
...ty/move/tools/move-coverage/src/source_coverage.rs 10.8% 82 Missing ⚠️
...ird_party/move/tools/move-cli/src/base/coverage.rs 25.0% 6 Missing ⚠️
...ove/tools/move-coverage/src/bin/source-coverage.rs 28.5% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##             main   #14449     +/-   ##
=========================================
- Coverage    59.7%    59.6%   -0.1%     
=========================================
  Files         844      844             
  Lines      205737   205834     +97     
=========================================
+ Hits       122831   122845     +14     
- Misses      82906    82989     +83     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@vineethk vineethk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have one comment about checking whether an issue is fixed and marking it as so, approving assuming that will be done.

@brmataptos brmataptos enabled auto-merge (squash) August 29, 2024 17:33

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite framework_upgrade success on d1bf834728a0cf166d993f4728dfca54f3086fb0 ==> 394145acf8d4429f589a6ffb5bec44a8708a04c2

Compatibility test results for d1bf834728a0cf166d993f4728dfca54f3086fb0 ==> 394145acf8d4429f589a6ffb5bec44a8708a04c2 (PR)
Upgrade the nodes to version: 394145acf8d4429f589a6ffb5bec44a8708a04c2
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 650.69 txn/s, submitted: 652.12 txn/s, failed submission: 1.43 txn/s, expired: 1.43 txn/s, latency: 5040.19 ms, (p50: 5500 ms, p90: 7500 ms, p99: 8800 ms), latency samples: 54740
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1042.62 txn/s, submitted: 1044.69 txn/s, failed submission: 2.07 txn/s, expired: 2.07 txn/s, latency: 3002.90 ms, (p50: 2300 ms, p90: 6100 ms, p99: 7500 ms), latency samples: 90560
5. check swarm health
Compatibility test for d1bf834728a0cf166d993f4728dfca54f3086fb0 ==> 394145acf8d4429f589a6ffb5bec44a8708a04c2 passed
Upgrade the remaining nodes to version: 394145acf8d4429f589a6ffb5bec44a8708a04c2
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1118.34 txn/s, submitted: 1121.89 txn/s, failed submission: 3.55 txn/s, expired: 3.55 txn/s, latency: 2705.57 ms, (p50: 2400 ms, p90: 4800 ms, p99: 7200 ms), latency samples: 100680
Test Ok

This comment has been minimized.

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 394145acf8d4429f589a6ffb5bec44a8708a04c2

two traffics test: inner traffic : committed: 12297.64 txn/s, latency: 3274.53 ms, (p50: 2700 ms, p90: 3600 ms, p99: 11900 ms), latency samples: 4675820
two traffics test : committed: 99.89 txn/s, latency: 2657.42 ms, (p50: 2400 ms, p90: 2800 ms, p99: 10300 ms), latency samples: 1820
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.233, avg: 0.219", "QsPosToProposal: max: 0.407, avg: 0.212", "ConsensusProposalToOrdered: max: 0.326, avg: 0.298", "ConsensusOrderedToCommit: max: 0.551, avg: 0.473", "ConsensusProposalToCommit: max: 0.858, avg: 0.771"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.90s no progress at version 1368925 (avg 0.21s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 7.80s no progress at version 1368923 (avg 7.59s) [limit 15].
Test Ok

This comment has been minimized.

Copy link
Contributor

✅ Forge suite compat success on d1bf834728a0cf166d993f4728dfca54f3086fb0 ==> 394145acf8d4429f589a6ffb5bec44a8708a04c2

Compatibility test results for d1bf834728a0cf166d993f4728dfca54f3086fb0 ==> 394145acf8d4429f589a6ffb5bec44a8708a04c2 (PR)
1. Check liveness of validators at old version: d1bf834728a0cf166d993f4728dfca54f3086fb0
compatibility::simple-validator-upgrade::liveness-check : committed: 13166.97 txn/s, latency: 2530.73 ms, (p50: 2400 ms, p90: 4200 ms, p99: 4900 ms), latency samples: 436400
2. Upgrading first Validator to new version: 394145acf8d4429f589a6ffb5bec44a8708a04c2
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 5553.81 txn/s, latency: 4696.73 ms, (p50: 4700 ms, p90: 6500 ms, p99: 8700 ms), latency samples: 117920
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 6732.66 txn/s, latency: 4729.47 ms, (p50: 4800 ms, p90: 7000 ms, p99: 7200 ms), latency samples: 233620
3. Upgrading rest of first batch to new version: 394145acf8d4429f589a6ffb5bec44a8708a04c2
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 7467.54 txn/s, latency: 3854.82 ms, (p50: 4300 ms, p90: 4700 ms, p99: 4800 ms), latency samples: 140080
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 7420.97 txn/s, latency: 4372.28 ms, (p50: 4700 ms, p90: 5000 ms, p99: 5200 ms), latency samples: 242560
4. upgrading second batch to new version: 394145acf8d4429f589a6ffb5bec44a8708a04c2
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 10967.35 txn/s, latency: 2462.34 ms, (p50: 2700 ms, p90: 2900 ms, p99: 3100 ms), latency samples: 192700
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 11136.48 txn/s, latency: 2826.71 ms, (p50: 2700 ms, p90: 3800 ms, p99: 4800 ms), latency samples: 359980
5. check swarm health
Compatibility test for d1bf834728a0cf166d993f4728dfca54f3086fb0 ==> 394145acf8d4429f589a6ffb5bec44a8708a04c2 passed
Test Ok

@brmataptos brmataptos merged commit 5eb0f1d into main Aug 29, 2024
48 of 51 checks passed
@brmataptos brmataptos deleted the brm-issue-14360 branch August 29, 2024 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants