Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix split post-processing of LLVM-based coverage #18634

Closed
wants to merge 1 commit into from

Conversation

fmeum
Copy link
Collaborator

@fmeum fmeum commented Jun 11, 2023

--experimental_split_coverage_postprocessing now works in combination with --experimental_generate_llvm_lcov, which required adding the runtime objects as inputs to the separate coverage post-processing spawn. Previously, they were only added to the runfiles of the test action, which aren't (and shouldn't) be added to the coverage spawn.

As a side effect, this provides a more natural fix for #15121 than bb6f1a7, which added an additional check to only add those runtime objects to the llvm-cov command line that were present in runfiles. Now, since all runtime objects are staged as inputs, they are known to be available. This improves coverage accuracy when e.g. runtime objects are packaged into jars and loaded at runtime.

Along the way the common setup for LLVM-based coverage tests has been extracted into a helper function and the tests have been updated to work with a broader range of clang versions. Previously, tests weren't run at all due to the clang version on the runners not falling into the restrictive range. Now, the tests are executed on the Ubuntu 20.04 runner (but not on any other runners, including macOS).

@fmeum fmeum changed the title TMP: Test sandboxed llvm-cov Fix split post-processing of LLVM-based coverage Jun 11, 2023
@fmeum fmeum force-pushed the fix-sandboxed-llvm-cov branch 8 times, most recently from 166de10 to bb6706f Compare June 11, 2023 19:58
@fmeum fmeum force-pushed the fix-sandboxed-llvm-cov branch 4 times, most recently from d166f8c to 54bd646 Compare June 12, 2023 13:07
`--experimental_split_coverage_postprocessing` now works in combination
with `--experimental_generate_llvm_lcov`, which required adding the
runtime objects as inputs to the separate coverage post-processing
spawn. Previously, they were only added to the runfiles of the test
action, which aren't (and shouldn't) be added to the coverage spawn.

As a side effect, this provides a more natural fix for bazelbuild#15121 than
bb6f1a7, which added an additional
check to only add those runtime objects to the `llvm-cov` command line
that were present in runfiles. Now, since all runtime objects are staged
as inputs, they are known to be available. This improves coverage
accuracy when e.g. runtime objects are packaged into jars and loaded at
runtime.
@fmeum fmeum requested a review from oquenchil June 12, 2023 13:23
@fmeum fmeum marked this pull request as ready for review June 12, 2023 13:23
@fmeum fmeum requested review from a team and lberki as code owners June 12, 2023 13:23
@fmeum fmeum removed request for a team and lberki June 12, 2023 13:23
@github-actions github-actions bot added awaiting-review PR is awaiting review from an assigned reviewer team-Remote-Exec Issues and PRs for the Execution (Remote) team labels Jun 12, 2023
@fmeum
Copy link
Collaborator Author

fmeum commented Jun 12, 2023

@oquenchil Could you review this PR?

@UebelAndre noticed the problem when trying to implement support for --experimental_split_coverage_postprocessing in rules_rust.

@fmeum
Copy link
Collaborator Author

fmeum commented Jun 12, 2023

@UebelAndre With this fix, would rules_rust still need access to the private metadata_files parameter of coverage_common.instrumented_files_info?

@UebelAndre
Copy link
Contributor

@UebelAndre With this fix, would rules_rust still need access to the private metadata_files parameter of coverage_common.instrumented_files_info?

I don’t think it would be necessary but we are currently adding llvm-profdata and llvm-cov to runfiles of the test, it feels like it’d be nice to be able to separate those out. But again, not necessary and we should be fine with this fix.

@UebelAndre
Copy link
Contributor

@oquenchil friendly ping. It'd be great if this could get in for the 6.3 release.

@oquenchil oquenchil requested review from c-mita and removed request for oquenchil June 16, 2023 14:56
@UebelAndre
Copy link
Contributor

@c-mita another friendly ping. I'm really hoping to have this in for 6.3 to fix both llvm based Cc coverage and to support Rust coverage (which uses llvm).

Copy link
Member

@c-mita c-mita left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@UebelAndre
Copy link
Contributor

@c-mita Awesome! Anything else blocking the merge?

@sgowroji
Copy link
Member

Hi @c-mita, Since I can see that this PR has been approved, please let me know whether I should proceed with importing it.

@fmeum
Copy link
Collaborator Author

fmeum commented Jun 20, 2023

@bazel-io flag

@bazel-io bazel-io added the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Jun 20, 2023
@c-mita c-mita added awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally and removed awaiting-review PR is awaiting review from an assigned reviewer labels Jun 20, 2023
@c-mita
Copy link
Member

c-mita commented Jun 20, 2023

Hi @c-mita, Since I can see that this PR has been approved, please let me know whether I should proceed with importing it.

Yes please; forgot to set the tags again...

@UebelAndre
Copy link
Contributor

@c-mita @sgowroji are there tags needed to add this to the 6.3 release?

@sgowroji
Copy link
Member

@bazel-io fork 6.3.0

@bazel-io bazel-io removed the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Jun 20, 2023
@github-actions github-actions bot removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label Jun 20, 2023
iancha1992 pushed a commit to iancha1992/bazel that referenced this pull request Jun 21, 2023
`--experimental_split_coverage_postprocessing` now works in combination with `--experimental_generate_llvm_lcov`, which required adding the runtime objects as inputs to the separate coverage post-processing spawn. Previously, they were only added to the runfiles of the test action, which aren't (and shouldn't) be added to the coverage spawn.

As a side effect, this provides a more natural fix for bazelbuild#15121 than bb6f1a7, which added an additional check to only add those runtime objects to the `llvm-cov` command line that were present in runfiles. Now, since all runtime objects are staged as inputs, they are known to be available. This improves coverage accuracy when e.g. runtime objects are packaged into jars and loaded at runtime.

Along the way the common setup for LLVM-based coverage tests has been extracted into a helper function and the tests have been updated to work with a broader range of clang versions. Previously, tests weren't run at all due to the clang version on the runners not falling into the restrictive range. Now, the tests are executed on the Ubuntu 20.04 runner (but not on any other runners, including macOS).

Closes bazelbuild#18634.

PiperOrigin-RevId: 542055078
Change-Id: Id8851f8685bb7724891acca395bb91556ee2b29a
@fmeum fmeum deleted the fix-sandboxed-llvm-cov branch June 21, 2023 17:45
iancha1992 added a commit that referenced this pull request Jun 21, 2023
`--experimental_split_coverage_postprocessing` now works in combination with `--experimental_generate_llvm_lcov`, which required adding the runtime objects as inputs to the separate coverage post-processing spawn. Previously, they were only added to the runfiles of the test action, which aren't (and shouldn't) be added to the coverage spawn.

As a side effect, this provides a more natural fix for #15121 than bb6f1a7, which added an additional check to only add those runtime objects to the `llvm-cov` command line that were present in runfiles. Now, since all runtime objects are staged as inputs, they are known to be available. This improves coverage accuracy when e.g. runtime objects are packaged into jars and loaded at runtime.

Along the way the common setup for LLVM-based coverage tests has been extracted into a helper function and the tests have been updated to work with a broader range of clang versions. Previously, tests weren't run at all due to the clang version on the runners not falling into the restrictive range. Now, the tests are executed on the Ubuntu 20.04 runner (but not on any other runners, including macOS).

Closes #18634.

PiperOrigin-RevId: 542055078
Change-Id: Id8851f8685bb7724891acca395bb91556ee2b29a

Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-Remote-Exec Issues and PRs for the Execution (Remote) team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants