Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added --skip-benchmarks command-line option. #2408

Merged

Conversation

danifeist
Copy link
Contributor

This PR adds the feature requested by #2392 "Command line option to skip benchmarks" by introducing the command-line option "--skip-benchmarks" so that Catch runs no benchmarks when this option is set. I'd be glad about feedback.

@codecov
Copy link

codecov bot commented Apr 11, 2022

Codecov Report

Merging #2408 (468f80e) into devel (f526ff0) will increase coverage by 0.00%.
The diff coverage is 100.00%.

❗ Current head 468f80e differs from pull request most recent head fe44137. Consider uploading reports for the commit fe44137 to get more accurate results

@@           Coverage Diff           @@
##            devel    #2408   +/-   ##
=======================================
  Coverage   91.13%   91.13%           
=======================================
  Files         157      157           
  Lines        7426     7430    +4     
=======================================
+ Hits         6767     6771    +4     
  Misses        659      659           

Copy link
Member

@horenmar horenmar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also needs tests.

It should be enough to pick some TEST_CASE that's already in the SelfTest project and has benchmarks, and then add new CTest test that invokes SelfTest with --skip-benchmarks and test spec for that one test. The test then checks that the benchmark was skipped by looking for the benchmark output.

Or add a new TEST_CASE just for this, that way is more resilient to later changes.

docs/command-line.md Show resolved Hide resolved
docs/command-line.md Outdated Show resolved Hide resolved
src/catch2/benchmark/catch_benchmark.hpp Show resolved Hide resolved
danifeist and others added 6 commits April 23, 2022 12:47
- Added version placeholder in documentation
- Improved description of as suggested
The new CTest runs a dedicated TEST_CASE with the --skip-benchmarks option.
The TEST_CASE can only succeed when the two benchmarks inside are skipped.
@horenmar
Copy link
Member

Thanks.

@horenmar horenmar merged commit 291b35b into catchorg:devel Apr 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants