Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement exact same CLI test filtering as in Catch2 #113

Closed
cschreib opened this issue Aug 25, 2023 · 0 comments · Fixed by #111
Closed

Implement exact same CLI test filtering as in Catch2 #113

cschreib opened this issue Aug 25, 2023 · 0 comments · Fixed by #111
Labels
enhancement New feature or request
Milestone

Comments

@cschreib
Copy link
Member

cschreib commented Aug 25, 2023

Currently, we aren't implementing the same CLI test filtering API as Catch2. Some of it is missing features, some of it is that we are implementing what the Catch2 docs say, but that is not what Catch2 does...

  • For tags, "[tag1][tag2]" in Catch2 means "select tests which have both [tag1] AND [tag2] specified". Currently in snitch this filter is never a match. This is a missing feature that is documented in Catch2.
  • For names and tags, "<filter1>,<filter2>" in Catch2 means "select tests which match EITHER <filter1> OR <filter2>". Currently in snitch this filter is just interpreted literally, and will try to match a name with a comma in it, so unlikely to ever be a match. This is not a feature fully specified in the docs, it is specified as if it applies to tags only, but in reality it applies to both tags and names.
  • As a result of the above, Catch2 needs to escape the character ,, which we currently don't.
  • For names and tags, "<filter1>" "<filter2>" in Catch2 means "select tests which match BOTH <filter1> AND <filter2>". Currently in snitch we use OR (somewhat), which is what the Catch2 docs implied, but their implementation does the opposite.
  • Hidden tests in Catch2 are only selected if positively matched, not from a negative exclusion. Currently in snitch they are treated like any test, and are only hidden in the default (unfiltered) run. The Catch2 docs specify this new behavior (which was introduced in v3.x, and we match the behavior from v2.x).
@cschreib cschreib added the enhancement New feature or request label Aug 25, 2023
@cschreib cschreib added this to the v1.2 milestone Aug 25, 2023
@cschreib cschreib changed the title Implement exact same CLI test filtering as in *Catch2* Implement exact same CLI test filtering as in Catch2 Aug 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant