-
Notifications
You must be signed in to change notification settings - Fork 42
add in testrunner support for @skip tag - we need to be able to skip a test scenario (or whole file) #490
Comments
@mdelapenya hi - this doesn't sound too hard, I'd love to know more about the testrunner flow we use here. I poked into the repo and didn't discover it... if you wish, you could point me in the right direction and I could try to push it on. |
Godog supports filtering the execution by tags (See https://github.com/cucumber/godog#tags), and we are leveraging this capability in the automation in place to run thee tests. We can filter by tags in two places:
... TAGS="fleet_mode && ~skip"...
- suite: "fleet"
tags: "agent_endpoint_integration && ~skip"
- suite: "fleet"
tags: "stand_alone_mode && ~skip"
- suite: "fleet"
tags: "fleet_mode && ~skip" Both tags will be evaluated in the automation and put in the godog context for tags. An example: SUITE="fleet" \
TAGS="fleet_mode && ~skip" DEVELOPER_MODE=true \
TIMEOUT_FACTOR=3 LOG_LEVEL=TRACE \
make -C e2e functional-test |
This issue is really related to #373: I'd like to refactor the names making them more reusable, also involving product management team in defining priorities. |
I don't disagree this is related, it surely is - I have a pr that has the skip functionality implemented for CI as needed, so I submit we can close this. and leave 373 open to discuss the refactor of tags as a separate topic. If you agree @mdelapenya pls close, or we can chat it. |
per is merged, closing this as well |
This is from discussion with Manu off and on over a few months, we never added test runner support so here is a ticket to do the work. hopefully small-ish effort.
A/C:
A file's main tag can be added to with @Skip and the whole file will be skipped
An individual scenario's tagging can be added to with @Skip and the scenario will be skipped and reported as such
The text was updated successfully, but these errors were encountered: