Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SDTEST-173] Early flake detection support for Cucumber #231

Merged
merged 13 commits into from
Sep 17, 2024

Conversation

anmarchenko
Copy link
Member

What does this PR do?
Adds early flake detection support for Cucumber

Additional Notes
To achieve this I had to:

  • refactor TestRetries::Component a bit: what was called Strategy before now called Driver; most of the component's logic was extracted to new Strategy classes
  • Cucumber test retries instrumentation has changed: instead of using built-in Cucumber retries, we use our own Filter to provide this functionality. Built-in cucumber retries don't support any use case except retrying failed tests.

How to test the change?
Unit tests are provided

@anmarchenko anmarchenko requested review from a team as code owners September 17, 2024 12:00
@anmarchenko anmarchenko changed the title [SDTEST-173] Early flaked detection support for Cucumber [SDTEST-173] Early flake detection support for Cucumber Sep 17, 2024
@anmarchenko anmarchenko added this to the 1.5.0 milestone Sep 17, 2024
@codecov-commenter
Copy link

codecov-commenter commented Sep 17, 2024

Codecov Report

Attention: Patch coverage is 97.42268% with 10 lines in your changes missing coverage. Please review.

Project coverage is 98.95%. Comparing base (c842a90) to head (cdd105b).
Report is 14 commits behind head on main.

Files with missing lines Patch % Lines
lib/datadog/ci/test_retries/strategy/retry_new.rb 88.52% 7 Missing ⚠️
lib/datadog/ci/contrib/cucumber/instrumentation.rb 87.50% 1 Missing ⚠️
lib/datadog/ci/test_retries/null_component.rb 66.66% 1 Missing ⚠️
...b/datadog/ci/test_retries/strategy/retry_failed.rb 94.73% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #231      +/-   ##
==========================================
+ Coverage   98.89%   98.95%   +0.05%     
==========================================
  Files         267      273       +6     
  Lines       13157    13285     +128     
  Branches      597      597              
==========================================
+ Hits        13012    13146     +134     
+ Misses        145      139       -6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@anmarchenko anmarchenko merged commit 1052c9c into main Sep 17, 2024
30 checks passed
@anmarchenko anmarchenko deleted the anmarchenko/refactor_test_retry_strategies branch September 17, 2024 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants