Skip to content
This repository has been archived by the owner on Oct 9, 2023. It is now read-only.

Skip core tests when not core testing #1330

Merged
merged 8 commits into from
May 6, 2022
Merged

Conversation

ethanwharris
Copy link
Collaborator

@ethanwharris ethanwharris commented May 5, 2022

What does this PR do?

Currently, all CI workflows run all of the tests that do not have specific requirements ('core' tests).
WIth this PR, the core tests will only run in core workflows, speeding up the CI and reducing issues created by test flakiness.
This PR additionally filters some serve tests into their own workflow too.

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests? [not needed for typos/docs]
  • Did you verify new and existing tests pass locally with your changes?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

  • Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@ethanwharris ethanwharris added the tests / CI Tests and continuous integration label May 5, 2022
@ethanwharris ethanwharris added this to the 0.7.x milestone May 5, 2022
@codecov
Copy link

codecov bot commented May 5, 2022

Codecov Report

Merging #1330 (3a3797e) into master (3faf9f6) will increase coverage by 9.82%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1330      +/-   ##
==========================================
+ Coverage   81.65%   91.48%   +9.82%     
==========================================
  Files         287      287              
  Lines       12846    12881      +35     
==========================================
+ Hits        10490    11784    +1294     
+ Misses       2356     1097    -1259     
Flag Coverage Δ
unittests 91.48% <100.00%> (+9.82%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
flash/core/serve/utils.py 100.00% <ø> (+3.44%) ⬆️
flash/core/data/data_module.py 94.23% <100.00%> (+0.48%) ⬆️
flash/core/data/utilities/classification.py 98.27% <100.00%> (+1.19%) ⬆️
flash/core/data/utils.py 89.47% <100.00%> (+0.28%) ⬆️
flash/core/model.py 88.91% <100.00%> (+0.05%) ⬆️
flash/core/optimizers/lamb.py 84.93% <100.00%> (+0.64%) ⬆️
flash/core/optimizers/lars.py 82.75% <100.00%> (+0.94%) ⬆️
flash/core/optimizers/lr_scheduler.py 93.75% <100.00%> (+0.64%) ⬆️
flash/core/serve/dag/optimization.py 93.77% <100.00%> (-0.59%) ⬇️
flash/core/serve/dag/order.py 96.40% <100.00%> (-0.37%) ⬇️
... and 86 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3faf9f6...3a3797e. Read the comment docs.

@ethanwharris ethanwharris changed the title [WIP] Skip core tests when not core testing Skip core tests when not core testing May 5, 2022
Copy link
Contributor

@krshrimali krshrimali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! 🚀 LGTM 😃

@ethanwharris ethanwharris merged commit 07d63e3 into master May 6, 2022
@ethanwharris ethanwharris deleted the tests/isolate_core_tests branch May 6, 2022 09:31
ethanwharris added a commit that referenced this pull request May 11, 2022
ethanwharris added a commit that referenced this pull request May 11, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
tests / CI Tests and continuous integration
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants