Skip to content
This repository has been archived by the owner on Oct 9, 2023. It is now read-only.

Fix CI and attempt to improve flaky tests #1394

Merged
merged 20 commits into from
Jul 18, 2022

Conversation

krshrimali
Copy link
Contributor

@krshrimali krshrimali commented Jul 15, 2022

What does this PR do?

This PR was an attempt to fix the CI, and it turned out to be successful. :) Here is the summary of what's done:

  1. In the style_transfer.py script, the dataset we download contains an image with one number of channels (data/coco128/images/train2017/000000000086.jpg) - and pystiche raised an error while trying to normalize. It looks like the image wasn't converted to RGB through our loader, and that is fixed in this PR.
  2. test_speed seems to be flaky for Windows/Mac, and we are trying to be softer in this PR. Let's investigate again if they appear to be flaky in the future.
  3. Minor changes in the example scripts, like changing batch_size to match input samples, and reducing epochs to 1.

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests? [not needed for typos/docs]
  • Did you verify new and existing tests pass locally with your changes?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

  • Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@codecov
Copy link

codecov bot commented Jul 15, 2022

Codecov Report

Merging #1394 (520fea6) into master (3e4c8bb) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #1394   +/-   ##
=======================================
  Coverage   92.91%   92.91%           
=======================================
  Files         285      286    +1     
  Lines       12778    12835   +57     
=======================================
+ Hits        11873    11926   +53     
- Misses        905      909    +4     
Flag Coverage Δ
unittests 92.91% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
flash/core/data/utilities/loading.py 98.52% <100.00%> (+1.42%) ⬆️
flash/text/question_answering/model.py 93.87% <0.00%> (-0.69%) ⬇️
flash/core/utilities/embedder.py 91.48% <0.00%> (ø)
flash/core/model.py 91.42% <0.00%> (+0.23%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3e4c8bb...520fea6. Read the comment docs.

@Borda Borda added the bug / fix Something isn't working label Jul 15, 2022
@krshrimali krshrimali marked this pull request as draft July 15, 2022 08:49
@krshrimali krshrimali marked this pull request as ready for review July 18, 2022 11:33
flash_examples/image_embedder.py Outdated Show resolved Hide resolved
flash_examples/image_embedder.py Outdated Show resolved Hide resolved
flash_examples/style_transfer.py Outdated Show resolved Hide resolved
requirements/datatype_image_extras.txt Outdated Show resolved Hide resolved
@krshrimali krshrimali changed the title Attempt to fix CI Fix CI and attempt to improve flaky tests Jul 18, 2022
@krshrimali
Copy link
Contributor Author

Once and if all the tests pass, I'll update the CHANGELOG. I don't want to trigger the CI again when it's not worth it. :)

Copy link
Collaborator

@ethanwharris ethanwharris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! LGTM 😃

@krshrimali krshrimali enabled auto-merge (squash) July 18, 2022 12:27
@krshrimali krshrimali disabled auto-merge July 18, 2022 13:18
CHANGELOG.md Outdated Show resolved Hide resolved
.azure-pipelines/testing-template.yml Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
flash_examples/image_embedder.py Outdated Show resolved Hide resolved
tests/examples/utils.py Outdated Show resolved Hide resolved
@ethanwharris ethanwharris merged commit 04f1b8f into master Jul 18, 2022
@ethanwharris ethanwharris deleted the ci/fix_examples_third_party branch July 18, 2022 17:28
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug / fix Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants