Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved flags fixturing for for repository unit tests #10190

Merged
merged 9 commits into from
May 21, 2024

Conversation

QMalcolm
Copy link
Contributor

resolves #8627

Problem

Our flags are a global. Although hopefully they won't be soon 🤞 Regardless because of their global nature, setting the global flags in one unit test, affected other unit tests. This meant either any unit test had to clean up the global flags at the end OR every test had to reset the global flags at start. The latter was more common because it's easier to code defensively than hope any upstream test appropriately cleans up after itself.

Additionally we didn't have a "standard" way of setting up flags for tests.

Solution

  • Create a fixture which sets and tears down the global flags for every pytest unit test
  • Create an intentionally overridable fixture for any args a test may want to have in its flags

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX
  • This PR includes type annotations for new and modified functions

QMalcolm added 8 commits May 20, 2024 19:23
…nit tests

In the previous commit we added a pytest fixture which sets and tears down
the global flags arg via `set_from_args` for every pytest based unit test.
Previously we had added a `set_from_args` in tests or test files to reset
the global flags from if they were modified by a previous test. This is no
longer necessary because of the work done in the previous commit.

Note: We did not modify any tests that use the `unittest.TestCase` class
because they don't use pytest fixtures. Thus those tests need to continue
operating as they currently do until we shift them to pytest unit tests.
We did this so in the next commit we can drop the unnecessary `set_from_args`
in the next commit. That will be it's own commit because converting these
tests is a restructuring that doing separately makes things easier to follow.
That is to say, all changes in this commit were just to convert the tests to
pytest, no other changes were made.
@QMalcolm QMalcolm added the Skip Changelog Skips GHA to check for changelog file label May 21, 2024
@QMalcolm QMalcolm requested a review from a team as a code owner May 21, 2024 02:47
@cla-bot cla-bot bot added the cla:yes label May 21, 2024
Copy link

codecov bot commented May 21, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.63%. Comparing base (341803d) to head (62a66d3).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #10190      +/-   ##
==========================================
- Coverage   88.65%   88.63%   -0.03%     
==========================================
  Files         180      180              
  Lines       22422    22434      +12     
==========================================
+ Hits        19879    19885       +6     
- Misses       2543     2549       +6     
Flag Coverage Δ
integration 86.08% <ø> (-0.06%) ⬇️
unit 63.00% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

In python 3.9 `Generator` was moved to `collections.abc` and deprecated
in `typing`. We still support 3.8 and thus need to be conditionally
importing `Generator`. We should remove this in the future when we drop
support for 3.8.
@QMalcolm
Copy link
Contributor Author

QMalcolm commented May 21, 2024

At the time of this comment it appears that Tests and Code Checks / (5) integration test/ python 3.11 / ubuntu-20.04 is In progress. However if you click on that action, you can see that it actually has finished. This is because there is an active GHA incident . All other GHA based checks have reported as Successful for this PR, going to move forward with merging.

@QMalcolm QMalcolm merged commit 09243d1 into main May 21, 2024
60 checks passed
@QMalcolm QMalcolm deleted the qmalcolm--8627-improved-flags-fixturing-for-unit-tests branch May 21, 2024 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla:yes Skip Changelog Skips GHA to check for changelog file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CT-3111] [Tech debt] Create setup fixture for setting flags in unit tests
2 participants