Use correct scope when parametrizing tests based on duts/asics #4961
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of PR
Summary:
Fixes parametrization based on duts/asics occuring per collected test instead of once per module
Type of change
Back port request
Approach
What is the motivation for this PR?
Currently we are using enum fixtures for parametrization based on available duts/asics (e.g.
enum_dut_hostname
,enum_frontend_asic_index
). These fixtures are parametrized insidepytest_generate_tests
function usingmetafunc.parametrize
with scope='module'.If any test already has parametrization using @pytest.mark.parametrize, these enum fixtures will be recreated for each collected test, ignoring above mentioned scope.
This behavior can be easily noticed during copp/test_copp.py. Even on testbed with single DUT, examining setup/teardown during test shows that
enum_rand_one_per_hwsku_frontend_hostname
is recreated after each pair of tests, triggering teardown/setup of dependent fixtures. This adds ~40 minutes to test_copp.py execution time.How did you do it?
Added override for enum fixtures outside of
pytest_generate_tests
with correct scope.How did you verify/test it?
Run copp/test_copp.py - execution time decreased from ~55 minutes to ~15 minutes.
Note: tested only on single-DUT testbed.
Any platform specific information?
Supported testbed topology if it's a new test case?
Documentation