Stop tracking of 2/3 of target models due to CI limitations #1044
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Due to our current CI limitations, our nightly jobs are mostly broken. As broken nightly isn't providing any value, our best effort is to lower the number of models we're targeting and focus on solving CI limitations. Main target should be to limit the number of variants and still keep plenty of unique architectures. That way, we'll not lower our model coverage quality even with 2/3 of variants removed as a target. In sum, majority of CI limitations come from limited disk space, host DRAM, etc. Fix #1043
vkovinicTT
approved these changes
Jan 14, 2025
mstojkovicTT
approved these changes
Jan 14, 2025
|
||||||||||||||
|
||||||||||||||
nvukobratTT
deleted the
nvukobrat/skip_model_tests_due_to_ci_limitations
branch
chandrasekaranpradeep
added a commit
that referenced
this pull request
Jan 17, 2025
…method in model analysis (#1060) In [this PR](#1044), we skipped some of the model variants due to CI limitation and those model variants will also be skipped in the model analysis pipeline. To avoid skipping the model variants in model analysis, added `--no-skips` argument in the subprocess run method in extraction and generation of unique ops config test function(i.e `generate_and_export_unique_ops_tests in scripts/model_analysis/unique_ops_utils.py`)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Due to our current CI limitations, our nightly jobs are mostly broken. As broken nightly isn't providing any value, our best effort is to lower the number of models we're targeting and focus on solving CI limitations.
Main target should be to limit the number of variants and still keep plenty of unique architectures. That way, we'll not lower our model coverage quality even with 2/3 of variants removed as a target.
In sum, majority of CI limitations come from limited disk space, host DRAM, etc.
Fix #1043