Put okify results in unique subdirectories to avoid file overwrites #1456
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #1454
This PR modifies the organization of okify results uploaded to artifactory to avoid overwriting results if multiple tests fail which share the same name (such as parameterized tests). To provide an example, if test_rampfit_step is made to fail (by adding an
assert False
). 4 errors will be produced (from the 4 parametrizations of the test):https://github.com/spacetelescope/RegressionTests/actions/runs/11325106885/job/31501401451#step:30:1020
as these 4 tests share the same name the results will be put into the same directory on artifactory "test_rampfit_step":
https://bytesalad.stsci.edu/ui/repos/tree/General/roman-pipeline-results/2024-10-14_GITHUB_CI_Linux-X64-py3.11-1145/test_rampfit_step
This can result in files being overwritten (if they share the same name).
This PR prefixes the artifactory subdirectory with a unique number (the index of the test for the test run) to avoid file overwrites. Here's an example run forcing the same test to fail:
https://bytesalad.stsci.edu/ui/repos/tree/General/roman-pipeline-results/2024-10-14_GITHUB_CI_Linux-X64-py3.11-1147
showing the 4 test results split into 4 subdirectories.
I was able to run a 'dry-run' okify on the above example and it picked up the 4 test failures in the newly named subdirectories.
Regression tests running here: https://github.com/spacetelescope/RegressionTests/actions/runs/11331347662
Tasks
24Q4_B15
(use the latest build if not sure)no-changelog-entry-needed
)changes/
:echo "changed something" > changes/<PR#>.<changetype>.rst
(see below for change types)docs/
pageokify_regtests
to update the truth filesnews fragment change types...
changes/<PR#>.general.rst
: infrastructure or miscellaneous changechanges/<PR#>.docs.rst
changes/<PR#>.stpipe.rst
changes/<PR#>.associations.rst
changes/<PR#>.scripts.rst
changes/<PR#>.mosaic_pipeline.rst
changes/<PR#>.patch_match.rst
steps
changes/<PR#>.dq_init.rst
changes/<PR#>.saturation.rst
changes/<PR#>.refpix.rst
changes/<PR#>.linearity.rst
changes/<PR#>.dark_current.rst
changes/<PR#>.jump_detection.rst
changes/<PR#>.ramp_fitting.rst
changes/<PR#>.assign_wcs.rst
changes/<PR#>.flatfield.rst
changes/<PR#>.photom.rst
changes/<PR#>.flux.rst
changes/<PR#>.source_detection.rst
changes/<PR#>.tweakreg.rst
changes/<PR#>.skymatch.rst
changes/<PR#>.outlier_detection.rst
changes/<PR#>.resample.rst
changes/<PR#>.source_catalog.rst