Adjust automation re: make_v2021_runoff_file
#185
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The relative path to the grid/ clone is different in Doug's test automation for the full test suite compared to what it is for just the test_daily_river_flows module. We're done with running the test for that module in isolation, so change the relative path to the value for the full test suite automation because it is how it will be used most often now.
This is the practical solution to ensuring that `make_v202111_runoff_file` runs after all the river discharge obs have been collected. We're forced into this because the race condition handling can't deal with ensuring that multiple instances of the same worker (`collect_river_data`) have completed. `make_runoff_file` and `make_v2021_runoff_file` are added to the `download_weather` race condition set to ensure that runoff forcing has been calculated before `upload_forcing` is launched after `grib_to_netcdf` in the early morning (06Z weather), or after `make_live_ocean_files` before the nowcast runs (12Z weather).
Having `make_ssh_files` (whose after function launches `make_runoff_file` and `make_v202111_runoff_file`) with those 2 workers results in an irresolvable condition for race condition management. The `make*_runoff_file` workers are now the end of one of the potential race condition paths.
Codecov Report
@@ Coverage Diff @@
## main #185 +/- ##
==========================================
- Coverage 77.42% 77.41% -0.02%
==========================================
Files 133 133
Lines 18491 18481 -10
Branches 1952 1953 +1
==========================================
- Hits 14317 14307 -10
Misses 4119 4119
Partials 55 55
Flags with carried forward coverage won't be shown. Click here to find out more.
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We need to ensure that
make_v202111_runoff_filerunsafter all the river discharge observations have been collected.
This PR implements a practical solution of launching
make_runoff_fileandmake_v2021_runoff_fileafter
make_ssh_files.We're forced into this solution because the race condition handling can't deal with ensuring that multiple
instances of the same worker (
collect_river_data) have completed.