Move sync archive update into normal task schedule #211
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This makes the sync archive update step run sequentially after the main archive update. This is better than separate jobs since it is robust when the main archive update takes longer than expected.
From some related investigation, it appears the time from comm EOT to L0 decom products appearing in the archive is less than 2 hours. EOT is very rarely past 2am, so we can do:
I looked into the cheta_sync code and it updates the archfiles.db3 (the master index file for a content type) last, so that I don't see an easy way for user fetch queries to fail even if done right in the middle of an update. Of course that doesn't mean it won't happen, so we'll revisit the cheru update timing if needed.
Testing
I don't see how to test on the side without just installing it. There is not much opportunity for this to fail except for file permissions.
We'll need to do a
chmod -R g+w /proj/sot/ska/data/eng_archive/sync
before promoting this.