-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Run the RFS Container's DocumentMigration application repeatedly as long as it's successful #1047
Run the RFS Container's DocumentMigration application repeatedly as long as it's successful #1047
Conversation
…ong it's successful. Unsuccessful means anything but a 0 exit code. The application has been cleaned up to return 3 for no work available. It used to return whatever a top-level exception would create. That part wasn't necessary, but it seemed like a good idea to test for that to make sure that after running repeatedly, processes would eventually do THAT instead of returning 0, causing infinite loops in the containers. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1047 +/- ##
============================================
- Coverage 80.57% 80.55% -0.02%
Complexity 2736 2736
============================================
Files 365 365
Lines 13611 13614 +3
Branches 941 941
============================================
Hits 10967 10967
- Misses 2067 2070 +3
Partials 577 577
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
…ess specification so that all 3rd party imports are in the same alphabetical block. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
…isk space. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
…ories Signed-off-by: Andre Kurait <akurait@amazon.com>
…tLoopOptimization Use sed for command line parsing in rfs entrypoint for cleanup directories
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. One question but I don't think it will substantively impact the application either way.
fi | ||
|
||
if [[ -n "$LUCENE_DIR" ]]; then | ||
echo "Will delete lucene local directory between runs: $LUCENE_DIR" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not a blocker, but - if we pre-delete the S3 directory (lines 52-23), why not pre-delete this one as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed lines 52-53. We'll run document migration once before deleting anything
Signed-off-by: Andre Kurait <akurait@amazon.com>
Remove s3 pre-delete
Unsuccessful means anything but a 0 exit code. The application has been cleaned up to return 3 for no work available. It used to return whatever a top-level exception would create. That part wasn't necessary, but it seemed like a good idea to test for that to make sure that after running repeatedly, processes would eventually do THAT instead of returning 0, causing infinite loops in the containers.
Description
This eliminates container startup costs while still cleaning out the java process so that we don't have to worry about runaway processes.
Issues Resolved
https://opensearch.atlassian.net/browse/MIGRATIONS-2042
Is this a backport? If so, please add backport PR # and/or commits #
Testing
Manual testing of the entrypoint.sh script
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.