feat: Dump retention managemnt #201
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Introduced retention parameters for the delete command. The delete command now supports the following retention parameters: - dry-run: Runs the deletion operation in test mode with verbose output, without actually deleting anything. - --before-date 2024-08-27T23:50:54+00:00: Deletes dumps older than the specified date. The date must be provided in RFC3339Nano format, for example: 2021-01-01T00:00:00Z. - --retain-recent 10: Retains the N most recent dumps, where N is specified by the user. - --retain-for 1w2d3h4m5s6ms7us8ns: Retains dumps for the specified duration. The format supports weeks (w), days (d), hours (h), minutes (m), seconds (s), milliseconds (ms), microseconds (us), and nanoseconds (ns). - --prune-failed: Prunes (removes) all dumps that have failed. - --prune-unsafe: Prunes dumps with "unknown-or-failed" statuses. This option only works in conjunction with --prune-failed. Refactored code and implemented helper functions to support the new features. Introduced two new statuses: `failed` and `in progress`. A dump is considered `failed` if it lacks a "done" heartbeat or if the last heartbeat timestamp exceeds 30 minutes. Heartbeats are recorded every 15 minutes by the dump command while it is in progress. If `greenmask` fails unexpectedly, the dump is marked as `failed` if the heartbeat is not finalized. The `in progress` status indicates that a dump is still in progress. The heartbeat mechanism determines dump status by storing a heartbeat file in the associated dump storage path, which is updated every 15 minutes. If the heartbeat is not updated for more than 30 minutes (twice the interval), the dump is classified as `failed`. Revised docs.
wwoytenko
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introduced retention parameters for the delete command. The delete command now supports the following retention parameters:
--dry-run: Runs the deletion operation in test mode with verbose output, without actually deleting anything.--before-date 2024-08-27T23:50:54+00:00: Deletes dumps older than the specified date. The date must be provided in RFC3339Nano format, for example: 2021-01-01T00:00:00Z.--retain-recent 10: Retains the N most recent dumps, where N is specified by the user.--retain-for 1w2d3h4m5s6ms7us8ns: Retains dumps for the specified duration. The format supports weeks (w), days (d), hours (h), minutes (m), seconds (s), milliseconds (ms), microseconds (us), and nanoseconds (ns).--prune-failed: Prunes (removes) all dumps that have failed.--prune-unsafe: Prunes dumps with "unknown-or-failed" statuses. This option only works in conjunction with--prune-failed.Refactored code and implemented helper functions to support the new features.
Introduced two new statuses:
failedandin progress. A dump is consideredfailedif it lacks a "done" heartbeat or if the last heartbeat timestamp exceeds 30 minutes. Heartbeats are recorded every 15 minutes by the dump command while it is in progress. Ifgreenmaskfails unexpectedly, the dump is marked asfailedif the heartbeat is not finalized. Thein progressstatus indicates that a dump is still in progress.The heartbeat mechanism determines dump status by storing a heartbeat file in the associated dump storage path, which is updated every 15 minutes. If the heartbeat is not updated for more than 30 minutes (twice the interval), the dump is classified as
failed.Revised docs.
Closes #142