Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-11775. Add tool to create RocksDB checkpoint #7664

Merged
merged 10 commits into from
Jan 19, 2025

Conversation

Tejaskriya
Copy link
Contributor

@Tejaskriya Tejaskriya commented Jan 8, 2025

What changes were proposed in this pull request?

Some ozone debug tools may require a consistent view into the RocksDB instance to function correctly and may give incorrect results if the DB is being used by a running OM. To fix this, we can have ozone debug operate on a RocksDB checkpoint if the Ozone process is running. This avoids users having to manually install RocksDB's ldb and make the checkpoint themselves. An optional flag --checkpoint in introduced, that instructs the command to create a checkpoint from the RocksDB instance provided and read that instead.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-11775

How was this patch tested?

Added a robot test in ozone-debug-ldb.

@errose28 errose28 added the tools Tools that helps with debugging label Jan 8, 2025
@errose28
Copy link
Contributor

errose28 commented Jan 8, 2025

Thanks for adding this @Tejaskriya. I wonder if it would be better to have checkpoint as its own command? This would provide a few potential benefits:

  • Ability to take a DB checkpoint for uses outside the LDB tools itself
  • No need to add --checkpoint to every other command that may need it (FSO repair in debug mode comes to mind)
  • User explicitly provides a file name so they know where the checkpoint is and that it is safe to remove. Terminal output tends to get lost with history.

I'm thinking something like ozone debug ldb checkpoint --db=<path-to-db> --output=<new-db-dir>

@Tejaskriya
Copy link
Contributor Author

@errose28 That does sound like a better idea. I'll update this PR with the new approach soon.

@Tejaskriya Tejaskriya marked this pull request as ready for review January 15, 2025 06:46
@adoroszlai adoroszlai changed the title HDDS-11775. Allow Ozone debug tools that need to read RocksDB to automatically create a checkpoint HDDS-11775. Add tool to create RocksDB checkpoint Jan 15, 2025
Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Tejaskriya for the patch. LGTM, some code reuse suggested.

@Tejaskriya
Copy link
Contributor Author

Tejaskriya commented Jan 15, 2025

Thanks for the review @adoroszlai, I have made the changes
@errose28 could you please review the updated patch too?

Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Tejaskriya for updating the patch.

@adoroszlai adoroszlai merged commit 7239594 into apache:master Jan 19, 2025
42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tools Tools that helps with debugging
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants