Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discard key versions during compaction #471

Closed
wants to merge 7 commits into from
Closed

Conversation

manishrjain
Copy link
Contributor

@manishrjain manishrjain commented May 1, 2018

  • In response to Discard invalid versions of keys during compaction #464 .
  • If a key version has deletion marker or is expired, we can safely drop all the older versions of that key.
  • If there's no overlap from lower levels, we can even drop the first such version.
  • To avoid an edge case bug, we need to ensure that all versions of the same key are contained by the same SSTable. This is reflected by not closing a table as long as more versions are found. And by picking key ranges to include all versions of the keys. Both these measures ensure that all versions at the same level get compacted together.
  • To avoid another edge case bug, we need to ensure that we don't drop versions above the current readTs. Note that managed DB would therefore not result in any version being discarded. Handling that is an open problem.

Badger Info:

  • Print out the key ranges for each SSTable via badger info.
  • Open an available port when running badger tool, so one can view the /debug/request and /debug/events links to understand what's going on behind the scenes.

This change is Reviewable

@manishrjain
Copy link
Contributor Author

Merged as e597fb7.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

1 participant