-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore(memtable): refactor code for memtable flush #1866
Conversation
a1e71f3
to
8200cf4
Compare
87f02a8
to
f084d1a
Compare
8200cf4
to
f837fb9
Compare
f084d1a
to
257895a
Compare
f837fb9
to
c2e9075
Compare
4b5225f
to
839781e
Compare
c2e9075
to
2fae4ee
Compare
839781e
to
a476694
Compare
2fae4ee
to
60b31d5
Compare
These are the places where we are sending the memtable to the
The only place this code will be useful when we open and close the DB. If the DB is closed and opened a few times in short duration, it may end up collecting a few small sized memtables and will require compaction in L0 -> L1 (or higher) unnecessarily. This will increase write amplification unnecessarily. I think this code only helps us, doesn't harm us in any way. |
7e82b6c
to
b0dbb49
Compare
464b124
to
ff60a72
Compare
ff60a72
to
162725e
Compare
162725e
to
72a81ff
Compare
72a81ff
to
248292f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- CI is failing on TestPenultimateMemCorruption (link)
- Please add a description to the PR explaining why this change is useful / necessary (I see your comment makes some of this clear, but it should also be in the description)
- This change was based on changes that were in another PR in Badger, this should be referenced
- We should benchmark the performance benefits of this change. Currently our assumption is this won't do any harm and may have some benefits when opening/closing the DB, but we should be able to quantify this benefit. Adding a benchmark would also be useful for future changes.
Looks like it is based off #1696 - is this change necessary to bring in the handover skiplist feature? |
=> looking at the failure
=> updated the description
=> It is not anymore. Earlier it was using some changes from another commit but I realized that we don't want those changes either because it was doubling the size of L0 sstables to twice of the expected size.
=> We are only making structural changes to the code, the logic is the same as it was before. I don't expect the performance to change at all. |
This change is not based off of #1696 any more. That is how the change started but I have removed the code to club together memtables. |
248292f
to
9ea992a
Compare
Removes the
flushTask
struct that was used in two different context before. Once place when we were sending the memtable for flushing it onto the disk after it becomes full, and other context is when we were sending the memtable to be written onto disk during flush.