perf: Make ResetBatch accumulate a minimum batch size #729
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nodedb.go
Outdated
Comment on lines
330
to
331
| // just don't do an optimization here. keep letting batch size increase. | ||
| return nil |
There was a problem hiding this comment.
If GetByteSize fails you fall back to disabling batching. Would it be safer to fall back to the old behaviour, batch size of 1?
There was a problem hiding this comment.
I feel like disabling batching is fine, but happy to change to a batch size of 1 (really would just like this PR to get in)
Right now, the only way this function errors in cosmos-DB is if the batch is nil.
There was a problem hiding this comment.
lets fallback to old behaviour, then we can merge this
tac0turtle
approved these changes
May 10, 2023
|
@Mergifyio backport release/v0.20.x |
✅ Backports have been created
|
mergify bot
pushed a commit
that referenced
this pull request
May 10, 2023
Co-authored-by: Marko <marbar3778@yahoo.com> (cherry picked from commit 747bc12) # Conflicts: # CHANGELOG.md
|
@Mergifyio backport release/v0.21.x |
✅ Backports have been created
|
mergify bot
pushed a commit
that referenced
this pull request
May 30, 2023
Co-authored-by: Marko <marbar3778@yahoo.com> (cherry picked from commit 747bc12) # Conflicts: # CHANGELOG.md
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ResetBatch is an optimization that is used for genesis, for not accumulating huge batches for node writes.
While a performance win, right now it goes to far, with having batch writes of one. Having some minimum size greatly reduces time here.
Here is benchmarks for doing this with a large osmosis import genesis file, where I batched on batch size, for every 512 keys. (In this PR its in 64kb chunks, a little bit annoying for me to test with our osmosis fork due to version differences. I can do it if people really insist, but this feels like a pretty clear speedup, with a sufficiently close test)
Here is pictures of the pprof time diffs (with triggering based on key amount, rather than bytesize as in this PR).
As this is genesis only logic, and it directionally is a significant speedup, I suggest merging it, and leaving to future work to figure out how to tune / configure this batch size.
Before (153 seconds):
After (27 seconds -- its only the contribution from SaveBranch thats in the same comparison):
This is on top of significant GC pressure being reduced. (I don't have precise numbers for how much of a reduction this change caused, I'd have to re-benchmark)
Note that there is a commit done after this, so theres no concern with a final remnant batch not getting committed: https://github.com/cosmos/iavl/blob/master/mutable_tree.go#L712-L728