Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use directly leveldb batch #1507

Merged
merged 1 commit into from
Jul 1, 2020

Conversation

manish-sethi
Copy link
Contributor

Signed-off-by: manish manish.sethi@gmail.com

Type of change

  • Improvement (improvement to code, performance, etc)

Description

This PR improves the leveldbhelper.UpdateBatch by replacing the map that it maintains by the actual leveldb batch.
This reduces the memory consumed and a sorting of key-values is avoided if the data added is already in sorted order (such as loading the data from snapshot files)

This commit improves the leveldbhelper.UpdateBatch by
replacing the map that it maintains by the actual leveldb batch.

This reduces the memory consumed and a sorting of key-values is avoided
if the data added is already in sorted order (such as loading the data
from snapshot files)

Signed-off-by: manish <manish.sethi@gmail.com>
@manish-sethi manish-sethi requested a review from a team as a code owner June 30, 2020 21:59
// WriteBatch writes a batch in an atomic way
func (h *DBHandle) WriteBatch(batch *UpdateBatch, sync bool) error {
if len(batch.KVs) == 0 {
if batch == nil || batch.Len() == 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not related to this PR. I was checking whether leveldb.WriteBatch returns err when the batch is nil (it does). You might be aware of the following. Then, I found that they use transaction (OpenTransaction(), Commit()) when the batch size is large to skip writing to journal. I remember once you said that we need to look at disabling the journal writes as we have a block store.

@cendhu cendhu merged commit 1ad6d5b into hyperledger:master Jul 1, 2020
@manish-sethi manish-sethi deleted the fix_leveldb_batch branch July 1, 2020 01:57
@cendhu
Copy link
Contributor

cendhu commented Sep 8, 2020

@Mergifyio backport release-2.2

@mergify
Copy link

mergify bot commented Sep 8, 2020

Command backport release-2.2: success

Backports have been created

mergify bot pushed a commit that referenced this pull request Sep 8, 2020
This commit improves the leveldbhelper.UpdateBatch by
replacing the map that it maintains by the actual leveldb batch.

This reduces the memory consumed and a sorting of key-values is avoided
if the data added is already in sorted order (such as loading the data
from snapshot files)

Signed-off-by: manish <manish.sethi@gmail.com>
(cherry picked from commit 1ad6d5b)
cendhu pushed a commit to cendhu/fabric that referenced this pull request Sep 8, 2020
This commit improves the leveldbhelper.UpdateBatch by
replacing the map that it maintains by the actual leveldb batch.

This reduces the memory consumed and a sorting of key-values is avoided
if the data added is already in sorted order (such as loading the data
from snapshot files)

Signed-off-by: manish <manish.sethi@gmail.com>
ale-linux pushed a commit that referenced this pull request Sep 8, 2020
This commit improves the leveldbhelper.UpdateBatch by
replacing the map that it maintains by the actual leveldb batch.

This reduces the memory consumed and a sorting of key-values is avoided
if the data added is already in sorted order (such as loading the data
from snapshot files)

Signed-off-by: manish <manish.sethi@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants