Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Leaf value collector #6795

Merged
merged 5 commits into from
Mar 5, 2024
Merged

Leaf value collector #6795

merged 5 commits into from
Mar 5, 2024

Conversation

asdacap
Copy link
Contributor

@asdacap asdacap commented Feb 29, 2024 •
edited
Loading

  • Collect leaf value to an arraypoollist of the desired types for snap server instead of encoding accounts to bytes first.
  • Increase snap server throughput by about 10-20%.
  • Total throughput right now + some optimization in halfpath is about 450k nodes writes per sec.

Types of changes

What types of changes does your code introduce?

  • Optimization

Testing

Requires testing

  • Yes
  • No

If yes, did you write tests?

  • Yes
  • No

Notes on testing

  • Full mainnet snap sync from one nethermind to another.
  • Hive tests

LukaszRozmej
LukaszRozmej reviewed Feb 29, 2024
View reviewed changes
src/Nethermind/Nethermind.Synchronization/SnapSync/AccountCollector.cs
Comment on lines +26 to +27
Account accnt = AccountDecoder.Instance.Decode(new RlpStream(value));
Accounts.Add(new PathWithAccount(path, accnt));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Check if we can have better performance with AccountStruct's?

src/Nethermind/Nethermind.Synchronization/SnapSync/PathWithStorageCollector.cs Show resolved Hide resolved
src/Nethermind/Nethermind.Synchronization/SnapSync/SnapServer.cs Outdated Show resolved Hide resolved
src/Nethermind/Nethermind.Synchronization/SnapSync/SnapServer.cs Outdated Show resolved Hide resolved
@asdacap asdacap merged commit bee4dde into master Mar 5, 2024
67 checks passed
@asdacap asdacap deleted the perf/snap-server-lower-allocation branch March 5, 2024 04:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@LukaszRozmej LukaszRozmej LukaszRozmej approved these changes

Assignees
No one assigned
Labels
performance
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@asdacap @LukaszRozmej