Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

perf: decrease max memory and increase speed, during chain root mmr migration #3664

Merged
merged 1 commit into from
Oct 27, 2022
Merged

perf: decrease max memory and increase speed, during chain root mmr migration #3664

merged 1 commit into from
Oct 27, 2022

Conversation

yangby-cryptape
Copy link
Collaborator

@yangby-cryptape yangby-cryptape commented Oct 26, 2022 •
edited
Loading

What problem does this PR solve?

Decrease max memory and increase speed, during chain root MMR migration.

What is changed and how it works?

Create a new DB transaction for every 10000 records inserted.

On an AWS 4C 8G instance, with latest testnet which tip number is 7139044:

  • Reduce the time for chain root MMR migration from 15 minutes to 12 minutes.
  • Reduce the max memory during chain root MMR migration from 6.5 GiB to 900 MiB.

p.s. These data has been tested 3 times.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code ci-runs-only: [ quick_checks,linters ]

Release note

Title Only: Include only the PR title in the release note.

@yangby-cryptape yangby-cryptape marked this pull request as ready for review October 26, 2022 09:45
@yangby-cryptape yangby-cryptape requested a review from a team as a code owner October 26, 2022 09:45
@yangby-cryptape yangby-cryptape requested review from doitian and removed request for a team October 26, 2022 09:45
@zhangsoledad
Copy link
Member

bors r+

@bors
Copy link
Contributor

bors bot commented Oct 27, 2022

Build succeeded:

@bors bors bot merged commit d8f0343 into nervosnetwork:develop Oct 27, 2022
@yangby-cryptape yangby-cryptape deleted the pr/perf-migration branch February 27, 2023 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhangsoledad zhangsoledad zhangsoledad approved these changes

@doitian doitian Awaiting requested review from doitian doitian is a code owner automatically assigned from nervosnetwork/ckb-dev

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@yangby-cryptape @zhangsoledad