Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

snapshot: merge loops for better performance #22160

Merged
merged 1 commit into from
Jan 25, 2021
Merged

snapshot: merge loops for better performance #22160

merged 1 commit into from
Jan 25, 2021

Conversation

ehnuje
Copy link
Contributor

@ehnuje ehnuje commented Jan 12, 2021

  • data size check loop can be merged with sanity check routine.

@holiman
Copy link
Contributor

holiman commented Jan 12, 2021

I suspect that the intent was to eventually remove the panicky loops, hence why they're not already merged

@karalabe
Copy link
Member

Perhaps, but I do think we should leave the panics in. If we mess up something, we should really blow up as soon as possible. Without the panics, we could end up with a consensus split, which arguably is worse.

@ehnuje
Copy link
Contributor Author

ehnuje commented Jan 13, 2021

I agree with the point that @karalabe mentioned, since data corruption here is the worst case we don't want in snapshot and panic here helps revealing the underlying problem, if exists.

holiman
holiman approved these changes Jan 25, 2021
View reviewed changes
Copy link
Contributor

@holiman holiman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@holiman holiman merged commit 04a7226 into ethereum:master Jan 25, 2021
@holiman holiman added this to the 1.10.0 milestone Jan 25, 2021
bulgakovk pushed a commit to bulgakovk/go-ethereum that referenced this pull request Jan 26, 2021
@ehnuje @bulgakovk
snapshot: merge loops for better performance (ethereum#22160)
08f3463
@quorumbot quorumbot mentioned this pull request Sep 3, 2021
Merged
9 tasks
@baptiste-b-pegasys baptiste-b-pegasys mentioned this pull request Sep 3, 2021
Closed
9 tasks
tony-ricciardi pushed a commit to tony-ricciardi/go-ethereum that referenced this pull request Jan 20, 2022
@gastonponti @karalabe
@rjl493456442 @ehnuje @holiman @fxfactorial
Stabilize Snapshots (ethereum#1529)
77a4294 
Cherry pick bug fixes from upstream for snapshots, which will enable higher transaction throughput. It also enables snapshots by default (which is one of the commits pulled from upstream).

Upstream commits included:

68754f3 cmd/utils: grant snapshot cache to trie if disabled (ethereum#21416)
3ee91b9 core/state/snapshot: reduce disk layer depth during generation
a15d71a core/state/snapshot: stop generator if it hits missing trie nodes (ethereum#21649)
43c278c core/state: disable snapshot iteration if it's not fully constructed (ethereum#21682)
b63e3c3 core: improve snapshot journal recovery (ethereum#21594)
e640267 core/state/snapshot: fix journal recovery from generating old journal (ethereum#21775)
7b7b327 core/state/snapshot: update generator marker in sync with flushes
167ff56 core/state/snapshot: gethring -> gathering typo (ethereum#22104)
d2e1b17 snapshot, trie: fixed typos, mostly in snapshot pkg (ethereum#22133)
c4deebb core/state/snapshot: add generation logs to storage too
5e9f5ca core/state/snapshot: write snapshot generator in batch (ethereum#22163)
18145ad core/state: maintain one more diff layer (ethereum#21730)
04a7226 snapshot: merge loops for better performance (ethereum#22160)
994cdc6 cmd/utils: enable snapshots by default
9ec3329 core/state/snapshot: ensure Cap retains a min number of layers
52e5c38 core/state: copy the snap when copying the state (ethereum#22340)
a31f6d5 core/state/snapshot: fix panic on missing parent
61ff3e8 core/state/snapshot, ethdb: track deletions more accurately (ethereum#22582)
c79fc20 core/state/snapshot: fix data race in diff layer (ethereum#22540)

Other changes
Commit f9b5530 (not from upstream) fixes an incorrect default DatabaseCache value due to an earlier bad merge.

Tested
Automated tests
Testing on a private testnet
Backwards compatibility
Enabling snapshots by default is a breaking change in terms of the CLI flags, but will not cause backwards incompatibility between the node and other nodes.

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
Co-authored-by: gary rong <garyrong0905@gmail.com>
Co-authored-by: Melvin Junhee Woo <melvin.woo@groundx.xyz>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Edgar Aroutiounian <edgar.factorial@gmail.com>
This was referenced Sep 23, 2022
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@holiman holiman holiman approved these changes

@karalabe karalabe Awaiting requested review from karalabe karalabe is a code owner

@rjl493456442 rjl493456442 Awaiting requested review from rjl493456442 rjl493456442 is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
1.10.0
Development

Successfully merging this pull request may close these issues.
3 participants
@ehnuje @holiman @karalabe