Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

core/state/snapshot: faster snapshot generation #22504

Merged
merged 75 commits into from
Apr 14, 2021
Merged

core/state/snapshot: faster snapshot generation #22504

merged 75 commits into from
Apr 14, 2021

Commits on Apr 9, 2021

  1. eth/protocols: persist received state segments

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    4c2c35f View commit details
    Browse the repository at this point in the history

  2. core: initial implementation

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    4a7650f View commit details
    Browse the repository at this point in the history

  3. core/state/snapshot: add tests

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    2b65233 View commit details
    Browse the repository at this point in the history

  4. core, eth: updates

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    d74a7f1 View commit details
    Browse the repository at this point in the history

  5. eth/protocols/snapshot: count flat state size

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    5a26f7f View commit details
    Browse the repository at this point in the history

  6. core/state: add metrics

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    d21e5a2 View commit details
    Browse the repository at this point in the history

  7. core/state/snapshot: skip unnecessary deletion

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    65cee53 View commit details
    Browse the repository at this point in the history

  8. core/state/snapshot: rename

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    557b81c View commit details
    Browse the repository at this point in the history

  9. core/state/snapshot: use the global batch

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    45fead8 View commit details
    Browse the repository at this point in the history

  10. core/state/snapshot: add logs and fix wiping

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    a9bea12 View commit details
    Browse the repository at this point in the history

  11. core/state/snapshot: fix

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    310db7c View commit details
    Browse the repository at this point in the history

  12. core/state/snapshot: save generation progress even if the batch is empty

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    6102044 View commit details
    Browse the repository at this point in the history

  13. core/state/snapshot: fixes

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    dd4ced3 View commit details
    Browse the repository at this point in the history

  14. core/state/snapshot: fix initial account range length

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    5eb07d4 View commit details
    Browse the repository at this point in the history

  15. core/state/snapshot: fix initial account range

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    62eb856 View commit details
    Browse the repository at this point in the history

  16. eth/protocols/snap: store flat states during the healing

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    b2dbe3b View commit details
    Browse the repository at this point in the history

  17. eth/protocols/snap: print logs

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    529147f View commit details
    Browse the repository at this point in the history

  18. core/state/snapshot: refactor (#4) 

    * core/state/snapshot: refactor

* core/state/snapshot: tiny fix and polish

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
    @holiman @rjl493456442
    holiman and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    da8f26b View commit details
    Browse the repository at this point in the history

  19. core, eth: fixes

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    7433151 View commit details
    Browse the repository at this point in the history

  20. core, eth: fix healing writer

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    7073457 View commit details
    Browse the repository at this point in the history

  21. core, trie, eth: fix paths

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    70c9b32 View commit details
    Browse the repository at this point in the history

  22. eth/protocols/snap: fix encoding

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    d865c47 View commit details
    Browse the repository at this point in the history

  23. eth, core: add debug log

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    6e94768 View commit details
    Browse the repository at this point in the history

  24. core/state/generate: release iterator asap (#5) 

    core/state/snapshot: less copy

core/state/snapshot: revert split loop

core/state/snapshot: handle storage becoming empty, improve test robustness

core/state: test modified codehash

core/state/snapshot: polish
    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    afecd67 View commit details
    Browse the repository at this point in the history

  25. core/state/snapshot: optimize stats counter

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    9cf7677 View commit details
    Browse the repository at this point in the history

  26. core, eth: add metric

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    13d4b9b View commit details
    Browse the repository at this point in the history

  27. core/state/snapshot: update comments

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    0cceb3c View commit details
    Browse the repository at this point in the history

  28. core/state/snapshot: improve tests

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    1f05908 View commit details
    Browse the repository at this point in the history

  29. core/state/snapshot: replace secure trie with standard trie

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    07e3575 View commit details
    Browse the repository at this point in the history

  30. core/state/snapshot: wrap return as the struct

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    1d3517e View commit details
    Browse the repository at this point in the history

  31. core/state/snapshot: skip wiping correct states

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    7c8e43c View commit details
    Browse the repository at this point in the history

  32. core/state/snapshot: updates

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    4539a8f View commit details
    Browse the repository at this point in the history

  33. core/state/snapshot: fixes

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    240e9d6 View commit details
    Browse the repository at this point in the history

  34. core/state/snapshot: fix panic due to reference flaw in closure

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    2fd4ce2 View commit details
    Browse the repository at this point in the history

  35. core/state/snapshot: fix errors in state generation logic + fix log o… 

    …utput
    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    b221f1a View commit details
    Browse the repository at this point in the history

  36. core/state/snapshot: remove an error case

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    6c93d5a View commit details
    Browse the repository at this point in the history

  37. core/state/snapshot: fix condition-check for exhausted snap state

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    2cf335c View commit details
    Browse the repository at this point in the history

  38. core/state/snapshot: use stackTrie for small tries

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    f25852e View commit details
    Browse the repository at this point in the history

  39. core/state/snapshot: don't resolve small storage tries in vain

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    dab43b3 View commit details
    Browse the repository at this point in the history

  40. core/state/snapshot: properly clean up storage of deleted accounts

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    2fce60d View commit details
    Browse the repository at this point in the history

  41. core/state/snapshot: avoid RLP-encoding in some cases + minor nitpicks

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    148bde6 View commit details
    Browse the repository at this point in the history

  42. core/state/snapshot: fix error (+testcase)

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    c24ef63 View commit details
    Browse the repository at this point in the history

  43. core/state/snapshot: clean up tests a bit

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    2625274 View commit details
    Browse the repository at this point in the history

  44. core/state/snapshot: work in progress on better tests

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    ac645a1 View commit details
    Browse the repository at this point in the history

  45. core/state/snapshot: polish code

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    823d0c5 View commit details
    Browse the repository at this point in the history

  46. core/state/snapshot: fix trie iteration abortion trigger

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    5a36506 View commit details
    Browse the repository at this point in the history

  47. core/state/snapshot: fixes flaws

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    664615f View commit details
    Browse the repository at this point in the history

  48. core/state/snapshot: remove panic

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    95118e7 View commit details
    Browse the repository at this point in the history

  49. core/state/snapshot: fix abort

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    bba0f1c View commit details
    Browse the repository at this point in the history

  50. core/state/snapshot: more tests (plus failing testcase)

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    3c64389 View commit details
    Browse the repository at this point in the history

  51. core/state/snapshot: more testcases + fix for failing test

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    47bc0ef View commit details
    Browse the repository at this point in the history

  52. core/state/snapshot: testcase for malformed data

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    71ea5ad View commit details
    Browse the repository at this point in the history

  53. core/state/snapshot: some test nitpicks

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    63f4998 View commit details
    Browse the repository at this point in the history

  54. core/state/snapshot: improvements to logging

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    ac8ebf3 View commit details
    Browse the repository at this point in the history

  55. core/state/snapshot: testcase to demo error in abortion

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    fda1c2a View commit details
    Browse the repository at this point in the history

  56. core/state/snapshot: fix abortion

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    f58f3c7 View commit details
    Browse the repository at this point in the history

  57. cmd/geth: make verify-state report the root

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    ea540aa View commit details
    Browse the repository at this point in the history

  58. trie: fix failing test

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    37b3426 View commit details
    Browse the repository at this point in the history

  59. core/state/snapshot: add timer metrics

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    d172f80 View commit details
    Browse the repository at this point in the history

  60. core/state/snapshot: fix metrics

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    3c9073c View commit details
    Browse the repository at this point in the history

  61. core/state/snapshot: udpate tests

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    1e42318 View commit details
    Browse the repository at this point in the history

  62. eth/protocols/snap: write snapshot account even if code or state is n… 

    …eeded
    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    4563da2 View commit details
    Browse the repository at this point in the history

  63. core/state/snapshot: fix diskmore check

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    c78f9f4 View commit details
    Browse the repository at this point in the history

  64. core/state/snapshot: review fixes

    @holiman @rjl493456442
    holiman authored and rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    c27ea4f View commit details
    Browse the repository at this point in the history

  65. core/state/snapshot: improve error message

    @rjl493456442
    rjl493456442 committed Apr 9, 2021
    Configuration menu
    Copy the full SHA
    46fc218 View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2021

  1. cmd/geth: rename 'error' to 'err' in logs

    @holiman
    holiman committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    bb149d9 View commit details
    Browse the repository at this point in the history

  2. core/state/snapshot: fix some review concerns

    @holiman
    holiman committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    78cdadd View commit details
    Browse the repository at this point in the history

  3. core/state/snapshot, eth/protocols/snap: clear snapshot marker when s… 

    …tarting/resuming snap sync
    @holiman
    holiman committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    1693ca3 View commit details
    Browse the repository at this point in the history

  4. core: add error log

    @rjl493456442
    rjl493456442 committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    c2e7952 View commit details
    Browse the repository at this point in the history

  5. core/state/snapshot: use proper timers for metrics collection

    @holiman
    holiman committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    1cc3ca3 View commit details
    Browse the repository at this point in the history

  6. core/state/snapshot: address some review concerns

    @holiman
    holiman committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    1a6143f View commit details
    Browse the repository at this point in the history

  7. eth/protocols/snap: improved log message

    @holiman
    holiman committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    920e2eb View commit details
    Browse the repository at this point in the history

  8. eth/protocols/snap: fix heal logs to condense infos

    @karalabe
    karalabe committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    fa8b467 View commit details
    Browse the repository at this point in the history

  9. core/state/snapshot: wait for generator termination before restarting

    @karalabe
    karalabe committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    3a54830 View commit details
    Browse the repository at this point in the history

  10. core/state/snapshot: revert timers to counters to track total time

    @karalabe
    karalabe committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    12a1a85 View commit details
    Browse the repository at this point in the history