Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(continuations): Improve initializations and reduce redundant computations #429

Merged
merged 158 commits into from
Aug 1, 2024

Conversation

Nashtare
Copy link
Collaborator

Savings on average over v0.5.0 move from ~8%/10% from late_hash_continuations branch, PR #402, to ~15% / 16% with this branch.

Also cleans up / refactors code a bit.
I would ideally move all the segmentation related stuff into a dedicated module, but leaving it for now as to not affect other PRs.

@Nashtare Nashtare added the performance Performance improvement related changes label Jul 23, 2024
@Nashtare Nashtare added this to the zk-continuations - Q3 2024 milestone Jul 23, 2024
@Nashtare Nashtare requested a review from LindaGuiga July 23, 2024 14:34
@Nashtare Nashtare requested a review from muursh as a code owner July 23, 2024 14:34
@github-actions github-actions bot added crate: trace_decoder Anything related to the trace_decoder crate. crate: proof_gen Anything related to the proof_gen crate. crate: evm_arithmetization Anything related to the evm_arithmetization crate. crate: zero_bin Anything related to the zero-bin subcrates. labels Jul 23, 2024
zero_bin/ops/src/lib.rs Outdated Show resolved Hide resolved
@Nashtare Nashtare self-assigned this Jul 24, 2024
.env Outdated Show resolved Hide resolved
Copy link
Contributor

@LindaGuiga LindaGuiga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

evm_arithmetization/src/prover.rs Outdated Show resolved Hide resolved
zero_bin/ops/src/lib.rs Outdated Show resolved Hide resolved
Base automatically changed from late_hash_continuations to feat/continuations August 1, 2024 14:39
@Nashtare Nashtare merged commit b0cda17 into feat/continuations Aug 1, 2024
13 of 14 checks passed
@Nashtare Nashtare deleted the perf/ll/init branch August 1, 2024 15:14
Nashtare added a commit that referenced this pull request Aug 6, 2024
…putations (#429)

* [WIP] add account to linked list

* Add account linked list and some unit tests

* Modify tests for insetions and fix bugs

* Fix deletion test

* [WIP] Getting trie data without leaves

* [WIP] changes during the flights

* Fix storage ll error

* Add missing file

* [WIP] checking consistency between linked list and state trie

* [WIP] addr 0x798c6047767c10f653ca157a7f66a592a1d6ca550cae352912be0b0745336afd not found in linked list

* [WIP] add missing nibbles to key

* Fix accounts insertions

* Check storage reads

* Add segments to preinitialization

* [WIP] Unreasonable offset

* [WIP] Erc721 error

* [WIP] uncomment ll reads

* [WIP] Fixing iterator

* Fix unit tests

* Fix erc721 error

* mend

* Remove debug info

* mend

* [WIP] hashing

* Provide constants non-deterministically

* Update journaling

* Update search_account and read_accounts_linked_lists

* Constraint state and storage leaves

* Update slot and account search methods in linked_lists and update revert_storage_change

* Fix asm

* Some fixes and debugging

* Fix hash mismatch

* Add functions for updating final trie

* [WIP] eliminating trie call

* [WIP] Fix kexit_data issue

* Merge with journaling branch

* [WIP] set final tries

* Mutate storage payload on insertions

* Fix deletions in final trie computation

* Remove double hashing of the initial state trie

* Misc

* trace_decoder/Cargo.toml

* Fix return from read_storage_linked_list_w_addr

* [WIP] debugging unit tests

* Fix set_payload_storage_extension and bring back final gas check

* Fix failing shanghai blocks

* Fix run_next_addresses_remove and some revertions

* Some journaling fixes

* FIx revert_storage_change and revert_account_created

* Fix delete account

* [WIP] fixing LoopCallsDepthThenRevert3_d0g0v0_Shanghai.json

* Add an overwrite version of account insertion and use it for mpt_insert_state_trie

* Fix final state trie hash mismatch

* Fix insert_new_slot_with_value

* Debugging variedContext_d10g0v0_Shanghai.json

* Fix variedContext_d10g0v0_Shanghai

* [WIP] Fixing vitalikTransactionTestParis_d0g0v0_Shanghai

* Delete all associated slots when an account is deleted

* Fix run_next_remove_address_slots

* Copy initial accounts and slots

* Fix find condition in run_next_remove_address_slots

* Deep copy of accounts and slots

* [WIP] Testing evm test suite

* Fix most unit tests and fix revert_account_created

* FIx linked list test and a bit of cleanup

* [WIP] Debugging /stExample/basefeeExample_d0g0v0_Shanghai

* Fix merge and add batch sizes to the benchmark

* Fix test_only in zero_bin

* Fix rlp pointers error

* Remove panic

* [WIP] Debugging InitCollisionParis_d2g0v0_Shanghai

* [WIP] Debugging stCallCodes/callcallcodecallcode_011_SuicideEnd_d0g0v0_Shanghai

* Fix remove_all_slots

* Clean code

* Fix account code initialize mpts

* Apply comments and fix CI.

* Improve remove_all_slots_loop

* Fix stack comment

* Misc, faster get_valid_slot_ptr

* [WIP] Debugging erc20

* Remove counter update and cold_access

* Fix log_opcode circuit sizes

* Fix stack comment and remove ctr update

* Fix preinitialization

* Fix unit tests

* Debugging stShift/shr01_d0g0v0_Shanghai

* Fixing shiftSignedCombinations_d0g0v0_Shanghai

* Remove counter assertions from linked_lists tests

* Fix preinitialization and start cleanup

* Fix test_process_receipt

* Fix MPT insert tests

* Add check in mpt_insert tests

* Additional fixes

* Pass preinitialized_segments when necessary

* Fix all unit tests

* Cleanup

* More comments cleanup

* Do not store memory content as vec

* Prevent needless conversion

* Remove needless copy

* Remove needless checks

* Use tuple_windows instead

* Typo

* Remove leftover

* Remove unrelated changes

* Fix clone_slot

* Remove assert 0 from clone_slot

* Fix next_node_ok_with_value

* Apply comments and cleanup

* Start

* Do not recompute LLs

* Remove unused preinit

* Remove useless function

* Rename for consistency

* Cleanup

* More cleanup

* Minor

* Lighten up initialization

* Cleaner

* Remove is_dummy

* Remove dummy run

* Fix next_node_ok_with_value and tiny cleanup

* Simplify iterator

* Tweak

* Start addressing comments

* Tweak

* Mighty clippy

* Revert num_procs

* Remove unnecessary linked_lists methods and payload_ptr

* Update .env file

* Address comment, bring back txn hashes

* Apply suggestion

* Update sizes

* Add explicit panic message

---------

Co-authored-by: 4l0n50 <alonso.gon@gmail.com>
Co-authored-by: Hamy Ratoanina <hamy.ratoanina@toposware.com>
Co-authored-by: Alonso Gonzalez <agonzalez@Polygon-F6GJJLCQFL.local>
Co-authored-by: Linda Guiga <lindaguiga3@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
crate: evm_arithmetization Anything related to the evm_arithmetization crate. crate: proof_gen Anything related to the proof_gen crate. crate: trace_decoder Anything related to the trace_decoder crate. crate: zero_bin Anything related to the zero-bin subcrates. performance Performance improvement related changes
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants