Eth protocol integration to p2p (don't merge, to be reviewed) #205

zelig · 2014-12-10T14:54:57Z

eth package

eth protocol interfacing with p2p peer and backend
blockpool interfacing chainmanager and protocol
backend gluing core components together

- changed backend interface - using callbacks for blockPool - use rlp stream for lazy decoding - use peer as logger - add id (peer pubkey) to ethProtocol fields - add testPeer to protocol test (temporary)

* Added filters / watches * Removed event dep

- changed backend interface - using callbacks for blockPool - use rlp stream for lazy decoding - use peer as logger - add id (peer pubkey) to ethProtocol fields - add testPeer to protocol test (temporary)

- new interface explicit backend components txPool, chainManager, blockPool - added protoErrorDisconnect for blockpool callback (FIXME: handling peer disconnects)

- use eventer events to broadcast transactions - CurrentTransactions -> GetTransactions - add AddTransactions

…ith eth/protocol

- eth p2p pkgs - new Ethereum initialiser - no caps param - use nat type - add NatType func to map nat type string to p2p.NAT - add pubkey to client identity

- nat UPNP|PMP - gateway for PMP NAT - obsolete usepnp - add pubkey to client identity

- changed backend interface - using callbacks for blockPool - use rlp stream for lazy decoding - use peer as logger - add id (peer pubkey) to ethProtocol fields - add testPeer to protocol test (temporary)

- new interface explicit backend components txPool, chainManager, blockPool - added protoErrorDisconnect for blockpool callback (FIXME: handling peer disconnects)

- add Status() to return td, currentblock hash, genesis hash - GetChainHashesFromHash -> GetBlockHashesFromHash

- new flag nat for nat support UPNP|PMP - new flag pmp for PMP gateway IP - add NatType to utils/cmd to get p2p.NAT from nat type string - obsolete usepnp flag - get rid of IsUpToDate and sleep in miner start - ethereum constructor takes nat type, port, maxpeer - add pubkey arg to client identity

Conflicts: eth/block_pool.go eth/error.go eth/protocol.go eth/protocol_test.go

obscuren · 2014-12-15T14:45:35Z

Has been merged in to poc-8 branch.

* Added getBlockRange endpoint * Fix a minor bug * lint: fix Co-authored-by: Kelvin Fichter <kelvinfichter@gmail.com>

* core, trie, eth, cmd: rework preimage store * fix ci * fix ci * Update zk_trie_database.go --------- Co-authored-by: rjl493456442 <garyrong0905@gmail.com> Co-authored-by: maskpp <maskpp266@gmail.com>

Merge v1.11.2

Squash the main verkle PR ahead of rebase don't call Bytes() in GetTreeKey (ethereum#137) trie: avoid endianness conversion in GetTreeKey (ethereum#140) * trie/utils: add concrete expected value in trie key generation test Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * mod: update to latest go-verkle Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie/utils: avoid endianness conversions Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * apply review changes & update to official go-verkle version Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> upgrade go-verkle to CoW version and get TestProcessVerkle to build (ethereum#138) updating ci to use self-hosted machine (ethereum#143) fix: storage offset in non-header group + reuse of value buffer (ethereum#145) dedup call to ChunkifyCode, same as replay branch (ethereum#156) * dedup call to ChunkifyCode, same as replay branch * fix some linter issues fix code offset in tree update (ethereum#157) fix REVERT in state processor test execution (ethereum#158) * fix code offset in tree update * fix REVERT in test execution save on key hashing: lump code size update with first code chunk group (ethereum#159) fix code chunk key calculation and storage key calculation (ethereum#161) * fix codeKey calculation * Remove * fix storageOffset * fix the fix to the fix to the offset fix * Remove copy/pasted, unused code in test * fix linter --------- Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com> fix: infinite loop when calling extcodecopy on empty code (ethereum#151) upgrade to latest go-verkle fix: only update code in the tree if it's dirty (ethereum#174) fix: read-touch the code size and Keccak of the origin (ethereum#175) List of changes for converting a sepolia database (ethereum#182) * naive conversion rebased on top of beverly hills * changes for the sepolia shadow fork conversion * fixes to please the linter * fixes to please the linter Unified point cache (ethereum#180) * Unified point cache * Use cache for Try*Account * alter Trie interface to use caching for slots (ethereum#181) * alter Trie interface to use caching for slots * fix: use a lock to protect the point cache (ethereum#185) * use fastest non-master go-verkle version & pull trie/Verkle.go changes to use new api (ethereum#184) * mod: update to fastest go-verkle version today Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie/verkle: use new batch serialization api Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> * fix: TryDelete signature in unit tests --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> trie/utils: fix potential overflow (ethereum#191) * trie/utils: fix potential overflow Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie/utils: receive storage key as a byte slice Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * revert formatter changes Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie/utils: fix mod 256 Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> trie/utils: fix incorrect bigint assignment (ethereum#193) Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> upgrade precomp link to fix CI fix: add missing code size&keccak leaves in empty accounts (ethereum#192) fixes to use the latest go-verkle@master (ethereum#197) * fixes to use the latest go-verkle@master * linter fixes * linter fixes for tests * fix: use jsign's go-verkle fix refactor: remove unused (*StateDB).GetXLittleEndian methods (ethereum#204) fix gas accounting issue in state_processor_test.go (ethereum#207) update go-verkle not to use StatelessNode anymore (ethereum#206) * update go-verkle not to use StatelessNode anymore * update go-verkle to latest refactor: move verkle gas accounting to its own block in TransitionDB (ethereum#208) fix a panic in deserializeVerkleProof if GetProofItems returns a nil ProofElements use the cachingDB instead of a custom VerkleDB (ethereum#209) * use the cachingDB instead of a custom VerkleDB * fix stack trace in LES remove holiman from CODEOWNERS as he gets too many emails read from tree in state object if the snapshot is nil (ethereum#205) add missing error checks for the root node type (ethereum#214) implement OpenStorageTrie for verkle trees (ethereum#210) * implement OpenStorageTrie for verkle trees * add a few comments for future maintenance * fix linter issue fix: copy balance leaf to new buffer in TryGetAccount (ethereum#217) implement some heretofore unimplemented iterator methods (ethereum#219) params: move verkle params to their own file (ethereum#228) fix: proper number of chunk evals (ethereum#215) overlay transition (ethereum#244) * overlay transition Fix some bugs identified in the code review Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> Include base -> overlay key-values migration logic (ethereum#199) * mod: add go-verkle version with key-value migration new apis Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * core/stateprocessor: use constant for max number of migrated key-values Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * core: add base->overlay key-values migration logic Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * core: fix some compiler errors Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie: consider removing transition trie api in the future Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * mod: use latest go-verkle Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> fix some unit tests errors get convresion block from file fix compilation issues fix initialization issue in migrator fix: changes needed to run the first 28 blocks important sutff: fix the banner fix: use nonce instead of balance in nonce leaf (ethereum#202) fixes for performing the overlay transition (ethereum#203) * fixes for performing the overlay transition * fixes for the full replay * fix: deletion-and-recreation of EoA * fixes to replay 2M+ blocks * upgrade to go-verkle@master * fix: proper number of chunk evals * rewrite conversion loop to fix known issues changes to make replay work with the overlay method (ethereum#216) * fixes for performing the overlay transition fixes for the full replay fix: deletion-and-recreation of EoA fixes to replay 2M+ blocks upgrade to go-verkle@master fix: proper number of chunk evals rewrite conversion loop to fix known issues changes to make replay work with the overlay method fixes to replay 2M+ blocks update to latest go-verkle@master * use a PBSS-like scheme for internal nodes (ethereum#221) * use a PBSS-like scheme for internal nodes * a couple of fixes coming from debugging replay * fix: use an error to notify the transition tree that a deleted account was found in the overlay tree (ethereum#222) * fixes for pbss replay (ethereum#227) * fixes for pbss replay * trie/verkle: use capped batch size (ethereum#229) * trie/verkle: use capped batch size Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * trie/verkle: avoid path variable allocation per db.Put Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * don't keep more than 32 state root conversions in RAM (ethereum#230) --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com> * cleanup some code * mod: update go-verkle Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * re-enable snapshot (ethereum#231) * re-enable cancun block / snapshot (ethereum#226) * clear storage conversion key upon translating account (ethereum#234) * clear storage conversion key upon translating account * mod: use latest go-verkle Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> * fix: self-deadlock with translated root map mutex (ethereum#236) * return compressed commitment as root commitment (ethereum#237) --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> fix first panic in *TransitionTrie.Copy() upgrade go-verkle to latest master mod: update go-verkle (ethereum#239) Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> core: print state root every 100 blocks (ethereum#240) Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> fix: only Commit the account trie (ethereum#242) fixes to get TestProcessVerkle to work with the overlay branch (ethereum#238) * fixes to get TestProcessVerkle to work with the overlay branch * fix all panics in verkle state processor test * fix proof verification move transition management to cachingDB * fix: mark the verkle transition as started if it's ended without being started * fix the verkle state processing test * fix linter errors * Add a function to clear verkle params for replay * fix: handle TransitionTrie in OpenStorageTrie * fix linter issue * fix the deleted account error (ethereum#247) * code cleanup (ethereum#248) * fix: don't error on a missing conversion.txt (ethereum#249) * Overlay Tree preimages exporting and usage (ethereum#246) * export overlay preimages tool Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * use preimages flat file in overlay tree migration logic Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * cmd/geth: add --roothash to overlay tree preimage exporting command Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * cleanup Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * review feedback Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> * fix: reduce the PR footprint (ethereum#250) * fix: don't fail when preimages.bin is missing (ethereum#251) * fix: don't fail when preimages.bin is missing * fix: don't open the preimages file when outside of transition --------- Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> review changes remove replay-specific code

zelig added 4 commits December 10, 2014 14:51

initial commit for eth-p2p integration

47793b6

no logging in error (to be refactored into p2p)

31a9fdc

eth protocol changes

e74f9f2

- changed backend interface - using callbacks for blockPool - use rlp stream for lazy decoding - use peer as logger - add id (peer pubkey) to ethProtocol fields - add testPeer to protocol test (temporary)

initial commit for eth blockpool + test

3e38f8a

zelig added the in progress label Dec 10, 2014

obscuren and others added 25 commits December 12, 2014 15:25

Accidentally changed max depth during testing & committed

25cf0c4

Generic filter interface

9e1689d

Moved filter to events

ed15382

Implemented watching using filter package

a17a1f9

* Added filters / watches * Removed event dep

Moved

1c7fd62

Added length checkes

06e7642

Panic for "Vm"

8577e41

Implemented Qt whisper interface

bf566a6

updated tests

aec0c0e

Merge branch 'poc8' into develop

0122989

eth-go packages ended up in whisper

4c84db8

fixed tests

f8061fc

initial commit for eth-p2p integration

e5aa38c

no logging in error (to be refactored into p2p)

eb5cb04

eth protocol changes

d957dd2

- changed backend interface - using callbacks for blockPool - use rlp stream for lazy decoding - use peer as logger - add id (peer pubkey) to ethProtocol fields - add testPeer to protocol test (temporary)

initial commit for eth blockpool + test

4366fdf

initial commit for new backend (eth.Ethereum)

02017ed

protocol

c44e025

- new interface explicit backend components txPool, chainManager, blockPool - added protoErrorDisconnect for blockpool callback (FIXME: handling peer disconnects)

add protocol error types specific to blockpool

3308d82

blockpool rewritten , tests broken FIXME

76070b4

adapt xeth pkg to new backend. FIXME JSPeer peer info

148de1c

adapt miner to new backend. use events to broadcast new mined blocks

118862f

adapt javascript pkg to new backend, use SuggestPeer

b89ed8e

transaction pool changes

c2af10d

- use eventer events to broadcast transactions - CurrentTransactions -> GetTransactions - add AddTransactions

add NewMinedBlockEvent

5022a31

zelig added 20 commits December 14, 2014 20:30

adapt to new eth pkg and p2p

39d86a2

add Status to retrieve TD, currentBlock, genesis for easy interface w…

d84810d

…ith eth/protocol

import eth pkg

50e1dcc

adapt to new backend

ae7c1e3

- eth p2p pkgs - new Ethereum initialiser - no caps param - use nat type - add NatType func to map nat type string to p2p.NAT - add pubkey to client identity

new cli options

d9b16c1

- nat UPNP|PMP - gateway for PMP NAT - obsolete usepnp - add pubkey to client identity

remove old ethereum/p2p related top level files

7d02c4f

remove wire package

e847aac

initial commit for eth-p2p integration

0add0c4

eth protocol changes

2dd8f41

- changed backend interface - using callbacks for blockPool - use rlp stream for lazy decoding - use peer as logger - add id (peer pubkey) to ethProtocol fields - add testPeer to protocol test (temporary)

initial commit for eth blockpool + test

5e4d77b

protocol

4dfce46

- new interface explicit backend components txPool, chainManager, blockPool - added protoErrorDisconnect for blockpool callback (FIXME: handling peer disconnects)

blockpool rewritten , tests broken FIXME

01dc1c1

adapt chain_manager to eth protocol interface

7b39cc8

- add Status() to return td, currentblock hash, genesis hash - GetChainHashesFromHash -> GetBlockHashesFromHash

blockmanager start/stop obsolete

e77b720

adapt blockpool/backend to use pow/ezp with pow.Block for VerifyPoW func

4c89d53

resolve merge conflict hell

72290f6

Merge branch 'ethp2p' of github.com:ethersphere/go-ethereum into ethp2p

702801a

Conflicts: eth/block_pool.go eth/error.go eth/protocol.go eth/protocol_test.go

rename logger pkg and logger instance var

835bf56

protocol test: basic status message tests pass

2740588

obscuren closed this Dec 15, 2014

obscuren removed the in progress label Dec 15, 2014

maoueh pushed a commit to streamingfast/go-ethereum that referenced this pull request Aug 13, 2021

rpc: getBlockByRange (ethereum#205)

8084020

* Added getBlockRange endpoint * Fix a minor bug * lint: fix Co-authored-by: Kelvin Fichter <kelvinfichter@gmail.com>

sduchesneau pushed a commit to streamingfast/go-ethereum that referenced this pull request Aug 1, 2023

Merge pull request ethereum#205 from OffchainLabs/merge-v1.11.2

2e6c09f

Merge v1.11.2

atenjin pushed a commit to alt-research/go-ethereum that referenced this pull request Apr 4, 2024

L2 Cancun support, activate on Ecotone (ethereum#205)

a79bde2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Eth protocol integration to p2p (don't merge, to be reviewed) #205

Eth protocol integration to p2p (don't merge, to be reviewed) #205

zelig commented Dec 10, 2014

obscuren commented Dec 15, 2014

Eth protocol integration to p2p (don't merge, to be reviewed) #205

Eth protocol integration to p2p (don't merge, to be reviewed) #205

Conversation

zelig commented Dec 10, 2014

obscuren commented Dec 15, 2014