Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Check for snap server #6609

Merged
merged 50 commits into from
Jul 3, 2024
Merged

Check for snap server #6609

merged 50 commits into from
Jul 3, 2024

Conversation

pinges
Copy link
Contributor

@pinges pinges commented Feb 23, 2024 •
edited
Loading

PR description

When we are connecting to an EthPeer and we are doing snap sync find out whether that peer does serve snap data.
This is needed to make sure that we have a minimum of snap data serving peers and to make sure that we only send snap requests to peers that serve snap data.

pinges and others added 17 commits February 13, 2024 18:59
@pinges
commit building version
81840ff 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
Merge branch 'main' of github.com:hyperledger/besu into CheckForSnapS…
40d9ea1 
…erver
@pinges
EthPeer add isServingSnap to be able to make sure that we have enough…
7f2fd43 
… snap servers connected when we are snap syncing

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
add real check for snap server using a snap task
dcbb895 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
Merge branch 'main' of github.com:hyperledger/besu into CheckForSnapS…
5449acc 
…erver
@pinges
some cleanup
d3455c9 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
Merge branch 'main' of github.com:hyperledger/besu into CheckForSnapS…
f2d7beb 
…erver
@pinges
add log for snap server check
01f9101 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
change the way we handle the max incoming connections
5d7f1f1 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
fix unit tests
80534d3 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
Merge branch 'main' of github.com:hyperledger/besu into CheckForSnapS…
edbc9d4 
…erver
@pinges
Merge branch 'main' of github.com:hyperledger/besu into CheckForSnapS…
734bb89 
…erver
@pinges
fix acceptance test
ea43fd0 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
Merge branch 'main' of github.com:hyperledger/besu into CheckForSnapS…
d6a2999 
…erver
@pinges
merge main
98fbcce 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@macfarla
Merge branch 'main' into CheckForSnapServer
fbe6b52
@macfarla
merge
446ef1b 
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
This was referenced Apr 11, 2024
Closed
Closed
@macfarla
merge
4203a55 
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
@macfarla macfarla marked this pull request as ready for review April 11, 2024 08:15
garyschulte
garyschulte reviewed Apr 11, 2024
View reviewed changes
Copy link
Contributor

@garyschulte garyschulte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this is still in development, with a lot of commented code, debug and TODOs.
The method of checking if a server is serving snap data seems solid, use their reported chain head and ensure they serve account data for it 👍

Overall, I'd prefer that we only weight snap servers while we are in the process of snap sync however. I think a less intrusive way to do this would be to schedule a peer check in the snap sync pipeline that stops once we have downloaded all of the snap data. It can maintain the peer ratio and disconnect peers as needed, and we can encapsulate that behavior into just sync where it is relevant.

besu/VERSION_METADATA.json Outdated Show resolved Hide resolved
besu/src/main/java/org/hyperledger/besu/RunnerBuilder.java Show resolved Hide resolved
ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/EthProtocolManager.java Outdated Show resolved Hide resolved
@macfarla
merge
61d7265 
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
@macfarla
extra logging from hyperledger#6924
de6899f 
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
@macfarla
formatting
272c1bb 
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
@macfarla
delete version metadata
ae030e7 
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
@macfarla macfarla mentioned this pull request Apr 12, 2024
Closed
@macfarla
reuse ethPeers
68934cc 
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
@macfarla
make a switching peer task
a95e4f4 
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
@macfarla
Merge branch 'main' of github.com:hyperledger/besu into CheckForSnapS…
13fb105 
…erver
@macfarla macfarla mentioned this pull request Apr 24, 2024
Closed
pinges added 11 commits June 13, 2024 13:15
@pinges
merge
14528e8 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
fix after merge
b148347 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
stop prefering snap servers after sync
12c8350 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
fix compile
1354979 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
add tests and some clean-up
048765f 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
make constant
de7d55b 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
merge and fix conflicts
a3e17d1 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
add newline
0099f11 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
spotless
f1f2073 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
fix tests, fix trailing peers, and clean up
13807b1 
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
@pinges
Merge branch 'main' of github.com:hyperledger/besu into CheckForSnapS…
9bf96ff 
…erver
@pinges pinges marked this pull request as ready for review June 26, 2024 01:02
usmansaleem
usmansaleem reviewed Jul 2, 2024
View reviewed changes
ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/EthPeers.java

private Comparator<EthPeer> bestPeerComparator;
private final Bytes localNodeId;
private RlpxAgent rlpxAgent;

private final Counter connectedPeersCounter;
// private List<ProtocolManager> protocolManagers;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can delete commented code.

usmansaleem
usmansaleem reviewed Jul 2, 2024
View reviewed changes
Copy link
Member

@usmansaleem usmansaleem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor comments otherwise looks good to me.

usmansaleem
usmansaleem approved these changes Jul 3, 2024
View reviewed changes
Copy link
Member

@usmansaleem usmansaleem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pinges pinges merged commit 8a9a84a into hyperledger:main Jul 3, 2024
40 checks passed
daniellehrner pushed a commit to daniellehrner/besu that referenced this pull request Jul 16, 2024
@pinges @macfarla @daniellehrner
Check for snap server (hyperledger#6609)
a8c4cb8 
* EthPeer add isServingSnap to be able to make sure that we have enough snap servers connected when we are snap syncing

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
daniellehrner pushed a commit to daniellehrner/besu that referenced this pull request Jul 16, 2024
@pinges @macfarla @daniellehrner
Check for snap server (hyperledger#6609)
55a2f1f 
* EthPeer add isServingSnap to be able to make sure that we have enough snap servers connected when we are snap syncing

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
daniellehrner added a commit that referenced this pull request Jul 17, 2024
@daniellehrner @cdivitotawela
@pinges @macfarla @shemnon @Gabriel-Trintinalia @matthew1001 @jframe @TiesD @non-fungible-nelson @Wolmin @fab-10 @leniram159 @jflo @usmansaleem @siladu @garyschulte @matkt @snazzysam933 @gtebrean @Matilda-Clerke
EIP 7702 (#7237)
895c17d 
* EIP 7702 first draft

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* added CHANGELOG.md entry

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* bug fixes, added first tests

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* container verify GitHub workflow (#7239)

Container verification step in release process automated with the container verify GitHub workflow. New workflow is triggered at the end of the release workflow which will check the release container images starts successfully. Verification test only checks container starts and reach the Ethereum main loop

Signed-off-by: Chaminda Divitotawela <cdivitotawela@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Investigate chain halts when syncing (#7162)

Fix some reasons for chain download halts when syncing

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Signed-off-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Check for EOFCreate subcontainer rules (#7232)

Check and test for the unused container rule, and only returncontract
targets can have truncated data rule.
Also test the other subcontainer rules in unit tests.

Signed-off-by: Danno Ferrin <danno@numisight.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Remove deprecation message for `--Xp2p-peer-lower-bound` (#7247)

Signed-off-by: Gabriel-Trintinalia <gabriel.trintinalia@consensys.net>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* less invasive code injection approach

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* added missing java doc & fixed test

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* added (currently non-working) acceptance test, some bug fixes in the transaction validation and tx pool logic

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* fix spotless

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* updated acceptance test, still not working, newPayload request seems to be necessary before final fork choice update

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* use correct world state to inject temporary code, inject code in existing accounts as well

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* renamed test service to prague, because the engine versions used are only available in the prague hard fork

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* fixed acceptance test, some bug fixes if authorized account does not yet exist

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Add build version option to prefix git hash with custom version property (#7222)

* Add build version option to prefix git hash with custom version property
* Refactor to make appending the git hash a boolean property. Include a commented-out example of how to use the properties in the gradle file

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Handle invalid snap getTrieNode requests with empty paths gracefully (#7221)

Signed-off-by: Jason Frame <jason.frame@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* fix typos in CHANGELOG (#7226)

Signed-off-by: Ties <71668189+TiesD@users.noreply.github.com>
Co-authored-by: Matt Nelson <85905982+non-fungible-nelson@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* feat: Add network option for LUKSO Mainnet (#7223)

* Add option for LUKSO network

Signed-off-by: Wolmin <lamonos123@gmail.com>

* Add tests for LUKSO

Signed-off-by: Wolmin <lamonos123@gmail.com>

* Apply spotless

Signed-off-by: Wolmin <lamonos123@gmail.com>

* Add changelog entry

Signed-off-by: Wolmin <lamonos123@gmail.com>

* Fix duplicate func

Signed-off-by: Wolmin <lamonos123@gmail.com>

* Fix changelog

Signed-off-by: Wolmin <lamonos123@gmail.com>

* Add bootnodes to genesis

Signed-off-by: Wolmin <lamonos123@gmail.com>

---------

Signed-off-by: Wolmin <lamonos123@gmail.com>
Signed-off-by: Wolmin <44748271+Wolmin@users.noreply.github.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Update Docker base image to Ubuntu 24.04 (#7251)

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Reconfigure how Protocol Specs are created WRT EVM condiguration (#7245)

Make the max code size and max initcode size a part of the EVM
configuration. As part of the change we need to move the tasks
CodeFactory once handled as a static class and move it into the EVM.
This has a nice follow on effect that we don't need to pass in max EOF
versions or max code sizes anymore.

Signed-off-by: Danno Ferrin <danno@numisight.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Fix the wrong 'Identifier' and 'Synchronizer' usage (#7252)

* fix the synchronizer usage

Signed-off-by: Leni <leniram159@gmail.com>

* fix Identifier usage

Signed-off-by: Leni <leniram159@gmail.com>

---------

Signed-off-by: Leni <leniram159@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Fix flaky SECP256R1 test (#7249)

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* update to work with the new max retries value (#7253)

Signed-off-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Temporary CancunEOF fork for EOF testing. (#7227)

Add Genesis ("CancunEOFTime") and reference test ("CancunEOF") support
for a temporary Cancun+EOF fork, in anticipation of potential devnets.

Signed-off-by: Danno Ferrin <danno@numisight.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* fixed bug introduced through merge of main, made acceptance test easier to understand

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* added missing java docs

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* removed unnecessary tag

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* make encodeSingleSetCode public again

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Snapserver responses to return at least one response (#7190)

Signed-off-by: Jason Frame <jason.frame@consensys.net>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* copy setCodeTransactionPayloads as well

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* fixed bug during tests with forrest db

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Snapserver GetTrieNodes request to handle short hash for storage (#7264)

Signed-off-by: Jason Frame <jason.frame@consensys.net>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* javadoc: Adding javadoc for ethstats module (#7269)

* javadoc: Adding javadoc for ethstats module

---------

Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Fix javadoc for ethereum:core top level package (#7270)

* javadoc - Apply javadoc to ethereum core top level package

---------

Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Disable Flaky tests - permissioning (#7256)

* disable some flaky tests

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* correct name for test

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* formatting

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* disable some flaky tests

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

---------

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Add bootnodes to the maintained peer list (#7257)

* Add bootnodes to the maintained peer list

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Update unit tests

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Add entry in changelog

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Tweak unit test

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Refactor to keep common steps the same for both cases

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Add debug log, call sanitizePeers() only once

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

---------

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Fix javadoc for ethereum api module, graphql package (#7272)

* javadoc - Adding missing javadocs ethereum:api graphql package

Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* T8n support for isStateTest and empty accounts (#7275)

Update t8n executor to support new isStateTest env flag that will
disable extra-transactional processing such as block rewards and beacon
root.
Also, make sure such extra-transactional commits don't create empty
accounts.

Signed-off-by: Danno Ferrin <danno@numisight.com>
Co-authored-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Promote storage x-trie-log subcommand to trie-log (#7278)

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Evm tool readme update (#7274)

* update paths to binary. update docker build to use java 21
* updated suggested jdk

---------

Signed-off-by: Justin Florentine <justin+github@florentine.us>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* javadoc - Add missing javadoc for evmtool module (#7277)

Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Rename ValidatorPublicKey to ValidatorPubKey (#7280)

Adapt to EIP-7002 name change for validator public key in all places.

Signed-off-by: Danno Ferrin <danno@numisight.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Add info-level diagnostic logs to aid with resolving stalled BFT chains (#7271)

* Add info-level diagnostic logs to aid with resolving stalled BFT chains

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Add javadoc

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

---------

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>
Signed-off-by: Matt Whitehead <matthew.whitehead@kaleido.io>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Update EIP-2935 contract (#7281)

Use the updated contract and address for EIP-2539.

Signed-off-by: Danno Ferrin <danno@numisight.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* add evmtool compability, fixing bugs related to sender recovery of 7702 txs and handling authorizations to empty accounts

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Deeper tracing of self-destructed accounts (#7284)

Consider previously self-destructed accounts when creating accounts.

Signed-off-by: Danno Ferrin <danno@numisight.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* next release version after 24.7.0 (#7285)

Signed-off-by: garyschulte <garyschulte@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Add experimental `--Xsnapsync-bft-enabled` which enables snap sync for BFT chains (#7140)

* Create a BFT-specific pivot block handler

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Change visibility

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Refactor sync-peer-count internal variable to match name, add experimental flag to enabled snap + BFT

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Merge with main

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Fix uppercase

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Address synchronization issue with trie pruning. Create BFT-specific account range handler. Add pipeline name and logs

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Remove debug log

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* fixing snapsync for empty state

Signed-off-by: Karim Taam <karim.t2am@gmail.com>

* Don't queue up events we can't handle

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Fix timing window where a validator with an empty data dir sometimes falls back to full sync if peer status isn't received quickly enough

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Remove BFT-specific account request class. Not needed

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Refactor some more 'fast' sync variables that are common to all pivot-based sync types

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* In FULL sync mode, disable bonsai-limit-trie-logs-enabled instead of failing to start

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Add javadoc comments, clarify overriding bonsai-limit-trie-logs-enabled

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Add BFT pivot block selector tests

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Fix failure error message

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Remove the unnamed Pipe constructor and update tests to set a pipe name

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Revert some info logs back to debug given the feedback on noise in the logs syncing with holesky

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Refactor fastSyncPivotDistance to syncPivotDistance

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Incomplete refactoring

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Update BFT event queueing tests

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Event queue test fixes

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Remove automatic setting of bonsai-limit-trie-logs-enabled to false if sync-mode = FULL (moving to another PR)

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

---------

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>
Signed-off-by: Karim Taam <karim.t2am@gmail.com>
Signed-off-by: Matt Whitehead <matthew.whitehead@kaleido.io>
Co-authored-by: Karim Taam <karim.t2am@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Turn off CicleCI for Besu (#7291)

All the CI jobs run in GitHub actions and Circle CI it no longer needed in Besu project

Signed-off-by: Chaminda Divitotawela <cdivitotawela@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Check for snap server (#6609)

* EthPeer add isServingSnap to be able to make sure that we have enough snap servers connected when we are snap syncing

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Implement System Calls (#7263)

Signed-off-by: Gabriel-Trintinalia <gabriel.trintinalia@consensys.net>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* wrap WorldUpdater inside a WorldUpdaterService to inject the authorized code whenever needed

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Update limit trie logs validation message for sync-mode=FULL (#7279)

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>

Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Execute requests before block persist (#7295)

Signed-off-by: Gabriel-Trintinalia <gabriel.trintinalia@consensys.net>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* fixed MainnetTransactionProcessor retrieval of correctn `to` account with injected code, fixed code injection

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* only first authorization is accepted, all the following ones are ignored

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* don't cache account with empty code

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* revert wrapping of world updater, as its `updater()` method creates a type of nesting that is not compatible with wrapping it. Instead a service is injected in the world updater to inject the code into the authorized accounts

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Fixed outdated tech redirect link. (#7297)

* fix wiki link

Signed-off-by: Snazzy <snazzysam933@gmail.com>

* fix format

Signed-off-by: Snazzy <snazzysam933@gmail.com>

* change knownHash

Signed-off-by: Snazzy <snazzysam933@gmail.com>

---------

Signed-off-by: Snazzy <snazzysam933@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Increment private nonce even if transaction failed. (#6593)

Increment private nonce even if transaction failed

Signed-off-by: George Tebrean <george@web3labs.com>
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Co-authored-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Co-authored-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* feat: Enhance --profile to load external profiles (#7292)

* feat: --profile can load external profiles
* fix external profile name method
* fix ProfilesCompletionCandidate
* test: Add unit tests
* changelog: Update changelog
* test: Fix TomlConfigurationDefaultProviderTest
* test: Fix BesuCommandTest

---------

Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Fix status badge for documentation (#7304)

Documentation has been moved to GitHub pages and no longer use readthedocs. Updated the README status badge for docs with correct link

Signed-off-by: Chaminda Divitotawela <cdivitotawela@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* [MINOR] Fixed some typos (#7299)

* typos

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

---------

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* refactored to share one AuthorizedAccountService between the different instances of the world updater, renamed some classes

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* spotless

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* load code for authorization at the beginning of the transaction

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* ignore authorization if chain id doesn't match

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* cache authority address, evmtool: do not fail if sender address is wrong

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Add evmtool block-test subcommand (#7293)

* Add evmtool block-test subcommand

Add an evmtool subcommand that will run non-hive blockchain tests.

Signed-off-by: Danno Ferrin <danno@numisight.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Make the retrying snap tasks switching (#7307)

* make snap tasks switching

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* 6612: Remove deprecated sync modes and related helper methods (#7309)

* 6612: Remove deprecated sync modes and related helper methods

Signed-off-by: Matilda-Clerke <matilda.shay.clerke@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* EOF Reference Test Fixes (#7306)

Fix a number of issues found in reference tests and evmone tests.

- Be tolerant of more nulls in json
- Support ContainerKind in reference tests
- re-order EXTCALL oeprands
- correct return value for REVERT in EXT*CALL
- re-order EOFCREATE code validation

Signed-off-by: Danno Ferrin <danno@numisight.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* test template refactor, bump besu-native to 0.9.2 (#7315)

Signed-off-by: garyschulte <garyschulte@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Feature/use gnark-crypto for eip-2537 (#7316)

* use gnark-crypto for bls precompiles

Signed-off-by: garyschulte <garyschulte@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* 6612 update changelog with removed syncmodes (#7320)

* 6612: Update changelog with removal of deprecated sync modes

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 6612: Update changelog with removal of deprecated sync modes

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

* 6612: Update changelog with removal of deprecated sync modes

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>

---------

Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Update datacopy (#7319)

Check for OOG earlier in DataCopy.
Add unit tests to cover operation branches.

Signed-off-by: Danno Ferrin <danno@numisight.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* disable flaky test (#7308)

* disable flaky test

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* disable flaky test

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

---------

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Update unit test (#7317)

* Update parameterized unit tests so the enumerate with --dry-run
* Update the prague-withdrawal.json unit test to handle current code

Signed-off-by: Danno Ferrin <danno@numisight.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* persist accounts that have storage updates, but no nonce, balance nor code

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Revert "persist accounts that have storage updates, but no nonce, balance nor code"

This reverts commit 9c9121a.

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* removed PKI backed QBFT (#7310)

* removed PKI backed QBFT

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* changelog

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

---------

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* EIP-7251 add consolidation request type (#7266)

* add request type for consolidations, encoder, decoder and tests

* added raw tx for consolidation

* add consolidation reqs to EngineGetPayloadResultV4

* set storage slot value to 0 initially and value for tx

* updates plugin api

Signed-off-by: Justin Florentine <justin+github@florentine.us>

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

---------

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* fix: `eth_call` deserialization to correctly ignore unknown fields in the transaction object (#7323)

* fix: Use Builder for JsonCallParameter
* changelog
* add additional unit tests
* fix: Update builder to withGas to match the json eth_call
---------

Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Stop transaction selection on TX_EVALUATION_TOO_LONG (#7330)

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* message frame buider will create AuthorizedCodeService by itsef if it isn't set

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* get correct nonce for authorization

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* nonce only returns a vaid nonce, new method nonceList returns all the nonces

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* plugs leaky abstraction

Signed-off-by: Justin Florentine <justin+github@florentine.us>

* some renaming, acceptance tests checks for exact balance of tx sponsor at the end.

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* inject the 7702 code into DiffBasedWorldStateUpdateAccumulator.createAccount

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* spotless

Signed-off-by: Justin Florentine <justin+github@florentine.us>

* spotless fix, removed todos

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* make AuthorityProcessor & chain id for it optional

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

---------

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Signed-off-by: Chaminda Divitotawela <cdivitotawela@gmail.com>
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Signed-off-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Signed-off-by: Danno Ferrin <danno@numisight.com>
Signed-off-by: Gabriel-Trintinalia <gabriel.trintinalia@consensys.net>
Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>
Signed-off-by: Jason Frame <jason.frame@consensys.net>
Signed-off-by: Ties <71668189+TiesD@users.noreply.github.com>
Signed-off-by: Wolmin <lamonos123@gmail.com>
Signed-off-by: Wolmin <44748271+Wolmin@users.noreply.github.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Leni <leniram159@gmail.com>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Matt Whitehead <matthew.whitehead@kaleido.io>
Signed-off-by: garyschulte <garyschulte@gmail.com>
Signed-off-by: Karim Taam <karim.t2am@gmail.com>
Signed-off-by: Snazzy <snazzysam933@gmail.com>
Signed-off-by: George Tebrean <george@web3labs.com>
Signed-off-by: Matilda-Clerke <matilda.shay.clerke@gmail.com>
Signed-off-by: Matilda Clerke <matilda.clerke@consensys.net>
Co-authored-by: Chaminda Divitotawela <cdivitotawela@users.noreply.github.com>
Co-authored-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Danno Ferrin <danno@numisight.com>
Co-authored-by: Gabriel-Trintinalia <gabriel.trintinalia@consensys.net>
Co-authored-by: Matt Whitehead <matthew.whitehead@kaleido.io>
Co-authored-by: Jason Frame <jason.frame@consensys.net>
Co-authored-by: Ties <71668189+TiesD@users.noreply.github.com>
Co-authored-by: Matt Nelson <85905982+non-fungible-nelson@users.noreply.github.com>
Co-authored-by: Wolmin <44748271+Wolmin@users.noreply.github.com>
Co-authored-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: leniram159 <leniram159@gmail.com>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: Usman Saleem <usman@usmans.info>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: Karim Taam <karim.t2am@gmail.com>
Co-authored-by: gringsam <snazzysam933@gmail.com>
Co-authored-by: George Tebrean <99179176+gtebrean@users.noreply.github.com>
Co-authored-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Co-authored-by: Matilda-Clerke <matilda.shay.clerke@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@garyschulte garyschulte garyschulte left review comments

@macfarla macfarla macfarla left review comments

@usmansaleem usmansaleem usmansaleem approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@pinges @macfarla @usmansaleem @garyschulte