Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix IBFT2 chains using BONSAI DB #12

Closed
wants to merge 741 commits into from

Conversation

matthew1001
Copy link

PR description

This PR fixes a bug with IBFT2 chains where blocks are persisted at block-proposal time instead of block-import time. With Bonsai DB this causes issues updating the flat DB because if a round change happens the original proposal isn't actually the block that's accepted for that round.

Fixed Issue(s)

Fixes hyperledger#7629

krsh24 and others added 30 commits June 11, 2024 08:01
* Error out on permissions config accounts-allowlist validation errors.

Signed-off-by: krishnannarayanan <krsh24@gmail.com>

* Fixing compilation errors

Signed-off-by: krishnannarayanan <krsh24@gmail.com>

* Incorrect file check in

Signed-off-by: krishnannarayanan <krsh24@gmail.com>

* Support for eth_maxPriorityFeePerGas

Signed-off-by: krishnannarayanan <krsh24@gmail.com>

* Change log update for my PRs

Signed-off-by: krishnannarayanan <krsh24@gmail.com>

---------

Signed-off-by: krishnannarayanan <krsh24@gmail.com>
… to the method (hyperledger#7188)

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* Add new acceptance test to soak test BFT chains

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Spotless

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Tidy up a little with re-usable start and stop functions with built in delays

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Add shanghai version of Simple Storage contract

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Put commented gradle code back in. Fix the web3j example commands in .sol files

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Spotless

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Set publication artifacts to avoid clash

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Exclude from regular acceptance tests

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Add shanghai fork to the test. Stall the chain for less time to reduce the time taken to mine new blocks

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Tidy up

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Update acceptance-tests/tests/shanghai/build.gradle

Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Matt Whitehead <matthew1001@hotmail.com>

* Tidy up var names

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Fix ports for IBFT2 as well as QBFT

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Remove maven publish spec, disable jar building for shanghai contract project

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* web3j version

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Make fixed port optional when creating a BFT node

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Only check artifact coordinates for those starting 'org.*'

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

---------

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>
Signed-off-by: Matt Whitehead <matthew1001@hotmail.com>
Signed-off-by: Matt Whitehead <matthew.whitehead@kaleido.io>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* Refactor genesis options file management

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Make loading allocation from genesis lazy

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update tests

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Memory optimization with streaming

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Improve loading and storgin genesis state at startup

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Remove comments

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Avoid parsing genesis file allocations twice

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update javadoc

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Fix

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Ignore unknown objects in allocations

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* avoid keeping genesis allocation data in memory

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update CHANGELOG

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

---------

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: ahamlat <ameziane.hamlat@consensys.net>
Co-authored-by: ahamlat <ameziane.hamlat@consensys.net>
Enable JFR continuous profiling with default setting

Signed-off-by: Ameziane H <ameziane.hamlat@consensys.net>
* fix(test): Fix dns daemon periodic test

Increase second run to 3 seconds

Signed-off-by: Usman Saleem <usman@usmans.info>

* fix(test): Fix dns daemon periodic test

change vertx testContext verify to failNow

Signed-off-by: Usman Saleem <usman@usmans.info>

* fix(test): Fix dns daemon periodic test

Remove Disabled

Signed-off-by: Usman Saleem <usman@usmans.info>

* fix(test): Code formatting

Signed-off-by: Usman Saleem <usman@usmans.info>

---------

Signed-off-by: Usman Saleem <usman@usmans.info>
…idatorPubkey) (hyperledger#7209)

Signed-off-by: Lucas Saldanha <lucascrsaldanha@gmail.com>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Karim Taam <karim.t2am@gmail.com>
* Creates a new issue template for releases

Signed-off-by: Justin Florentine <justin+github@florentine.us>

* aligns release process with wiki docs

Signed-off-by: Justin Florentine <justin+github@florentine.us>

---------

Signed-off-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Remove all EIP-3074 code from Besu. Since EIP-3074 has been replaced
with EIP-7702 in Pectra, and there is no intent to schedule it for a
future fork there is no need to retain the code.

Signed-off-by: Danno Ferrin <danno@numisight.com>
A complete and up to date implementation of EIP-7692 EOF v.1. For genesis 
file activation use "PragueEOFTime", for references tests it activates as 
part of Prague.

Signed-off-by: Danno Ferrin <danno@numisight.com>
* Update Dockerfile

reuse NO_PROXY_CACHE env

Signed-off-by: Enrico Del Fante <enrico.delfante@consensys.net>

* Update Dockerfile

Signed-off-by: Enrico Del Fante <enrico.delfante@consensys.net>

---------

Signed-off-by: Enrico Del Fante <enrico.delfante@consensys.net>
Co-authored-by: Usman Saleem <usman@usmans.info>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
…" (hyperledger#7219)

This reverts commit dfe256e.

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
)

* Don't persist BFT proposed blocks, only committed ones

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Fix unit tests, update copyright

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Update changelog

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

---------

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>
Signed-off-by: Matt Whitehead <matthew.whitehead@kaleido.io>
Signed-off-by: cangqiaoyuzhuo <850072022@qq.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Gabriel-Trintinalia <gabriel.trintinalia@consensys.net>
…erledger#7180)

* Avoid keeping txpool lock during block creation

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update CHANGELOG

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Remove unneeded synchronized

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

---------

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
…perledger#6734)

* Change eth trace from 'Message not expected' to 'Request message'

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Add braces to log message

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Spotless fix

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

---------

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>
Signed-off-by: Matt Whitehead <matthew.whitehead@kaleido.io>
Container security scanning workflow added. This runs on schedule everyday. Also possible to run on-demand for a given image tag

Signed-off-by: Chaminda Divitotawela <cdivitotawela@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
Repository follow standard to use git hash to pin the GitHub actions. Updated the container security scan workflow actions with their git hashes

Signed-off-by: Chaminda Divitotawela <cdivitotawela@gmail.com>
Release workflow publish step was missing the depepndency of artifacts jobs. Due to this reason it could not collect the artifact hashes from the artifacts job. This was introduced in the release workflow consolidation

Signed-off-by: Chaminda Divitotawela <cdivitotawela@gmail.com>
Fail earlier with the initcode size check

Signed-off-by: Danno Ferrin <danno@numisight.com>
Trivy scan result upload to GitHub fails due to permission issue. Added permission security-events=write to the workflow file as a fix. Since workflow permission explicitly defined, it requires contents=read explicity set as well

Signed-off-by: Chaminda Divitotawela <cdivitotawela@gmail.com>
Container verification step in release process automated with the container verify GitHub workflow. New workflow is triggered at the end of the release workflow which will check the release container images starts successfully. Verification test only checks container starts and reach the Ethereum main loop

Signed-off-by: Chaminda Divitotawela <cdivitotawela@gmail.com>
Fix some reasons for chain download halts when syncing

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Signed-off-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Check and test for the unused container rule, and only returncontract
targets can have truncated data rule.
Also test the other subcontainer rules in unit tests.

Signed-off-by: Danno Ferrin <danno@numisight.com>
…#7247)

Signed-off-by: Gabriel-Trintinalia <gabriel.trintinalia@consensys.net>
…rty (hyperledger#7222)

* Add build version option to prefix git hash with custom version property
* Refactor to make appending the git hash a boolean property. Include a commented-out example of how to use the properties in the gradle file

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>
garyschulte and others added 26 commits September 4, 2024 18:31
Signed-off-by: garyschulte <garyschulte@gmail.com>
Signed-off-by: Lucas Saldanha <lucascrsaldanha@gmail.com>
…rs (hyperledger#7539)


Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
…ger#7538)


Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
…AccessList (hyperledger#7502)

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
…yperledger#7576)

* Remove tx from pool when its score is lower than a configured value

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update besu/src/main/java/org/hyperledger/besu/cli/options/TransactionPoolOptions.java

Co-authored-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Check for below min score after the penalization

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

---------

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
* implement engine_getBlobsV1

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Signed-off-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* wrapped WorldUpdater into `EVMWorldupdater` to remove the authority code injection from the implementation of the actual world updaters

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* add CHANGELOG entry

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* first draft for 7702 v2

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* change return value of DelegatedCodeGasCostHelper

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* fix upfront gas cost calculation, fix setting code multiple times in MutableDelegatedCodeAccount

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* fix gas refund for delegated code when account already exists, added gas cost deduction for code delegation resolution to ExtCodeSizeOperation

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* allow accounts with delegated code to send transactions

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* add refund for already existing account after nonce check

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* resolve delegated code only the first time to avoid delegation loops

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* handle invalid authorization signatures properly

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* refactored CodeDelegationProcessor to compute authorizer of a code delegation after the chain id check

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* fix canSetDelegatedCode method by checking code how it is in the trie and not the resolved code

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* optimize code hash calculation for empty code, fix check for empty code delegation list

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* check the all code delegation signatures hava a valid s value

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* rename encoder & decoder, handle invalid signature values in T8nExecutor

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* create the signatures for code delegation authorizations in T8nExecutor without checking if they are valid to test them later during the tx execution

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* check that recid is either 0 or 1

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* fixed acceptance tests, renamed the the remaining instances of set code to code delegation

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* fix delegate encoder & encoder unit tests

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* spotless

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* changed empty check for delegated accounts, fixed test

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* fix hasDelegatedCode method when code is null

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* run acceptance tests without deamon

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* disable CodeDelegationTransactionAcceptanceTest to check if it is causing the stuck ci pipeline

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* check if shouldTransferAllEthOfAuthorizerToSponsor is causing pipeline to stall

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* check if shouldCheckNonceAfterNonceIncreaseOfSender is causing pipeline to stall

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* check if closing the cluster after every test is causing pipeline to stall

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* spotless

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

---------

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Co-authored-by: Danno Ferrin <danno@numisight.com>
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
* wip decoupled parent block header from block creators

---------

Signed-off-by: Justin Florentine <justin+github@florentine.us>
* Added help for setting up Stratum mining

Signed-off-by: ITStarMan100 <danhopeman@outlook.com>

* removed duplication from help message; fixed spotless

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

---------

Signed-off-by: ITStarMan100 <danhopeman@outlook.com>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
…ion (hyperledger#7593)

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
…yperledger#7592)

* feat: exported metrics for current finalized and safe block number

Signed-off-by: Jones Ho <jones.ho@consensys.net>

* Update ethereum/core/src/main/java/org/hyperledger/besu/ethereum/chain/DefaultBlockchain.java

Co-authored-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: jonesho <81145364+jonesho@users.noreply.github.com>

* Update ethereum/core/src/main/java/org/hyperledger/besu/ethereum/chain/DefaultBlockchain.java

Co-authored-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: jonesho <81145364+jonesho@users.noreply.github.com>

* fix: spotless issue

Signed-off-by: Jones Ho <jones.ho@consensys.net>

---------

Signed-off-by: Jones Ho <jones.ho@consensys.net>
Signed-off-by: jonesho <81145364+jonesho@users.noreply.github.com>
Co-authored-by: Fabio Di Fabio <fabio.difabio@consensys.net>
…eMarket (hyperledger#7565)

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
* fix 7702 tx validation in tx pool

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* check that 7702 transactions always have a mandatory to field

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

---------

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Luis Pinto <luis.pinto@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: garyschulte <garyschulte@gmail.com>
Signed-off-by: garyschulte <garyschulte@gmail.com>
* remove unused classes

Signed-off-by: George Tebrean <george@web3labs.com>

* removed duplication from changelog

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

---------

Signed-off-by: George Tebrean <george@web3labs.com>
Signed-off-by: George Tebrean <99179176+gtebrean@users.noreply.github.com>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
… initilization order (hyperledger#7607)

* Change test to tease NPEs during DefaultBlockchain.<init>() caused by initilization order
* fixup! Change test to tease NPEs during DefaultBlockchain.<init>() caused by initilization order
* move counters into their own method

Signed-off-by: Luis Pinto <luis.pinto@consensys.net>
…roBaseFeeMarket (hyperledger#7565)" (hyperledger#7613)

This reverts commit 07adb41.

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
)

* Fix debug_traceCall to handle underpriced transactions

Signed-off-by: 7suyash7 <suyashnyn1@gmail.com>

* remove unused methods

Signed-off-by: 7suyash7 <suyashnyn1@gmail.com>

* Add test case and changelog entry for debug_traceCall fix

Signed-off-by: 7suyash7 <suyashnyn1@gmail.com>

---------

Signed-off-by: 7suyash7 <suyashnyn1@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
…perledger#7580)

Signed-off-by: ITStarMan100 <danhopeman@outlook.com>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
* upcoming breaking change for k8s nat
* log warning if deprecated NAT method used

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

---------

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
@matthew1001 matthew1001 marked this pull request as draft September 17, 2024 14:35
Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

IBFT2 fails block validation with BONSAI DB