Skip to content

Commit

Permalink
Feat/rebasehive (#6)
Browse files Browse the repository at this point in the history
* Update besu branch back to main (ethereum#836)

update besu branch back to main

* cmd/hiveview: limit log bytes in suite view

It was already limited to a certain number of lines, but that's not enough when
the test outputs very long lines.

* simulators/ethereum/engine: Fix ForkID Tests (ethereum#837)

* simulators/ethereum/pyspec: go.mod: geth update

* simulators/ethereum/engine: go.mod: geth update

* simulators/ethereum/engine: ForkID: check no disconnect

* simulators/ethereum/pyspec: fix genesis

* Incorrect comment.

* simulators/ethereum/engine: Add incorrect engine versions tests (ethereum#834)

* simulators/ethereum/engine: use customizer interfaces

* simulators/ethereum/engine: Expect generic methods

* simulators/ethereum/engine: GetPayload generic methods

* simulators/ethereum/engine: CLMock add callback on payload attributes

* simulators/ethereum/engine: GetPayload customizers

* simulators/ethereum/engine: Add tests for incorrect FcU and GetPayload versions

* simulators/ethereum/engine: Add time delta modifier

* simulators/ethereum/engine: Fix Cancun time increments

* simulators/ethereum/engine: Add multiple FcU payload requests on different forks test

* simulators/ethereum/engine: CLMock: add payload id history

* simulators/ethereum/engine: Add payload id check on beacon root change

* simulators/ethereum/engine: Fix test starting directly on Cancun

* configs/cancun: Besu update (ethereum#841)

configs: Besu cancun branch update

* clients/reth: add cancun fork in mapper (ethereum#842)

* clients/erigon: Fix `Dockerfile.git` (ethereum#861)

clients/erigon: fix git dockerfile

* simulators/ethereum/engine: Refactor Legacy Engine API Tests (ethereum#840)

* Refactor previous tests to run on new forks

* Fix configuration on Cancun

* Fix spec fork time

* Run all tests using blob transactions when possible

* Allow blob transaction modifications

* Add more Cancun invalid field tests

* Update expected invalid detection on versioned hashes

* Use multiple source accounts for test transactions by default

* Get full block only when necessary

* Fixes on some of the tests

* Update suggested fee recipient tests

* Update prev-randao tests

* NewPayload Tests Refactor

* Refactor last re-org tests

* Simplify ForkID tests

* Incorrect comment

* Fix pyspec

* More Versioned Hashes Tests

* simulators/ethereum/engine: Beacon root pre-deploy address update (ethereum#839)

simulators/ethereum/engine: Update beacon root to latest spec.

* clients/reth: Dockerfile support to build Reth from git or locally (ethereum#860)

* clients/besu: sh - Remove legacy option from startup flags (ethereum#863)

Remove legacy tx pool option

Signed-off-by: Gabriel-Trintinalia <gabriel.trintinalia@consensys.net>

* simulators/ethereum/pyspec: update execution-spec-tests (ethereum#865)

Updates execution-spec-tests version, which includes the same beacon
root contract address as the existing engine-api tests

* cmd/hiveview: persist column filters in URL (ethereum#864)

* cmd/hiveview: avoid creating history for column filter changes (ethereum#866)

* simulators/ethereum/engine: run tests in parallel (ethereum#869)

sim/eth/engine: run tests in parallel

* hiveproxy: upgrade dependencies

* go.mod: upgrade dependencies

go-ethereum v1.13 no longer provides ethash mining, so this feature
had to be removed from hivechain.

* go.mod: upgrade hiveproxy dependency

* .circleci: use Go 1.21

* .circleci: enable internal static linking

* .circleci: disable libc dependencies in CI build

* simulators/devp2p: upgrade dependencies

* simulators/smoke: upgrade dependencies

* simulators/smoke/genesis: add some new forks

* simulators/ethereum/sync: upgrade dependencies

* simulators/ethereum/rpc: upgrade dependencies

* simulators/ethereum/rpc-compat: upgrade dependencies

* simulators/ethereum/pyspec: upgrade dependencies

* simulators/ethereum/graphql: upgrade dependencies

* simulators/ethereum/consensus: upgrade dependencies

* simulators/ethereum/engine: upgrade dependencies

Also remove the module replacement, since it should not be needed anymore.

* simulators/ethereum: update Go version in go.work file

* simulators/ethereum/engine: go mod tidy

* simulators/ethereum/engine: Add London non-zero-number test (ethereum#867)

* configs/cancun: Update Geth (ethereum#871)

* simulators/ethereum/engine: Update beacon root address: 0xbEAC..7d7a (ethereum#873)

* simulators/ethereum/pyspec: Use hive specific fixture format (ethereum#844)

* simulators/pyspec: Update to use correct fcuVersion.

* simulators/ethereum/pyspec: Refactor to use new fixture format.

* simulators/ethereum/pyspec: Fix logging, tests links and go.sum.

* simulators/ethereum/pyspec: fixes

* simulators/ethereum/pyspec: update fixtures source

---------

Co-authored-by: Mario Vega <marioevz@gmail.com>

* Add reth (ethereum#875)

Add reth to default cancun config

* clients/prysm-vc: Fix docker args

* clients/lodestar-vc: Remove builder option

* clients/teku-bn: Add trusted setup file

* configs/cancun: add all consensus clients

* bump besu (ethereum#882)

* simulators/ethereum: Update beacon root address 0x000F..Beac02 (ethereum#884)

* simulators/ethereum/engine: Update beacon root address 0x000F..Beac02

* simulators/ethereum/pyspec: Download latest fixtures

* simulators/ethereum/engine: Update beacon root contract

* simulators/ethereum/engine: Update History buffer length

Co-authored-by: spencer <spencer.taylor-brown@ethereum.org>

---------

Co-authored-by: Mario Vega <marioevz@gmail.com>
Co-authored-by: spencer <spencer.taylor-brown@ethereum.org>

* simulators/ethereum/engine: rm 0x prefix on 4788 code (ethereum#885)

* clients/erigon: Increase allowed blobcount in remote txs (ethereum#890)

Increase allowed blobcount in remote txs

Erigon now has a built-in spam protector for blob txs, this flag would increase the count of allowed blobs per account from the default 48 to a 1000

* simulators/ethereum/engine: use replaced version of geth with updated beacon root (ethereum#888)

* simulators/ethereum/engine: use replaced version of geth with updated beacon root

* simulators/ethereum/pyspec: use replaced version of geth with updated beacon root

---------

Co-authored-by: Mario Vega <marioevz@gmail.com>

* simulators/ethereum/engine: Engine API Test Fixes (ethereum#891)

* simulators/ethereum/engine: fix non-zero pre-merge fork test

* simulators/ethereum/engine: fix invalid versioned hashes tests

* clients/besu: use ubuntu base image and install java (ethereum#889)

use ubuntu base image and install java

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* simulators/ethereum/engine: Fix several tests (ethereum#886)

* simulators/ethereum/engine: fix versioning tests

* simulators/ethereum/engine: fix transaction replacement

* simulators/ethereum/engine: CLMock: Per-client payload id map

* simulators/ethereum/engine: Unique Payload ID tests in Cancun

* clients/ethereumjs: Remove `transports` parameter (ethereum#892)

* simulators/ethereum/engine: Fix incomplete transactions test (ethereum#893)

* clients/nimbus-el: use admin_nodeInfo instead of net_nodeInfo (ethereum#896)

Most/all EL client seem to use admin_nodeInfo so we adjusted this in nimbus-eth1

* simulators/ethereum/engine: Optional `INVALID` on sync

* simulators/ethereum/engine: Fix invalid nonce test on zero nonce

* simulators/ethereum/engine: Fix invalid Tx tests to use more realistic values

* simulators/ethereum/engine: Remove blob tx on Shanghai genesis, before fork

* simulators/ethereum/engine: Tx Invalid Signature Modifies S

* simulators/ethereum/engine: Update post-merge forks to activate later in pre-merge test (ethereum#901)

simulators/eth/engine: Update pre-merge tests.

* simulators/ethereum/engine: Fix naming, safe/finalized on reorg tests (ethereum#907)

* simulators/ethereum/engine: use generic error check for missing block (ethereum#874)

simulators/ethereum/engine: use generic error check for missing block on rpc

* clients/go-ethereum: disable min free disk check (ethereum#910)

* internal/libhive: return clients in the order defined in the client-file (ethereum#898)

* cmd/hivechain: support for newer forks, more chain outputs (ethereum#909)

* simulators/ethereum/engine,pyspec: Update Trusted Setup (ethereum#915)

* simulators/ethereum/engine: Update geth branch, use mainnet trusted setup

* simulators/ethereum: go.work.sum

* simulators/ethereum/pyspec: Update geth branch

* configs/cancun: Update geth to devnet-10

* simulators/ethereum/engine: Send different blobs depending on the sender

* cmd/hivechain: export chain starting after genesis (ethereum#912)

* cmd/hivechain: write ttd passed to genesis if merged (ethereum#913)

* cmd/hivechain: add forkenv output (ethereum#921)

* cmd/hivechain: add HIVE_NETWORK_ID in forkenv output

* cmd/hivechain: use smaller network ID

jq doesn't like it when numbers are > 64bit

* cmd/hivechain: even smaller chainid

* cmd/hivechain: add headnewpayload output

* cmd/hivechain: track preimages during generation (ethereum#924)

* cmd/hivechain: remove trie/triedb/hashdb package dependency

* Nethermind to use master branch (ethereum#925)

* simulators/ethereum/engine,pyspec: Use `golang:1.20-alpine` (ethereum#926)

* simulators/ethereum/engine: Trim log output of requests (ethereum#928)

* Enable blob tx pool for Nethermind

* simulators/ethereum/engine: Fix engine-cancun fcu version tests (ethereum#902)

* simulators/ethereum/engine: Fix bugs within fcu version tests (Shanghai/Cancun).

* simulators/ethereum/engine: Fix Cancun fcUV2 tests bugs

---------

Co-authored-by: jangko <jangko128@gmail.com>

* clients: only output full genesis file at high log levels (ethereum#927)

* clients: Trim genesis output for low log levels.

* clients: Specific echo for trimmed genesis output.

* clients: remove openethereum as its deprecated (ethereum#932)

* simulators/ethereum/engine: update test names for distinction (ethereum#914)

* simulators: fix typos in comments (ethereum#933)

* simulators/ethereum/engine: Add test to re-org to mempool and back in (ethereum#900)

simulators/ethereum/engine: Add re-org to mempool and back intest

* simulators/ethereum/engine: Minor fix to re-org tests (ethereum#934)

simulators/ethereum/engine: minor fix to re-org tests

* clients/besu: remove HIVE_SKIP_POW

* clients/erigon: remove HIVE_SKIP_POW

* clients/erigon: set terminalTotalDifficultyPassed

* clients/ethereumjs: remove mention of HIVE_SKIP_POW

* clients/go-ethereum: remove -gcmode=archive in import

* clients/nethermind: remove mentions of HIVE_SKIP_POW

* clients/nimbus-el: remove mentions of HIVE_SKIP_POW

* clients/reth: remove mention of HIVE_SKIP_POW

* hivesim: only log test and suite skips at high log levels (ethereum#923)

* graphql: add test case for eip-4844 fields (ethereum#894)

* cmd/hivechain: add ability to create valid clique chain (ethereum#938)

The hivechain tool can now create a signed clique chain. It can be important to have a chain with
actually valid block seal in some test scenarios, and go-ethereum can't mine ethash blocks anymore,
so clique was the only option.

This pulls in a go-ethereum update to get access to an improved GenerateChain implementation.

* simulators/ethereum/sync: use engine API to trigger sync (ethereum#922)

This updates the ethereum/sync simulator to use a newer post-merge chain.
Sync is now triggered via the engine API.

* go.mod: upgrade github.com/docker/docker

* simulators/devp2p: upgrade hivesim

* simulators/smoke: upgrade hivesim

* simulators/ethereum/sync: upgrade hivesim

* simulators/ethereum/rpc: upgrade hivesim

* simulators/ethereum/rpc-compat: upgrade hivesim

* simulators/ethereum/pyspec: upgrade hivesim

* simulators/ethereum/graphql: upgrade hivesim

* simulators/ethereum/engine: upgrade hivesim

* simulators/ethereum/consensus: upgrade hivesim

* simulators/ethereum: remove go.work

Having a Go workspace file ties all the simulators together in terms of their dependency
versions. This is a problem when one of them wants to replace a dependency version. When
building the simulator, hive will not use the workspace file, so it makes little sense to
have it.

* simulators/ethereum/pyspec: module replacement to reference engine sim

* simulators/ethereum/engine: Fix bugs for ForkID and Missing Ancenstor Tests (ethereum#936)

* clients/go-ethereum: use --state.scheme=path

* clients: remove unused HIVE_TESTNET variable

* docs: remove HIVE_SKIP_POW

* cmd/hivechain: disable difficulty-bomb-only forks

* cmd/hivechain: remove HIVE_SKIP_POW in forkenv output

* clients/ethereumjs: verbose RPC logging (ethereum#939)

* clients: add 'glacier' forks in mapper (ethereum#941)

* clients/go-ethereum: remove 'yolo' fork blocks

* clients/ethereumjs: remove 'yolo' forks in mapper

* clients: add missing 'glacier' forks in mapper

* cmd/hivechain: put back difficulty-only forks

This reverts commit f01b2d3.

* simulators/smoke: add 'glacier' forks

* cmd/hivechain: improve uncle generation

* hive: add --sim.randomseed flag (ethereum#943)

* simulators/ethereum/engine: Add randomness seed input (ethereum#944)

* simulators/ethereum/engine: randomness seed input

* simulators/ethereum/engine: Handle zero HIVE_RANDOM_SEED

* clients/besu: Remove trusted setup (ethereum#946)

* cmd/hivechain: add 4788 contract to genesis and set beacon root post-cancun

Co-authored-by: Felix Lange <fjl@twurst.com>

* clients/ethereumjs: add faster local test dockerfile (ethereum#940)

* clients/lodestar-vc: add `--builder` flag (ethereum#949)

* clients/teku-bn: Add rate limit parameters (ethereum#899)

* clients/teku: Add rate limit parameters

* clients/teku-bn: Use hive variable for peer scoring parameters

* simulators/ethereum/engine,pyspec: Upgrade geth branch (ethereum#951)

simulators/ethereum: upgrade geth branch

* configs/cancun: Use go-ethereum master (ethereum#952)

configs/cancun: use geth master

* clients/teku-bn: Remove trusted setup file, add `--ignore-weak-subjectivity-period-enabled` (ethereum#950)

* clients/teku-bn: Remove trusted setup file

* clients/teku-bn: Add `--ignore-weak-subjectivity-period-enabled`

* hivesim: dd documentation generation (ethereum#948)

* clients/erigon, simulators/ethereum/rpc: patch RPC tests for erigon (ethereum#953)

TTDPassed=true would make Erigon consider it POS chain - not applicable for clique or fakepow-based tests

* cmd/hivechain: fix typos in contracts

* cmd/hivechain: add snap test contract in genesis

* clients/nethermind: upgrade to .NET 8 (ethereum#955)

* simulators/devp2p: add support for new eth tests (ethereum#942)


Co-authored-by: lightclient <lightclient@protonmail.com>

* clients/nethermind: update TxPool config (ethereum#956)

Nethermind recently refactored blob pool to support blob transaction reorgs, introducing an additional flag. We decided to consolidate three blob pool-related flags into one enum. This PR enables support for blob transactions + persistent blob storage + blob transactions reorgs.

* clients/erigon: small fix to mapper jq (ethereum#962)

Fix: true instead of 1

* simulators/ethereum/consensus: Fix `excessBlobGas` genesis (ethereum#964)

simulators/ethereum/consensus: Fix excessBlobGas genesis

* clients/erigon: Fix git dockerfile (ethereum#965)

* clients/besu: Fix git dockerfile (ethereum#966)

* all: fix some typos in README.md files (ethereum#968)

* cmd/hivechain: improve modifier schedule

* cmd/hivechain: add more transaction types

* cmd/hivechain: add simple test for generate

* cmd/hivechain: deploy revert contract

* cmd/hivechain: improve spacing in callrevert

* cmd/hivechain: improve deploy tx gaslimit

* cmd/hivechain: avoid OOG in randomcode

Reduce output size. 256 bytes of code cost ~50k gas.

* cmd/hivechain: correct selectors in callrevert

* cmd/hivechain: add blob transaction

* clients/reth: use ghcr.io/paradigmxyz/reth as base image (ethereum#971)

* clients/reth: use ghcr.io/paradigmxyz/reth as base image

* clients/reth: use latest tag

* cmd/hivechain: use calldata hash as log topic in emit contract

* clients/nethermind: Fix Dockerfile.git (ethereum#972)

* simulators/ethereum/rpc-compat: load chain env from tests

* simulators/ethereum/rpc-compat: add explainer in diff message

* clients/nimbus-el: change command line options to use combo HTTP server for RPC (ethereum#973)

* .circleci: Add CI for rust simulators (ethereum#980)

* .circleci: add support for running ci's based off code modified

* .circleci: Add CI for rust simulators

* clients: add portal network client definitions (ethereum#977)

* .ciricleci: fix circleci script after breaking from ethereum#980 (ethereum#982)

* hivesim: add built-in support for connecting to the engine API

We need to do that in a lot more tests now and it gets tiring to redeclare the
token everywhere.

* go.mod: go.mod tidy

* simulators/ethereum/rpc-compat: send forkchoiceUpdated to client (ethereum#983)

* simulators/ethereum/sync: use EngineAPI method of hivesim

* simulators/ethereum/rpc-compat: handle test comments and avoid error message comparison (ethereum#984)

* clients/besu: remove --engine-jwt-enabled

* clients/reth: fix merge netsplit block in mapper (ethereum#986)

* simulators/ethereum/pyspec: Use latest EEST fixture release. (ethereum#985)

* simulators/ethereum/pyspec: Update dockerfile tests release structure.

* simulators/ethereum/pyspec: Tweak to adhere to fixture format change.

* simulators/portal/history/rpc-compat: add portal network history rpc-compat simulator (ethereum#978)

* clients: add 'portal' role

* simulators/portal/history/portal-interop: add portal network history simulator (ethereum#987)

* simulators/portal/history/portal-mesh: add portal network history portal-mesh simulator (ethereum#988)

* simulators/portal/history/trin-bridge: add portal network history trin-bridge simulator (ethereum#989)

* simulators/portal/beacon/rpc-compat: add portal network beacon rpc-compat simulator (ethereum#990)

* clients/prysm-vc: Use REST api (ethereum#906)

* simulators/eth2/engine: Fix error checking (ethereum#897)

* clients/lodestar-bn: Add optional flag to disable peer scoring (ethereum#911)

* clients/erigon: Configure snapshots flag (ethereum#992)

clients/erigon/erigon.sh: Configure snapshots flag.

* clients/go-ethereum: Remove miner gasprice flag (ethereum#993)

clients/go-ethereum/geth.sh: Remove miner gasprice flag.

* simulators/ethereum/engine: Fixes for execution-apis#498 (ethereum#974)

* simulators/ethereum/engine: Fix expected error on fcu tests

* simulators/ethereum/engine: Get Payload delay at spec level

* simulators/ethereum/engine: Correct error code on inconsistent FcU

* simulators/ethereum/engine: Invaild Payload Attributes Expectation Modification

* simulators/portal: refactor simulators to run multiple suites (ethereum#994)

This is just to cut down on the number of Rust projects and Dockerfiles.

* simulators/ethereum/engine: Expect error code for invalid payload attributes tests (ethereum#996)

simulators/ethereum/engine: Expect error code for invalid payload attributes tests.

* clients/lighthouse-bn: Add trusted peers env var (ethereum#908)

* clients/besu: fix merge block in mapper (ethereum#999)

fix fork id problem

Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>

* simulators/ethereum/consensus: add Paris as additional merge fork (ethereum#997)

* simulators/portal/history: trin_bridge suite remove redundant extra client (ethereum#998)

* clients/erigon: Increase blobpool limit in Erigon (ethereum#1000)

* simulators/ethereum/pyspec: Refactor, add sync payload verification (ethereum#995)

* simulators/ethereum/pyspec: Refactor, add sync payload verification

* simulators/ethereum/pyspec: Declare sync timeout constant

* simulators/ethereum/pyspec: Disable timing print

* simulators/ethereum/consensus: add more 'paris' forks

* simulators/portal/history: fix bug where only headers were tested (ethereum#1001)

* clients/reth: make sure pruner doesn't start on tests (ethereum#1002)

* simulators/eth2: Fix Docker golang version (ethereum#1003)

* simulators/eth2/engine: Dockerfile, golang:1.20

* simulators/eth2/testnet: Dockerfile, golang:1.20

* simulators/eth2/withdrawals: Dockerfile, golang:1.20

* clients/portal: update Dockerfiles to use baseimage/tag arg format (ethereum#1005)

---------

Signed-off-by: Gabriel-Trintinalia <gabriel.trintinalia@consensys.net>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: stefan.pingel@consensys.net <stefan.pingel@consensys.net>
Co-authored-by: Parithosh Jayanthi <parithosh.jayanthi@ethereum.org>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Mario Vega <marioevz@gmail.com>
Co-authored-by: Dan Cline <6798349+Rjected@users.noreply.github.com>
Co-authored-by: spencer <spencer@spencertaylorbrown.uk>
Co-authored-by: Gabriel-Trintinalia <gabriel.trintinalia@consensys.net>
Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com>
Co-authored-by: Dan <danceratopz@gmail.com>
Co-authored-by: spencer <spencer.taylor-brown@ethereum.org>
Co-authored-by: Somnath <snb895@outlook.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Kim De Mey <kim.demey@gmail.com>
Co-authored-by: Delweng <delweng@gmail.com>
Co-authored-by: Alexey <me@flcl.me>
Co-authored-by: jangko <jangko128@gmail.com>
Co-authored-by: vuittont60 <81072379+vuittont60@users.noreply.github.com>
Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
Co-authored-by: Jochem Brouwer <jochembrouwer96@gmail.com>
Co-authored-by: lightclient <lightclient@protonmail.com>
Co-authored-by: Marcin Sobczak <77129288+marcindsobczak@users.noreply.github.com>
Co-authored-by: Vehorny <153144728+vehorny@users.noreply.github.com>
Co-authored-by: Martin HS <martin@swende.se>
Co-authored-by: Kolby Moroz Liebl <31669092+KolbyML@users.noreply.github.com>
Co-authored-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Co-authored-by: joshieDo <93316087+joshieDo@users.noreply.github.com>
  • Loading branch information
1 parent e2e6833 commit 47449f6
Show file tree
Hide file tree
Showing 264 changed files with 20,934 additions and 14,823 deletions.
81 changes: 8 additions & 73 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -1,79 +1,14 @@
version: 2.1
orbs:
go: circleci/go@1.5.0

jobs:
# This job builds the hive executable and stores it in the workspace.
build:
docker:
- image: cimg/go:1.19
steps:
# Build it.
- checkout
- go/load-cache
- go/mod-download
- go/save-cache
- run: {command: 'go build -ldflags="-extldflags=-static" .'}
# Store the executable.
- persist_to_workspace:
root: .
paths: ["hive"]

# This job runs the smoke test simulations. This requires a virtual
# machine instead of the container-based build environment because
# hive needs to be able to talk to the docker containers it creates.
smoke-tests:
machine:
image: ubuntu-2004:202201-02
steps:
- checkout
- attach_workspace: {at: "/tmp/build"}
- run:
command: "/tmp/build/hive --sim=smoke/genesis --client=go-ethereum"
- run:
command: "/tmp/build/hive --sim=smoke/network --client=go-ethereum"
setup: true

# This job also runs the smoke test simulations, but against a remote dockerd.
smoke-tests-remote-docker:
docker:
- image: cimg/base:2022.04
steps:
- checkout
- attach_workspace: {at: "/tmp/build"}
- setup_remote_docker: {version: 20.10.14}
- run:
command: "/tmp/build/hive --sim=smoke/genesis --client=go-ethereum --loglevel 5"
- run:
command: "/tmp/build/hive --sim=smoke/network --client=go-ethereum --loglevel 5"

# This job runs the go unit tests.
go-test:
docker:
- image: cimg/go:1.19
steps:
# Get the source.
- checkout
- go/load-cache
- go/mod-download
- go/save-cache
# Run the tests.
- run:
name: "hive module tests"
command: "go test -cover ./..."
- run:
name: "hiveproxy module tests"
command: "go test -cover ./..."
working_directory: "./hiveproxy"
- run:
name: "Compile Go simulators"
command: ".circleci/compile-simulators.sh"
orbs:
path-filtering: circleci/path-filtering@0.0.1

workflows:
main:
setup-workflow:
jobs:
- go-test
- build
- smoke-tests:
requires: ["build"]
- smoke-tests-remote-docker:
requires: ["build"]
- path-filtering/filter:
mapping: |
simulators/portal/.* rust-ci true
base-revision: origin/master
106 changes: 106 additions & 0 deletions .circleci/continue_config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
version: 2.1
orbs:
go: circleci/go@1.5.0

parameters:
rust-ci:
type: boolean
default: false

jobs:
# This job builds the hive executable and stores it in the workspace.
build:
docker:
- image: cimg/go:1.21
steps:
# Build it.
- checkout
- go/load-cache
- go/mod-download
- go/save-cache
- run: {command: 'go build -ldflags="-s -extldflags=-static" -tags "osusergo netgo static_build" .'}
# Store the executable.
- persist_to_workspace:
root: .
paths: ["hive"]

# This job runs the smoke test simulations. This requires a virtual
# machine instead of the container-based build environment because
# hive needs to be able to talk to the docker containers it creates.
smoke-tests:
machine:
image: ubuntu-2004:202201-02
steps:
- checkout
- attach_workspace: {at: "/tmp/build"}
- run:
command: "/tmp/build/hive --sim=smoke/genesis --client=go-ethereum"
- run:
command: "/tmp/build/hive --sim=smoke/network --client=go-ethereum"

# This job also runs the smoke test simulations, but against a remote dockerd.
smoke-tests-remote-docker:
docker:
- image: cimg/base:2022.04
steps:
- checkout
- attach_workspace: {at: "/tmp/build"}
- setup_remote_docker: {version: 20.10.14}
- run:
command: "/tmp/build/hive --sim=smoke/genesis --client=go-ethereum --loglevel 5"
- run:
command: "/tmp/build/hive --sim=smoke/network --client=go-ethereum --loglevel 5"

# This job runs the go unit tests.
go-test:
docker:
- image: cimg/go:1.21
steps:
# Get the source.
- checkout
- go/load-cache
- go/mod-download
- go/save-cache
# Run the tests.
- run:
name: "hive module tests"
command: "go test -cover ./..."
- run:
name: "hiveproxy module tests"
command: "go test -cover ./..."
working_directory: "./hiveproxy"
- run:
name: "Compile Go simulators"
command: ".circleci/compile-simulators.sh"
# this makes sure the rust code is good
rust-simulators:
docker:
- image: cimg/rust:1.75.0
steps:
- checkout
- run:
name: Install rustfmt
command: rustup component add rustfmt
- run:
name: Install Clippy
command: rustup component add clippy
- run:
name: Install Clang
command: sudo apt update && sudo apt-get install clang -y
- run:
name: "Lint, build, test Rust simulators"
command: ".circleci/rust-simulators.sh"

workflows:
main:
jobs:
- go-test
- build
- smoke-tests:
requires: ["build"]
- smoke-tests-remote-docker:
requires: ["build"]
rust-jobs:
when: << pipeline.parameters.rust-ci >>
jobs:
- rust-simulators
16 changes: 16 additions & 0 deletions .circleci/rust-simulators.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash

# This causes the bash script to exit immediately if any commands errors out
set -e

failed=""
sims=$(find simulators -name Cargo.toml)
for d in $sims; do
d="$(dirname "$d")"
echo "Lint, build, test $d"
( cd "$d" || exit 1;
cargo fmt --all -- --check;
cargo clippy --all --all-targets --all-features --no-deps -- --deny warnings;
cargo test --workspace -- --nocapture;
)
done
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,7 @@ workspace
.idea/
# build output
/hive

# build output for rust simulators files
simulators/**/Cargo.lock
simulators/**/target
1 change: 0 additions & 1 deletion clients/besu/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ COPY genesis.json /genesis.json
COPY mapper.jq /mapper.jq
COPY besu.sh /opt/besu/bin/besu-hive.sh
COPY enode.sh /hive-bin/enode.sh
COPY trusted_setup.txt /trusted_setup.txt

# Set execute permissions for scripts
RUN chmod +x /opt/besu/bin/besu-hive.sh /hive-bin/enode.sh
Expand Down
15 changes: 9 additions & 6 deletions clients/besu/Dockerfile.git
Original file line number Diff line number Diff line change
@@ -1,24 +1,28 @@
### Build Besu From Git:

## Builder stage: Compiles besu from a git repository
FROM openjdk:17-jdk-slim as builder
FROM ubuntu:22.04 as builder

ARG tag=main
ARG github=hyperledger/besu

RUN echo "Cloning: $github - $tag" && \
apt-get update && apt-get install -y git libsodium-dev libnss3-dev \
RUN echo "installing java on ubuntu base image" \
&& apt-get update && apt-get install -y git libsodium-dev libnss3-dev \
&& apt-get install --no-install-recommends -q --assume-yes ca-certificates-java=20190909* \
&& apt-get install --no-install-recommends -q --assume-yes openjdk-17-jre-headless=17* libjemalloc-dev=5.* \
&& echo "Cloning: $github - $tag" \
&& git clone --depth 1 --branch $tag https://github.com/$github \
&& cd besu && ./gradlew installDist

## Final stage: Sets up the environment for running besu
FROM openjdk:17-jdk-slim
FROM ubuntu:22.04

# Copy compiled binary from builder
COPY --from=builder /besu/build/install/besu /opt/besu

RUN apt-get update && apt-get install -y curl jq libsodium23 libnss3-dev \
&& apt-get clean && rm -rf /var/lib/apt/lists/*
&& apt-get install --no-install-recommends -q --assume-yes openjdk-17-jre-headless=17* libjemalloc-dev=5.* \
&& apt-get clean && rm -rf /var/lib/apt/lists/*

# Create version.txt
RUN /opt/besu/bin/besu --version > /version.txt
Expand All @@ -28,7 +32,6 @@ COPY genesis.json /genesis.json
COPY mapper.jq /mapper.jq
COPY besu.sh /opt/besu/bin/besu-hive.sh
COPY enode.sh /hive-bin/enode.sh
COPY trusted_setup.txt /trusted_setup.txt

# Set execute permissions for scripts
RUN chmod +x /opt/besu/bin/besu-hive.sh /hive-bin/enode.sh
Expand Down
9 changes: 6 additions & 3 deletions clients/besu/Dockerfile.local
Original file line number Diff line number Diff line change
@@ -1,22 +1,26 @@
### Build Besu Locally:

## Builder stage: Compiles besu from a local directory
FROM openjdk:17-jdk-slim as builder
FROM ubuntu:22.04 as builder

# Default local client path: clients/besu/<besu>
ARG local_path=besu
COPY $local_path besu

RUN apt-get update && apt-get install -y git libsodium-dev libnss3-dev \
&& apt-get install --no-install-recommends -q --assume-yes ca-certificates-java=20190909 \
&& apt-get install --no-install-recommends -q --assume-yes openjdk-17-jre-headless=17* libjemalloc-dev=5.* \
&& cd besu && ./gradlew installDist

## Final stage: Sets up the environment for running besu
FROM openjdk:17-jdk-slim
FROM ubuntu:22.04

# Copy compiled binary from builder
COPY --from=builder /besu/build/install/besu /opt/besu

RUN apt-get update && apt-get install -y curl jq libsodium23 libnss3-dev \
&& apt-get install --no-install-recommends -q --assume-yes ca-certificates-java=20190909 \
&& apt-get install --no-install-recommends -q --assume-yes openjdk-17-jre-headless=17* libjemalloc-dev=5.* \
&& apt-get clean && rm -rf /var/lib/apt/lists/*

# Create version.txt
Expand All @@ -27,7 +31,6 @@ COPY genesis.json /genesis.json
COPY mapper.jq /mapper.jq
COPY besu.sh /opt/besu/bin/besu-hive.sh
COPY enode.sh /hive-bin/enode.sh
COPY trusted_setup.txt /trusted_setup.txt

# Set execute permissions for scripts
RUN chmod +x /opt/besu/bin/besu-hive.sh /hive-bin/enode.sh
Expand Down
33 changes: 17 additions & 16 deletions clients/besu/besu.sh
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,11 @@
#
# - HIVE_MINER enables mining. value is coinbase.
# - HIVE_MINER_EXTRA extra-data field to set for newly minted blocks
# - HIVE_SKIP_POW If set, skip PoW verification
# - HIVE_LOGLEVEL Client log level
# - HIVE_GRAPHQL_ENABLED If set, GraphQL is enabled on port 8545 and RPC is disabled
#
# These flags are not supported by the Besu hive client
#
# - HIVE_TESTNET whether testnet nonces (2^20) are needed
# - HIVE_FORK_DAO_VOTE whether the node support (or opposes) the DAO fork

set -e
Expand All @@ -63,9 +61,19 @@ esac
FLAGS="--logging=$LOG --data-storage-format=BONSAI"

# Configure the chain.
jq -f /mapper.jq /genesis.json > /besugenesis.json
echo -n "Genesis: "; cat /besugenesis.json
FLAGS="$FLAGS --genesis-file=/besugenesis.json "
mv /genesis.json /genesis-input.json
jq -f /mapper.jq /genesis-input.json > /genesis.json
FLAGS="$FLAGS --genesis-file=/genesis.json "

# Dump genesis.
if [ "$HIVE_LOGLEVEL" -lt 4 ]; then
echo "Supplied genesis state (trimmed, use --sim.loglevel 4 or 5 for full output):"
jq 'del(.alloc[] | select(.balance == "0x123450000000000000000"))' /genesis.json
else
echo "Supplied genesis state:"
cat /genesis.json
fi


# Enable experimental 'berlin' hard-fork features if configured.
#if [ -n "$HIVE_FORK_BERLIN" ]; then
Expand All @@ -75,10 +83,8 @@ FLAGS="$FLAGS --genesis-file=/besugenesis.json "
# The client should start after loading the blocks, this option configures it.
IMPORTFLAGS="--run"

# Disable PoW check if requested.
if [ -n "$HIVE_SKIP_POW" ]; then
IMPORTFLAGS="$IMPORTFLAGS --skip-pow-validation-enabled"
fi
# Skip PoW checks on import.
IMPORTFLAGS="$IMPORTFLAGS --skip-pow-validation-enabled"

# Load chain.rlp if present.
if [ -f /chain.rlp ]; then
Expand Down Expand Up @@ -113,7 +119,7 @@ fi
if [ "$HIVE_MINER_EXTRA" != "" ]; then
FLAGS="$FLAGS --miner-extra-data=$HIVE_MINER_EXTRA"
fi
FLAGS="$FLAGS --min-gas-price=1 --tx-pool-price-bump=0 --tx-pool-limit-by-account-percentage=1"
FLAGS="$FLAGS --min-gas-price=1 --tx-pool-price-bump=0"

# Configure peer-to-peer networking.
if [ "$HIVE_BOOTNODE" != "" ]; then
Expand Down Expand Up @@ -150,12 +156,7 @@ RPCFLAGS="$RPCFLAGS --rpc-ws-enabled --rpc-ws-api=DEBUG,ETH,NET,WEB3,ADMIN --rpc
# Enable merge support if needed
if [ "$HIVE_TERMINAL_TOTAL_DIFFICULTY" != "" ]; then
echo "0x7365637265747365637265747365637265747365637265747365637265747365" > /jwtsecret
RPCFLAGS="$RPCFLAGS --engine-host-allowlist=* --engine-jwt-enabled --engine-jwt-secret /jwtsecret"
fi

# Enable KZG trusted setup if cancun timestamp is set, needed for custom genesis on Besu wtih Cancun
if [ "$HIVE_CANCUN_TIMESTAMP" != "" ]; then
FLAGS="$FLAGS --kzg-trusted-setup=/trusted_setup.txt"
RPCFLAGS="$RPCFLAGS --engine-host-allowlist=* --engine-jwt-secret /jwtsecret"
fi

# Start Besu.
Expand Down
4 changes: 3 additions & 1 deletion clients/besu/mapper.jq
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,9 @@ def to_int:
"muirGlacierBlock": env.HIVE_FORK_MUIR_GLACIER|to_int,
"berlinBlock": env.HIVE_FORK_BERLIN|to_int,
"londonBlock": env.HIVE_FORK_LONDON|to_int,
"parisBlock": env.HIVE_MERGE_BLOCK_ID|to_int,
"arrowGlacierBlock": env.HIVE_FORK_ARROW_GLACIER|to_int,
"grayGlacierBlock": env.HIVE_FORK_GRAY_GLACIER|to_int,
"mergeNetsplitBlock": env.HIVE_MERGE_BLOCK_ID|to_int,
"terminalTotalDifficulty": env.HIVE_TERMINAL_TOTAL_DIFFICULTY|to_int,
"shanghaiTime": env.HIVE_SHANGHAI_TIMESTAMP|to_int,
"cancunTime": env.HIVE_CANCUN_TIMESTAMP|to_int,
Expand Down
Loading

0 comments on commit 47449f6

Please sign in to comment.