refactor: move to revm #918

onbjerg · 2022-03-12T20:32:23Z

Ports our testing backend to REVM. This PR simplifies some code, removes the evm-adapters crate, and generally increases testing speed.

More specifically the following parts have been refactored:

The EVM executor
Cheatcodes
Call tracing
The debugger
The test runner (some clean up in ContractRunner/MultiContractRunner)
Decoding of logs/traces etc.
Gas reports

The following new features were implemented:

forge test --debug (see REVM debugger #920 (comment))
forge run can now run functions that take arguments (see REVM debugger #920 (comment))
Fuzzing dictionary see (REVM fuzz dictionary #985)
Data from forked endpoints is now optionally cached (see feat: add support for storage caching #1006)

The following things were modified:

The debugger had some minor changes to text, such as titles
Zero bytes in the stack and memory view are now greyed out

The following bugs were squashed:

The following tests have been added/modified:

geb was re-enabled in our integration tests
Lil' Web3 was added to our integration tests
Maple Labs' Loan was added to our integration tests
We now have a common set of Solidity tests we can test against. The idea is that we expand this suite to include every aspect of Foundry that touches Solidity files, including the linter/formatter and new test types (see Cross-crate testdata #965)

Tasks

Blocking issues:

Tracing inspector #753 (REVM tracing and gas reports #867)
Port gas reports #775 (REVM tracing and gas reports #867)
Debugging inspector #758 (REVM debugger #920)
Port forge run #761 (REVM debugger #920)
Tracking issue for new features that need to be ported: Re-implement new features for REVM #799

Things that would be nice to address:

Things that would be nice to add:

4byte trace identifier
ethers-addressbook trace identifier (feat(config): add static address label registry #640)
Etherscan trace identifier (feat(forge): stack traces grab contracts from etherscan when forking #447)
Common Solidity tests in root of project: currently some crates duplicate tests or reference tests in other crates (https://github.com/onbjerg/foundry-test)
forge test --debug

Checklist before merge:

Compare gas usage between the port and master
Compare trace decoding between the port and master
Link all fixes issues
Create issues in the Foundry Book repository with updates
Create issues for unresolved stretch tasks

Everything in the PR has been tested against https://github.com/onbjerg/foundry-test and solmate

gakonst · 2022-03-13T16:15:31Z

I think the Foundry-Test repo would be nice to get in the main PR and have MultiContract runner run against it, as it gets us a lot of confidence about the safety of the code. Otherwise agree that Etherscan/4Byte can happen in separate PR, as long as we believe that the current architecture is generic enough to support batching most of the RPC requests for signatures/abis etc so we don't have big performance overheads

onbjerg · 2022-03-13T20:35:10Z

Current rough benchmarks. For reference, solmate is currently 186 fuzz tests and 209 unit tests

Tracing off (256 fuzz runs)

Full tracing on (256 fuzz runs)

gakonst · 2022-03-17T22:53:08Z

Turns out compiling with RUSTFLAGS="-C target-cpu=native" cargo build --release --bin forge yields non-trivial speed increase, on the order of ~10%

onbjerg · 2022-03-18T00:04:25Z

Ok so this is pretty much good to go with the exception of a memory allocation bug in REVM that I will address tomorrow. Gonna take tomorrow to work on this for the final time before merge, will try to squeeze in some optimizations in REVM and #731 / #849 / an Etherscan decoder if possible

REVM tasks:

Memory allocation bug: some fuzz tests make REVM think it needs to allocate a few petabytes (fix: impose a memory limit bluealloy/revm#86)
REVM analysis optimization: REVM does a contract analysis step every time a call is made which in our case adds up to a lot of time, since for fuzz tests we (by default) run the tests with the same contracts 256 times. We can at a minimum save 255 redundant analysis steps per fuzz test (more if there are more contracts involved)
REVM inspector log hook: We currently semi-duplicate some logic from REVM to peek into the state of the interpreter and see if there is a log we can grab. We need to do this to get logs even if we revert - we can probably upstream a hook for inspectors that collect logs instead. Turns out the extract_log function in this port takes a considerable amount of cycles for duplicate work (about 5% in my flamegraphs running against solmate) (feat: Inspector::log bluealloy/revm#85)

Stretch goals:

Fuzz dictionary (feat(forge): fuzz dictionary #731)
Invariant tests (feat(forge): invariant tests #849)
Etherscan decoder

onbjerg · 2022-03-21T02:58:59Z

Going to fix the failing tests, otherwise this is good to go

* refactor: nuke `evm-adapters` * refactor: simple revm test runner Current features: - Can run unit tests - Works with both revert-type tests and DSTest-type tests - Collects logs, albeit not for reverting tests - Integrated with config and CLI flags Disabled features: - Gas reports - Tracing - Cheatcodes - Fuzzing - Log decoding - Forking mode - Hardhat-style `console.log`, since those require us to decode calls to a specific address (HH does not emit logs) - The debugger In addition to this, I've disabled some tests that could never pass under the current circumstances, but that should be adjusted and re-enabled when their respective features are implemented (such as fuzz tests) * refactor: adjust CLI to new runner API * feat: log collector inspector * feat: hardhat logs * chore: lint * refactor: extract hh log converter to helper fn * refactor: return single test result if setup fails * build: use upstream revm chore: renuke `evm-adapters`

* feat: add `InspectorStack` Adds `InspectorStack`, an inspector that calls a stack of other inspectors sequentially. Closes #752 * feat: port cheatcodes to revm * feat: port `expectCall` cheatcode * feat: extract labels from cheatcode inspector * feat: port `expectEmit` cheatcode * refactor: move log decoding into `forge` crate * chore: remove unused evm patch * test: re-enable debug logs test * fix: record reads on `SSTORE` ops * refactor: rename `record` to `start_record` * docs: clarify why `DUMMY_CALL_OUTPUT` is 320 bytes * fix: handle `expectRevert` with no return data * build: bump revm * chore: remove outdated todo * refactor: use static dispatch in `InspectorStack` * build: use k256 * fix: make gas usage not so crazy

* feat: copy-paste old forking provider * feat(fork): convert to REVM traits * chore: remove unnecessary codehash handler * feat: impl Database for shared backend * chore: fix tests * chore: fmt * fix(fork): correctly convert H256 <> U256 for storage * refactor: separate storage from accounts in cache * feat(fork): fetch block hashes * chore: remove unused DB parameter * test: add test for block hashes * feat: add forked backend to executor builder * feat(cli): set fork url on the executor * refactor: move shared backend to separate file * feat(fork): add fn for instantiating forked env * feat(cli): allow pinning block number * fix(fork): install missing listeners * feat(fork): instantiate environment with forked state * fix: use a CALLER address with maxed out balance for calls this is required because in forking mode otherwise the account wont have enough balance to transact * chore: fmt Co-authored-by: Oliver Nordbjerg <hi@notbjerg.me>

* feat: very simple traces * feat: creation traces * feat: setup and revert traces * fix: fix lib addresses * refactor: simplify tracer inspector * fix: fill traces in correct order * build: bump revm * fix: get code for newly created contracts * refactor: unify log extraction logic * feat: trace logs * refactor: unify labels and names * refactor: return string from trace Instead of passing in an empty string we then pass around inside the trace display logic, we just return strings where appropriate. * refactor: remove identified contracts * refactor: remove unused vars * refactor: simplify `construct_func_call` * refactor: name special characters in traces * refactor: rework all display logic * feat: first pass identify/decode for traces * refactor: move tracing to own module * refactor: simplify `test` * feat: traces for fuzz tests * fix: make fuzz revert reasons less verbose * feat: port gas reports * refactor: small readability nits * feat: run fuzz *and* unit tests in parallel Previously we would run each test contract in parallel, but within each `ContractRunner` we would run unit tests first (in parallel) and then fuzz tests (in parallel). * refactor: move colouring logic to its own function * fix: test contract identification We now include three kinds of traces that are used for identification of contracts: - Deployment traces: these are the initial deployments of the test contract and libraries - Setup traces: these are traces of calls to the `setUp` function - Execution traces: these are the traces of calls to the test contract itself * fix: mark setup trace as a setup trace * fix: get correct nonce in tracer * fix: log extraction outside of current memory * chore: clean up complex types * chore: remove outdated comment * fix: make tests compile * fix: add missing test filter function * feat: display full address in traces * fix: color "new" keyword in traces * fix: filter out `console.log` calls from traces * chore: remove unnecessary comment * feat: add gas cost to creation traces * fix: properly decode outputs * refactor: destructure `TestSetup` in test funcs * fix: ignore address for func output decoding * fix: fix expect emit Co-authored-by: Georgios Konstantopoulos <me@gakonst.com> Co-authored-by: brockelmore <brockelmore@users.noreply.github.com>

onbjerg · 2022-03-22T10:03:09Z

Just noticed SSTORE2Test::testFailReadInvalidPointer(address) in Solmate will correctly revert but sometimes still be marked as failed. Investigating

Edit: Ok, figured it out. Seems like they are supposed to fail, but because of the way proptest works the counterexample + logs + traces do not match the actual failure, they match the next call. proptest will try to run at least one more case to find the minimal failure case

mattsse

couple nits, not necessarily introduced in this PR.

we should consider cleaning up some Test/TestRunner related code but would recommend doing this after this is merged.

evm/src/executor/abi.rs

evm/src/executor/inspector/cheatcodes/expect.rs

evm/src/executor/inspector/stack.rs

evm/src/executor/inspector/tracer.rs

evm/src/executor/mod.rs

evm/src/fuzz/mod.rs

* fix: report correct fuzz failure case * docs: improve some docs in fuzzer

* Simple REVM test runner (#788) * refactor: nuke `evm-adapters` * refactor: simple revm test runner Current features: - Can run unit tests - Works with both revert-type tests and DSTest-type tests - Collects logs, albeit not for reverting tests - Integrated with config and CLI flags Disabled features: - Gas reports - Tracing - Cheatcodes - Fuzzing - Log decoding - Forking mode - Hardhat-style `console.log`, since those require us to decode calls to a specific address (HH does not emit logs) - The debugger In addition to this, I've disabled some tests that could never pass under the current circumstances, but that should be adjusted and re-enabled when their respective features are implemented (such as fuzz tests) * refactor: adjust CLI to new runner API * feat: log collector inspector * feat: hardhat logs * chore: lint * refactor: extract hh log converter to helper fn * refactor: return single test result if setup fails * build: use upstream revm chore: renuke `evm-adapters` * REVM fuzzer (#789) * REVM cheatcodes (#841) * feat: add `InspectorStack` Adds `InspectorStack`, an inspector that calls a stack of other inspectors sequentially. Closes #752 * feat: port cheatcodes to revm * feat: port `expectCall` cheatcode * feat: extract labels from cheatcode inspector * feat: port `expectEmit` cheatcode * refactor: move log decoding into `forge` crate * chore: remove unused evm patch * test: re-enable debug logs test * fix: record reads on `SSTORE` ops * refactor: rename `record` to `start_record` * docs: clarify why `DUMMY_CALL_OUTPUT` is 320 bytes * fix: handle `expectRevert` with no return data * build: bump revm * chore: remove outdated todo * refactor: use static dispatch in `InspectorStack` * build: use k256 * fix: make gas usage not so crazy * feat(revm): add forking mode (#835) * feat: copy-paste old forking provider * feat(fork): convert to REVM traits * chore: remove unnecessary codehash handler * feat: impl Database for shared backend * chore: fix tests * chore: fmt * fix(fork): correctly convert H256 <> U256 for storage * refactor: separate storage from accounts in cache * feat(fork): fetch block hashes * chore: remove unused DB parameter * test: add test for block hashes * feat: add forked backend to executor builder * feat(cli): set fork url on the executor * refactor: move shared backend to separate file * feat(fork): add fn for instantiating forked env * feat(cli): allow pinning block number * fix(fork): install missing listeners * feat(fork): instantiate environment with forked state * fix: use a CALLER address with maxed out balance for calls this is required because in forking mode otherwise the account wont have enough balance to transact * chore: fmt Co-authored-by: Oliver Nordbjerg <hi@notbjerg.me> * chore: fmt * REVM tracing and gas reports (#867) * feat: very simple traces * feat: creation traces * feat: setup and revert traces * fix: fix lib addresses * refactor: simplify tracer inspector * fix: fill traces in correct order * build: bump revm * fix: get code for newly created contracts * refactor: unify log extraction logic * feat: trace logs * refactor: unify labels and names * refactor: return string from trace Instead of passing in an empty string we then pass around inside the trace display logic, we just return strings where appropriate. * refactor: remove identified contracts * refactor: remove unused vars * refactor: simplify `construct_func_call` * refactor: name special characters in traces * refactor: rework all display logic * feat: first pass identify/decode for traces * refactor: move tracing to own module * refactor: simplify `test` * feat: traces for fuzz tests * fix: make fuzz revert reasons less verbose * feat: port gas reports * refactor: small readability nits * feat: run fuzz *and* unit tests in parallel Previously we would run each test contract in parallel, but within each `ContractRunner` we would run unit tests first (in parallel) and then fuzz tests (in parallel). * refactor: move colouring logic to its own function * fix: test contract identification We now include three kinds of traces that are used for identification of contracts: - Deployment traces: these are the initial deployments of the test contract and libraries - Setup traces: these are traces of calls to the `setUp` function - Execution traces: these are the traces of calls to the test contract itself * fix: mark setup trace as a setup trace * fix: get correct nonce in tracer * fix: log extraction outside of current memory * chore: clean up complex types * chore: remove outdated comment * fix: make tests compile * fix: add missing test filter function * feat: display full address in traces * fix: color "new" keyword in traces * fix: filter out `console.log` calls from traces * chore: remove unnecessary comment * feat: add gas cost to creation traces * fix: properly decode outputs * refactor: destructure `TestSetup` in test funcs * fix: ignore address for func output decoding * fix: fix expect emit Co-authored-by: Georgios Konstantopoulos <me@gakonst.com> Co-authored-by: brockelmore <brockelmore@users.noreply.github.com> * REVM debugger (#920) * feat: port debugger data structures * feat: initial port of `ui` crate * chore: add `ui` crate as a workspace member * refactor: adjust ui contract identification * feat: grey out 0 values in debugger memory Closes #902 * style: minor debugger ui beautification * feat: better stack display in debugger ui * feat: gray out zero bytes in stack view * feat: debugger inspector * refactor: minor code cleanup * feat: port `forge run` * fix: temp fix for failing `DsTest.sol` include * chore: fix lints * test: adjust `forge run` tests * refactor: use simple bool for revert checks * chore: remove unused display impl * chore: remove unused comment * fix: display number of stack items in ui * docs: prettify cli help for some commands * feat: `forge test --debug` * refactor: `get_create_address` util * refactor: `InspectorData` * docs: more detailed err for `forge test --debug` * feat: support hardhat artifacts in `vm.getCode` (#956) Ports #903 * REVM: FFI cheatcode updates (#955) * feat: only strip 0x in ffi output if present Ports #904 * Update forge/src/executor/inspector/cheatcodes/ext.rs Co-authored-by: Georgios Konstantopoulos <me@gakonst.com> * REVM gas fixes (#950) * feat: account for gas refunds * refactor: merge `call_raw` and committing variant * fix: actually use refund quotient * feat: strip tx gas stipend * fix: fix reported gas usage in debugger * build: use upstream revm * test: adjust `forge run` gas values in tests * chore: remove unused copy * chore: add note on push maths * feat: make stipend reduction optional * fix: remove tx stipend in `forge run` * REVM: Pull EVM executor into own crate (#961) * refactor: move evm executor to own crate * refactor: `evm::executor::fuzz` -> `evm::fuzz` * refactor: `evm::debugger` -> `evm::debug` * test: fix multi runner test * feat: better ux for expect revert without reason (#962) * Cross-crate testdata (#965) * feat: cross-crate shared testdata * refactor: move `foundry-utils` to common tests * fix: fix getcode test * fix: compile once in tests * fix: fix prank cheatcode (#973) Correctly apply `msg.sender` prank to both transfers and calls. * fix: prank depth math * test: fix lib linking test * refactor: use revm `log` hook (#984) * refactor: use revm `log` hook * chore: bump revm Co-authored-by: Georgios Konstantopoulos <me@gakonst.com> * test: add lil-web3 to integration tests * test: add maple labs loans to integration tests Closes #959 * REVM fuzz dictionary (#985) * feat: fuzz dictionary Co-authored-by: brockelmore <31553173+brockelmore@users.noreply.github.com> * fix: handle malformed bytecode * fix: limit search for push bytes * feat: collect fuzz state from logs * feat: build initial fuzz state from db * perf: use `Index` instead of `Selector` Co-authored-by: brockelmore <31553173+brockelmore@users.noreply.github.com> * feat(config): add caching settings * feat: add none option * feat: add foundry data dir * feat: add storage map support * bump ethers * chore(clippy): make clippy happy * refactor: diskmap * feat: add rpc caching support * feat: add no storage cache option * refactor: rename cnfig value * docs: more storage caching docs * fix: with config builder function * refactor: address review Co-authored-by: Bjerg <onbjerg@users.noreply.github.com> Co-authored-by: Georgios Konstantopoulos <me@gakonst.com> Co-authored-by: Oliver Nordbjerg <hi@notbjerg.me> Co-authored-by: brockelmore <brockelmore@users.noreply.github.com> Co-authored-by: brockelmore <31553173+brockelmore@users.noreply.github.com>

mattsse

can't spot anything obvious anymore.
let's fix the things we missed once they come in

gakonst

Ok let's do it - needs a follow up on the caching to work with account nonces/code/balance etc, and we also want the etherscan decoder.

Monster PR big kudos

* feat(cache): proper json cache * refactor: use new db types * chore(clippy): make clippy happy * bump revm * docs: some docs * refactor: extend Fork type * remove diskmap types * test: refactor tests * remove sharedmemcache * add tests * more tracing * chore(clippy): make clippy happy

* Simple REVM test runner (foundry-rs#788) * refactor: nuke `evm-adapters` * refactor: simple revm test runner Current features: - Can run unit tests - Works with both revert-type tests and DSTest-type tests - Collects logs, albeit not for reverting tests - Integrated with config and CLI flags Disabled features: - Gas reports - Tracing - Cheatcodes - Fuzzing - Log decoding - Forking mode - Hardhat-style `console.log`, since those require us to decode calls to a specific address (HH does not emit logs) - The debugger In addition to this, I've disabled some tests that could never pass under the current circumstances, but that should be adjusted and re-enabled when their respective features are implemented (such as fuzz tests) * refactor: adjust CLI to new runner API * feat: log collector inspector * feat: hardhat logs * chore: lint * refactor: extract hh log converter to helper fn * refactor: return single test result if setup fails * build: use upstream revm chore: renuke `evm-adapters` * REVM fuzzer (foundry-rs#789) * REVM cheatcodes (foundry-rs#841) * feat: add `InspectorStack` Adds `InspectorStack`, an inspector that calls a stack of other inspectors sequentially. Closes foundry-rs#752 * feat: port cheatcodes to revm * feat: port `expectCall` cheatcode * feat: extract labels from cheatcode inspector * feat: port `expectEmit` cheatcode * refactor: move log decoding into `forge` crate * chore: remove unused evm patch * test: re-enable debug logs test * fix: record reads on `SSTORE` ops * refactor: rename `record` to `start_record` * docs: clarify why `DUMMY_CALL_OUTPUT` is 320 bytes * fix: handle `expectRevert` with no return data * build: bump revm * chore: remove outdated todo * refactor: use static dispatch in `InspectorStack` * build: use k256 * fix: make gas usage not so crazy * feat(revm): add forking mode (foundry-rs#835) * feat: copy-paste old forking provider * feat(fork): convert to REVM traits * chore: remove unnecessary codehash handler * feat: impl Database for shared backend * chore: fix tests * chore: fmt * fix(fork): correctly convert H256 <> U256 for storage * refactor: separate storage from accounts in cache * feat(fork): fetch block hashes * chore: remove unused DB parameter * test: add test for block hashes * feat: add forked backend to executor builder * feat(cli): set fork url on the executor * refactor: move shared backend to separate file * feat(fork): add fn for instantiating forked env * feat(cli): allow pinning block number * fix(fork): install missing listeners * feat(fork): instantiate environment with forked state * fix: use a CALLER address with maxed out balance for calls this is required because in forking mode otherwise the account wont have enough balance to transact * chore: fmt Co-authored-by: Oliver Nordbjerg <hi@notbjerg.me> * chore: fmt * REVM tracing and gas reports (foundry-rs#867) * feat: very simple traces * feat: creation traces * feat: setup and revert traces * fix: fix lib addresses * refactor: simplify tracer inspector * fix: fill traces in correct order * build: bump revm * fix: get code for newly created contracts * refactor: unify log extraction logic * feat: trace logs * refactor: unify labels and names * refactor: return string from trace Instead of passing in an empty string we then pass around inside the trace display logic, we just return strings where appropriate. * refactor: remove identified contracts * refactor: remove unused vars * refactor: simplify `construct_func_call` * refactor: name special characters in traces * refactor: rework all display logic * feat: first pass identify/decode for traces * refactor: move tracing to own module * refactor: simplify `test` * feat: traces for fuzz tests * fix: make fuzz revert reasons less verbose * feat: port gas reports * refactor: small readability nits * feat: run fuzz *and* unit tests in parallel Previously we would run each test contract in parallel, but within each `ContractRunner` we would run unit tests first (in parallel) and then fuzz tests (in parallel). * refactor: move colouring logic to its own function * fix: test contract identification We now include three kinds of traces that are used for identification of contracts: - Deployment traces: these are the initial deployments of the test contract and libraries - Setup traces: these are traces of calls to the `setUp` function - Execution traces: these are the traces of calls to the test contract itself * fix: mark setup trace as a setup trace * fix: get correct nonce in tracer * fix: log extraction outside of current memory * chore: clean up complex types * chore: remove outdated comment * fix: make tests compile * fix: add missing test filter function * feat: display full address in traces * fix: color "new" keyword in traces * fix: filter out `console.log` calls from traces * chore: remove unnecessary comment * feat: add gas cost to creation traces * fix: properly decode outputs * refactor: destructure `TestSetup` in test funcs * fix: ignore address for func output decoding * fix: fix expect emit Co-authored-by: Georgios Konstantopoulos <me@gakonst.com> Co-authored-by: brockelmore <brockelmore@users.noreply.github.com> * REVM debugger (foundry-rs#920) * feat: port debugger data structures * feat: initial port of `ui` crate * chore: add `ui` crate as a workspace member * refactor: adjust ui contract identification * feat: grey out 0 values in debugger memory Closes foundry-rs#902 * style: minor debugger ui beautification * feat: better stack display in debugger ui * feat: gray out zero bytes in stack view * feat: debugger inspector * refactor: minor code cleanup * feat: port `forge run` * fix: temp fix for failing `DsTest.sol` include * chore: fix lints * test: adjust `forge run` tests * refactor: use simple bool for revert checks * chore: remove unused display impl * chore: remove unused comment * fix: display number of stack items in ui * docs: prettify cli help for some commands * feat: `forge test --debug` * refactor: `get_create_address` util * refactor: `InspectorData` * docs: more detailed err for `forge test --debug` * feat: support hardhat artifacts in `vm.getCode` (foundry-rs#956) Ports foundry-rs#903 * REVM: FFI cheatcode updates (foundry-rs#955) * feat: only strip 0x in ffi output if present Ports foundry-rs#904 * Update forge/src/executor/inspector/cheatcodes/ext.rs Co-authored-by: Georgios Konstantopoulos <me@gakonst.com> * REVM gas fixes (foundry-rs#950) * feat: account for gas refunds * refactor: merge `call_raw` and committing variant * fix: actually use refund quotient * feat: strip tx gas stipend * fix: fix reported gas usage in debugger * build: use upstream revm * test: adjust `forge run` gas values in tests * chore: remove unused copy * chore: add note on push maths * feat: make stipend reduction optional * fix: remove tx stipend in `forge run` * REVM: Pull EVM executor into own crate (foundry-rs#961) * refactor: move evm executor to own crate * refactor: `evm::executor::fuzz` -> `evm::fuzz` * refactor: `evm::debugger` -> `evm::debug` * test: fix multi runner test * feat: better ux for expect revert without reason (foundry-rs#962) * Cross-crate testdata (foundry-rs#965) * feat: cross-crate shared testdata * refactor: move `foundry-utils` to common tests * fix: fix getcode test * fix: compile once in tests * fix: fix prank cheatcode (foundry-rs#973) Correctly apply `msg.sender` prank to both transfers and calls. * fix: prank depth math * test: fix lib linking test * refactor: use revm `log` hook (foundry-rs#984) * refactor: use revm `log` hook * chore: bump revm Co-authored-by: Georgios Konstantopoulos <me@gakonst.com> * test: add lil-web3 to integration tests * test: add maple labs loans to integration tests Closes foundry-rs#959 * REVM fuzz dictionary (foundry-rs#985) * feat: fuzz dictionary Co-authored-by: brockelmore <31553173+brockelmore@users.noreply.github.com> * fix: handle malformed bytecode * fix: limit search for push bytes * feat: collect fuzz state from logs * feat: build initial fuzz state from db * perf: use `Index` instead of `Selector` Co-authored-by: brockelmore <31553173+brockelmore@users.noreply.github.com> * feat(cli): Refactor cli/cmd over forge and cast (foundry-rs#1009) * ⚙️ refactor cli * 🧪 refactor casts * REVM: Support cheatcodes in `setUp` (foundry-rs#997) * fix: support cheatcodes in `setUp` * fix: subtract stipend without panic * chore: rename test * fix: set tx gas price to block basefee * fix: use `CALLER` for `is_success` check * chore: remove duplicate clap attribute * fix: set chain id correctly in fork mode * fix: separate evm block env from execution env * chore: clippy * refactor: block override without `block_env` fn * test: explain why git clone failed * test: disable maple-labs/loan * refactor: make addresses statics instead of lazies * docs: fix console address comment * refactor: make `DUMMY_CREATE_ADDRESS` a static * chore: minor nits * refactor: move inspector state collection * fix: report correct fuzz failure case (foundry-rs#1017) * fix: report correct fuzz failure case * docs: improve some docs in fuzzer * feat: add support for storage caching (foundry-rs#1006) * Simple REVM test runner (foundry-rs#788) * refactor: nuke `evm-adapters` * refactor: simple revm test runner Current features: - Can run unit tests - Works with both revert-type tests and DSTest-type tests - Collects logs, albeit not for reverting tests - Integrated with config and CLI flags Disabled features: - Gas reports - Tracing - Cheatcodes - Fuzzing - Log decoding - Forking mode - Hardhat-style `console.log`, since those require us to decode calls to a specific address (HH does not emit logs) - The debugger In addition to this, I've disabled some tests that could never pass under the current circumstances, but that should be adjusted and re-enabled when their respective features are implemented (such as fuzz tests) * refactor: adjust CLI to new runner API * feat: log collector inspector * feat: hardhat logs * chore: lint * refactor: extract hh log converter to helper fn * refactor: return single test result if setup fails * build: use upstream revm chore: renuke `evm-adapters` * REVM fuzzer (foundry-rs#789) * REVM cheatcodes (foundry-rs#841) * feat: add `InspectorStack` Adds `InspectorStack`, an inspector that calls a stack of other inspectors sequentially. Closes foundry-rs#752 * feat: port cheatcodes to revm * feat: port `expectCall` cheatcode * feat: extract labels from cheatcode inspector * feat: port `expectEmit` cheatcode * refactor: move log decoding into `forge` crate * chore: remove unused evm patch * test: re-enable debug logs test * fix: record reads on `SSTORE` ops * refactor: rename `record` to `start_record` * docs: clarify why `DUMMY_CALL_OUTPUT` is 320 bytes * fix: handle `expectRevert` with no return data * build: bump revm * chore: remove outdated todo * refactor: use static dispatch in `InspectorStack` * build: use k256 * fix: make gas usage not so crazy * feat(revm): add forking mode (foundry-rs#835) * feat: copy-paste old forking provider * feat(fork): convert to REVM traits * chore: remove unnecessary codehash handler * feat: impl Database for shared backend * chore: fix tests * chore: fmt * fix(fork): correctly convert H256 <> U256 for storage * refactor: separate storage from accounts in cache * feat(fork): fetch block hashes * chore: remove unused DB parameter * test: add test for block hashes * feat: add forked backend to executor builder * feat(cli): set fork url on the executor * refactor: move shared backend to separate file * feat(fork): add fn for instantiating forked env * feat(cli): allow pinning block number * fix(fork): install missing listeners * feat(fork): instantiate environment with forked state * fix: use a CALLER address with maxed out balance for calls this is required because in forking mode otherwise the account wont have enough balance to transact * chore: fmt Co-authored-by: Oliver Nordbjerg <hi@notbjerg.me> * chore: fmt * REVM tracing and gas reports (foundry-rs#867) * feat: very simple traces * feat: creation traces * feat: setup and revert traces * fix: fix lib addresses * refactor: simplify tracer inspector * fix: fill traces in correct order * build: bump revm * fix: get code for newly created contracts * refactor: unify log extraction logic * feat: trace logs * refactor: unify labels and names * refactor: return string from trace Instead of passing in an empty string we then pass around inside the trace display logic, we just return strings where appropriate. * refactor: remove identified contracts * refactor: remove unused vars * refactor: simplify `construct_func_call` * refactor: name special characters in traces * refactor: rework all display logic * feat: first pass identify/decode for traces * refactor: move tracing to own module * refactor: simplify `test` * feat: traces for fuzz tests * fix: make fuzz revert reasons less verbose * feat: port gas reports * refactor: small readability nits * feat: run fuzz *and* unit tests in parallel Previously we would run each test contract in parallel, but within each `ContractRunner` we would run unit tests first (in parallel) and then fuzz tests (in parallel). * refactor: move colouring logic to its own function * fix: test contract identification We now include three kinds of traces that are used for identification of contracts: - Deployment traces: these are the initial deployments of the test contract and libraries - Setup traces: these are traces of calls to the `setUp` function - Execution traces: these are the traces of calls to the test contract itself * fix: mark setup trace as a setup trace * fix: get correct nonce in tracer * fix: log extraction outside of current memory * chore: clean up complex types * chore: remove outdated comment * fix: make tests compile * fix: add missing test filter function * feat: display full address in traces * fix: color "new" keyword in traces * fix: filter out `console.log` calls from traces * chore: remove unnecessary comment * feat: add gas cost to creation traces * fix: properly decode outputs * refactor: destructure `TestSetup` in test funcs * fix: ignore address for func output decoding * fix: fix expect emit Co-authored-by: Georgios Konstantopoulos <me@gakonst.com> Co-authored-by: brockelmore <brockelmore@users.noreply.github.com> * REVM debugger (foundry-rs#920) * feat: port debugger data structures * feat: initial port of `ui` crate * chore: add `ui` crate as a workspace member * refactor: adjust ui contract identification * feat: grey out 0 values in debugger memory Closes foundry-rs#902 * style: minor debugger ui beautification * feat: better stack display in debugger ui * feat: gray out zero bytes in stack view * feat: debugger inspector * refactor: minor code cleanup * feat: port `forge run` * fix: temp fix for failing `DsTest.sol` include * chore: fix lints * test: adjust `forge run` tests * refactor: use simple bool for revert checks * chore: remove unused display impl * chore: remove unused comment * fix: display number of stack items in ui * docs: prettify cli help for some commands * feat: `forge test --debug` * refactor: `get_create_address` util * refactor: `InspectorData` * docs: more detailed err for `forge test --debug` * feat: support hardhat artifacts in `vm.getCode` (foundry-rs#956) Ports foundry-rs#903 * REVM: FFI cheatcode updates (foundry-rs#955) * feat: only strip 0x in ffi output if present Ports foundry-rs#904 * Update forge/src/executor/inspector/cheatcodes/ext.rs Co-authored-by: Georgios Konstantopoulos <me@gakonst.com> * REVM gas fixes (foundry-rs#950) * feat: account for gas refunds * refactor: merge `call_raw` and committing variant * fix: actually use refund quotient * feat: strip tx gas stipend * fix: fix reported gas usage in debugger * build: use upstream revm * test: adjust `forge run` gas values in tests * chore: remove unused copy * chore: add note on push maths * feat: make stipend reduction optional * fix: remove tx stipend in `forge run` * REVM: Pull EVM executor into own crate (foundry-rs#961) * refactor: move evm executor to own crate * refactor: `evm::executor::fuzz` -> `evm::fuzz` * refactor: `evm::debugger` -> `evm::debug` * test: fix multi runner test * feat: better ux for expect revert without reason (foundry-rs#962) * Cross-crate testdata (foundry-rs#965) * feat: cross-crate shared testdata * refactor: move `foundry-utils` to common tests * fix: fix getcode test * fix: compile once in tests * fix: fix prank cheatcode (foundry-rs#973) Correctly apply `msg.sender` prank to both transfers and calls. * fix: prank depth math * test: fix lib linking test * refactor: use revm `log` hook (foundry-rs#984) * refactor: use revm `log` hook * chore: bump revm Co-authored-by: Georgios Konstantopoulos <me@gakonst.com> * test: add lil-web3 to integration tests * test: add maple labs loans to integration tests Closes foundry-rs#959 * REVM fuzz dictionary (foundry-rs#985) * feat: fuzz dictionary Co-authored-by: brockelmore <31553173+brockelmore@users.noreply.github.com> * fix: handle malformed bytecode * fix: limit search for push bytes * feat: collect fuzz state from logs * feat: build initial fuzz state from db * perf: use `Index` instead of `Selector` Co-authored-by: brockelmore <31553173+brockelmore@users.noreply.github.com> * feat(config): add caching settings * feat: add none option * feat: add foundry data dir * feat: add storage map support * bump ethers * chore(clippy): make clippy happy * refactor: diskmap * feat: add rpc caching support * feat: add no storage cache option * refactor: rename cnfig value * docs: more storage caching docs * fix: with config builder function * refactor: address review Co-authored-by: Bjerg <onbjerg@users.noreply.github.com> Co-authored-by: Georgios Konstantopoulos <me@gakonst.com> Co-authored-by: Oliver Nordbjerg <hi@notbjerg.me> Co-authored-by: brockelmore <brockelmore@users.noreply.github.com> Co-authored-by: brockelmore <31553173+brockelmore@users.noreply.github.com> * fix: default to 80m gas * fix(evm): gracefully shutdown backendhandler (foundry-rs#1021) * feat(evm/cache): improve json file caching (foundry-rs#1025) * feat(cache): proper json cache * refactor: use new db types * chore(clippy): make clippy happy * bump revm * docs: some docs * refactor: extend Fork type * remove diskmap types * test: refactor tests * remove sharedmemcache * add tests * more tracing * chore(clippy): make clippy happy * release: 0.2.0 Co-authored-by: Georgios Konstantopoulos <me@gakonst.com> Co-authored-by: brockelmore <brockelmore@users.noreply.github.com> Co-authored-by: brockelmore <31553173+brockelmore@users.noreply.github.com> Co-authored-by: abigger87 <abigger87@gmail.com> Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

onbjerg added the L-ignore Log: ignore PR in changelog label Mar 12, 2022

onbjerg force-pushed the revm branch from fcc981f to 7b706a1 Compare March 12, 2022 20:52

onbjerg force-pushed the revm branch from d5bed39 to 2ab9081 Compare March 13, 2022 20:04

This was linked to issues Mar 14, 2022

integrate revm in evm-adapters #116

Closed

fix(tracing): Decode collided function signatures #646

Closed

This was referenced Mar 14, 2022

fix(tracing): Decode collided function signatures #646

Closed

Make clippy happy #944

Closed

onbjerg linked an issue Mar 15, 2022 that may be closed by this pull request

Grey zero values in debugger memory #902

Closed

onbjerg force-pushed the revm branch from d879ded to 7483c90 Compare March 15, 2022 18:36

onbjerg mentioned this pull request Mar 17, 2022

dapptools-style fuzzer counterexample replay #893

Closed

onbjerg linked an issue Mar 17, 2022 that may be closed by this pull request

dapptools-style fuzzer counterexample replay #893

Closed

gakonst mentioned this pull request Mar 17, 2022

forge test displays incorrect gas for function call, but displays correct gas in forge test -vvvv #812

Closed

2 tasks

onbjerg linked an issue Mar 17, 2022 that may be closed by this pull request

forge test displays incorrect gas for function call, but displays correct gas in forge test -vvvv #812

Closed

2 tasks

onbjerg force-pushed the revm branch from 2ddf7a6 to 7b21a5d Compare March 17, 2022 11:56

onbjerg linked an issue Mar 17, 2022 that may be closed by this pull request

vm.expectRevert should support an empty signature to catch reverts with no message #885

Closed

onbjerg mentioned this pull request Mar 17, 2022

Target native CPU on foundryup -b #974

Closed

onbjerg linked an issue Mar 21, 2022 that may be closed by this pull request

Add maple protocol to integration tests #959

Closed

onbjerg force-pushed the revm branch from 9c671de to 8430b26 Compare March 21, 2022 02:27

onbjerg changed the title ~~Draft: REVM~~ refactor: move to revm Mar 21, 2022

onbjerg and others added 6 commits March 22, 2022 03:36

REVM fuzzer (#789)

c1046ae

chore: fmt

04892dd

mattsse requested changes Mar 22, 2022

View reviewed changes

onbjerg mentioned this pull request Mar 22, 2022

fix: report correct fuzz failure case #1017

Merged

onbjerg and others added 7 commits March 22, 2022 12:43

refactor: make addresses statics instead of lazies

ff83f81

docs: fix console address comment

ac66716

refactor: make DUMMY_CREATE_ADDRESS a static

7263f10

chore: minor nits

47e2695

refactor: move inspector state collection

c2f15ae

fix: report correct fuzz failure case (#1017)

f3f43ad

* fix: report correct fuzz failure case * docs: improve some docs in fuzzer

onbjerg added T-feature Type: feature and removed L-ignore Log: ignore PR in changelog labels Mar 22, 2022

onbjerg linked an issue Mar 22, 2022 that may be closed by this pull request

Have same behaviour as on EVM for blockhash(blockNumber) #837

Closed

fix: default to 80m gas

71cd0e9

onbjerg requested review from mattsse and gakonst March 22, 2022 17:27

mattsse approved these changes Mar 22, 2022

View reviewed changes

fix(evm): gracefully shutdown backendhandler (#1021)

bd7fc8d

gakonst approved these changes Mar 23, 2022

View reviewed changes

onbjerg mentioned this pull request Mar 23, 2022

No gas-report for property-based tests #1023

Closed

2 tasks

release: 0.2.0

91879d3

gakonst merged commit 33f1df0 into master Mar 23, 2022

gakonst deleted the revm branch March 23, 2022 19:51

m1guelpf mentioned this pull request Mar 24, 2022

Allow calling expectRevert without any arguments foundry-rs/forge-std#31

Merged

mds1 mentioned this pull request Mar 25, 2022

bug: foundryup fails once a branch you're on is deleted #1045

Closed

2 tasks

mattsse mentioned this pull request Apr 8, 2022

docs: add note back about large toml numbers #1237

Merged

DaniPopes mentioned this pull request May 4, 2023

feat(forge): Paris & Shanghai support & add prevrandao cheatcode #4856

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: move to revm #918

refactor: move to revm #918

onbjerg commented Mar 12, 2022 •

edited

Loading

gakonst commented Mar 13, 2022

onbjerg commented Mar 13, 2022

gakonst commented Mar 17, 2022

onbjerg commented Mar 18, 2022 •

edited

Loading

onbjerg commented Mar 21, 2022

onbjerg commented Mar 22, 2022 •

edited

Loading

mattsse left a comment

mattsse left a comment

gakonst left a comment •

edited

Loading

refactor: move to revm #918

refactor: move to revm #918

Conversation

onbjerg commented Mar 12, 2022 • edited Loading

Tasks

gakonst commented Mar 13, 2022

onbjerg commented Mar 13, 2022

gakonst commented Mar 17, 2022

onbjerg commented Mar 18, 2022 • edited Loading

onbjerg commented Mar 21, 2022

onbjerg commented Mar 22, 2022 • edited Loading

mattsse left a comment

Choose a reason for hiding this comment

mattsse left a comment

Choose a reason for hiding this comment

gakonst left a comment • edited Loading

Choose a reason for hiding this comment

onbjerg commented Mar 12, 2022 •

edited

Loading

onbjerg commented Mar 18, 2022 •

edited

Loading

onbjerg commented Mar 22, 2022 •

edited

Loading

gakonst left a comment •

edited

Loading