feat(test): allow custom txes before unit and fuzz test

[grandizzy]

Motivation

Closes #8485
Closes #1543

Atm unit / fuzz tests are performed only against the state resulting from setup call, but would be nice to support configuration of other txes / state modification for individual tests. This PR adds the possibility of per unit / fuzz test configuration of before test selectors / txes.

Configuration can be specified by implementing beforeTestSetup function

function beforeTestSetup(bytes4 testSelector) public pure returns (bytes[] memory beforeTestCalldata)

where

• bytes4 testSelector is the selector of the test that will run on before txes modified state
• bytes[] memory beforeTestCalldata is an array of arbitrary calldatas that will be applied before test run
  E.g. for configuring testC to be executed on state modified by testA and setB(uint256) functions the beforeTestCalldata can be implemented as

    function beforeTestSetup(bytes4 testSelector) public pure returns (bytes[] memory beforeTestCalldata) {
        if (testSelector == this.testC.selector) {
            beforeTestCalldata = new bytes[](2);
            beforeTestCalldata[0] = abi.encodePacked(this.testA.selector);
            beforeTestCalldata[1] = abi.encodeWithSignature("setB(uint256)", 111);
        }
    }

• To continue test execution the before test call should be success.

Solution

[mattsse]

the new comments are very helpful, ty

[DaniPopes]

we should generalize this "check fn exists in abi -> call" to all the callSolDefault, can be followup

[mds1]

What does this give us over just having users write a regular solidity function that they call at the beginning of a test? You can always use --isolate mode to separate out transactions. (I only skimmed the linked issues as I catch up on some github notifications so apologies if I missed the rationale).

Edit: rereading it, I'm unclear if this executes before setUp, or between setUp and the test itself? In either case there are workarounds though through standard solidity patterns

[grandizzy]

What does this give us over just having users write a regular solidity function that they call at the beginning of a test? You can always use --isolate mode to separate out transactions. (I only skimmed the linked issues as I catch up on some github notifications so apologies if I missed the rationale).

Edit: rereading it, I'm unclear if this executes before setUp, or between setUp and the test itself? In either case there are workarounds though through standard solidity patterns

@mds1 I added docs here, pls lmk if this makes sense
https://book.getfoundry.sh/forge/writing-tests#before-test-setups

[mds1]

In those docs, wouldn't an alternate implementation of the first code block simply be the below? The below feels more clear to read, since the beforeTestSetup can be obfuscated when it lives in a contract you inherit from, so test behavior is less obvious

// run with `forge test --isolate` if you need tx side effects like selfdestruct
contract ContractTest is Test {
    uint256 a;
    uint256 b;

    function testA() public {
        require(a == 0);
        a += 1;
    }

    function setB(uint256 value) public {
        b = value;
    }

    function testC() public {
        testA();
        setB(1);
        assertEq(a, 1);
        assertEq(b, 1);
    }
}

[grandizzy]

So could #1543 scenario be accomplished with isolate?

[mds1]

I believe it should, cc @klkvr who implemented isolate mode to confirm

[gsalzer]

In those docs, wouldn't an alternate implementation of the first code block simply be the below? The below feels more clear to read, since the beforeTestSetup can be obfuscated when it lives in a contract you inherit from, so test behavior is less obvious

@mds1 Is it possible to specify isolation mode within the test file, maybe even for a single test alone? Like by asserting at the beginning of a test that we are in isolation mode, or by setting it programmatically? I didn't find it in the docs.

One nice thing about beforeTestSetup is that it explicitly fixes the intended transaction patterns for the tests. If the isolation flag has to be set in the foundry environment, then the execution of the test with wrong flags might show passed tests, which actually would fail with the correct options. On the other hand, if we consider foundry.toml as part of the test specification, then we can set the isolation flag there.

[gsalzer]

@mds1 It seems that your version of the code with isolation behaves differently from the example in the foundry book using beforeTestSetup. The debug output shows separate tx for the latter, but only one for the version with isolation.

$ forge test --match-path mds1.t.sol --isolate -vvvvv
[⠒] Compiling...
No files changed, compilation skipped

Ran 2 tests for test/mds1.t.sol:ContractTest
[PASS] testA() (gas: 22553)
Traces:
  [22553] ContractTest::testA()
    └─ ← [Stop] 

[PASS] testC() (gas: 44986)
Traces:
  [44986] ContractTest::testC()
    └─ ← [Stop] 

$ forge test --match-path fbook.t.sol -vvvvv
[⠒] Compiling...
No files changed, compilation skipped

Ran 2 tests for test/fbook.t.sol:ContractTest
[PASS] testA() (gas: 22487)
Traces:
  [22487] ContractTest::testA()
    └─ ← [Stop] 

[PASS] testC() (gas: 4524)
Traces:
  [22487] ContractTest::testA()
    └─ ← [Stop] 

  [22336] ContractTest::setB(1)
    └─ ← [Stop] 

  [4524] ContractTest::testC()
    └─ ← [Stop] 

$ forge --version
forge 0.2.0 (6822860 2024-07-29T00:20:30.057117769Z)