feat(cheatcodes): add ability to ignore (multiple) specific and partial reverts in fuzz and invariant tests

[emo-eth]

Motivation

As raised in #4271, sometimes the fuzzer dictionary will uncover real but irrelevant errors in contract code, especially when testing against stateful forks. Runs that encounter anticipated errors should be thrown out and count towards the global assume counter.

Closes #4271.

Solution

This PR adds overloaded and related function definitions to vm.assumeNoRevert, introduced in #8780, as follows:

    /// Represents a "potential" revert reason from a single subsequent call when using `vm.assumeNoReverts`.
    /// Reverts that match will result in a FOUNDRY::ASSUME rejection, whereas unmatched reverts will be surfaced
    /// as normal.
    struct PotentialRevert {
        /// The allowed origin of the revert opcode; address(0) allows reverts from any address
        address reverter;
        /// When true, only matches on the beginning of the revert data, otherwise, matches on entire revert data
        bool partialMatch;
        /// The data to use to match encountered reverts
        bytes revertData;
    }

    /// Discard this run's fuzz inputs and generate new ones if next call reverts with the potential revert parameters.
    #[cheatcode(group = Evm, safety = Safe)]
    function assumeNoRevert(PotentialRevert calldata potentialRevert) external pure;

    /// Discard this run's fuzz inputs and generate new ones if next call reverts with the any of the potential revert parameters.
    #[cheatcode(group = Evm, safety = Safe)]
    function assumeNoRevert(PotentialRevert[] calldata potentialReverts) external pure;

Behavior

• it should be possible to to reject any specific error without data (by encoding a 4-byte selector as revertData)
• it should be possible to reject any specific error with arbitrary data (by encoding a signature including data as revertData)
• it should be possible to reject any specific of error regardless of extra data (by encoding a 4-byte selector and specifying partialMatch as true)
• it should be possible to reject multiple errors that a single call may produce (overloaded array param)
• it should be possible to restrict these rejections to a specific reverter (the optional overloaded address reverter param)
• anticipated reverts should reset after next non-cheatcode external call
• combining a catch-all vm.assumeNoRevert() with a vm.assumeNoRevert(args) call should result in an error

(Currently an invariant dict related test is failing)

[grandizzy]

looks good! left couple of comments, going to move back in draft until fix added

[grandizzy]

@mds1 would be great to have your thoughts re new proposed cheatcodes. Thanks!

[emo-eth]

@grandizzy should be in a good state now – apologies, had a hectic week. let me know if anything else looks off.

[jenpaff]

@grandizzy should be in a good state now – apologies, had a hectic week. let me know if anything else looks off.

moving PR to ready for review

[grandizzy]

thank you, left couple of comments.

[grandizzy]

Thank you, looks good! I pushed a change to simplify test and have one more nit re assume_no_revert fn. @zerosnacks @yash-atreya @klkvr pls chime in when time permits. thank you!

[grandizzy]

Lgtm, thank you! Pending additional review before merging

[mds1]

In general UX LGTM! One thing is let's make sure the behavior between this and the various expectRevert cheats is consistent, e.g. if you pass a 4byte selector does it only match if the revert data is exact, or do partials match also? Given that this cheat is assumeNoRevert for partial match I think it's inconsistent as spec'd in the PR description

https://github.com/foundry-rs/forge-std/blob/da591f56d8884c5824c0c1b3103fbcfd81123c4c/src/Vm.sol#L2102-L2124

[emo-eth]

In general UX LGTM! One thing is let's make sure the behavior between this and the various expectRevert cheats is consistent, e.g. if you pass a 4byte selector does it only match if the revert data is exact, or do partials match also? Given that this cheat is assumeNoRevert for partial match I think it's inconsistent as spec'd in the PR description

https://github.com/foundry-rs/forge-std/blob/da591f56d8884c5824c0c1b3103fbcfd81123c4c/src/Vm.sol#L2102-L2124

@mds1 apologies, i can update the description (pending final decision) – @grandizzy suggested having the default behavior to accept partial matches.

having the granularity of assumeNoRevert vs assumeNoPartialRevert would be useful in the scenario of colliding selectors – but that's pretty rare unless intentional. i am having a hard time thinking of cases where a test might be pulling in (arbitrary) external or random revert bytes that might have collisions – but there are possibly some, and it might be worth having the disambiguation.

i do appreciate the explicitness, though i do think assumeNoRevert is already kind of a counter-intuitive name, and assumeNoPartialRevert adds onto that complexity. eg assumeNoRevertPartial is maybe slightly less confusing, but breaks with the expectPartialRevert naming convention...

happy to add back in if there's consensus

[mds1]

Thanks for those details, so my thoughts are:

• We already have expectRevert(bytes4) and expectPartialRevert(bytes4). To this point—"would be useful in the scenario of colliding selectors – but that's pretty rare unless intentional"—I'm unsure of the motivation for this split
• The least surprising assume behavior would therefore be analogous meaning assumeNoRevert(bytes4) and assumeNoPartialRevert(bytes4)
• It sounds like there are some breaking change considerations here, which I am not fully up to speed on, in which case that is suitable rationale for having inconsistent behavior between the expect and assume cheats
• Therefore I'll defer to you and @grandizzy to decide
• But, we should track this and unify / simplify for foundry v1

[emo-eth]

It looks like the CI is failing related to the proposed changes:
https://github.com/foundry-rs/foundry/actions/runs/12635122448/job/35204363739?pr=9179#step:12:361

weird, I don't see why this test would fail with proposed changes and cannot reproduce the test failure locally either ... 🤔

Likewise not sure why my changes would affect it - but does seem to consistently be failing for polygon between runs 🤔

[grandizzy]

It looks like the CI is failing related to the proposed changes:
https://github.com/foundry-rs/foundry/actions/runs/12635122448/job/35204363739?pr=9179#step:12:361

weird, I don't see why this test would fail with proposed changes and cannot reproduce the test failure locally either ... 🤔

Likewise not sure why my changes would affect it - but does seem to consistently be failing for polygon between runs 🤔

I rebased PR with master containing fixes for other flaky tests (was thinking that failure could be an effect of those) but same test failing, checking some more why's that

[grandizzy]

thank you, lgtm! had to rework the failing test and split tested contracts and test contract in different files as they were affecting outcome. waiting for one more review before merging

[emo-eth]

@zerosnacks @yash-atreya @klkvr would love to get this merged if there aren't any more blockers

[zerosnacks]

Thanks @emo-eth!

LGTM

[DaniPopes]

this looks unused?

[grandizzy]

yeah, tests within test_cmd.rs are not importing this file but rather adding same source inline, so this can be removed