Document invariant testing + patterns

[gakonst]

foundry-rs/foundry#1572

https://github.com/maple-labs/revenue-distribution-token/tree/e0eca03c2ff05c36000a097de678543d7234f7cc/contracts/test

document simple invariant breaking, the actor model from Maple's RDT, usage of targetContract(s) etc

[Perelyn-sama]

Getting this done.

[lucas-manuel]

Thoughts on patterns:

Open Invariant Testing

This testing pattern deploys all relevant contracts in the setUp function and exposes all the functions in these contracts to the fuzzer. This pattern must use revert_on_failure = false in the foundry.toml, since this pattern will make calls with unbounded values.

Pros	Cons
- Checks all functions - Unbounded params	- Single actor - Reversions

Bounded Actor-Based Invariant Testing

This testing pattern deploys all relevant contracts in the setUp function and deploys wrapper contracts to call the functions on the contract. These contracts act as actors in a more realistic simulation of interactions against the desired contracts. These contracts can have relevant permissions and will perform calls with parameters bounded to reasonable values. This pattern must use revert_on_failure = true in the foundry.toml, since this pattern should expose any unexpected reversions over the course of the fuzzing campaign.

User contract reference.

To take it a step further, a contract can be used to dynamically deploy new actors during the fuzzing campaign and perform actions from these actors (example).

Another use case for this pattern is to create a contract that will call the vm.warp cheatcode at random intervals to simulate the passage of time and ensure that invariants are still held (example).

This pattern must add all non-wrapper contracts deployed in the setUp function into the excludeContracts array as to ensure that these contracts are only called through the wrapper contracts.

Pros	Cons
- Many actors - Can simulate passage of time - Can expose unexpected reversions	- Potential human error in incorrect implementations of wrapper functions

Unbounded Actor-Based Invariant Testing

Similar pattern to above, except with unbounded parameters. This allows for the contract to be called without any artificially bounded parameters, potentially exposing previously unconceived vectors to break invariants. This pattern must use revert_on_failure = false in the foundry.toml, since this pattern will make calls with unbounded values.

Pros	Cons
- Many actors - Can simulate passage of time - Unbounded parameters	- Reversions

[Perelyn-sama]

Hey @lucas-manuel can we get examples of the first and last example too please, thanks

[lucas-manuel]

Currently all the invariant examples we have are private, but here is an open source implementation of bounded actor-based invariant testing: https://github.com/maple-labs/revenue-distribution-token/blob/add-forge-invariants/contracts/test/Invariants.t.sol

[JacoboLansac]

When are Invariant tests expected to be released to the public? Looking forward to it. Thanks!

[gakonst]

They are already released!

[JacoboLansac]

They are already released!

What!
According to the foundry book they are not yet supported. https://book.getfoundry.sh/forge/advanced-testing

Am I looking at the wrong version of the foundry book? Or, can I have a link to the documentation of invariant tests? Damn I'm excited!

[Perelyn-sama]

https://github.com/foundry-rs/foundry/tree/master/testdata%2Ffuzz%2Finvariant

[JacoboLansac]

https://github.com/foundry-rs/foundry/tree/master/testdata%2Ffuzz%2Finvariant

And an example project where it is used?
Thank you very much!

[Perelyn-sama]

Scroll up

[azflin]

There is no official docs for this still?

[gakonst]

Soon :) For now take a look at https://github.com/lucas-manuel/invariant-example and #760

[PaulRBerg]

take a look at https://github.com/lucas-manuel/invariant-example and #760

Also maple-core-v2.

[JacoboLansac]

what is the best way of managing blockchain time in invariant tests?
So far I did a work around using a storage variable of the handler contract and a modifier that would warp time before each of the target functions, but I was wondering if there was a better way. Thanks

[aviggiano]

What does dictionary_weight mean and how does it affect the fuzzer performance? I can't find any docs related to it.

[gakonst]

Could you guys please open new issues about these questions, for us to document them in the book?