Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invariant Testing #69

Closed
gakonst opened this issue Sep 30, 2021 · 8 comments
Closed

Invariant Testing #69

gakonst opened this issue Sep 30, 2021 · 8 comments
Labels
A-evm Area: EVM C-forge Command: forge Cmd-forge-test Command: forge test D-hard Difficulty: hard

Comments

@gakonst
Copy link
Member

gakonst commented Sep 30, 2021

@transmissions11 leaked some alpha by @MrChico from the dapphub chat

telegram-cloud-photo-size-1-5096339203830163703-x
telegram-cloud-photo-size-1-5096339203830163702-x

This algorithm should be easy to do inside our fuzzer logic.

you should run the invariant func once before you do any txs
Because sometimes it doesn’t hold off the bat but then that changes after txs or if all the txs revert then the invariant will show as passed when it may have totally failed if ever called
dapphub/dapptools#745

@gakonst gakonst removed dapp labels Jan 12, 2022
@onbjerg onbjerg added Cmd-forge-test Command: forge test C-forge Command: forge D-hard Difficulty: hard D-average Difficulty: average and removed D-hard Difficulty: hard labels Jan 24, 2022
@brockelmore brockelmore added D-hard Difficulty: hard and removed D-average Difficulty: average labels Jan 28, 2022
@transmissions11
Copy link
Contributor

transmissions11 commented Feb 5, 2022

personal wishlist:

  1. would like a way to specify that a test can revert, just not fail an assert
  2. would like to fuzz a set of invariants for time instead of fixed fuzz runs, or as long as i want
    a) ui showing failures as they occur and the progress of the campaign
    b) don't stop the campaign after a failure is found, keep fuzzing parts of it if possible
  3. would like a way to control the list of accounts that the fuzzer calls things from
  4. would like a way to control the contracts the fuzzer targets

@lucas-manuel
Copy link

Working on invariants right now so will come back with a wish list as well, is this currently being worked on btw?

@gakonst
Copy link
Member Author

gakonst commented Feb 10, 2022

I think @brockelmore had hacked something?

@brockelmore
Copy link
Member

no that was just in solidity didnt impl in foundry

@lucas-manuel
Copy link

One thing that would be really nice is a summary of how many times each function was called and the mean values like in the current fuzz run output. Also logging numbers of reversions for each function call if any.

@mattsse
Copy link
Member

mattsse commented Aug 25, 2022

vote close @joshieDo

@gakonst
Copy link
Member Author

gakonst commented Aug 25, 2022

would like a way to control the list of accounts that the fuzzer calls things from

I think we don't have this yet? Probably for separate issue.

@joshieDo
Copy link
Collaborator

joshieDo commented Aug 26, 2022

We do with targetSenders and excludeSenders.

So feel free to close!

@gakonst gakonst closed this as completed Aug 26, 2022
Repository owner moved this from Todo to Done in Foundry Aug 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-evm Area: EVM C-forge Command: forge Cmd-forge-test Command: forge test D-hard Difficulty: hard
Projects
Archived in project
Development

No branches or pull requests

7 participants