Deep Fuzzing on Newly Modified Solidity Files

[0x4007]

Random thought I had but since I wrote that action that will:

1. read the bounty-bot repo and
2. build if any source code changes occur, and then
3. commit the built file to the repo

It inspired me to think of what other compute we can delegate to the cloud to make the dev experience better.

What if we do something similar with deep fuzzing, but only on modified Solidity files? I understand that deep fuzzing takes a lot of time, so we could run it for the entire codebase one time, and then only run it on modified Solidity files moving forward? I vaguely recall a discussion related to how deep fuzzing is super slow.

transmissions11 has a Foundry template that does a deep fuzz on everything when merged to main, which served as another point of inspiration for me.

      - name: Run tests
        run: forge test
        env:
          # Only fuzz intensely if we're running this action on a push to main or for a PR going into main:
          FOUNDRY_PROFILE: ${{ (github.ref == 'refs/heads/main' || github.base_ref == 'main') && 'intense' }}

https://github.com/transmissions11/foundry-template/blob/0bf8e53589f2e8f72e41baf672c25aa35ba57295/.github/workflows/tests.yml#L28-L32

[gitcoindev]

/start

[ubiquibot]

Deadline	Tue, 22 Aug 2023 14:38:37 UTC
Registered Wallet	0x7e92476D69Ff1377a8b45176b1829C4A5566653a

Tips:

• Use /wallet 0x0000...0000 if you want to update your registered payment wallet address @user.
• Be sure to open a draft pull request as soon as possible to communicate updates on your progress.
• Be sure to provide timely updates to us when requested, or you will be automatically unassigned from the bounty.

[gitcoindev]

Hi! I found a way to achieve this, my QA is available at gitcoindev#2

I will submit a PR today.

[gitcoindev]

The solution requires an explanation. I did numerous experiments on my local workspace and figured out that in order to find tests related to modified contract files forge tree command can be used. This command provides project dependency tree from which the test case that uses a certain Solidity contract can be found.

An example: modified Diamond.sol, CreditClock.sol and StakingShare.sol:

$ forge tree | grep -E "^test|^src|Diamond.sol" | grep -B1 Diamond.sol | grep ^test
test/diamond/DiamondTest.t.sol ^0.8.19

$ forge tree | grep -E "^test|^src|CreditClock.sol" | grep -B1 CreditClock.sol | grep ^test
test/dollar/core/CreditClock.t.sol ^0.8.19

$ forge tree | grep -E "^test|^src|StakingShare.sol" | grep -B1 StakingShare.sol | grep ^test
test/diamond/DiamondTest.t.sol ^0.8.19
test/diamond/facets/ChefFacet.t.sol ^0.8.19
test/diamond/facets/StakingFacet.t.sol ^0.8.19
test/diamond/facets/StakingFormulasFacet.t.sol ^0.8.19
test/diamond/facets/UbiquityPoolFacet.t.sol ^0.8.19
test/dollar/core/StakingShare.t.sol ^0.8.19

As it can be seen above, some tests are duplicated, therefore I used sort | uniq commands to get only unique tests and executed forge test with FOUNDRY_PROFILE: intense only on this subset of tests.

I tried to implement the workflow to be intelligent and exit fuzzing if no Solidity contracts were modified.
FOUNDRY_PROFILE is set to 10000 runs, should be more than enough.

For pull request deep fuzzing is executed on all tests to avoid missing any scenarios.

[gitcoindev]

Also, deep fuzzing was already triggered on my pull request related to this issue and logs are available at https://github.com/ubiquity/ubiquity-dollar/actions/runs/5927880413/job/16072306637

[gitcoindev]

Adding a comment for the bot: waiting for PR approval from @rndquu and merge.

[ubiquibot]

Task Assignee Reward

[ CLAIM 200 WXDAI ]

0x7e92476D...A5566653a

If you've enjoyed your experience in the DevPool, we'd appreciate your support. Follow Ubiquity on GitHub and star this repo. Your endorsement means the world to us and helps us grow!

We are excited to announce that the DevPool and UbiquiBot are now available to partners! Our ideal collaborators are globally distributed crypto-native organizations, who actively work on open source on GitHub, and excel in research & development. If you can introduce us to the repository maintainers in these types of companies, we have a special bonus in store for you!

[ubiquibot]

Task Creator Reward

pavlovcik: [ CLAIM 88 WXDAI ]