Gas Profiling #8323
Replies: 3 comments
-
Any progress on this |
Beta Was this translation helpful? Give feedback.
-
Any Answer for this... it is important for developers... |
Beta Was this translation helpful? Give feedback.
-
We've since added the option to run tests in isolation: https://book.getfoundry.sh/reference/cli/forge/test?highlight=isolate#forge-test, this greatly improves the accuracy of tracking gas.
Other improvements like those developed in https://github.com/marktoda/forge-gas-snapshot/ can potentially be upstreamed We also support the Converting this ticket into a discussion as there are no direct actionable items. |
Beta Was this translation helpful? Give feedback.
-
Component
Forge
Describe the feature you would like
Hey everyone, first off thank you for the work on foundry so far, I've been enjoying working with it.
I'd like to ask about gas metering though - what dev workflows do people use/recommend for profiling functions at the moment, and what's the current general plans for future work on this aspect.
Here's the methods I'm presently aware of:
There's
forge snapshot
which gives you gas costs for test functions. This suffers from not giving per function breakdown, and whilst you can structure your tests to minimise the code running that isn't the function you're trying to profile that's pretty messy.There's
forge test --gas-report
which summarises gas costs for function calls as well. At a glance this looks just like what I'm after, but re-running it a few times reveals that the min,avg,max values can differ by significant margins from run to run. It seems that it only tallies the gas cost for a function once per test function, even if the test is fuzzed? In doing so it would appear that it's perhaps inadvertently often using threshold fuzzed input that makes the function behave more drastically different, perhaps. I think I read in passing that only profiling a single run from a fuzzed test was a deliberate decision to keep execution times short, but I think it would benefit from a cli flag that enabled the slower but more representative profiling every run.Then there's the vm cheatcodes to pause and resume gas consumption. In theory you can use these to ignore the gas cost for the code that sets up test scenarios, only metering the gas consumed by the function call that the test is testing. Unfortunately, judging by several other issues (GasMetering generates different results compared to gasleft() diff #4523, Gas meter shows huge number when using vm.pauseGasMetering() along with
delete X
#4370, etc) this doesn't always work as expected.There's also the
gasleft()
approach where end users can manually keep track of gas consumed between two arbitrary points in a test function. This is perhaps the most reliable approach I know of right now, but even that isn't flawless unfortunately. Warm/cold access, memory expansion, and probably some other EVM details mean that even if you're ignoring the gas cost of the test setup code, it's still impacting the execution cost of the function you're trying to profile.In order to address the shortcomings with approach 4 it seems to me that it requires fairly extensive work: some sort of cheatcode that can be used to tell forge to run the subsequent code as if it were a new transaction, giving it fresh state to work with. I'm not remotely familiar with forge internal working so no idea how practical attempting that even is, alas.
So to recap:
Additional context
No response
Beta Was this translation helpful? Give feedback.
All reactions