-
Notifications
You must be signed in to change notification settings - Fork 644
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update ethereum/tests fixtures #1224
Update ethereum/tests fixtures #1224
Conversation
Rolls forwards up to 2018-01-22. Passing of test_state_fixtures determined using `git bisect run`.
The test I'm evaluating currently was introduced in ethereum/tests#405 - you can try it by I couldn't find a rationale for it, but looks like it deals with the EDIT: So, the purpose of this test is likely: the precompile allocates and returns the correct amount of memory, given a very big Running pytest --cache-clear --capture=no -k "test_state_fixtures[/home/veox/src/py-evm/fixtures/GeneralStateTests/stReturnDataTest/modexp_modsize0_returndatasize.json:modexp_modsize0_returndatasize:Byzantium:4]" fails on py-evm/tests/json-fixtures/test_state.py Line 339 in 3fef4bd
The helper contract puts some results of the call to the precompile into storage, and that produces a mismatch in the state. ATM, still looking for the culprit. |
Maybe we can uncover what is wrong with opcode level test cases assuming we know what the expected result is for the inputs you mentioned? |
FYI, the mismatched state root failure case is hard to debug. It normally requires enabling Also maybe look to see if there are VM level tests for this as they would provide explicit output for the call. |
This comment (of mine) is incorrect. Was inspecting EVM memory, but data returned from the precompile call is stored in The EVM memory footprint is correct: a 32-byte value needs to be written to location Took a glance y-day, there is a modexp test file, but no test case for this is particular. Tried inspecting the
|
I think the computation fails to allocate additional 999188 bytes EDIT: no, the above is incorrect. There is no failure to provide The contract in the test
(These are written to storage locations IIUC, I haven't gotten to inspecting actual storage post-computation - trying to find out how... For ref, disassembly of the helper contract:
For ref also - LLL source:
|
Have you looked at the |
EDIT: Eventually, this turned out to be the culprit. Gas calculation was wrong in OK, changed I haven't checked yet if 24023296 is the precise expected gas consumption for the transaction (the test case seems to be under-specified in this regard); the values put to storage, according to the
JIC, here's a gist with |
EDIT: In this comment, I'm confused, thinking two different What's more confusing to me is, doing this printf-style debugging, I can see that the return value length from print('computation.should_erase_return_data:', computation.should_erase_return_data)
print('len(computation._output):', len(computation._output)) at the end of So, seems like somewhere between the result being |
EDIT: The below is also incorrect. The existing value is specified in the test JSON. :( Maybe found it?.. When writing (EDIT: Close accidental, sorry.) |
I've been stuck on this single test case for too long, and, to the best of my understanding, My inclination is to mark this test Brought up my findings in ethereum/tests#405 (where it was incuded), and requested more info. |
Thanks for all the work you've put into this. I think |
The upstream generated test is not sufficiently specific, and it's hard to determine which of the two implementations is incorrect. The principal author of the test case (Yoichi Hirai, github @pirapira) seems currently unavailable, so it's difficult for me to get specific details.
1d8d470
to
b11a7b8
Compare
Squashed/rebased previous linting failure. I was unable to apply the same Also, accidentally found a typo in Running full state tests locally while CircleCI checks linting, will submit afterwards. |
There was a typo in the "complexity" calculation function, a special interim value from EIP-198 used to determine total gas use. The code path was never previously exercised. NOTE: previous-commit "fixtures bump" was to commit: 9b1f07c58a70d1b17c4489c49eb9bebf4a27d290 Squashed commit: tests: update "very big number" in test_modexp_gas_fee_calculation(). ... and also fix that test's name, from "calculTation". The very-big-number is not actually in EIP-198; the latter has this to say: > it’s not possible to provide enough gas to make that computation. That's a bit cryptic, but the gist is that the most that can be represented in a 256-bit number is 0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff which is 115792089237316195423570985008687907853269984665640564039457584007913129639935 and that's less than the 10684346944173007063723051170445283632835119638284563472873463025465780712173320789629146724657549280936306536701227228889744512638312451529980055895215896 required by this vector, or even the (erroneous) 708647586132375115992254428253169996062012306153720251921480414128428353393856280 that was in the test previously.
e93255c
to
7752f05
Compare
Evaluation of the next failure - that unearthed by ethereum/tests#419 - starts here. Try it using LLL helper contract:
Its disassembly:
Expected values at storage (
|
Candidate, narrowed down to a difference in gas usage:
The difference is 15000, which could suggest a miscalculation of gas use on |
@veox not sure what commit you are on, but the net gas metering for SSTORE hasn't been implemented yet so if those are constantinople tests that could be the case. If not, the simple case is a small miscalculation, but I've found that often the gas usage difference is actually indicative of a previous error where either a value should not have been written to storage (but it was) and thus a subsequent write to that storage slot costs less than it should, or some other variant of the same thing. |
@pipermerriam Thanks for the suggestion! I'm trying not to assume things ("15000 therefore This is a Byzantium test, so net gas metering isn't in play yet. I've edited the relevant previous comment to indicate the commit used, for the sake of easier reproduction. |
OK, here's what I think is going on in this test. There is an account A call is made by another account, At this point - question: what is the value at storage location
(Irrelevant section snipped on 2018-12-11.) |
Is the tl;dr that a (nice bug hunting btw) |
It would seem so; except the yellowpaper (Section 7, "Contract Creation") either explicitly states otherwise, is contradictory, or is under-specified:
But then later:
The YP does not consider a "synthetic" case, like in this test. @holiman (near-copy from gitter chat) It seems that ethereum/EIPs#684 considers {some- The proposition is somewhat ludicrous, but here we are!.. I assume the test case is being run, and passes against all the implementations handled by Hive?.. |
@veox it appears you've got a handle on the why (your explanation makes sense to me). Let me know if you need guidance on how to proceed with fixing things. |
@pipermerriam Cool! Definitely confused here, might take a day or two to formulate something concise. It seems to me that the test case is in contradiction to the yellowpaper. If so, then all the major implementations may be as well (need to verify).
Would be nice to get confirmation from |
29f11ac
to
e02123e
Compare
TL;DR: I won't be fixing this one. q:D The "correct" approach from the very beginning would have been immediately marking the test As to the test itself - IMO this really should be fixed upstream. Leaving a newly- This difference between However, it seems that this particular detail is a quirk in the test definition; it's probably testing for something else. By marking the test If someone's inclined, they can push A-a-and a test still fails on CircleCI. Will investigate another day. |
We recently fixed a bug for checkpointing in openethereum/parity-ethereum#9319, and I believe right now, parity's behavior on this is the same as py-evm -- on |
58b8192
to
bdfba46
Compare
…evertInCreateInInit as xfail. Rolls forwards up to 2018-03-01. Break in `RevertInCreateInInit.json` determined by `git bisect run`. The test is marked `xfail` to expicitly highlight the fact. This is done in 3 places - all are run as part of CI.
bdfba46
to
11d61b0
Compare
Determined state tests as "good" by `git bisect run`.
Current state might be where the "simple" fixes end. The next merge commit in Unmarking as Should be OK to squash-merge, especially since there's an interim commit that failed CI. |
…e-appear (#9532) Because more tests won't hurt. :) Add a test case for ethereum/py-evm#1224 (comment) where if contract creation fails, old storage values (if ever existed) should re-appear.
What was wrong?
See issue #1152, in particular #1152 (comment).
ethereum/tests
git submodule infixtures
is rather old:py-evm
PR Enable state tests for Constantinople #1181 enablesConstantinopleVM
testing with a much newer test set; having these changes "lumped together" makes it harder to attribute CI run failures to one or the other, at least at a cursory look.How was it fixed?
Incrementally:
py-evm
modifications.MODEXP
precompile was too cheap.RevertInCreateInInit
asxfail
, for bothStateTests
andBlockchainTests
.WIP: current state
MODEXP
precompile.py-evm
calculated the gas price wrong (this is not the same bug Parity had). Discussion starts at Update ethereum/tests fixtures #1224 (comment).CREATE
in init code, followed by returndatasize/copy". Discussion starts at Update ethereum/tests fixtures #1224 (comment).Cute Animal Picture
"Love this fixture..."
Source: eleejhon @ imgur