-
Notifications
You must be signed in to change notification settings - Fork 759
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SpuriousDragon HF Support #791
Conversation
Codecov Report
@@ Coverage Diff @@
## master #791 +/- ##
==========================================
+ Coverage 84.19% 84.44% +0.25%
==========================================
Files 16 18 +2
Lines 1221 1241 +20
Branches 246 246
==========================================
+ Hits 1028 1048 +20
Misses 125 125
Partials 68 68
Continue to review full report at Codecov.
|
Ok, this is actually very much doable. Not completely there yet but state tests are passing already. 😄 The TODO list from above has still to be completed. Was just working on re-adding pre-Byzantium EIP-658 (Tx receipts) when running out of time. This is the change commit here and I came up with the following not yet working test code for the async function runWithHf(hardfork) {
const vm = setupVM({ hardfork: hardfork })
const suite = setup(vm)
const block = new Block(util.rlp.decode(suite.data.blocks[0].rlp))
await setupPreConditions(suite.vm.stateManager._trie, suite.data)
let res = await suite.p.runBlock({
block,
generate: true,
skipBlockValidation: true,
})
return res
}
tape('should return correct HF receipts', async (t) => {
const res = await runWithHf('byzantium')
t.equal(res.receipts[0].status, 1, 'should return correct post-Byzantium receipt format')
res = await runWithHf('spuriousDragon')
console.log(res)
//t.equal(res.receipts[0].stateRoot, 1, 'should return correct pre-Byzantium receipt format')
t.end()
}) Feel free everyone to pick this up - didn't want to commit incomplete here - and generally continue working on this (SpuriousDragon support). Shouldn't be too much work left. |
"test:state:allForks": "npm run test:state -- --fork=Byzantium && npm run test:state -- --fork=Constantinople && npm run test:state -- --fork=Petersburg && npm run test:state -- --fork=Istanbul && npm run test:state -- --fork=MuirGlacier", | ||
"test:state:selectedForks": "npm run test:state -- --fork=Petersburg && npm run test:state -- --fork=Istanbul && npm run test:state -- --fork=MuirGlacier", | ||
"test:state:allForks": "npm run test:state -- --fork=SpuriousDragon && npm run test:state -- --fork=Byzantium && npm run test:state -- --fork=Constantinople && npm run test:state -- --fork=Petersburg && npm run test:state -- --fork=Istanbul && npm run test:state -- --fork=MuirGlacier", | ||
"test:state:selectedForks": "npm run test:state -- --fork=Petersburg && npm run test:state -- --fork=SpuriousDragon", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
State tests are our bottleneck, I was thinking in splitting state tests in two jobs, so instead of a 12min job, it could be two jobs of 6:30s each. Let me know if that would speed up your work @holgerd77, so I can implement it right into this PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the offer, not necessary here on this round, but definitely useful to split these up. We can do test:state:selectedLatestForks
and test:state:selectedNonLatestForks
to give this some (somewhat blurry) semantics for better orientation, rather that test:state:selectedForks1
and ...2
or something.
Just a first-shot idea, feel free to take on or not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've been thinking about an experiment re state tests, and that'd be to have the test runner accept multiple forks, prepare everything once and run each test for all the forks consecutively. It probably won't yield much of a speed-up, but might be worth trying out if the effort required isn't that high.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you re-add Istanbul
and MuirGlacier
to seletedForks
state test since those are run by CI?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My suggestion was to leave this to Everton to readd (see comment below) along splitting the state test run CI command into two (for performance reasons). Could you live with that as well?
...abstractTxReceipt, | ||
} as PreByzantiumTxReceipt | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, @s1na @alcuadrado please have a look if you are satisfied with this implementation, I thought it is not worth to update the checkpointing mechanism here, since this is a feature (which we then doesn't fully provide for pre-Byzantium) rather than consensus critical code.
We can alternatively also just fill in an empty Buffer and avoid changing the StateManager interface (might generally be a useful addition, not sure though). 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some working note: just tested to run the blockchain tests on older HFs with node -r ts-node/register --stack-size=1500 ./tests/tester --blockchain --fork=Byzantium
.
These are currently NOT working and throwing errors like invalid block stateRoot
and invalid receiptTrie
. I can confirm though that this is also happening on master
so this is not induced by the changes here and rather need some separate investigation. @evertonfraga We should also run - I would suggest - 1 additional fork config on the blockchain tests on the nightly test runs.
The Istanbul
tests are running ('node -r ts-node/register --stack-size=1500 ./tests/tester --blockchain --fork=Istanbul'). The older tests not running doesn't necessarily mean that this originates in one or several bugs in our code but can also have its origin on the tests
side and eventually the older HF blockchain tests don't get accurately generated any more respectively have been at the state of the tests
snapshot.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would rather suggest not to do this investigation here but do this separately. Otherwise the scope here gets too big.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The receiptTrie field in the block header is a commitment to the tx receipts, so I wouldn't be surprised if they were incorrect. But I agree we can probably handle this separately because as you said it needs deeper changes. I'd suggest however till then to keep the StateManager interface unchanged and if the intermediate state root is incorrect anyway then just return a dummy value until we dig deeper.
703e806
to
0230d05
Compare
Had another check, the difficulty code in Gave this a last rebase, this is now ready for review. @evertonfraga For practical reasons and to see the |
0230d05
to
2e1508d
Compare
Rebased this on top of #787 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great PR overall and thanks for taking initiative on this Holger! Just had a few minor comments.
@@ -401,9 +401,15 @@ export const handlers: { [k: string]: OpHandler } = { | |||
runState.stack.push(new BN(keccak256(code))) | |||
}, | |||
RETURNDATASIZE: function (runState: RunState) { | |||
if (!runState._common.gteHardfork('byzantium')) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a side-note: we won't be needing these checks anymore after getOpcodesForHF
supports all the forks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup these can be removed as these changes are in master.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, but this will be another PR.
Can you approve here again so that we can merge? Sina already did but I dismissed by doing the rebase. 😋
...abstractTxReceipt, | ||
} as PreByzantiumTxReceipt | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The receiptTrie field in the block header is a commitment to the tx receipts, so I wouldn't be surprised if they were incorrect. But I agree we can probably handle this separately because as you said it needs deeper changes. I'd suggest however till then to keep the StateManager interface unchanged and if the intermediate state root is incorrect anyway then just return a dummy value until we dig deeper.
2e1508d
to
5f55c10
Compare
This is now ready for re-review. //cc @s1na |
5f55c10
to
2616b99
Compare
2616b99
to
f3017fd
Compare
@ryanio thanks for the branch update, @s1na (or @evertonfraga if still awake 😛 , but don't stay up for this), can one of you give this another look? |
Update: please wait with merging this in here, I will do some other reviews this morning and then do a rebase here on top. |
a0160af
to
36f3d4e
Compare
Ok, rebased this. This is now open for another look. |
(also feel free to directly merge in upon review) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me, thanks Holger!
…VM and package.json
…ASIZE and RETURNDATACOPY
…14) and REVERT (EIP-140)
…ForHF functionality
Rebased this, this would need a re-approval now. |
Part of #652
References
TODO
Byzantium HF Activation Switches
Byzantium Meta Issue: #209
Check that all EIPs from the respective newer HF get disabled/reverted, taking in the list from #161 here:
Block
libraryREVERT
instructionRETURNDATA
opcodesSTATICCALL
opcodeBlock
librarySpuriousDragon HF EIPs - Recap and Activation Switch Checks
Rather for preparing older HF integration, technically not needed.
Tx
library Support spurious dragon and later ethereumjs-tx#132 with HF switch in placeCommon
libraryTangerineWhistle
integrationTest Situation
Test run with HF activated and no changes applied: