-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
op-node/rollup/derive: Implement pipeline stage multiplexing #12506
Conversation
9853bf0
to
a841424
Compare
a82dad4
to
0d6b571
Compare
a841424
to
12eae24
Compare
0d6b571
to
f24a759
Compare
Semgrep found 1 Prefer Semgrep found 1 No |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few remarks for reviewers.
defbbf4
to
8899e8b
Compare
8da10a1
to
7f8b473
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, but seem to have some test failures.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approving as I don't think there's anything really blocking in my comments. Obviously the tests will have to pass though. :)
7f8b473
to
52e741a
Compare
…m-optimism#12506) * op-node/rollup/derive: Implement pipeline stage multiplexing * fix BatchStage empty batch generation * fix fork configuration in LargeL1Gaps test
Description
This PR implements derivation pipeline stage multiplexing to achieve stage transformations of the channel and batch stage for Holocene.
Stage Multiplexing
The channel and batch stage of the derivation pipeline have two separate implementations for those stages pre- and post-Holocene, and a transformation should happen between those stages when the L1 origin goes past the Holocene activation timestamp (and back if a reset occurs).
BatchQueue
->BatchStage
ChannelBank
->ChannelAssembler
This PR adds the concept of stage transformations to the derivation pipeline. A stage can be a
to implement stage-specific transformation behavior for a particular fork. When the derivation pipeline's
Step
function detect an L1 origin timestamp going past a Fork activation point, it iterates over all stages and calls the transformation function. This is currently optional per stage, so Go interface conversion tests are used to check whether a stage has transformation functionality implemented.This design allows for a modular stage transformation design that encapsulates fork-specific logic to each individual stage.
In this PR, the new architecture is used in two different ways:
FrameQueue
implementation type and instance stay the same pre- and post-Holocene. However, Holocene consensus rules require the frame queue to be reset at Holocene activation. So theFrameQueue
was made aForkTransformer
for Holocene, and simply resets itself.ChannelMux
andBatchMux
were added which orchestrate the transformation of the pre-Holocene stages into the post-Holocene stages.transformHolocene()
functions on the legacyChannelBank
andBatchQueue
stages to obtain the Holocene stages. In the case of theBatchQueue
this allows to port over the collectedl1Blocks
into the new HoloceneBatchStage
.Fork modularity
In anticipation of improved modularity of the codebase around forks, this PR prepares some generic fork functionality for the deploy config and in action test. Also, a common action test environment was added. In future work, existing action tests will be simplified to use the new environment.
Tests
Added action tests that check for the correct stage transformation log messages. The two cases tested are
Metadata