Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

op-node/rollup/derive: Implement pipeline stage multiplexing #12506

Merged
merged 3 commits into from
Oct 28, 2024

Conversation

sebastianst
Copy link
Member

@sebastianst sebastianst commented Oct 17, 2024

Description

This PR implements derivation pipeline stage multiplexing to achieve stage transformations of the channel and batch stage for Holocene.

Stage Multiplexing

The channel and batch stage of the derivation pipeline have two separate implementations for those stages pre- and post-Holocene, and a transformation should happen between those stages when the L1 origin goes past the Holocene activation timestamp (and back if a reset occurs).

  • BatchQueue -> BatchStage
  • ChannelBank -> ChannelAssembler

This PR adds the concept of stage transformations to the derivation pipeline. A stage can be a

type ForkTransformer interface {
	Transform(rollup.ForkName)
}

to implement stage-specific transformation behavior for a particular fork. When the derivation pipeline's Step function detect an L1 origin timestamp going past a Fork activation point, it iterates over all stages and calls the transformation function. This is currently optional per stage, so Go interface conversion tests are used to check whether a stage has transformation functionality implemented.

This design allows for a modular stage transformation design that encapsulates fork-specific logic to each individual stage.

In this PR, the new architecture is used in two different ways:

  • The FrameQueue implementation type and instance stay the same pre- and post-Holocene. However, Holocene consensus rules require the frame queue to be reset at Holocene activation. So the FrameQueue was made a ForkTransformer for Holocene, and simply resets itself.
  • The channel and batch stages have two different implementations pre- and post-Holocene. So-called stage multiplexers ChannelMux and BatchMux were added which orchestrate the transformation of the pre-Holocene stages into the post-Holocene stages.
    • They call the underlying transformHolocene() functions on the legacy ChannelBank and BatchQueue stages to obtain the Holocene stages. In the case of the BatchQueue this allows to port over the collected l1Blocks into the new Holocene BatchStage.
    • When Reset would go back to an L1 origin pre-Holocene, the multiplexers also transform back to the reset pre-Holocene stages.

Fork modularity

In anticipation of improved modularity of the codebase around forks, this PR prepares some generic fork functionality for the deploy config and in action test. Also, a common action test environment was added. In future work, existing action tests will be simplified to use the new environment.

Tests

Added action tests that check for the correct stage transformation log messages. The two cases tested are

  • Holocene at Genesis: immediate transformation
  • Holocene post-Genesis: transformation at the Holocene L1 origin and reset back to a pre-Holocene derivation state.

Metadata

Copy link
Contributor

semgrep-app bot commented Oct 20, 2024

Semgrep found 1 sol-style-notice-over-dev-natspec finding:

  • packages/contracts-bedrock/src/cannon/interfaces/IPreimageOracle.sol

Prefer @notice over @dev in natspec comments

Ignore this finding from sol-style-notice-over-dev-natspec.

Semgrep found 1 golang_fmt_errorf_no_params finding:

  • op-deployer/pkg/deployer/bootstrap/bootstrap.go

No fmt.Errorf invocations without fmt arguments allowed

Ignore this finding from golang_fmt_errorf_no_params.

@sebastianst sebastianst marked this pull request as ready for review October 21, 2024 14:45
Copy link
Member Author

@sebastianst sebastianst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few remarks for reviewers.

op-e2e/actions/helpers/env.go Show resolved Hide resolved
op-e2e/actions/upgrades/helpers/config.go Show resolved Hide resolved
op-e2e/actions/helpers/env.go Show resolved Hide resolved
@sebastianst sebastianst added the M-do-not-merge Meta: Do not merge label Oct 21, 2024
@sebastianst sebastianst linked an issue Oct 22, 2024 that may be closed by this pull request
@sebastianst sebastianst force-pushed the seb/holocene-batch-stage branch 2 times, most recently from defbbf4 to 8899e8b Compare October 22, 2024 15:26
Base automatically changed from seb/holocene-batch-stage to develop October 22, 2024 15:54
@sebastianst sebastianst added A-op-node Area: op-node T-protocol Team: Protocol and removed M-do-not-merge Meta: Do not merge labels Oct 22, 2024
@sebastianst sebastianst added this to the Holocene: Derivation milestone Oct 22, 2024
Copy link
Contributor

@ajsutton ajsutton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but seem to have some test failures.

op-chain-ops/genesis/config.go Outdated Show resolved Hide resolved
op-chain-ops/genesis/config.go Outdated Show resolved Hide resolved
op-node/rollup/chain_spec.go Outdated Show resolved Hide resolved
op-node/rollup/types.go Show resolved Hide resolved
op-node/rollup/derive/channel_mux.go Outdated Show resolved Hide resolved
op-node/rollup/derive/channel_mux_test.go Show resolved Hide resolved
op-node/rollup/derive/batch_mux_test.go Show resolved Hide resolved
op-node/rollup/derive/batch_mux.go Outdated Show resolved Hide resolved
Copy link
Contributor

@ajsutton ajsutton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving as I don't think there's anything really blocking in my comments. Obviously the tests will have to pass though. :)

@sebastianst sebastianst added this pull request to the merge queue Oct 28, 2024
Merged via the queue into develop with commit dbaedac Oct 28, 2024
49 checks passed
@sebastianst sebastianst deleted the seb/holocene-pipeline branch October 28, 2024 20:59
samlaf pushed a commit to samlaf/optimism that referenced this pull request Nov 10, 2024
…m-optimism#12506)

* op-node/rollup/derive: Implement pipeline stage multiplexing

* fix BatchStage empty batch generation

* fix fork configuration in LargeL1Gaps test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-op-node Area: op-node T-protocol Team: Protocol
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Holocene-D: Activation pipeline reset and stages wiring
2 participants