Batch sequencer blocks to post more compact blobs to Celestia #1049
Comments
github-merge-queue bot
pushed a commit
that referenced
this issue
May 8, 2024
… Celestia fees (#1045) ## Summary Batch mutiple Sequencer blocks into single Celestia blobs to reduce fee payments. ## Background Until now, each Sequencer block was turned into multiple blobs (one for overall sequencer block metadata, and one blob per rollup that had transactiosn in the sequencer block). This wasn't as efficient as it could be because the new compression scheme introduced in #1006 can only come to bear with more bytes to compress. Relayer will collect sequencer blocks up to a total (compressed) size of 1MB (1/2 of the current max of 2MB that Celestia blocks can be). ## Changes - Introduce protobuf messages `astria.sequencerblock.v1alpha1.CelestiaHeaderList` and `astria.sequencerblock.v1alpha1.CelestiaRollupDataList` - Rename `astria.sequencerblock.v1alpha1.CelestiaRollupBlob` to `astria.sequencerblock.v1alpha1.CelestiaRollupData` - Rename `astria.sequencerblock.v1alpha1.CelestiaSequencerBlob` to `astria.sequencerblock.v1alpha1.CelestiaHeader` - Collect Sequencer Blocks into the `*List` protobuf messages before posting them to Celestia (instead of splitting up each Sequencer block into mutiple blobs and posting them one by one). ## Testing Add unit tests around the next submission aggregation logic. Update conductor blackbox tests. ## Metrics + `CELESTIA_PAYLOAD_CREATION_LATENCY`: histogram with microsecond units to track the time it takes to create a payload of Celestia blobs (encoding + compressing all protobufs) + metrics for reporting compression ratio and total compressed payload size were moved from the payload/blob construction phase to the submission phase. ## Breaking Changelist - Relayer and Conductor write/read new protobuf messages to/from Celestia. ## Related Issues Closes #1042 Closes #1049
github-merge-queue bot
pushed a commit
that referenced
this issue
May 27, 2024
## Summary Limits the number of requests conductor sends to the Sequencer CometBFT endpoint to 100 per minute. ## Background During sync conductor can DOS Sequencer's CometBFT node by sending too many requests for commits and validator sets. With the batching logic introduced in #1049 there can be dozens (or more) blocks stored in each Celestia blob, each of which needs to be checked separately. With several blobs being fetched at once during, this can quickly spiral into hundreds (if not thousands) requests per minute. Note that only calls to `/commit` and `/validators` are rate limited, because there is currently no way to enforce this at the transport layer, see this issue: informalsystems/tendermint-rs#1420 However, the only other calls are to `/genesis` (once at startup), and `/abci_info` (every block-time period, usually every 2 seconds), which is rare enough to not need a rate limit. ## Changes - Use a tower `RateLimit` middleware around a tendermint-rs `HttpClient` only send up to 100 requests per minute. ## Breaking changes - Adds an environment variable `ASTRIA_CONDUCTOR_SEQUENCER_REQUESTS_PER_SECOND` to configure rate-limiting of requests sent to the Sequencer CometBFT node for verification of Sequencer block data fetched from Celestia blobs ## Testing This needs to be observed end-to-end, potentially letting conductor run for a very long time with only soft commits, and then turning firm commits on. ## Related Issues closes #1064 --------- Co-authored-by: Jordan Oroshiba <jordan@astria.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
No description provided.
The text was updated successfully, but these errors were encountered: