Provide a byte-size capacity check on the blob queue on the sequencer-relayer #1042
Labels
Comments
Fraser999
added
sequencer-relayer
github-merge-queue bot
pushed a commit
that referenced
this issue
May 8, 2024
… Celestia fees (#1045) ## Summary Batch mutiple Sequencer blocks into single Celestia blobs to reduce fee payments. ## Background Until now, each Sequencer block was turned into multiple blobs (one for overall sequencer block metadata, and one blob per rollup that had transactiosn in the sequencer block). This wasn't as efficient as it could be because the new compression scheme introduced in #1006 can only come to bear with more bytes to compress. Relayer will collect sequencer blocks up to a total (compressed) size of 1MB (1/2 of the current max of 2MB that Celestia blocks can be). ## Changes - Introduce protobuf messages `astria.sequencerblock.v1alpha1.CelestiaHeaderList` and `astria.sequencerblock.v1alpha1.CelestiaRollupDataList` - Rename `astria.sequencerblock.v1alpha1.CelestiaRollupBlob` to `astria.sequencerblock.v1alpha1.CelestiaRollupData` - Rename `astria.sequencerblock.v1alpha1.CelestiaSequencerBlob` to `astria.sequencerblock.v1alpha1.CelestiaHeader` - Collect Sequencer Blocks into the `*List` protobuf messages before posting them to Celestia (instead of splitting up each Sequencer block into mutiple blobs and posting them one by one). ## Testing Add unit tests around the next submission aggregation logic. Update conductor blackbox tests. ## Metrics + `CELESTIA_PAYLOAD_CREATION_LATENCY`: histogram with microsecond units to track the time it takes to create a payload of Celestia blobs (encoding + compressing all protobufs) + metrics for reporting compression ratio and total compressed payload size were moved from the payload/blob construction phase to the submission phase. ## Breaking Changelist - Relayer and Conductor write/read new protobuf messages to/from Celestia. ## Related Issues Closes #1042 Closes #1049
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently the relayer has a hard-coded cap of 128 blobs in its
QueuedConvertedBlocksqueue. This is overly pessimistic for the case of syncing a relayer on startup.We should look to replace this limit with one based on the total size in bytes of the blobs, with an initial hard-coded cap of just under 50% of the Celestia blob-size limit.
From experimentation, the blob-size limits on Celestia are:
where these represent the maximum value of
BroadcastTxRequest::tx_bytes.len() + 4(there are 4 bytes consumed in packing the length into the serialized field).The limit for our blob-size should therefore be 981,218 bytes.
The text was updated successfully, but these errors were encountered: