Make max_blobs_per_block a config parameter

[pawanjay176]

Issue Addressed

N/A

Proposed Changes

Change max_blobs_per_block from a preset value to a config value. This affects a lot of the codebase where we have to use runtime variants of ssz List and Vector.

[jimmygchen]

I've just added the 6.0.0 label to this - I think it would be great if we can release this with the getBlobsV1 changes, as we're keen to get some data on blob count increase with the optimisation. Ideally this get merged relatively soon given the amount of files touched, however this is a nice to have, so we can drop this in favour of higher priority issues.

[jimmygchen]

The fix looks good, thanks! Lets' goooo 🚀

[jimmygchen]

We just found a bug with this:

lighthouse/beacon_node/lighthouse_network/src/rpc/methods.rs

Lines 336 to 345 in 04b3743

	impl BlobsByRangeRequest {
	/// This function provides an upper bound on number of blobs expected in
	/// a certain slot range.
	///
	/// Note: **must not** use for anything consensus critical, only for
	/// bounds checking and rate limiting.
	pub fn max_blobs_requested<E: EthSpec>(&self) -> u64 {
	self.count.saturating_mul(MAX_BLOBS_PER_BLOCK_CEILING)
	}
	}

This function is used in a validation here:

lighthouse/beacon_node/network/src/network_beacon_processor/rpc_methods.rs

Lines 893 to 894 in 3838897

	// Should not send more than max request blocks
	if req.max_blobs_requested::<T::EthSpec>() > self.chain.spec.max_request_blob_sidecars {

and this will always fail if someone sends a range request with more than max_request_blob_sidecars sidecars or 768 / 16 = 48 blocks.

We need to update the validation here to use the spec max_blob_per_block config instead of using a ceiling value.

[pawanjay176]

@jimmygchen Nice catch. I moved the check from the rpc_methods to the decoding of the request.
Also threaded the chainspec and current_fork to the max_responses function. Its not ideal but it isn't too bad imo because it does make sense that the max_response size depends on the current_fork.

@jxs had to add the ForkContext into the rate limiter in 70917f7 .I did not like this initially but I feel that's the best way to avoid the ugliness with keeping different max values in sync. Having the fork context in the rate limiter doesn't seem ugly because rate limiting is also fork dependent. Happy to explore a different solution if you feel otherwise.

[jimmygchen]

Looks great now \o/

[jimmygchen]

@mergify queue

[mergify]

queue

✅ The pull request has been merged automatically

The pull request has been merged automatically at 0572729