Historical block / state pruning

[nflaig]

Problem description

Right now, the longer a user is running Lodestar, the more storage is required. This is not ideal and the only way right now to free up space is by deleting the db and checkpoint sync.

Ideally, we want to clean up historical blocks after a certain time period (e.g MIN_EPOCHS_FOR_BLOCK_REQUESTS = ~5 months) to keep storage requirements constant and at the same time be able to serve historical blocks to the network (required by range sync).

This will become even more relevant once we enable backfill sync.

Solution description

Clean up historical blocks after certain time period. We might wanna consider putting this behind a feature flag as nodes that are intentionally synced from genesis (or earlier checkpoint) do not want that data to be pruned.

Additional context

Discord discussion

[wemeetagain]

also pruning old states which (depending on configuration) may be equal or greater source of disk usage

[nflaig]

also pruning old states which (depending on configuration) may be equal or greater source of disk usage

Can apply the same to states but I think we prune most of these already and store them very sparsely

lodestar/packages/beacon-node/src/chain/archiver/archiveStates.ts

Lines 67 to 70 in ff253a7

	const statesSlotsToDelete = computeStateSlotsToDelete(storedStateSlots, archiveStateEpochFrequency);
	if (statesSlotsToDelete.length > 0) {
	await this.db.stateArchive.batchDelete(statesSlotsToDelete);
	}

We might wanna get some metrics on this, could be done via db.approximateSize by passing the corresponding range

[twoeths]

regarding pruning old state, if we follow the design to only store state diff instead of state to disk then it becomes really lightweight and we don't need to think about this anymore, see https://github.com/dapplion/tree-states-review-guide/blob/main/persisted_hdiff.md

also it'll help with the historical state regen as stated in #6033 (comment)

we should prioritize that work instead imo