Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove ballast from genesis block #1671

Merged
merged 2 commits into from
Jul 18, 2023
Merged

Conversation

nazar-pc
Copy link
Member

We had a piece of data in the header, which worked acceptably well when segments were tiny. When we increased recorded history segment to 128MiB, suddenly we started getting issues with RPC that can't return the data, when returned JS will be slow to decode and moving ballast to extrinsics requires even more hacks since object mapping fails to allocate enough memory in WASM for such a big extrinsic.

In the end I decided to leverage the fact that in SCALE encoding size of following data (except in special cases) is known beforehand. So by appending some extra data after encoded data we can store whatever we want without affecting decoding. This is exact what I did here, adding essentially invisible pseudo-random data to the end of the encoding of genesis block.

The first commit is necessary, but it was also useful regardless, hence separate commit.

@dariolina I do not recall if we mention somewhere how we do this, but if we do, then we need to update older description to reflect this change. Ended up being quite elegant actually.

Code contributor checklist:

@nazar-pc nazar-pc added the breaking-consensus This PR introduces breaking changes to the consensus label Jul 18, 2023
@nazar-pc nazar-pc requested a review from rg3l3dr as a code owner July 18, 2023 04:25
Copy link
Member

@dariolina dariolina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Smart! Added a sentence on how archiving is bootstraped to consensus spec here

@nazar-pc nazar-pc merged commit 437da57 into main Jul 18, 2023
@nazar-pc nazar-pc deleted the remove-ballast-from-genesis-block branch July 18, 2023 10:34
ParthDesai added a commit to autonomys/subspace-pulsar-sdk that referenced this pull request Aug 4, 2023
ParthDesai added a commit to autonomys/subspace-pulsar-sdk that referenced this pull request Aug 15, 2023
ParthDesai added a commit to autonomys/subspace-pulsar-sdk that referenced this pull request Aug 15, 2023
ParthDesai added a commit to autonomys/subspace-pulsar-sdk that referenced this pull request Aug 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-consensus This PR introduces breaking changes to the consensus
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants