Cherry pick basic DA functionality from celestia-core #536
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* move Messages field to the end of Block.Data
* Add some constants for share computation and the NMT:
- also a bunch of todos regarding shares computation
* First (compiling) stab on creating shares
* Test with Evidence and fix bug discovered by test
* remove resolved todos
* introduce split method
* Introduce LenDelimitedMarshaler interface and some reformatting
* Introduce TxLenDelimitedMarshaler
* add some test cases
* fix some comments
* fix some comments & linter
* Add reserved namespaces to params
* Move ll-specific consts into a separate file (consts.go)
* Add MarshalDelimited to HexBytes
* Add tail-padding shares
* Add ComputeShares method on Data to compute all shares
* Fix compute the next square num and not the next power of two
* lints
* Unexport MakeShares function:
- it's likely to change and it doesn't have to be part of the public API
* lints 2
* First stab on computing row/column roots
* fix rebase glitches:
- move DA related constants out of params.go
* refactor MakeBlock to take in interm. state roots and messages
* refactor state.MakeBlock too
* Add todos LenDelimitedMarshaler and extract appendShares logic
* Simplify shares computation: remove LenDelimitedMarshaler abstraction
* actually use DA header to compute the DataRoot everywhere (will lead to failing tests for sure)
* WIP: Update block related core data structures in protobuf too
* WIP: fix zero shares edge-case and get rid of Block.Data.hash (use dataAvailabilityHeader.Hash() instead)
* Fixed tests, only 3 failing tests to go: TestReapMaxBytesMaxGas, TestTxFilter, TestMempoolFilters
* Fix TestTxFilter:
- the size of the wrapping Data{} proto message increased a few bytes
* Fix Message proto and `DataFromProto`
* Fix last 2 remaining tests related to the increased block/block.Data size
* Use infectious lib instead of leopard
* proto-lint: snake_case
* some lints and minor changes
* linter
* panic if pushing to tree fails, extend Data.ToProto()
* revert renaming in comment
* add todo about refactoring as soon as the rsmt2d allows the user to choose the merkle tree
evan-forbes
commented
Sep 20, 2021
Comment on lines
-126
to
+128
| {20, 240, -1, 10}, | ||
| {20, 240, 10, 10}, | ||
| {20, 240, 15, 10}, | ||
| {20, 280, -1, 10}, | ||
| {20, 280, 10, 10}, | ||
| {20, 280, 15, 10}, |
There was a problem hiding this comment.
Comment on lines
+112
to
+121
| // // fillDataAvailabilityHeader fills in any remaining DataAvailabilityHeader fields | ||
| // // that are a function of the block data. | ||
| // func (b *Block) fillDataAvailabilityHeader() { | ||
| // namespacedShares := b.Data.computeShares() | ||
| // shares := namespacedShares.RawShares() | ||
| // if len(shares) == 0 { | ||
| // // no shares -> no row/colum roots -> hash(empty) | ||
| // b.DataHash = b.DataAvailabilityHeader.Hash() | ||
| // return | ||
| // } |
There was a problem hiding this comment.
planning to do this in a different PR, as share splitting and merging isn't added yet
Comment on lines
48
to
+50
| Header `json:"header"` | ||
| Data `json:"data"` | ||
| Evidence EvidenceData `json:"evidence"` | ||
| LastCommit *Commit `json:"last_commit"` | ||
| LastCommit *Commit `json:"last_commit"` |
evan-forbes
changed the title
|
Will these be squashed or merged btw? |
liamsi
added a commit
that referenced
this pull request
Sep 23, 2021
* Basic DA functionality (#83) * move Messages field to the end of Block.Data * Add some constants for share computation and the NMT: - also a bunch of todos regarding shares computation * First (compiling) stab on creating shares * Test with Evidence and fix bug discovered by test * remove resolved todos * introduce split method * Introduce LenDelimitedMarshaler interface and some reformatting * Introduce TxLenDelimitedMarshaler * add some test cases * fix some comments * fix some comments & linter * Add reserved namespaces to params * Move ll-specific consts into a separate file (consts.go) * Add MarshalDelimited to HexBytes * Add tail-padding shares * Add ComputeShares method on Data to compute all shares * Fix compute the next square num and not the next power of two * lints * Unexport MakeShares function: - it's likely to change and it doesn't have to be part of the public API * lints 2 * First stab on computing row/column roots * fix rebase glitches: - move DA related constants out of params.go * refactor MakeBlock to take in interm. state roots and messages * refactor state.MakeBlock too * Add todos LenDelimitedMarshaler and extract appendShares logic * Simplify shares computation: remove LenDelimitedMarshaler abstraction * actually use DA header to compute the DataRoot everywhere (will lead to failing tests for sure) * WIP: Update block related core data structures in protobuf too * WIP: fix zero shares edge-case and get rid of Block.Data.hash (use dataAvailabilityHeader.Hash() instead) * Fixed tests, only 3 failing tests to go: TestReapMaxBytesMaxGas, TestTxFilter, TestMempoolFilters * Fix TestTxFilter: - the size of the wrapping Data{} proto message increased a few bytes * Fix Message proto and `DataFromProto` * Fix last 2 remaining tests related to the increased block/block.Data size * Use infectious lib instead of leopard * proto-lint: snake_case * some lints and minor changes * linter * panic if pushing to tree fails, extend Data.ToProto() * revert renaming in comment * add todo about refactoring as soon as the rsmt2d allows the user to choose the merkle tree * clean up some unused test helper functions * linter * still debugging the exact right number of bytes for max data... Co-authored-by: Ismail Khoffi <Ismail.Khoffi@gmail.com>
evan-forbes
pushed a commit
that referenced
this pull request
Jun 9, 2023
* removing non updated java and kotlin guides * upgradeing guides order * removing stale java and kotlin guides
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Mainly cherry picks #83, with a few modifications to better fit in a newer version of tendermint, and this PR doesn't add the data availability header to the block.
part of #528