docs: data square layout specs update part 1 #1905

evan-forbes · 2023-06-12T02:00:09Z

Overview

closes #1606, although I still think there is work to update this portion of the specs. I didn't want to leap to that yet, so I've limited the changes here to mostly updating to the new non-interaction rules

Checklist

New and updated code has appropriate documentation
New and updated code has new and/or updated testing
Required CI checks are passing
Visual proof for any user facing features like CLI or documentation updates
Linked issues closed with keywords

github-actions · 2023-06-12T02:00:34Z

PR Preview Action v1.4.4
🚀 Deployed preview to https://celestiaorg.github.io/celestia-app/pr-preview/pr-1905/
on branch `gh-pages` at 2023-06-15 19:34 UTC

codecov-commenter · 2023-06-12T02:13:15Z

Codecov Report

Merging #1905 (1948b44) into main (b370d72) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    celestiaorg/celestia-app#1905   +/-   ##
=======================================
  Coverage   21.21%   21.21%           
=======================================
  Files         120      120           
  Lines       13695    13695           
=======================================
  Hits         2905     2905           
  Misses      10500    10500           
  Partials      290      290

Impacted Files	Coverage Δ
pkg/shares/non_interaction_rules.go	`100.00% <ø> (ø)`

specs/src/specs/data_square_layout.md

specs/src/specs/block_proposer.md

cmwaters · 2023-06-13T12:22:47Z

specs/src/specs/data_square_layout.md

@@ -37,16 +36,16 @@ Specifically, blobs must begin at a new share. We note a nice property from this

 This, however, requires the block producer to interact with the transaction sender to provide them the starting location of their blob, so that the sender can sign over the commitment based on that starting location. This can be done selectively, but is not ideal as a default for e.g. end-user wallets.

-### Non-Interactive Default Rules
+### Non-Interaction Rules


Suggested change

### Non-Interaction Rules

### Share Commitment Rules

or Blob Commitment Rules

[not blocking] or Blob Share Commitment Rules 🙂

Yeah I'm a fan of that

cc also curious what @adlerjohn thinks about the name

we've changed them to "Blob Share Commitment Rules" for now. Personally, I'd opt for something like "Blob Layout Rules" or just "Blob Commitment Rules", but bscr are suitable as well

I'm not too picky though, whatever people can get behind I'm good with

specs/src/specs/data_square_layout.md

specs/src/specs/shares.md

specs/src/specs/data_square_layout.md

cmwaters · 2023-06-15T08:16:43Z

I'm just going to pick this up (hope that's okay @evan-forbes)

specs/src/specs/data_square_layout.md

Co-authored-by: Rootul P <rootulp@gmail.com>

cmwaters · 2023-06-15T08:20:46Z

Meta comment here: In all honesty, you probably should not be able to approve a PR that you made commits too but whatever

rootulp · 2023-06-15T17:31:48Z

specs/src/specs/data_square_layout.md


-We want to arrange this data into a `k * k` matrix of fixed-sized shares, which will later be committed to in [Namespace Merkle Trees (NMTs)](../specs/data_structures.md#namespace-merkle-tree) so that individual shares in this matrix can be proven to belong to a single data root.
+We want to arrange this data into a `k * k` matrix of fixed-sized [shares](../specs/shares.md), which will later be committed to in [Namespace Merkle Trees (NMTs)](https://github.com/celestiaorg/nmt/blob/v0.16.0/docs/spec/nmt.md) so that individual shares in this matrix can be proven to belong to a single data root.


[no change needed][question] for consistency, should we refer to it as "Namepsace" or "Namespaced" Merkle Trees?

https://github.com/celestiaorg/nmt README says "Namespaced" but TBH I prefer "Namespace"

I'm good with either, but the LL paper uses "namespaced"

rootulp · 2023-06-15T17:34:10Z

specs/src/specs/data_square_layout.md


-This is similar to [Merkle Mountain Ranges](https://www.usenix.org/legacy/event/sec09/tech/full_papers/crosby.pdf), though with the largest subtree bounded by the blob minimum square size rather than being unbounded.
+In the constraint mentioned above, the number of rows/columns in the minimum square size should be a power of 2.


[question] is this line still applicable? There is no reference to minimum square size above

resolved in bfa854b

rootulp · 2023-06-15T17:35:21Z

specs/src/specs/networking.md


-Receiving a `MsgWirePayForData` object from the network follows the reverse process: verify using the [non-interactive default rules](../rationale/data_square_layout.md#non-interactive-default-rules) that the signature is valid.
+Receiving a `MsgWirePayForData` object from the network follows the reverse process: verify using the [non-interaction rules](../specs/data_square_layout.md#non-interaction-rules) that the signature is valid.


[blocking][question] if we're renaming non-interactive default rules to non-interaction default rules, should we do a global find + replace of the codebase in this PR?

good point 2a24af8

I didn't touch the ADRs as there are an insane number of references, although we might want to in the future

I thought we were going to rename it from non-interactive default rules to blob share commitment rules 🙃

evan-forbes · 2023-06-15T19:37:37Z

there were some merge conflicts, so I went ahead and updated them along with feedback from Rootul

cmwaters

Nooice

* docs: data square layout part 1 * docs: add power of 2 note * Apply suggestions from code review Co-authored-by: Rootul P <rootulp@gmail.com> --------- Co-authored-by: Callum Waters <cmwaters19@gmail.com> Co-authored-by: Rootul P <rootulp@gmail.com>

docs: data square layout part 1

9bd0730

evan-forbes added the specs directly relevant to the specs label Jun 12, 2023

evan-forbes self-assigned this Jun 12, 2023

evan-forbes requested review from liamsi, adlerjohn, MSevey, rootulp and cmwaters as code owners June 12, 2023 02:00

MSevey requested a review from a team June 12, 2023 02:00

evan-forbes mentioned this pull request Jun 12, 2023

docs: update rational for specs non-interactive defaults #1630

Closed

5 tasks

evan-forbes mentioned this pull request Jun 12, 2023

docs: update the blob module specs part 1 #1906

Merged

5 tasks

cmwaters reviewed Jun 12, 2023

View reviewed changes

specs/src/specs/data_square_layout.md Outdated Show resolved Hide resolved

docs: add power of 2 note

5fed294

MSevey requested a review from a team June 12, 2023 14:42

Merge branch 'main' into evan/data-square-layout-specs-part-1

735d516

cmwaters reviewed Jun 13, 2023

View reviewed changes

rootulp previously approved these changes Jun 13, 2023

View reviewed changes

cmwaters mentioned this pull request Mar 11, 2024

docs: update the spec with how square construction is currently implemented celestiaorg/go-square#50

Open

cmwaters reviewed Jun 15, 2023

View reviewed changes

specs/src/specs/data_square_layout.md Outdated Show resolved Hide resolved

Apply suggestions from code review

33a1169

Co-authored-by: Rootul P <rootulp@gmail.com>

cmwaters dismissed rootulp’s stale review via 33a1169 June 15, 2023 08:18

MSevey requested a review from a team June 15, 2023 08:18

Merge branch 'main' into evan/data-square-layout-specs-part-1

1948b44

cmwaters requested review from cmwaters and rootulp June 15, 2023 08:18

cmwaters previously approved these changes Jun 15, 2023

View reviewed changes

evan-forbes enabled auto-merge (squash) June 15, 2023 11:48

MSevey previously approved these changes Jun 15, 2023

View reviewed changes

rootulp reviewed Jun 15, 2023

View reviewed changes

evan-forbes added 3 commits June 15, 2023 14:05

Merge branch 'main' into evan/data-square-layout-specs-part-1

4f01d46

docs: remove old line

bfa854b

docs: replace non-interactive/non-interaction

2a24af8

evan-forbes dismissed stale reviews from MSevey and cmwaters via 2a24af8 June 15, 2023 19:34

MSevey requested a review from a team June 15, 2023 19:34

cmwaters approved these changes Jun 16, 2023

View reviewed changes

evan-forbes requested review from MSevey and rootulp June 16, 2023 14:32

Merge branch 'main' into evan/data-square-layout-specs-part-1

404c6c6

MSevey requested a review from a team June 16, 2023 14:33

rootulp approved these changes Jun 16, 2023

View reviewed changes

evan-forbes merged commit 922cdf6 into main Jun 16, 2023

evan-forbes deleted the evan/data-square-layout-specs-part-1 branch June 16, 2023 15:06

evan-forbes mentioned this pull request Jun 20, 2023

specs: update non-interactive default rules #1063

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: data square layout specs update part 1 #1905

docs: data square layout specs update part 1 #1905

evan-forbes commented Jun 12, 2023

github-actions bot commented Jun 12, 2023 •

edited

Loading

codecov-commenter commented Jun 12, 2023 •

edited

Loading

cmwaters Jun 13, 2023

rootulp Jun 13, 2023 •

edited

Loading

cmwaters Jun 14, 2023

evan-forbes Jun 15, 2023

evan-forbes Jun 15, 2023

cmwaters commented Jun 15, 2023

cmwaters commented Jun 15, 2023

rootulp Jun 15, 2023

evan-forbes Jun 15, 2023

rootulp Jun 15, 2023

evan-forbes Jun 15, 2023

rootulp Jun 15, 2023 •

edited

Loading

evan-forbes Jun 15, 2023

evan-forbes Jun 15, 2023

cmwaters Jun 16, 2023 •

edited

Loading

evan-forbes commented Jun 15, 2023

cmwaters left a comment


		We want to arrange this data into a `k * k` matrix of fixed-sized shares, which will later be committed to in [Namespace Merkle Trees (NMTs)](../specs/data_structures.md#namespace-merkle-tree) so that individual shares in this matrix can be proven to belong to a single data root.
		We want to arrange this data into a `k * k` matrix of fixed-sized [shares](../specs/shares.md), which will later be committed to in [Namespace Merkle Trees (NMTs)](https://github.com/celestiaorg/nmt/blob/v0.16.0/docs/spec/nmt.md) so that individual shares in this matrix can be proven to belong to a single data root.


		This is similar to [Merkle Mountain Ranges](https://www.usenix.org/legacy/event/sec09/tech/full_papers/crosby.pdf), though with the largest subtree bounded by the blob minimum square size rather than being unbounded.
		In the constraint mentioned above, the number of rows/columns in the minimum square size should be a power of 2.


		Receiving a `MsgWirePayForData` object from the network follows the reverse process: verify using the [non-interactive default rules](../rationale/data_square_layout.md#non-interactive-default-rules) that the signature is valid.
		Receiving a `MsgWirePayForData` object from the network follows the reverse process: verify using the [non-interaction rules](../specs/data_square_layout.md#non-interaction-rules) that the signature is valid.

docs: data square layout specs update part 1 #1905

docs: data square layout specs update part 1 #1905

Conversation

evan-forbes commented Jun 12, 2023

Overview

Checklist

github-actions bot commented Jun 12, 2023 • edited Loading

codecov-commenter commented Jun 12, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

rootulp Jun 13, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmwaters commented Jun 15, 2023

cmwaters commented Jun 15, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rootulp Jun 15, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmwaters Jun 16, 2023 • edited Loading

Choose a reason for hiding this comment

evan-forbes commented Jun 15, 2023

cmwaters left a comment

Choose a reason for hiding this comment

github-actions bot commented Jun 12, 2023 •

edited

Loading

codecov-commenter commented Jun 12, 2023 •

edited

Loading

rootulp Jun 13, 2023 •

edited

Loading

rootulp Jun 15, 2023 •

edited

Loading

cmwaters Jun 16, 2023 •

edited

Loading