Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

das light client (mvp) #323

Merged
merged 91 commits into from
May 24, 2021
Merged

das light client (mvp) #323

merged 91 commits into from
May 24, 2021

Conversation

liamsi
Copy link
Member

@liamsi liamsi commented May 11, 2021

Description

MVP data availability sampling light client

For instructions how to run this: #323 (comment)

ref: #311

@codecov-commenter
Copy link

codecov-commenter commented May 23, 2021

Codecov Report

Merging #323 (fe55882) into master (880ede1) will decrease coverage by 0.19%.
The diff coverage is 23.72%.

@@            Coverage Diff             @@
##           master     #323      +/-   ##
==========================================
- Coverage   61.98%   61.79%   -0.20%     
==========================================
  Files         263      262       -1     
  Lines       22931    22919      -12     
==========================================
- Hits        14213    14162      -51     
- Misses       7207     7258      +51     
+ Partials     1511     1499      -12     
Impacted Files Coverage Δ
ipfs/plugin/nmt.go 30.58% <ø> (ø)
light/detector.go 81.30% <ø> (ø)
light/provider/http/http.go 40.86% <0.00%> (-15.03%) ⬇️
light/verifier.go 69.23% <0.00%> (-2.06%) ⬇️
p2p/ipld/validate.go 54.54% <0.00%> (ø)
rpc/client/mock/client.go 20.00% <0.00%> (-1.22%) ⬇️
rpc/core/blocks.go 30.76% <0.00%> (-4.02%) ⬇️
rpc/core/routes.go 0.00% <ø> (ø)
rpc/core/types/responses.go 33.33% <ø> (ø)
types/light.go 72.52% <ø> (ø)
... and 15 more

@liamsi liamsi removed the request for review from adlerjohn May 23, 2021 07:43
// persistent kvstore application and special consensus wal instance
// (byteBufferWAL) and waits until numBlocks are created.
// If the node fails to produce given numBlocks, it returns an error.
func WALGenerateNBlocks(t *testing.T, wr io.Writer, numBlocks int) (err error) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are only used in consensus/wal_test.go. Hence, I've moved them there (and it also helped with re-using the same ipfsAPI object in TestMain, see consensus/replay_test.go).

Comment on lines +30 to +34
// waitingTime estimates how long it should take for a node to reach the height.
// More nodes in a network implies we may expect a slower network and may have to wait longer.
func waitingTime(nodes int) time.Duration {
return time.Duration(30+(nodes*4)) * time.Second
}
Copy link
Member Author

@liamsi liamsi May 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slightly modified version of: tendermint/tendermint#6202

cmd/tendermint/commands/light.go Outdated Show resolved Hide resolved
@@ -177,6 +178,14 @@ func ValidateTrustLevel(lvl tmmath.Fraction) error {
return nil
}

func ValidateNumSamples(numSamples uint32) error {
maxShares := consts.MaxSquareSize * consts.MaxSquareSize
if numSamples == 0 && numSamples < uint32(maxShares) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maxShares should never be 0, so the second condition is trivially true if the former is true. Should remove (and ideally place these assumptions in a separate file with static assertions like https://github.com/bitcoin-core/secp256k1/blob/3dc8c072b6d84845820c1483a2ee21094fb87cc3/src/assumptions.h).

Also, this should be numSamples > uint32(maxShares).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great find. Removed the 2nd condition as the code will calculate the actual number of samples as:

numSamples := min(c.numSamples, uint32(numRows*numRows))

ideally place these assumptions in a separate file with static assertions like https://github.com/bitcoin-core/secp256k1/blob/3dc8c072b6d84845820c1483a2ee21094fb87cc3/src/assumptions.h).

What does static mean in this context? The other checks like this are in this file. I kept the new one here for consistency with the existing code.

Copy link
Member

@adlerjohn adlerjohn May 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static as in your program doesn't even compile without an error if these assumptions are violated. I don't know if Go has such a thing, but something like a small module that runs when the program starts that panics if certain fundamental assumptions are violated (e.g. the platform not being 64 bits, or having set a parameter to a value that was assumed impossible like setting the max size to 0) would serve that purpose. Either way it's orthogonal to this PR.

Copy link
Member Author

@liamsi liamsi May 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static as in your program doesn't even compile without an error if these assumptions are violated.

Yeah, that does not work here as the inputs that are checked are user input (not known during compile time).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if Go has such a thing, but something like a small module that runs when the program starts

https://golang.org/doc/effective_go#init

but this is also during run time, not compile time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant that the max size parameter is not set to 0. That's a compile-time constant, and parts of the code assume it's > 0.

Crashing the program before it even does anything, deterministically, isn't great...but it's better than crashing in the middle randomly based on user input.

light/verifier.go Outdated Show resolved Hide resolved
types/block.go Outdated Show resolved Hide resolved
@liamsi liamsi requested a review from adlerjohn May 23, 2021 16:18
cmd/tendermint/commands/light.go Show resolved Hide resolved
cmd/tendermint/commands/light.go Show resolved Hide resolved
cmd/tendermint/commands/light.go Outdated Show resolved Hide resolved
cmd/tendermint/commands/light.go Show resolved Hide resolved
consensus/mempool_test.go Show resolved Hide resolved
consensus/state.go Show resolved Hide resolved
consensus/state.go Outdated Show resolved Hide resolved
types/block.go Outdated Show resolved Hide resolved
rpc/core/routes.go Show resolved Hide resolved
Comment on lines +180 to +185
func ValidateNumSamples(numSamples uint32) error {
if numSamples == 0 {
return fmt.Errorf("number of samples must be > 0")
}
return nil
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: are we planning on adding more checks in this function? perhaps it would be slightly simpler to just add this check to the place where it's needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: are we planning on adding more checks in this function?

not sure yet.

perhaps it would be slightly simpler to just add this check to the place where it's needed?

I think the code is more consistent with the already existing code and I would argue that that place is more readable like this (alternatively, we could introduce a variable e.g. isValidNumSamples := numSamples != 0. That would the same effect).

Copy link
Member

@evan-forbes evan-forbes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two 👍 from me!
mvp

@liamsi liamsi merged commit 4d99fe6 into master May 24, 2021
@liamsi liamsi deleted the ismail/light-mvp branch May 24, 2021 09:55
@liamsi
Copy link
Member Author

liamsi commented May 24, 2021

eeek, this was not meant to be merged as a merge commit 🤦🏼 Forgot to cleanup the commit history or squash merge instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants