Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

shares: Implement the SharesService #23

Closed
5 tasks done
renaynay opened this issue Aug 23, 2021 · 8 comments · Fixed by #167
Closed
5 tasks done

shares: Implement the SharesService #23

renaynay opened this issue Aug 23, 2021 · 8 comments · Fixed by #167
Labels
area:shares Shares and samples

Comments

@renaynay
Copy link
Member

renaynay commented Aug 23, 2021

Implement SharesService in package shares.

Implement:

  • type Share
  • Constructor function for SharesService

The SharesService contains the following methods / functionality:

  • Start
  • Stop

For ShareExchange, a light node only requires the ability to request shares and perform sampling (random share requests across random peers), and store the shares they receive, while a full node should be able to serve those share requests as well as perform requests.

ShareStore only applies to light nodes as full nodes store full blocks, so if the node is a full node, ShareStore should be disabled.


type Service interface {
	GetShare(ctx context.Context, dah header.DataAvailabilityHeader, row, col int) (Share, error)
	GetShares(context.Context, DataAvailabilityHeader) ([]Shares, error)
	GetSharesByNamespace(context.Context, DataAvailabilityHeader, namespace.ID) ([]Shares, error)
	
	Start(context.Context) error
	Stop(context.Context) error
}
@renaynay renaynay added the area:shares Shares and samples label Aug 23, 2021
@renaynay
Copy link
Member Author

Should there be a pruning mechanism / threshold for light nodes ShareStore? cc @Wondertan @liamsi

@liamsi
Copy link
Member

liamsi commented Aug 23, 2021

I think full storage nodes certainly need pruning (not necessarily for devnet/testnet but for mainnet).

Light nodes download even less shares. Hence, pruning for light nodes is even less of a priority. That said, if light nodes get subjectively re-initialized we might consider overwriting the existing header (and shares) store. So that way we would have implicit pruning: whenever the light node gets re-initialized it throws away old headers and corresponding shares.

@renaynay renaynay changed the title shares: Implement the constructor function for the SharesService shares: Implement the SharesService Aug 23, 2021
@renaynay renaynay changed the title shares: Implement the SharesService shares: Implement the SharesService Aug 23, 2021
@Wondertan
Copy link
Member

Light nodes download even less shares.

Light nodes do DAS and still download a reasonable amount of shares, which should be pruned. I am not sure though what should be the default behavior between storing and pruning, or storing only for last n heights and prune.

@renaynay
Copy link
Member Author

I believe @liamsi said something along the lines of we'll only prune when node is subjectively re-initialised? So we'll store all shares we get in the meantime. I don't know if pruning is within the scope of devnet tho but good to think about. We should make an issue for it.

@renaynay
Copy link
Member Author

renaynay commented Nov 5, 2021

TODO:

  • Integrate GetSharesByNamespace into shares.Service inside service/share, ipld: adds ability to retrieve shares by namespace ID #170
  • Implement GetShares (ability to reconstruct whole square) -- this is already implemented somewhere, so mostly copy and paste, but better to refactor it a bit for networking, concurrency, and storage optimisations (but this can be left for post Devnet, just need to document it in an issue).

@Wondertan
Copy link
Member

Wondertan commented Nov 13, 2021

Reopen as there is still one simple todo left:

@renaynay
Copy link
Member Author

renaynay commented Nov 29, 2021

TODO:

  • GetShares

@Wondertan
Copy link
Member

@renaynay, GetShares are required for Full Availability so start with this one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:shares Shares and samples
Projects
No open projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants