Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependency replication for blobs #492

Closed
sandreae opened this issue Aug 8, 2023 · 0 comments · Fixed by #514
Closed

Dependency replication for blobs #492

sandreae opened this issue Aug 8, 2023 · 0 comments · Fixed by #514

Comments

@sandreae
Copy link
Member

sandreae commented Aug 8, 2023

For context, a similar, but more general, idea has already been raised: p2panda/website#202

We need something along these lines for blobs so that a node can be configured to replicate only certain types of blob documents and their pieces, rather than all blob_v1 and blob_piece_v1 documents. To summarise the behaviour we want is:

  • blob_v1 and blob_piece_v1 are built-in system schema which describe binary data. Nodes are not expected to directly replicate documents following these schema as that would cause a flood of data they are likely not interested in.
  • a user can define their own schema which contains a field relating to blob_v1 , if a node is configured to support this application schema, then it is expected that they also want all related blob_v1 but not necessarily the blob_piece_v1 documents
  • a further step the node can take is to specify that, yes, they actually do want to replicate the blob_piece_v1 documents as well

This should be achievable by adding logic to the replication service which includes dependencies of application schema in their target sets when initiating sync requests.

The desired replication behaviour (eg. "I want all blobs and blob pieces for schema X, Y & Z") would need to be expressed in Announce messages as well.

@sandreae sandreae added this to the Blobs milestone Aug 17, 2023
@adzialocha adzialocha changed the title Dependency replication for blobs Dependency replication for blobs Aug 18, 2023
@adzialocha adzialocha removed this from the Blobs milestone Aug 18, 2023
@sandreae sandreae added this to the Blobs milestone Aug 22, 2023
@sandreae sandreae linked a pull request Aug 31, 2023 that will close this issue
4 tasks
@adzialocha adzialocha removed this from the Blobs milestone Sep 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants