MetadataBook

[vasco-santos]

Overview

With the PeerStore improvements work for libp2p@0.28 #582 , we currently have the AddressBook, ProtoBook and KeyBook instead of the old mapping between peer identifiers and PeerInfo. In this context, we will also be working on the MetadataBook.

The MetadataBook will allow libp2p users to associate data to a specific peer. This data should also be persisted, so that users can rely on them after a node restart. Examples of this data can be peer nicknames, roles in the network, associated apps, etc

Implementation Design

Taking into account the generic purpose use of this book, we need to think about how this data will be stored. Since one peer can have n metadata entries, we would need to have the following type of data structure in the MetadataBook:

Map<string, Map<string, *>>

This is, a map of peer identifier strings to their metadata, which should also be a Map indexed by a string identifier of the metadata and the content associated to that metadata to store.

In the datastore side of things, we will be able to use the following as key:

/peers/metadata/<b32 peer id no padding>/<key>

The metadata data will be stored as ... [TO BE COMPLETED, SEE CONCERNS]

API

[TO BE COMPLETED, SEE CONCERNS]

Concerns

Stored Data

While the "in-memory" book and the datastore key are straightforward, the data we can store might rise some issues. Regarding the "in-memory" book, users would be able to just add any type of content to the store and get it, which would be completely transparent to the users. However, considering that we will need to persist this data to offer a bigger value to the users, we need to know how this data is structured, in order to properly store it in the datastore by marshalling and unmarshalling it on retrieve.

go-libp2p PeerStore MetadataBook accepts any type of object by using the encoding/gob package. This allows them to just marshal any kind of received data for storing it, and unmarshal it for the users.

In JS land, we can potentially use a module like surrial to offer the same capabilities. This module adds 1.3kB to the bundle being minified and gzipped.

Since most users will probably not use this feature, I am reluctant on adding a module for this specific use case. So, other alternatives might be to require users to provide a Buffer to store the data (and users can use this type of module for the marshalling without being part of the libp2p bundle), or to simply accept strings, which we can convert into Buffer easily. Users would also be able to simply convert JS objects into strings and provide it (same for buffers).

API

While for the AddressBook, ProtoBook and KeyBook are similar to JS Maps regarding their API, the MetadataBook will probably not be the case.

The metadata book will have 3 inputs: PeerId, MetadataKey, MetadataValue, instead of 2 like the remaining ones.

• peerStore.metadataBook.set(peerId: PeerId, key: string, value: *)
• peerStore.metadataBook.get(peerId: PeerId) => Get all metadata for the Peer
• peerStore.metadataBook.getValue(peerId: PeerId, key: string) => Get a specific part of the metadata
• peerStore.metadataBook.delete(peerId: PeerId) => Delete all peer metadata from the book
• peerStore.metadataBook.deleteValue(peerId: PeerId, key: string) => Delete part of the metadata of a peer from the book.

[vasco-santos]

cc @jacobheun @lidel

[lidel]

cc @Gozala – would this be useful in your work on ipfs/in-web-browsers#158 ? Namely, ability to persist metadata about a peer (eg. "its a local go-ipfs node", "its a node participating in cross-origin scharing scheme" etc?

[vasco-santos]

FYI: I have a working draft of this in this commit.

It is keeping the metadata records as a Buffer (the user should encode it according to the preference, ProtoBuf, JSON serialization + convert to Buffer). Moreover, it has the structure and API that I described above. It is swill WIP and I was waiting on some feedback before getting a PR for it public

[jacobheun]

The proposed API looks reasonable to me. I think starting with Buffer only support is fine. We could add support later for type serialization/deserialization if needed, but I think it's reasonable for the application layer to handle the marshaling abstraction.

[vasco-santos]

Closing this as #638 got merged 🎉