Skip to content
This repository has been archived by the owner on Jun 14, 2024. It is now read-only.

feat(66/WAKU2-METADATA): add WakuMetadata Protocol #619

Merged
merged 7 commits into from
Oct 3, 2023
Merged

feat(66/WAKU2-METADATA): add WakuMetadata Protocol #619

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
24a6e28
Create README.md
alrevuelta Sep 25, 2023
97dc818
Update index.md
alrevuelta Sep 25, 2023
3f01712
Update README.md
alrevuelta Sep 26, 2023
d381ee1
Update README.md
alrevuelta Sep 29, 2023
2856707
Update README.md
alrevuelta Oct 2, 2023
7d8cbec
Update README.md
alrevuelta Oct 3, 2023
79831da
Update README.md
alrevuelta Oct 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions content/docs/rfcs/66/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
slug: 66
title: 66/WAKU2-METADATA
name: Waku Metadata Protocol
status: raw
editor: Alvaro Revuelta <alrevuelta@status.im>
contributors:
---

# Metadata Protocol

Waku specifies a req/resp protocol that provides information about the node's medatadata. Such metadata is meant to be used
by the node to decide if a peer is worth connecting or not. The node that makes the request, includes its metadata
so that the receiver is aware of it, without requiring an extra interaction. The parameters are the following:
* `networkId`: Unique identifier of the network that the node is running in.
* `shards`: Shard indexes that the node is subscribed to.


## Protocol id

`/vac/waku/metadata/1.0.0`

## Request

```proto
message WakuMetadataRequest {
jm-clius marked this conversation as resolved.
Show resolved Hide resolved
uint32 network_id = 1;
repeated unt32 shards = 2;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jm-clius I'm not fully aware on the implications of static sharding in here. Any thought on wether it should be considered here? Or these shards should be only for auto sharding?

In other words. If a node is running with static sharding, should the metadata protocol be able to "advertise" those shards?

Another way could be to use string instead of uint32 and just put the whole topic there.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it stands, this would work for static sharding as well (as the cluster ID and shard numbers are encoded exactly the same). The one outstanding thing to decide is between network_id vs. cluster_id as terminology. I think there are OK arguments either side, but we should pick one and stick with it. Perhaps let's just put it up for a vote somewhere?

Interesting idea re string, but I see this exchange as more valuable (at least for now) for the numbered sharding as in the Waku Network. Exchanging the whole topic could as well just reuse the hello messages that already exist in some libp2p implementations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The one outstanding thing to decide is between network_id vs. cluster_id as terminology

sure. Will ask.

Interesting idea re string, but I see this exchange as more valuable (at least for now) for the numbered sharding as in the Waku Network

Not sure i follow. If shard is just an int, how do we know if that refers to a shard or to a static "shard"?

Exchanging the whole topic could as well just reuse the hello messages that already exist in some libp2p implementations.

mm why then using this metadata protocol for the shards? Perhaps it makes more sense to use the hello message and the whole topics?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure i follow. If shard is just an int, how do we know if that refers to a shard or to a static "shard"?

The network ID would be different. Static shards (despite the different name) follows the same pattern - the shard numbers are fully qualified only in conjunction with the network ID. In other words, the static sharded networks are just Waku Networks using their own network IDs (!= 1). In fact, network IDs 0-15 are (potentially) reserved so we have some scope to define our own main/test networks, but any other network ID can be used by applications interested in creating their own Waku Networks.

mm why then using this metadata protocol for the shards?

Well, why use it for network ID if that is also encoded in the topic? My understanding was that the intention is to conveniently exchange the minimum information that is necessary to determine if a connection is interesting or not - that would be network ID in conjunction with shard numbers. Neither of these pieces of metadata is currently enough on its own to make any final conclusions about a connection. The reason not to simply have this be implicit in a list of subscribed topics is to be explicit about our structured approach to network ID and shard numbers in topic construction. Or am I missing something behind the intention of the metadata protocol?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, why use it for network ID if that is also encoded in the topic?

mm because there are some protocols that don't know about "topics". I mean, if i dial with store protocol, how do i know that the store protocol is running in the same network?. afaik right now there is no way?

but yeah, agree with "exchange the minimum information to determine if a connection is interesting or not". But networkid is one level up than shards. Different network id is a deal breaker. Different shard(s), well guess we are not interested, but not as "strong".

Copy link
Contributor

@jm-clius jm-clius Sep 29, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, if i dial with store protocol, how do i know that the store protocol is running in the same network?

True, but again I would need to know what shards within the network the store protocol supports. In all (current) cases, the network ID is just a logical grouping of shards. Since the shards (in fact the pubsub topics constructed from these) define how messages are routed, this is the minimum information I would need to know if a node is interesting for relay, store, filter and lightpush. (A future exception would be that network-ID defines the RLN contract regardless of shard, although this wouldn't affect my decision about which nodes I connect to or not. But RLN contract is the only protocol element that I can think of that belongs to a network ID and not to specific shards within the network.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
repeated unt32 shards = 2;
repeated uint32 shards = 2;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed 2856707

}
```

## Response

```proto
message WakuMetadataResponse {
uint32 network_id = 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would still make this optional in the proto3 definition (both in request and response).
This is not because populating the field is necessarily optional in the protocol, but because proto3 limitations currently imply that there will be no way for us to know if the remote party populated the field or not without the optional keyword. As it stands, if this field is not explicitly populated it will always deserialise as network ID 0, whether that was the intention or not.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be clear already, but just thought of a better way to state the implication. If we have a statement in an RFC such as:

The network_id MUST be populated with the configured network ID...

there is no way for us to check this stipulation without the optional keyword. A bit silly in proto3 IMO.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed 2856707

repeated unt32 shards = 2;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
repeated unt32 shards = 2;
repeated uint32 shards = 2;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed 2856707

}
```
1 change: 1 addition & 0 deletions content/menu/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ bookMenuLevels: 1
- [61/STATUS-Community-History-Archives]({{< relref "/docs/rfcs/61/README.md" >}})
- [63/STATUS-Keycard-Usage]({{< relref "/docs/rfcs/63/README.md" >}})
- [64/WAKU2-NETWORK]({{< relref "/docs/rfcs/64/README.md" >}})
- [66/WAKU2-METADATA]({{< relref "/docs/rfcs/66/README.md" >}})
- Draft
- [1/COSS]({{< relref "/docs/rfcs/1/README.md" >}})
- [3/REMOTE-LOG]({{< relref "/docs/rfcs/3/README.md" >}})
Expand Down