From 395aba4ff9966596eae5c98d78d12f72624ff5b6 Mon Sep 17 00:00:00 2001 From: Thomas Eizinger Date: Sun, 8 Oct 2023 19:41:34 +1100 Subject: [PATCH 01/11] Draft 1 --- ambient-peer/README.md | 66 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 66 insertions(+) create mode 100644 ambient-peer/README.md diff --git a/ambient-peer/README.md b/ambient-peer/README.md new file mode 100644 index 000000000..48cf71b5a --- /dev/null +++ b/ambient-peer/README.md @@ -0,0 +1,66 @@ +# Ambient peer discovery + +| Lifecycle Stage | Maturity | Status | Latest Revision | +|-----------------|---------------|--------|-----------------| +| 1A | Working Draft | Active | r0, 2023-10-08 | + +Authors: [@thomaseizinger] + +Interest Group: + +[@thomaseizinger]: https://github.com/thomaseizinger + +See the [lifecycle document][lifecycle-spec] for context about the maturity level and spec status. + +[lifecycle-spec]: https://github.com/libp2p/specs/blob/master/00-framework-01-spec-lifecycle.md + +## Table of Contents + + + +## Overview + +The ambient peer discovery protocol allows peers to share some of their ambient peers with each other. +Ambient in this case simply means "known peers". +There is no definition of distance like in kademlia and neither do peers have to be connected to consider another peer as ambient. + +## Usecase + +Ambient peer discovery is most useful when a node either starts with or is left with a few or perhaps only a single connection. + +For example, a user may start a libp2p web app and enter another browser's relayed `/webrtc` address. +The connection will succeed but because both nodes are browsers, further discovery of nodes via e.g. kademlia is not possible. +Ambient peer discovery allows the web app to inquire for further nodes from the new connection. + +## Protocol + +1. Node _A_ opens a new stream to node _B_ with the protocol name `/libp2p/ambient-peers`. +2. Node _B_ chooses a subset of at most 5 known peer records received from other peers. + The chosen peer records SHOULD at least have one address that share the same transport technology as the the connection between node _A_ and node _B_. + For example, if node _A_ and node _B_ are connected via WebRTC, node _B_ SHOULD select 5 peer records where each one of them has at least one WebRTC address. +3. Node _B_ writes these peer records onto the stream in their [protobuf encoding](https://github.com/libp2p/specs/blob/master/RFC/0003-routing-records.md#address-record-format), each record being length-prefixed using an unsigned varint and closes the stream after the last one. +4. Node _A_ reads peer records from the stream until EOF or 5 have been received, whichever comes earlier. + +## Security considerations + +Revealing even just some of your peers has serious privacy and security implications for a network. +Care has been taken to mitigate some of these at the design level of the ambient peer discovery protocol. +However, users should still be aware that usage of this protocol does reveal _some_ of your either current or past connections. + + + +### Pre-sample peer records + +Creating new nodes is cheap, meaning we have to assume that repeated requests for e.g. TCP nodes may all come from the same actor trying to map the network. +Implementations therefore MUST pre-sample which peer records they will return for a particular transport protocol and return the same set, regardless of which peer is asking. + +Implementations MAY group transports as follows to further reduce how many peers they reveal: + +1. **Anything on top of TCP:** We support several encryption protocols on top of TCP like noise or TLS. + Some nodes may choose to embed this in their multiaddress using `/tls` or `/noise`. + Nodes MAY consider these to be the equivalent and return a peer record containing a `/tcp/noise` address on a connection that is using `/tcp/tls`. +2. **All versions of QUIC:** QUIC is in itself a versioned protocol and we have for the moment two multiaddress protocols: `/quic` and `/quicv1`. + For the purpose of ambient peer discovery, nodes MAY assume all current and future versions of QUIC are equal. +3. **Anything Web:** If a peer connects over `/webrtc`, `/webrtc-direct`, `/webtransport` or `/ws`, chances are they are a browser node. + As such, nodes MAY assume that any peer record with one of these is useful. +4. **IPv4 & IPv6**: Nodes MAY assume that the requesting peer is capable of dialing either version of IP, regardless of which one was used to make the connection. From 3868a1798b5339c726787b4f005939c595c0d9af Mon Sep 17 00:00:00 2001 From: Thomas Eizinger Date: Mon, 9 Oct 2023 12:55:24 +1100 Subject: [PATCH 02/11] Only return peers we have been connected to in the past --- ambient-peer/README.md | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/ambient-peer/README.md b/ambient-peer/README.md index 48cf71b5a..6bfa20cc4 100644 --- a/ambient-peer/README.md +++ b/ambient-peer/README.md @@ -21,8 +21,7 @@ See the [lifecycle document][lifecycle-spec] for context about the maturity leve ## Overview The ambient peer discovery protocol allows peers to share some of their ambient peers with each other. -Ambient in this case simply means "known peers". -There is no definition of distance like in kademlia and neither do peers have to be connected to consider another peer as ambient. +Ambient in this case means "peers I used to be connected to". ## Usecase @@ -35,26 +34,31 @@ Ambient peer discovery allows the web app to inquire for further nodes from the ## Protocol 1. Node _A_ opens a new stream to node _B_ with the protocol name `/libp2p/ambient-peers`. -2. Node _B_ chooses a subset of at most 5 known peer records received from other peers. - The chosen peer records SHOULD at least have one address that share the same transport technology as the the connection between node _A_ and node _B_. - For example, if node _A_ and node _B_ are connected via WebRTC, node _B_ SHOULD select 5 peer records where each one of them has at least one WebRTC address. -3. Node _B_ writes these peer records onto the stream in their [protobuf encoding](https://github.com/libp2p/specs/blob/master/RFC/0003-routing-records.md#address-record-format), each record being length-prefixed using an unsigned varint and closes the stream after the last one. -4. Node _A_ reads peer records from the stream until EOF or 5 have been received, whichever comes earlier. +1. Node _B_ chooses a subset of at most 5 known peer records received from other peers. + 1. The chosen peer records SHOULD at least have one address that share the same transport technology as the the connection between node _A_ and node _B_. + For example, if node _A_ and node _B_ are connected via WebRTC, node _B_ SHOULD select 5 peer records where each one of them has at least one WebRTC address. + 1. Node _B_ MUST NOT be currently connected to any of these nodes. +1. Node _B_ writes these peer records onto the stream in their [protobuf encoding](https://github.com/libp2p/specs/blob/master/RFC/0003-routing-records.md#address-record-format), each record being length-prefixed using an unsigned varint and closes the stream after the last one. +1. Node _A_ reads peer records from the stream until EOF or 5 have been received, whichever comes earlier. ## Security considerations Revealing even just some of your peers has serious privacy and security implications for a network. -Care has been taken to mitigate some of these at the design level of the ambient peer discovery protocol. -However, users should still be aware that usage of this protocol does reveal _some_ of your either current or past connections. +As a result, this specification forbids nodes to return peer records of nodes they are connected to. -### Pre-sample peer records +## Implementation considerations -Creating new nodes is cheap, meaning we have to assume that repeated requests for e.g. TCP nodes may all come from the same actor trying to map the network. -Implementations therefore MUST pre-sample which peer records they will return for a particular transport protocol and return the same set, regardless of which peer is asking. +### Bound local peer storage -Implementations MAY group transports as follows to further reduce how many peers they reveal: +This protocol requires nodes to store records of peers they used to be connected to. +This is useful independently of this protocol to e.g. reconnect to a peer you've once been connected to. +Implementations should take care that the resulting memory or disk usage is bounded and only store a number of peers appropriate for their deployment target (mobile, server, etc). + +### Group transport technologies + +Implementations MAY group transports as follows: 1. **Anything on top of TCP:** We support several encryption protocols on top of TCP like noise or TLS. Some nodes may choose to embed this in their multiaddress using `/tls` or `/noise`. From 66d877ddda73c52271861a16d6c36c615aacb344 Mon Sep 17 00:00:00 2001 From: Thomas Eizinger Date: Mon, 9 Oct 2023 13:06:48 +1100 Subject: [PATCH 03/11] Add prior art and FAQ --- ambient-peer/README.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/ambient-peer/README.md b/ambient-peer/README.md index 6bfa20cc4..9378074cf 100644 --- a/ambient-peer/README.md +++ b/ambient-peer/README.md @@ -68,3 +68,32 @@ Implementations MAY group transports as follows: 3. **Anything Web:** If a peer connects over `/webrtc`, `/webrtc-direct`, `/webtransport` or `/ws`, chances are they are a browser node. As such, nodes MAY assume that any peer record with one of these is useful. 4. **IPv4 & IPv6**: Nodes MAY assume that the requesting peer is capable of dialing either version of IP, regardless of which one was used to make the connection. + +## Prior art + +Exchanging peers one knows is a common thing in the peer-to-peer space: + +1. [PEX](https://en.wikipedia.org/wiki/Peer_exchange) augments the BitTorrent protocol. +2. Bitcoin nodes can send [`addr`](https://en.bitcoin.it/wiki/Protocol_documentation#addr) messages to exchange peers with one another. +3. WAKU has an [ambient-peer discovery](https://github.com/vacp2p/rfc/blob/master/content/docs/rfcs/34/README.md) protocol built on top of libp2p. + +There have been several discussions in the libp2p space about adding such a protocol: + +- https://github.com/libp2p/specs/issues/222 +- https://github.com/libp2p/notes/issues/3 +- https://github.com/libp2p/notes/issues/7 + +## FAQ + +### Why not use the rendezvous? + +The rendezvous protocol could be repurposed as a kind of peer exchange protocol. +We would have to agree on an identifier that all peers use to register themselves within a certain topic. +Other peers can then go and query a node for all peers registered under this topic. + +We consider this impractical for the given problem because: + +- It requires three parties to support the protocol instead of just two and thus would take a lot longer to be rolled out. +- It creates a lot of traffic. + Nodes have to actively register themselves without knowing whether their peer record will ever be distributed / requested. + To be effective, every node would have to register themselves with every other node. From 42dd0cd9df4f3c66a5fea4ee9f923290441f5392 Mon Sep 17 00:00:00 2001 From: Thomas Eizinger Date: Wed, 11 Oct 2023 09:41:56 +1100 Subject: [PATCH 04/11] Next revision --- ambient-peer/README.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/ambient-peer/README.md b/ambient-peer/README.md index 9378074cf..8b19b1967 100644 --- a/ambient-peer/README.md +++ b/ambient-peer/README.md @@ -37,14 +37,15 @@ Ambient peer discovery allows the web app to inquire for further nodes from the 1. Node _B_ chooses a subset of at most 5 known peer records received from other peers. 1. The chosen peer records SHOULD at least have one address that share the same transport technology as the the connection between node _A_ and node _B_. For example, if node _A_ and node _B_ are connected via WebRTC, node _B_ SHOULD select 5 peer records where each one of them has at least one WebRTC address. - 1. Node _B_ MUST NOT be currently connected to any of these nodes. + 1. Node _B_ SHOULD NOT be currently connected to any of these nodes. 1. Node _B_ writes these peer records onto the stream in their [protobuf encoding](https://github.com/libp2p/specs/blob/master/RFC/0003-routing-records.md#address-record-format), each record being length-prefixed using an unsigned varint and closes the stream after the last one. 1. Node _A_ reads peer records from the stream until EOF or 5 have been received, whichever comes earlier. ## Security considerations Revealing even just some of your peers has serious privacy and security implications for a network. -As a result, this specification forbids nodes to return peer records of nodes they are connected to. +By default, implementations MUST NOT share records of peers they are currently connected to. +Implementations MAY add a configuration flag that allows users to override this. @@ -69,6 +70,12 @@ Implementations MAY group transports as follows: As such, nodes MAY assume that any peer record with one of these is useful. 4. **IPv4 & IPv6**: Nodes MAY assume that the requesting peer is capable of dialing either version of IP, regardless of which one was used to make the connection. +### Separating networks + +Libp2p is used across a range of networks and many of them may not actually have a useful overlap in compatible protocols. +To avoid sharing addresses of peers that don't support useful protocols, implementations SHOULD allow configuration of the protocol identifier. +For example, instead of `/libp2p/ambient-peers` a node may use `/my-cool-p2p-network/ambient-peers`. + ## Prior art Exchanging peers one knows is a common thing in the peer-to-peer space: From f6bb9ab854838f992cfb2ed1ec18622132f876a8 Mon Sep 17 00:00:00 2001 From: Thomas Eizinger Date: Wed, 11 Oct 2023 09:43:48 +1100 Subject: [PATCH 05/11] Add recommendation about retaining suffix --- ambient-peer/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/ambient-peer/README.md b/ambient-peer/README.md index 8b19b1967..8e83aeb5b 100644 --- a/ambient-peer/README.md +++ b/ambient-peer/README.md @@ -75,6 +75,7 @@ Implementations MAY group transports as follows: Libp2p is used across a range of networks and many of them may not actually have a useful overlap in compatible protocols. To avoid sharing addresses of peers that don't support useful protocols, implementations SHOULD allow configuration of the protocol identifier. For example, instead of `/libp2p/ambient-peers` a node may use `/my-cool-p2p-network/ambient-peers`. +It is RECOMMENDED that implementations retain the `/ambient-peers` suffix to communicate the semantics of this protocol. ## Prior art From 4d350369e1670d0dd8d5c8c6b32fc480670c3134 Mon Sep 17 00:00:00 2001 From: Thomas Eizinger Date: Wed, 11 Oct 2023 09:48:32 +1100 Subject: [PATCH 06/11] Add comment about interest group --- ambient-peer/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ambient-peer/README.md b/ambient-peer/README.md index 8e83aeb5b..865905d1c 100644 --- a/ambient-peer/README.md +++ b/ambient-peer/README.md @@ -6,7 +6,7 @@ Authors: [@thomaseizinger] -Interest Group: +Interest Group: [@thomaseizinger]: https://github.com/thomaseizinger From 3806fcfd2a0ab70dfc3033326ddbda7be57eb684 Mon Sep 17 00:00:00 2001 From: Thomas Eizinger Date: Mon, 16 Oct 2023 15:22:01 +1100 Subject: [PATCH 07/11] Remove line from overview --- ambient-peer/README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/ambient-peer/README.md b/ambient-peer/README.md index 865905d1c..7aace1745 100644 --- a/ambient-peer/README.md +++ b/ambient-peer/README.md @@ -21,7 +21,6 @@ See the [lifecycle document][lifecycle-spec] for context about the maturity leve ## Overview The ambient peer discovery protocol allows peers to share some of their ambient peers with each other. -Ambient in this case means "peers I used to be connected to". ## Usecase From f4e2573f56dbb7593be3b20a6b09e9d229f3c612 Mon Sep 17 00:00:00 2001 From: Thomas Eizinger Date: Mon, 16 Oct 2023 15:26:58 +1100 Subject: [PATCH 08/11] Add note about resource footprint --- ambient-peer/README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/ambient-peer/README.md b/ambient-peer/README.md index 7aace1745..11cf56dca 100644 --- a/ambient-peer/README.md +++ b/ambient-peer/README.md @@ -30,6 +30,9 @@ For example, a user may start a libp2p web app and enter another browser's relay The connection will succeed but because both nodes are browsers, further discovery of nodes via e.g. kademlia is not possible. Ambient peer discovery allows the web app to inquire for further nodes from the new connection. +The protocol is designed to compliment other discovery mechanism like kademlia. +It features a very small resource footprint and can thus also be used by lite-clients within browser or mobile environments. + ## Protocol 1. Node _A_ opens a new stream to node _B_ with the protocol name `/libp2p/ambient-peers`. From 20f84791eee2437628c0b227cdc3110d9ce01a7e Mon Sep 17 00:00:00 2001 From: Thomas Eizinger Date: Mon, 16 Oct 2023 15:30:11 +1100 Subject: [PATCH 09/11] Minor wording change --- ambient-peer/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ambient-peer/README.md b/ambient-peer/README.md index 11cf56dca..cf5ff4d12 100644 --- a/ambient-peer/README.md +++ b/ambient-peer/README.md @@ -67,7 +67,7 @@ Implementations MAY group transports as follows: Some nodes may choose to embed this in their multiaddress using `/tls` or `/noise`. Nodes MAY consider these to be the equivalent and return a peer record containing a `/tcp/noise` address on a connection that is using `/tcp/tls`. 2. **All versions of QUIC:** QUIC is in itself a versioned protocol and we have for the moment two multiaddress protocols: `/quic` and `/quicv1`. - For the purpose of ambient peer discovery, nodes MAY assume all current and future versions of QUIC are equal. + For the purpose of ambient peer discovery, nodes MAY assume all current and future versions of QUIC are supported by the remote node. 3. **Anything Web:** If a peer connects over `/webrtc`, `/webrtc-direct`, `/webtransport` or `/ws`, chances are they are a browser node. As such, nodes MAY assume that any peer record with one of these is useful. 4. **IPv4 & IPv6**: Nodes MAY assume that the requesting peer is capable of dialing either version of IP, regardless of which one was used to make the connection. From cba9ee915074d1c9f42a808c23a39573f032fee4 Mon Sep 17 00:00:00 2001 From: Thomas Eizinger Date: Mon, 16 Oct 2023 15:30:38 +1100 Subject: [PATCH 10/11] Update date --- ambient-peer/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ambient-peer/README.md b/ambient-peer/README.md index cf5ff4d12..e91d00f33 100644 --- a/ambient-peer/README.md +++ b/ambient-peer/README.md @@ -2,7 +2,7 @@ | Lifecycle Stage | Maturity | Status | Latest Revision | |-----------------|---------------|--------|-----------------| -| 1A | Working Draft | Active | r0, 2023-10-08 | +| 1A | Working Draft | Active | r0, 2023-10-16 | Authors: [@thomaseizinger] From 55bbfdda444e35b3ff89e0090a42a4f6d23d2826 Mon Sep 17 00:00:00 2001 From: Thomas Eizinger Date: Thu, 19 Oct 2023 14:42:32 +1100 Subject: [PATCH 11/11] Update ambient-peer/README.md Co-authored-by: Russell Dempsey <1173416+SgtPooki@users.noreply.github.com> --- ambient-peer/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ambient-peer/README.md b/ambient-peer/README.md index e91d00f33..3468534b4 100644 --- a/ambient-peer/README.md +++ b/ambient-peer/README.md @@ -66,7 +66,7 @@ Implementations MAY group transports as follows: 1. **Anything on top of TCP:** We support several encryption protocols on top of TCP like noise or TLS. Some nodes may choose to embed this in their multiaddress using `/tls` or `/noise`. Nodes MAY consider these to be the equivalent and return a peer record containing a `/tcp/noise` address on a connection that is using `/tcp/tls`. -2. **All versions of QUIC:** QUIC is in itself a versioned protocol and we have for the moment two multiaddress protocols: `/quic` and `/quicv1`. +2. **All versions of QUIC:** QUIC is in itself a versioned protocol and we have for the moment two multiaddress protocols: `/quic` and `/quic-v1`. For the purpose of ambient peer discovery, nodes MAY assume all current and future versions of QUIC are supported by the remote node. 3. **Anything Web:** If a peer connects over `/webrtc`, `/webrtc-direct`, `/webtransport` or `/ws`, chances are they are a browser node. As such, nodes MAY assume that any peer record with one of these is useful.