Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

p2p-circuit v2 spec #325

Merged
merged 46 commits into from
Jun 29, 2021
Merged
Show file tree
Hide file tree
Changes from 43 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
119e237
move README to circuit-v1.md
vyzo May 5, 2021
22e2721
circuit v2 spec preliminaries
vyzo May 5, 2021
91f2ab6
copy edit
vyzo May 5, 2021
6b791ab
copy edit
vyzo May 5, 2021
e36fe48
copy edit
vyzo May 5, 2021
f8ca5d2
copy edit
vyzo May 5, 2021
16027bc
copy edit
vyzo May 5, 2021
bbb2337
copy edit
vyzo May 5, 2021
bfa6918
copy edit
vyzo May 5, 2021
3c0e757
flesh out spec
vyzo May 10, 2021
5efd8e8
relay/circuit-v2: Replace excalidraw with plantuml
mxinden May 10, 2021
e2cbf5f
copy edit
vyzo May 11, 2021
e242414
copy edit
vyzo May 11, 2021
186bf1c
copy edit
vyzo May 11, 2021
803682e
copy edit
vyzo May 11, 2021
8bea5d0
copy edit
vyzo May 11, 2021
cf3e6d2
Merge pull request #326 from mxinden/rfc/circuit-v2
vyzo May 11, 2021
5af6126
copy edit
vyzo May 11, 2021
01ea1ea
copy edit
vyzo May 11, 2021
af6819b
copy edit
vyzo May 18, 2021
f9dda94
copy edit
vyzo May 18, 2021
53e4fdd
copy edit
vyzo May 18, 2021
d5bc92e
copy edit
vyzo May 18, 2021
4538e2a
copy edit
vyzo May 18, 2021
b47a1db
copy edit
vyzo May 18, 2021
50bbead
copy edit
vyzo May 18, 2021
d74f42c
copy edit
vyzo May 18, 2021
d47247b
copy edit
vyzo May 18, 2021
8591ede
copy edit
vyzo May 18, 2021
90deee6
copy edit
vyzo May 18, 2021
2b7c32d
copy edit
vyzo May 18, 2021
15788ce
copy edit
vyzo May 18, 2021
ad64f2f
copy edit
vyzo May 18, 2021
0bebcc1
copy edit
vyzo May 18, 2021
fb477c7
copy edit
vyzo May 18, 2021
9cb054f
copy edit
vyzo May 18, 2021
f5c1d62
copy edit
vyzo May 18, 2021
962a80d
copy edit
vyzo May 18, 2021
b1ac56d
copy edit
vyzo May 18, 2021
d00f2c7
copy edit
vyzo May 18, 2021
1b63664
copy edit
vyzo May 18, 2021
4d52042
copy edit
vyzo May 18, 2021
62c641b
add wording about relay over relay
vyzo Jun 2, 2021
e943af6
copy edit
vyzo Jun 8, 2021
c3b40e4
editorial changes
vyzo Jun 8, 2021
c541d2c
add multicoded and envelope domain for vouchers
vyzo Jun 8, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
297 changes: 5 additions & 292 deletions relay/README.md
Original file line number Diff line number Diff line change
@@ -1,294 +1,7 @@
# Circuit Relay v0.1.0
# p2p-circuit relay

> Circuit Switching for libp2p, also known as TURN or Relay in Networking literature.
Circuit Switching for libp2p, also known as TURN or Relay in Networking literature.

| Lifecycle Stage | Maturity | Status | Latest Revision |
|-----------------|----------------|--------|-----------------|
| 3A | Recommendation | Active | r1, 2018-06-03 |

Authors: [@daviddias]

Interest Group: [@lgierth], [@hsanjuan], [@jamesray1], [@vyzo], [@yusefnapora]

[@daviddias]: https://github.com/daviddias
[@lgierth]: https://github.com/lgierth
[@hsanjuan]: https://github.com/hsanjuan
[@jamesray1]: https://github.com/jamesray1
[@vyzo]: https://github.com/vyzo
[@yusefnapora]: https://github.com/yusefnapora

See the [lifecycle document][lifecycle-spec] for context about maturity level
and spec status.

[lifecycle-spec]: https://github.com/libp2p/specs/blob/master/00-framework-01-spec-lifecycle.md

## Implementations

- [js-libp2p-circuit](https://github.com/libp2p/js-libp2p-circuit)
- [go-libp2p-circuit](https://github.com/libp2p/go-libp2p-circuit)
- [rust-libp2p-relay](https://github.com/libp2p/rust-libp2p/tree/master/protocols/relay)

## Table of Contents

- [Overview](#overview)
- [Dramatization](#dramatization)
- [Addressing](#addressing)
- [Wire protocol](#wire-protocol)
- [Interfaces](#interfaces)
- [Implementation Details](#implementation-details)
- [Removing existing relay protocol](#removing-existing-relay-protocol)

## Overview

The circuit relay is a means to establish connectivity between libp2p nodes (e.g. IPFS nodes) that wouldn't otherwise be able to establish a direct connection to each other.

Relay is needed in situations where nodes are behind NAT, reverse proxies, firewalls and/or simply don't support the same transports (e.g. go-ipfs vs. browser-ipfs). Even though libp2p has modules for NAT traversal ([go-libp2p-nat](https://github.com/libp2p/go-libp2p-nat)), piercing through NATs isn't always an option. The circuit relay protocol exists to overcome those scenarios.

Unlike a transparent **tunnel**, where a libp2p peer would just proxy a communication stream to a destination (the destination being unaware of the original source), a circuit relay makes the destination aware of the original source and the circuit followed to establish communication between the two. This provides the destination side with full knowledge of the circuit which, if needed, could be rebuilt in the opposite direction.

Apart from that, this relayed connection behaves just like a regular connection would, but over an existing swarm stream with another peer (instead of e.g. TCP). A node asks a relay node to connect to another node on its behalf. The relay node short-circuits streams between the two nodes, enabling them to reach each other.

Relayed connections are end-to-end encrypted just like regular connections.

The circuit relay is both a tunneled transport and a mounted swarm protocol. The transport is the means of ***establishing*** and ***accepting*** connections, and the swarm protocol is the means to ***relaying*** connections.

```
+-----+ /ip4/.../tcp/.../ws/p2p/QmRelay +-------+ /ip4/.../tcp/.../p2p/QmTwo +-----+
|QmOne| <------------------------------------>|QmRelay|<----------------------------------->|QmTwo|
+-----+ (/libp2p/relay/circuit multistream) +-------+ (/libp2p/relay/circuit multistream) +-----+
^ +-----+ ^
| /p2p-circuit/QmTwo | | |
+-----------------------------------------+ +-----------------------------------------+
```

**Notes for the reader:**

- In this document, we use `/p2p/Qm...` multiaddrs. Libp2p previously used `/ipfs/Qm...` for multiaddrs you'll likely see uses of this notation in the wild. `/ipfs` and `/p2p` multiaddrs are equivalent but `/ipfs` is deprecated `/p2p` should be preferred.
- You may also see `/ipfs/Qm...` used for _file paths_ in IPFS. These are _not_ multiaddrs and this confusion is one of the many motivations for switching to `/p2p/Qm...` multiaddrs.

## Dramatization

Cast:
- QmOne, the dialing node (browser).
- QmTwo, the listening node (go-ipfs).
- QmRelay, a node which speaks the circuit relay protocol (go-ipfs or js-ipfs).

Scene 1:
- QmOne wants to connect to QmTwo, and through peer routing has acquired a set of addresses of QmTwo.
- QmTwo doesn't support any of the transports used by QmOne.
- Awkward silence.

Scene 2:
- All three nodes have learned to speak the `/ipfs/relay/circuit` protocol.
- QmRelay is configured to allow relaying connections between other nodes.
- QmOne is configured to use QmRelay for relaying.
- QmOne automatically added `/p2p-circuit/p2p/QmTwo` to its set of QmTwo addresses.
- QmOne tries to connect via relaying, because it shares this transport with QmTwo.
- A lively and prolonged dialogue ensues.

## Addressing

`/p2p-circuit` multiaddrs don't carry any meaning of their own. They need to encapsulate a `/p2p` address, or be encapsulated in a `/p2p` address, or both.

As with all other multiaddrs, encapsulation of different protocols determines which metaphorical tubes to connect to each other.

A `/p2p-circuit` circuit address, is formated following:

`[<relay peer multiaddr>]/p2p-circuit/<destination peer multiaddr>`

Examples:

- `/p2p-circuit/p2p/QmVT6GYwjeeAF5TR485Yc58S3xRF5EFsZ5YAF4VcP3URHt` - Arbitrary relay node
- `/ip4/127.0.0.1/tcp/5002/p2p/QmdPU7PfRyKehdrP5A3WqmjyD6bhVpU1mLGKppa2FjGDjZ/p2p-circuit/p2p/QmVT6GYwjeeAF5TR485Yc58S3xRF5EFsZ5YAF4VcP3URHt` - Specific relay node

This opens the room for multiple hop relay, where the second relay is encapsulated in the first relay multiaddr, such as:

`<1st relay>/p2p-circuit/<2nd relay>/p2p-circuit/<dst multiaddr>`

A few examples:

Using any relay available:

- `/p2p-circuit/p2p/QmTwo`
- Dial QmTwo, through any available relay node (or find one node that can relay).
- The relay node will use peer routing to find an address for QmTwo if it doesn't have a direct connection.
- `/p2p-circuit/ip4/../tcp/../p2p/QmTwo`
- Dial QmTwo, through any available relay node, but force the relay node to use the encapsulated `/ip4` multiaddr for connecting to QmTwo.

Specify a relay:

- `/p2p/QmRelay/p2p-circuit/p2p/QmTwo`
- Dial QmTwo, through QmRelay.
- Use peer routing to find an address for QmRelay.
- The relay node will also use peer routing, to find an address for QmTwo.
- `/ip4/../tcp/../p2p/QmRelay/p2p-circuit/p2p/QmTwo`
- Dial QmTwo, through QmRelay.
- Includes info for connecting to QmRelay.
- The relay node will use peer routing to find an address for QmTwo.

Double relay:

- `/p2p-circuit/p2p/QmTwo/p2p-circuit/p2p/QmThree`
- Dial QmThree, through a relayed connection to QmTwo.
- The relay nodes will use peer routing to find an address for QmTwo and QmThree.
- We'll **not support nested relayed connections for now**, see [Future Work](#future-work) section.

## Wire format

We start the description of the Wire format by illustrating a possible flow scenario and then describing them in detail by phases.

### Relay Message

Every message in the relay protocol uses the following protobuf:

```
message CircuitRelay {

enum Status {
SUCCESS = 100;
HOP_SRC_ADDR_TOO_LONG = 220;
HOP_DST_ADDR_TOO_LONG = 221;
HOP_SRC_MULTIADDR_INVALID = 250;
HOP_DST_MULTIADDR_INVALID = 251;
HOP_NO_CONN_TO_DST = 260;
HOP_CANT_DIAL_DST = 261;
HOP_CANT_OPEN_DST_STREAM = 262;
HOP_CANT_SPEAK_RELAY = 270;
HOP_CANT_RELAY_TO_SELF = 280;
HOP_BACKOFF = 290;
STOP_SRC_ADDR_TOO_LONG = 320;
STOP_DST_ADDR_TOO_LONG = 321;
STOP_SRC_MULTIADDR_INVALID = 350;
STOP_DST_MULTIADDR_INVALID = 351;
STOP_RELAY_REFUSED = 390;
MALFORMED_MESSAGE = 400;
}

enum Type { // RPC identifier, either HOP, STOP or STATUS
HOP = 1;
STOP = 2;
STATUS = 3;
CAN_HOP = 4; // is peer a relay?
}

message Peer {
required bytes id = 1; // peer id
repeated bytes addrs = 2; // peer's known addresses
}

optional Type type = 1; // Type of the message

optional Peer srcPeer = 2; // srcPeer and dstPeer are used when Type is HOP or STOP
optional Peer dstPeer = 3;

optional Status code = 4; // Status code, used when Type is STATUS
}
```

### High level overview of establishing a relayed connection

**Setup:**
- Peers involved, A, B, R
- A wants to connect to B, but needs to relay through R

**Assumptions:**
- A has connection to R, R has connection to B

**Events:**
- phase I: Open a request for a relayed stream (A to R).
- A dials a new stream `sAR` to R using protocol `/libp2p/circuit/relay/0.1.0`.
- A sends a CircuitRelay message with `{ type: 'HOP', srcPeer: '/p2p/QmA', dstPeer: '/p2p/QmB' }` to R through `sAR`.
- R receives stream `sAR` and reads the message from it.
- phase II: Open a stream to be relayed (R to B).
- R opens a new stream `sRB` to B using protocol `/libp2p/circuit/relay/0.1.0`.
- R sends a CircuitRelay message with `{ type: 'STOP', srcPeer: '/p2p/QmA', dstPeer: '/p2p/QmB' }` on `sRB`.
- R sends a CircuitRelay message with `{ type: 'STATUS', code: 'SUCCESS' }` on `sAR`.
- phase III: Streams are piped together, establishing a circuit
- B receives stream `sRB` and reads the message from it
- B sends a CircuitRelay message with `{ type: 'STATUS', code: 'SUCCESS' }` on `sRB`.
- B passes stream to `NewConnHandler` to be handled like any other new incoming connection.

### Under the microscope

- We've defined a max length for the multiaddrs of arbitrarily 1024 bytes
- Multiaddrs are transfered on its binary packed format
- Peer Ids are transfered on its non base encoded format (aka byte array containing the multihash of the Public Key).


### Status codes table

This is a table of status codes and sample messages that may occur during a relay setup. Codes in the 200 range are returned by the relay node. Codes in the 300 range are returned by the destination node.


| Code | Message | Meaning |
| ----- |:--------------------------------------------------|:----------:|
| 100 | "success" | Relay was setup correctly |
| 220 | "src address too long" | |
| 221 | "dst address too long" | |
| 250 | "failed to parse src addr: no such protocol ipfs" | The `<src>` multiaddr in the header was invalid |
| 251 | "failed to parse dst addr: no such protocol ipfs" | The `<dst>` multiaddr in the header was invalid |
| 260 | "passive relay has no connection to dst" | |
| 261 | "active relay couldn't dial to dst: conn refused" | relay could not form new connection to target peer |
| 262 | "couldn't' dial to dst" | relay has conn to dst, but failed to open a stream |
| 270 | "dst does not support relay" | |
| 280 | "can't relay to itself" | The relay got its own address as destination |
| 290 | "temporary backoff" | The relay wants us to backoff and try again later |
| 320 | "src address too long" | |
| 321 | "dst address too long" | |
| 350 | "failed to parse src addr" | src multiaddr in the header was invalid |
| 351 | "failed to parse dst addr" | dst multiaddr in the header was invalid |
| 390 | "connection refused by stop endpoint" | The stop endpoint couldn't accept the connection |
| 400 | "malformed message" | A malformed or too long message was received |

## Implementation details

### Interfaces

> These are go-ipfs specific

As explained above, the relay is both a transport (`tpt.Transport`) and a mounted stream protocol (`p2pnet.StreamHandler`). In addition it provides a means of specifying relay nodes to listen/dial through.

Note: the usage of p2pnet.StreamHandler is a little bit off, but it gets the point across.

```go
import (
tpt "github.com/libp2p/go-libp2p-transport"
p2phost "github.com/libp2p/go-libp2p-host"
p2pnet "github.com/libp2p/go-libp2p-net"
p2proto "github.com/libp2p/go-libp2p-protocol"
)

const ID p2proto.ID = "/libp2p/circuit/relay/0.1.0"

type CircuitRelay interface {
tpt.Transport
p2pnet.StreamHandler

EnableRelaying(enabled bool)
}

fund NewCircuitRelay(h p2phost.Host)
```

### Removing existing relay protocol in Go

Note that there is an existing swarm protocol colloqiually called relay. It lives in the go-libp2p package and is named `/ipfs/relay/line/0.1.0`.

- Introduced in ipfs/go-ipfs#478 (28-Dec-2014).
- No changes except for ipfs/go-ipfs@de50b2156299829c000b8d2df493b4c46e3f24e9.
- Changed to use multistream muxer.
- Shortcomings
- No end-to-end encryption.
- No rate limiting (DoS by resource exhaustion).
- Doesn't verify src id in ReadHeader(), easy to fix.
- Capable of *accepting* connections, and *relaying* connections.
- Not capable of *connecting* via relaying.

Since the existing protocol is incomplete, insecure, and certainly not used, we can safely remove it.

## Future work

We have considered more features but won't be adding them on the first iteration of Circuit Relay, the features are:

- Multihop relay - With this specification, we are only enabling single hop relays to exist. Multihop relay will come at a later stage as Packet Switching.
- Relay discovery mechanism - At the moment we're not including a mechanism for discovering relay nodes. For the time being, they should be configured statically.
Specifications:
- [p2p-circuit v1](circuit-v1.md)
- [p2p-cricuit v2](circuit-v2.md)
Loading