Distributed hash tables

[max-mapper]

Related to the file sharing requirement is the need to do decentralized peer discovery using e.g. a distributed hash table. The existing file sharing requirements focus on throughput and backpressure, but the other ingredient missing for p2p file sharing is a low latency, highly parallel (low overhead) socket.

Motivation

Making a WebRTC connection now is very slow compared to say UDP hole punching w/ BitTorrent, with a ~5RTT handshake and ICE/STUN negotiation. However, ianswett@google.com was apparently working on a version of DataChannels powered by QUIC protocol which boasts 0RTT handshakes in the best case. https://www.youtube.com/watch?v=mIvyOFu1c1Q

In order to implement a fast DHT lookup for example you need low latency, low overhead connections, so you can do wide fanout style network searches with high concurrency. BitTorrent for example contacts hundreds of peers in Kademlia in the first few hundred milliseconds of opening a torrent.

I don't imagine we will ever get raw UDP or TCP sockets added to JS in the future because of the lack of encryption, but QUIC has a great story for stateless mode performance as well as built in TLS-like encryption. It would allow the number of concurrent WebSocket and WebRTC connections on a page to go up from the current limits (~10 data channels per page or so?) to many more (depending on how it's implemented, but in general the amount of state per connection should be much lower).

Ian Swett has stated that google wants to do more standards work with it, but in more recent posts like this https://cloudplatform.googleblog.com/2018/06/Introducing-QUIC-support-for-HTTPS-load-balancing.html they no longer mention Data Channels, and I can't really find much info about if that's still a planned feature.

[lgrahl]

Connection Setup Time

IMO, reducing the amount of RTTs for the connection setup is an optimisation. But I wouldn't mind explicitly adding it as a requirement to the IoT use cases or adding a new DHT use case section:

Nxx The amount of RTTs required to set up a peer-to-peer connection should be minimised.

I guess this is what was meant by low latency?

Parallelise

Agree. This could base on N02 (to reduce the required amount of sockets and thus resources). The following requirement could be added along with a new DHT use case:

Nxx The user agent must be able to handle many peer-to-peer connections at the same time efficiently.

While many is of course up for discussion.

Low Overhead

How would you phrase that into a requirement?

[lgrahl]

@feross raised another requirement for DHTs. To quote:

• DHTs are decentralized distributed systems that provide a lookup
  service similar to a hash table. DHTs are useful for finding peers without
  a central signaling infrastructure, and they're used in almost every
  widely-deployed peer-to-peer system, including BitTorrent, Bitcoin, IPFS,
  Dat, etc.
• With the current WebRTC connection model, DHTs are nearly impossible to
  build. We need the ability to store the "contact information" for a peer,
  close the connection to that peer, and then re-connect to that peer at some
  point in the future (if they're still online). With TCP/UDP, this is quite
  easy to accomplish. Simply store the ip:port (12.34.56.78:9000) and try to
  connect. If the peer is still online, it will just work.
• Peers need the ability to "listen" for incoming connections. Peers also
  need the ability to publish a "reusable SDP" that multiple peers can use to
  connect to listening peers (currently an SDP is usable only once)

Let's phrase these into requirements.

Reusable Connections

Nxx The application must be able to move the peer-to-peer context into persistent storage and reuse it at a later stage without having to do the signalling process again.

If that wasn't controversial enough, let's go to the next requirement.

Incoming Connections

Nxx The user agent must signal incoming connections to the application which must be able to accept them as a new peer-to-peer connection.

This raises a couple of questions:

1. How is a new incoming connection being detected? A different username in a STUN binding request? An incoming DTLS connection?
2. How to verify the fingerprint of the remote peer's certificate?

[the8472]

However, ianswett@google.com was apparently working on a version of DataChannels powered by QUIC protocol which boasts 0RTT handshakes in the best case.

Isn't that conditional on a session resumption token being available? That is not applicable for DHTs because most DHT RPCs are single-shot, i.e. during lookups you iterate down a path through nodes that you have never seen before and are not likely to visit again anytime soon. Only the starting set from your local routing table and the final set (the target nodes) may be contacted again.

Since an iterative lookup forms a dependency chain any additional RTTs multiply by the number of hops you have to visit during a lookup. For a million-node DHT this can be around ~20 hops with geographically diverse distributions. Locality-aware optimizations are possible but complex to implement. Racing parallel lookups for speed would either compound with or force you to give up multi-path verification as necessary for S/Kademlia.

In other words, many DHT routing algorithms are designed for 1RTT exchanges without setup or teardown as possible with UDP sockets.

Some level of secrecy can be achieved by using tying node IDs to their public key, but for 1RTT exchanges you're limited to non-PFS for the query part of your exchange.

[max-mapper]

Yes sorry 0RTT is not really relevant here, but based on a talk Ian Swett gave about the project a few years ago now the latency for 1RTT with QUIC is also supposed to be greatly reduced, which when you are doing many hops could have a big impact on overall query time (because as stated above WebRTC is currently something like 5RTT per connection)

[the8472]

My understanding is that there are no deployed webrtc based DHT implementation. So the improvement from 5RTT to 1RTT initialization is entirely hypothetical. Compared to actually deployed UDP-based implementations it's still 2 times worse. For interactive uses that would also mean the latency floor is twice as large as it is in native applications.

[max-mapper]

We'll never see one deployed if we don't get a lower latency WebRTC transport :)

[aboba]

@maxogden There is now a proposal in WICG for client/server QUIC: https://discourse.wicg.io/t/webtransport-proposal/3508
The explainer is here: https://github.com/pthatcherg/web-transport/blob/master/explainer.md
The API is here: https://pthatcherg.github.io/web-transport/

Note that this proposal is for a client/server API, so it doesn't relate to P2P WebRTC use cases (that is covered by WebRTC-QUIC: https://w3c.github.io/webrtc-quic/

Do any of these proposals appear interesting for the scenarios described in this issue?

[lgrahl]

Sorry for jumping in, but from my perspective the use case has much more fundamental issues as described above. We should not mix this with a discussion involving a different transport protocol. And the web transport API is not peer-to-peer.

[aboba]

@lgrahl Agreed. Is there someone willing to present the issue (or a PR) at the June Virtual interim?

[lgrahl]

@aboba Can you give us an estimate of available time to present this use case and the requirements? @feross agreed to present the DHT use case.

[aboba]

I would assume roughly 10 minutes for the presentation and discussion. The slides are available here:
https://docs.google.com/presentation/d/1UrXARLbAfwmfK686rX_9W_-FGIc62ZDwVd079wIoX7I/edit#slide=id.g30f5366cbe_0_1

[jhiesey]

Some use case proposals here: libp2p/js-libp2p-webrtc-star#177 (comment)

[the8472]

Note that the lack of STUN servers and high rate of connection setup/teardowns for iterative lookups in a distributed environment means that the browser would have to work harder to make the listening sockets reachable through NATs and firewalls, e.g. by negotiating via UPnP IGD/PMP/PCP with the gateway devices and allowing clients to coordinate hole punching (simultaneous open) through other nodes in the network.

[juberti]

Webtransport does have a p2p mode IIUC, I think that will help a fair amount with setup delays, and I think it would also address the reusable SDP question (although may need some ICE work too to better support 'forking')

STUN servers should still be usable and able to achieve direct connections the vast majority of the time, as in current cases/

[the8472]

STUN servers are still not exactly distributed and AIUI peers would have to agree on using a specific set of servers, no? It doesn't seem adequate for many p2p protocols anyway where you really want your ports opened to unsolicited incoming connection.

[juberti]

STUN servers are pretty ubiquitous and peers don't need to agree on which ones.

[lgrahl]

The WebTransport spec currently does not define a P2P mode. For WebRTC, @pthatcherg did propose APIs on last TPAC (which I couldn't attend) how to make it work.

[juberti]

right, it seems like that hasn't made it into the WT spec yet. Generally, I think it's simply replacing host:port with ICETransport/certificate fingerprint, and the rest is straightforward.

[aboba]

@juberti @lgrahl @pthatcherg
The RTCQuicTransport object supports P2P operation and is defined as inheriting from QuicTransportBase, which is defined in WebTransport. We haven't reviewed it since moving WebTransport over to WHATWG Streams, but I believe the intent is still for the RTCQuicTransport object to address the P2P needs.

If there is a deficiency in the P2P spec, can you file Issues?

[pthatcherg]

The scope of WebTransport is currently the general framework plus specific client-server implementations. RTCQuicTransport uses the general framework of WebTransport for p2p QUIC. However, almost all of the focus of those involved with RTCQuicTransport has lately been on client-server, so RTCQuicTransport has fallen behind a bit and could at some point use some updates. What I presented at TPAC was how we could get ICE to support DHTs (mainly with ICE forking), but that is only the first layer in solving the whole problem (an IceTransport by itself is not very useful). The second layer is putting a data channel on top of that. The choices are: a. PeerConnection, b. ORTC-style SctpTransport/DtlsTransport and c. RTCQuicTransport.

…

On Wed, Oct 16, 2019 at 7:58 PM Bernard Aboba ***@***.***> wrote: @juberti <https://github.com/juberti> @lgrahl <https://github.com/lgrahl> @pthatcherg <https://github.com/pthatcherg> The RTCQuicTransport object <https://w3c.github.io/webrtc-quic/#rtcquic-transport*> supports P2P operation and is defined as inheriting from QuicTransportBase <https://wicg.github.io/web-transport/#quic-transport-base>, which is defined in WebTransport. We haven't reviewed it since moving WebTransport over to WHATWG Streams, but I believe the intent is still for the RTCQuicTransport and RTCQuicStream objects to address the P2P needs. If there is a deficiency in the P2P spec, can you file Issues? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#15?email_source=notifications&email_token=AB3J76XXVY3UG3D267YGD2LQO7IGFA5CNFSM4FZPTKZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBOSNXA#issuecomment-542975708>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB3J76SYX37P46ZZUGPDGQTQO7IGFANCNFSM4FZPTKZA> .

[juberti]

c) seems like the most straightforward approach.

[aboba]

Agree that c) is the most promising. Ortc lib supports b) and is in use by games developers who care about footprint and generally are not SDP-savvy, thereby preferring b) to a). However, when asked about the future, games developers seem to prefer c) over b) since it would allow them to utilize WebTransport with stats support to address both client/server and p2p aspects of their games, as well as providing for operational monitoring.

[lgrahl]

Would find it sad to see b) not being pursued but I can see the value in shedding off some complexity (that DTLS/SCTP undoubtedly brings) iff this is incompatible with WebRTC/ORTC anyway. If it isn't inherently incompatible, then there's definitely value in keeping both DTLS/SCTP and QUIC.