Specialized DHT

[justheuristic]

Based on discussion with @UARTman on 09.02.2020 - we can greatly improve TesseractNetwork by specializing Kademlia DHT to our needs. As of now, there are two issues with the default implementation:

Operating under NAT
At the time of writing, a RemoteExpert client always connects to a runtime via an IP and a port. This is impractical since participants may have have no open ports. We could instead implement direct p2p connections via dht.

Faster lookup
The majority of keys stored in our DHT are expert prefixes used by gating function. These prefixes are not meant to live indefinitely: if no runtime sends prefix information in $t$ seconds, we can safely delete it.

However, Kademlia assumes that if you store a key, this key must be still there after some time. This causes many inefficiencies such as wasting memory on stale keys or "refreshing" keys every 1 or 24 hours.

[justheuristic]

Status:

• Uperating under NAT: tl;dr Anton found out that nat traversal can be done over UPnP or hole-punching. @unconst mentioned that these methods are implemented in libp2p. Further discussion will be moved to a dedicated issue
• Faster lookup: we implemented custom DHT with conflict Re-write DHT #37 , switched to grpc in [part 2] grpc-based dht #51 ; Further enhancements will be dedicated to different issues