feat: ipns over dht

[vasco-santos]

Here is an initial implementation of IPNS over DHT. Until the dht is enabled by default in js-ipfs we need to use the --enable-dht-experiment experimental flag, or it will only use the local repo as datastore.

Needs:

• libp2p/js-libp2p#288
• libp2p/js-libp2p-kad-dht#57
• js-libp2p release with libp2p/js-libp2p#288

Unblocks:

• ipfs/interop#47

[dirkmc]

@vasco-santos it's great to see you moving ahead with this work

It looks like the code only gets a single IPNS value from the DHT, whereas in the go version it retrieves multiple versions of the IPNS value (I believe the default is 16).

The go version provides a streaming API. Do you plan to support streaming in JS?

[vasco-santos]

hey @dirkmc

So the stream was added recently to go-ipfs, in the latest ipfs@.4.18iirc!

We should add support for it in js-ipfs, as well, but I believe the main focus now should be to have this initial implementation as go had before and have interop between go and js. After that, I think it will be reasonable to add a stream API for this. We have been discussing adding stream's in the API for other things, and probably we should look into this too.

Just other detail, we currently do the dht.getMany with 16 occurences too, in the js-libp2p-kad-dht libp2p/js-libp2p-kad-dhtsrc/private.js#L271. However, we select the best record afterwards (using a select function provided), and return the "best" value.

[dirkmc]

@vasco-santos I agree about the streaming interface, it can wait until a later version. The reason I bring it up is because in the go version of IPNS, retrieving those 16 records takes so long that it almost always times out, so IPNS is painfully slow. I added an option to retrieve less than 16 records to mitigate this problem, but the real solution is streaming.

I think a lot of people were turned away from IPNS and even IPFS because of IPNS slowness so it would be great to get ahead of the issue as soon as possible

[jacobheun]

This looks good. Interop tests for ipns are also passing. Ci failures there are unrelated to the ipns tests.

[alanshaw]

@vasco-santos would you mind rebasing this now that the IPNS over pubsub work is merged? 🙏

[alanshaw]

Could you please also look at

js-ipfs/src/core/ipns/publisher.js

Line 79 in 83b8e9b

(cb) => embedPublicKeyRecord ? this._publishPublicKey(keys.routingPubKey, publicKey, peerId, cb) : cb()

I think it is currently the wrong way round, but we should always publish it for the time being until there's no more go-ipfs nodes that require it.

[dirkmc]

W00t 🎉