Decentralized Mutable Torrents #34
Comments
|
I see big overlap with #19 considering that it'll also need a way to point at a DHT put key and retrieve a torrent from that. And a (possibly old) torrent that just points to the dht which points to another (new) torrent is the wrong tool for the job, a magnet would convey the intent of pointing to content instead of directly describing the content more clearly, which brings us even closer to the my feed idea. It would basically be a building block thereof. |
|
It is indeed inspired mostly by #19. However, I think DHT feeds are at a higher level of abstraction - more details, and hence more things to implement. On the other hand, this concept of simply having mutable DHT keys point to torrents, and not care about their contents (it could be an .rss file or simply anything else), seems to be more generic and hence would allow for more use cases. For instance, I could be wanting to share data that doesn't necessarily have a time component to it, like feeds have - one day I may want to share a Linux distro, the next day I may simply want to share my personal video. Not everything is a stream of growing content. |
|
how about a new magnet link format, that instead of using an info-hash as key, uses an ed25519 public key + an optional salt. The DHT data blob it resolves to is expected to be an info-hash, or maybe even just a regular "?xt=urn:btih"-style magnet link. I'm not sure what the best magnet link arguments would be to represent this. I believe magnet links are meant to be content addresses. "?xt=" (exact topic) may not be suitable for instance, even if we would say "?xt=urn:btpk" or something like that. |
|
@arvidn why do you need the pubkey + optional salt in the actual magnet link URI rather than just the bare minimum (the hash)? |
|
Your question is poorly phrased. If the pubkey is needed then the hash cannot be the bare minimum because it would be less than the minimum. And BEP44 clearly states
Therefore knowledge of the salt is needed. |
|
@the8472 ah I see. Anyway the implementation could default to no salt (or empty string), in which case the hash is the bare minimum needed for the magnet URI. |
|
To fully support DHT store perhaps you're right, pubkey + optional salt is needed. Can we base64 stuff in the magnet URI :) ? Just thinking how we can make it smaller to share addresses around. |
|
@lmatteis I take it when you say "hash", you mean public key. The "address" of a mutable DHT blob (mutable put/get) is the public key used to sign it. info-hashes in regular magnet links are hex encoded (and initially they were base32 encoded). sure it's possible to base64 encode it, assuming the "web-safe" base64 alphabet. |
|
@arvidn yeah I just forgot the salt part. In which case perhaps Do you think I should perhaps start writing a BEP about this? I was trying to follow the steps here (http://bittorrent.org/beps/bep_0001.html) but the forums seem down, so Im doubtful about whether the process is still the same? |
|
yeah, the google group permissions seems to have been difficult and I believe the plan is to change the link on bittorrent.org to just point to the issue-tracker on this github project instead. |
|
regarding a magnet link, I imagined that the salt could be a separate argument, similar to other optional values in current magnet links, like &dn= and &tr= etc. |
|
How about |
|
but, if all you have is the hash of the public key, how are you going to use it? You can't reverse the hash and reproduce the key. |
|
@arvidn isn't the hash of the publickey also the key of the item in the DHT? So you'd |
|
So I guess what I meant was that hash of pubkey + salt. That would be enough, right? Always forgetting about the damn salt. Edited to make it clearer. |
|
Also with a base64 alphabet for the address (in the magnet link) there can be some brute forcing to allow users to find more "readable" addresses. Similar to how some Bitcoin/Tor addresses are semi-readable (for instance I tried reading a bit on the magnet spec and I'm not sure. Perhaps we can use Also, it's really an address if you think about it. Usually when there's a string that points to some data that changes, it's called an address. So maybe |
|
right, if you have a salt, you need to hash the key and salt together to compute the target id. But more importantly, in order to validate the response you get back from the DHT, you need to verify its signature, which means you need to know the public key. I get the impression that the (original) intention in the magnet link spec, "?xt=" is meant to specify a content address. An info-hash is a content address, but a public key to a mutable slot is not. As you point out, it's more like a URL whose content may change. I don't really have strong opinions about this, but it may make sense to look for an alternative top-level key, other than "?xt=". |
|
@arvidn if the response returns the pub key, why do you need to know it beforehand? |
|
because the person responding may lie. that entity just needs to find any data and any public key that end up producing the same hash as you requested. This is not nearly as difficult as forging a signature, especially with SHA-1. At least this is my understanding. some crypto expert may want to correct or corroborate. |
|
SHA-1 is considered weak enough that it is generally not recommended for new applications, but in practice its preimage resistance remains intact, for now. On the other hand, given SHA-1's weakened state, there is a good chance we'll want to migrate bittorrent to SHA-2 or 3 at some point in the future. If this happens magnet links to mutable torrents would not need to change if we were already including the full pubkey. |
|
We're talking about And if we assume that the user obtained the magnet from a non-malicious source then a malicious agent could not execute a collision attack on sha1, he would need to perform a preimage attack. There currently is no known preimage attack against sha1. That said, sha1 would be a weaker link than the other cryptographic primitives used in the chain of trust. |
|
How about the
so a real life magnet URI would be
a base64 alternative (which I actually don't like because slashes break copy&paste):
|
|
|
Here's a first draft of the reference implementation. It's an extension of the |
|
I think this can be closed |
|
I'm a little late, like 6 year... anyone know if the proximity attack detailed here https://gist.github.com/substack/eadd13302d785dc13aac can be used even with bep 46? |
Perhaps I'm missing something, but that appears to be a misunderstanding on how DHT routing works. The set of nodes responsible for a specific key does not change due to other keys being present. What an attacker could do is try to exhaust the storage of the target nodes, but the nodes can make that more difficult by throttling write requests, applying per-IP(-range)-limits and giving more weight to entries that are republished by more readers (thus forcing an attacker to match the popularity of content).
46 builds on top of 44, so yes, it would be affected if 44 were. |
Yes, this sort of attack is still possible. Implementing BEP 42, specifically the part about having IDs be determined by your IP, on the DHT should help protect against this sort of attack. I'm not sure if anyone has actually implemented this for the mainline DHT, but the Hyperswarm DHT does something along these lines to help improve its security. (on top of other things). |
I've been searching for ways to have users benefit from DHT store (BEP 44) more directly. One use case would be the idea of "decentralized mutable torrents".
To me this is the bare minimum to allow for content creators to finally start using DHT store functionality directly, and benefit from its decentralized properties.
From a publishers perspective I imagine this to be really simple: in the clients preferences there could be a "mutable torrents" section which contains a table mapping various mutable keys to torrents that are being seeded. The publisher can accordingly change which torrent maps to which mutable key.
From a consumers perspective, they could add such mutable .torrent (or magnet link) and there could be an "auto download" checkbox/option. Obviously it would function differently from normal torrents, because it would need to DHT get for updates, and change swarms when there're updates. I'm wondering though how the UI for a mutable torrent download would be like; I guess with "auto download" on, you'd have the torrent change automatically if the publisher changed it.
Thoughts?
@the8472 @arvidn
The text was updated successfully, but these errors were encountered: