Fully resolve ipfs paths down to its final hash #1320

travisperson · 2015-06-02T08:52:31Z

Given an ipfs path, this will resolve it down to it's furthest point.

Tests to come.

Given an ipfs path, this will resolve it down to it's furthest point.

whyrusleeping · 2015-06-02T14:49:00Z

core/commands/resolve.go

@@ -84,6 +84,23 @@ Resolve the value of another name recursively:
 			return
 		}

+		if len(output.Segments()) > 2 {
+			merkNode, err := n.Resolver.ResolvePath(n.Context(), output)


i'm gonna use merkNode from now on.

whyrusleeping · 2015-06-02T14:49:23Z

what case wasnt being handled previously?

travisperson · 2015-06-02T16:19:19Z

Given a path after a hash it would just return the same input and not actually do anything.
Before

$ ipfs resolve /ipfs/Qmf2yWpPKY19GqqfCidT9cCKMZnpvGfo1tHdVAYHGs6b2z/something
/ipfs/Qmf2yWpPKY19GqqfCidT9cCKMZnpvGfo1tHdVAYHGs6b2z/something⏎

After

$ ipfs resolve /ipfs/Qmf2yWpPKY19GqqfCidT9cCKMZnpvGfo1tHdVAYHGs6b2z/something
/ipfs/QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn⏎

fixed copy paste error

This was because ResolveN just called parsePath and make a Path out of the input and didn't actually do anything else.

I was going to put this logic under ResolveN, but I don't have access to the node. However, it might be important that these steps take place there instead.

whyrusleeping · 2015-06-02T17:29:04Z

cc @wking

whyrusleeping · 2015-06-02T17:34:57Z

also the output /ipfs//ipfs/QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn seems wrong. got an extra ipfs on there

wking · 2015-06-02T17:42:26Z

I want this feature (see also #1082), but I'm not currently clear on
the intended UI (see #1307). And I also think we want to flip the
logic around and add an IPFS resolver (the stuff that's currently in
core/pathresolver.go) to our multi-protocol resolver (currently in
namesys/namesys.go) and teach the multi-protocol logic (currently in
namesys/base.go) to distinguish between symlinks (which replace the
looked-up path in the segment stack) and non-symlinks (which reduce
the segment stack). See also 1.

Because as far as path-resolution goes, it doesn't matter if we're
using Merkle nodes, DHT links, DNS links, … to figure out what a path
means.

travisperson · 2015-06-02T18:48:33Z

also the output /ipfs//ipfs/QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn seems wrong. got an extra ipfs on there

Copy and paste error, it it's not there when ran.

whyrusleeping · 2015-06-02T18:54:04Z

I like the functionality that this command has in this PR.

whyrusleeping · 2015-06-02T22:44:01Z

i must be really out of the loop. Didnt this command used to be ipfs name resolve ?

jbenet · 2015-06-03T07:07:23Z

@travisperson i think full resolution should be under a -r / --recursive flag. (as @wking points out)

jbenet · 2015-06-03T07:12:24Z

@wking

i think there's many parts here:

resolving object links (just ipfs objects)
resolving key names (via routing records)
resolving dns names (via dns)
resolving .bit names (via namecoin)
resolving onion names (via tor)
resolving all names (everything currently under /ipns/..., key names and dns names, etc)
all of the above

(this is over-exhaustive to show that there may be many different sort of names to resolve)

today, we have 6.) under ipfs name resolve, and that's fine.

i'd like:

ipfs resolve to mean (7.),
ipfs object resolve to be (1.) above.
ipfs name resolve and/or just ipns resolve to mean (6.) above
ipfs dns resolve and/or just dns resolve to be (3.)

2., 4., 5. could have their own tools / distinctions down the road, but not common enough (or not even implemented yet)

(hit return too early)

jbenet · 2015-06-03T07:30:32Z

(btw, ipfs object will be renamed to ipfs dag)

travisperson · 2015-06-03T07:54:24Z

@travisperson i think full resolution should be under a -r / --recursive flag. (as @wking points out)

Prior to this PR, the resolve command did nothing for ipfs paths, it just returned the same input. As it is, it's not really a recursive resolve, unless you only want it to resolve a single folder at a time, such as:

$ ipfs resolve /ipfs/<hash>/path/to/end
/ipfs/<hash>/to/end

wking · 2015-06-03T17:28:02Z

On Tue, Jun 02, 2015 at 03:44:02PM -0700, Jeromy Johnson wrote:

i must be really out of the loop. Didnt this command used to be
ipfs name resolve ?

It did until #1208, and it likely will have that name again soon.
That's what #1307 is about.

wking · 2015-06-03T17:47:01Z

On Wed, Jun 03, 2015 at 12:12:25AM -0700, Juan Batiz-Benet wrote:

i think there's many parts here:

I responded in #1307 to keep the intended interface conversation in
one place.

wking · 2015-06-03T17:53:46Z

On Wed, Jun 03, 2015 at 12:30:33AM -0700, Juan Batiz-Benet wrote:

(btw, ipfs object will be renamed to ipfs dag

Spun off into #1328.

wking · 2015-06-03T18:13:36Z

On Wed, Jun 03, 2015 at 12:54:25AM -0700, Travis Person wrote:

i think full resolution should be under a -r / --recursive
flag. (as @wking points out)

Prior to this PR, the resolve command did nothing for ipfs paths, it
just returned the same input. As it is, it's not really a recursive
resolve, unless you only want it to resolve a single folder at a
time, such as:

$ ipfs resolve /ipfs//path/to/end
/ipfs//to/end

This is how I think a non-recursive Merkle-link lookup should work, so
I want that in the all-protocol resolver. Although I'd phrase it
like this:

With the Merkle DAG:

Object        Link
           Name Hash
========   ==== ========
<hash-1>   path <hash-2>
<hash-2>   to   <hash-3>
<hash-3>   end  <hash-4>

Then:

$ ipfs resolve /ipfs/<hash-1>/path/to/end
/ipfs/<hash-2>/to/end
$ ipfs resolve --recursive /ipfs/<hash-1>/path/to/end
/ipfs/<hash-4>

But I'd also like to see support for symlink objects with arbitrary
data (including any supported protocol):

/ipns//child

Then we could have:

With the Merkle DAG:

Object     Type           Link        Data
                       Name Hash
========   ==========  ==== ========  =====
<hash-1>   unixfs-dir  a    <hash-2>
<hash-2>   unixfs-dir  b    <hash-3>
<hash-3>   symlink                    /ipns/<pk-hash>/d
<hash-4>   unixfs-dir  c    <hash-5>
<hash-5>   fanout      d*   <hash-6>
<hash-6>   fanout-leaf d    <hash-7>
<hash-7>   unixfs-dir  e    <hash-8>

And the DHT-namesys entries:

 Pubkey     Record
 =========  ==============
 <pk-hash>  /ipfs/<hash-4>/c

Then:

$ ipfs resolve /ipfs/<hash-2>/b/e
/ipfs/<hash-3>/e
$ ipfs resolve /ipfs/<hash-3>/e
/ipns/<pk-hash>/d/e
$ ipfs resolve /ipns/<pk-hash>/d/e
/ipfs/<hash-4>/c/d/e
$ ipfs resolve /ipfs/<hash-4>/c/d/e
/ipfs/<hash-5>/d/e
$ ipfs resolve /ipfs/<hash-5>/d/e
/ipfs/<hash-7>/e
$ ipfs resolve --recursive /ipfs/<hash-2>/b/e
/ipfs/<hash-8>

Besides the link through IPNS space, this also shows the non-recursive
form silently traversing the fanout tree.

jbenet · 2015-06-03T21:32:47Z

This is how I think a non-recursive Merkle-link lookup should work, so
I want that in the all-protocol resolver. Although I'd phrase it
like this:

yes, i like that resolve sequence.

But I'd also like to see support for symlink objects with arbitrary
data (including any supported protocol):

Yes, that looks right to me. the links are a bit tricky. we need to think carefully how to "symlink" from a merkledag object to an ipns mutable record safely. The nice replicated dag guarantees are a bit broken if we dont do it carefully.

btw, dm (mazieres) pointed out to me that getting symlinks right was going to be hard. :)

this also shows the non-recursive
form silently traversing the fanout tree.

yes agreed with this. this is actually similar to how git traverses commit -> first tree object. we need a good, reliable way to do this. btw, the fanout links thing should work like a union or a unixfs file-- each node should work as a valid root (i.e. no fanout and fanout-leaf distinction).

jbenet · 2015-06-03T22:12:44Z

@travisperson clear on next things to do?

wking · 2015-06-03T22:41:12Z

On Wed, Jun 03, 2015 at 02:32:47PM -0700, Juan Batiz-Benet wrote:

But I'd also like to see support for symlink objects with
arbitrary data (including any supported protocol):

Yes, that looks right to me. the links are a bit tricky.

Symlinks are #1104. Their impact here is just: if we want links out
of the immutable Merkle DAG at all, we'll need to figure out how
they'll be represented in the resolver UI. For me, they motivate
moving Merkle-DAG resolution into NameSys 1.

btw, the fanout links thing should work like a union or a unixfs
file-- each node should work as a valid root (i.e. no fanout and
fanout-leaf distinction).

Spun off into ipfs/specs#10.

rht · 2015-10-16T12:58:30Z

@whyrusleeping can be closed

rht · 2015-10-16T12:58:35Z

(here goes)

Fully resolve ipfs paths down to its final hash

c1ba446

Given an ipfs path, this will resolve it down to it's furthest point.

jbenet added the backlog label Jun 2, 2015

whyrusleeping reviewed Jun 2, 2015
View reviewed changes

wking mentioned this pull request Jun 2, 2015

initial implementation of save #1187

Closed

wking mentioned this pull request Jun 3, 2015

Command line interface for name resolution and publishing #1307

Closed

wking mentioned this pull request Jun 3, 2015

Rename 'ipfs object' to 'ipfs dag' #1328

Closed

wking mentioned this pull request Jun 3, 2015

Distinguish fanout from non-fanout links ipfs/specs#10

Closed

rht mentioned this pull request Sep 9, 2015

Resolve IPFS DAG paths in ipfs resolve command #1662

Merged

rht closed this Oct 16, 2015

jbenet removed the backlog label Oct 16, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fully resolve ipfs paths down to its final hash #1320

Fully resolve ipfs paths down to its final hash #1320

travisperson commented Jun 2, 2015

whyrusleeping Jun 2, 2015

whyrusleeping commented Jun 2, 2015

travisperson commented Jun 2, 2015

whyrusleeping commented Jun 2, 2015

whyrusleeping commented Jun 2, 2015

wking commented Jun 2, 2015

travisperson commented Jun 2, 2015

whyrusleeping commented Jun 2, 2015

whyrusleeping commented Jun 2, 2015

jbenet commented Jun 3, 2015

jbenet commented Jun 3, 2015

jbenet commented Jun 3, 2015

travisperson commented Jun 3, 2015

wking commented Jun 3, 2015

wking commented Jun 3, 2015

wking commented Jun 3, 2015

wking commented Jun 3, 2015

jbenet commented Jun 3, 2015

jbenet commented Jun 3, 2015

wking commented Jun 3, 2015

rht commented Oct 16, 2015

rht commented Oct 16, 2015

Fully resolve ipfs paths down to its final hash #1320

Fully resolve ipfs paths down to its final hash #1320

Conversation

travisperson commented Jun 2, 2015

whyrusleeping Jun 2, 2015

Choose a reason for hiding this comment

whyrusleeping commented Jun 2, 2015

travisperson commented Jun 2, 2015

whyrusleeping commented Jun 2, 2015

whyrusleeping commented Jun 2, 2015

wking commented Jun 2, 2015

travisperson commented Jun 2, 2015

whyrusleeping commented Jun 2, 2015

whyrusleeping commented Jun 2, 2015

jbenet commented Jun 3, 2015

jbenet commented Jun 3, 2015

jbenet commented Jun 3, 2015

travisperson commented Jun 3, 2015

wking commented Jun 3, 2015

wking commented Jun 3, 2015

wking commented Jun 3, 2015

wking commented Jun 3, 2015

jbenet commented Jun 3, 2015

jbenet commented Jun 3, 2015

wking commented Jun 3, 2015

rht commented Oct 16, 2015

rht commented Oct 16, 2015