Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: pin: context deadline exceeded #1630

Closed
davidar opened this issue Aug 31, 2015 · 13 comments
Closed

Error: pin: context deadline exceeded #1630

davidar opened this issue Aug 31, 2015 · 13 comments

Comments

@davidar
Copy link
Member

davidar commented Aug 31, 2015

Pinning seems to time out on large (10GB) directories:

# load into cache
$ ipfs refs -r QmYaFMPTgDkwp3vun6o3zfVv6gFwDWVSJWwZQHieXj1Ys6
<snip>
# make sure it's definitely cached
$ ipfs get QmYaFMPTgDkwp3vun6o3zfVv6gFwDWVSJWwZQHieXj1Ys6

$ ipfs pin add -r QmYaFMPTgDkwp3vun6o3zfVv6gFwDWVSJWwZQHieXj1Ys6
Error: pin: context deadline exceeded

Affects: ipfs-inactive/archives#2

Edit: Actually, it would also be nice if you didn't have to do the caching step separately (i.e. make pin act like get, but without actually creating files in the current directory).

@whyrusleeping
Copy link
Member

@davidar thats interesting. how long did the pin call take?

@davidar
Copy link
Member Author

davidar commented Sep 2, 2015

@whyrusleeping A minute or two before error, I guess (refs -r also takes a while just to list blocks from the cache). I can time it exactly later today if that would help.

@rht
Copy link
Contributor

rht commented Sep 3, 2015

Actually, it would also be nice if you didn't have to do the caching step separately (i.e. make pin act like get, but without actually creating files in the current directory).

Related #875

@whyrusleeping
Copy link
Member

I've got to be honest, i cant tell where that timeout in pin is coming from... @rht can you see where?

@davidar
Copy link
Member Author

davidar commented Sep 3, 2015

@rht not exactly, what I mean is download to the block store and pin it simultaneously (without timing out). A progress bar too would be nice

@rht
Copy link
Contributor

rht commented Sep 3, 2015

@whyrusleeping In corerepo.Pin, it comes from either

  1. core.Resolve(ctx,...) which calls n.Resolver(ctx,...) (which timeout happens only in path.Resolver.ResolveLinks)
  2. n.Pinning.Pin(ctx, ...) which calls Blockservice.GetBlock (which timeout happens somewhere in bitswap).

Currently the two can't be distinguished because both has error with prefix "pin: ".
Maybe @davidar can change the message in https://github.com/ipfs/go-ipfs/blob/master/core/corerepo/pinning.go#L32 to "pin: resolve: ", rebuild, then rerun the ipfs pin add and see whether the error message has this prefix?

Edit: clarified the path

@rht
Copy link
Contributor

rht commented Sep 3, 2015

@davidar I ref-ed the issue because this is another view of not wanting to "duplicate" files (you only want the cache).

@rht
Copy link
Contributor

rht commented Sep 3, 2015

Actually, since ipfs refs -r uses core.Resolve and it works just fine, the err is constrained to come from the 2nd one (n.Pinning.Pin). @davidar sorry no need to run the test.

@whyrusleeping
Copy link
Member

@rht it has to be the one in ResolveLinks, i'm fairly certain that bitswap wont ever timeout a request.

@rht
Copy link
Contributor

rht commented Sep 3, 2015

@rht
Copy link
Contributor

rht commented Sep 3, 2015

Wish this could return an iterator rht@6d55bc9

@jbenet
Copy link
Member

jbenet commented Sep 3, 2015

@rht can use channels as iterators

i like iterators in python. probably the best support i've seen in a language for them. (without going c++ operator overloading crazy).

@whyrusleeping
Copy link
Member

This has since been improved, ipfs pin add is much better about timeouts and resource consumption

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants