Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ipfs add -r --nocopy Error: Failed to get block for zb2rh... The system cannot find the path specified. #4260

Open
ivan386 opened this issue Sep 21, 2017 · 8 comments

Comments

@ivan386
Copy link
Contributor

ivan386 commented Sep 21, 2017

Version information:

go-ipfs version: 0.4.11-rc2-
Repo version: 6
System version: amd64/windows
Golang version: go1.9

Type: Bug

Severity: High

Description:

I try to add folder with --nocopy folder and get error.

Reproduce:

some_folder\test.txt

ipfs add -r -nocopy some_folder

rename "some_folder" to "some_folder_renamed"

ipfs add -r -nocopy some_folder_renamed

Error: Failed to get block for zb2rh...: open C:...\some_folder\test.txt: The system cannot find the path specified.

@whyrusleeping
Copy link
Member

When adding content with the experimental filestore and the --nocopy option, moving files after you have added them will break things.

cc @kevina whats the recovery strategy here?

@kevina
Copy link
Contributor

kevina commented Sep 21, 2017

@whyrusleeping we don't have one, in the short to mid term I would like to see several things

  1. Prove an --overwrite option that will replace existing filestore entries (I have talked about this elsewhere: Implement basic filestore 'no-copy' functionality #3629 (comment))
  2. Merge filestore rm filestore rm #3815
  3. Implement filestore clean to remove invalid entries

Note that (1) is useful because it can fix existing entries without having to worry about pinning. (2) is another way to manually clean up and is a prerequisite of (3).

In the long term I would like to allow multiple filestore entries per block. I have implemented this in my own implementation but will likely require a decent degree of work to get into your implementation.

@kevina
Copy link
Contributor

kevina commented Sep 21, 2017

Note that if --overright is implemented @ivan386 code will just work.

@NiKiZe
Copy link

NiKiZe commented Jul 20, 2018

Hitting this as well and it seems that you can;
ipfs pin rm hash
That might tell you it has a parent then unpin that.
And finally run ipfs repo gc, then rerun the add command.
(ipfs pin rm might not be needed?)

Is that correct? Is there any better way until filestore clean or filestore rm is available?

@ivan386
Copy link
Contributor Author

ivan386 commented Jul 20, 2018

Try filestore verify

@NiKiZe
Copy link

NiKiZe commented Jul 21, 2018

Try filestore verify

That does indeed remove the error
What I'm working on is ipfs/notes#296
And I expect this to be around 4-500GB of data, so running a full verify after each rsync will not really be acceptable. (we know this is all experimental but just mentioning the issue).

What I however did discover is that ipfs repo gc actually works just fine to resolve issues with missing files quite quickly. I don't understand how this does solve it, but it "does for me", Is it a bug that it works? is it causing more hidden issues that I have not yet seen?

Update: This only helps to get ipfs add to not error out, nothing does actually seen to be cleaned, not with verify either, It does seem to be an misunderstanding on my part, See ipfs/notes#296 (comment)

@Stebalien
Copy link
Member

Make sure your files are pinned. If they aren't, a GC may fix this issue because it may delete references to the problematic blocks.

@ochilan
Copy link

ochilan commented Sep 26, 2018

Hello! I'm currently looking into using IPFS for publishing videos through DTube. Organizing my videos in the regular filesystem and adding them to IPFS using --nocopy seems to be a natural thing to me. Since it has been a year since plans to add functions like moving and removing files in filestore have popped up, I wanted to ask if there is any progress.

I also have two related questions:

  1. Am I understanding correctly that a workaround is to "pin rm" the nonexistent hash and subsequently run a gc? In a quick test, this fixed the issue of not being able to re-add the file using a different filename.

  2. If I choose to host some files on IPFS now, but not using filestore just yet, would it be wise to use "--raw-leaves" when adding files even when not using filestore? I'm asking because using raw leaves results in different hashes than without using them. If I migrate to filestore at a later point in time (which requires raw leaves), would the probability be greater that the hashes will stay the same? If not, migrating would require editing possibly many, many (DTube) postings to point to the correct hashes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants