Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix level db panic #6186

Merged
merged 1 commit into from
Apr 9, 2019
Merged

Fix level db panic #6186

merged 1 commit into from
Apr 9, 2019

Conversation

michaelavila
Copy link
Contributor

This PR exists for me to trouble shoot some CI issues. Disregard.

@ghost ghost assigned michaelavila Apr 4, 2019
@ghost ghost added the status/in-progress In progress label Apr 4, 2019
@Stebalien
Copy link
Member

#6185?

@michaelavila
Copy link
Contributor Author

@Stebalien I'm actually using this PR to troubleshoot the leveldb panics as I can't reproduce them locally. Are you saying #6185 should fix that?

@michaelavila michaelavila changed the title [DO NOT MERGE] Fix/rc1 [DO NOT MERGE] Fix level db panic Apr 4, 2019
@Stebalien
Copy link
Member

Ah, no. Sorry, different issue.

In terms of the leveldb panics, I think we're closing the datastore while reading from it (or reading from it after closing it?).

@michaelavila
Copy link
Contributor Author

@Stebalien yeah, there are those panics and there's also the panic from the pinning test:

panic: interface conversion: cache.Value is nil, not *table.Reader

goroutine 103 [running]:
github.com/syndtr/goleveldb/leveldb.(*tOps).find(0xc00034fc20, 0xc0001bc0f0, 0xc000456090, 0x2b, 0x2b, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/circleci/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/table.go:383 +0x221
github.com/syndtr/goleveldb/leveldb.(*version).get.func1(0x0, 0xc0001bc0f0, 0xc000456090)
	/home/circleci/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/version.go:176 +0x452
github.com/syndtr/goleveldb/leveldb.(*version).walkOverlapping(0xc000330a50, 0x0, 0x0, 0x0, 0xc000456090, 0x2b, 0x2b, 0xc000347910, 0xc0003478e0)
	/home/circleci/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/version.go:116 +0x2a7
github.com/syndtr/goleveldb/leveldb.(*version).get(0xc000330a50, 0x0, 0x0, 0x0, 0xc000456090, 0x2b, 0x2b, 0x0, 0x0, 0x0, ...)
	/home/circleci/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/version.go:160 +0x2b2
github.com/syndtr/goleveldb/leveldb.(*DB).get(0xc000286000, 0x0, 0x0, 0x0, 0x0, 0xc000456060, 0x23, 0x30, 0x1d, 0x0, ...)
	/home/circleci/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:779 +0x397
github.com/syndtr/goleveldb/leveldb.(*DB).Get(0xc000286000, 0xc000456060, 0x23, 0x30, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/circleci/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:845 +0x12f
github.com/ipfs/go-ds-leveldb.(*accessor).Get(0xc000190a20, 0xc000456030, 0x23, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/circleci/go/pkg/mod/github.com/ipfs/go-ds-leveldb@v0.0.1/datastore.go:84 +0x82
github.com/ipfs/go-ds-measure.(*measure).Get(0xc0001de000, 0xc000456030, 0x23, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/circleci/go/pkg/mod/github.com/ipfs/go-ds-measure@v0.0.1/measure.go:149 +0xd4
github.com/ipfs/go-datastore/mount.(*Datastore).Get(0xc0001942c0, 0xc000456000, 0x23, 0x24bcc00, 0xbf21969c66d5cebf, 0x1336e5a, 0x249da20, 0x23)
	/home/circleci/go/pkg/mod/github.com/ipfs/go-datastore@v0.0.2/mount/mount.go:197 +0x70
github.com/ipfs/go-ds-measure.(*measure).Get(0xc0001de240, 0xc000456000, 0x23, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/circleci/go/pkg/mod/github.com/ipfs/go-ds-measure@v0.0.1/measure.go:149 +0xd4
github.com/ipfs/go-datastore/keytransform.(*ktds).Get(0xc000336400, 0xc0001ac040, 0x11, 0x1, 0xc0001ae090, 0x10, 0x14a5580, 0x24bdfe0)
	/home/circleci/go/pkg/mod/github.com/ipfs/go-datastore@v0.0.2/keytransform/keytransform.go:40 +0x76
github.com/ipfs/go-ipfs/provider.(*Queue).nextEntry(0xc0004d8a50, 0x0, 0x0, 0xc00009d601, 0xc00009d698)
	/home/circleci/ipfs/go-ipfs/provider/queue.go:73 +0xc2
github.com/ipfs/go-ipfs/provider.(*Queue).work.func1(0xc0004d8a50)
	/home/circleci/ipfs/go-ipfs/provider/queue.go:107 +0x6d1
created by github.com/ipfs/go-ipfs/provider.(*Queue).work
	/home/circleci/ipfs/go-ipfs/provider/queue.go:101 +0x4f

They could be related. The other panics (the 10:56:51.864 ERROR provider: Failed to enqueue cid: leveldb: closed queue.go:121) are just being logged and swallowed up, but the panic in the middle of the pinning test is different and actually panicking ipfs, so I'm a little more worried about it.

@michaelavila michaelavila force-pushed the fix/rc1 branch 6 times, most recently from 29bf7bb to 723bb8b Compare April 5, 2019 20:02
@michaelavila michaelavila force-pushed the fix/rc1 branch 8 times, most recently from 67653f6 to 126f613 Compare April 8, 2019 22:26
provider/queue.go Outdated Show resolved Hide resolved
provider/queue.go Outdated Show resolved Hide resolved
@michaelavila
Copy link
Contributor Author

@Stebalien we had something like your suggestion earlier, but ran into some issues that I didn't this time around. Changes are in. Let me know if you had something else in mind.

License: MIT
Signed-off-by: Michael Avila <davidmichaelavila@gmail.com>
@Stebalien Stebalien changed the title [DO NOT MERGE] Fix level db panic Fix level db panic Apr 9, 2019
@Stebalien Stebalien merged commit fbbf062 into master Apr 9, 2019
@Stebalien Stebalien deleted the fix/rc1 branch April 9, 2019 05:09
@ghost ghost removed the status/in-progress In progress label Apr 9, 2019
@Stebalien
Copy link
Member

Ok, one more blocker down. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants