Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import images from ISFDB to IPFS / Neo4j #2

Open
dysbulic opened this issue Jul 6, 2020 · 1 comment
Open

Import images from ISFDB to IPFS / Neo4j #2

dysbulic opened this issue Jul 6, 2020 · 1 comment

Comments

@dysbulic
Copy link
Member

dysbulic commented Jul 6, 2020

The IPFS database with all the images from the ISFDB became corrupted somehow and the daemon would not boot with it.

There is a rake task, isfdb:covers which will search the filesystem in the pattern output by export:covers.

Some of the images have zero size, I know from the import log. There are likely other errors as well since I know the /#{author}/#{title}/#{isbn}.#{ext} sometimes produced files that were just .jpg with no filename.

@dysbulic
Copy link
Member Author

The import script is running and checking the filesystem for existing images. When an image is not found, a Cover node is created with a url, but no cid. Another rake task checks for any such covers, downloads them, and saves the CID to the node.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant