Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow continuation after ComicVine time out #404

Open
GoogleCodeExporter opened this issue Jun 13, 2015 · 1 comment
Open

Allow continuation after ComicVine time out #404

GoogleCodeExporter opened this issue Jun 13, 2015 · 1 comment

Comments

@GoogleCodeExporter
Copy link

When scraping a large amount of issues, if comicvine times out the scraper pops 
up a window and leaves you to pick up where it left off. I suggest an option to 
"Try to continue" or something like that, maybe with a countdown, so that large 
batches can be done without having to reselect the issues it skipped.

Original issue reported on code.google.com by Rurik...@gmail.com on 5 Feb 2015 at 10:31

@GoogleCodeExporter
Copy link
Author

Hi, thanks for the suggestion.

The best way to deal with the timeout problem that you are seeing is to use the 
SCRAPE_DELAY feature, described here:

http://comicrack.cyolito.com/forum/32-news-and-announcements/33534-comic-vine-sc
raper?start=470#39770

That link also explains why I haven't already implemented the feature you are 
asking for.

Another useful tip is here:

http://comicrack.cyolito.com/forum/8-help/38668-solved-show-unscraped-files#4162
5

This should help you easily discover your currently unscraped comics, in case 
you do hit the timeout and your need to pick up where you left off.

Original comment by cban...@gmail.com on 5 Feb 2015 at 6:02

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants