-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ability to skip a problematic page and move on #75
Comments
rivernews
added a commit
to rivernews/review-scraper-java-development-environment
that referenced
this issue
May 22, 2020
Click approachThere's issues when enabling click element approach - it will cause
|
Infinite Job
|
rivernews
added a commit
to rivernews/review-scraper-java-development-environment
that referenced
this issue
May 27, 2020
We approach this by bringing in click approach again, and put it after guess url approach. No error reported again. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Some webpage failed locating review panel, because of a bug in the webpage. No matter how much we re-try, the result is the same. Probably this is due to glassdoor using server side rendering to generate static pages so the page doesn't change until the next update cycle. But we encounter the same problems as well, so looks like the update doesn't help.
Closer look into the cause (increase survival rate)
We currently don't have any idea of tackling this to try to scrape the content
click()
method may lead to the "page state has changed so cannot findElement on tainted DOM".Overall approach to increase throughput
We may focus on another approach to have the ability to skip
The text was updated successfully, but these errors were encountered: