Unable to access the page contents until it fully loads

[Byperion]

After successfully bypassing Cloudflare, the page takes a total of 90-120 seconds to load. The contents I need shows up within the first 20 seconds. I have a large number of pages to process, in the neighborhood of 450k pages. Is there a way to access the contents before the page fully loads?

[mdmintz]

There are ways of accessing the page sooner:

--pls="none" / pls="none" --> Set pageLoadStrategy to "none": This strategy causes Selenium to return immediately after the initial HTML content is fully received by the browser.

--sjw / sjw=True --> Skip JS Waits, such as wait_for_ready_state_complete().

With those combined, you'll have access to the page quickly, but you may need to add waits on top of that if things are too fast.

[Byperion]

Thanks!