How to get the html of the web page ScrapeGraphAI received?

[Tony20221]

I would like to get the html of the web page that ScrapeGraphAI received as a response. I want to compare its content vs what I receive when I use a browser.
How to save the web page's raw HTML in a file?

[dosubot[bot]]

Hello @Tony20221! I'm here to help you out with any bugs, questions, or contributions you have. If you need assistance, feel free to ask.

To get and save the raw HTML of the web page that ScrapeGraphAI received, you can use the scrape_do_fetch function from the scrape_do.py module. Here's how you can do it:

# Import the function
from scrape_do import scrape_do_fetch

# Define your parameters
token = 'your_api_token'
target_url = 'http://example.com'
use_proxy = False  # or True if you want to use a proxy
geoCode = None  # or specify a country code if needed
super_proxy = False  # or True if you want to use Residential & Mobile Proxy Networks

# Fetch the raw HTML
raw_html = scrape_do_fetch(token, target_url, use_proxy, geoCode, super_proxy)

# Save the raw HTML to a file
with open('webpage.html', 'w', encoding='utf-8') as file:
    file.write(raw_html)

This code will fetch the raw HTML of the specified web page and save it to a file named webpage.html ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}