Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for webkit #56

Merged
merged 2 commits into from
Apr 11, 2022
Merged

Add support for webkit #56

merged 2 commits into from
Apr 11, 2022

Conversation

rdmurphy
Copy link
Contributor

Not sure why webkit was excluded seeing as it's one of the three major browsers supported by Playwright, but this PR adds it.

@simonw
Copy link
Owner

simonw commented Apr 8, 2022

I skipped webkit because it needs a bit more work - right now I get the following error:

% shot-scraper https://www.whatismybrowser.com/detect/what-is-my-user-agent/ -b webkit
Traceback (most recent call last):
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/bin/shot-scraper", line 33, in <module>
    sys.exit(load_entry_point('shot-scraper', 'console_scripts', 'shot-scraper')())
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/Users/simon/Dropbox/Development/shot-scraper/shot_scraper/cli.py", line 166, in shot
    context, browser_obj = _browser_context(
  File "/Users/simon/Dropbox/Development/shot-scraper/shot_scraper/cli.py", line 217, in _browser_context
    browser_obj = p.chromium.launch(**browser_kwargs)
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/playwright/sync_api/_generated.py", line 11417, in launch
    self._sync(
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/playwright/_impl/_sync_base.py", line 111, in _sync
    return task.result()
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/playwright/_impl/_browser_type.py", line 90, in launch
    Browser, from_channel(await self._channel.send("launch", params))
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 39, in send
    return await self.inner_send(method, params, False)
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 63, in inner_send
    result = next(iter(done)).result()
playwright._impl._api_types.Error: Unsupported chromium channel "webkit"

I'm confident it's possible to get it working, but I think the code here needs some fixes:

browser_kwargs = dict(headless=not interactive, devtools=devtools)
if browser == "chromium":
browser_obj = p.chromium.launch(**browser_kwargs)
elif browser == "firefox":
browser_obj = p.firefox.launch(**browser_kwargs)
else:
browser_kwargs["channel"] = browser
browser_obj = p.chromium.launch(**browser_kwargs)

@rdmurphy
Copy link
Contributor Author

rdmurphy commented Apr 9, 2022

@simonw Derp — sorry about that. I had another branch in flight where I added that but forgot to pull it into this. Should be good now!

@simonw
Copy link
Owner

simonw commented Apr 11, 2022

Tested this with:

shot-scraper 'https://www.whatismybrowser.com/detect/what-is-my-user-agent/' --browser webkit --height 400

www-whatismybrowser-com-detect-what-is-my-user-agent 2

And:

% shot-scraper javascript https://www.example.com/ 'navigator.userAgent' --browser webkit
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.4 Safari/605.1.15"

@simonw simonw merged commit 0b0b3c7 into simonw:main Apr 11, 2022
simonw added a commit that referenced this pull request Apr 11, 2022
simonw added a commit that referenced this pull request Apr 11, 2022
So that WebKit can work, refs #56
simonw added a commit that referenced this pull request Apr 11, 2022
simonw added a commit that referenced this pull request Apr 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants