Running Playwright in Heroku

[diervo]

Today we are running pupeteer in Heroku by using this buildpack:
https://github.com/jontewks/puppeteer-heroku-buildpack

When trying to deploy with playwright with the trivial webkit example (the one described in this repo) I get the following error:

(node:4) UnhandledPromiseRejectionWarning: Error: WebKit revision is not downloaded. Run "npm install" or "yarn install"
2020-01-24T19:39:52.135779+00:00 app[web.1]:     at WKPlaywright.launchServer (/app/node_modules/playwright-core/lib/server/wkPlaywright.js:94:23)
2020-01-24T19:39:52.135781+00:00 app[web.1]:     at async WKPlaywright.launch (/app/node_modules/playwright-core/lib/server/wkPlaywright.js:67:24)
2020-01-24T19:39:52.135788+00:00 app[web.1]:     at async /app/index.js:5:19
2020-01-24T19:39:52.135801+00:00 app[web.1]:   -- ASYNC --
2020-01-24T19:39:52.135809+00:00 app[web.1]:     at WKPlaywright.<anonymous> (/app/node_modules/playwright-core/lib/helper.js:54:23)
2020-01-24T19:39:52.135811+00:00 app[web.1]:     at WKPlaywright.launch (/app/node_modules/playwright-core/lib/server/wkPlaywright.js:67:35)
2020-01-24T19:39:52.135813+00:00 app[web.1]:     at WKPlaywright.<anonymous> (/app/node_modules/playwright-core/lib/helper.js:55:31)
2020-01-24T19:39:52.135815+00:00 app[web.1]:     at /app/index.js:5:35
2020-01-24T19:39:52.135823+00:00 app[web.1]:     at Object.<anonymous> (/app/index.js:19:3)
2020-01-24T19:39:52.135830+00:00 app[web.1]:     at Module._compile (internal/modules/cjs/loader.js:955:30)
2020-01-24T19:39:52.135837+00:00 app[web.1]:     at Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10)
2020-01-24T19:39:52.135840+00:00 app[web.1]:     at Module.load (internal/modules/cjs/loader.js:811:32)
2020-01-24T19:39:52.135842+00:00 app[web.1]:     at Function.Module._load (internal/modules/cjs/loader.js:723:14)
2020-01-24T19:39:52.135844+00:00 app[web.1]:     at Function.Module.runMain (internal/modules/cjs/loader.js:1043:10)
2020-01-24T19:39:52.135846+00:00 app[web.1]:   -- ASYNC --
2020-01-24T19:39:52.135848+00:00 app[web.1]:     at WKPlaywright.<anonymous> (/app/node_modules/playwright-core/lib/helper.js:54:23)
2020-01-24T19:39:52.135850+00:00 app[web.1]:     at /app/index.js:5:35
2020-01-24T19:39:52.135852+00:00 app[web.1]:     at Object.<anonymous> (/app/index.js:19:3)
2020-01-24T19:39:52.135853+00:00 app[web.1]:     at Module._compile (internal/modules/cjs/loader.js:955:30)
2020-01-24T19:39:52.135855+00:00 app[web.1]:     at Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10)
2020-01-24T19:39:52.135857+00:00 app[web.1]:     at Module.load (internal/modules/cjs/loader.js:811:32)
2020-01-24T19:39:52.135859+00:00 app[web.1]:     at Function.Module._load (internal/modules/cjs/loader.js:723:14)
2020-01-24T19:39:52.135861+00:00 app[web.1]:     at Function.Module.runMain (internal/modules/cjs/loader.js:1043:10)
2020-01-24T19:39:52.135863+00:00 app[web.1]:     at internal/main/run_main_module.js:17:11

Certainly node_modules dependencies are installed otherwise the error would be different. Any ideas what would be missing?

I will keep debugging, but any hint might be useful.

[JoelEinbinder]

Does firefox or chromium work?

[diervo]

All of them seem to fail.

[diervo]

Seems that the root cause is due to the lazy install that playwright does. Without debbuging further seems that heroku will not allow outbound network request at build time.

I ssh in the dyno and all dependencies and paths seems to be there (playwright-core/.local-chromium/linux-733125/chrome-linux)

So at this point not sure whats going on.

[diervo]

Final update, chrome works, firefox and webkit does not:

My demo repo to test:
https://github.com/diervo/testing-heroku-playwright

Demo test url:
https://infinite-wildwood-37558.herokuapp.com/

[aslushnikov]

@diervo The https://github.com/jontewks/puppeteer-heroku-buildpack is doing some magic, like installing dependencies. However, webkit requires a different set of dependencies to run, e.g. check out our github workflow.

My bet would be that there's a need for a new playwright-heroku-buildpack that adds all the dependencies necessary for all three browsers.

I ssh in the dyno and all dependencies and paths seems to be there (playwright-core/.local-chromium/linux-733125/chrome-linux)

Also, can you check that there's also playwright-core/.local-webkit and playwright-core/.local-firefox folders with binaries? Can you also try running binaries there? (for webkit, please run pw_run.sh --headless)

[pavelfeldman]

Folding this into #581 and #1085

[peter]

@diervo I tried out your testing-heroku-playwright repo and it works awesomely well with Playwright 0.9 but unfortunately it doesn't work with newer versions, i.e. 0.11 and 0.12.1. Currently this is a blocking me from deploying Playwright. I'm hoping to not need to revert back to Puppeteer. Any pointers would be greatly appreciated for deploying Playwright in the cloud, on Heroku or otherwise. The error I get is "Protocol error (Target.setDiscoverTargets): Target closed" cc @aslushnikov

Error: Protocol error (Target.setDiscoverTargets): Target closed.
2020-03-26T13:41:13.423047+00:00 app[web.1]: at /app/node_modules/playwright-core/lib/chromium/crConnection.js:124:63
2020-03-26T13:41:13.423048+00:00 app[web.1]: at new Promise (<anonymous>)
2020-03-26T13:41:13.423049+00:00 app[web.1]: at CRSession.send (/app/node_modules/playwright-core/lib/chromium/crConnection.js:123:16)
2020-03-26T13:41:13.423049+00:00 app[web.1]: at CRSession.<anonymous> (/app/node_modules/playwright-core/lib/helper.js:84:31)
2020-03-26T13:41:13.423049+00:00 app[web.1]: at Function.connect (/app/node_modules/playwright-core/lib/chromium/crBrowser.js:60:21)
2020-03-26T13:41:13.423050+00:00 app[web.1]: at Chromium.launch (/app/node_modules/playwright-core/lib/server/chromium.js:44:53)
2020-03-26T13:41:13.423050+00:00 app[web.1]: at async /app/index.js:23:21 {

[arjunattam]

@peter - the underlying buildpack being used is probably out of date. Specifically, this bit around system packages will need to be updated. The list of updated packages is available in our docker setup. Another option that can be explored is to move away from the buildpack and our docker image directly, since Heroku can work with docker directly.

[peter]

@arjun27 thanks, yeah, both of those suggestions make sense. I tried deploying to Heroku with your Dockerfile but couldn't immediately get that to work. I'm not clear on the interplay between Heroku and Docker and my Docker skills are limited in the first place. As far as the second approach of updating packages in the Buildpack I'm confused though. Your official Dockerfile for Playwright only lists three packages as Chromium dependencies (libnss3, libxss1, and libasound2) and they are all included in the Puppeteer buildpack. I don't need Firefox/Webkit for now, just trying to get Chromium to run. The only thing I see missing is nodejs 12, but I think have that covered in my app as I am using two buildpacks for my app:

1. heroku/nodejs
2. https://github.com/jontewks/puppeteer-heroku-buildpack

[peter]

@arjun27 an interesting point of reference is that my tests run just fine without customization on Travis and they appear to be using Ubuntu 14 (Trusty) whereas Heroku defaults to 18 (Bionic) but Ubuntu 16 is available as an alternative. Not sure if that is relevant here. Unfortunately Travis doesn't fit my use case as they can only schedule jobs to run once a day.

[peter]

Folding this into #581 and #1085

@pavelfeldman I think this issue should be re-opened or should I create a New issue? Playwright not working on Heroku (even with Chromium) isnt really covered by the issues you reference, or at least that has to be verified if that is the case.