[BUG] Google Cloud Function error (deploy)

[MaiaVinicius]

Context:

• Operating System: Linux
• Node version: Node.js 10
• Browser: Webkit
• Extra: [any specific details about your environment]

Code Snippet

const {chromium, webkit, firefox} = require('playwright');

(async () => {
      const browser = await webkit.launch({ headless: true });
      const context = await browser.newContext();
      const page = await context.newPage();
})();

Describe the bug

I'm trying to deploy my playwright application in Google Cloud Functions. It seems like there is a problem to spawn the browser.

Error: function terminated. Recommended action: inspect logs for termination reason. Details:
Failed to launch browser: Error: spawn /root/.cache/ms-playwright/chromium-764964/chrome-linux/chrome ENOENT

[aslushnikov]

@MaiaVinicius Google Cloud Functions environment is missing the needed libraries for webkit. And IIRC there's no way to add missing dependencies to Google Cloud from the user land :( The only hope is to submit a feature request to google cloud and hope they'll fix them.

If you end up filing this to Google, please post a link to the filed bug here for the future reference.

[aslushnikov]

Oh, wait!

Browser: Webkit

@MaiaVinicius I see you try launching webkit, but your error log is mentioning chrome:

Failed to launch browser: Error: spawn /root/.cache/ms-playwright/chromium-764964/chrome-linux/chrome ENOENT

Is this log correct?

[MaiaVinicius]

Actually, I have tried all 3 drivers (chrome, firefox and webkit). All returned the same error.

Thank you for the feedback.

@aslushnikov

[gluck]

Got this issue as well, I'm assuming that either firebase doesn't run install scripts, or doesn't wait for it to finish (install script is async, but nothing's waiting for it AFAICT).
Got it working by either:

• executing installBrowsersWithProgressBar within the function (and waiting for it), but it'll consume function bandwidth $$$
• using chrome-aws-lambda package, which bundles chrome linux binary in the npm package itself, and tricking playwright into using it

Although both of these are far from ideal 😿

[aslushnikov]

Got this issue as well, I'm assuming that either firebase doesn't run install scripts, or doesn't wait for it to finish (install script is async, but nothing's waiting for it AFAICT).

@gluck 🤷‍♂️ we're not very familiar with firebase, unfortunately...

I'll close this since it's unclear how we can help here.

[billfeng]

I'm getting this issue as well. Is it possible to add an option to bundle the browser being used as a part of the NPM package, instead of using the browser on-device?

[gluck]

Hi @billfeng, since my comment the situation has improved (a bit) and you can use the package 'playwright-core' to avoid downloading the browsers and chrome-aws-lambda package can be used to get a npm-packaged-chromium.
Two drawbacks with that approach:

• you need to match playwright version with chrome-aws-lambda so that chromium version is the expected one
• chrome-aws-lambda only works on cloud, and running locally won't work

Now that playwright has already tons of packaged ways (playwright-core, playwright-chromium ...), having another one like playwright-chromium-cloud would be ideal indeed.

[billfeng]

Hi @gluck, thanks for the info, this is very helpful! I will try the method you described. Quick questions -

• Do the chromium versions have to match exactly? The latest from chrome-aws-lambda is 856583, but the one that playwright installs is a close but different number, and there isn't a chrome-aws-lambda release with the exact match.
• How did you get playwright-core to use the chromium binary from chrome-aws-lambda?

Also, I made some interesting hacky progress with a different method. By forcing Firebase cloud functions to install the browser binaries locally through running PLAYWRIGHT_BROWSERS_PATH=0 npm i playwright as a postinstall script, and then setting executablePath to the browser binaries in node_modules, it seems to almost work, except then I get this different error:

error while loading shared libraries: libharfbuzz-icu.so.0: cannot open shared object file: No such file or directory

I'm guessing playwright or the browser binary it installed needs this libharfbuzz-icu.so.0 thing? Is this a Linux package that the Ubuntu container of Firebase functions is missing? If this looks familiar to you at all or if you have any suggestions on how to get around this, that would be much appreciated too. I feel like I'm so close to getting it to work!

[gluck]

I'm currently using:

• playwright 1.6.2 (chromium 88.0.4324.0)
• chrome-aws-lambda 5.5.0 (chromium 88.0.4298.0)

Without issues (I had issues when the major version was different).

Pseudo code:

import { chromium } from 'playwright-core';
import * as bundledChromium from 'chrome-aws-lambda';

....
  const browser = await Promise.resolve(bundledChromium.executablePath).then(
      (executablePath) => {
        if (!executablePath) {
          // local execution
          return chromium.launch({});
        }
        return chromium.launch({ executablePath });
      }
    );

Your approach seems promising as well, not sure why the shared library is missing.

[tverilytt]

This seems to work well now (I have only tested with playwright-chromium).

Only initial issue was that the launch method did not find the executable:

browserType.launch: Failed to launch chromium because executable doesn't exist at /root/.cache/ms-playwright/chromium-888113/chrome-linux/chrome

Setting the environment variable PLAYWRIGHT_BROWSERS_PATH=0 resolved this, and I can now use playwright-chromium in Google Cloud Function to e.g. generate pdf 😃

Cheers
-jo

BTW - had to raise memory limit to 1 GB to avoid running out of memory...

[pawelmhm]

For some reasons just using PLAYWRIGHT_BROWSERS_PATH=0 didn't work for me here.

What worked for me when deploying Playwright to Google Cloud Function.

1. Passing executablePath option to browser.launch, example

 // tmp is folder where browser binary should be found, 1295 is browser version and it can change
  const context = await browser.newContext({"executablePath":  "tmp/firefox-1295/firefox/firefox"})

2. Passing environment variable to GCF PLAYWRIGHT_BROWSERS_PATH=<some_folder>, in my case this was tmp, passed from gcloud command line tool via flag --set-build-env-vars=PLAYWRIGHT_BROWSERS_PATH=tmp but it can also bet set via Web interface.

[tverilytt]

HI! Hm, ok. I set PLAYWRIGHT_BROWSERS_PATH for both build and runtime environment, e.g. for gcloud CLI:

--set-build-env-vars="PLAYWRIGHT_BROWSERS_PATH=0" --set-env-vars="PLAYWRIGHT_BROWSERS_PATH=0"

[pawelmhm]

thanks for info, I'll double check it again and verify

[jdexyz]

I'm currently using:

* playwright 1.6.2 (chromium 88.0.4324.0)

* chrome-aws-lambda 5.5.0 (chromium 88.0.4298.0)

Without issues (I had issues when the major version was different).

Pseudo code:

import { chromium } from 'playwright-core';
import * as bundledChromium from 'chrome-aws-lambda';

....
  const browser = await Promise.resolve(bundledChromium.executablePath).then(
      (executablePath) => {
        if (!executablePath) {
          // local execution
          return chromium.launch({});
        }
        return chromium.launch({ executablePath });
      }
    );

Your approach seems promising as well, not sure why the shared library is missing.

I can confirm that the above solution works well. Thank you for sharing this !

Here is a complete example that returns a screenshot of google when queried.
(Using NodeJS 14, 2GB RAM, latest chrome-aws-lambda version)

package.json

{
  "dependencies": {
        "chrome-aws-lambda": "10.1.0",
        "playwright-core": "1.14.1"
  }
}

index.js

const { chromium } = require('playwright-core');
const bundledChromium = require('chrome-aws-lambda');

module.exports.screenshot = async function (req, res) {
    const url = "https://google.com/";
    
  const browser = await Promise.resolve(bundledChromium.executablePath).then(
      (executablePath) => {
        if (!executablePath) {
          // local execution
          return chromium.launch({});
        }
        return chromium.launch({ executablePath });
      }
    );
    const page = await browser.newPage();
    await page.goto(url);
    const screenshotBuffer = await page.screenshot({ fullPage: true });
    await browser.close();
    res.set("Content-Type", "image/png");
    res.status(500).send(screenshotBuffer)        
};

[mcm-gr]

Could you also run this setup in the local firebase function emulator? Best, MM

[MuhammadM1998]

I've tried to do what @jdexyz suggested and got the following error when deploying to vercel

Function Status:500
Edge Status:500
Duration:4514.31 ms
Init Duration: 184.41 ms
Memory Used:462 MB
ID:fra1::iad1::kj8sg-1669238499279-b87cc4c1dc5a
User Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0

: Protocol error (Page.enable): Browser closed.
==================== Browser output: ====================
[pid=23][err] #30 0x55996e66e8ac (/tmp/chromium+0x54008ab)
[pid=23][err] #31 0x7f998e5be44b start_thread
[pid=23][err] #32 0x7f998db6756f __GI___clone
[pid=23][err] [pid=23][err] Received signal 6
[pid=23][err] #0 0x55996e65efc9 (/tmp/chromium+0x53f0fc8)
[pid=23][err] #1 0x55996e5daf33 (/tmp/chromium+0x536cf32)
[pid=23][err] #2 0x55996e65eaf1 (/tmp/chromium+0x53f0af0)
[pid=23][err] #3 0x7f998e5c88e0 (/usr/lib64/libpthread-2.26.so+0x118df)
[pid=23][err] #4 0x7f998daabca0 __GI_raise
[pid=23][err] #5 0x7f998daad148 __GI_abort
[pid=23][err] #6 0x55996e65ddb5 (/tmp/chromium+0x53efdb4)
[pid=23][err] #7 0x55996e5ec227 (/tmp/chromium+0x537e226)
[pid=23][err] #8 0x55996d27a6c7 (/tmp/chromium+0x400c6c6)
[pid=23][err] #9 0x55996d27807e (/tmp/chromium+0x400a07d)
[pid=23][err] #10 0x55996d276d03 (/tmp/chromium+0x4008d02)
[pid=23][err] #11 0x55996d2806f2 (/tmp/chromium+0x40126f1)
[pid=23][err] #12 0x55996d2822e4 (/tmp/chromium+0x40142e3)
[pid=23][err] #13 0x55996d10d5ec (/tmp/chromium+0x3e9f5eb)
[pid=23][err] #14 0x55996eefc653 (/tmp/chromium+0x5c8e652)
[pid=23][err] #15 0x55996eed4e7f (/tmp/chromium+0x5c66e7e)
[pid=23][err] #16 0x55996ef00963 (/tmp/chromium+0x5c92962)
[pid=23][err] #17 0x55996ef01ced (/tmp/chromium+0x5c93cec)
[pid=23][err] #18 0x55996eed1a09 (/tmp/chromium+0x5c63a08)
[pid=23][err] #19 0x55996eeea65e (/tmp/chromium+0x5c7c65d)
[pid=23][err] #20 0x55996eeeaa0a (/tmp/chromium+0x5c7ca09)
[pid=23][err] #21 0x55996eee9ec0 (/tmp/chromium+0x5c7bebf)
[pid=23][err] #22 0x55996d716ebb (/tmp/chromium+0x44a8eba)
[pid=23][err] #23 0x55996d716790 (/tmp/chromium+0x44a878f)
[pid=23][err] #24 0x55996d71303f (/tmp/chromium+0x44a503e)
[pid=23][err] #25 0x55996d70a53d (/tmp/chromium+0x449c53c)
[pid=23][err] #26 0x55996d71bf47 (/tmp/chromium+0x44adf46)
[pid=23][err] #27 0x55996e68c040 (/tmp/chromium+0x541e03f)
[pid=23][err] #28 0x55996e7c388d (/tmp/chromium+0x555588c)
[pid=23][err] #29 0x55996e68c2b3 (/tmp/chromium+0x541e2b2)
[pid=23][err] #30 0x55996e636045 (/tmp/chromium+0x53c8044)
[pid=23][err] #31 0x55996e60f5fb (/tmp/chromium+0x53a15fa)
[pid=23][err] #32 0x55996e64c878 (/tmp/chromium+0x53de877)
[pid=23][err] #33 0x55996d1293b0 (/tmp/chromium+0x3ebb3af)
[pid=23][err] #34 0x55996e64ca02 (/tmp/chromium+0x53dea01)
[pid=23][err] #35 0x55996e66e8ac (/tmp/chromium+0x54008ab)
[pid=23][err] #36 0x7f998e5be44b start_thread
[pid=23][err] #37 0x7f998db6756f __GI___clone
[pid=23][err]   r8: 0000000000000000  r9: 00007f998488fb00 r10: 0000000000000008 r11: 0000000000000246
[pid=23][err]  r12: 00007f9984890dc0 r13: 00007f998488fd60 r14: 00007f9984890dd0 r15: aaaaaaaaaaaaaaaa
[pid=23][err]   di: 0000000000000002  si: 00007f998488fb00  bp: 00007f998488fd50  bx: 0000000000000006
[pid=23][err]   dx: 0000000000000000  ax: 0000000000000000  cx: 00007f998daabca0  sp: 00007f998488fb00
[pid=23][err]   ip: 00007f998daabca0 efl: 0000000000000246 cgf: 002b000000000033 erf: 0000000000000000
[pid=23][err]  trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[pid=23][err] [end of stack trace]
    at captureStackTrace (/var/task/node_modules/playwright-core/lib/utils/stackTrace.js:60:17)
    at BrowserContext._wrapApiCall (/var/task/node_modules/playwright-core/lib/client/channelOwner.js:101:58)
    at BrowserContext.newPage (/var/task/node_modules/playwright-core/lib/client/browserContext.js:237:17)
    at Browser.newPage (/var/task/node_modules/playwright-core/lib/client/browser.js:97:32)
    at async playwright (file:///var/task/api/playwright.js:23:18)
    at async Server.<anonymous> (/var/task/___vc/__helpers.js:813:13) {
  name: 'Error'
}

[2dareis2do]

I had a similar issue with running playwright python. I was wondering why I could run from the command line but it was failing when running as a web user process.

Turns out the path for browsers are stored in ~/.cache folder when running playwright install command

For me the solution was to make sure that I ran playwright install command as the web user.