Automated large-scale testing of real-world Node.js CLIs

[jviotti]

This is an idea for much later down the road, but writing it here so I don't forget.

Once we have an SEA PoC implementation ready, it would be interesting to mine NPM for i.e. the 100 most popular Node.js CLI packages, and attempt to package them all as SEAs.

We could automate that effort on i.e. some Dockerfiles and have a growing large-scale e2e test setup that exercises our SEA approach with a large number of real CLIs and surfaces any possible thing we are missing

Node.js CLIs we know about

• Playwright driver CLI (https://github.com/microsoft/playwright/blob/main/utils/build/build-playwright-driver.sh)

[mxschmitt]

A bit related to that, should we create a document with a bunch of projects, which would be good to validate if the created APIs / ideas would work for them? Especially projects which either rely on existing single executable software or projects which would like to use it.

[jviotti]

I'm all for that! What are the ones you know about? We can start the list in this discussion and then convert it into a PR

[mxschmitt]

I'm not familiar with others, but we at Playwright use it for bundling our driver (used for our language ports).

The driver is essentially Node.js executable, JavaScript files, web assets, some scripts (ps1, shell scripts). Before we also embedded node_modules folder into it, but it ended up being a lot of files (5+k). Now we bundle them with esbuild and have around 278 files, which includes 2 web-applications which could be served "in memory" since we provide a web-server for that, and some shell scripts, which could be copied either on-demand from memory or being piped into bash/PowerShell.

We could put some structure into it, aka. e.g. Repository URL, Bundled Content, Challenges, Operating Systems

[jviotti]

We could put some structure into it, aka. e.g. Repository URL, Bundled Content, Challenges, Operating Systems

That'd be great. To start with, I at least created a list on my original message in this discussion.

Do you have a link to the driver CLI code?

[mxschmitt]

https://github.com/microsoft/playwright/blob/main/utils/build/build-playwright-driver.sh

[jviotti]

I sent a PR collecting these: #26