Epic: Enable secure, friction-less & reproducible bug reports

[lukekarrys]

We want to accomplish two things:

1. Not make it necessary to run code on local machines to triage bug reports
2. Lower the friction for reporters (and maintainers) to create fully encapsulated and reproducible bug reports

Ideas

• A workspace or separate repo that has the following:
  • Template repository (or just forkable)
  • Minimal tools and templated files for common tasks (managing node versions, npm versions, asserting behavior)
    • package.json
    • nvm
    • tap
    • run npm from source via a registry tarball
  • devcontainer.json for codespace settings
  • Github action workflows that run CI for requested os/version matrix
• Update bug report with instructions for creating VM/codespace
  • Prior art statelyai/xstate@c9e83c0#diff-637f7b97bba458badb691a1557c3d4648686292e948dbe3e8360564378b653efR40-R47
• Encapsulate all this in an npx-able bin (npx @npmcli/bug-report --npm 8.5.3 --node 16 ?)

Caveats

• URL rot. It would be nice if any links to reproductions in issues would not 404
• Codespaces require admin rights to repo or a PAT with a codespace scope. Offloading this to non-member reporters might not be possible
  • https://docs.github.com/en/billing/managing-billing-for-github-codespaces/about-billing-for-codespaces#how-billing-is-handled-for-forked-repositories

[nlf]

another thing to note here is that codespaces are linux only. virtual machines are nearly trivial for linux guests too. windows and osx, on the other hand, are one or more of: surprisingly slow, annoying to setup, requires the host os to be the same as the guest.

i wonder if we could use github actions for this?

[lukekarrys]

another thing to note here is that codespaces are linux only...i wonder if we could use github actions for this?

that's what i was thinking too, and was attempting to describe above. i don't see codespaces as strictly necessary, but just as the easiest way to go from browser -> running the cli. then whatever gets created in a codespace would be run via a github action that would be similar to our ci.yml but with some extra debugging steps like cat debug.log, etc

[ljharb]

Also repl.it might be useful (or runkit, if it offered a way to prepopulate the code snippet)