Intercept workerd logs + write all debug logs to a hidden file

[RamIdeas]

Fixes #3631

What this PR solves / how to test:

• fixes a bug where Miniflare's Log class rethrew the error when the log level meant the error wouldn't be logged
• exposes a new Miniflare option: handleRuntimeStdio which allows intercepting workerd stdout/stderr
• wrangler now intercepts workerd logs using the above option
• wrangler now writes all debug logs to .wrangler/debug-logs/wrangler-debug-<TIMESTAMP>.log (superseding this PR)

Implementation notes:

• Logs with log level debug are now never sent to the terminal so the terminal remains clean but the extra observability is provided via the log file
• The default debug log filepath can be overridden using the new WRANGLER_LOG_DIR env var
  • which allows you to specify a directory path or a specific .log filepath
  • the default value for this env var is <package>/.wrangler/debug-logs
  • the filename will contain a timestamp, unless the WRANGLER_LOG_DIR value ends in .log
• The e2e test workflow will upload the debug-log file as an artefact for inspection upon failures (primarily, but also available on successful runs too)
• The logger level debug is what triggers the use of the log file but all logs are appended to the log file so that the order amongst other log-level logs (info, error, warn, etc) is preserved

Exposing a handleRuntimeStdio option, instead of hooks for each readline interface of the current implementation, should also satisfy #4357

Author has addressed the following:

• Tests
  • Included
  • Not necessary because:
• Changeset (Changeset guidelines)
  • Included
  • Not necessary because:
• Associated docs
  • Issue(s)/PR(s):
  • Not necessary because:

Note for PR author:

We want to celebrate and highlight awesome PR review! If you think this PR received a particularly high-caliber review, please assign it the label highlight pr review so future reviewers can take inspiration and learn from it.

[changeset-bot]

🦋 Changeset detected

Latest commit: 664e4a4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
miniflare	Minor
wrangler	Minor
@cloudflare/pages-shared	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[github-actions]

A wrangler prerelease is available for testing. You can install this latest build in your project with:

npm install --save-dev https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/6932507257/npm-package-wrangler-4341

You can reference the automatically updated head of this PR with:

npm install --save-dev https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/prs/6932507257/npm-package-wrangler-4341

Or you can use npx with this latest build directly:

npx https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/6932507257/npm-package-wrangler-4341 dev path/to/script.js

Additional artifacts:

npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/6932507257/npm-package-miniflare-4341

npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/6932507257/npm-package-cloudflare-pages-shared-4341

Note that these links will no longer work once the GitHub Actions artifact expires.

---

wrangler@3.16.0 includes the following runtime dependencies:

Package	Constraint	Resolved
miniflare	workspace:*	3.20231030.0
workerd	1.20231030.0	1.20231030.0
workerd --version	1.20231030.0	2023-10-30

|

Please ensure constraints are pinned, and miniflare/workerd minor versions match.

[admah]

@RamIdeas are users notified somewhere that this new file exists?

[mrbbot]

Miniflare changes look good! I've added some comments to some of the Wrangler ones... 👍

[codecov]

Codecov Report

Merging #4341 (664e4a4) into main (1747d21) will decrease coverage by 0.24%.
Report is 1 commits behind head on main.
The diff coverage is 36.48%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4341      +/-   ##
==========================================
- Coverage   75.46%   75.23%   -0.24%     
==========================================
  Files         225      226       +1     
  Lines       12482    12555      +73     
  Branches     3242     3256      +14     
==========================================
+ Hits         9420     9446      +26     
- Misses       3062     3109      +47     

Files	Coverage Δ
...ages/wrangler/src/environment-variables/factory.ts	71.42% <ø> (ø)
packages/wrangler/src/logger.ts	98.27% <83.33%> (-1.73%)	⬇️
packages/wrangler/src/dev/miniflare.ts	55.20% <17.39%> (-5.15%)	⬇️
packages/wrangler/src/utils/debug-log-file.ts	40.00% <40.00%> (ø)

[mrbbot]

Is this function used anywhere?