Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: use esbuild-plugins-node-modules-polyfill #3832

Conversation

MichaelDeBoey
Copy link

@MichaelDeBoey MichaelDeBoey commented Aug 24, 2023

This PR replaces @esbuild-plugins/node-globals-polyfill & @esbuild-plugins/node-modules-polyfill with the up-to-date & maintained esbuild-plugins-node-modules-polyfill

The esbuild-plugins repo itself points towards using esbuild-plugin-polyfill-node instead
https://github.com/remorses/esbuild-plugins/blob/373b44902ad3e669f7359c857de09a930ce1ce90/README.md?plain=1#L15-L16

After doing this in the Remix repo (see remix-run/remix#5274), we got quite some new bugs, so we chose to go for @imranbarbhuiya's esbuild-plugins-node-modules-polyfill instead (see remix-run/remix#6562), which is an up-to-date and well maintained alternative

Added benefit is that we won't get the following deprecation warnings when installing @esbuild-plugins/node-modules-polyfill:

npm WARN deprecated rollup-plugin-inject@3.0.2: This package has been deprecated and is no longer maintained. Please use @rollup/plugin-inject.
npm WARN deprecated sourcemap-codec@1.4.8: Please use @jridgewell/sourcemap-codec instead

Resolves #1232

@MichaelDeBoey MichaelDeBoey requested review from a team as code owners August 24, 2023 17:38
@changeset-bot
Copy link

changeset-bot bot commented Aug 24, 2023

⚠️ No Changeset found

Latest commit: defb3d5

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions
Copy link
Contributor

github-actions bot commented Aug 29, 2023

A wrangler prerelease is available for testing. You can install this latest build in your project with:

npm install --save-dev https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/8279466977/npm-package-wrangler-3832

You can reference the automatically updated head of this PR with:

npm install --save-dev https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/prs/3832/npm-package-wrangler-3832

Or you can use npx with this latest build directly:

npx https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/8279466977/npm-package-wrangler-3832 dev path/to/script.js
Additional artifacts:
npx https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/8279466977/npm-package-create-cloudflare-3832 --no-auto-update
npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/8279466977/npm-package-cloudflare-kv-asset-handler-3832
npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/8279466977/npm-package-miniflare-3832
npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/8279466977/npm-package-cloudflare-pages-shared-3832
npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/8279466977/npm-package-cloudflare-vitest-pool-workers-3832

Note that these links will no longer work once the GitHub Actions artifact expires.


wrangler@3.34.0 includes the following runtime dependencies:

Package Constraint Resolved
miniflare workspace:* 3.20240304.1
workerd 1.20240304.0 1.20240304.0
workerd --version 1.20240304.0 2024-03-04

Please ensure constraints are pinned, and miniflare/workerd minor versions match.

@codecov
Copy link

codecov bot commented Aug 29, 2023

Codecov Report

Attention: Patch coverage is 92.77778% with 39 lines in your changes are missing coverage. Please review.

Project coverage is 71.48%. Comparing base (84eeee5) to head (38e08fc).
Report is 4 commits behind head on v4.

❗ Current head 38e08fc differs from pull request most recent head 9da3533. Consider uploading reports for the commit 9da3533 to get more accurate results

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##               v4    #3832      +/-   ##
==========================================
+ Coverage   70.35%   71.48%   +1.13%     
==========================================
  Files         298      309      +11     
  Lines       15567    16083     +516     
  Branches     4007     4103      +96     
==========================================
+ Hits        10952    11497     +545     
+ Misses       4615     4586      -29     
Files Coverage Δ
packages/wrangler/e2e/helpers/normalize.ts 93.75% <100.00%> (ø)
...angler/src/__tests__/helpers/collect-cli-output.ts 100.00% <100.00%> (ø)
...ler/src/__tests__/helpers/msw/handlers/versions.ts 100.00% <100.00%> (ø)
...ckages/wrangler/src/__tests__/helpers/msw/index.ts 100.00% <100.00%> (ø)
packages/wrangler/src/d1/migrations/apply.tsx 60.63% <100.00%> (+1.06%) ⬆️
packages/wrangler/src/d1/migrations/create.tsx 94.11% <ø> (ø)
packages/wrangler/src/d1/migrations/helpers.ts 94.54% <100.00%> (-0.20%) ⬇️
packages/wrangler/src/d1/migrations/list.tsx 71.79% <100.00%> (+0.74%) ⬆️
packages/wrangler/src/d1/migrations/options.ts 100.00% <ø> (ø)
packages/wrangler/src/deploy/index.ts 95.74% <100.00%> (-1.03%) ⬇️
... and 16 more

... and 9 files with indirect coverage changes

@MichaelDeBoey MichaelDeBoey force-pushed the use-esbuild-plugins-node-modules-polyfill branch from f2c0092 to 5e2436c Compare August 30, 2023 12:49
@GregBrimble GregBrimble mentioned this pull request Aug 30, 2023
2 tasks
@MichaelDeBoey
Copy link
Author

Failing CI seems to be unrelated

@mrbbot
Copy link
Contributor

mrbbot commented Sep 8, 2023

Hey! 👋 Thanks for this PR! We discussed this as a team, and I think we're going to hold off merging this until we're ready to do a major release: #1232 (comment). We're also going to do an audit of these polyfills and our natively supported Node builtins added with the nodejs_compat compatibility flag, to see how many differences there are. This will inform whether we still need to provide a Node polyfilling solution in Wrangler, or can just rely on native support from the Workers runtime.

@MichaelDeBoey
Copy link
Author

@mrbbot It would be nice if we could drop the polyfills, but I don't think it hurts to switch to a more up-to-date & maintained version for now either 🤷‍♂️

@workers-devprod
Copy link
Contributor

hey @MichaelDeBoey :) definitely agree it's good to stay up to date in the meantime. however per #1232 (comment), we believe this represents a breaking change and as such will hold off for now. as @mrbbot noted, we plan to audit the polyfills -- if this update is still necessary we'll schedule it as part of the next major release 👍

@petebacondarwin
Copy link
Contributor

These new polyfills are higher fidelity than the old ones, which causes the Cloudflare socket detection to fail in the node-postgres library. I have created a fix there to tighten that detection up - brianc/node-postgres#3170

But we will need that fix to go hand in hand with landing this PR.

@imranbarbhuiya
Copy link

imranbarbhuiya commented Mar 13, 2024

These new polyfills are higher fidelity than the old ones, which causes the Cloudflare socket detection to fail in the node-postgres library. I have created a fix there to tighten that detection up - brianc/node-postgres#3170

But we will need that fix to go hand in hand with landing this PR.

if you want to exclude net, tls module then you can pass builtinModules.filter(a => !['net', 'tls'].includes(a)) to https://github.com/imranbarbhuiya/esbuild-plugins-node-modules-polyfill#configure-which-modules-to-polyfill

@petebacondarwin
Copy link
Contributor

These new polyfills are higher fidelity than the old ones, which causes the Cloudflare socket detection to fail in the node-postgres library. I have created a fix there to tighten that detection up - brianc/node-postgres#3170
But we will need that fix to go hand in hand with landing this PR.

if you want to exclude net module then you can pass builtinModules.filter(a => a==='net') to https://github.com/imranbarbhuiya/esbuild-plugins-node-modules-polyfill#configure-which-modules-to-polyfill

That sounds like a good solution for now. Thanks

@petebacondarwin petebacondarwin force-pushed the use-esbuild-plugins-node-modules-polyfill branch from fc5895a to abd6eee Compare March 13, 2024 19:13
@MichaelDeBoey MichaelDeBoey force-pushed the use-esbuild-plugins-node-modules-polyfill branch 2 times, most recently from 79f36c0 to 38e08fc Compare March 14, 2024 10:52
@petebacondarwin
Copy link
Contributor

Emptying the net package polyfill resolves the problem with the pg-cloudflare usage.

Here is a link to other projects that might rely upon these polyfills: https://github.com/search?q=node_compat+%3D+true+language%3ATOML&type=code&l=TOML

We should contact these projects and ask them to test out the prerelease of v4.

@petebacondarwin
Copy link
Contributor

(There was no need to avoid polyfilling the tls library because the connect function in that polyfill is left unimplemented: https://github.com/jspm/jspm-core/blob/7af7d7413f472305d08d0d78ec3d1f15588be50a/nodelibs/browser/tls.js#L34).

@petebacondarwin petebacondarwin changed the base branch from main to v4 March 16, 2024 07:53
@petebacondarwin petebacondarwin requested review from a team as code owners March 16, 2024 07:53
Copy link

gitguardian bot commented Mar 16, 2024

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
9997298 Triggered Generic Database Assignment bd935cf packages/wrangler/src/tests/hyperdrive.test.ts View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secret safely. Learn here the best practices.
  3. Revoke and rotate this secret.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Our GitHub checks need improvements? Share your feedbacks!

@petebacondarwin petebacondarwin force-pushed the use-esbuild-plugins-node-modules-polyfill branch from 9da3533 to defb3d5 Compare March 16, 2024 11:51
@petebacondarwin
Copy link
Contributor

Oops - sorry. I accidentally closed this by pushing the wrong commits when rebasing.
Will open a new PR.

@MichaelDeBoey MichaelDeBoey deleted the use-esbuild-plugins-node-modules-polyfill branch June 3, 2024 11:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change Change that will result in breaking existing behavior
Projects
None yet
Development

Successfully merging this pull request may close these issues.

maintenance: wrangler depends on deprecated npm package rollup-plugin-inject
5 participants