Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build packages with tsup #2120

Merged
merged 18 commits into from
Feb 22, 2024
Merged

Build packages with tsup #2120

merged 18 commits into from
Feb 22, 2024

Conversation

Mrtenz
Copy link
Member

@Mrtenz Mrtenz commented Jan 22, 2024

This changes all packages to be built with tsup, instead of SWC. tsup uses esbuild under the hood, so performance should be comparable.

More context here: MetaMask/utils#144.

Copy link

socket-security bot commented Jan 22, 2024

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package New capabilities Transitives Size Publisher
npm/@esbuild/aix-ppc64@0.19.12 None 0 10.4 MB evanw
npm/@esbuild/android-arm@0.19.12 None 0 11.5 MB evanw
npm/@esbuild/android-arm64@0.19.12 None 0 9.83 MB evanw
npm/@esbuild/android-x64@0.19.12 None 0 11.5 MB evanw
npm/@esbuild/darwin-arm64@0.19.12 None 0 9.76 MB evanw
npm/@esbuild/darwin-x64@0.19.12 None 0 10.2 MB evanw
npm/@esbuild/freebsd-arm64@0.19.12 None 0 8.85 MB evanw
npm/@esbuild/freebsd-x64@0.19.12 None 0 9.61 MB evanw
npm/@esbuild/linux-arm@0.19.12 None 0 9.18 MB evanw
npm/@esbuild/linux-arm64@0.19.12 None 0 8.85 MB evanw
npm/@esbuild/linux-ia32@0.19.12 None 0 9.14 MB evanw
npm/@esbuild/linux-loong64@0.19.12 None 0 9.37 MB evanw
npm/@esbuild/linux-mips64el@0.19.12 None 0 10.4 MB evanw
npm/@esbuild/linux-ppc64@0.19.12 None 0 9.18 MB evanw
npm/@esbuild/linux-riscv64@0.19.12 None 0 9.11 MB evanw
npm/@esbuild/linux-s390x@0.19.12 None 0 10 MB evanw
npm/@esbuild/linux-x64@0.19.12 None 0 9.6 MB evanw
npm/@esbuild/netbsd-x64@0.19.12 None 0 9.59 MB evanw
npm/@esbuild/openbsd-x64@0.19.12 None 0 9.62 MB evanw
npm/@esbuild/sunos-x64@0.19.12 None 0 9.59 MB evanw
npm/@esbuild/win32-arm64@0.19.12 None 0 8.98 MB evanw
npm/@esbuild/win32-ia32@0.19.12 None 0 9.45 MB evanw
npm/@esbuild/win32-x64@0.19.12 None 0 9.81 MB evanw
npm/@rollup/rollup-android-arm-eabi@4.12.0 None 0 1.63 MB lukastaegert
npm/@rollup/rollup-android-arm64@4.12.0 None 0 2.49 MB lukastaegert
npm/@rollup/rollup-darwin-arm64@4.12.0 None 0 2.41 MB lukastaegert
npm/@rollup/rollup-darwin-x64@4.12.0 None 0 2.58 MB lukastaegert
npm/@rollup/rollup-linux-arm-gnueabihf@4.12.0 None 0 2.43 MB lukastaegert
npm/@rollup/rollup-linux-arm64-gnu@4.12.0 None 0 2.47 MB lukastaegert
npm/@rollup/rollup-linux-arm64-musl@4.12.0 None 0 2.37 MB lukastaegert
npm/@rollup/rollup-linux-riscv64-gnu@4.12.0 None 0 2.54 MB lukastaegert
npm/@rollup/rollup-linux-x64-gnu@4.12.0 None 0 2.73 MB lukastaegert
npm/@rollup/rollup-linux-x64-musl@4.12.0 None 0 2.73 MB lukastaegert
npm/@rollup/rollup-win32-arm64-msvc@4.12.0 None 0 2.93 MB lukastaegert
npm/@rollup/rollup-win32-ia32-msvc@4.12.0 None 0 2.64 MB lukastaegert
npm/@rollup/rollup-win32-x64-msvc@4.12.0 None 0 3.44 MB lukastaegert
npm/@sindresorhus/is@5.3.0 None 0 58.9 kB sindresorhus
npm/any-promise@1.3.0 None 0 22.2 kB kevinbeaty
npm/bundle-require@4.0.2 environment, filesystem, unsafe Transitive: network, shell +25 224 MB egoist
npm/cac@6.7.14 None 0 81.8 kB egoist
npm/chokidar@3.6.0 environment, filesystem +14 531 kB paulmillr
npm/esbuild@0.19.12 environment, filesystem, network, shell +23 224 MB evanw
npm/joycon@3.1.1 environment, filesystem 0 14 kB egoist
npm/load-tsconfig@0.2.5 None 0 17.1 kB egoist
npm/lodash.sortby@4.7.0 None 0 75.8 kB jdalton
npm/mz@2.7.0 filesystem, network, shell +4 52 kB jongleberry
npm/postcss-load-config@4.0.2 environment, unsafe Transitive: filesystem, shell +38 466 MB ai
npm/rollup@4.12.0 environment, filesystem +15 35.8 MB lukastaegert
npm/sucrase@3.35.0 Transitive: environment, filesystem, network, shell, unsafe +38 3.74 MB alangpierce
npm/thenify-all@1.6.0 None +2 36.7 kB dead_horse
npm/thenify@3.3.1 None +1 30.1 kB dead_horse
npm/tr46@1.0.1 None +1 356 kB sebmaster
npm/tree-kill@1.2.2 shell 0 7.82 kB wmhilton
npm/ts-interface-checker@0.1.13 None 0 68 kB dsagal2
npm/tsup@8.0.2 environment, eval, filesystem Transitive: network, shell, unsafe +169 731 MB egoist
npm/webidl-conversions@4.0.2 None 0 19.3 kB domenic
npm/whatwg-url@7.1.0 None +4 529 kB domenic

🚮 Removed packages: npm/@esbuild/android-arm64@0.20.1, npm/@esbuild/android-arm@0.20.1, npm/@esbuild/android-x64@0.20.1, npm/@esbuild/darwin-arm64@0.20.1, npm/@esbuild/darwin-x64@0.20.1, npm/@esbuild/freebsd-arm64@0.20.1, npm/@esbuild/freebsd-x64@0.20.1, npm/@esbuild/linux-arm64@0.20.1, npm/@esbuild/linux-arm@0.20.1, npm/@esbuild/linux-ia32@0.20.1, npm/@esbuild/linux-loong64@0.20.1, npm/@esbuild/linux-mips64el@0.20.1, npm/@esbuild/linux-ppc64@0.20.1, npm/@esbuild/linux-riscv64@0.20.1, npm/@esbuild/linux-s390x@0.20.1, npm/@esbuild/linux-x64@0.20.1, npm/@esbuild/netbsd-x64@0.20.1, npm/@esbuild/openbsd-x64@0.20.1, npm/@esbuild/sunos-x64@0.20.1, npm/@esbuild/win32-arm64@0.20.1, npm/@esbuild/win32-ia32@0.20.1, npm/@esbuild/win32-x64@0.20.1, npm/@mole-inc/bin-wrapper@8.0.1, npm/@sindresorhus/is@4.6.0, npm/@swc/cli@0.1.65, npm/@szmarczak/http-timer@4.0.6, npm/@tokenizer/token@0.3.0, npm/arch@2.2.0, npm/bin-check@4.1.0, npm/bin-version-check@5.1.0, npm/bin-version@6.0.0, npm/cacheable-lookup@5.0.4, npm/cacheable-request@7.0.4, npm/commander@7.2.0, npm/esbuild@0.20.1, npm/executable@4.1.1, npm/ext-list@2.2.2, npm/ext-name@5.0.0, npm/file-type@17.1.6, npm/filename-reserved-regex@3.0.0, npm/filenamify@5.1.1, npm/find-versions@5.1.0, npm/got@11.8.6, npm/http2-wrapper@1.0.3, npm/is-plain-obj@1.1.0, npm/lowercase-keys@2.0.0, npm/normalize-url@6.1.0, npm/os-filter-obj@2.0.0, npm/p-cancelable@2.1.1, npm/p-finally@1.0.0, npm/peek-readable@5.0.0, npm/responselike@2.0.1, npm/semver-regex@4.0.5, npm/semver-truncate@3.0.0, npm/shebang-command@1.2.0, npm/sort-keys-length@1.0.1, npm/sort-keys@1.1.2, npm/strip-eof@1.0.0, npm/strip-outer@2.0.0, npm/strtok3@7.0.0, npm/token-types@5.0.1, npm/trim-repeated@2.0.0

View full report↗︎

Copy link

socket-security bot commented Jan 22, 2024

👍 Dependency issues cleared. Learn more about Socket for GitHub ↗︎

This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored.

Ignoring: npm/mz@2.7.0, npm/source-map@0.8.0-beta.0, npm/tree-kill@1.2.2

View full report↗︎

Next steps

Take a deeper look at the dependency

Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support [AT] socket [DOT] dev.

Remove the package

If you happen to install a dependency that Socket reports as Known Malware you should immediately remove it and select a different dependency. For other alert types, you may may wish to investigate alternative packages or consider if there are other ways to mitigate the specific risk posed by the dependency.

Mark a package as acceptable risk

To ignore an alert, reply with a comment starting with @SocketSecurity ignore followed by a space separated list of ecosystem/package-name@version specifiers. e.g. @SocketSecurity ignore npm/foo@1.0.0 or ignore all packages with @SocketSecurity ignore-all

@rygine
Copy link

rygine commented Jan 25, 2024

just a general comment on tsup. if the treeshake option is ever used, the source maps will be inaccurate. more info here.

it can make debugging a huge pain. we've since moved away from tsup in favor of rollup.

@Mrtenz
Copy link
Member Author

Mrtenz commented Jan 26, 2024

just a general comment on tsup. if the treeshake option is ever used, the source maps will be inaccurate. more info here.

it can make debugging a huge pain. we've since moved away from tsup in favor of rollup.

Thanks for the heads up! I'll see how big of a difference treeshake makes in our case. It seems like there's a couple open PRs to fix it as well, so potentially we can patch tsup until one of those is merged.

@Mrtenz
Copy link
Member Author

Mrtenz commented Feb 1, 2024

@SocketSecurity ignore npm/tree-kill@1.2.2
@SocketSecurity ignore npm/source-map@0.8.0-beta.0

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tsup moves everything to separate chunk files (when code splitting is enabled), which breaks code relying on relative imports, in our case, the Webpack loader. As a workaround I added this new loader, which executes a function. This works since the worker path points to __filename, so regardless of where the file is placed, it will load properly.

Copy link

codecov bot commented Feb 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (e682522) 96.59% compared to head (1eba380) 96.60%.

Additional details and impacted files
@@           Coverage Diff           @@
##             tsup    #2120   +/-   ##
=======================================
  Coverage   96.59%   96.60%           
=======================================
  Files         332      334    +2     
  Lines        7582     7595   +13     
  Branches     1175     1175           
=======================================
+ Hits         7324     7337   +13     
  Misses        258      258           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Mrtenz
Copy link
Member Author

Mrtenz commented Feb 18, 2024

@SocketSecurity ignore npm/mz@2.7.0

@@ -1,35 +1,14 @@
{
"name": "@metamask/snaps-simulator",
"version": "2.4.3",
"private": true,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that I added private: true here. We previously published this to NPM for snaps-jest, but we no longer use it.

@@ -100,38 +73,6 @@ jobs:
exit 1
fi

post-build:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed all post-tsc scripts for simplicity.

conditions: os=freebsd & cpu=x64
languageName: node
linkType: hard

"@esbuild/linux-arm64@npm:0.18.20":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we dedupe our esbuild deps or does that require bumping WDIO again?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it requires bumping WDIO yeah.

@Mrtenz Mrtenz changed the base branch from main to tsup February 21, 2024 15:21
@Mrtenz Mrtenz merged commit a785f62 into tsup Feb 22, 2024
120 of 148 checks passed
@Mrtenz Mrtenz deleted the mrtenz/tsup branch February 22, 2024 09:48
Mrtenz added a commit that referenced this pull request Feb 22, 2024
This changes all packages to be built with `tsup`, instead of SWC.
`tsup` uses `esbuild` under the hood, so performance should be
comparable.

More context here: MetaMask/utils#144.
Mrtenz added a commit that referenced this pull request Feb 23, 2024
This changes all packages to be built with `tsup`, instead of SWC.
`tsup` uses `esbuild` under the hood, so performance should be
comparable.

More context here: MetaMask/utils#144.
Mrtenz added a commit that referenced this pull request Feb 26, 2024
This changes all packages to be built with `tsup`, instead of SWC.
`tsup` uses `esbuild` under the hood, so performance should be
comparable.

More context here: MetaMask/utils#144.
Mrtenz added a commit that referenced this pull request Feb 26, 2024
This swaps out the build system for `tsup`, and adds a proper ESM build
to each package. In addition to that, the following changes have been
made:

- All packages with Node.js-specific code have been refactored to make
them browser compatible. Node.js exports were moved to a separate
`/node` export, i.e., `@metamask/snaps-utils/node`.
- This also applies to the React Native webview service in
`snaps-controllers`, which can now be imported from
`@metamask/snaps-controllers/react-native`.
- The `snaps-simulator` package is no longer published to NPM. We were
previously using it for `snaps-jest`, but it's no longer used now.

---

This pull request consists of the following pull requests:

- #2120.
- #2211.
naugtur pushed a commit that referenced this pull request Mar 28, 2024
This changes all packages to be built with `tsup`, instead of SWC.
`tsup` uses `esbuild` under the hood, so performance should be
comparable.

More context here: MetaMask/utils#144.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants