Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support patching dependencies, similar to pnpm patch/patch-package #2336

Closed
josefaidt opened this issue Mar 7, 2023 · 43 comments · Fixed by #11470
Closed

support patching dependencies, similar to pnpm patch/patch-package #2336

josefaidt opened this issue Mar 7, 2023 · 43 comments · Fixed by #11470
Assignees
Labels
bun install Something that relates to the npm-compatible client enhancement New feature or request

Comments

@josefaidt
Copy link
Contributor

What is the problem this feature would solve?

I am currently working with a few dependencies that will cause errors at runtime due to their imports, and I'd also like to apply general changes without suggesting changes to the projects.

Today I'm using pnpm to manage dependencies and bun to run the project, which is fine for the most part. Unfortunately patch-package will not work as it is looking for npm & yarn lockfiles.

What is the feature you are proposing to solve the problem?

bun patch

  • behaves similar to pnpm patch
  • executing bun patch <dependency-name> allows the developer to modify dependencies
  • executing bun patch --apply <dependency-name> applies the patch
    • pnpm creates a copy of the dependency into /private/var/folders and requires the developer to pass the full directory path when applying a patch with pnpm patch-commit <directory-path>

What alternatives have you considered?

No response

@josefaidt josefaidt added the enhancement New feature or request label Mar 7, 2023
@Electroid Electroid added the bun install Something that relates to the npm-compatible client label Jun 28, 2023
@harrytran998
Copy link

Hello @josefaidt - Can you have some time to make this issue progress? Thank 😆

@birkskyum
Copy link
Collaborator

Alternatively, if it's easier to add support for using patch-package, that would go a long way too.

@OnurGvnc
Copy link

OnurGvnc commented Sep 9, 2023

Attempting to create a patch using bunx patch-package <package-name> fails. The patch-package outputs the following error:

Copy code
**ERROR** No package-lock.json, npm-shrinkwrap.json, or yarn.lock file.

You must use either npm@>=5, yarn, or npm-shrinkwrap to manage this project's
dependencies.

However, if you have pre-existing patches in the ./patches directory, you can successfully apply them using the
bunx patch-package command (or via package.json -> scripts -> "postinstall": "patch-package")

As a temporary workaround to generate a patches/package+name+version.patch file

  • Run npm install to generate a package-lock.json file (temporarily, we'll remove it later)
  • Make the necessary modifications inside node-modules/<package-name>
  • Execute npx patch-package <package-name> to create the patches/package+name+version.patch file
  • Run rm package-lock.json && bun install

I found a new workaround:

  • run bun install --yarn it just converts bun.lockb into a yarn.lock . (or bun ./bun.lockb > ./yarn.lock )
  • make the necessary modifications inside node-modules/<package-name>
  • bunx patch-package <package-name>
  • rm yarn.lock
  • package.json -> scripts -> "postinstall": "bunx patch-package"
    https://github.com/ds300/patch-package

@SimonVerhoeven
Copy link

It would be nice to see this to facilitate angular/angular#46719

@glekner
Copy link

glekner commented Sep 10, 2023

Any plans to implement this? @Jarred-Sumner

@deathemperor
Copy link

deathemperor commented Sep 11, 2023

I found a new workaround:

  • run bun install --yarn it just converts bun.lockb into a yarn.lock
  • make the necessary modifications inside node-modules/<package-name>
  • bunx patch-package <package-name>
  • package.json -> scripts -> "postinstall": "patch-package"
    ds300/patch-package

this works with one minor tweak: "postinstall": "bunx patch-package" if patch-package is not in your dependencies

@juliusmarminge
Copy link

+1 pnpm patch is no nice to have and definetely a blocking feature before adopting bun in some of my repos

@juliusmarminge
Copy link

I found a new workaround:

  • run bun install --yarn it just converts bun.lockb into a yarn.lock
  • make the necessary modifications inside node-modules/<package-name>
  • bunx patch-package <package-name>
  • package.json -> scripts -> "postinstall": "patch-package"
    ds300/patch-package

This wont work though for platforms like vercel that determines the package manager by which lockfile you have in the repo, so if you have a yarn.lock they will use yarn to install deps unless i put in manual overrides :/

@intrnl
Copy link

intrnl commented Sep 12, 2023

I'm very certain you only need the yarn.lock so that patch-package can determine what package to retrieve for you to be able to modify them. You don't need to commit the yarn.lock file after creating the patch.

@nandorojo
Copy link

nandorojo commented Sep 13, 2023

I got patch-package working with Bun! Simply add --yarn when you bun install. This creates a yarn.lock file for you.

bun install --yarn

You can now use patch-package:

bunx patch-package moti

@deathemperor
Copy link

I found a new workaround:

  • make the necessary modifications inside node-modules/<package-name>
  • bunx patch-package <package-name>
  • package.json -> scripts -> "postinstall": "patch-package"

ds300/patch-package

This wont work though for platforms like vercel that determines the package manager by which lockfile you have in the repo, so if you have a yarn.lock they will use yarn to install deps unless i put in manual overrides :/

It works, my production repos are using it. Moreover, vercel already supports bun for building.

@remorses
Copy link

remorses commented Oct 14, 2023

Instead of adding a bun patch command i propose adding a bun vendor package command that

  • moves the package form node_modules/package to a vendor/package folder
  • symlinks this folder to node_modules
  • implicitly adds this package to the install overrides so all other dependencies resolve package to the vendor folder

This command can be used

  • to patch the contents of a package without relying on a diff file
  • to debug a package behaviour without modifying node_modules
  • to let tools like Next.js update the browser when the package contents change (by default Next.js ignores changes in node_modules, you have to delete the cache to see node_modules changes)

@Jarred-Sumner
Copy link
Collaborator

we probably will do bun pm patch

we've looked into adding it but are unhappy with the performance of various diff/patch libraries

@frogkind
Copy link

frogkind commented Dec 6, 2023

seem patch-package not work for non-npm dependencies such as git even i manually prepare the patch file,
is there any workaround?

@xlc
Copy link
Contributor

xlc commented Dec 31, 2023

please don't worry about performance for the first version (of course unless it is unreasonably slow). a good enough is better than not having this feature at all

@rikur
Copy link

rikur commented Jan 12, 2024

+1 for having some kind of solution even if its performance is not on par with rest of the project.

@oasaleh
Copy link

oasaleh commented Jan 29, 2024

  • unx patch-package

I followed your steps but I'm getting the following error.

bunx patch-package drizzle-kit
patch-package 8.0.0
• Creating temporary folder
• Installing drizzle-kit@0.20.13 with yarn
21 | function spawnSync(command, args, options) {
22 |     // Parse the arguments
23 |     const parsed = parse(command, args, options);
24 |
25 |     // Spawn the child process
26 |     const result = cp.spawnSync(parsed.command, parsed.args, parsed.options);
                        ^
TypeError: Executable not found in $PATH: "yarn"
 code: "ERR_INVALID_ARG_TYPE"

      at node:child_process:173:47
      at spawnSync (/private/tmp/patch-package@latest--bunx/node_modules/cross-spawn/index.js:26:20)
      at spawnSafeSync (/private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/spawnSafe.js:11:20)
      at makePatch (/private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/makePatch.js:131:17)
      at /private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/index.js:72:13
      at forEach (:1:21)
      at /private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/index.js:71:9
      at /private/tmp/patch-package@latest--bunx/node_modules/patch-package/index.js:3:1

21 | function spawnSync(command, args, options) {
22 |     // Parse the arguments
23 |     const parsed = parse(command, args, options);
24 |
25 |     // Spawn the child process
26 |     const result = cp.spawnSync(parsed.command, parsed.args, parsed.options);
                        ^
TypeError: Executable not found in $PATH: "yarn"
 code: "ERR_INVALID_ARG_TYPE"

      at node:child_process:173:47
      at spawnSync (/private/tmp/patch-package@latest--bunx/node_modules/cross-spawn/index.js:26:20)
      at spawnSafeSync (/private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/spawnSafe.js:11:20)
      at makePatch (/private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/makePatch.js:131:17)
      at /private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/index.js:72:13
      at forEach (:1:21)
      at /private/tmp/patch-package@latest--bunx/node_modules/patch-package/dist/index.js:71:9
      at /private/tmp/patch-package@latest--bunx/node_modules/patch-package/index.js:3:1

Is installing yarn a must?

@corysimmons
Copy link

In the meantime, I just slapped together some naive implementation of a patcher (just copies packages by name from node_modules to a ./patches dir with a command, then destructively pastes them back to node_modules with another command) to scratch this itch. It should work with anything (npm, yarn, pnpm, bun, etc.) with the same API (just swap out npx for bunx where appropriate).

https://github.com/corysimmons/patcheer

It seems to work well for my needs and should be as fast as $ cp

@corysimmons
Copy link

@resthedev I can't speak for patch-package but @rikur's fix worked for my patching package with my Vercel build. I just set my Vercel build command to bun i (I didn't append the --yarn flag), then bun add -D postinstall-postinstall & committed/pushed. And the build succeeded. 🎉

@resthedev
Copy link

@resthedev I can't speak for patch-package but @rikur's fix worked for my patching package with my Vercel build. I just set my Vercel build command to bun i (I didn't append the --yarn flag), then bun add -D postinstall-postinstall & committed/pushed. And the build succeeded. 🎉

I see. Do you mean you put bun i in the Vercel "build" command or the "install" command?
CleanShot 2024-01-31 at 22 57 12@2x

@corysimmons
Copy link

Yes, it should recognize it based on there only being a bun.lockb file in your project root and automatically run bun install but I don't trust it so I just manually add bun i to it.

@gustavopch
Copy link

I was migrating to Bun and realized I wouldn't be able to use patch-package. I guess I'll have to wait.

@corysimmons
Copy link

@gustavopch I provide a workaround a couple of comments above yours.

  1. bun add -D patcheer postinstall-postinstall
  2. bunx patcheer copy <package_name>
  3. Make your tweaks in ./patches/<package_name>
  4. bunx patcheer apply
  5. Add "postinstall": "bunx patcheer apply" to your scripts in your package.json.
  6. Make your build command on your host bun install.

This should work locally and in any hosting env that supports Bun (i.e. it's working on Vercel for me). Ping me if something doesn't work.

Unsubscribing from this thread now. 💨

@km-tr
Copy link

km-tr commented Mar 15, 2024

@corysimmons
Even if I do this, it only modifies the node_modules, and the actual running code doesn't change. But are there cases where this actually works? I'm using expo with bun.

@AngeloCloudAcademy
Copy link

@Jarred-Sumner some workaround in the meantime?

@rikur
Copy link

rikur commented Mar 18, 2024

@AngeloCloudAcademy you can try this with postinstall-postinstall. Been using it for months now without issues.

#2336 (comment)

@bryanltobing
Copy link

bryanltobing commented Mar 26, 2024

i'm doing this in the meantime

I got patch-package working with Bun! Simply add --yarn when you bun install. This creates a yarn.lock file for you.

bun install --yarn

You can now use patch-package:

bunx patch-package moti

this works, also adding yarn.lock to .gitignore so that it doesn't get commited

@ReoHakase
Copy link

After numerous amount of experiments, I found that patching patch-package by itself to rewrite a called package manager to bun from yarn, works. (Confirmed in local environment without yarn, GitHub Actions, Vercel)

Steps

Configure bun to generate yarn lockfile

Specify a --yarn flag like bun install --yarn or add the following lines to your bunfig.toml in order to generate yarn.lock file.
This is needed since patch-package currently does not support bun's binary lockfile format. (See ds300/patch-package#489)

[install.lockfile]
print = "yarn" # Whether to generate a non-Bun lockfile alongside bun.lockb

Warning

I personally recommend the later one since you need to generate the yarn v1 lockfile every time you update dependencies.

Install patch-package

Then, add patch-package to dev dependencies.

bun add -D patch-package

Manually modify a source file of patch-package

Open node_modules/patch-package/dist/makePatch.js and look for spawnSafe_1.spawnSafeSync that spawns yarn, then replace it with bun like this diff:

diff --git a/node_modules/patch-package/dist/makePatch.js b/node_modules/patch-package/dist/makePatch.js
index d8d0925..874284a 100644
--- a/node_modules/patch-package/dist/makePatch.js
+++ b/node_modules/patch-package/dist/makePatch.js
@@ -120,7 +120,7 @@ function makePatch({ packagePathSpecifier, appPath, packageManager, includePaths
             try {
                 // try first without ignoring scripts in case they are required
                 // this works in 99.99% of cases
-                spawnSafe_1.spawnSafeSync(`yarn`, ["install", "--ignore-engines"], {
+                spawnSafe_1.spawnSafeSync(`bun`, ["install"], {
                     cwd: tmpRepoNpmRoot,
                     logStdErrOnError: false,
                 });
@@ -128,7 +128,7 @@ function makePatch({ packagePathSpecifier, appPath, packageManager, includePaths
             catch (e) {
                 // try again while ignoring scripts in case the script depends on
                 // an implicit context which we haven't reproduced
-                spawnSafe_1.spawnSafeSync(`yarn`, ["install", "--ignore-engines", "--ignore-scripts"], {
+                spawnSafe_1.spawnSafeSync(`bun`, ["install", "--ignore-scripts"], {
                     cwd: tmpRepoNpmRoot,
                 });
             }

Note

You can remove --ignore-engines flag without worrying since bun does not support engines field in package.json at the moment. (See #5846)

Save the patch you've made to patches/

This command looks like pure madness, but surprisingly it worked 😮

bun patch-package patch-package

Now, patches/patch-package+8.0.0.patch should be created.

Add postinstall command to package.json

As it is written in patch-package documentation.

{
  "name": "reoiam-dev",
  "version": "0.0.0",
  "workspaces": [
    "apps/*",
    "packages/*"
  ],
  "scripts": {
    "postinstall": "patch-package",
  },

Modify source files of your desired package

bun patch-package @acme/some-package

Also, run git clean -dfX && bun i to check if it is working.


Example commits:
ReoHakase/reoiam-dev@2c01637
ReoHakase/reoiam-dev@8aae34e

@rChaoz
Copy link

rChaoz commented Apr 22, 2024

I would also like to add that it would be awesome to support something like Yarn does with resolutions:

package.json:

{
    "resolutions": {
        "some-library:1.2.3": "patch:patch-file",
        "other-library:1.0.0": "http://alternate-source/file.tgz",
        "final-library:100.0.0": "./dir/final-library-fork"
    }
}

This support pretty much every use case:

  1. simple bugfixes can be made directly on the modules using a patch tool
  2. more complex changes can be made by forking a library repo, cloning, changing & building it locally
  3. sometimes, the process from 2 is simplified by CI tools that provide a release version for PRs, so including the library clone in your code is not needed

@vijaybritto
Copy link

vijaybritto commented May 3, 2024

To make it work with Private registries

I followed the steps by @ReoHakase and it worked for public packages. But to make it work with our private registry I had to do the following changes:

diff --git a/node_modules/patch-package/dist/makePatch.js b/node_modules/patch-package/dist/makePatch.js
index d8d0925..7c488cd 100644
--- a/node_modules/patch-package/dist/makePatch.js
+++ b/node_modules/patch-package/dist/makePatch.js
@@ -109,18 +109,18 @@ function makePatch({ packagePathSpecifier, appPath, packageManager, includePaths
             resolutions: resolveRelativeFileDependencies_1.resolveRelativeFileDependencies(appPath, appPackageJson.resolutions || {}),
         }));
         const packageVersion = getPackageVersion_1.getPackageVersion(path_1.join(path_1.resolve(packageDetails.path), "package.json"));
-        [".npmrc", ".yarnrc", ".yarn"].forEach((rcFile) => {
+        [".npmrc", ".yarnrc", ".yarn", "bunfig.toml", ".env"].forEach((rcFile) => {
             const rcPath = path_1.join(appPath, rcFile);
             if (fs_extra_1.existsSync(rcPath)) {
                 fs_extra_1.copySync(rcPath, path_1.join(tmpRepo.name, rcFile), { dereference: true });
             }
         });
         if (packageManager === "yarn") {
-            console_1.default.info(chalk_1.default.grey("•"), `Installing ${packageDetails.name}@${packageVersion} with yarn`);
+            console_1.default.info(chalk_1.default.grey("•"), `Installing ${packageDetails.name}@${packageVersion} with bun`);
             try {
                 // try first without ignoring scripts in case they are required
                 // this works in 99.99% of cases
-                spawnSafe_1.spawnSafeSync(`yarn`, ["install", "--ignore-engines"], {
+                spawnSafe_1.spawnSafeSync(`bun`, ["install"], {
                     cwd: tmpRepoNpmRoot,
                     logStdErrOnError: false,
                 });
@@ -128,7 +128,7 @@ function makePatch({ packagePathSpecifier, appPath, packageManager, includePaths
             catch (e) {
                 // try again while ignoring scripts in case the script depends on
                 // an implicit context which we haven't reproduced
-                spawnSafe_1.spawnSafeSync(`yarn`, ["install", "--ignore-engines", "--ignore-scripts"], {
+                spawnSafe_1.spawnSafeSync(`bun`, ["install", "--ignore-scripts"], {
                     cwd: tmpRepoNpmRoot,
                 });
             }

I had to specify "bunfig.toml", ".env" extra in the list of files to copy into the temporary folder. This is to make sure that bun can find the private registry in the bunfig.toml file.

To Summarize:

  1. Change bunfig.toml to add the following
[install.lockfile]
print = "yarn" # Whether to generate a non-Bun lockfile alongside bun.lockb
  1. Add the package
bun add -D patch-package
  1. Make changes in node_modules/patch-package/dist/makePatch.js as shown above
  2. Create the patch for patch-package
bun patch-package patch-package
  1. Add postinstall script
{
  "name": "reoiam-dev",
  "version": "0.0.0",
  "workspaces": [
    "apps/*",
    "packages/*"
  ],
  "scripts": {
    "postinstall": "patch-package",
  }
  1. Optionally verify it by forcing a full reinstall
bun install --force
  1. Follow regular patch-package flows

@zackradisic
Copy link
Contributor

For those tracking this issue, I am currently working on this. Patching a package will behave similarly to pnpm, through the bun patch <pkg> and bun patch-commit <path> commands.

Yarn's patch protocol will also be supported.

My work-in-progress PR for this feature is here.

@Jarred-Sumner
Copy link
Collaborator

The merged implementation of bun patch does not meet our quality bar. Leaving this open until #11719 is completed

@Jarred-Sumner
Copy link
Collaborator

Fixed by @zackradisic in #11858

@Scalahansolo
Copy link

Attempted to use this and am running into this issue? Am I doing something wrong here?

CleanShot 2024-06-19 at 22 03 31@2x

@zackradisic
Copy link
Contributor

@Scalahansolo
My fixes in this PR (#12022) might fix this. There were issues running bun patch in workspaces and with scoped packages

@Scalahansolo
Copy link

Scalahansolo commented Jun 24, 2024

Still doesn't work in our monorepo on 1.1.16

cc @zackradisic

CleanShot 2024-06-23 at 21 23 26@2x

@Jarred-Sumner
Copy link
Collaborator

@Scalahansolo can you file an issue and @zackradisic will fix it?

@nobkd
Copy link

nobkd commented Jun 25, 2024

Is there a way to prepare multiple packages for patching at once?
Something like bun patch pkg1 pkg2
and same for bun patch --commit pkg1 pkg2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bun install Something that relates to the npm-compatible client enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.