Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] npm install removes linked packages, npm link replaces linked package contents #2380

Open
hdodov opened this issue Dec 18, 2020 · 19 comments
Labels
Bug thing that needs fixing platform:windows is Windows-specific Release 7.x work is associated with a specific npm 7 release

Comments

@hdodov
Copy link

hdodov commented Dec 18, 2020

I have specified a package in my package.json that I have also linked with a local version I use for development. When I run npm i in the parent package, the symlink is removed, the package is installed from npm, and I use the live version instead, not my local development one.

Later, if I run npm link, the package correctly get linked with my local version, but everything is replaced with the downloaded version, meaning that all my local development files are replaced with the contents of the tarball downloaded from npm. This means I have to go in my package folder, clone the repo again, checkout the files from the master branch... Not to mention that if I have any unsaved work, it would get completely erased with no chance to get it back even via Git - because the .git folder is also deleted.

Current Behavior:

  • running npm i removes linked packages and installs their published versions instead
  • running npm link <package> replaces the contents of the symlinked package folder with the published package's tarball contents

Expected Behavior:

  • running npm i should ignore packages in node_modules that are symlinked
  • running npm link <package> should only create a symlink and leave everything in the symlinked folder intact

Steps To Reproduce:

  1. Install a package from npm in a project
  2. Run npm link in a different, local version of the package
  3. Run npm link <package> in the project
  4. Run npm i
  5. Linked package is removed
  6. Run npm link <package> in the project
  7. Local version is replaced with the published tarball contents

Environment:

  • OS: Microsoft Windows 10 Pro, 10.0.18363 Build 18363
  • nvm: 1.1.7
  • Node: 15.4.0
  • npm: 7.0.15 (can't install 7.2 due to nvm)
@hdodov hdodov added Bug thing that needs fixing Needs Triage needs review for next steps Release 7.x work is associated with a specific npm 7 release labels Dec 18, 2020
@jeremy-coleman
Copy link

Yikes, thats scary.

@ljharb
Copy link
Contributor

ljharb commented Dec 18, 2020

@hdodov in what way would nvm prevent npm install -g npm@7? edit: ah, i see you're using nvm-windows, not actual nvm.

@darcyclarke darcyclarke added platform:windows is Windows-specific and removed Needs Triage needs review for next steps labels Jan 29, 2021
@darcyclarke
Copy link
Contributor

darcyclarke commented Jan 29, 2021

@hdodov can you install the latest v7 & try reproducing this? (ie. npm i -g npm@7)

@hdodov
Copy link
Author

hdodov commented Feb 4, 2021

@darcyclarke I can't reach the point of being able to reproduce. With a fresh install of Node 15.8.0 and npm 7.5.1, I create a symlink by running npm link in the package folder:

added 1 package, and audited 4 packages in 231ms

...which creates the symlink (oblik). However, when I open it, it seems to be broken:

image

...and indeed if I try to run npm link oblik in a project, I get:

npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path C:/Program Files/nodejs/node_modules/oblik/package.json
npm ERR! errno -4058
npm ERR! enoent ENOENT: no such file or directory, open 'C:\Program Files\nodejs\node_modules\oblik\package.json'    
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\dodov\AppData\Local\npm-cache\_logs\2021-02-04T07_50_48_106Z-debug.log

Here's the log: 2021-02-04T07_50_48_106Z-debug.log.


I tried to link and unlink several times, but nothing seems to fix it. I guess that's for another issue, though? Let me know if I should open a new one. Maybe there's some issue with nvm for Windows? Although I had no issues with it specifically.

@darcyclarke
Copy link
Contributor

darcyclarke commented Feb 4, 2021

@hdodov good to hear the initial issue seems to be resolved. I have heard of other folks using nvs or volta on Windows for version management (there's also nvm-windows, not sure if that's what you're using already, but I'm not sure how well it's supported) - could you maybe try one of those & see if linking breaks there as well?

@hdodov
Copy link
Author

hdodov commented Feb 5, 2021

@hdodov good to hear the initial issue seems to be resolved.

@darcyclarke actually, I don't know if it's resolved. That was my point in my previous comment - I couldn't get past the npm link <package> part to see what the npm install after it would do - whether the link would stay or my child project would get erased.

I'll experiment with other version management tools and post the update here. 👍

@hdodov
Copy link
Author

hdodov commented Feb 5, 2021

I installed Volta 1.0.1 and I didn't have the problems mentioned in my previous comment. I tried to reproduce the issue and it seems that it's half solved. Here's what I tested:

  1. npm link oblik (without the package in my package.json): adds the package as a link
  2. npm i: removes the linked package (maybe not expected?)
  3. npm i oblik: adds the package from npm and lists it in package.json
  4. npm link oblik: changes the package from the downloaded version to the local one via symlink
  5. npm i: changes the package from the linked version to the downloaded one, even though it's listed in package.json (not expected)

Basically, whenever I run a command that installs or uninstalls a package, the linked package is switched with the downloaded version if it's listed in package.json, or the link is simply removed if it isn't listed.

What's different this time is that my child package (the one I'm linking) didn't get erased at any point during the testing, which is wonderful. The only real issue now is that I have to run npm link <package> every time my dependencies change.

OS: Microsoft Windows 10 Pro, 10.0.18363 Build 18363
Volta: 1.0.1
Node: 15.8.0
npm: 7.5.1

ssbarnea added a commit to ssbarnea/ansible-language-server that referenced this issue Nov 17, 2021
To prevent npm install from overriding `npm link` outcomes or getting
errors when running `npm ls` we must be sure we do always add `--save`

Related: npm/cli#2380
ssbarnea added a commit to ssbarnea/vscode-ansible that referenced this issue Nov 18, 2021
To prevent npm install from overriding `npm link` outcomes or getting
errors when running `npm ls` we must be sure we do always add `--save`

Related: npm/cli#2380
Related: ansible/ansible-language-server#130
ssbarnea added a commit to ssbarnea/vscode-ansible that referenced this issue Nov 18, 2021
To prevent npm install from overriding `npm link` outcomes or getting
errors when running `npm ls` we must be sure we do always add `--save`

Related: npm/cli#2380
Related: ansible/ansible-language-server#130
@tangobravo
Copy link

npm i replacing any links does appear to be the intended default behaviour. With --save you can update package.json so it will persist through npm install operations.
https://docs.npmjs.com/cli/v8/commands/npm-link#caveat

@ssbarnea
Copy link

ssbarnea commented Feb 4, 2022

As package.json is almost for sure tracked, this cannot reliably be used on CI as it is lightly that the pipeline will fail because a tracked file was modified (a common security measure for ensuring that what you test is what you track).

@joepie91
Copy link

joepie91 commented May 7, 2022

npm i replacing any links does appear to be the intended default behaviour.

That behaviour doesn't make any sense for the intended usecase, though, and this is not how it worked in old npm versions (before npm link was broken) either. The whole point of this functionality is to temporarily substitute a dependency while developing or debugging on multiple packages at once.

So neither "overriding package.json with the substitute" nor "have the substitute be deleted every time you add a completely unrelated dependency" make sense here. The correct behaviour would be to retain the substitute across npm commands, without changing the package.json (and ideally not changing the package-lock.json either).

Edit: This was supposed to be fixed in v7, by the way.

@eliot-akira
Copy link

eliot-akira commented Jun 3, 2022

The first part of the originally reported issue is still happening for me, with npm version 8.12.1.

  • running npm install removes linked packages and installs their published versions instead

Apparently this is now the intended behavior, but it wasn't always like this. I used to have a workflow using npm link to temporarily substitute modules for local development. Running npm install would keep any symlinked modules - I thought that was the whole purpose of npm link.

It was working well for a number of projects and linked modules, but at some point (I seem to recall it started in npm version 5.x) the behavior changed, where npm install would remove and replace linked modules.

(This was never solved, and after a while I had to switch to yarn which works as expected for my local setup. I'm actually trying to move back to using npm fully since it's the canonical package manager for Node.js - and I have issues with the direction yarn is going, but that's another story..)

With the current behavior, every time I run npm install, I have to re-run npm link <package> to manually restore the removed links.

If this is the intended default behavior, would it be possible to add a CLI option to preserve linked modules as before?

@vzakharov-rxnt
Copy link

I find it odd the one of the most important development workflows hasn't been fixed for almost 2 years now.
I am using Node 16.15.0.

@ssbarnea
Copy link

ssbarnea commented Jul 4, 2022

@vzakharov-rxnt While this bug is marked like linking would be broken only on Windows, the reality is that npm link is also broken on platforms fully supporting symlinks too, where npm i does override the links. That is why we switched from npm to yarn on one of our projects (vscode-ansible) last month. We also got some serious performance benefits and much better UX for upgrading dependencies but these were only perks, the linking was key.

@vzakharov-rxnt
Copy link

vzakharov-rxnt commented Jul 4, 2022

@ssbarnea We had issues with unit tests when dependencies were installed with yarn. Had to switch back to npm.
Unit tests are using karma/jasmine.

@Vasile-Peste
Copy link

After many years using npm link it's still one of my worst nightmares...

@scott-lc
Copy link

@Vasile-Peste - The link package along with a prepare hook has been an excellent alternative for me.

npx link = A safer version of npm link.

{
  "scripts": {
    "prepare": "(test -f link.config.json && npx link) || true"
   ...
 }
}

@mryellow
Copy link

npm link A should not invalidate npm link B.

Both packages should exist as symlinks.

@pjdon
Copy link

pjdon commented Apr 15, 2024

Still getting this with npm version 9.5.0. I get the message removed 1 package when running npm install. When re-running npm link <source-package> it just recreates the symlink, doesn't add a tarball or anything.

@tjsr
Copy link

tjsr commented May 7, 2024

I've been having issues with my dev workflow and this lately. While it seems counter-intuitive to blow away a linked local package, I've decided that the answer is to have an npm postinstall script that then links the packages you want to know about relative to the project.

To help me around this I'm going to write a script/tool in the coming day or so that goes through and looked at linked global packages (either symlink or junction), and calls link on each of either the specified one, or all of them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug thing that needs fixing platform:windows is Windows-specific Release 7.x work is associated with a specific npm 7 release
Projects
None yet
Development

No branches or pull requests