Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

npm is installed wrongly on windows machine #53538

Closed
anonrig opened this issue Jun 21, 2024 · 42 comments
Closed

npm is installed wrongly on windows machine #53538

anonrig opened this issue Jun 21, 2024 · 42 comments
Labels
npm Issues and PRs related to the npm client dependency or the npm registry. windows Issues and PRs related to the Windows platform.

Comments

@anonrig
Copy link
Member

anonrig commented Jun 21, 2024

I'm getting the following error when trying to run a simple npx command on Windows. I opened the issue to this repository because we are distributing npm, we define the location of npm in a Windows machine.

I reinstalled Node.js from nodejs.org with the latest v22, and I can not use npm at all. Any input/help is appreciated.

PS: I'm using PowerShell.

PS C:\Users\yagiz\Desktop\coding\node> npx node-benchmark-compare .\bench.txt
npm error code ENOENT
npm error syscall lstat
npm error path C:\Users\yagiz\AppData\Roaming\npm
npm error errno -4058
npm error enoent ENOENT: no such file or directory, lstat 'C:\Users\yagiz\AppData\Roaming\npm'
npm error enoent This is related to npm not being able to find a file.
npm error enoent
npm error A complete log of this run can be found in: C:\Users\yagiz\AppData\Local\npm-cache\_logs\2024-06-21T20_29_57_155Z-debug-0.log

cc @nodejs/platform-windows

@anonrig anonrig added the windows Issues and PRs related to the Windows platform. label Jun 21, 2024
@richardlau
Copy link
Member

Maybe related to npm/cli#7089?

@anonrig
Copy link
Member Author

anonrig commented Jun 21, 2024

cc @nodejs/npm this impacts all windows npm users.

@richardlau
Copy link
Member

Also we used to create AppData\Roaming\npm in the msi installer, but stopped doing so: 0ae8bf8

@abrowncownow
Copy link

@anonrig Can you post the details of the log C:\Users\yagiz\AppData\Local\npm-cache\_logs\2024-06-21T20_29_57_155Z-debug-0.log

@anonrig
Copy link
Member Author

anonrig commented Jun 21, 2024

@anonrig Can you post the details of the log C:\Users\yagiz\AppData\Local\npm-cache\_logs\2024-06-21T20_29_57_155Z-debug-0.log

0 verbose cli C:\Program Files\nodejs\node.exe C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js
1 info using npm@10.8.1
2 info using node@v22.3.0
3 silly config load:file:C:\Program Files\nodejs\node_modules\npm\npmrc
4 silly config load:file:C:\Users\yagiz\Desktop\coding\node\.npmrc
5 silly config load:file:C:\Users\yagiz\.npmrc
6 silly config load:file:C:\Users\yagiz\AppData\Roaming\npm\etc\npmrc
7 verbose title npm exec node-benchmark-compare .\bench.txt
8 verbose argv "exec" "--" "node-benchmark-compare" ".\\bench.txt"
9 verbose logfile logs-max:10 dir:C:\Users\yagiz\AppData\Local\npm-cache\_logs\2024-06-21T20_29_57_155Z-
10 verbose logfile C:\Users\yagiz\AppData\Local\npm-cache\_logs\2024-06-21T20_29_57_155Z-debug-0.log
11 silly logfile start cleaning logs, removing 1 files
12 silly logfile done cleaning log files
13 silly packumentCache heap:4345298944 maxSize:1086324736 maxEntrySize:543162368
14 http fetch GET 200 https://registry.npmjs.org/node-benchmark-compare 152ms (cache revalidated)
15 silly packumentCache heap:4345298944 maxSize:1086324736 maxEntrySize:543162368
16 verbose stack Error: ENOENT: no such file or directory, lstat 'C:\Users\yagiz\AppData\Roaming\npm'
16 verbose stack     at async lstat (node:internal/fs/promises:1026:18)
16 verbose stack     at async #loadActual (C:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\arborist\load-actual.js:112:20)
16 verbose stack     at async exec (C:\Program Files\nodejs\node_modules\npm\node_modules\libnpmexec\lib\index.js:203:26)
16 verbose stack     at async Npm.exec (C:\Program Files\nodejs\node_modules\npm\lib\npm.js:207:9)
16 verbose stack     at async module.exports (C:\Program Files\nodejs\node_modules\npm\lib\cli\entry.js:74:5)
17 error code ENOENT
18 error syscall lstat
19 error path C:\Users\yagiz\AppData\Roaming\npm
20 error errno -4058
21 error enoent ENOENT: no such file or directory, lstat 'C:\Users\yagiz\AppData\Roaming\npm'
22 error enoent This is related to npm not being able to find a file.
22 error enoent
23 verbose cwd C:\Users\yagiz\Desktop\coding\node
24 verbose os Windows_NT 10.0.22631
25 verbose node v22.3.0
26 verbose npm  v10.8.1
27 verbose exit -4058
28 verbose code -4058
29 error A complete log of this run can be found in: C:\Users\yagiz\AppData\Local\npm-cache\_logs\2024-06-21T20_29_57_155Z-debug-0.log

@tniessen
Copy link
Member

Does npm i -g npm also fail or does it perhaps resolve the issue?

@anonrig
Copy link
Member Author

anonrig commented Jun 23, 2024

Does npm i -g npm also fail or does it perhaps resolve the issue?

It might but running it would destroy the repro in case someone needs it?

@lukekarrys
Copy link
Member

@anonrig does it work if you use cmd instead of PowerShell? this is just to figure out if it is also a side effect of #52682

@huseyinacacak-janea
Copy link
Contributor

I've looked into this issue and seen that this is caused by stopping creating AppData\Roaming\npm as @richardlau said. Since this is a security concern, it is unlikely that this commit will be reverted.

There are workarounds mentioned on the internet. Their basic idea is to create the appropriate folder, either manually by creating an empty folder or by installing a global module. The basic command to create this folder would be npm i -g npm, as this does not install an additional module, but moves the npm folder to global.

Additionally, you could install the module as an extra step before running npx node-benchmark-compare .\bench.txt. For example, in this case, you could run npm i -g node-benchmark-compare.

The npx documentation might be misleading for some users. It can be updated accordingly.

@lukekarrys, I tried from cmd and it is the same.

@anonrig
Copy link
Member Author

anonrig commented Jun 29, 2024

This might be a security issue, but regardless we should never ship a broken/buggy software. If this is indeed the default behavior, than I ask myself "why are we shipping a broken feature on Windows?"

cc @nodejs/tsc

@anonrig
Copy link
Member Author

anonrig commented Jun 29, 2024

@anonrig does it work if you use cmd instead of PowerShell? this is just to figure out if it is also a side effect of #52682

Didn't fix the issue.

@anonrig anonrig added npm Issues and PRs related to the npm client dependency or the npm registry. tsc-agenda Issues and PRs to discuss during the meetings of the TSC. labels Jun 29, 2024
@anonrig
Copy link
Member Author

anonrig commented Jun 29, 2024

@MylesBorins I think this requires your attention

@GeoffreyBooth
Copy link
Member

@nodejs/npm

@benjamingr
Copy link
Member

@anonrig why did you tag this tsc-agenda? What benefit would this get from a 20 person meeting?

@tniessen
Copy link
Member

Here are excerpts from some of my own comments from internal discussions that led up to 0ae8bf8.

Creating AppData\Roaming\npm was an intentional addition in order to resolve nodejs/node-v0.x-archive#8141, which appears to have been a common issue for users of npm. However, this was implemented before 4cfe5eb, which changed the MSI installation scope to perMachine. There were concerns about creating the npm directory in that PR, albeit not related to security (see nodejs/node-v0.x-archive#25640).

FWIW, I am not quite happy with this patch. My main concern is nodejs/node-v0.x-archive#8141 becoming an issue again, but I am not sure under what circumstances it would. The only thing I can think of is if AppData\Roaming\npm somehow cannot be created by npm (i.e., by an unprivileged user), but then we probably also cannot create it from within the installer through an unprivileged action.

So while we weren't certain at the time, we did consider the possibility that on some Windows systems, this patch might re-introduce some old npm issues.

I still don't see a reasonable way for the Node.js installer to create that directory. What I'm wondering is if npm is assuming the directory exists (and it doesn't) or if npm would create the directory (but can't).

@joyeecheung
Copy link
Member

It seems like a UX issue that can be addressed with better docs and better suggestions in the errors. But it seems more appropriate to just switch to npm/cli#7089

@anonrig anonrig removed the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Jun 29, 2024
@targos
Copy link
Member

targos commented Jun 29, 2024

I don't see why npm couldn't create the directory when needed. It is in a path that belongs to the current user so permissions shouldn't be an issue

@ruyadorno
Copy link
Member

iirc node tweaks the npm prefix on windows in: tools/msvs/npm/npmrc

I believe that custom path does solve a few problems in windows but given that is not the default path that the standalone npm client is using it creates a split for windows users and can easily be a blind spot for the npm team to maintain.

If my assumptions above are not wrong, the fix would be for the npm cli team to clean up that prefix story on windows so that's straightforward to support it moving forward and of course start to create that missing folder (in case that's the one they want to standardize on).

@anonrig
Copy link
Member Author

anonrig commented Jun 29, 2024

@anonrig why did you tag this tsc-agenda? What benefit would this get from a 20 person meeting?

@benjamingr I understand you but you should know that: I've added the tsc-agenda because this is a major bug which is not addressed for more than a week on an operating system we have tier 1 support. The benefit of getting this to a 20 person meeting is to raise awareness, and expedite the fix of this problem, and potentially navigate to why no-one addressed this in a timely manner.

@richardlau
Copy link
Member

richardlau commented Jun 29, 2024

iirc node tweaks the npm prefix on windows in: tools/msvs/npm/npmrc

I believe that custom path does solve a few problems in windows but given that is not the default path that the standalone npm client is using it creates a split for windows users and can easily be a blind spot for the npm team to maintain.

That's interesting because the npm docs say:

prefix Configuration

The prefix config defaults to the location where
node is installed. On most systems, this is /usr/local. On Windows, it's
%AppData%\npm. On Unix systems, it's one level up, since node is typically
installed at {prefix}/bin/node rather than {prefix}/node.exe.

When the global flag is set, npm installs things into this prefix.
When it is not set, it uses the root of the current package, or the
current working directory if not in a package already.

Node Modules

Packages are dropped into the node_modules folder under the prefix.
When installing locally, this means that you can
require("packagename") to load its main module, or
require("packagename/lib/path/to/sub/module") to load other modules.

Global installs on Unix systems go to {prefix}/lib/node_modules.
Global installs on Windows go to {prefix}/node_modules (that is, no
lib folder.)

@ruyadorno
Copy link
Member

@richardlau sounds like my assumptions might be wrong then 😅

@tniessen
Copy link
Member

Does this affect all Windows users or just a subset? I'm a bit surprised we haven't heard more complaints about this if it does affect all users.

@anonrig
Copy link
Member Author

anonrig commented Jun 30, 2024

Does this affect all Windows users or just a subset? I'm a bit surprised we haven't heard more complaints about this if it does affect all users.

I installed it through nodejs website and running on windows 11. I assume it is all users.

@hzhang20902
Copy link

I'm getting the following error when trying to run a simple npx command on Windows. I opened the issue to this repository because we are distributing npm, we define the location of npm in a Windows machine.

I reinstalled Node.js from nodejs.org with the latest v22, and I can not use npm at all. Any input/help is appreciated.

PS: I'm using PowerShell.

PS C:\Users\yagiz\Desktop\coding\node> npx node-benchmark-compare .\bench.txt
npm error code ENOENT
npm error syscall lstat
npm error path C:\Users\yagiz\AppData\Roaming\npm
npm error errno -4058
npm error enoent ENOENT: no such file or directory, lstat 'C:\Users\yagiz\AppData\Roaming\npm'
npm error enoent This is related to npm not being able to find a file.
npm error enoent
npm error A complete log of this run can be found in: C:\Users\yagiz\AppData\Local\npm-cache\_logs\2024-06-21T20_29_57_155Z-debug-0.log

cc @nodejs/platform-windows

This issue has happened a few times in recent years, and has been answered but not really solved.

npm i -g npm is the cleanest way to fix it

@StefanStojanovic
Copy link
Contributor

Does this affect all Windows users or just a subset?

This should only affect users that never installed Node.js while installer was making that folder. As I recall when uninstalling Node, that directory wasn't removed, so it'd stay there unless you manually delete it, which I doubt a lot of people did.

@tniessen
Copy link
Member

tniessen commented Jul 1, 2024

Creating the folder from the MSI is problematic for multiple reasons, not only because of security concerns. I don't think this is something we can cleanly fix inside the installer, except through some user instructions or best-effort approach.

@RedYetiDev
Copy link
Member

nodejs/help#4436 seems to be reporting a similar issue, are they related?

@anonrig
Copy link
Member Author

anonrig commented Jul 5, 2024

It has been 2 weeks, and there is still no solution available from NPM team. @nodejs/npm @nodejs/tsc

@anonrig anonrig added the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Jul 5, 2024
@hzhang20902
Copy link

It has been 2 weeks, and there is still no solution available from NPM team. @nodejs/npm @nodejs/tsc

No it's been years, but if you feel so strongly about this, you can fix it yourself as it is open source and you can submit code for review. Otherwise, just gonna have to live with:

npm i -g npm

@RedYetiDev
Copy link
Member

Does npm.cmd work?

@tniessen
Copy link
Member

tniessen commented Jul 5, 2024

It has been 2 weeks, and there is still no solution available from NPM team

If 0ae8bf8 did indeed re-introduce this issue, then it's been there for more than a year already. I am not sure what the TSC can do about this since we don't maintain npm and no solution has been suggested within Node.js itself.

@anonrig
Copy link
Member Author

anonrig commented Jul 5, 2024

@tniessen It means we are shipping unmaintained code for more than a year. How does this don't concern TSC?

@targos
Copy link
Member

targos commented Jul 5, 2024

What do you want to discuss in a meeting?

@benjamingr
Copy link
Member

How does this don't concern TSC?

You're going to have to accept the fact this isn't in fact a high severity bug from most people's PoV given you're the first one to complain about it afaict and it's been this way for about a year and it's a bug in a third-party program that can be fixed in that program (and it looks like it isn't a high priority for them either).

When stuff breaks people typically complain very quickly, if you believe this is in error tagging this tsc-agenda multiple times and pinging people multiple times won't help.

You know exactly what happens when this gets brought up in a TSC meeting:

  • Michael says "ok, the next item on our agenda is npm is installed wrongly on windows machine #53538, anyone here has context to give".
  • Someone volunteers and says "Yagiz asked to add this to the meeting agenda because he thinks it's important and the TSC should look into fixing it".
  • Everyone says "ok", people mention stuff we all already know from this issue like why creating the path was removed (security), the pain (npm can create that path or users need to run another command) etc.
  • someone adds a comment to the meeting notes, nothing gets fixed.

The TSC is a leadership body not the owner of Node.js, collaborators own the code. You need to motivate someone to fix it or to fix it yourself. You were a TSC voting member you realize how the process works.

You have a lot of trust, friends and good faith in the project - I feel silly telling you all of this but I feel like you're needlessly frustrated around this all so I felt it's all worth explaining super clearly.

@hzhang20902
Copy link

nodejs/help#4436 seems to be reporting a similar issue, are they related?

don't think so, their issue was non-output to Powershell using npm vs npm.cmd, so most likely an AutoRun config is on their global or user scope for that particular command path. but if their issue wasn't caused by user error or Chocolately/3rd party package manager install (ie at some point the default was changed to enable AutoRun), then it probably would be related as that and this issue affect the same type of system process for command aliasing, however that works under the hood.

@anonrig
Copy link
Member Author

anonrig commented Jul 9, 2024

You're going to have to accept the fact this isn't in fact a high severity bug from most people's PoV given you're the first one to complain about it

@benjamingr I'm not the first one complaining about it. I'm just the first one raising it to the TSC. If you don't think this is not true, here's the result of a quick Google search about this exact problem in the past year and a half.

Similar issues:

Stack Overflow:

Youtube:

Reddit:

More websites:

@mhdawson
Copy link
Member

Was on the TSC agenda as an FYI to raise awareness. It was discussed in meeting today so taking offf the agenda.

@RafaelGSS RafaelGSS removed the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Jul 10, 2024
@joyeecheung
Copy link
Member

joyeecheung commented Jul 10, 2024

I don't know enough about Windows so pardon my ignorance, but why exactly can't npm create %appdata%\npm when it's missing? nodejs/node-v0.x-archive#8141 described it as

npm uses $APPDATA\npm so that non-Administrator roles have a writable place to stash global packages. Since creating that directory if it doesn't exist does require Administrator privileges, npm -g install won't / can't work if the base directory doesn't exist (with the correct ACL) before npm is run. The installer should ensure that the whole path is set up at boot time.

I don't have a Windows machine right now so can't verify but my impression is that creating a folder at %appdata% doesn't require administrator privilege if running user already has access to it (that is, it's not trying to create a folder in the administrator's %appdata% folder when the user running npm isn't administrator). Does that have something to do with the installer having to be run as administrator because it installs things to the program files folder by default? But then the windows installer is not the only way you install Node.js on Windows. nvs, for example, installs Node.js to %localappdata%, so there are no administrator privileges required during installation. It seems fine for npm to at least try creating %appdata\npm if it doesn't exist, and only fail when it cannot actually create that folder, instead of always counting on someone else to create that folder because it assumes only administrators are allowed to do so? (but why?)

@leobalter
Copy link
Contributor

Hi everyone! This might be my first time in a Node forum as the (new) PM for npm. I'm glad to be here!

Thanks everyone for raising the concerns and the good discussion. I've briefly watched the TSC recording and I agree there are areas for improvement. While the npm i -g npm does the trick, it's not ideal to live with an extra step workaround.

My team was able to pick this up and a fix should be on the way.

@electrovir
Copy link

electrovir commented Aug 2, 2024

Oddly, I was unable to get around this issue in GitHub Actions until I ran npm i npm, installing npm directly into my repository. I tried npm i -g npm and it didn't help.

@anonrig anonrig closed this as completed Aug 2, 2024
@anonrig
Copy link
Member Author

anonrig commented Aug 2, 2024

Issue is fixed with npm/cli#7640

@sxlijin
Copy link

sxlijin commented Sep 16, 2024

Thanks @anonrig for making sure that this got solved! Our users are running into this issue right now, so I'm glad to see that a fix is in place.

I'm a bit confused about why it's still happening right now, though, on new installs of Node.js: npm/cli#7640 was merged in July and got released in NPM 10.8.3, if I'm reading npm/cli@9214be9 correctly.

The various installers currently appear to still ship 10.8.2 by default though (see below image) - I assume because there's some bake time before the next NPM release gets promoted up, but can I ask when NPM 10.8.3 will be shipped by default in the Node.js installer?

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
npm Issues and PRs related to the npm client dependency or the npm registry. windows Issues and PRs related to the Windows platform.
Projects
None yet
Development

No branches or pull requests