[BUG] npm install fails on scoped package override with cryptic "Invalid comparator" error

[gthb]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

We have a package override in our project that's working OK in NPM 8 (after some workarounds for #5007):

  "overrides": {
    "pgsql-ast-parser": "@grid-is/pgsql-ast-parser@^111.0.0"
  },

Upgrading npm to 9.1.1 or 9.2.0 makes npm install fail in this project, with:

npm ERR! Invalid comparator: @grid-is/pgsql-ast-parser@^111.0.0

and in the debug logfile:

35 verbose shrinkwrap failed to load node_modules/.package-lock.json out of date, updated: node_modules/@grid-is/pgsql-ast-parser
36 timing idealTree Completed in 1300ms
37 timing command:i Completed in 1305ms
38 verbose stack TypeError: Invalid comparator: @grid-is/pgsql-ast-parser@^111.0.0
38 verbose stack     at Comparator.parse (/Users/gthb/.nvm/versions/node/v16.18.0/lib/node_modules/npm/node_modules/semver/classes/comparator.js:38:13)
38 verbose stack     at new Comparator (/Users/gthb/.nvm/versions/node/v16.18.0/lib/node_modules/npm/node_modules/semver/classes/comparator.js:22:10)
38 verbose stack     at /Users/gthb/.nvm/versions/node/v16.18.0/lib/node_modules/npm/node_modules/semver/classes/range.js:133:47
38 verbose stack     at Array.map (<anonymous>)
38 verbose stack     at Range.parseRange (/Users/gthb/.nvm/versions/node/v16.18.0/lib/node_modules/npm/node_modules/semver/classes/range.js:133:35)
38 verbose stack     at /Users/gthb/.nvm/versions/node/v16.18.0/lib/node_modules/npm/node_modules/semver/classes/range.js:34:22
38 verbose stack     at Array.map (<anonymous>)
38 verbose stack     at new Range (/Users/gthb/.nvm/versions/node/v16.18.0/lib/node_modules/npm/node_modules/semver/classes/range.js:34:8)
38 verbose stack     at Object.intersects (/Users/gthb/.nvm/versions/node/v16.18.0/lib/node_modules/npm/node_modules/semver/ranges/intersects.js:3:8)
38 verbose stack     at OverrideSet.getEdgeRule (/Users/gthb/.nvm/versions/node/v16.18.0/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/override-set.js:53:18)
39 verbose cwd /Users/gthb/git/GRID-client
40 verbose Darwin 22.1.0
41 verbose node v16.18.0
42 verbose npm  v9.2.0
43 error Invalid comparator: @grid-is/pgsql-ast-parser@^111.0.0

Expected Behavior

I expect npm install to succeed with NPM 9 like it does with NPM 8 (or else output a more informative error message, if there is something wrong with the package override that NPM 8 was not revealing).

Steps To Reproduce

I would extract minimal steps to reproduce this, except, I put in that work in #5007 and that's been totally ignored for half year. I'll happily put in the work here if there's a hint that someone cares for it. :)

Environment

• npm: 9.2.0
• Node.js: 16.18.0
• OS Name: Darwin 22.1.0
• System Model Name: Macbook Pro
• npm config:

; "user" config from /Users/gthb/.npmrc

@grid-is:registry = "https://registry.npmjs.org/" 
//registry.npmjs.org/:_authToken = (protected) 

; "project" config from /Users/gthb/git/GRID-client/.npmrc

engine-strict = true 

; "cli" config from command line options

location = "project" 

; node bin location = /Users/gthb/.nvm/versions/node/v16.18.0/bin/node
; node version = v16.18.0
; npm local prefix = /Users/gthb/git/GRID-client
; npm version = 9.2.0
; cwd = /Users/gthb/git/GRID-client
; HOME = /Users/gthb
; Run `npm config ls -l` to show all defaults.

[gthb]

This may well be related to #5843 but I'm guessing it is distinct in that this one has nothing to do with file paths.

[neulad]

It seems that this issue is quite fresh, because I started to get it only few days ago too. What is the latest version without this issue, it would be very helpful because it literally stops me from building my project now :(

[everett1992]

Here's a simple reproduction. I used different versions of react and react-dom to force a ERESOLVE. react-keyed-flatten-children has a devDependnecy on react ^16.12.00 which is not a valid semver range.

$ cat <<-HERE > package.json
{
  "dependencies": {
    "react": "^18.0.0",
    "react-dom": "^17.0.0",
    "react-keyed-flatten-children": "^1.0.0"
  },
  "overrides": {
    "react": "\$react"
  }
}
HERE

$ npx npm@9.2.0 -- install
npm ERR! Invalid comparator: ^16.12.00

npm ERR! A complete log of this run can be found in:
npm ERR!     ~/.npm/_logs/2022-12-30T01_03_13_310Z-debug-0.log

And the full stack trace from the debug log.

TypeError: Invalid comparator: ^16.12.00
    at Comparator.parse (npm/node_modules/semver/classes/comparator.js:38:13)
    at new Comparator (npm/node_modules/semver/classes/comparator.js:22:10)
    at npm/node_modules/semver/classes/range.js:133:47
    at Array.map (<anonymous>)
    at Range.parseRange (npm/node_modules/semver/classes/range.js:133:35)
    at npm/node_modules/semver/classes/range.js:34:22
    at Array.map (<anonymous>)
    at new Range (npm/node_modules/semver/classes/range.js:34:8)
    at Object.intersects (npm/node_modules/semver/ranges/intersects.js:3:8)
    at OverrideSet.getEdgeRule (npm/node_modules/@npmcli/arborist/lib/override-set.js:53:18)

npm 8 installs the package fine.

[trusktr]

Same here. NPM 8 works fine. I added a reproduction here: #5843 (comment)

[IGx89]

FYI, #5843 was fixed in v9.3.0 and v9.3.0 has fixed this issue for us as well.

[gthb]

For me, with v9.3.0 and v9.3.1, npm install now fails with:

npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path /Users/gthb/git/GRID-client/node_modules/pg-mem/@grid-is/pgsql-ast-parser@^111.0.0/package.json
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, open '/Users/gthb/git/GRID-client/node_modules/pg-mem/@grid-is/pgsql-ast-parser@^111.0.0/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent 

... instead of the Invalid comparator message. Note that once again the @grid-is/pgsql-ast-parse package is being incorrectly looked for inside node_modules/pg-mem/ but should be looked for directly under node_modules/ (or else under the symlink node_modules/pg-mem/node_modules

I tried rm -rf package-lock.json node_modules and then again npm install, and then it fails on a different incorrect path:

npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path /Users/gthb/git/GRID-client/@grid-is/pgsql-ast-parser@^111.0.0/package.json
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, open '/Users/gthb/git/GRID-client/@grid-is/pgsql-ast-parser@^111.0.0/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent 

... where note that node_modules/ is missing after the GRID-client/ project root directory. Looks like this went two levels up, skipping past not only pg-mem/ but also node_modules/.