[BUG] NPM ci seems to be broken on Azure DevOps

[terwort]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

When running npm ci on Azure Devops with the node version set to 16.14.2, and npm version set to 8.5.0 we get The npm ci command can only install with an existing package-lock.json. Even though we definitely have a package-lock file.

Reverting our node version back to 16.14.0, and npm to 8.3.1 fixes the issue with no other changes required.

Expected Behavior

I would expect npm ci to complete successfully.

Steps To Reproduce

I'm not sure exactly the reproduction steps. I just noticed on our Azure Devops pipeline it fails npm ci as mentioned in the Current Behavior section.

Environment

Environment: ubuntu-20.04
Version: 20220330.0
npm: 8.5.0
node: 16.14.2

[ljharb]

Any chance you have an env var configured with “npm” in the name, case-insensitively?

[terwort]

@ljharb Not that I can find. This is one of the yaml files used in our pipelines, and it fails on the npm ci step. So all its done so far is set the node version and then immediately call npm ci.

`jobs:

• job: Job_2
  displayName: ESLint
  pool:
  vmImage: ubuntu-latest
  steps:
  • task: NodeTool@0
    inputs:
    versionSpec: "16.x"
    displayName: "Set node version"
  • task: Npm@1
    displayName: npm ci preamble-ui-hcms
    inputs:
    command: ci
    workingDir: packages/preamble-ui-hcms
    verbose: false
  • task: Npm@1
    displayName: npm ci testing
    inputs:
    command: ci
    workingDir: packages/testing
    verbose: false
  • task: Npm@1
    displayName: npm run eslint:hcms
    inputs:
    command: custom
    workingDir: packages/testing
    verbose: false
    customCommand: run eslint:hcms
  • task: PublishTestResults@2
    displayName: Jest Test Results
    condition: succeededOrFailed()
    inputs:
    testResultsFiles: junit.xml
    searchFolder: packages/testing
    failTaskOnFailedTests: true
    testRunTitle: ESLint HCMS
    `

[ljharb]

And which CI command is failing (in which dir)?

[terwort]

This is the path azure devops is using /opt/hostedtoolcache/node/16.14.2/x64/bin/npm ci

[ljharb]

No i mean, your yaml file runs npm ci multiple times, in different dirs. Which one fails?

[terwort]

the first one in preamble-ui-hcms. Although, it fails on about 6 of our yaml files all in different directories with their own package.json files.

[jonofoz]

@terwort My team has been having the same issue. Today I tried npm ci --legacy-peer-deps locally, which allowed me to circumvent the issues we were having. Sticking that in the pipeline, for you it'd be something like

task: Npm@1
  displayName: [something]
  inputs:
    command: custom
    customCommand: "ci --legacy-peer-deps"
    workingDir: [something]
  [other stuff here]
  [other stuff here]
  [...]

I just did something similar and my pipelines using ci are passing now.

Try that out! Please let me know if that works for you.

[Y-LyN-10]

We are experiencing a similar issue with GitHub Actions that could be related to this one. In our case the error is that 'npm ci' can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with 'npm install' before continuing and then there is a list of "missing" libraries from the lock file.

Such error is not true because they ARE in sync (nothing changes if I do npm install) and reverting back to Node v16.14.0 (with older NPM version) is our only solution so far. There is no error there. The worst thing is that such breaking change creeped into a patch version of Node.js.

Let me know if I should open a new issue for that or/and if I can help you to reproduce the issue.

[ljharb]

@Y-LyN-10 are you sure you’re using the same node and npm version locally and in production?

[nlf]

We are experiencing a similar issue with GitHub Actions that could be related to this one. In our case the error is that 'npm ci' can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with 'npm install' before continuing and then there is a list of "missing" libraries from the lock file.

Such error is not true because they ARE in sync (nothing changes if I do npm install) and reverting back to Node v16.14.0 (with older NPM version) is our only solution so far. There is no error there. The worst thing is that such breaking change creeped into a patch version of Node.js.

Let me know if I should open a new issue for that or/and if I can help you to reproduce the issue.

this issue you're describing is not the same as the original one, and i think #4664 will help you out

[nlf]

this is an interesting one. can you reproduce this in a public repository you can share with us? the validation steps that have brought to light issues with npm ci for other folks weren't introduced until npm@8.6.0 while you're on 8.5.0.

[lukekarrys]

Let me know if I should open a new issue for that or/and if I can help you to reproduce the issue.

@Y-LyN-10 If you can share a public repository or reproduction that would be very helpful! Opening a new issue would be best, since it might not be the same issue, and we can look in to it. Thanks!

[lukekarrys]

@terwort Did the suggestion from #4660 (comment) to use --legacy-peer-deps (or to use the same flags locally as you do in azure with npm ci) work in your case? We've added some documentation now to highlight this (#4666). Please let me know if you're still having issues with this!

[Y-LyN-10]

Sorry for the late response!

Yes, I believe our issue is different - npm ci does not work for us with lockfile ver 1. --legacy-peer-deps doesn't help. We are testing with all major LTS versions of Node.js, so we are kinda stuck to old lockfile format (re-creating it with with newer npm version via npm install solves the problem, but it's not an option for us due to other NPM issues) and all tests work fine except for Node.js >=16.14.2. Our workaround is to pin the NPM version to v8.3.1 for Node 16.x.x. After all - the purpose of these tests is to test Node, not NPM.

However, if you are still interested in that issue and believe that NPM >=8.5.0 should be compatible with old lockfiles, I can try to create a public repository to reproduce it and I will open another issue for that. Thanks!

[swenson]

We ran into this as well in GitHub Actions for Node 16.14.2, but reverting to 16.14.0 fixed it.

[wraithgar]

However, if you are still interested in that issue and believe that NPM >=8.5.0 should be compatible with old lockfiles, I can try to create a public repository to reproduce it and I will open another issue for that. Thanks!

Yes npm should be fully backwardly compatible with old lockfiles. If you have a way to reproduce an error please open a new issue