cliui git dependency breaks lerna installation

[richallanb]

It appears the introduction of cliui using a git path for the dependency has broken installing lerna on npm 8 and 9.
https://github.com/isaacs/jackspeak/blob/main/package.json#L70

richard.bull@MSAN42AD2EED git % npm i -g lerna
npm WARN deprecated @npmcli/move-file@2.0.1: This functionality has been moved to @npmcli/fs
npm ERR! code 127
npm ERR! git dep preparation failed
npm ERR! command /Users/richard.bull/.nvm/versions/node/v16.16.0/bin/node /Users/richard.bull/.nvm/versions/node/v16.16.0/lib/node_modules/npm/bin/npm-cli.js install --force --cache=/Users/richard.bull/.npm --prefer-offline=false --prefer-online=false --offline=false --no-progress --no-save --no-audit --include=dev --include=peer --include=optional --no-package-lock-only --no-dry-run
npm ERR! npm WARN using --force Recommended protections disabled.
npm ERR! npm ERR! code 127
npm ERR! npm ERR! path /Users/richard.bull/.npm/_cacache/tmp/git-cloneFiix7g
npm ERR! npm ERR! command failed
npm ERR! npm ERR! command sh -c npm run compile
npm ERR! npm ERR! > cliui@8.0.1 precompile
npm ERR! npm ERR! > rimraf build
npm ERR! npm ERR! npm WARN using --force Recommended protections disabled.
npm ERR! npm ERR! sh: rimraf: command not found
npm ERR!

Tested on node v16 and v18, with npm 9.6.5 and 8.19.4

[DavidAnson]

Agreed - @richallanb beat me by an hour (nice!), but I think I'm getting reports of the same problem for the markdownlint-cli project, see here: igorshubovych/markdownlint-cli#394. (Includes a link to a similar report on Stack Overflow.) It's only an issue when installing --global for reasons I don't yet know.

[DavidAnson]

Not sure you need receipts, but here's some supporting evidence:

node@7ed5037cb4c0:/home/workdir$ npm ls cliui
workdir@ /home/workdir
`-- markdownlint-cli@0.34.0
  `-- glob@10.2.2
    `-- jackspeak@2.1.4
      `-- cliui@8.0.1 (git+ssh://git@github.com/isaacs/cliui.git#9f97090165675fdda63a79c29bc36bb1033506b0)

npm http fetch GET 200 https://registry.npmjs.org/color-name 44ms (cache revalidated)
npm http fetch GET 200 https://codeload.github.com/isaacs/cliui/tar.gz/9f97090165675fdda63a79c29bc36bb1033506b0 445ms (cache revalidated)
npm verb stack Error: git dep preparation failed
npm verb stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/@npmcli/promise-spawn/lib/index.js:53:27)

[isaacs]

I can roll it back, but what's the problem with git deps? npm has supported git deps for ages, I've used them in published packages lots of times.

[isaacs]

I'll just publish it as a normal npm fork for now I guess. But cliui failing prepare seems like an npm bug to me.

[richallanb]

I can roll it back, but what's the problem with git deps? npm has supported git deps for ages, I've used them in published packages lots of times.

@isaacs Yeah I think you're right. This actually worked perfectly fine in npm 6.

[6XGate]

It seems there is a circular dependencies between cliui -> rimraf -> glob -> jackspeak -> cliui

I started having this issue with npm-check-updates

[isaacs]

@6XGate one issue is enough, no need to repost this on everything that uses glob. (Everything uses glob, you'll be busy for some time.)

[isaacs]

But yes, the cycle is what I was missing. It can't build cliui without rimraf, which needs glob, which needs jackspeak, which needs cliui, so it tries to run the prepare before it has what it needs.

[6XGate]

It's weird you can install cliui without issue by itself.

[isaacs]

It's weird you can install cliui without issue by itself.

That's because cyclical deps are normally fine, npm handles them no problem. And in fact, they're usually fine when installing deps from git, as well.

Because a git repository typically doesn't have built artifacts, in order to install a git dep, npm has to do a little bit of extra work. It clones the repo, installs its dev dependencies, and then runs the prepare and prepublish scripts, the same as it would when publishing, so that the end result is the same sort of tarball artifact that you'd get from the registry.

Unfortunately, in this case, cliui depends on rimraf, and uses the rimraf bin in its prepare script. Rimraf depends on glob, which depends on jackspeak, which depends on that same git version of cliui.

npm usually is fairly clever about building dependencies before their dependents. However, once it encounters a cycle, it has to pick something, and the algorithm it uses it "upon a cycle, don't continue to recurse, just hope for the best". It's easy to see with our human brains that if it had chosen to build rimraf prior to building cliui, it would have worked, but what if rimraf had required glob to do its build? Then choosing to build rimraf first would have failed in the same way.

Bootstrap problems are tricky.

Anyway, published a new jackspeak that depends on @isaacs/cliui instead of the git fork. Whenever they resolve that string wrapping issue in esm, I'll deprecate it and move back to using main.

[ljharb]

Thanks for fixing; fyi the use of aliases also is still breaking installation of my dev dep that's pulling this in.