Installing npm dependency from public GitHub repository fails

[TimDaub]

In one of my projects I use simple-caldav which contains the following line in its package.json:

dependencies: {
  "ical.js": "github:TimDaub/ical.js#feat/detect-module-mode-build",
  ...
}

It points to a branch here. I've submitted a PR to the upstream repo, but it seems they're not having much time for maintenance.

Anyways, my GH action in the project that has simple-caldav as a dependency looks like this

# This workflow will do a clean install of node dependencies, build the source code and run tests across different versions of node
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions

name: Node.js CI

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [10.x, 12.x, 14.x]

    steps:
    - uses: actions/checkout@v2
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v1
      with:
        node-version: ${{ matrix.node-version }}
    - run: npm ci
    - run: npm test

However, when it runs npm ci, it fails like this

npm ERR! Error while executing:
npm ERR! /usr/bin/git ls-remote -h -t ssh://git@github.com/TimDaub/ical.js.git
npm ERR! 
npm ERR! Warning: Permanently added the RSA host key for IP address '140.82.113.4' to the list of known hosts.
npm ERR! git@github.com: Permission denied (publickey).
npm ERR! fatal: Could not read from remote repository.
npm ERR! 
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.
npm ERR! 
npm ERR! exited with error code: 128

[Antonio-Laguna]

@TimDaub did you find any workarounds?

[TimDaub]

Nope, I'm quite confused by this problem. E.g. why does it say git ls-remote? Is npm using git internally?
I could imagine another notation within package.dependencies could make a difference. But I haven't tested that yet.

[Antonio-Laguna]

Thought the same and even tried that locally and works fine locally

[Antonio-Laguna]

@TimDaub I ended up releasing the forks as my own libraries publicly scoped to NPM since a project was breaking deployments because of this issue. It's a workaround but it's something :)

[carlocorradini]

@TimDaub

Same issue here.

A very annoying problem...

Have you find any solution?

[TimDaub]

ended up releasing the forks as my own libraries publicly scoped to NPM since a project was breaking deployments because of this issue. It's a workaround but it's something :)

Unfortunately, it's not an option I have.

[TimDaub]

What ended up fixing it for me is adding the unknown host in my ssh config before npm ci:

...
- run: mkdir -p $HOME/.ssh/ && echo "140.82.113.4" >> $HOME/.ssh/known_hosts
- run: npm ci
...

It's far from perfect, but works well as a work around for now. Additionally, disabling ssh's key checking via config may be an option too. I prefer to go with this more narrow solution.

Edit: Turns out this won't work all the time as the IPs that the package is requested from change

• 140.82.113.*
• 140.82.112.*
• 140.82.114.*

I've tested adding ranges and ssh-keyscan, but so far I wasn't successful.

Edit2:

I think I finally ended up solving it for good. This is what you'll have to do:

1. Backup your current RSA keypair at ~/.ssh
2. Generate a new RSA keypair on your system ssh-keygen -t rsa -C "your_email@example.com". Ideally don't overwrite your existing keypair at ~/.ssh by entering a custom path.
3. Take the contents of the generated *.pub key and add it to your SSH keys in your GitHub account settings
4. In your repo that has the action, navigate to Settings > Secrets and add SSH_PRIVATE_KEY the contents of the private key file that was generated
5. Then in your repo's workflow file, add the following before -run: npm ci

...
- uses: webfactory/ssh-agent@v0.4.1
   with:
     ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
- run: npm ci
...

For more details, check https://github.com/webfactory/ssh-agent

[Siilwyn]

This is still an issue... An alternative solution is replacing the resolved url in the package lock file, for example:
git+ssh://git@github.com/zspecza/common-tags.git#946fcbf8cfc1a14c2183ef5a81b23727a2b531e3 becomes: git+https://git@github.com/zspecza/common-tags.git#946fcbf8cfc1a14c2183ef5a81b23727a2b531e3

[natekwilson]

Having the same issue, but within my workflows' docker build step. There has to be easier ways to disseminate SSH agent keys/known hosts info to different contexts when SSH-git actions are so commonplace

[daonb]

This is still a bug. I'm now going to implement the workaround @TimDaub suggested and it'll probably work but still... It hit me just as I released an version and I had no time to fiddle with private keys so github action can read public github URLs. It was a bummer but I released a version whose README starts with "test: failing"

Please fix this as the attached screenshot does not compute.

[akaltar]

For me this issue started occurring when I tried to switch from npm install to npm ci so for some of you switching to npm install may be another workaround.

[Xunnamius]

I fixed this in my workflows by adding an extra step after the actions/checkout@v2 (with persist-credentials: false) step:

      - name: Checkout
        uses: actions/checkout@v2
        with:
          persist-credentials: false

      - name: Reconfigure git to use HTTP authentication
        run: >
          git config --global url."https://github.com/".insteadOf
          ssh://git@github.com/

Changing from SSH to HTTP makes everything work across all workflows using npm ci (which has several benefits over npm install). If you need to authenticate, use a PAT instead of SSH:

git config --global url."https://${{ secrets.GH_TOKEN }}@github.com/".insteadOf ssh://git@github.com/

[TimDaub]

An update on my earlier workaround in this thread. A problem that I've discovered is that according to GitHub settings:

Secrets are not passed to workflows that are triggered by a pull request from a fork.

Hence, it becomes useless when trying to collaborate with others.

[wallind]

I fixed this in my workflows by adding an extra step after the actions/checkout@v2 (with persist-credentials: false) step:

      - name: Checkout
        uses: actions/checkout@v2
        with:
          persist-credentials: false

      - name: Reconfigure git to use HTTP authentication
        run: >
          git config --global url."https://github.com/".insteadOf
          ssh://git@github.com/

Changing from SSH to HTTP makes everything work across all workflows using npm ci (which has several benefits over npm install). If you need to authenticate, use a PAT instead of SSH:

git config --global url."https://${{ secrets.GH_TOKEN }}@github.com/".insteadOf ssh://git@github.com/

this did fix my problem so thank you for that. Out of curiosity though is there any security lost by doing this? I don't care enough to not use this fix for the project I need it on but I am left wondering.

[andreyvolokitin]

@AndrewJHart the only thing I don't understand is how the github account is relevant here. If CI can be triggered without any action from this specific account — why it even considers this account's SSH key? The SSH connection is between github action runner and github repo, I don't even understand where/when gihub account could become relevant here, but it indeed works (CI is triggered by pushing git tags by my account, I wonder if it will stop working if different account will be pushing tags)

[jdmarshall]

I haven't had any luck getting the workarounds to succeed for this.

There are several very worrying aspects of this bug.

1. npm ci returns a success status when an error has clearly occurred
2. there are no hints in --verbose about how or if it's trying to fetch the files