grpc-tools install fails roughly 50% of the time with "bad download"

[michaelbeaumont]

Problem description

When installing, node-pre-gyp throws a bad download error.

Reproduction steps

git clone https://github.com/grpc/grpc-node && cd grpc-node/packages/grpc-tools
npm i until it fails

Dockerfile:

FROM node:13.13.0
RUN npm install --unsafe-perm -g grpc-tools

Environment

• Linux Arch amd64
• 13.12 & 13.13
• nvm
• grpc-tools@1.8.1

but we're also seeing this on Google Cloud Build and Github Actions

Additional context

$ npm install
> grpc-tools@1.8.1 install /home/mike/projects/gastromatic/grpc-node/packages/grpc-tools
> node-pre-gyp install

node-pre-gyp WARN Using needle for node-pre-gyp https download 
node-pre-gyp ERR! install error 
node-pre-gyp ERR! stack Error: bad download
node-pre-gyp ERR! stack     at afterExtract (/home/mike/projects/gastromatic/grpc-node/packages/grpc-tools/node_modules/node-pre-gyp/lib/install.js:98:46)
node-pre-gyp ERR! stack     at Unpack.afterTarball (/home/mike/projects/gastromatic/grpc-node/packages/grpc-tools/node_modules/node-pre-gyp/lib/install.js:178:9)
node-pre-gyp ERR! stack     at Unpack.emit (events.js:315:20)
node-pre-gyp ERR! stack     at Unpack.[maybeClose] (/home/mike/projects/gastromatic/grpc-node/packages/grpc-tools/node_modules/tar/lib/unpack.js:171:12)
node-pre-gyp ERR! stack     at Unpack.[unpend] (/home/mike/projects/gastromatic/grpc-node/packages/grpc-tools/node_modules/tar/lib/unpack.js:394:21)
node-pre-gyp ERR! stack     at /home/mike/projects/gastromatic/grpc-node/packages/grpc-tools/node_modules/tar/lib/unpack.js:308:46
node-pre-gyp ERR! stack     at FSReqCallback.oncomplete (fs.js:159:23)
node-pre-gyp ERR! System Linux 5.6.4-arch1-1
node-pre-gyp ERR! command "/usr/bin/node" "/home/mike/projects/gastromatic/grpc-node/packages/grpc-tools/node_modules/.bin/node-pre-gyp" "install"
node-pre-gyp ERR! cwd /home/mike/projects/gastromatic/grpc-node/packages/grpc-tools
node-pre-gyp ERR! node -v v13.13.0
node-pre-gyp ERR! node-pre-gyp -v v0.14.0
node-pre-gyp ERR! not ok 
bad download
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! grpc-tools@1.8.1 install: `node-pre-gyp install`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the grpc-tools@1.8.1 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/mike/.npm/_logs/2020-04-15T16_48_03_193Z-debug.log

The request is emitting the err event with:

Error: Remote end closed socket abruptly

[badsyntax]

I started experiencing this once a new LTS node version was released.

To fix, use a known working node version (for which grpc-tools binaries are available), for example 12.16.2. I wouldn't use non-LTS for CI.

[NikoGrano]

Weird issue 12.16.2 works fine. However patch version 12.16.3 fails. How is it possible node always breaks so easily...

[Alabate]

Seems that this "fix" might also work : #922 (comment)

Edit: found the issue in needle: tomas/needle#312

[murgatroid99]

I have published a new version of grpc-tools, which should pick up the new version of needle and fix this issue.

[bsyk]

This has surfaced again today with node v12.18.0 for me.
As suggested in the comment above, installing request first circumvents the issue.
My grpc-tools are at 1.9.0

[netpoetica]

I am using 14.6.0, latest LTS, and this is the case for me as well with version 1.8.1 of grpc-tools in my package.json. It is 1/2 to 1/3 of the time the same failure.

$ npm i

> grpc-tools@1.8.1 install C:\Users\keithl\repos\MY_PROJECT\node_modules\grpc-tools
> node-pre-gyp install

node-pre-gyp WARN Using needle for node-pre-gyp https download
node-pre-gyp ERR! install error
node-pre-gyp ERR! stack Error: bad download
node-pre-gyp ERR! stack     at Unpack.afterTarball (C:\Users\keithl\repos\MY_PROJECT\node_modules\grpc-tools\node_modules\node-pre-gyp\lib\instal
l.js:100:46)
node-pre-gyp ERR! stack     at Unpack.emit (events.js:315:20)
node-pre-gyp ERR! stack     at Unpack.[maybeClose] (C:\Users\keithl\repos\MY_PROJECT\node_modules\grpc-tools\node_modules\tar\lib\unpack.js:171:1
2)
node-pre-gyp ERR! stack     at Unpack.[unpend] (C:\Users\keithl\repos\MY_PROJECT\node_modules\grpc-tools\node_modules\tar\lib\unpack.js:394:21)
node-pre-gyp ERR! stack     at C:\Users\keithl\repos\MY_PROJECT\n
ode_modules\grpc-tools\node_modules\tar\lib\unpack.js:308:46
node-pre-gyp ERR! stack     at FSReqCallback.oncomplete (fs.js:171:23)
node-pre-gyp ERR! System Windows_NT 10.0.18363
node-pre-gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\keithl\\repos\MY_PROJECT\\node_modules\\grpc-tools\\node_modules\\n
ode-pre-gyp\\bin\\node-pre-gyp" "install"
node-pre-gyp ERR! cwd C:\Users\keithl\repos\MY_PROJECT\node_modules\grpc-tools
node-pre-gyp ERR! node -v v14.16.0
node-pre-gyp ERR! node-pre-gyp -v v0.12.0
node-pre-gyp ERR! not ok
bad download

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! grpc-tools@1.8.1 install: `node-pre-gyp install`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the grpc-tools@1.8.1 install script.
npm ERR! This is probably not a problem with npm. There is likely additional log
ging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\keithl\AppData\Roaming\npm-cache\_logs\2021-03-03T18_21_18
_131Z-debug.log

Re-running npm i 1-2 times seems to cause it to correct itself - sometimes it will fail too many times in a row, and only a rm package-lock.json seems to do the trick.

[murgatroid99]

The new version of grpc-tools that I referred to in my previous comment was version 1.9.0. The latest version is 1.10.0. You are still experiencing the bug because you are installing version 1.8.1, which does not include the fix for this bug.