Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

node-gyp keeps failing unreliably on github windows ci. #2683

Closed
nodech opened this issue Jun 18, 2022 · 10 comments · Fixed by #2846
Closed

node-gyp keeps failing unreliably on github windows ci. #2683

nodech opened this issue Jun 18, 2022 · 10 comments · Fixed by #2846
Labels
bcrypt Windows

Comments

@nodech
Copy link

nodech commented Jun 18, 2022

  • Node Version: 12.x - 18.x

  • Platform: Windows (System Windows_NT 10.0.20348) Github Actions

  • Compiler: VS2022 (17.2.32602.215)

  • Module: https://github.com/nodech/hsd/actions/runs/2515981257

    This issue happens in the Github CI with windows-latest os, everything else is fine.
    Setup is the matrix of windows with multiple nodejs version from v12 to v18. Because different version of npm are shipped with different versions of node, first step CI does is updates npm so project has latest node-gyp that's shipped with npm. Unfortunately, all of them fail, BUT even worse is they fail unreliably.. Sometimes test will pass, but most of the time it fails. So I am not sure if it's issue with node-gyp or not. But all of the failures happen when looking for common.gypi file.
    Looking at the code, it seems common.gypi file is not being found at C:\\Users\\runneradmin\\AppData\\Local\\node-gyp\\Cache\\16.15.1\\include\\node\\common.gypi and then it fallbacks to what is shown in the output C:\\Users\\runneradmin\\AppData\\Local\\node-gyp\\Cache\\16.15.1\\common.gypi.
    Here are the example of it failing multiple times:

    This happens inconsistently and same test will sometimes pass, mostly fail.
    Interesting thing to note is that, the dependency itself is building fine: https://github.com/nodech/bcrypto/runs/6939480240, it only fails with hsd which has multiple native dependencies other than bcrypto. I tried to remove all other deps and only leave bcrypto. Switch from matrix to separate tests and etc. Always end up in the similar situation.

Example output for the failure from the Github CI: 
npm WARN config global `--global`, `--local` are deprecated. Use `--location=global` instead.
npm ERR! code 1
npm ERR! path D:\a\hsd\hsd\node_modules\bcrypto
npm ERR! command failed
npm ERR! command C:\Windows\system32\cmd.exe /d /s /c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@9.0.0
npm ERR! gyp info using node@16.15.1 | win32 | x64
npm ERR! gyp info find Python using Python version 3.9.13 found at "C:\hostedtoolcache\windows\Python\3.9.13\x64\python3.exe"
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.1/node-v16.15.1-headers.tar.gz
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.1/node-v16.15.1-headers.tar.gz
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.1/SHASUMS256.txt
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.1/win-x86/node.lib
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.1/win-x64/node.lib
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.1/win-arm64/node.lib
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.1/SHASUMS256.txt
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.1/win-x64/node.lib
npm ERR! gyp http 404 https://nodejs.org/download/release/v16.15.1/win-arm64/node.lib
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.1/win-x86/node.lib
npm ERR! gyp info find VS using VS2022 (17.2.32602.215) found at:
npm ERR! gyp info find VS "C:\Program Files\Microsoft Visual Studio\2022\Enterprise"
npm ERR! gyp info find VS run with --verbose for detailed information
npm ERR! gyp info spawn C:\hostedtoolcache\windows\Python\3.9.13\x64\python3.exe
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   'C:\\npm\\prefix\\node_modules\\npm\\node_modules\\node-gyp\\gyp\\gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'msvs',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   'D:\\a\\hsd\\hsd\\node_modules\\bcrypto\\build\\config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   'C:\\npm\\prefix\\node_modules\\npm\\node_modules\\node-gyp\\addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   'C:\\Users\\runneradmin\\AppData\\Local\\node-gyp\\Cache\\16.15.1\\common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=C:\\Users\\runneradmin\\AppData\\Local\\node-gyp\\Cache\\16.15.1',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=C:\\npm\\prefix\\node_modules\\npm\\node_modules\\node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=C:\\\\Users\\\\runneradmin\\\\AppData\\\\Local\\\\node-gyp\\\\Cache\\\\16.15.1\\\\<(target_arch)\\\\node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=D:\\a\\hsd\\hsd\\node_modules\\bcrypto',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'D:\\a\\hsd\\hsd\\node_modules\\bcrypto\\build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp: C:\Users\runneradmin\AppData\Local\node-gyp\Cache\16.15.1\common.gypi not found (cwd: D:\a\hsd\hsd\node_modules\bcrypto) while reading includes of binding.gyp while trying to load binding.gyp
npm ERR! gyp ERR! configure error 
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack     at ChildProcess.onCpExit (C:\npm\prefix\node_modules\npm\node_modules\node-gyp\lib\configure.js:261:16)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:527:28)
npm ERR! gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
npm ERR! gyp ERR! System Windows_NT 10.0.20348
npm ERR! gyp ERR! command "C:\\hostedtoolcache\\windows\\node\\16.15.1\\x64\\node.exe" "C:\\npm\\prefix\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd D:\a\hsd\hsd\node_modules\bcrypto
npm ERR! gyp ERR! node -v v16.15.1
npm ERR! gyp ERR! node-gyp -v v9.0.0
npm ERR! gyp ERR! not ok
(...)
@cclauss
Copy link
Contributor

cclauss commented Jun 18, 2022 •
edited
Loading

actions/setup-node@v1 != v3.3.0

actions/checkout should also be upgraded.

@cclauss cclauss closed this as completed Jun 18, 2022
@nodech
Copy link
Author

nodech commented Jun 19, 2022 •
edited
Loading

That is main branch which does not have windows tests, that's what I am trying to upgrade and add windows to the mix (already have macos). On the runners you can see branch is update-ci-2: https://github.com/nodech/hsd/blob/update-ci-2/.github/workflows/node.js.yml. Which uses setup-node@v3.

@cclauss cclauss reopened this Jun 19, 2022
@nodech
Copy link
Author

nodech commented Jun 20, 2022 •
edited
Loading

So I tried isolating dependencies to see how the failures happen in https://github.com/nodech/ci-test.

Dependencies used for testing (all three are using node-gyp):

Used workflow: https://github.com/nodech/ci-test/blob/main/.github/workflows/nodejs.yml

@nodech nodech mentioned this issue Jun 21, 2022
Merged
3 tasks
@Apollon77
Copy link

I have the same issue since longer times. In my case it is always "noble" which is the reason for the issues... as soon as I use windows-latest runner on GithUb actiopns it fails mainly with Node.js 16!

See https://github.com/Apollon77/hap-controller-node/runs/7120903854?check_suite_focus=true#step:5:53

I use node setup action 3.3.0 and also upgrade node-gyp manually globally.

@cclauss
Copy link
Contributor

cclauss commented Jul 14, 2022

Please create a pull request that fails to https://github.com/nodejs/node-gyp/tree/main/.github/workflows so that everyone can see these failures.

@cclauss cclauss added the bcrypt label Aug 2, 2022
@shazron shazron mentioned this issue Aug 31, 2022
Merged
10 tasks
@michalzaq12 michalzaq12 mentioned this issue Oct 19, 2022
Closed
@dsanders11 dsanders11 mentioned this issue May 11, 2023
Merged
4 tasks
@LabhanshAgrawal
Copy link

Looks like this is still an issue. I tried using v9.4.0 and I'm still getting this common.gypi not found error.
E.g. https://github.com/vercel/hyper/actions/runs/5271238701/jobs/9531822058

@rithvikvibhu
Copy link

@LabhanshAgrawal looks like it didn't use 9.4.0. From the logs:

gyp ERR! node -v v18.16.0
gyp ERR! node-gyp -v v9.3.1

@LabhanshAgrawal
Copy link

@LabhanshAgrawal looks like it didn't use 9.4.0. From the logs:

gyp ERR! node -v v18.16.0
gyp ERR! node-gyp -v v9.3.1

Weird, thanks for pointing out, yarn.lock only has the 9.4.0 version. I will check how 9.3.1 is getting used.

@LabhanshAgrawal
Copy link

It was picking up v9.3.1 from global packages, setting it up to use the local one solved this. Anyone else facing similar issue on github actions, feel free to check vercel/hyper#7270

@rithvikvibhu rithvikvibhu mentioned this issue Jun 17, 2023
Merged
@rathboma
Copy link

@LabhanshAgrawal++ Thank you for posting this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bcrypt Windows
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: extract tarball to temp directory on Windows dsanders11/node-gyp
6 participants
@rathboma @nodech @cclauss @rithvikvibhu @Apollon77 @LabhanshAgrawal