-
Notifications
You must be signed in to change notification settings - Fork 29.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
coverage stats have plummeted #37368
Comments
I've rerun https://ci.nodejs.org/job/node-test-commit-linux-coverage-daily/ against 6ea9af9906cd74ed and this has returned coverage numbers in line with the expected:
this compares closely to the original coverage build for 6ea9af9906cd74ed in https://ci.nodejs.org/job/node-test-commit-linux-coverage-daily/779/nodes=benchmark/console of
(I don't know why the numbers aren't exactly the same) Starting with 88d9268d087eea0d we appear to be getting an error during compilation:
Commits between 6ea9af9...88d9268
88d9268 errors: align source-map stacks with spec Perhaps it's the V8 8.8 update? |
Build against fa3997d also shows the coverage drop:
I'm not sure it's easily possible to narrow down the commits further as the remaining range was the V8 8.8 update. |
Looking at the GitHub Actions, they seem healthy: https://github.com/nodejs/node/pull/37402/checks?check_run_id=1913015747 Do we think the low coverage is due to the compilation issue, vs., a regression with our coverage collection? |
It's the compilation issue. Specifically it appears to be failing to compile during the recursive make call to Line 323 in 9b32762
node earlier in the build and use it to build the addons (i.e. Lines 319 to 322 in 9b32762
The GitHub Actions coverage workflow isn't using the node/.github/workflows/coverage-linux.yml Lines 38 to 42 in 9b32762
|
Is |
The fact that tests are now taking several hours to complete on master doesn't seem very healthy (4h on |
It's what https://ci.nodejs.org/job/node-test-commit-linux-coverage-daily/ runs (to be exact, it runs
I tried throttling the number of jobs ( I've not been able to reproduce locally (Fedora 33, gcc/g++ 10). |
I logged into
🤔 I'll try restarting (/upgrading) the Jenkins agent on the machine. |
I ran the The
|
Failing to update packages via ansible on |
There was an issue to switch to codecov.io (subject to figuring out some access issues). I think the only reason the job runs on a specific machine is so that we can generate/transfer the results to the machine were the data is pulled for coverage.nodejs.org. If we believe we are ready to move to codecov.io only, maybe we just let the job run on a different machine and stop using coverage.nodejs.org ? |
Summary so far:
I believe it's also related to #36139 (comment) and #37429 in that when the |
I've been itching for us to get to this point, because it does mean we don't have to maintain the bespoke coverage machines. Also, I think it's worth noting that it's much easier for folks to run coverage locally now. Even though coverage takes several hours (40% slower than the normal test run right now) it still means we're getting regular metrics shipped to codecov.io. I still think there are some hiccups to sort out on codecov.io before I'd fully endorse it mind you, e.g., lack of commit context: |
What is the difference between running manually and running the the code coverage jenkins job? I would have thought they should be doing the same thing + some extra scripting to capture and provide the data. |
Running under the Jenkins agent? The job doesn't get to the extra scripting as it's failing during In any case #37502 avoids the unnecessary V8 rebuilds and the coverage job with that no longer fails to build during the
|
Or #37505, which cherry-picks a fix for torque from upstream V8: https://ci.nodejs.org/job/node-test-commit-linux-coverage-daily/805/nodes=benchmark/console
|
Original commit message: [torque] Don't replace unmodified empty files To improve incremental builds. Bug: v8:7793 Change-Id: I6990a97e058d22d34acd1f609167cd30ca7518ad Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2596789 Reviewed-by: Nico Hartmann <nicohartmann@chromium.org> Commit-Queue: Seth Brenith <seth.brenith@microsoft.com> Cr-Commit-Position: refs/heads/master@{#72053} Refs: v8/v8@373f4ae PR-URL: #37505 Fixes: #37368 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Not sure what happened but both coverage.nodejs.org and codecov.io show our code coverage plummeting recently.
Not sure if this is a problem in our code coverage task in
Makefile
/vcbuild.bat
or if it's a build/infra problem or what.@bcoe @nodejs/build
The text was updated successfully, but these errors were encountered: