chore: replaces c8 with node native coverage #203
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Removed dependencies detected. Learn more about Socket for GitHub ↗︎ 🚮 Removed packages: npm/c8@9.1.0 |
sverweij
force-pushed
the
chore/replace-c8-with-native-coverage
branch
from
6b5f99d to
813c540
Compare
sverweij
force-pushed
the
chore/replace-c8-with-native-coverage
branch
4 times, most recently
from
dc72a04 to
8dbdaf9
Compare
sverweij
force-pushed
the
chore/replace-c8-with-native-coverage
branch
from
8dbdaf9 to
e1fecc1
Compare
McMeadow
approved these changes
May 11, 2024
sverweij
added a commit
that referenced
this pull request
May 11, 2024
## Description - removes c8 dependency & config - uses the --experimental-test-coverage cli switch to get coverage - updates `./tools/dot-with-summary.reporter.js` reporter to emit coverage information - updates the ci workflow to emit information from that report instead of the markdown derived from the codecov run (on node 20 only for now as there it's stable). ## Motivation and Context In our setup c8 doesn't provide any real coverage information anymore (see below - note the number of lines, functions and branches). Root cause probably lies [how node handles ESM/ how v8-to-istanbul can process that](bcoe/c8#34). As nodejs now also has a --experimental-test-coverage built in we migrate this to that. > --experimental-test-coverage isn't perfect either b.t.w. In node 20 it's stable, but on node 22 (at least in our set up) it tends to report different results on each run with unchanged source code. <details> <summary>recent c8 output on virtual-code-owners main branch</summary> ```shell $ npm test > virtual-code-owners@8.0.5 test > c8 tsx --test-reporter ./tools/dot-with-summary.reporter.js --test src/*.test.ts src/**/*.test.ts ..................................................... 53 passing (1.159 ms) =============================== Coverage summary =============================== Statements : 100% ( 11/11 ) Branches : 100% ( 0/0 ) Functions : 100% ( 0/0 ) Lines : 100% ( 11/11 ) ================================================================================ ``` </details> ## How Has This Been Tested? - [x] green ci - [x] manually (the script in tools should probably be put in a separate repo and get its own set of unit tests c.s., but for fixing the issue at hand a bit out of scope. ## Screenshots ``` ..................................................... 53 passing (1.199 ms) =============================== Coverage summary =============================== Branches : 100 % (205/205) Functions : 100 % (123/123) Lines : 100 % (1.533/1.533) ================================================================================ ``` With uncovered lines/ functions (b.t.w. same, unchanged codebase as above ..., but using node 22, after a run or two) ```shell ..................................................... 53 passing (1.216 ms) =============================== Coverage summary =============================== Branches : 100 % (184/184) Functions : 98,31 % (116/118) NOK Lines : 99,74 % (1.529/1.533) ================================================================================ Uncovered lines: /Users/sander/prg/js/virtual-code-owners/src/labeler-yml/generate.ts:102,103,104 /Users/sander/prg/js/virtual-code-owners/src/virtual-code-owners/anomalies.ts:40 Uncovered functions: /Users/sander/prg/js/virtual-code-owners/src/virtual-code-owners/anomalies.ts:40,40 ``` ## Types of changes <!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] Documentation only change - [ ] Refactor (non-breaking change which fixes an issue without changing functionality) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Chore (thing that needs doing in e.g. coding infrastructure)
sverweij
added a commit
that referenced
this pull request
May 11, 2024
## Description - removes c8 dependency & config - uses the --experimental-test-coverage cli switch to get coverage - updates `./tools/dot-with-summary.reporter.js` reporter to emit coverage information - updates the ci workflow to emit information from that report instead of the markdown derived from the codecov run (on node 20 only for now as there it's stable). ## Motivation and Context In our setup c8 doesn't provide any real coverage information anymore (see below - note the number of lines, functions and branches). Root cause probably lies [how node handles ESM/ how v8-to-istanbul can process that](bcoe/c8#34). As nodejs now also has a --experimental-test-coverage built in we migrate this to that. > --experimental-test-coverage isn't perfect either b.t.w. In node 20 it's stable, but on node 22 (at least in our set up) it tends to report different results on each run with unchanged source code. <details> <summary>recent c8 output on virtual-code-owners main branch</summary> ```shell $ npm test > virtual-code-owners@8.0.5 test > c8 tsx --test-reporter ./tools/dot-with-summary.reporter.js --test src/*.test.ts src/**/*.test.ts ..................................................... 53 passing (1.159 ms) =============================== Coverage summary =============================== Statements : 100% ( 11/11 ) Branches : 100% ( 0/0 ) Functions : 100% ( 0/0 ) Lines : 100% ( 11/11 ) ================================================================================ ``` </details> ## How Has This Been Tested? - [x] green ci - [x] manually (the script in tools should probably be put in a separate repo and get its own set of unit tests c.s., but for fixing the issue at hand a bit out of scope. ## Screenshots ``` ..................................................... 53 passing (1.199 ms) =============================== Coverage summary =============================== Branches : 100 % (205/205) Functions : 100 % (123/123) Lines : 100 % (1.533/1.533) ================================================================================ ``` With uncovered lines/ functions (b.t.w. same, unchanged codebase as above ..., but using node 22, after a run or two) ```shell ..................................................... 53 passing (1.216 ms) =============================== Coverage summary =============================== Branches : 100 % (184/184) Functions : 98,31 % (116/118) NOK Lines : 99,74 % (1.529/1.533) ================================================================================ Uncovered lines: /Users/sander/prg/js/virtual-code-owners/src/labeler-yml/generate.ts:102,103,104 /Users/sander/prg/js/virtual-code-owners/src/virtual-code-owners/anomalies.ts:40 Uncovered functions: /Users/sander/prg/js/virtual-code-owners/src/virtual-code-owners/anomalies.ts:40,40 ``` ## Types of changes <!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] Documentation only change - [ ] Refactor (non-breaking change which fixes an issue without changing functionality) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Chore (thing that needs doing in e.g. coding infrastructure)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
./tools/dot-with-summary.reporter.jsreporter to emit coverage informationMotivation and Context
In our setup c8 doesn't provide any real coverage information anymore (see below - note the number of lines, functions and branches). Root cause probably lies how node handles ESM/ how v8-to-istanbul can process that. As nodejs now also has a --experimental-test-coverage built in we migrate this to that.
recent c8 output on virtual-code-owners main branch
How Has This Been Tested?
Screenshots
With uncovered lines/ functions (b.t.w. same, unchanged codebase as above ..., but using node 22, after a run or two)
Types of changes
Checklist
📖
⚖️