Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workspaces generate inaccurate c8 coverage reports #3220

Closed
6 tasks done
kherock opened this issue Apr 20, 2023 · 5 comments · Fixed by #3226
Closed
6 tasks done

Workspaces generate inaccurate c8 coverage reports #3220

kherock opened this issue Apr 20, 2023 · 5 comments · Fixed by #3226
Labels
feat: coverage Issues and PRs related to the coverage feature

Comments

@kherock
Copy link

kherock commented Apr 20, 2023

Describe the bug

The C8 test coverage provider reports different statistics and inaccurate line numbers when the reporter runs on a workspace. When I preview the HTML report, the code highlighted is about 10 lines offset from where the actual uncovered code is. The column number offset also appear to shift 1 or 2 characters.

Reproduction

Vitest hangs when generating coverage in StackBlitz - I will provide a reproduction of this on GitHub when I have a chance.

System Info

System:
    OS: macOS 12.6.3
    CPU: (10) arm64 Apple M1 Pro
    Memory: 112.97 MB / 16.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 16.19.1 - ~/.local/share/homebrew/bin/node
    Yarn: 1.22.19 - ~/.local/share/homebrew/bin/yarn
    npm: 8.19.3 - ~/.local/share/homebrew/bin/npm
  Browsers:
    Firefox: 102.2.0
    Safari: 16.3
  npmPackages:
    @vitest/coverage-c8: ^0.30.1 => 0.30.1 
    vitest: ^0.30.1 => 0.30.1

Used Package Manager

npm

Validations

@github-actions
Copy link

Hello @kherock. Please provide a minimal reproduction using a GitHub repository or StackBlitz. Issues marked with need reproduction will be closed if they have no activity within 3 days.

@AriPerkkio AriPerkkio added bug feat: coverage Issues and PRs related to the coverage feature and removed needs reproduction labels Apr 22, 2023
@kherock
Copy link
Author

kherock commented Apr 24, 2023

Does this still need a reproduction? I think the linked PR #3226 covers the problems I'm experiencing.

@AriPerkkio
Copy link
Member

There was a clear bug in how @vitest/coverage-c8 handles sources maps when workspaces are used. The internal test cases of Vitest were enough to reproduce the issue.

@ocavue
Copy link
Contributor

ocavue commented Apr 25, 2023

Here is a reproduction I created. I hope this can help to fix and test the bug

Reproduce Steps:

$ git clone https://github.com/issueset/vitest-workspace-c8-bug
$ cd vitest-workspace-c8-bug
$ pnpm install
$ pnpm run test:c8
$ pnpm run test:istanbul

Open the report html files under coverage/. It's clear that console.log("A3") should never be covered, but the c8 reporter shows that this line is been covered.

CleanShot 2023-04-25 at 17 32 34@2x

@AriPerkkio
Copy link
Member

Thanks @ocavue - this is how the reproduction case looks like with the changes of #3226:

image

@github-actions github-actions bot locked and limited conversation to collaborators Jun 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
feat: coverage Issues and PRs related to the coverage feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants