Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider adding coverage reports (e.g. for lcov format) #117

Open
alxgrk opened this issue May 18, 2021 · 9 comments · May be fixed by #352
Open

Consider adding coverage reports (e.g. for lcov format) #117

alxgrk opened this issue May 18, 2021 · 9 comments · May be fixed by #352
Labels
enhancement New feature or request

Comments

@alxgrk
Copy link

alxgrk commented May 18, 2021

Hey there,

first, thanks for your work, this github action is awesome!

Nevertheless, it would be even more awesome, if one could see code coverage information next to the test reports. Maybe an additional property coverage-files could be added (see this one as an example), which takes lcov files as an input and displays the information if the property is set? What are your thoughts about that?

Thanks in advance.
Best regards,
Alex

@dorny
Copy link
Owner

dorny commented May 19, 2021

Hello,

I was thinking about this some time ago but came to the conclusion it has low priority because there are other projects doing it.
As an eventual user of this feature - why would you prefer having this in a test-reporter instead of using the action you linked or codecov.io for example?

@alxgrk
Copy link
Author

alxgrk commented May 19, 2021

The one feature that differentiates your work from all other projects I could find is that the results will be displayed as a Github Action Check Run (and not e.g. as a comment on the PR or on an external site like codecov). This approach is very concise (not cluttering the PR) and only relatable to one specific run. So combining test results and their coverage in a single view would be great.

@dorny
Copy link
Owner

dorny commented May 23, 2021

Thanks, that makes sense.

For the beginning, I would like to support lcov and cobetura formats. Results should be in the form of a table - either by files, classes, or whole projects. I'm not sure if it should be in the same check run as test results or a separate one. Check run summaries are limited to 65k bytes - there might be not enough space for both test results and coverage results.

Existing actions, including the one you linked, just wrap other tools that convert coverage files into HTML. Here I would have to do parsing and visualization from scratch. It will need some time. I will put this on my to-do list but unfortunately, it won't be done anytime soon. I just got a new job and my availability for this project is quite limited now. In the next months, I will focus rather on bug fixes and simpler improvements than a completely new feature.

@alxgrk
Copy link
Author

alxgrk commented May 23, 2021

Perfect! Personally, I only care about lcov and splitting it into two check runs is perfectly fine, but that's just my personal opinion.

Additionally, I totally understand that this feature is not your highest priority - but with that in mind, I'm wondering why wrapping other tools is not an option in this case? I thought, content needs to be delivered as markdown, so basically creating HTML and converting that to markdown should be fine as well, shouldn't it?

@PavanMudigondaTR
Copy link

PavanMudigondaTR commented Oct 31, 2021

Hi Dorny, I have the same request. If you could add Jacoco and Corbetura icov support by parsing XML and displaying in table similar to test results display. I understand you are busy with new job. Just when ever you can get to it.

Thank you for this awesome github action you have created.

@j-catania
Copy link
Collaborator

j-catania commented Aug 12, 2023

Hi guys,

I'm looking to make this work on my fork. PR #259

I will be in touch with you,
Juu'

@j-catania j-catania linked a pull request Jan 3, 2024 that will close this issue
@j-catania j-catania added this to the v1.8.0 milestone Jan 3, 2024
@j-catania j-catania added the enhancement New feature or request label Jan 3, 2024
@j-catania
Copy link
Collaborator

j-catania commented Jan 3, 2024

Hi guys,

I worked a little bit in a branch for adding lcov support.

I use it in another projet to me : https://github.com/j-catania/emusk/actions/runs/7390220838/job/20104669346

Here is the actual report :
image

@j-catania
Copy link
Collaborator

j-catania commented Jan 12, 2024

@alxgrk, @PavanMudigondaTR

Could you please have a look at the PR and tell me what you think please ?

@jozefizso jozefizso removed this from the v1.8.0 milestone Feb 4, 2024
@alxgrk
Copy link
Author

alxgrk commented Jun 12, 2024

@j-catania the PR looks very concise, but I agree with @dorny 's comment:

I think if lcov should be supported by test-reporter, it would require a specialized implementation for producing output in a different layout and also adding input variables for a passing threshold. The output should probably not only show if it passed but also an actual number.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants