Jacoco reports false 0% test coverage in version 1.7.0 but worked properly in version 1.5.3.

[timKraeuter]

Describe the bug
I upgraded the plugin from version 1.5.3 to version 1.7.0.
My test coverage using Jacoco was working in 1.5.3 when executing gradle build. See excerpt of the jacoco.xml:

However, after the upgrade, Jacoco reports 0% test coverage. See excerpt of the jacoco.xml:

To Reproduce
Clone my IntelliJ plugin from here and run gradle build through IntelliJ idea after importing the project.

Look at the file jacoco.xml in build/reports/jacoco.xml:

Downgrade the plugin to version 1.5.3 to get the correct behavior.

Expected behavior
Jacoco should be able to determine test coverage correctly during gradle build.

Environment:

• OS: Windows
• Gradle IntelliJ Plugin Version: 1.7.0 (worked in 1.5.3, errors in 1.6.0)
• Gradle Version 6.8 (wrapper)

Additional context
Nothing.

[monosoul]

Plugin version 1.5.3 can't build a plugin for the latest Intellij version (222.*), getting:

Error: LinkageError occurred while loading main class com.intellij.idea.Main
	java.lang.ExceptionInInitializerError: null

So downgrading is only an option until you try to build the plugin for the latest intellij.

[magneticB]

Have the same problem upgrading from 1.5.3 to version 1.7.0. Is there a fix available?

[timKraeuter]

I am not aware of a fix yet :(

[monosoul]

I have switched to kover for coverage report generation. It doesn't work with JaCoCo engine as well (shows 0% coverage), but works with Intellij engine (used by default). That was good enough for me, since the XML report it generates works fine with Codecov.

[hsz]

I have tested the project provided by @timKraeuter using the latest sources of the Gradle IntelliJ Plugin locally.
It turned out that everything works fine if I comment out the following part:

https://github.com/JetBrains/gradle-intellij-plugin/blob/1a82757fb3c4f0bfa8fbbc7b96a7e3738b76d98b/src/main/kotlin/org/jetbrains/intellij/IntelliJPlugin.kt#L1132-L1134

Unfortunately, this is required for since 221 to make everything up-and-running within the IDEA.

I have already compared all the configuration differences between runs that have PathClassLoader set as on and off and sources/classes paths are the same. JaCoCo classpaths also don't differ.

What is actually different is the file size of build/jacoco/test.exec: with PathClassLoader enabled it's significantly smaller: – 24K. Without it: 130K.

[timKraeuter]

I am using sonar cloud in a CI/CD pipeline to determine the code coverage for my plugin. As far as I can tell, this does not work with kover.
As @hsz pointed out, it seems to have something to do with the changes to the class loader. I hope this can get fixed since JaCoCO is a standard tool to determine code coverage for Java projects.

[hsz]

I have removed mentioned part as it is redundant from the IDE point of view, when running tests.
Such a change is now available in 1.8.0-SNAPSHOT would you mind checking if all's good with that version?

[timKraeuter]

I can check it. What is the easiest way to get the new dependency into my project? (since it does not seem to be in mavenCentral yet)

[hsz]

Check our docs: Snapshot Release

[timKraeuter]

Unfortunately, I don't see any changes in the behavior using 1.8.0-SNAPSHOT. I changed to the snapshot version in a new branch but got 0% coverage locally and on sonarcloud.

[hsz]

@timKraeuter My apologies – 1.8.1-SNAPSHOT.

Looks good to me. :)

[timKraeuter]

Thanks, the new release 1.8.0 works for me!