0% coverage is being shown after running tests #110

robvanpamel · 2018-05-28T10:42:07Z

Hi,

After running my tests with Nunit I get an unexpected result from coverlet.
My folder structure is

root 
  \src 
     \projectfolder\project.csproj
  \tests
     \testprojectfolder\testproject.csproj
  \solution.sln

I executed the tests from within the testprojectfolder
The result is printed below.

dotnet test /p:CollectCoverage=true
Build started, please wait...
Build completed.

Test run for /home/rob/git/svc_system_center/tests/Sag.SystemCenter.Api.Tests/bin/Debug/netcoreapp2.0/Sag.SystemCenter.Api.Tests.dll(.NETCoreApp,Version=v2.0)
Microsoft (R) Test Execution Command Line Tool Version 15.5.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
NUnit Adapter 3.9.0.0: Test execution started
Running all tests in /home/rob/git/svc_system_center/tests/Sag.SystemCenter.Api.Tests/bin/Debug/netcoreapp2.0/Sag.SystemCenter.Api.Tests.dll
NUnit3TestExecutor converted 456 of 456 NUnit test cases

NUnit Adapter 3.9.0.0: Test execution complete

Total tests: 456. Passed: 452. Failed: 0. Skipped: 4.
Test Run Successful.
Test execution time: 3.8804 Seconds

Calculating coverage result...
  Generating report '/home/rob/git/svc_system_center/tests/Sag.SystemCenter.Api.Tests/coverage.json'

+--------------------------------------------+--------+--------+--------+
| Module                                     | Line   | Branch | Method |
+--------------------------------------------+--------+--------+--------+
| Sag.SystemCenter.LoginManagement.BL        | 0%     | 0%     | 0%     |
+--------------------------------------------+--------+--------+--------+
| Sag.SystemCenter.UserManagement.BL         | 0%     | 0%     | 0%     |
+--------------------------------------------+--------+--------+--------+
| Sag.SystemCenter.Identity                  | 0%     | 0%     | 0%     |
+--------------------------------------------+--------+--------+--------+
| Sag.SystemCenter.UserManagement.DAL        | 0%     | 0%     | 0%     |
+--------------------------------------------+--------+--------+--------+
| Sag.SystemCenter.ApiManagement             | 0%     | 0%     | 0%     |
+--------------------------------------------+--------+--------+--------+
| Sag.SystemCenter.Captcha.BL                | 0%     | 0%     | 0%     |
+--------------------------------------------+--------+--------+--------+
| Sag.SystemCenter.Domain                    | 0%     | 0%     | 0%     |
+--------------------------------------------+--------+--------+--------+
| Sag.SystemCenter.Api                       | 0%     | 0%     | 0%     |
+--------------------------------------------+--------+--------+--------+
| Sag.SystemCenter.Messaging                 | 0%     | 0%     | 0%     |
+--------------------------------------------+--------+--------+--------+
| Sag.SystemCenter.ApplicationManagement.DAL | 0%     | 0%     | 0%     |
+--------------------------------------------+--------+--------+--------+

I'm running on a Linux Mint machine with this dotnet configuration

.NET Command Line Tools (2.1.4)

Product Information:
 Version:            2.1.4
 Commit SHA-1 hash:  5e8add2190

Runtime Environment:
 OS Name:     linuxmint
 OS Version:  18.3
 OS Platform: Linux
 RID:         linux-x64
 Base Path:   /usr/share/dotnet/sdk/2.1.4/

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.5
  Build    : 17373eb129b3b05aa18ece963f8795d65ef8ea54

The text was updated successfully, but these errors were encountered:

tonerdo · 2018-05-28T21:02:20Z

Hi @robvanpamel can you try with the previous version of Coverlet (v1.2.0)? This seems to be a 2.0.0 issue. Let me know

aburgett87 · 2018-05-29T02:02:03Z

Hi @tonerdo,
I can see the issue as well, though only on Linux. Using v1.2.0 coverage results output as expected, though on 2.0.0 it's as @robvanpamel described.

Using macOS 10.13.4, both 2.0.0 and 1.2.0 work as expected.

Linux dotnet cofiguration

.NET Command Line Tools (2.1.4)

Product Information:
 Version:            2.1.4
 Commit SHA-1 hash:  5e8add2190

Runtime Environment:
 OS Name:     debian
 OS Version:  9
 OS Platform: Linux
 RID:         linux-x64
 Base Path:   /usr/share/dotnet/sdk/2.1.4/

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.5
  Build    : 17373eb129b3b05aa18ece963f8795d65ef8ea54

macOS dotnet configuration

.NET Command Line Tools (2.1.200)

Product Information:
 Version:            2.1.200
 Commit SHA-1 hash:  2edba8d7f1

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.13
 OS Platform: Darwin
 RID:         osx.10.13-x64
 Base Path:   /usr/local/share/dotnet/sdk/2.1.200/

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.7
  Build    : 2d61d0b043915bc948ebf98836fefe9ba942be11

aburgett87 · 2018-05-29T02:33:39Z

@tonerdo,
Interestingly, if I run my project with coverlet 2.0.0 on Linux using dotnet core 2.0.7 or 2.1-rc everything works as expected.

Linux dotnet 2.0.7 configuration

.NET Command Line Tools (2.1.200)

Product Information:
 Version:            2.1.200
 Commit SHA-1 hash:  2edba8d7f1

Runtime Environment:
 OS Name:     debian
 OS Version:  9
 OS Platform: Linux
 RID:         debian.9-x64
 Base Path:   /usr/share/dotnet/sdk/2.1.200/

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.7
  Build    : 2d61d0b043915bc948ebf98836fefe9ba942be11

Linux dotnet configuration 2.1-rc1

.NET Core SDK (reflecting any global.json):
 Version:   2.1.300-rc1-008673
 Commit:    f5e3ddbe73

Runtime Environment:
 OS Name:     debian
 OS Version:  9
 OS Platform: Linux
 RID:         debian.9-x64
 Base Path:   /usr/share/dotnet/sdk/2.1.300-rc1-008673/

Host (useful for support):
  Version: 2.1.0-rc1
  Commit:  eb9bc92051

.NET Core SDKs installed:
  2.1.300-rc1-008673 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.0-rc1-final [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.0-rc1-final [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.0-rc1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

FYI @robvanpamel

robvanpamel · 2018-05-29T07:13:17Z

Hi,
For your information it is working with the 1.2 release

basilfx · 2018-05-29T08:04:17Z

@robvanpamel Can you try to exclude NUnit3 Test Adapter assemblies: dotnet test /p:CollectCoverage=true /p:Exclude=[NUnit3.TestAdapter]*. See #101 for more information.

mikkokupsu · 2018-06-06T10:21:46Z

I'm facing a similar problem.
I'm running below commands in .NET Docker container (2.0-sdk) and coverlet.msbuild version 2.0.0.
By removing /p:Threshold=90, I'm getting a coverage result. When the threshold is present, I get a 0 percentage report.

dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=lcov /p:ThresholdType=line /p:Threshold=90 /p:CoverletOutput=./lcov Test

Test run for /builds/Test/bin/Debug/netcoreapp2.0/Test.dll(.NETCoreApp,Version=v2.0)
Microsoft (R) Test Execution Command Line Tool Version 15.7.0
Copyright (c) Microsoft Corporation. All rights reserved.

Starting test execution, please wait...

Total tests: 98. Passed: 98. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 4.8294 Seconds

Calculating coverage result...
Generating report './lcov.info'

+-------------------+--------+--------+--------+
| Module | Line | Branch | Method |
+-------------------+--------+--------+--------+
| XXXX | 0% | 0% | 0% |
+-------------------+--------+--------+--------+

/root/.nuget/packages/coverlet.msbuild/2.0.0/build/netstandard2.0/coverlet.msbuild.targets(23,5): error : 'XXXX' has a line coverage '0%' below specified threshold '90%' [/builds/Test/Test.csproj]

dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=lcov /p:ThresholdType=line /p:CoverletOutput=./lcov Test

Test run for /builds/Test/bin/Debug/netcoreapp2.0/Test.dll(.NETCoreApp,Version=v2.0)
Microsoft (R) Test Execution Command Line Tool Version 15.7.0
Copyright (c) Microsoft Corporation. All rights reserved.

Starting test execution, please wait...

Total tests: 98. Passed: 98. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 4.7816 Seconds

Calculating coverage result...
Generating report './lcov.info'

+-------------------+--------+--------+--------+
| Module | Line | Branch | Method |
+-------------------+--------+--------+--------+
| XXXX | 100% | 85% | 100% |
+-------------------+--------+--------+--------+

tlogik · 2018-06-08T12:36:27Z

I have the issue of 0% coverage even on coverlet version 1.2.0
I am using
Xunit2,
Tested on Coverlet 1.2.0 and latest 2.0.0
on a netstand 2.0 projects.

Note: It works on other netstandard/netcore projects i have, running same framework versions etc.
Currently i do not see any pattern to the reason why some projects does not generate coverage where others do.

Update 2:
Message from dotnet test --no-build --filter Category=Unit /p:CollectCoverage=true
using XUNIT2 for tests.

No test is available in /Users/thomasblitz/Private/code/gitlab/Framework/framework-logging-applicationlogging/src/ApplicationLogging.DAL.Tests/bin/Debug/netcoreapp2.0/ApplicationLogging.DAL.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again. Additionally, path to test adapters can be specified using /TestAdapterPath command. Example /TestAdapterPath:<pathToCustomAdapters>.

Update 3:
Got it working

I updated Microsoft.NET.Test.Sdk to 15.7.2 and maintained coverlet at 1.2.
Still same result.
I ran dotnet test --no-build --filter Category=Unit and because i had been writing some testcode i now could see some test failing. The failures were suppressed when i had /p:CollectCoverage=true
Fixed the code, rebuild and run tests with success.
ran dotnet test --no-build --filter Category=Unit /p:CollectCoverage=true and now i had coverage

Update 4:

Bumped coverlet to 2.0.0 and i still get coverage results :-D

There is a discrepancy in the results from coverlet 1.2 to 2.0.
Observe:
Coverlet 1.2

Module	Coverage
ApplicationLogging.Dal	0.3%

Coverlet 2.0

Module	Line	Branch	Method
ApplicationLogging.Dal	2.3%	2.7%	3.9%

queen-of-code · 2018-08-16T20:50:42Z

I'm seeing this as well in one of my test suites. I moved to using the nuget package, just to eliminate potential variables:

    <PackageReference Include="coverlet.msbuild" Version="2.2.1">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.7.0" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.3.1" />
    <PackageReference Include="TeamCity.VSTest.TestAdapter" Version="1.0.10" />
    <PackageReference Include="xunit" Version="2.3.1" />
    <PackageReference Include="Moq" Version="4.8.3" />

Test run for C:\GitHub\api\test\unit\API.Partner.Service.Test\bin\Debug\netcoreapp2.1\API.Partner.Service.Test.dll(.NETCoreApp,Version=v2.1)
Microsoft (R) Test Execution Command Line Tool Version 15.8.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

Total tests: 53. Passed: 53. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 3.1755 Seconds

Calculating coverage result...
  Generating report 'C:\GitHub\api\test\unit\API.Partner.Service.Test\coverage.json'

+------------------------------------+--------+--------+--------+
| Module                             | Line   | Branch | Method |
+------------------------------------+--------+--------+--------+
| API                                 | 0%     | 0%     | 0%     |
+------------------------------------+--------+--------+--------+
| API.Partner.Service          | 0%     | 0%     | 0%     |
+------------------------------------+--------+--------+--------+

I know that coverage should actually be around 60-70% - certainly not 0. Interestingly, it generates an enormous coverage.json file full of lines, but all the Hits are :0.

"System.Collections.Generic.Dictionary`2<System.String,System.Func`2<System.Object,System.Object>> API.Partner.Service.AppCreatorBase`1::CreateCompiledGetters(System.Collections.Generic.IEnumerable`1<System.Reflection.PropertyInfo>)": {
          "Lines": {
            "114": 0,
            "115": 0,
            "117": 0,
            "118": 0,
            "119": 0,
            "121": 0,
            "123": 0,
            "125": 0,
            "126": 0,
            "127": 0,
            "129": 0,
            "130": 0
          },
          "Branches": [
            {
              "Line": 117,
              "Offset": 135,
              "EndOffset": 17,
              "Path": 1,
              "Ordinal": 1,
              "Hits": 0
            },
            {
              "Line": 117,
              "Offset": 135,
              "EndOffset": 137,
              "Path": 0,
              "Ordinal": 0,
              "Hits": 0
            }
          ]
        },

Is there a diagnostic logger for coverlet that I can enable?

tonerdo · 2018-08-16T22:03:38Z

Hi @queen-of-code @robvanpamel @tlogik take a look at #72. Try adding the following line to your csproj:

<CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>

Is there a diagnostic logger for coverlet that I can enable?

Not at this time

queen-of-code · 2018-08-17T19:03:21Z

Unfortunately no luck. I added that, and also tried switching my test csproj from

    <PropertyGroup Condition="'$(Configuration)'=='Debug'">
        <DebugType>pdbonly</DebugType>
    </PropertyGroup>

    <PropertyGroup Condition="'$(Configuration)'=='Debug'">
        <DebugType>full</DebugType>
    </PropertyGroup>

Aside from making my test run exponentially slower, it didn't provide any coverage information. Just a dump full of 0% coverage-claims.

tonerdo · 2018-09-01T16:04:08Z

#181 @queen-of-code completely changes the way Coverlet instruments assemblies and makes the test run orders of magnitude faster. You can see perf improvements here: #181 (comment).

I expect this change should fix the 0% coverage issue you're all getting, should make a release in 48 hours so you can test it

queen-of-code · 2018-09-05T21:32:33Z

Looking forward to it @tonerdo - when you cut the new release, I will test it out ASAP.

tonerdo · 2018-09-07T18:55:30Z

@queen-of-code The new releases are out

Coverlet Global Tool: https://www.nuget.org/packages/coverlet.console/1.2.0
Coverlet MSBuild: https://www.nuget.org/packages/coverlet.msbuild/2.3.0

Let me know if your issue gets fixed when you upgrade.
Thanks!

tonerdo · 2018-09-09T22:28:15Z

@queen-of-code I didn't notice this before but you should try changing DebugType to portable

queen-of-code · 2018-09-12T13:25:14Z

I have good news and I have bad news.

The good news is that our total execution time is down from ~50 minutes to ~22 minutes. That's an amazing improvement! Great work!

The bad news is that I'm still getting no coverage results. I've included the problematic test csproj. The command I'm running is dotnet test --verbosity normal --no-build --configuration Debug /p:CollectCoverage=true

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>
    <AssemblyName>MP.Test</AssemblyName>
    <PackageId>MP.Test</PackageId>
    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
  </PropertyGroup>

    <PropertyGroup Condition="'$(Configuration)'=='Debug'">
        <DebugType>portable</DebugType>
    </PropertyGroup>
    
  <ItemGroup>
    <None Include="*.json" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="PreserveNewest" />
    <None Include="*.config" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="PreserveNewest" />
    <None Include="TestData\**\*.csv" CopyToOutputDirectory="PreserveNewest" CopyToPublishDirectory="PreserveNewest" />
  </ItemGroup>
  <ItemGroup>
    <ProjectReference Include="..\..\..\src\MP\MP.csproj" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="coverlet.msbuild" Version="2.3.0">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.6.0" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.3.1" />
    <PackageReference Include="TeamCity.VSTest.TestAdapter" Version="1.0.10" />
    <PackageReference Include="xunit" Version="2.3.1" />
    <PackageReference Include="Moq" Version="4.8.3" />
  </ItemGroup>
</Project>

tonerdo · 2018-09-16T09:23:17Z

@queen-of-code I'm bummed that you're still having issues. Your csproj looks accurate enough and you shouldn't still be having problems. Unfortunately, I'm unable to debug without having access to the problematic project.

As a result of this limitation, I'm adding a new flag to log diagnostics information, making it easy for me to debug similar issues without needing a repro project. This will be ready/released in ~2 days

The good news is that our total execution time is down from ~50 minutes to ~22 minutes

Glad to hear that. Out of curiousity, how long do your tests take to run when Coverlet isn't included?

queen-of-code · 2018-09-19T00:18:02Z

Our current test suite (using dotnet test test.sln) runs in just over 1 minute. I just verified test-only numbers with the latest coverlet version, and it's at about 8.5 minutes. So it's still a significant overhead, most of which I suspect is due to running tests serially via **/*.csproj. SonarQube is involved in the build process as well, so it's certainly possible that it's leaving cruft in the dotnet assemblies.

twcclegg · 2018-10-05T20:53:57Z

I having what seems to be a similar issue of 0% results for code coverage using 2.3.0, however I do get coverage for 2 of the 4 projects with the .sln I'm trying to get coverage for. I do get results using 1.2.0. Happening on both centos7 and osx.

No coverage:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>
  </PropertyGroup>
  <PropertyGroup>
    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="coverlet.msbuild" Version="2.3.0">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.8.0" />
    <PackageReference Include="xunit" Version="2.4.0" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.0" />
  </ItemGroup>
  <ItemGroup>
    <ProjectReference Include="..\AsteriskARIClient\AsteriskARIClient.csproj" />
    <ProjectReference Include="..\CallController.Test\CallController.Test.csproj" />
    <ProjectReference Include="..\CallControllerApi\CallControllerApi.csproj" />
    <ProjectReference Include="..\CallController\CallController.csproj" />
    <ProjectReference Include="..\Utilities\Utilities.csproj" />
  </ItemGroup>
  <ItemGroup>
    <None Update="appsettings.json">
      <CopyToOutputDirectory>Always</CopyToOutputDirectory>
    </None>
    <None Update="log4net.config">
      <CopyToOutputDirectory>Always</CopyToOutputDirectory>
    </None>
  </ItemGroup>
  <ItemGroup>
    <Service Include="{82a7f48d-3b50-4b1e-b82e-3ada8210c358}" />
  </ItemGroup>
</Project>

Coverage:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="coverlet.msbuild" Version="2.3.0">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="Microsoft.Extensions.Logging" Version="2.1.1" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.8.0" />
    <PackageReference Include="Moq" Version="4.10.0" />
    <PackageReference Include="xunit" Version="2.4.0" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.0" />
    <PackageReference Include="FluentAssertions" Version="5.4.2" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\Utilities\Utilities.csproj" />
  </ItemGroup>

  <ItemGroup>
    <Service Include="{82a7f48d-3b50-4b1e-b82e-3ada8210c358}" />
  </ItemGroup>
</Project>

p10tyr · 2018-10-23T12:51:12Z

Hello everyone on the thread.
A user of PrestoCoverage which uses Coverlet reported that he has issues while running the tests with Moq.

This error message appears.
Could not load file or assembly "Moq" Strong name signature could not be verified. The assembly may have been tampered with, or it was delay signed but not fully signed with the correct private key

Now all I did was a search on the bugs and found this post.. I notice you are referencing Moq which may probably be the reason you do not get any coverage results?

Is there anybody that can confirm that Coverlet works with Moq?

StephenMP · 2018-10-30T19:24:39Z

I'm also getting 0% code coverage on one of my projects. You can view the source here: https://github.com/StephenMP/SLOBSharp/tree/cc9bfd373730f9fa46454bf9cca65039edae4651

Also @ppumkin not sure if this is the correct place for your question, but yes, it works with Moq :)

Update:
With some experimenting, I found that Coverlet returns 0% code coverage when I run my tests located in SLOBSharp.Tests.Domain.Services. It seems that Coverlet struggles when a test spins up a NamedPipeServerStream to run it's tests.

I use NCrunch to run my tests locally and it has no issue calculating metrics in this scenario, so not sure why Coverlet struggles?

Update 2:
I've since changed the way I was testing the named pipe parts in my project by mocking streams. So, I've updated the link above to point to the point in time where the issue was occuring.

p10tyr · 2018-11-05T09:39:04Z

That is interesting. Coverlet needs to rewrite parts of the DLL and I have noticed if it fails to do so it wont crash and burn it will just return 0% coverage.

(I need to double check this but that is what I remember happening when I was messing around with my extension using coverlet - Probably need a message to say that the DLL's was not attached instead of returning 0%)

I wonder if there is a race condition or just a security constraint on rewriting the DLL

nCrunch uses a different technique to monitor for coverage.

queen-of-code · 2018-11-28T23:58:17Z

I can confirm that I have at least one test project using Moq that works fine, but it's true that the one that doesn't work is using Moq.

Nonary · 2018-12-03T04:13:50Z

I also get 0% test coverage on my project at work, also have Moq installed.

amitwats · 2018-12-06T04:05:31Z

Facing the same issue. Got a feeling it is something on the lines of caching but cant put my figure to the pattern followed. I tried running the test with different options and sometimes it works. But this is not consistent

bednar · 2018-12-06T05:06:45Z

I can confirm same issue. The coverage have been 0% after we added a dependency to Reactive Extensions (Rx) for .NET.

See Travis CI Log...

SlowLogicBoy · 2018-12-06T09:44:11Z

For me coverage doesn't work unless I add references to:

XUnit projects:
1. Microsoft.NET.Test.Sdk
2. xunit.runner.visualstudio
NUnit Projects:
1. NUnit3TestAdapter
2. Microsoft.NET.Test.Sdk

And remove DebugType property from csproj files. Sometimes it doesnt work with DebugType Full, that includes projects you are testing.

ViktorHofer · 2018-12-10T20:08:08Z

Same here in corefx. Spinning up a Ubuntu machine now to debug what's going on.

ViktorHofer · 2018-12-10T21:34:15Z

Ok sorry, it is working for me, I was missing passing the include-directory with the right formatted path to coverlet.

amitwats · 2018-12-12T06:23:00Z

That is interesting. Coverlet needs to rewrite parts of the DLL and I have noticed if it fails to do so it wont crash and burn it will just return 0% coverage.

(I need to double check this but that is what I remember happening when I was messing around with my extension using coverlet - Probably need a message to say that the DLL's was not attached instead of returning 0%)

I wonder if there is a race condition or just a security constraint on rewriting the DLL

nCrunch uses a different technique to monitor for coverage.

Based on this comment I tried running the console in Admin mode and for atleast one time it gave a accurate coverage. Not sure this would be reproducible though.

ghost · 2019-07-20T21:08:41Z

Continuing to get more info on the 0% coverage, I'm running coverlet.console on the debugger, with references to the coverlet.core project. I added traces to ModuleTrackerTemplate class and I can see calls to RecordHit and Registering the Unload Events. However, the UnloadModule method doesn't seem to be called for the tests for which I get 0% coverage. The only other thing I see is a lonely message in the Application Output window that reads:

EventSource Error: ERROR: Exception during construction of EventSource System.Collections.Concurrent.ConcurrentCollectionsEventSource: 1

without any additional context. Does this message rings a bell to anybody as to whether or not may be related to an exception that would prevent the UnloadModule from running?

MarcoRossignoli · 2019-09-06T13:48:14Z

I have released vstest with this, but this will flow into dotnet sdk 2.2.4 only.

@vagisha-nidhi can you confirm that last release of sdk 2.2.401(runtime 2.2.6) inject inproc data collector?

queen-of-code · 2019-09-06T15:57:21Z

I can confirm that at least some of my issues were FIXED in this combo of releases. Thank you all for your effort here!

…

On Fri, Sep 6, 2019 at 6:48 AM Marco Rossignoli ***@***.***> wrote: I have released vstest with this, but this will flow into dotnet sdk 2.2.4 only. @vagisha-nidhi <https://github.com/vagisha-nidhi> can you confirm that last release of sdk 2.2.401(runtime 2.2.6) inject inproc data collector? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#110>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEIOSMJYPTER34BZUKBBH5DQIJNSNANCNFSM4FCAAE2A> .

MarcoRossignoli · 2019-09-06T16:00:42Z

I can confirm that at least some of my issues were FIXED in this combo

Glad to hear thank's @queen-of-code for info.

I'll update docs to reflect this.
I'll close this issue after @vagisha-nidhi double confirm!

AmirSasson · 2019-09-11T14:48:00Z

I am having the same problem.
Code covarage is 0%
without any error or clue what is wrong.
using dotnet core version 2.2.402 ,coverlet version 2.6.3,
any way to get a clue what is wrong?


Test Run Successful.
Total tests: 510
     Passed: 510
 Total time: 11.9588 Seconds

Calculating coverage result...
  Generating report 'C:\Users\Amir\source\repos\tracking-api\tests\tests\coverage.json'

+-----------------------+------+--------+--------+
| Module                | Line | Branch | Method |
+-----------------------+------+--------+--------+
| TrackingApi.Apps.Core | 0%   | 0%     | 0%     |
+-----------------------+------+--------+--------+
| TrackingApi.Common    | 0%   | 0%     | 0%     |
+-----------------------+------+--------+--------+
| TrackingApi.Contracts | 0%   | 0%     | 0%     |
+-----------------------+------+--------+--------+
| TrackingApi.Domains   | 0%   | 0%     | 0%     |
+-----------------------+------+--------+--------+
| TrackingApi.Web       | 0%   | 0%     | 0%     |
+-----------------------+------+--------+--------+

+---------+------+--------+--------+
|         | Line | Branch | Method |
+---------+------+--------+--------+
| Total   | 0%   | 0%     | 0%     |
+---------+------+--------+--------+
| Average | 0%   | 0%     | 0%     |
+---------+------+--------+--------+

MarcoRossignoli · 2019-09-11T15:03:41Z

using dotnet core version 2.2.402 ,coverlet version 2.6.3,

@AmirSasson you're using msbuild integration...you should try collectors to understand if this is the correct issue you're experiencing https://github.com/tonerdo/coverlet/blob/master/Documentation/VSTestIntegration.md

AmirSasson · 2019-09-12T07:13:13Z

hi @MarcoRossignoli , i tried the vstest integration using the collectors.
running dotnet test --collect:"XPlat Code Coverage"
its still giving me zero coverage.

does it matter if use xunit and not "mstest"?

(btw i was using msbuild integration cause it has "out of the box" minimum threshold that i have enforced in our CI).
i wonder if there is an error that i can see to understand what is wrong.. cause obviously everything worked before i changed some of the tests.

MarcoRossignoli · 2019-09-12T07:35:54Z

@AmirSasson you mean dotnet vstest --collect:"XPlat Code Coverage"?
And if you use vstest you need to publish before https://github.com/tonerdo/coverlet/blob/master/Documentation/VSTestIntegration.md#coverlet-integration-with-vstest

AmirSasson · 2019-09-12T07:50:18Z

hi @MarcoRossignoli GOOD NEWS

I have managed to solve this.

apparently i had a running thread that probably was still running when the code coverage was calculated.
and now, that i wait for it in the Test Fixture Dispose method, all starting to work out!
i still dont understand how the average is so low. (but this is another problem)
thanks for your help.
if someone experience the same, i would recommend to double check if all resources are cleaned in the end of the tests. (especially when using Fixture collection).

                                                                                                                                                                                              
Test Run Successful.
Total tests: 510
     Passed: 510
 Total time: 20.7397 Seconds

Calculating coverage result...
  Generating report 'C:\Users\Amir\source\repos\tracking-api\tests\tests\coverage.json'

+-----------------------+--------+--------+--------+
| Module                | Line   | Branch | Method |
+-----------------------+--------+--------+--------+
| TrackingApi.Apps.Core | 100%   | 87.5%  | 100%   |
+-----------------------+--------+--------+--------+
| TrackingApi.Common    | 91.57% | 86.45% | 94.44% |
+-----------------------+--------+--------+--------+
| TrackingApi.Contracts | 99.15% | 76.27% | 97.16% |
+-----------------------+--------+--------+--------+
| TrackingApi.Domains   | 94.32% | 85.64% | 90.97% |
+-----------------------+--------+--------+--------+
| TrackingApi.Web       | 91.88% | 66.92% | 85.71% |
+-----------------------+--------+--------+--------+

+---------+---------+---------+---------+
|         | Line    | Branch  | Method  |
+---------+---------+---------+---------+
| Total   | 94.73%  | 81.99%  | 92.74%  |
+---------+---------+---------+---------+
| Average | 18.946% | 16.398% | 18.548% |
+---------+---------+---------+---------+

Rjae · 2019-09-14T03:33:31Z

@MarcoRossignoli I just finished configuring a GH Actions workflow. All is well except my 100% coverage solution is reported as 0% running under GH Actions. The same project running on Travis (xenial) and local (Windows) is reported as 100%.

I've tried 2.2.108, 204, 401, 402 - along with 2.6.3. I tried all the workarounds listed on this thread, no luck. If you're aware of any other fixes I may try please let me know.

name: CI

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v1

    - name: Runtime
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 2.2.204

    - name: Tools
      run: dotnet tool install --global dotnet-reportgenerator-globaltool

    - name: Build
      run: dotnet build

    - name: Test
      run: dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=OpenCover /p:Exclude=\"[xunit.*]*,[*]*.Migrations.*\"

    - name: Report
      run: |
        export DOTNET_ROOT=/opt/hostedtoolcache/dncs/2.2.204/x64
        export MSBuildSDKsPath=$DOTNET_ROOT/sdk/$(${DOTNET_ROOT}/dotnet --version)/Sdks
        export PATH="$PATH:$HOME/.dotnet/tools"
        ~/.dotnet/tools/reportgenerator "-reports:*Tests/coverage.opencover.xml" "-targetdir:coverage-reports"

    - name: Artifacts
      uses: actions/upload-artifact@master
      with:
        name: Coverage
        path: coverage-reports

MarcoRossignoli · 2019-09-14T06:58:22Z

@Rjae you should try with collectors and version v2.2.401 or newer sdk, the issue could be related to vs test platform that kill process before hit files are written and could be dependant on ci machine load.

Rjae · 2019-09-15T23:19:56Z

Thanks @MarcoRossignoli I'm giving collectors a shot right now. Getting

$ dotnet test --settings <absolute path to runsettings>
Data collection : Could not find data collector 'XPlat code coverage'

My runsettings file:

<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="XPlat code coverage">
        <Configuration>
          <Format>opencover</Format>
          <Exclude>[xunit.*]*,[*]*.Migrations.*</Exclude>
          <ExcludeByAttribute>Obsolete,GeneratedCodeAttribute,CompilerGeneratedAttribute</ExcludeByAttribute>
          <SingleHit>false</SingleHit>
          <UseSourceLink>true</UseSourceLink>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
</RunSettings>

I've followed the doc - but must be missing something. I'll keep digging and update thread here.

Rjae · 2019-09-15T23:27:06Z

Force rebuild was necessary, so locally all good. Will use in GH workflow now and update thread with results.

Rjae · 2019-09-16T00:10:57Z

Sadly, switching to collector did not resolve issue. Again, works locally (both collector and msbuild), on Travis, and on Jenkins.

MarcoRossignoli · 2019-09-16T08:29:46Z

@Rjae can you try to enable coverlet logging? msbuild collectors

MarcoRossignoli · 2019-09-23T08:53:44Z

I'm going to close this issue as resolved by in-proc collectors.
If case of issue you can enable collectors logging through https://github.com/tonerdo/coverlet/blob/master/Documentation/Troubleshooting.md#collectors-integration

Rjae · 2019-09-28T15:33:22Z

@MarcoRossignoli I finally had time to enable logging (see attached). Not knowing coverlet code yet, I mostly skimmed through. I did notice many socket exceptions in host log. Please let me know if there is something else I may try.
Logs.zip

MarcoRossignoli · 2019-09-28T16:20:26Z

@Rjae can you open new issue(I'm oof at the moment) with copy paste your messages from this thread so we can go on there on clean one.

andrei-m-code · 2019-12-06T14:53:37Z

Just curious, is this even necessary?

<IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
<PrivateAssets>all</PrivateAssets>

MarcoRossignoli · 2019-12-06T16:02:37Z

Needed to avoid that msbuild harness package coverlet.msbuild(also props,targets) flow to project that uses it https://docs.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files#controlling-dependency-assets

aburgett87 mentioned this issue May 29, 2018

Migrating to 2.0.0 #107

Closed

This was referenced Jul 23, 2019

Running coverlet global tool over dotnet build intermittently fails to open file when calculating coverage #491

Closed

Coverage is not calculated if there is rabbitmq used #506

Closed

MarcoRossignoli mentioned this issue Aug 8, 2019

run coverlet as global tool with vstest #513

Closed

MarcoRossignoli mentioned this issue Sep 6, 2019

Update guide for inproc collector #535

Merged

ViktorHofer mentioned this issue Sep 11, 2019

Add support for code coverage using Coverlet + ReportGenerator dotnet/arcade#3919

Closed

MarcoRossignoli mentioned this issue Sep 12, 2019

Windows build produces much higher coverage results than macOS. #239

Closed

MarcoRossignoli mentioned this issue Sep 18, 2019

Fix exclude by files #524

Merged

MarcoRossignoli closed this as completed Sep 23, 2019

MarcoRossignoli unpinned this issue Sep 23, 2019

Rjae mentioned this issue Sep 28, 2019

0% Coverage Reported in GitHub Workflow #575

Closed

0% coverage is being shown after running tests #110

0% coverage is being shown after running tests #110

Comments

robvanpamel commented May 28, 2018

tonerdo commented May 28, 2018

aburgett87 commented May 29, 2018 • edited Loading

aburgett87 commented May 29, 2018

robvanpamel commented May 29, 2018

basilfx commented May 29, 2018 • edited Loading

mikkokupsu commented Jun 6, 2018

tlogik commented Jun 8, 2018 • edited Loading

queen-of-code commented Aug 16, 2018 • edited Loading

tonerdo commented Aug 16, 2018

queen-of-code commented Aug 17, 2018 • edited Loading

tonerdo commented Sep 1, 2018

queen-of-code commented Sep 5, 2018 • edited Loading

tonerdo commented Sep 7, 2018

tonerdo commented Sep 9, 2018

queen-of-code commented Sep 12, 2018

tonerdo commented Sep 16, 2018

queen-of-code commented Sep 19, 2018

twcclegg commented Oct 5, 2018 • edited Loading

p10tyr commented Oct 23, 2018

StephenMP commented Oct 30, 2018 • edited Loading

p10tyr commented Nov 5, 2018

queen-of-code commented Nov 28, 2018

Nonary commented Dec 3, 2018

amitwats commented Dec 6, 2018

bednar commented Dec 6, 2018

SlowLogicBoy commented Dec 6, 2018

ViktorHofer commented Dec 10, 2018

ViktorHofer commented Dec 10, 2018

amitwats commented Dec 12, 2018

ghost commented Jul 20, 2019

MarcoRossignoli commented Sep 6, 2019

queen-of-code commented Sep 6, 2019 via email

MarcoRossignoli commented Sep 6, 2019

AmirSasson commented Sep 11, 2019

MarcoRossignoli commented Sep 11, 2019

AmirSasson commented Sep 12, 2019 • edited Loading

does it matter if use xunit and not "mstest"?

MarcoRossignoli commented Sep 12, 2019 • edited Loading

AmirSasson commented Sep 12, 2019 • edited Loading

I have managed to solve this.

Rjae commented Sep 14, 2019

MarcoRossignoli commented Sep 14, 2019 • edited Loading

Rjae commented Sep 15, 2019

Rjae commented Sep 15, 2019

Rjae commented Sep 16, 2019

MarcoRossignoli commented Sep 16, 2019

MarcoRossignoli commented Sep 23, 2019

Rjae commented Sep 28, 2019

MarcoRossignoli commented Sep 28, 2019 • edited Loading

andrei-m-code commented Dec 6, 2019

MarcoRossignoli commented Dec 6, 2019

aburgett87 commented May 29, 2018 •

edited

Loading

basilfx commented May 29, 2018 •

edited

Loading

tlogik commented Jun 8, 2018 •

edited

Loading

queen-of-code commented Aug 16, 2018 •

edited

Loading

queen-of-code commented Aug 17, 2018 •

edited

Loading

queen-of-code commented Sep 5, 2018 •

edited

Loading

twcclegg commented Oct 5, 2018 •

edited

Loading

StephenMP commented Oct 30, 2018 •

edited

Loading

AmirSasson commented Sep 12, 2019 •

edited

Loading

MarcoRossignoli commented Sep 12, 2019 •

edited

Loading

AmirSasson commented Sep 12, 2019 •

edited

Loading

MarcoRossignoli commented Sep 14, 2019 •

edited

Loading

MarcoRossignoli commented Sep 28, 2019 •

edited

Loading