BUG: (Completely) Missing Coverage

[nopara73]

Maybe related: #39

Behavior.

What I do

1. Add coverlet.msbuild NuGet to my test project.
2. Run Tor (needed for my tests to successfully complete.)
3. Run dotnet test /p:CollectCoverage=true.

I also tried creating a new project and everything works fine, however with my current project (https://github.com/zkSNACKs/WalletWasabi/) it silently fails.

[nopara73]

Note: Neither the coverage.xml, nor the coverage.json files are generated.

[nopara73]

To add to the mystery, I tried to add it to Appveyor, and it "kind of" worked:
https://ci.appveyor.com/project/zkSNACKs/walletwasabi/build/1.0.118

So it worked for the WalletWasabi.Backend module (ASP.NET Core web API project), but not for the WalletWasabi (.NET Core library project) module and exited with an error? I am not sure if I interpret the results correctly or if it's even related.

My changes were nothing special:

[nopara73]

And travis (both on Linux and OSX) is even stranger:

https://travis-ci.org/zkSNACKs/WalletWasabi/jobs/373441646
https://travis-ci.org/zkSNACKs/WalletWasabi/jobs/373441646

[tonerdo]

Thanks for reporting this. I'll take a look at your sample project and see if I can figure it out

[nopara73]

@tonerdo Thanks! I put together some build instructions for Windows so to not get stuck:

1. Get .NET Core
2. Download the Tor Experd Bundle (Not the browser.) https://www.torproject.org/download/download
3. Run tor.exe
4. Fire up command line:

git clone https://github.com/zkSNACKs/WalletWasabi
cd WalletWasabi
dotnet restore && dotnet build
cd WalletWasabi.Tests
dotnet test

Notes:

1. dotnet test takes about 3 minutes, the first time it'll take 1-2 minutes longer, because it downloads bitcoind. This daemon is used by the tests in regression test mode to simulate the Bitcoin network.

2. If you'd happen to terminate the tests immaturely, make sure you also terminate the bitcoind daemon process from the Task Manager, otherwise next time your tests would fail.

3. Tests are using %appdata%\WalletWasabi folder to log and for other things, so you can delete it after you no longer wish to do anything with the software anymore.

[SteveGilham]

Having tried this, the first obvious problem is that there's an assembly being (indirectly) referenced that's in ~/.nuget, but is not copied locally. That manifests as an assembly resolution failure when trying to write an instrumented assembly -- the process then just carries on obliviously.

05/04/2018 07:52:09>dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=opencover
Build started, please wait...
Build completed.

Test run for C:\Users\steve\Documents\GitHub\WalletWasabi\WalletWasabi.Tests\bin\Debug\netcoreapp2.0\WalletWasabi.Tests.dll(.NETCoreApp,Version=v2.0)
C:\Users\steve\.nuget\packages\coverlet.msbuild\1.2.0\build\netstandard2.0\coverlet.msbuild.targets(14,5): error : Failed to resolve assembly: 'NBitcoin, Version=4.1.1.3, Culture=neutral, PublicKeyToken=null' [C:\Users\steve\Documents\GitHub\WalletWasabi\WalletWasabi.Tests\WalletWasabi.Tests.csproj]

[nopara73]

@SteveGilham I actually tried it. Copying the assembly manually only solved the problem in altcover, but not in coverlet, it still silently fails.

[pwelter34]

My project is also not getting code coverage results. Its not covering all the dependent assemblies. My project is located here. https://github.com/loresoft/FluentCommand

Is there a way to setup debug local?

The build output is here: https://ci.appveyor.com/project/LoreSoft/fluentcommand

[Drag13]

I think I have related bug. On Windows all is ok, but on Linux with Travis I have 0 coverage report.

dotnet build CIDemo.sln -c Release
dotnet test -c Release test/test.csproj /p:CollectCoverage=true /p:CoverletOutputFormat=opencover

Any idea?

PackageReference Include="coverlet.msbuild" Version="2.0.0"
Demo repository
Travis report

[Drag13]

Downgrading to 1.2.0 Fixed problem. But for 2.0 it still exists

[coenm]

I'm experiencing the same issue.

2.0 -> Travis = no coverage
2.0 -> Local windows => has coverage
1.2 -> Travis => has coverage

[n00j]

Have you tried adding the following to your test.csproj?

<CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>

I was running into problems with coverlet not being able to locate dll's and seems to fix it.

[varsnotwars]

@nopara73, this works correctly now after adding <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
Thanks @n00j 👍

[n00j]

@varnotwars no problem! It would be useful if this was added to the documentation as I believe it solves a lot of the 0% coverage issues and errors

[tonerdo]

Will keep this open until I add it to the documentation, thanks @n00j

[rubberduck203]

In my case, upgrading my test project from netcoreapp1.1 to netcoreapp2.0 fixed this.

[asbjornu]

Just as a FYI, my test project is netcoreapp2.1 and has CopyLocalLockFileAssemblies set to true, but dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=opencover still produces no code coverage file on disk.

[abhurtun]

the same as @asbjornu no code coverage 👎

[abhurtun]

I found out if you run in Release mode is doesn't cover all the dlls but if you do Debug it works

[nopara73]

@asbjornu @abhurtun Neither the trick works for us anymore.

[asbjornu]

I can't remember exactly what I did, but I managed to generate a code coverage report and upload it to Codecov in linked-data-dotnet/json-ld.net#38.

[nopara73]

Also it might be also helpful to point out that @SteveGilham's altcover is still working for .NET Core projects. It's less straightforward to use but at least it works:

Install altcover NuGet to the WalletWasabi.Tests project and Coverage Glutters extension to vscode.

dotnet build
dotnet C:\Users\user\.nuget\packages\altcover\5.0.663\tools\netcoreapp2.0\altcover.dll --save --inplace "-i=bin\Debug\netcoreapp2.2"
dotnet test
dotnet C:\Users\user\.nuget\packages\altcover\5.0.663\tools\netcoreapp2.0\altcover.dll runner --collect "-r=bin\Debug\netcoreapp2.2" "-l=lcov.info"

Run vscode and click on "Watch":

[SteveGilham]

Just interjecting to remark that doing

dotnet test /p:AltCover=true /p:AltCoverLcovReport=lcov.info

to have the same effect has been supported since early last summer.

[MarcoRossignoli]

Hi guys I saw a lot of activity on this issue..I would like to know if this issue is still relevant, if so I would like to repro on my local and in case fix or update documentation accordingly.
Do you have some repro that still fail?

[MarcoRossignoli]

close for stale...further discussion on similar issue https://github.com/dotnet/cli/issues/12705#issuecomment-535150372
With 3.0 all assets will be copied so should resolve by design this issue...for shared framework at the moment we don't have an elegant solution, only manually copy dll to output folder.

[PSanetra]

We are still facing this issue. The workarounds mentioned here did not help.

[MarcoRossignoli]

@PSanetra please can you file an new issue with you case?