Unit testing example does not work in docker

[alexellis]

Issue Title

With this article:

https://github.com/dotnet/core-docs/blob/master/docs/core/testing/unit-testing-with-dotnet-test.md

General

I was able to set up a unit test project and have this restored/running on my Mac but when it came to using the Docker image for Preview 1 or 2 it failed.

Errors in /root/dn/test/project.json
    Unable to resolve 'RateCalc.Engine' for '.NETCoreApp,Version=v1.0'.

The code is available here:

https://github.com/alexellis/dncore-market-rates

I am a bit confused about these two lines in the unit test project:

      "imports": [
        "dotnet5.4",
        "portable-net451+win8"
      ]

The other projects created are using dncore 5.0 - does this mean we need Mono or a full .NET runtime in order to use xunit?

1. Suggest reviewers
   • If you know someone who can provide feedback, use '@' to ask them to review.

@BillWagner / @ Steve Smith

I've also provided a Dockerfile.

[BillWagner]

@alexellis Thanks for writing the issue, and providing a link to your project.

I think you are seeing the error because your tests are in the 'test' directory, and not it in a subdirectory of 'test'. Can you move that project into a sub-directory (e.g. 'test/EngineTests') and let us know if it works?

We can update the article to make that more clear.

Thanks.

[alexellis]

@BillWagner I'm afraid that didn't solve the problem. This works on my Mac, but not within Docker with either of the official images. Here are more details.

This is my resulting structure: (You can clone it and try it - if you're on Mac or Windows 10 there is a 1-click install for Docker that runs without VirtualBox)

./Dockerfile
./global.json
./LICENSE
./src
./src/RateCalc.App
./src/RateCalc.App/market.csv
./src/RateCalc.App/Program.cs
./src/RateCalc.App/project.json
./src/RateCalc.App/project.lock.json
./src/RateCalc.Engine
./src/RateCalc.Engine/CsvParser.cs
./src/RateCalc.Engine/project.json
./src/RateCalc.Engine/project.lock.json
./src/RateCalc.Engine/Rates.cs
./test
./test/RateCalc.Engine.Tests
./test/RateCalc.Engine.Tests/CsvParserTest.cs
./test/RateCalc.Engine.Tests/project.json
./test/RateCalc.Engine.Tests/project.lock.json
./test/RateCalc.Engine.Tests/Tests.cs

Same issue, works on Mac locally, but not in Docker image.

error: Unable to resolve 'RateCalc.Engine' for '.NETCoreApp,Version=v1.0'.
info : Committing restore...
log  : Writing lock file to disk. Path: /root/dn/test/RateCalc.Engine.Tests/project.lock.json
log  : /root/dn/test/RateCalc.Engine.Tests/project.json
log  : Restore failed in 7406ms.

Errors in /root/dn/test/RateCalc.Engine.Tests/project.json
    Unable to resolve 'RateCalc.Engine' for '.NETCoreApp,Version=v1.0'.

NuGet Config files used:
    /root/.nuget/NuGet/NuGet.Config

Feeds used:
    https://api.nuget.org/v3/index.json

Installed:
    124 package(s) to /root/dn/test/RateCalc.Engine.Tests/project.json
The command '/bin/sh -c dotnet restore' returned a non-zero code: 1

Whatever was installed on my Mac

dotnet --info
.NET Command Line Tools (1.0.0-preview2-003121)

Product Information:
 Version:            1.0.0-preview2-003121
 Commit SHA-1 hash:  1e9d529bc5

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.11
 OS Platform: Darwin
 RID:         osx.10.11-x64

Preview 1 on Docker:

dotnet --info
.NET Command Line Tools (1.0.0-preview1-002702)

Product Information:
 Version:     1.0.0-preview1-002702
 Commit Sha:  6cde21225e

Runtime Environment:
 OS Name:     debian
 OS Version:  8
 OS Platform: Linux
 RID:         debian.8-x64

Preview 2 on Docker

docker run microsoft/dotnet:latest dotnet --info
.NET Command Line Tools (1.0.0-preview1-002702)

Product Information:
 Version:     1.0.0-preview1-002702
 Commit Sha:  6cde21225e

Runtime Environment:
 OS Name:     debian
 OS Version:  8
 OS Platform: Linux
 RID:         debian.8-x64

What is interesting is that the RateCalc.App application compiles without warnings.

Can you also speak to the odd part of the project.json file.. ?

      "imports": [
        "dotnet5.4",
        "portable-net451+win8"
      ]

[BillWagner]

Thanks for doing that research. Let me do a bit of research, and I'll get back to you.

[blackdwarf]

@alexellis in your Dockerfile, could you test out what happens when you change the WORKDIR to point to the root folder, where the global.json is and not to the actual test folder?

Actually, come to think of it, you could run dotnet restore in the solution folder first and then just WORKDIR into other folders and do dotnet build.

[jeremymeng]

I think @alexellis is missing the global.json in the Docker image. See devlooped/moq#168 (comment).

[blackdwarf]

@jeremymeng the Dockerfile in the repo shows that global.json is being added.

[alexellis]

It is now working inside Docker, but does a complete restore every build. Is there a way of changing the instruction order to take advantage of the step caching?

I.e. in nodejs with npm you can restore before adding any code.

[mairaw]

@blackdwarf @jeremymeng can you help with @alexellis's last question?

[jeremymeng]

@glennc might have some ideas for this scenario (dotnet restore caching in Docker).

[mairaw]

@spboyer can you take a look at this one?

[spboyer]

@alexellis when you say that it is doing the restore every time, you are stating that each time you build the image?

The microsoft/dotnet:1.0.0-preview2-sdk image pre-loads the Nuget cache in the base image. When you see Step 8 in the output

Step 8 : RUN dotnet restore
 ---> Running in 8a3175ee5bf2
log  : Restoring packages for /root/dn/test/RateCalc.Engine.Tests/project.json...
log  : Restoring packages for /root/dn/src/RateCalc.App/project.json...
log  : Restoring packages for /root/dn/src/RateCalc.Engine/project.json...
log  : Writing lock file to disk. Path: /root/dn/src/RateCalc.Engine/project.lock.json
log  : Writing lock file to disk. Path: /root/dn/src/RateCalc.App/project.lock.json
log  : /root/dn/src/RateCalc.Engine/project.json
log  : Restore completed in 1053ms.
log  : /root/dn/src/RateCalc.App/project.json
log  : Restore completed in 1054ms.
log  : Installing System.Threading.Overlapped 4.0.1-rc2-24027.
log  : Installing System.IO.FileSystem.Watcher 4.0.0-rc2-24027.
log  : Installing System.Security.Cryptography.Csp 4.0.0-rc2-24027.
log  : Installing System.Security.Cryptography.Cng 4.1.0-rc2-24027.
.
.
.

the build is not downloading these files from Nuget, but copying them from the local cache that exists within the image. Within your Dockerfile, you are adding the application files to the image each time it is built.

ADD ./src ./src
ADD ./test ./test

In my test, I never called dotnet restore on my local machine which means that the needed assets would not exists in the ./src or ./test directories and thus needing and Installation when dotnet restore is called.

Given the amount of time to run this, Time elapsed 00:00:01.1501594; I would say that the cache being used within the Docker image is acceptable given the scenario of not ever having the dotnet resources on the local machine.

The alternative way to accomplish this would be to run the restore locally and use microsoft/dotnet:latest as the base image within your Dockerfile. This change would also require you to run dotnet publish and copy only the compiled resource to the image.

cc:/ @glennc @SteveLasker

[alexellis]

I'm sure it was not taking 1 second (a lot longer). Let me try to reproduce the issue - I guess this was raised 3 months ago maybe things have changed.

[spboyer]

@alexellis - 3 months ago I don't believe we were loading the Nuget cache in the base image, there has been some improvements there.

[alexellis]

$ git clone https://github.com/alexellis/dncore-market-rates

@spboyer so please could you clone the repo - do your initial build and then just edit the name of one of the tests in the CsvParserTests.cs file?

$ time docker build -t market-rates .

real    0m41.416s
user    0m0.031s
sys 0m0.032s

[alexellis]

Output from editing a unit test name:

Sending build context to Docker daemon 1.154 MB
Step 1 : FROM microsoft/dotnet:1.0.0-preview2-sdk
 ---> 96d122fe36cb
Step 2 : MAINTAINER alexellis2@gmail.com
 ---> Using cache
 ---> f79361382308
Step 3 : RUN mkdir -p /root/dn/
 ---> Using cache
 ---> c12ccc8b1ed9
Step 4 : WORKDIR /root/dn/
 ---> Using cache
 ---> d7eac690bf40
Step 5 : ADD ./global.json ./
 ---> Using cache
 ---> d64afc65de75
Step 6 : ADD ./src ./src
 ---> Using cache
 ---> 9fd1ce68be73
Step 7 : ADD ./test ./test
 ---> 40868161d6fd
Removing intermediate container f82a6236d2c4
Step 8 : RUN dotnet restore
 ---> Running in be7c12d59a7a
log  : Restoring packages for /root/dn/src/RateCalc.Engine/project.json...
log  : Restoring packages for /root/dn/src/RateCalc.App/project.json...
log  : Installing Microsoft.NETCore.Jit 1.0.2.
log  : Installing Microsoft.NETCore.Runtime.CoreCLR 1.0.2.
log  : Installing Microsoft.NETCore.App 1.0.0.
log  : Writing lock file to disk. Path: /root/dn/src/RateCalc.App/project.lock.json
log  : Writing lock file to disk. Path: /root/dn/src/RateCalc.Engine/project.lock.json
log  : /root/dn/src/RateCalc.App/project.json
log  : Restore completed in 8414ms.
log  : /root/dn/src/RateCalc.Engine/project.json
log  : Restore completed in 8571ms.
log  : Restoring packages for /root/dn/test/RateCalc.Engine.Tests/project.json...
log  : Installing System.Threading.Overlapped 4.0.1-rc2-24027.
log  : Installing System.IO.FileSystem.Watcher 4.0.0-rc2-24027.
log  : Installing System.Security.Cryptography.Cng 4.1.0-rc2-24027.
log  : Installing System.Security.Cryptography.Csp 4.0.0-rc2-24027.
log  : Installing Microsoft.NETCore.Windows.ApiSets 1.0.1-rc2-24027.
log  : Installing Microsoft.Win32.Registry 4.0.0-rc2-24027.
log  : Installing System.Threading.ThreadPool 4.0.10-rc2-24027.
log  : Installing System.Security.Cryptography.X509Certificates 4.1.0-rc2-24027.
log  : Installing System.Security.Cryptography.Encoding 4.0.0-rc2-24027.
log  : Installing System.Security.Cryptography.OpenSsl 4.0.0-rc2-24027.
log  : Installing runtime.native.System.Net.Http 4.0.1-rc2-24027.
log  : Installing runtime.native.System 4.0.0-rc2-24027.
log  : Installing System.Diagnostics.DiagnosticSource 4.0.0-rc2-24027.
log  : Installing runtime.native.System.Security.Cryptography 4.0.0-rc2-24027.
log  : Installing System.Buffers 4.0.0-rc2-24027.
log  : Installing Microsoft.NETCore.Targets 1.0.1-rc2-24027.
log  : Installing Microsoft.NETCore.Runtime.CoreCLR 1.0.2-rc2-24027.
log  : Installing System.Runtime.Serialization.Primitives 4.1.1.
log  : Installing Microsoft.NETCore.Runtime.Native 1.0.2-rc2-24027.
log  : Installing System.Diagnostics.Process 4.1.0-rc2-24027.
log  : Installing FluentAssertions 4.12.0.
log  : Installing runtime.native.System.IO.Compression 4.1.0-rc2-24027.
log  : Installing NuGet.Packaging.Core.Types 3.5.0-beta-final.
log  : Installing System.Threading.Tasks.Extensions 4.0.0-rc2-24027.
log  : Installing System.Net.Http 4.0.1-rc2-24027.
log  : Installing System.Net.Primitives 4.0.11-rc2-24027.
log  : Installing System.IO.FileSystem 4.0.1-rc2-24027.
log  : Installing System.AppContext 4.1.0-rc2-24027.
log  : Installing Microsoft.Win32.Primitives 4.0.1-rc2-24027.
log  : Installing System.IO.Compression.ZipFile 4.0.1-rc2-24027.
log  : Installing Microsoft.NETCore.Platforms 1.0.1-rc2-24027.
log  : Installing System.Globalization.Calendars 4.0.1-rc2-24027.
log  : Installing System.Runtime.Numerics 4.0.1-rc2-24027.
log  : Installing Microsoft.NETCore.Runtime 1.0.2-rc2-24027.
log  : Installing System.Net.Sockets 4.1.0-rc2-24027.
log  : Installing System.Threading.Timer 4.0.1-rc2-24027.
log  : Installing System.Runtime.InteropServices.RuntimeInformation 4.0.0-rc2-24027.
log  : Installing System.IO.FileSystem.Primitives 4.0.1-rc2-24027.
log  : Installing xunit 2.1.0.
log  : Installing Moq 4.6.25-alpha.
log  : Installing System.Console 4.0.0-rc2-24027.
log  : Installing System.Diagnostics.TraceSource 4.0.0-rc2-24027.
log  : Installing System.Runtime.InteropServices.PInvoke 4.0.0-rc2-24027.
log  : Installing dotnet-test-xunit 1.0.0-rc2-build10015.
log  : Installing Microsoft.DotNet.InternalAbstractions 1.0.0-rc2-002702.
log  : Installing NuGet.Common 3.5.0-beta-final.
log  : Installing System.IO.Compression 4.1.0-rc2-24027.
log  : Installing xunit.assert 2.1.0.
log  : Installing xunit.core 2.1.0.
log  : Installing System.Diagnostics.Tools 4.0.1-rc2-24027.
log  : Installing NuGet.Packaging.Core 3.5.0-beta-final.
log  : Installing System.Xml.ReaderWriter 4.0.11-rc2-24027.
log  : Installing System.Reflection.Emit.ILGeneration 4.0.1-rc2-24027.
log  : Installing System.Reflection.Extensions 4.0.1-rc2-24027.
log  : Installing System.Linq.Queryable 4.0.1-rc2-24027.
log  : Installing System.Collections.Immutable 1.2.0-rc2-24027.
log  : Installing System.Resources.ResourceManager 4.0.1-rc2-24027.
log  : Installing System.Text.Encoding.Extensions 4.0.11-rc2-24027.
log  : Installing System.Reflection.TypeExtensions 4.1.0-rc2-24027.
log  : Installing NETStandard.Library 1.5.0-rc2-24027.
log  : Installing NuGet.Versioning 3.5.0-beta-final.
log  : Installing NuGet.Frameworks 3.5.0-beta-final.
log  : Installing System.Runtime.InteropServices 4.1.0-rc2-24027.
log  : Installing System.Globalization 4.0.11-rc2-24027.
log  : Installing System.Reflection.Primitives 4.0.1-rc2-24027.
log  : Installing System.Security.Cryptography.Primitives 4.0.0-rc2-24027.
log  : Installing System.Diagnostics.Debug 4.0.11-rc2-24027.
log  : Installing System.Runtime.Handles 4.0.1-rc2-24027.
log  : Installing Castle.Core 4.0.0-alpha001.
log  : Installing Microsoft.Extensions.DependencyModel 1.0.0-rc2-002702.
log  : Installing Microsoft.CSharp 4.0.1-rc2-24027.
log  : Installing System.Linq 4.1.0-rc2-24027.
log  : Installing NuGet.Packaging 3.5.0-beta-final.
log  : Installing System.Xml.XDocument 4.0.11-rc2-24027.
log  : Installing System.Runtime.Extensions 4.1.0-rc2-24027.
log  : Installing System.Dynamic.Runtime 4.0.11-rc2-24027.
log  : Installing System.Collections 4.0.11-rc2-24027.
log  : Installing System.Reflection.Metadata 1.3.0-rc2-24027.
log  : Installing NuGet.RuntimeModel 3.5.0-beta-final.
log  : Installing System.Collections.Concurrent 4.0.12-rc2-24027.
log  : Installing System.Threading.Thread 4.0.0-rc2-24027.
log  : Installing System.Runtime.Loader 4.0.0-rc2-24027.
log  : Installing System.Security.Cryptography.Algorithms 4.1.0-rc2-24027.
log  : Installing System.Text.RegularExpressions 4.0.12-rc2-24027.
log  : Installing System.Text.Encoding 4.0.11-rc2-24027.
log  : Installing System.Threading 4.0.11-rc2-24027.
log  : Installing System.Linq.Expressions 4.0.0.
log  : Installing System.Reflection 4.1.0-rc2-24027.
log  : Installing System.Resources.ResourceManager 4.0.0.
log  : Installing System.Reflection.Primitives 4.0.0.
log  : Installing System.Reflection.Emit 4.0.1-rc2-24027.
log  : Installing System.IO 4.0.0.
log  : Installing System.Threading.Tasks 4.0.11-rc2-24027.
log  : Installing Microsoft.DotNet.ProjectModel 1.0.0-rc2-002702.
log  : Installing Microsoft.DiaSymReader.Native 1.3.3.
log  : Installing System.Linq.Expressions 4.0.11-rc2-24027.
log  : Installing System.Runtime 4.1.0-rc2-24027.
log  : Installing Microsoft.DiaSymReader 1.0.6.
log  : Installing xunit.runner.utility 2.1.0.
log  : Installing Newtonsoft.Json 7.0.1.
log  : Installing xunit.runner.reporters 2.1.0.
log  : Installing Microsoft.Extensions.Testing.Abstractions 1.0.0-preview1-002702.
log  : Installing System.Diagnostics.Debug 4.0.0.
log  : Installing System.Reflection.Extensions 4.0.0.
log  : Installing System.Text.RegularExpressions 4.0.0.
log  : Installing System.Globalization 4.0.0.
log  : Installing System.Net.Http 4.0.0.
log  : Installing System.Threading.Tasks 4.0.0.
log  : Installing System.Text.Encoding 4.0.0.
log  : Installing System.Net.Primitives 4.0.0.
log  : Installing System.ObjectModel 4.0.0.
log  : Installing System.Reflection 4.0.0.
log  : Installing System.Threading 4.0.0.
log  : Installing System.Runtime.Extensions 4.0.0.
log  : Installing System.Reflection.Emit.Lightweight 4.0.1-rc2-24027.
log  : Installing System.Linq 4.0.0.
log  : Installing System.ObjectModel 4.0.12-rc2-24027.
log  : Installing System.Collections 4.0.0.
log  : Installing System.IO 4.1.0-rc2-24027.
log  : Installing System.Runtime 4.0.0.
log  : Installing System.Diagnostics.Tracing 4.1.0-rc2-24027.
log  : Installing xunit.extensibility.execution 2.1.0.
log  : Installing xunit.abstractions 2.0.0.
log  : Installing xunit.extensibility.core 2.1.0.
log  : Writing lock file to disk. Path: /root/dn/test/RateCalc.Engine.Tests/project.lock.json
log  : /root/dn/test/RateCalc.Engine.Tests/project.json
log  : Restore completed in 10797ms.
 ---> 7c97a9ddad3c
Removing intermediate container be7c12d59a7a
Step 9 : WORKDIR /root/dn/test/RateCalc.Engine.Tests/
 ---> Running in e75557d84379
 ---> c3bc574c578c
Removing intermediate container e75557d84379
Step 10 : RUN dotnet build
 ---> Running in be5bb892aee7
Project RateCalc.Engine (.NETCoreApp,Version=v1.0) will be compiled because expected outputs are missing
Compiling RateCalc.Engine for .NETCoreApp,Version=v1.0

Compilation succeeded.
    0 Warning(s)
    0 Error(s)

Time elapsed 00:00:02.0088077
Project RateCalc.Engine.Tests (.NETCoreApp,Version=v1.0) will be compiled because dependencies changed
Compiling RateCalc.Engine.Tests for .NETCoreApp,Version=v1.0

Compilation succeeded.
    0 Warning(s)
    0 Error(s)

Time elapsed 00:00:01.7288984


 ---> 60b053b9458c
Removing intermediate container be5bb892aee7
Step 11 : RUN dotnet test
 ---> Running in 256918ae4540
Project RateCalc.Engine (.NETCoreApp,Version=v1.0) was previously compiled. Skipping compilation.
Project RateCalc.Engine.Tests (.NETCoreApp,Version=v1.0) was previously compiled. Skipping compilation.
xUnit.net .NET CLI test runner (64-bit debian.8-x64)
  Discovering: RateCalc.Engine.Tests
  Discovered:  RateCalc.Engine.Tests
  Starting:    RateCalc.Engine.Tests
Taking 150 off Alex @ rate of 0.5, left= 50
Taking 50 off Dave @ rate of 1, left= 0
Taking 150 off Alex @ rate of 0.005833333, left= 50
Taking 50 off Dave @ rate of 0.005833333, left= 0
  Finished:    RateCalc.Engine.Tests
=== TEST EXECUTION SUMMARY ===
   RateCalc.Engine.Tests  Total: 14, Errors: 0, Failed: 0, Skipped: 0, Time: 0.666s
SUMMARY: Total: 1 targets, Passed: 1, Failed: 0.
 ---> 7c3124ebfe16
Removing intermediate container 256918ae4540
Step 12 : WORKDIR /root/dn/src/RateCalc.App/
 ---> Running in 6d5e0f8418fc
 ---> f2a0108e1c24
Removing intermediate container 6d5e0f8418fc
Step 13 : CMD dotnet run
 ---> Running in d659664ac735
 ---> 12d1bbce2142
Removing intermediate container d659664ac735
Successfully built 12d1bbce2142

real    0m41.416s
user    0m0.031s
sys 0m0.032s

[spboyer]

@alexellis - see the results below. I want to say that given the scenario of restoring and building the application within the application; its pretty good. The Nuget cache is within the base container.

First run - no rates image

real    0m27.870s
user    0m0.021s
sys 0m0.023s

Second run - after change

real    0m22.084s
user    0m0.018s
sys 0m0.022s

[JRAlexander]

@BillWagner It looks like we could close this and replace with an update for 2.0 issue. Am I right?

[SteveLasker]

I recently posted an example for Unit Testing with Multistage dockerfiles:
Repo: https://github.com/SteveLasker/AspNetCoreMultiProject/tree/AddingUnitTests
Example: https://github.com/SteveLasker/AspNetCoreMultiProject/blob/AddingUnitTests/Web/Dockerfile

[JRAlexander]

I think we can close this. The articles have been updated.