Exception when targeting .NET Framework with .NET 7 installed

[YuhanKun]

Exception System.TypeInitializationException,    Exception thrown executing tests in B:\repos\Libraries\New folder\bin\Debug\net48\New folder.dll
The type initializer for 'NUnit.Engine.Services.RuntimeFrameworkService' threw an exception.
   at NUnit.Engine.Services.RuntimeFrameworkService.ApplyImageData(TestPackage package)
   at NUnit.Engine.Services.RuntimeFrameworkService.SelectRuntimeFramework(TestPackage package)
   at NUnit.Engine.Runners.MasterTestRunner.GetEngineRunner()
   at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnitEngine.NUnitEngineAdapter.Explore(TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\NUnitEngineAdapter.cs:line 88
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, IGrouping`2 testCases, TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 275
InnerException: System.ArgumentException: Unknown framework version 7.0
Parameter name: version
   at NUnit.Engine.RuntimeFramework.GetClrVersionForFramework(Version frameworkVersion)
   at NUnit.Engine.RuntimeFramework..ctor(RuntimeType runtime, Version version, String profile)
   at NUnit.Engine.RuntimeFramework.GetNetCoreRuntimesFromDirectoryNames(IEnumerable`1 dirNames)
   at NUnit.Engine.RuntimeFramework.FindDotNetCoreFrameworks()
   at NUnit.Engine.RuntimeFramework.FindAvailableFrameworks()
   at NUnit.Engine.RuntimeFramework.get_AvailableFrameworks()
   at NUnit.Engine.Services.RuntimeFrameworkService..cctor()
No test is available in B:\repos\Libraries\New folder\bin\Debug\net48\New folder.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

Reproduce

1. Install .NET 7 SDK
2. Create an unit test project by using dotnet new nunit
3. Update all dependencies to latest version
4. Target net48

Full csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFrameworks>net48</TargetFrameworks>
    <RootNamespace>New_folder</RootNamespace>
    <ImplicitUsings>disable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <LangVersion>preview</LangVersion>
    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.1.0" />
    <PackageReference Include="NUnit" Version="3.13.2" />
    <PackageReference Include="NUnit3TestAdapter" Version="4.2.1" />
    <PackageReference Include="NUnit.Analyzers" Version="3.3.0" />
    <PackageReference Include="coverlet.collector" Version="3.1.2" />
  </ItemGroup>

</Project>

[YuhanKun]

This issue should probably be moved to https://github.com/nunit/nunit-console repo.

Sorry for it that I didn't find that repo before.

[CharliePoole]

@YuhanKun Thanks for reporting this. I transferred the issue, as you see.

[madelson]

This is affecting me as well (see linked duplicate issue).

[manfred-brands]

Unless you need something specific from the .net 7 sdk, can you create a global.json file on your project limiting the sdk used for your net4x project to 6 or lower? I also installed the net7 sdk but had no problem running tests in my existing repositories.

[madelson]

@manfred-brands it feels like there ought to be a fix here given that the error is failing to parse in NUnit itself. Why shouldn't NUnit work with the net7.0 SDK?

[CharliePoole]

@madelson The PR to fix this issue (#1175) has been merged. Are you having a different problem than what's fixed there? Have you tried out the latest dev release on our myget feed?

That said, I see from #4085 that you actually ran into the problem using the NUnit 3 Visual Studio adapter. The adapter bundles its own copy of the NUnit engine, which is normally the latest release version at the time the adapter is released. Consequently, nothing we fix in the engine (which is where the problem lies) will be reflected in the adapter.

@OsirisTerje any thoughts on how we should deal with this?

[manfred-brands]

My suggested trick with global.json doesn't work.

NUnitEngine does call FindAvailableFrameworks regardless.
@CharliePoole Would it be an idea to only look for requested frameworks or alternatively ignore unknown frameworks so we don't get in the same situation when then next one is released?

[CharliePoole]

@manfred-brands The engine is intended to work in different environments. In some cases (like a GUI) it may be called on to load different assemblies at different times, requiring differnt frameworks. So it needs to know what frameworks are avaiable and does that at the time it is initialized.

At the specific point where we are throwing an exception, we could, conceivably not throw it. However, now that 7.0 is fixed, I think we have time to do better than that.

The error occurs when we are determining the CLR version of the framework. I wonder if we really need to know that for any framework beyond 4.x. The call is only made to Runtime.cs to populate the ClrVersion property. What code uses that? Why is it needed? Can we get rid of it. That would be a step toward simplifying the RuntimeFramework class, which I'd like to get rid of anyway!

Would you care to look into this?

[manfred-brands]

@madelson Downgrading NUnit3TestAdapter to version 4.1.0 allows you to run tests again, regardless of which SDK used for compilation.

[manfred-brands]

Would you care to look into this?

@CharliePoole Yes, I wouldn't mind. I now ran into this issues twice. Can you create a new ticket outlining what you expect (copy what you said above?) and assign it to me.

[madelson]

@madelson Downgrading NUnit3TestAdapter to version 4.1.0 allows you to run tests again, regardless of which SDK used for compilation.

I tried this but it did not work for me :-/

[CharliePoole]

@madelson This issue was reported against the console runner, so you may want to deal with your own problem separately, by bringing it to @OsirisTerje 's attention in the adapter repository.

I just updated my post about engine version conflicts when using the adapter and the console runner to bring it up to the latest adapter release. There's a table that shows what engine version you are using for an version of the adapter. That may possibly help.

[OsirisTerje]

@CharliePoole This is an error appearing in the adapter, but its roots are here in the engine, so I suggest we reopen this issue and solve it here.
I see there are a lot of changes around detecting the framework, and it seems the 3.12 engine is handling it correctly, from what I can read from the adapter issue.

Running the tests on the console/engine sln:

[CharliePoole]

Fixed in 3.16 with the installation of a .NET 7.0 agent.

[fedefrei]

I'm facing a similar error. Any probably date for 3.16 to be published to Nuget?

[CharliePoole]

@fedefrei Are you encountering this in version 3.15.2? The source indicates it's fixed there.

[fedefrei]

@CharliePoole maybe I'm in the wrong neighborhood.
I'm using the Visual Studio Test Explorer, under NUnit v3.13.3 and NUnit3TestAdapter v4.2.0. Is Nunit Console involved in this flow?

[CharliePoole]

@fedefrei The test adapter uses the NUnit engine, which is part of this project... in fact the biggest part. When you use the adapter, you get the particular engine version, against which your particular adapter version was built. For adapter 4.2.0, I believe that's version 3.13, which still has this problem.

Obviously, no engine release will help you if you aren't using it and you can't make the adapter use a different version from the one that is built in.

[CharliePoole]

This issue has been resolved in version 3.16.0

The release is available on:
GitHub.
NuGet packages are also available NuGet.org and
Chocolatey Packages may be found at Chocolatey.org

[Thorium]

We'd need a fix for NUnit3TestAdapter too...

[CharliePoole]

This is the console project. However, I happen to know that the adapter incorporated the fix in version 4.3.0.

[BlythMeister]

@CharliePoole i looked at #1175

WIth the way i read that code, when .net 8 is released in 12 months, this same problem will happen again and everyone will need to do the nuget upgrade dance?

Does this code have to throw the exception?

[mikkelbu]

@BlythMeister If we get version 4.0 released before that, then it will not be an issue - see #1177 or Charlie's and Manfred's comments above in this issue for more information.

[CharliePoole]

@BlythMeister Unfortunately, it's not good enough to not throw an exception. The code actually has to return some value. The best solution is to do away with the method entirely. I tried to do that in 3.15 but was not able to do so... it's used in one case I'm unable to eliminate. The 4.0 code makes breaking changes and does away with the entire RuntimeFramework class.

[Patrickkk]

We are facing this issue with .net 8 as well. As far as i can quickly tell is the same problem

[OsirisTerje]

@BlythMeister @Patrickkk Please use 3.15.4 , it works on .net 8 and all forward going frameworks.

We do have issue with the 3.16.X series, that breaks in different ways, and the 4.0 version have its own problems, it is not ready, and last but not least, we do lack resources (aka people) to work on this. But the 3.15.X series is being maintained. It is the same used with the adapter.

Also, if there are specific issues in the 3.16.X series that you miss in the 3.15.X series, we can probably port them over.

[Patrickkk]

@OsirisTerje Thanks for the quick reply, will give it a go.