MSTest V2 tests won't run when targeting net5.0-windows10.0.19041

[JJenkinsDaxsonics]

Description

Tests will not run when targeting net5.0-windows10.0.19041 with MSTest version greater than 2.2.3.

Steps to reproduce

• Create MSTest project targeting net5.0-windows10.0.19041.
• Attempt to run tests
• Change MSTest version to 2.2.3
• Clean solution and run tests again

Expected behavior

Tests should run in the test explorer

Actual behavior

All tests are skipped. VS Test log incorrectly reports .dll does not exist.

VS Test Log Output:

[MSTest][Discovery][WinUI3-Demo\WinUI3-UnitTests\bin\Debug\net5.0-windows10.0.19041\WinUI3-UnitTests.dll] Failed to discover tests from assembly WinUI3-Demo\WinUI3-UnitTests\bin\Debug\net5.0-windows10.0.19041\WinUI3-UnitTests.dll. Reason:File does not exist: WinUI3-Demo\WinUI3-UnitTests\bin\Debug\net5.0-windows10.0.19041\WinUI3-UnitTests.dll

MSTest log:

pTrace Information: 0 : 4580, 4, 2021/10/01, 16:05:24.725, 12341249999384, testhost.x86.dll, AssemblyResolver.OnResolve: Resolved assembly: Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices, from path: \WinUI3-Demo\WinUI3-UnitTests\bin\Debug\net5.0-windows10.0.19041\Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll
TpTrace Information: 0 : 4580, 4, 2021/10/01, 16:05:24.725, 12341250002103, testhost.x86.dll, MSTestExecutor.RunTests: Running tests from sources.
TpTrace Information: 0 : 4580, 4, 2021/10/01, 16:05:24.728, 12341250024421, testhost.x86.dll, CollectSourceInformation value Found : False 
TpTrace Error: 0 : 4580, 4, 2021/10/01, 16:05:24.773, 12341250480636, testhost.x86.dll, Exception of type 'System.InvalidOperationException' was thrown.

Environment

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.11.0" />
<PackageReference Include="MSTest.TestAdapter" Version="2.2.5" />
<PackageReference Include="MSTest.TestFramework" Version="2.2.5" />

AB#1415159

[dotMorten]

I'm hitting the same issue with net6.0-windows10.0.18362.0 (even without WinUI involved). I'm getting the same "File does not exist", with a path to my test assembly which indeed does exist.

[jlaanstra]

I'm seeing the same problem.

[dotMorten]

Duplicate of #982 ?

[mikeKuester]

I posted #851 half a year ago. Just tested it with v2.2.8 and .NET 6, but it still doesn't work. The root cause could be the same?

[jlaanstra]

Quoting @Scott1js's reply in microsoft/CsWinRT#1047 for reference:

There's no reason MSTest cannot execute unpackaged C#/WinRT projection code. This is a regression in MSTest, which was introduced with the WinUI support. That support is triggered via a specific test method attribute, UITestMethod. But independently, the test project's TFM also determines the test adapter to copy from the nuget. E.g., a value of net5.0-windows10.0.19041 copies the PlatformServices.WinUI adapter from "%NUGET_PACKAGES%\mstest.testadapter\2.2.5\build\net5.0-windows10.0.18362.0\Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll". The problem is that the WinUI test adapter assumes packaged execution in calls [like this]:

testfx/src/Adapter/PlatformServices.WinUI/Services/WinUIFileOperations.cs

Line 109 in 19acefa

public string GetFullFilePath(string assemblyFileName)

        public string GetFullFilePath(string assemblyFileName)
        {
            return (SafeInvoke<string>(() => Path.Combine(Windows.ApplicationModel.Package.Current.InstalledLocation.Path, assemblyFileName)) as string) ?? assemblyFileName;
        }

The test code here is not using WinUI, so it could just rely on the PlatformService.Desktop adapter code, but the TFM behavior above doesn't support that. You can back off to an earlier version of MSTest (I verified that 2.23 works) and you could also enter a bug in the MSTest repo to support unpackaged: https://github.com/microsoft/testfx

@Haplois can we get some traction on this?

[dotMorten]

Honestly I'd prefer the back out that WinUI Support, and make support for packaged apps a different separate test adapter you reference. The UITestMethod attribute that was added is pretty much useless anyway, as it doesn't support async task tests, but is easily worked around by using the existing test method and dispatching to the current window. (WinUIEx does this and uses a code generator to help you avoid having to write that extra code each time, although a simple helper will do the trick for you).
It's a much bigger issue that the newer test adapters don't work at all for Win32 apps.

[japj]

We are also running into the same issue where a basic net5 based unittest project is working fine when not pinning to a specific Windows Sdk version, but fails to execute at all when a specific Windows Sdk is used (e.g. 10.0.19041) .

Downgrading to 2.2.3 also fixed this specific issue.

[BetaOrbiter]

I'm seeing the same problem with .net6-windows10.0.19041.0

[nseveno]

The same problem with .net 6 also. I would like to test some C++/winrt components wrapped with a C#/winrt project... I hope this will be solved because I have no way to test those components !

[japj]

@Haplois is there anything we can do to help get this fixed in a next version?

[Haplois]

I am currently working on fixing this. Our next preview (probably on next friday) will include a fix. I'll update this thread with PR once it is ready. @japj thank you for the offer.