Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: use mutex for file determination #1113

Merged
merged 8 commits into from
Aug 3, 2023

Conversation

Lorilatschki
Copy link
Contributor

Ensure the file determination process is aware of multiple test runs in different processes by using a mutex for the assembly path

…up a custom result file name rather than the name of the assembly to be tested
…test runs in different processes by using a mutex for the assembly path
@OsirisTerje
Copy link
Member

@Lorilatschki Two tests failing :-)

@Lorilatschki
Copy link
Contributor Author

Lorilatschki commented Aug 3, 2023

@OsirisTerje I have seen it but I don't know why. I tried to debug the test locally (it is only enabled for net462) but I cannot run it inside my VS2022. It always says the following:

Processing C:\Users\xxx\source\repos\github\nunit3-vs-adapter\src\NUnit3AdapterExternalTests\bin\Debug\net462\NUnit.VisualStudio.TestAdapter.ExternalTests.dll
Workdir set to: C:\Users\xxx\source\repos\github\nunit3-vs-adapter\src\NUnit3AdapterExternalTests\bin\Debug\net462
Execution engine run time with filter length 9 :Elapsed: 00:00:00.00
Filter: <filter/>
Discovery/Processing/Converting: :Elapsed: 00:00:00.00
Discovered 0 test cases
Processing C:\Users\xxx\source\repos\github\nunit3-vs-adapter\src\NUnitTestAdapterTests\bin\Debug\net462\NUnit.VisualStudio.TestAdapter.Tests.dll
Workdir set to: C:\Users\xxx\source\repos\github\nunit3-vs-adapter\src\NUnitTestAdapterTests\bin\Debug\net462
Execution engine run time with filter length 9 :Elapsed: 00:00:00.28
Filter: <filter/>
NUnit failed to load C:\Users\xxx\source\repos\github\nunit3-vs-adapter\src\NUnitTestAdapterTests\bin\Debug\net462\NUnit.VisualStudio.TestAdapter.Tests.dll
NUnit Adapter 4.5.0.0: Test discovery complete
No test is available in C:\Users\xxx\source\repos\github\nunit3-vs-adapter\src\empty-assembly\bin\Debug\net462\empty-assembly.dll C:\Users\xxx\source\repos\github\nunit3-vs-adapter\src\mock-assembly\bin\Debug\net462\mock-assembly.dll C:\Users\xxx\source\repos\github\nunit3-vs-adapter\src\NUnit3AdapterExternalTests\bin\Debug\net462\NUnit.VisualStudio.TestAdapter.ExternalTests.dll C:\Users\xxx\source\repos\github\nunit3-vs-adapter\src\NUnitTestAdapterTests\bin\Debug\net462\NUnit.VisualStudio.TestAdapter.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Could not find testhost
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.InitializeDiscovery(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler, Boolean skipDefaultAdapters)
Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Could not find testhost
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.InitializeDiscovery(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler, Boolean skipDefaultAdapters)
Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Could not find testhost
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.InitializeDiscovery(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler, Boolean skipDefaultAdapters)
System.InvalidOperationException: The provided manager was not found in any slot.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.ClearCompletedSlot(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.RunNextWork(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.HandlePartialDiscoveryComplete(IProxyDiscoveryManager proxyDiscoveryManager, Int64 totalTests, IEnumerable`1 lastChunk, Boolean isAborted)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelDiscoveryEventsHandler.HandleDiscoveryComplete(DiscoveryCompleteEventArgs discoveryCompleteEventArgs, IEnumerable`1 lastChunk)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.InitializeDiscovery(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler, Boolean skipDefaultAdapters)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.DiscoverTests(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler)
System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: The provided manager was not found in any slot.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.ClearCompletedSlot(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.RunNextWork(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.HandlePartialDiscoveryComplete(IProxyDiscoveryManager proxyDiscoveryManager, Int64 totalTests, IEnumerable`1 lastChunk, Boolean isAborted)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelDiscoveryEventsHandler.HandleDiscoveryComplete(DiscoveryCompleteEventArgs discoveryCompleteEventArgs, IEnumerable`1 lastChunk)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.DiscoverTests(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.<>c__DisplayClass27_0.<DiscoverTestsOnConcurrentManager>b__0()
   at System.Threading.Tasks.Task.Execute()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.InvalidOperationException: The provided manager was not found in any slot.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.ClearCompletedSlot(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ParallelOperationManager`3.RunNextWork(TManager completedManager)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.HandlePartialDiscoveryComplete(IProxyDiscoveryManager proxyDiscoveryManager, Int64 totalTests, IEnumerable`1 lastChunk, Boolean isAborted)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelDiscoveryEventsHandler.HandleDiscoveryComplete(DiscoveryCompleteEventArgs discoveryCompleteEventArgs, IEnumerable`1 lastChunk)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.DiscoverTests(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.Parallel.ParallelProxyDiscoveryManager.<>c__DisplayClass27_0.<DiscoverTestsOnConcurrentManager>b__0()
   at System.Threading.Tasks.Task.Execute()<---

Any idea what I am doing wrong?

Running dotnet test --framework net462 on a command line doesn't work either.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
No test is available in C:\Users\xxx\source\repos\github\nunit3-vs-adapter\src\NUnitTestAdapterTests\bin\Debug\net462\NUnit.VisualStudio.TestAdapter.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

Additionally, path to test adapters can be specified using /TestAdapterPath command. Example  /TestAdapterPath:<pathToCustomAdapters>.

@OsirisTerje
Copy link
Member

OsirisTerje commented Aug 3, 2023

I don't think you're doing anything wrong. I've got the same message from time to time lately. I wonder if this is some bug in the testhost or some related component. If you tro to run it from the command line, using dotnet test I believe it will work.

SORRY: Didnt see your last lines....

Try to run: build --target=Test

About VS: I clean the solution, sometimes even delete all bin and obj files and even the -vs folder, then it works. In some cases I change the targetframeworks to ONLY be one specific framework. Not sure what is wrong here, but I have seen it on other solutions as well.

@Lorilatschki
Copy link
Contributor Author

Try to run: build --target=Test

This worked! I found the issue, in that 2 tests the assemblyPath is null and creating a mutex with a name equals null isn't allowed. I have adapted the name determination.

@Lorilatschki
Copy link
Contributor Author

Lorilatschki commented Aug 3, 2023

@OsirisTerje locally the tests are passing with my last commit. Can you please double check it on your end?

@OsirisTerje
Copy link
Member

The build is still failing with the same errors, files not found.

@OsirisTerje
Copy link
Member

OsirisTerje commented Aug 3, 2023

I cloned your PR repo, and on net core there are no errors. And net462 doesnt run at all..... in VS that is.
Using the build --target=Test I get 2 errors with .net fw from the tests ThatTestOutputXmlHasBeenCreatedBelowAssemblyFolder and ThatTestOutputXmlHasBeenAtWorkDirLocation. These two tests only exist for net fw.
image

I tested the master branch, and all tests there works, so the tests are actually failing.

@OsirisTerje
Copy link
Member

Cool! That last commit/change did the trick !!

@OsirisTerje
Copy link
Member

@Lorilatschki Do you fix the conflict too, and then we merge

@OsirisTerje
Copy link
Member

OsirisTerje commented Aug 3, 2023

@Lorilatschki Some merge stuff, it seems:
Error: D:\a\nunit3-vs-adapter\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\NUnitEngineAdapter.cs(203,23): error CS0111: Type 'NUnitEngineAdapter' already defines a member called 'GetTestOutputFileName' with the same parameter types [D:\a\nunit3-vs-adapter\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit.TestAdapter.csproj::TargetFramework=net462] Error: D:\a\nunit3-vs-adapter\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\NUnitEngineAdapter.cs(203,23): error CS0111: Type 'NUnitEngineAdapter' already defines a member called 'GetTestOutputFileName' with the same parameter types [D:\a\nunit3-vs-adapter\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit.TestAdapter.csproj::TargetFramework=netcoreapp3.1]

@Lorilatschki
Copy link
Contributor Author

Lorilatschki commented Aug 3, 2023

@Lorilatschki Some merge stuff, it seems:
Error: D:\a\nunit3-vs-adapter\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\NUnitEngineAdapter.cs(203,23): error CS0111: Type 'NUnitEngineAdapter' already defines a member called 'GetTestOutputFileName' with the same parameter types [D:\a\nunit3-vs-adapter\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit.TestAdapter.csproj::TargetFramework=net462] Error: D:\a\nunit3-vs-adapter\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\NUnitEngineAdapter.cs(203,23): error CS0111: Type 'NUnitEngineAdapter' already defines a member called 'GetTestOutputFileName' with the same parameter types [D:\a\nunit3-vs-adapter\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit.TestAdapter.csproj::TargetFramework=netcoreapp3.1]

I will fix it.

@Lorilatschki
Copy link
Contributor Author

@OsirisTerje merge conflict is solved.

@OsirisTerje OsirisTerje merged commit f3b5c9c into nunit:master Aug 3, 2023
3 checks passed
@Lorilatschki Lorilatschki deleted the feat/mutex_for_filecreation branch August 3, 2023 20:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants