NUnit3TestAdapter integration: Exception when using NUnit.Engine 3.16.2

[OsirisTerje]

See the following branch in the adapter repo: https://github.com/nunit/nunit3-vs-adapter/tree/engine316-2

The adapter test project has to be set to run only netcore3.1 (but that is not this issue, just to be able to run it at all).
The test failing is: ThatDiscovererNUnitEngineAdapterIsInitialized

The exception happens here:

So it fails at NUnit.Engine.Internal.TestAsemblyResolver.FindBestVersionDir

The runner is MasterTestRunner

The filter contains the method being run, and that is that.

[CharliePoole]

You'll want to figure out what subdirectory name it is trying to process when it fails. It's likely your .NET 8.0 sdk directory.

[OsirisTerje]

No, it is not, I have not installed net 8. Going to debug it to find out :-)

[OsirisTerje]

The error starts here:

and fails here

Now, I can't see MonoTouch anywhere in the folders for the Sdks.

[CharliePoole]

Variable name is the AssemblyName of the assembly we are trying to resolve. It should be a test assembly or a dependency of a test assembly. It's apparently possible that some dependencies may be references without a version. I've never seen that before so I'd like to know how it happens. It may be an error we should report or perhaps we should accept any version.

[CharliePoole]

Also note that we wouldn't in FindBestVersionDir unless "monotouch" were found somewhere in scanning the AdditionalFrameworkDirectories, which are initialized in the static constructor of TestAssemblyResolver.

It may be useful to add some WriteLines in the loop that calls FindBestVersionDir.

[OsirisTerje]

The stacktrace is long, and I agree it should be written out, but also that the code should allow for null version. This might not be the only dll that have that. I believe this dll is rather old, and even if dotnet requires Version to be not null (ref dotnet/runtime#4467) we might stumble upon something like this. Also, there should be a fix in dotnet (ref dotnet/coreclr#1522) that set this to Version=0. There is no monotouch dll anywhere in the bin or lower, so it picks it up from somewhere else. It has something to do with Xamarin I think, so there might be some dependencies pointing there.

I think we should check this and just set Version to 0. It will make the engine more robust.

[OsirisTerje]

The line that crashes initializes a variable that is not in use. So I'll just remove that line.

[CharliePoole]

Ah it's outside of the loop!

The loop could still cause problems. Maybe the method shouldn't be called if name.Version is null.

[OsirisTerje]

I tested it locally and this one didn't crash at least, but I got another exception when creating the engine, the TestEngineActivator.CreateInstance() returned null. It may be related, or not.

[OsirisTerje]

@CharliePoole I added that nullcheck as you suggested, and that was the thing that was needed.

About the TestEngineActivator.CreateInstance() , if that returned null - which it seems to do in some cases, I simply do a new TestEngine() and that did the trick. I seem to remember you mentioned this some time back.