Block System.Numerics.Tensors tests on native AOT

[MichalStrehovsky]

Failing in the native AOT outerloops. These use the xUnit generic test method that xUnit will MakeGenericMethod at runtime. Needs someone to write RD.XML:

Running assembly:System.Numerics.Tensors.Tests, Version=9.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
[FAIL] System.Numerics.Tensors.Tests.TensorTests.TensorExtensionsTwoSpanInSpanOut
System.NotSupportedException : 'System.Numerics.Tensors.Tests.TensorTests.TensorExtensionsTwoSpanInSpanOut[System.Single](System.Numerics.Tensors.Tests.TensorTests+PerformCalculationTwoSpanInSpanOut`1[System.Single],System.Func`3[System.Numerics.Tensors.Tensor`1[System.Single],System.Numerics.Tensors.Tensor`1[System.Single],System.Numerics.Tensors.Tensor`1[System.Single]])' is missing native code. MethodInfo.MakeGenericMethod() is not compatible with AOT compilation. Inspect and fix AOT related warnings that were generated when the app was published. For more information see https://aka.ms/nativeaot-compatibility
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.GetUncachedMethodInvoker(RuntimeTypeInfo[], MemberInfo) + 0x29
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.MakeGenericMethod(Type[]) + 0x1f8
[FAIL] System.Numerics.Tensors.Tests.TensorTests.TensorExtensionsSpanInSpanOut
System.NotSupportedException : 'System.Numerics.Tensors.Tests.TensorTests.TensorExtensionsSpanInSpanOut[System.Single](System.Numerics.Tensors.Tests.TensorTests+PerformCalculationSpanInSpanOut`1[System.Single],System.Func`2[System.Numerics.Tensors.Tensor`1[System.Single],System.Numerics.Tensors.Tensor`1[System.Single]])' is missing native code. MethodInfo.MakeGenericMethod() is not compatible with AOT compilation. Inspect and fix AOT related warnings that were generated when the app was published. For more information see https://aka.ms/nativeaot-compatibility
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.GetUncachedMethodInvoker(RuntimeTypeInfo[], MemberInfo) + 0x29
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.MakeGenericMethod(Type[]) + 0x1f8
[FAIL] System.Numerics.Tensors.Tests.TensorTests.TensorExtensionsTwoSpanInFloatOut
System.NotSupportedException : 'System.Numerics.Tensors.Tests.TensorTests.TensorExtensionsTwoSpanInFloatOut[System.Single](System.Numerics.Tensors.Tests.TensorTests+PerformCalculationTwoSpanInFloatOut`1[System.Single],System.Func`3[System.Numerics.Tensors.Tensor`1[System.Single],System.Numerics.Tensors.Tensor`1[System.Single],System.Single])' is missing native code. MethodInfo.MakeGenericMethod() is not compatible with AOT compilation. Inspect and fix AOT related warnings that were generated when the app was published. For more information see https://aka.ms/nativeaot-compatibility
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.GetUncachedMethodInvoker(RuntimeTypeInfo[], MemberInfo) + 0x29
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.MakeGenericMethod(Type[]) + 0x1f8
[FAIL] System.Numerics.Tensors.Tests.TensorTests.TensorExtensionsSpanInTOut
System.NotSupportedException : 'System.Numerics.Tensors.Tests.TensorTests.TensorExtensionsSpanInTOut[System.Single](System.Numerics.Tensors.Tests.TensorTests+PerformCalculationSpanInTOut`1[System.Single],System.Func`2[System.Numerics.Tensors.Tensor`1[System.Single],System.Single])' is missing native code. MethodInfo.MakeGenericMethod() is not compatible with AOT compilation. Inspect and fix AOT related warnings that were generated when the app was published. For more information see https://aka.ms/nativeaot-compatibility
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.GetUncachedMethodInvoker(RuntimeTypeInfo[], MemberInfo) + 0x29
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.MakeGenericMethod(Type[]) + 0x1f8
[FAIL] System.Numerics.Tensors.Tests.TensorSpanTests.TensorExtensionsTwoSpanInFloatOut
System.NotSupportedException : 'System.Numerics.Tensors.Tests.TensorSpanTests.TensorExtensionsTwoSpanInFloatOut[System.Single](System.Numerics.Tensors.Tests.TensorSpanTests+TensorPrimitivesTwoSpanInTOut`1[System.Single],System.Numerics.Tensors.Tests.TensorSpanTests+TensorTwoSpanInTOut`1[System.Single])' is missing native code. MethodInfo.MakeGenericMethod() is not compatible with AOT compilation. Inspect and fix AOT related warnings that were generated when the app was published. For more information see https://aka.ms/nativeaot-compatibility
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.GetUncachedMethodInvoker(RuntimeTypeInfo[], MemberInfo) + 0x29
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.MakeGenericMethod(Type[]) + 0x1f8
[FAIL] System.Numerics.Tensors.Tests.TensorSpanTests.TensorExtensionsSpanInTOut
System.NotSupportedException : 'System.Numerics.Tensors.Tests.TensorSpanTests.TensorExtensionsSpanInTOut[System.Single](System.Numerics.Tensors.Tests.TensorSpanTests+TensorPrimitivesSpanInTOut`1[System.Single],System.Numerics.Tensors.Tests.TensorSpanTests+TensorSpanInTOut`1[System.Single])' is missing native code. MethodInfo.MakeGenericMethod() is not compatible with AOT compilation. Inspect and fix AOT related warnings that were generated when the app was published. For more information see https://aka.ms/nativeaot-compatibility
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.GetUncachedMethodInvoker(RuntimeTypeInfo[], MemberInfo) + 0x29
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.MakeGenericMethod(Type[]) + 0x1f8
[FAIL] System.Numerics.Tensors.Tests.TensorSpanTests.TensorExtensionsSpanInSpanOut
System.NotSupportedException : 'System.Numerics.Tensors.Tests.TensorSpanTests.TensorExtensionsSpanInSpanOut[System.Single](System.Numerics.Tensors.Tests.TensorSpanTests+TensorPrimitivesSpanInSpanOut`1[System.Single],System.Numerics.Tensors.Tests.TensorSpanTests+TensorSpanInSpanOut`1[System.Single])' is missing native code. MethodInfo.MakeGenericMethod() is not compatible with AOT compilation. Inspect and fix AOT related warnings that were generated when the app was published. For more information see https://aka.ms/nativeaot-compatibility
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.GetUncachedMethodInvoker(RuntimeTypeInfo[], MemberInfo) + 0x29
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.MakeGenericMethod(Type[]) + 0x1f8
[FAIL] System.Numerics.Tensors.Tests.TensorSpanTests.TensorExtensionsTwoSpanInSpanOut
System.NotSupportedException : 'System.Numerics.Tensors.Tests.TensorSpanTests.TensorExtensionsTwoSpanInSpanOut[System.Single](System.Numerics.Tensors.Tests.TensorSpanTests+TensorPrimitivesTwoSpanInSpanOut`1[System.Single],System.Numerics.Tensors.Tests.TensorSpanTests+TensorTwoSpanInSpanOut`1[System.Single])' is missing native code. MethodInfo.MakeGenericMethod() is not compatible with AOT compilation. Inspect and fix AOT related warnings that were generated when the app was published. For more information see https://aka.ms/nativeaot-compatibility
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.GetUncachedMethodInvoker(RuntimeTypeInfo[], MemberInfo) + 0x29
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.MakeGenericMethod(Type[]) + 0x1f8
Finished System.Numerics.Tensors.Tests, Version=9.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51

Cc @dotnet/ilc-contrib

[MichalStrehovsky]

/azp run runtime-nativeaot-outerloop

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-numerics-tensors
See info in area-owners.md if you want to be subscribed.

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[MichalStrehovsky]

@dotnet/ilc-contrib @michaelgsharp could someone have a look? native AOT outerloop is red.

[tannergooding]

Is there a corresponding tracking issue for adding the RD.xml so this can be handled by NAOT?

Has this always been failing and just in the outerloop?

[jkotas]

This was introduced by #103005.

The problems are generic theories that are instantiated via reflection, like https://github.com/dotnet/runtime/pull/103005/files#diff-6d2c0778b263a7596b6b3661a484ebbb83675976c086cd2cab2747d23a85d5e0R230-R232 . The ideal fix would be switch to source generated xunit driver where all instantiations of the generic theories are statically discoverable at compile time.

Authoring RD.xml is very labor intensive and fragile. I would recommend against that. If you care about these tests running in native aot, I recommend avoiding dynamic generic theories instead.

We have stopped opening issues on disabled libraries tests that are not AOT compatible and that do not test interesting AOT-specific runtime behaviors (tests for OOB packages in particular). The issues were just getting won't fixed. We can certainly open issue for this one if you would like.

[tannergooding]

No, it sounds fine if we're not concerned about it and it's standard to what we're doing elsewhere

I agree that there isn't really anything that is interesting to NAOT in particular here and a more general fix to avoid the need for dynamic reflection of generics is better.

In this case we should be able to do something more like we're doing in TensorPrimitives where we have a generic base class and then concrete subclasses for the exact types we want to test instead. That allows us to share most of the logic and then just instantiate them per T like this:

    public class DoubleGenericTensorPrimitives : GenericFloatingPointNumberTensorPrimitivesTests<double> { }
    public class SingleGenericTensorPrimitives : GenericFloatingPointNumberTensorPrimitivesTests<float> { }
    public class HalfGenericTensorPrimitives : GenericFloatingPointNumberTensorPrimitivesTests<Half>
    {
        protected override void AssertEqualTolerance(Half expected, Half actual, Half? tolerance = null) =>
            base.AssertEqualTolerance(expected, actual, tolerance ?? Half.CreateTruncating(0.001));
    }

    [OuterLoop]
    public class NFloatGenericTensorPrimitives : GenericFloatingPointNumberTensorPrimitivesTests<NFloat> { }

    [OuterLoop]
    public class SByteGenericTensorPrimitives : GenericSignedIntegerTensorPrimitivesTests<sbyte> { }
    public class Int16GenericTensorPrimitives : GenericSignedIntegerTensorPrimitivesTests<short> { }
    [OuterLoop]
    public class Int32GenericTensorPrimitives : GenericSignedIntegerTensorPrimitivesTests<int> { }
    public class Int64GenericTensorPrimitives : GenericSignedIntegerTensorPrimitivesTests<long> { }
    [OuterLoop]
    public class IntPtrGenericTensorPrimitives : GenericSignedIntegerTensorPrimitivesTests<nint> { }
    public class Int128GenericTensorPrimitives : GenericSignedIntegerTensorPrimitivesTests<Int128> { }

    public class ByteGenericTensorPrimitives : GenericIntegerTensorPrimitivesTests<byte> { }
    [OuterLoop]
    public class UInt16GenericTensorPrimitives : GenericIntegerTensorPrimitivesTests<ushort> { }
    [OuterLoop]
    public class CharGenericTensorPrimitives : GenericIntegerTensorPrimitivesTests<char> { }
    public class UInt32GenericTensorPrimitives : GenericIntegerTensorPrimitivesTests<uint> { }
    [OuterLoop]
    public class UInt64GenericTensorPrimitives : GenericIntegerTensorPrimitivesTests<ulong> { }

    public class UIntPtrGenericTensorPrimitives : GenericIntegerTensorPrimitivesTests<nuint> { }
    [OuterLoop]
    public class UInt128GenericTensorPrimitives : GenericIntegerTensorPrimitivesTests<UInt128> { }

CC. @michaelgsharp as that is probably a feasible refactoring we could make

[agocke]

FYI, I've been working on source generating tests in my spare time -- it got derailed with other infra stuff.

It's unfortunately a very complex problem and involves rewriting large parts of the xunit execution framework.