Replacing traits used to skip certain test cases with custom Theory and Fact attributes

[vlada-shubina]

Related to #8329

Context

Current approach skipping test cases using traits doesn't skip tests cases when running in VS.
In CI tests are ignored as never existed, without any messages.

Changes Made

Implemented platform specific Theory and Fact attributes
Applied them instead PlatformSpecific and SkipOnTargetFramework traits.
Created LongPathSupportDisabledFactAttribute to handle tests only allowed when long path support is disabled (though those tests never run, see #8349)
Removed attributes and traits related to mono.

Testing

Manually tested in VS to see that the tests are indeed skipped with proper error message.
Checked CI runs too.

Notes

[ladipro]

The SDK repo has something similar. I wonder if it wouldn't be worth aligning the names and signatures (or just copying those attributes).
Example: https://github.com/dotnet/sdk/blob/main/src/Tests/Microsoft.NET.TestFramework/Attributes/PlatformSpecificFact.cs

[vlada-shubina]

The SDK repo has something similar. I wonder if it wouldn't be worth aligning the names and signatures (or just copying those attributes). Example: https://github.com/dotnet/sdk/blob/main/src/Tests/Microsoft.NET.TestFramework/Attributes/PlatformSpecificFact.cs

SDK has most of this attributes too: https://github.com/dotnet/sdk/blob/main/src/Tests/Microsoft.NET.TestFramework/Attributes/WindowsOnlyFactAttribute.cs for example.

I just added the additional message as many tests gave the reason why they are skipped in comment which won't be visible in anywhere else.

The next step will be to try to contribute to arcade with same attributes.

If those are approved, they can be used from any repo using the arcade: sdk / installer / msbuild, and others.

[ladipro]

Ah, I'm blind.

That's a great plan. Although I'm still finding it odd that this functionality is not already in arcade. We (and the SDK) cannot be the first teams hitting this. Is there a way to make the tests honor the existing traits outside of build -test? Whether they show as skipped or are completely filtered is not a big deal, IMO.

[vlada-shubina]

Ah, I'm blind.

That's a great plan. Although I'm still finding it odd that this functionality is not already in arcade. We (and the SDK) cannot be the first teams hitting this. Is there a way to make the tests honor the existing traits outside of build -test? Whether they show as skipped or are completely filtered is not a big deal, IMO.

Traits is meant to be used for categorization; and not skipping the tests. Test Explorer UX is built around this: filtering and grouping by traits.

The "skipping" features based on traits in arcade were implemented long ago, note that they also have custom XUnit runner. The feature this PR is using is quite new: xunit/xunit#2073. I believe it's just legacy way of doing it as xunit/xunit#2073 was not available at the time and there was a need for this feature.
I can ask people around if they know anything about this part.

imo, the reason why it is implemented in each repo separately simply as it's very easy to do it; but i agree it's not a great practice.

[vlada-shubina]

The tests marked with LongPathSupportDisabledFactAttribute are never run on CI, and locally in most case too.
They need Windows OS where long path support is disabled, but it's not the case for CI.

Created issue to address that: #8349

[rainersigwald]

Please merge instead of squashing (if we wind up needing to back this out for some reason I don't want to have to lose the wonderful deletion of mono attributes :))

[JanKrivanek]

Thanks!