TestCaseFilter with custom properties?

[pkrukp]

Hello,

I'm trying to use TestCaseFilter with custom properties. Is that possible?

I was trying to pass various forms of /TestCaseFilter to vstest.console.exe but without success. Works with TestCategory but not with custom properties.

For example in MSTest:

        [TestProperty("Bug", "12345")]
        [TestMethod]
        public void TestMethod2()

For XUnit it's Trait and for NUnit it's Property.

In https://github.com/Microsoft/vstest-docs/blob/master/docs/filter.md#syntax there's a list of "Supported properties":

• for MSTest there's no TestProperty there :(
• for XUnit there's Traits listed but I could not make it work (filter didn't match any test)
• no NUnit listed

I'm attaching a simple solution with 3 UT projects (mstest, xunit, nunit): ClassLibrary1.zip

I have used vstest.console.exe 16.7.0 from Microsoft.TestPlatform nuget package.
(In fact I'd like to pass the TestCaseFilter through RunSettings when vstest is used programmatically through IVsTestConsoleWrapper API, but I have started by checking how it works in the vstest.console.exe - I assume it works the same in API, from microsoft/vstest#2273).

AB#2050968

[nohwnd]

Looking at the code in MSTest it does not seem to be supported at the moment.

For xUnit it works like this: dotnet test --filter Bug=12345 or vstest.console XUnitTestProject1.dll --testcasefilter:Bug=12345.

[pkrukp]

Looking at the code in MSTest it does not seem to be supported at the moment.

So it must be implemented in each test adapter separately? I was thinking about making this issue a feature request for vstest, but I should probably file it against mstest (and maybe nunit)?

For xUnit it works like this: dotnet test --filter Bug=12345 or vstest.console XUnitTestProject1.dll --testcasefilter:Bug=12345.

Thanks, that way it works.

[nohwnd]

Yes each test adapter needs to implement it separately because they have different representations of tests. The way this was implemented, there is no common abstraction over this filtering concept. If you want it in MSTest then microsoft/testfx is the repo to search for this issue, or creating a new one. And similarly for NUnit.

[nohwnd]

@pkruk2 moved your issue to testfx (MSTest), to make it easier for you. No need to file another issue.

Related #201

[pkrukp]

Thank you. I'll try to file FR for other test adapters soon

[testplatform-bot]

✅ Successfully linked to Azure Boards work item(s):

• User Story 2050968