-
Notifications
You must be signed in to change notification settings - Fork 253
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TestCaseFilter with custom properties #3015
Conversation
[ArgumentsProvider(nameof(TargetFrameworks.All), typeof(TargetFrameworks))] | ||
public async Task UsingTestPropertyForOwnerAndPriorityAndTestCategory_TestsFailed(string currentTfm) | ||
{ | ||
var testHost = TestHost.LocateFrom(_testAssetFixture.TargetAssetPath, AssetName, currentTfm); | ||
|
||
TestHostResult testHostResult = await testHost.ExecuteAsync("--filter tree!~one"); | ||
|
||
testHostResult.AssertOutputContains(""" | ||
failed PriorityTest 0ms | ||
UTA023: TestClass: Cannot define predefined property Priority on method PriorityTest. | ||
failed OwnerTest 0ms | ||
UTA023: TestClass: Cannot define predefined property Owner on method OwnerTest. | ||
failed TestCategoryTest 0ms | ||
UTA023: TestClass: Cannot define predefined property TestCategory on method TestCategoryTest. | ||
"""); | ||
} | ||
|
||
[ArgumentsProvider(nameof(TargetFrameworks.All), typeof(TargetFrameworks))] | ||
public async Task RunWithFilter_UsingTestPropertyForOwner_FilteredButTestsFailed(string currentTfm) | ||
{ | ||
var testHost = TestHost.LocateFrom(_testAssetFixture.TargetAssetPath, AssetName, currentTfm); | ||
|
||
TestHostResult testHostResult = await testHost.ExecuteAsync("--filter owner=testOwner"); | ||
|
||
testHostResult.AssertOutputContains(""" | ||
failed OwnerTest 0ms | ||
UTA023: TestClass: Cannot define predefined property Owner on method OwnerTest. | ||
"""); | ||
} | ||
|
||
[ArgumentsProvider(nameof(TargetFrameworks.All), typeof(TargetFrameworks))] | ||
public async Task RunWithFilter_UsingTestPropertyForPriorityAndTestCategory_NotFiltered(string currentTfm) | ||
{ | ||
var testHost = TestHost.LocateFrom(_testAssetFixture.TargetAssetPath, AssetName, currentTfm); | ||
|
||
TestHostResult testHostResult = await testHost.ExecuteAsync("--filter TestCategory=category|Priority=1"); | ||
|
||
testHostResult.AssertOutputContains("Zero tests ran"); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What are these tests proving? The errors seem unrelated to adding a new functionality that can filter on additional test categories.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yup it's not related to the fix, I just want to document that behaviour as it's related to TestProperty
if would change it in future
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@nohwnd I'll let you do the final approval here as you have been following more closely.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is useful functionality and it aligns well with traits on tests being used as properties of the test.
There is some slight perf impact when the property filter is enabled, but it is not the mainline scenario (discovery under VS), and we should be able to implement this in a more optimized way with ease if we drop object model from mstest.
Nice work @engyebrahim |
fix: #731
fix: #201