Removed Discover Warnings in Test Output | Issue #437

[parrainc]

This PR fixes the issue #437. After building the solution and the test discover starts, non-public tests won't report in the Test output.

[Update]
Created a "dummy test class attribute" that inherit from TestClassAttribute to be used in the dummy test classes that were generating warnings in the Output.

This is how is looks after these changes:

[jayaranigarg]

@parrainc : Thank you for your contribution but unfortunately we want to have test classes to work only with public access modifier and not with private/internal for performance reasons.

[parrainc]

Hi @jayaranigarg thanks for your comment. But, notice that the behaviour will be the same for discovering tests (just public test classes), the only thing I changed is that internal/private test classes won't report in the output, as stated the expected behaviour in the issue.

In any case, if you have suggestions that I can follow, so I can update the PR, please let me know.

[parrainc]

@jayaranigarg just following this one up... wanted to know if there's suggestions on what needs to be modified in this PR.

[jayaranigarg]

@parrainc : We indeed want internal/private classes to be reported in the output, otherwise a user will not be able to figure out the reason why his/her tests are not getting discovered.

We somehow just want to suppress these warnings for our unit-test classes, where we don't want testmethods to run as such as they are dummy classes which are merely used as assets in writing unit tests.

[parrainc]

Gotcha! I misunderstood the issue a little bit. @jayaranigarg
I'm updating this PR with new commit suppressing these warnings instead of just removing the reporting functionality on internal/private classes.

[jayaranigarg]

@parrainc : That's great! Let us know once you push in the new changes.

[parrainc]

@jayaranigarg new changes are ready for review (with some delay). Please take a look when available and let me know your thoughts :)

[jayaranigarg]

Please add [DummyTestClass] to class DummyTestClassWithCleanupMethods(Line 388) as well.

[parrainc]

Just did it, but when added the [DummyTestClass] attribute to DummyTestClassWithCleanupMethods class, one of the unit test failed RunCleanupShouldReturnCleanupResultsForAssemblyAndCla ssCleanupMethods because of the AssemblyCleanup method is not being called.

[jayaranigarg]

That is weird.

[parrainc]

Indeed it is. I did even try debugging the unit test that was using the DummyTestClassWithCleanupMethods with the [DummyTestClass] and [UTF.TestClass] but could not observe nothing. and got same behavior except for the part of the assemblyCleanup thing. Notice that when invoking the ClassCleanup it works properly.

[jayaranigarg]

Tried debugging this. Basically, when we get DeclaredMethods for DummyTestClassWithCleanupMethods, it is not returning AssemblyCleanup() method in the list of methods for that type and hence AssemblyCleanup() is not getting called. Now, the question is why AssemblyCleanup() is not present in the list when we mention [DummyTestClass] and is present in case of [UTF.TestClass]

[parrainc]

That's the same question I asked to myself...
I created a mstest project, added a class CustomTestClass that inherits from TestClass, and set up the assembly/class initalize and cleanup methods. It looks like there could be an issue when a test class is inheriting from a custom attribute that inherits from TestClassAttribute.
Have you seem any issues regarding to that?

Here's the debug output with CustomTestClass attribute:
expectedValue is a private static int field declared in the test class

ClassInitialize: before -> 0
ClassInitialize: after -> 321
TestMethod: expectedValue -> 321

Assert.AreEqual failed. Expected:<123>. Actual:<321>.

ClassCleanup: expectedValue -> 321

notice that with the custom attribute, neither the assembly initialize nor the cleanup were executed

Here's the debug output with TestClass attribute:

AssemblyInitialize: before -> 0
AssemblyInitialize: after -> 123
ClassInitialize: before -> 123
ClassInitialize: after -> 123
TestMethod: expectedValue -> 123
ClassCleanup: expectedValue -> 123
AssemblyCleanup: expectedValue -> 123

[parrainc]

@jayaranigarg any updates?
I'm thinking of opening an issue with my inputs in above comments? Would that be alright?

[jayaranigarg]

@parrainc : That should be fine. Please raise a separate issue for this and let's get this PR in.

[jayaranigarg]

@parrainc : LGTM 👍 Just a minor comment. Please address that and it should be good to push in.