Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make test libraries configuration agnostic #378

Merged
merged 7 commits into from
Dec 20, 2019

Conversation

safern
Copy link
Member

@safern safern commented Nov 28, 2019

In order to be able to build tests in one configuration and then use a shared framework that was built against another configuration we need to make our tests to test for specific behavior with an ||.

If this could introduce bugs where a Release code could be returning valid values for Debug but that's wrong, I'm happy to change this to do runtime checks instead, anyway we should not be testing framework configuration specific behavior based on the configuration the tests were built.

For Contracts tests, I did leave Configuration dependency, as that depends on what the test assembly was built against but not what shared framework we're running on. So Debug test assembly should work on Release and Debug shared framework. Also if the test assembly is built for Release mode it should work on any shared framework.

<DefineConstants>$(DefineConstants),DEBUG,TRACE</DefineConstants>
<!-- Test projects should be configuration agnostic. -->
<DefineConstants Condition="'$(IsTestProject)' != 'true'">$(DefineConstants),DEBUG</DefineConstants>
<DefineConstants>$(DefineConstants),TRACE</DefineConstants>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a good reason to keep TRACE defined?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't seem like it. From doing a grep, I saw that there are only a few usages of this constant, but the projects that use it, either define it on the files or on the csproj itself.

https://github.com/dotnet/runtime/blob/master/src/libraries/System.Diagnostics.TraceSource/src/System/Diagnostics/TraceSource.cs#L175

Should we just remove those conditionals and defines? It seems like it is always defined, therefore, those methods are always available?

@jkotas
Copy link
Member

jkotas commented Nov 28, 2019

Configuration dependency, as that depends on what the test assembly was built against

A better way to do this may be to #define the conditional symbol in specific files. Look for #define DEBUG in System.Diagnostics.Debug tests for prior art.

@jkotas
Copy link
Member

jkotas commented Nov 28, 2019

I would scope this change to:

  • Fixing all cases where debug build of the test fails on release build of the product, and vice versa. Are there any actually?
  • Improving the coverage of release builds of the tests such as removing DEBUG ifdefs around test cases in StackTraceTests.cs

But I would keep DEBUG defined for debug builds of the tests.

@ViktorHofer
Copy link
Member

But I would keep DEBUG defined for debug builds of the tests.

I gave this more thought as well and I agree with Jan that we shouldn't undefine DEBUG.

@safern
Copy link
Member Author

safern commented Dec 5, 2019

Are there any actually?

Yeah there where a couple in some Diagnostics tests.

But I would keep DEBUG defined for debug builds of the tests.

Ok. I will work on that.

@safern
Copy link
Member Author

safern commented Dec 18, 2019

@jkotas @ViktorHofer could you please take a look?

@safern
Copy link
Member Author

safern commented Dec 18, 2019

Fixed all comments. Thanks for the review.

@safern safern merged commit 0d596d2 into dotnet:master Dec 20, 2019
@safern safern deleted the TestsConfigurationAgnostic branch December 20, 2019 02:06
Copy link
Member

@ViktorHofer ViktorHofer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM.

@ghost ghost locked as resolved and limited conversation to collaborators Dec 11, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants