Diable System.Net.Http.FunctionalTests parallel execution on UAP run #35004

caesar-chen · 2019-01-31T21:02:04Z

Related: #35002

EDIT: I want to mention that, before the change, running System.Net.Http.FunctionalTests test suites with Outerloop tests locally takes around 17s. After the change, it's around 58s.

caesar-chen · 2019-01-31T21:03:16Z

@dotnet-bot help

dotnet-bot · 2019-01-31T21:04:33Z

Welcome to the dotnet/corefx Repository

The following is a list of valid commands on this PR. To invoke a command, comment the indicated phrase on the PR

The following commands are valid for all PRs and repositories.

Click to expand

Comment Phrase	Action
@dotnet-bot test this please	Re-run all legs. Use sparingly
@dotnet-bot test ci please	Generates (but does not run) jobs based on changes to the groovy job definitions in this branch
@dotnet-bot help	Print this help message

The following jobs are launched by default for each PR against dotnet/corefx:master.

Click to expand

Comment Phrase	Job Launched
@dotnet-bot test \QLinux arm Release Build\E	Linux arm Release Build

The following optional jobs are available in PRs against dotnet/corefx:master.

Click to expand

Comment Phrase	Job Launched
@dotnet-bot test innerloop CentOS7.1 Debug	Queues Innerloop CentOS7.1 Debug x64 Build and Test
@dotnet-bot test innerloop CentOS7.1 Release	Queues Innerloop CentOS7.1 Release x64 Build and Test
@dotnet-bot test code coverage	Queues Code Coverage Windows Debug
@dotnet-bot test innerloop Debian8.4 Debug	Queues Innerloop Debian8.4 Debug x64 Build and Test
@dotnet-bot test innerloop Debian8.4 Release	Queues Innerloop Debian8.4 Release x64 Build and Test
@dotnet-bot test innerloop Fedora24 Debug	Queues Innerloop Fedora24 Debug x64 Build and Test
@dotnet-bot test innerloop Fedora24 Release	Queues Innerloop Fedora24 Release x64 Build and Test
@dotnet-bot test Linux arm Debug	Queues Linux arm Debug Build
@dotnet-bot test code formatter check	Queues Code Formatter Check
@dotnet-bot test innerloop OSX10.12 Debug	Queues Innerloop OSX10.12 Debug x64 Build and Test
@dotnet-bot test innerloop OSX10.12 Release	Queues Innerloop OSX10.12 Release x64 Build and Test
@dotnet-bot test outerloop netcoreapp CentOS7.1 Debug	Queues OuterLoop netcoreapp CentOS7.1 Debug x64
@dotnet-bot test outerloop netcoreapp CentOS7.1 Release	Queues OuterLoop netcoreapp CentOS7.1 Release x64
@dotnet-bot test outerloop netcoreapp Debian8.4 Debug	Queues OuterLoop netcoreapp Debian8.4 Debug x64
@dotnet-bot test outerloop netcoreapp Debian8.4 Release	Queues OuterLoop netcoreapp Debian8.4 Release x64
@dotnet-bot test outerloop netcoreapp Fedora24 Debug	Queues OuterLoop netcoreapp Fedora24 Debug x64
@dotnet-bot test outerloop netcoreapp Fedora24 Release	Queues OuterLoop netcoreapp Fedora24 Release x64
@dotnet-bot test outerloop netcoreapp OSX10.12 Debug	Queues OuterLoop netcoreapp OSX10.12 Debug x64
@dotnet-bot test outerloop netcoreapp OSX10.12 Release	Queues OuterLoop netcoreapp OSX10.12 Release x64
@dotnet-bot test outerloop netcoreapp PortableLinux Debug	Queues OuterLoop netcoreapp PortableLinux Debug x64
@dotnet-bot test outerloop netcoreapp PortableLinux Release	Queues OuterLoop netcoreapp PortableLinux Release x64
@dotnet-bot test outerloop netcoreapp RHEL7.2 Debug	Queues OuterLoop netcoreapp RHEL7.2 Debug x64
@dotnet-bot test outerloop netcoreapp RHEL7.2 Release	Queues OuterLoop netcoreapp RHEL7.2 Release x64
@dotnet-bot test outerloop netcoreapp Ubuntu14.04 Debug	Queues OuterLoop netcoreapp Ubuntu14.04 Debug x64
@dotnet-bot test outerloop netcoreapp Ubuntu14.04 Release	Queues OuterLoop netcoreapp Ubuntu14.04 Release x64
@dotnet-bot test outerloop netcoreapp Ubuntu16.04 Debug	Queues OuterLoop netcoreapp Ubuntu16.04 Debug x64
@dotnet-bot test outerloop netcoreapp Ubuntu16.04 Release	Queues OuterLoop netcoreapp Ubuntu16.04 Release x64
@dotnet-bot test outerloop netcoreapp Ubuntu16.10 Debug	Queues OuterLoop netcoreapp Ubuntu16.10 Debug x64
@dotnet-bot test outerloop netcoreapp Ubuntu16.10 Release	Queues OuterLoop netcoreapp Ubuntu16.10 Release x64
@dotnet-bot test outerloop netcoreapp Windows 7 Debug	Queues OuterLoop netcoreapp Windows 7 Debug x64
@dotnet-bot test outerloop netcoreapp Windows 7 Release	Queues OuterLoop netcoreapp Windows 7 Release x64
@dotnet-bot test outerloop netcoreapp Windows_NT Debug	Queues OuterLoop netcoreapp Windows_NT Debug x64
@dotnet-bot test outerloop netcoreapp Windows_NT Release	Queues OuterLoop netcoreapp Windows_NT Release x64
@dotnet-bot test innerloop PortableLinux Debug	Queues Innerloop PortableLinux Debug x64 Build and Test
@dotnet-bot test innerloop PortableLinux Release	Queues Innerloop PortableLinux Release x64 Build and Test
@dotnet-bot test innerloop RHEL7.2 Debug	Queues Innerloop RHEL7.2 Debug x64 Build and Test
@dotnet-bot test innerloop RHEL7.2 Release	Queues Innerloop RHEL7.2 Release x64 Build and Test
@dotnet-bot test innerloop Ubuntu14.04 Debug	Queues Innerloop Ubuntu14.04 Debug x64 Build and Test
@dotnet-bot test innerloop Ubuntu14.04 Release	Queues Innerloop Ubuntu14.04 Release x64 Build and Test
@dotnet-bot test innerloop Ubuntu16.04 Debug	Queues Innerloop Ubuntu16.04 Debug x64 Build and Test
@dotnet-bot test innerloop Ubuntu16.04 Release	Queues Innerloop Ubuntu16.04 Release x64 Build and Test
@dotnet-bot test innerloop Ubuntu16.10 Debug	Queues Innerloop Ubuntu16.10 Debug x64 Build and Test
@dotnet-bot test innerloop Ubuntu16.10 Release	Queues Innerloop Ubuntu16.10 Release x64 Build and Test
@dotnet-bot test innerloop Windows_NT Debug	Queues Innerloop Windows_NT Debug x86 Build and Test
@dotnet-bot test innerloop Windows_NT Release	Queues Innerloop Windows_NT Release x64 Build and Test

Have a nice day!

davidsh · 2019-01-31T21:05:22Z

src/System.Net.Http/tests/FunctionalTests/XUnitAssemblyAttributes.cs

+
+using Xunit;
+
+// [ActiveIssue(35002)]: Disabling parallel execution of System.Net.Http.FunctionalTests


nit: you should modify the comment like this:

"Disable parallel execution of System.Net.Http.FunctionalTests on UAP test runs"

This way it is obvious if someone looks at this file they will know that this behavior is only for UAP test runs. Otherwise, they will have to look at the CSPROJ file.

davidsh · 2019-01-31T21:06:11Z

src/System.Net.Http/tests/FunctionalTests/XUnitAssemblyAttributes.cs

+using Xunit;
+
+// [ActiveIssue(35002)]: Disabling parallel execution of System.Net.Http.FunctionalTests
+[assembly: CollectionBehavior(CollectionBehavior.CollectionPerAssembly, DisableTestParallelization = true, MaxParallelThreads = 1)]


If "DisableTestParallelization = true" then does it matter whether or not you change "MaxParallelThread" at all, or even have it mentioned in this file?

I did some search online, having both is useful in some scenarios, for example: when some components in the test project are not compatible with the TPL, and xUnit uses TPL extensively, even when tests aren't being run in parallel.

I think it's fine to only keep DisableTestParallelization = true here.

ViktorHofer · 2019-01-31T21:27:09Z

src/System.Net.Http/tests/FunctionalTests/System.Net.Http.Functional.Tests.csproj

@@ -149,6 +149,7 @@
    <Compile Include="MultiInterfaceReadOnlyStream.cs" />
    <Compile Include="MultiInterfaceStreamContent.cs" />
    <Compile Include="PostScenarioUWPTest.cs" />
+    <Compile Include="XUnitAssemblyAttributes.cs" />


Don't you just want to import this for uap?

Yes. So I put it under <ItemGroup Condition="'$(TargetGroup)' == 'uap'"> line 146.

Ah didn't see that on my phone. Thx

wfurt

LGTM. I assume you got local repro and this change fixes it?

caesar-chen · 2019-01-31T21:48:07Z

I assume you got local repro and this change fixes it?

Yes.

caesar-chen · 2019-01-31T21:52:17Z

@safern Do we have a mechanism to run UWP outerloop test in CI? I cannot find it in the bot helper message.

davidsh · 2019-01-31T21:55:35Z

@dotnet-bot test Outerloop UWP CoreCLR x64 Debug Build

davidsh · 2019-01-31T21:56:55Z

In general, the Outerloop CI run is the same name as the Innerloop, but you add the word "Outerloop" to it.

caesar-chen · 2019-01-31T21:59:14Z

@dotnet-bot test Outerloop UWP CoreCLR x64 Debug Build

caesar-chen · 2019-01-31T23:55:36Z

CI is clean now. I will merge the PR.

…otnet/corefx#35004) * diable parallel execution * fix format.... * address feedback Commit migrated from dotnet/corefx@5543097

caesar-chen added 2 commits January 31, 2019 12:56

diable parallel execution

726f1a8

fix format....

45d83ae

caesar-chen added test enhancement Improvements of test source code area-System.Net.Http test-run-uwp-coreclr labels Jan 31, 2019

caesar-chen added this to the 3.0 milestone Jan 31, 2019

caesar-chen self-assigned this Jan 31, 2019

caesar-chen requested a review from a team January 31, 2019 21:02

davidsh reviewed Jan 31, 2019

View reviewed changes

davidsh requested review from stephentoub and ViktorHofer January 31, 2019 21:06

dotnet deleted a comment from dotnet-bot Jan 31, 2019

ViktorHofer reviewed Jan 31, 2019

View reviewed changes

wfurt approved these changes Jan 31, 2019

View reviewed changes

address feedback

8768af4

davidsh approved these changes Jan 31, 2019

View reviewed changes

caesar-chen merged commit 5543097 into dotnet:master Jan 31, 2019

caesar-chen deleted the uap_test branch January 31, 2019 23:55

ViktorHofer mentioned this pull request Feb 7, 2019

Update CoreFxOptimizationData, ProjectNTfs, ProjectNTfsTestILC to master-20190207.3, beta-27407-00, beta-27407-00, respectively (master) #35142

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Diable System.Net.Http.FunctionalTests parallel execution on UAP run #35004

Diable System.Net.Http.FunctionalTests parallel execution on UAP run #35004

caesar-chen commented Jan 31, 2019 •

edited

Loading

caesar-chen commented Jan 31, 2019

dotnet-bot commented Jan 31, 2019

davidsh Jan 31, 2019

caesar-chen Jan 31, 2019

davidsh Jan 31, 2019

caesar-chen Jan 31, 2019

ViktorHofer Jan 31, 2019

caesar-chen Jan 31, 2019

ViktorHofer Jan 31, 2019

wfurt left a comment

caesar-chen commented Jan 31, 2019

caesar-chen commented Jan 31, 2019

davidsh commented Jan 31, 2019

davidsh commented Jan 31, 2019

caesar-chen commented Jan 31, 2019

caesar-chen commented Jan 31, 2019


		using Xunit;

		// [ActiveIssue(35002)]: Disabling parallel execution of System.Net.Http.FunctionalTests

Diable System.Net.Http.FunctionalTests parallel execution on UAP run #35004

Diable System.Net.Http.FunctionalTests parallel execution on UAP run #35004

Conversation

caesar-chen commented Jan 31, 2019 • edited Loading

caesar-chen commented Jan 31, 2019

dotnet-bot commented Jan 31, 2019

davidsh Jan 31, 2019

Choose a reason for hiding this comment

caesar-chen Jan 31, 2019

Choose a reason for hiding this comment

davidsh Jan 31, 2019

Choose a reason for hiding this comment

caesar-chen Jan 31, 2019

Choose a reason for hiding this comment

ViktorHofer Jan 31, 2019

Choose a reason for hiding this comment

caesar-chen Jan 31, 2019

Choose a reason for hiding this comment

ViktorHofer Jan 31, 2019

Choose a reason for hiding this comment

wfurt left a comment

Choose a reason for hiding this comment

caesar-chen commented Jan 31, 2019

caesar-chen commented Jan 31, 2019

davidsh commented Jan 31, 2019

davidsh commented Jan 31, 2019

caesar-chen commented Jan 31, 2019

caesar-chen commented Jan 31, 2019

caesar-chen commented Jan 31, 2019 •

edited

Loading