Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testmerging jit64 #83151

Merged
merged 28 commits into from
Apr 14, 2023
Merged

Testmerging jit64 #83151

merged 28 commits into from
Apr 14, 2023

Conversation

BrianBohe
Copy link
Member

This another pr merging JIT tests as #81969. It starts with a commit removing args from test entrypoint, and then applies ILTransform with the arguments specified on each commit in the same directory (baseservices/threading). This is the version of the tool I used.
Last two commits are a patch on the test merging approach to avoid deduplicating test classes on il tests. These are taken from #81969.

@BrianBohe
Copy link
Member Author

Before:

Time [secs] | Total | Passed | Failed | Skipped | Assembly Execution Summary
    ============================================================================
        404.212 |   884 |    884 |      0 |       0 | JIT.jit64.XUnitWrapper.dll
    ----------------------------------------------------------------------------
        404.212 |   884 |    884 |      0 |       0 | (total)

Now:

Time [secs] | Total | Passed | Failed | Skipped | Assembly Execution Summary
============================================================================
    247.204 |   140 |    140 |      0 |       0 | JIT.jit64.jit64_d
     39.049 |   205 |    204 |      1 |       0 | JIT.jit64.jit64_do
    261.956 |   184 |    184 |      0 |       0 | JIT.jit64.jit64_others
    255.325 |   291 |    291 |      0 |       0 | JIT.jit64.jit64_r
    241.254 |   104 |    104 |      0 |       0 | JIT.jit64.jit64_ro
----------------------------------------------------------------------------
   1044.789 |   924 |    923 |      1 |       0 | (total)

@ghost
Copy link

ghost commented Mar 8, 2023

Tagging subscribers to this area: @dotnet/area-system-reflection-metadata
See info in area-owners.md if you want to be subscribed.

Issue Details

This another pr merging JIT tests as #81969. It starts with a commit removing args from test entrypoint, and then applies ILTransform with the arguments specified on each commit in the same directory (baseservices/threading). This is the version of the tool I used.
Last two commits are a patch on the test merging approach to avoid deduplicating test classes on il tests. These are taken from #81969.

Author: BrianBohe
Assignees: BrianBohe
Labels:

area-System.Reflection.Metadata

Milestone: -

@teo-tsirpanis teo-tsirpanis added area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI and removed area-System.Reflection.Metadata labels Mar 8, 2023
@ghost
Copy link

ghost commented Mar 8, 2023

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch, @kunalspathak
See info in area-owners.md if you want to be subscribed.

Issue Details

This another pr merging JIT tests as #81969. It starts with a commit removing args from test entrypoint, and then applies ILTransform with the arguments specified on each commit in the same directory (baseservices/threading). This is the version of the tool I used.
Last two commits are a patch on the test merging approach to avoid deduplicating test classes on il tests. These are taken from #81969.

Author: BrianBohe
Assignees: BrianBohe
Labels:

area-CodeGen-coreclr

Milestone: -

@BrianBohe
Copy link
Member Author

It seems I missed a few cases in issue.targets

@JulieLeeMSFT
Copy link
Member

@markples please take care of this PR. Thanks @BrianBohe for working on this.

@markples
Copy link
Member

markples commented Apr 7, 2023

/azp run runtime-coreclr outerloop, runtime-extra-platforms

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@markples
Copy link
Member

markples commented Apr 7, 2023

/azp run runtime-coreclr outerloop, runtime-extra-platforms

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@markples
Copy link
Member

markples commented Apr 8, 2023

/azp run runtime-coreclr outerloop, runtime-extra-platforms

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

Otherwise, a merged test group's wrapper's exit code will override
any out-of-proc test in the group.

This should be safe because other places that set CLRTestExpectedExitCode are either
- Later in the file so will continue to override this
- In src\tests\baseservices\TieredCompilation\RunBasicTestWithMcj.cmd/sh
  but setting/using it locally

This will break setting it in the environment and overriding all occurrences, but
such usage wouldn't have the expected per-test expected exit values.
@markples
Copy link
Member

/azp run runtime-coreclr outerloop, runtime-extra-platforms

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@markples
Copy link
Member

@trylek Thanks for approving an earlier iteration of this. I've made a few additional changes that will hopefully be a quick re-review.

I also added an unrelated fix. I noticed that Regression's b16102 was testing an empty Main method, so merged test conversion lost its purpose. First undo the change and use RequiresProcessIsolation and ReferenceXUnitWrapperGenerator. Then unconditionally set CLRTestExpectedExitCode in .cmd/.sh wrappers so that the merged wrapper expected value doesn't override embedded tests. I couldn't find anything in dotnet/runtime that appears to depend on the conditional. If this requires discussion, then I can separate it out in order to merge jit64. Thanks!

@markples
Copy link
Member

I've added a review to this since Brian started the work and I reviewed those portions, but I authored the final changes so obviously my review doesn't work for those.

Copy link
Member

@trylek trylek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Mark for finishing Brian's change, your change still looks great to me.

@markples
Copy link
Member

fyi - there are a lot of existing failures, but I went through (multiple iterations for some of) the runtime, outerloop, and extra-platforms jobs and did not find anything new.

@markples markples merged commit 4f7375c into dotnet:main Apr 14, 2023
@ghost ghost locked as resolved and limited conversation to collaborators May 14, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI test-enhancement Improvements of test source code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants