Move MSBuild projects out of the native build scripts #31701
Conversation
….Private.CoreLib into their own subsets in Subsets.props and drive their builds via Arcade instead of the coreclr scripts.
…ild.cmd and build.sh
…nstead of in the build.cmd/sh scripts. Clean up the build.sh usage documentation (lots of copy-paste errors).
…ldOS to enable building System.Private.CoreLib for the correct output given the script parameters.
…ile. Enable the test shell scripts to either resolve the dotnet CLI from the path or from relative to Core_Root if the __DotNet variable is undefined.
…into hoist-S.P.CL
…anaged tools after the runtime.
This renaming is a good idea that I strongly suggest. Instead of completely removing the |
Additional cleanup for crossgen-corelib step.
|
There's no single exit point for src/coreclr/build.sh, so I'm just going to put the warning at the beginning and at the "success" end. |
|
Official build run at: https://dev.azure.com/dnceng/internal/_build/results?buildId=543977&view=results |
…gets consumed correctly.
Clean up build-runtime parameters and passing the official build ID from runtime.proj.
|
Yep. We're passing it as an "unprocessed build arg", which gets passed along to the MSBuild invocation for GenerateNativeVersionFile. If you check the official build I linked to, the outputs of the CoreCLR native build have the correct version info. |
|
Only failure is due to #32764 Anyone else have any feedback? If not, I'm going to merge this at EOD. |
|
Hello @jkoritzinsky! Because this pull request has the p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (
|
ghost
merged commit 
ghost
locked as resolved and limited conversation to collaborators
Summary
Significantly simplify our native build scripts by moving all of the managed builds up to the Arcade build infrastructure and splitting them into subsets to help save developer time on their inner builds.
Additionally, since it isn't easily possible to make Arcade build a project as self-contained publish, this PR also changes the crossgen2 build to build framework-dependent, have the Helix SDK pull down a .NET CLI for the test runs, and update the test run scripts to resolve the local dotnet CLI or the one in the path based on environment variables.
Subsets
The coreclr build is split into these subsets:
Local build improvements
The managed eventing headers are generated during the System.Private.CoreLib build by discovering python and running the script during the build in MSBuild.
The ucrt copy and enforce-pgo steps run as part of the native build instead of the managed build, which makes significantly more sense logically.
Due to keeping all of the managed builds within a single MSBuild invocation instead of nested invocations, this change saves around a minute in local build times.
Workflow changes
CoreCLR developers will have to use the root
build.cmd/shscripts to build any of the managed components. Thesrc/coreclr/build-rutnime.cmd/shscripts only support building the native coreclr CMake project. Thesrc/coreclr/crossgen-corelib.cmd/shscripts support running crossgen on System.Private.CoreLib. The oldsrc/coreclr/build.cmd/shscripts are kept so as to give developers a transition period from the scripts to the new scripts.Workflow examples
I've included some workflow examples below.
Build CoreCLR, System.Private.CoreLib, crossgen System.Private.CoreLib, managed tools, CoreCLR packages in Checked configuration for x86:
D:\source\runtime>./build.cmd -subsetCategory coreclr -c checked -arch x86Build CoreCLR, System.Private.CoreLib, crossgen System.Private.CoreLib, skip managed tools, skip coreclr packages:
D:\source\runtime>./build.cmd -subsetcategory coreclr -subset runtime-corelib-nativecorelibBuild CoreCLR and managed tools only:
D:\source\runtime>./build.cmd -subsetcategory coreclr -subset runtime-toolsBuild CoreCLR, System.Private.CoreLib, crossgen System.Private.CoreLib, managed tools, CoreCLR packages in debug and libraries in release (already works today):
D:\source\runtime> ./build.cmd -subsetcategory coreclr-libraries -c Release -runtimeConfiguration debugFuture improvements
This PR will enable us to build System.Private.CoreLib in a job before running the coreclr or libraries builds. As a result, this moves us closer to being able to build coreclr and libraries in parallel in CI. Additionally, this does most of the work to Arcadify coreclr's build, so we can focus more on what our next steps and "final" build script design will be.