Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Script changes enabling composite R2R build with shared framework #34661

Merged
merged 5 commits into from
Apr 15, 2020

Conversation

trylek
Copy link
Member

@trylek trylek commented Apr 7, 2020

This change basically does three things:

  1. Adds code to build-test.cmd/sh to Crossgen2 framework in composite mode;

  2. Adds code to CLRTest.Crossgen.targets to run Crossgen2 in Helix in composite mode when told to do so via the CompositeBuildMode environment variable;

  3. Implements YML fixes for composite pipeline support (sadly the jobs don't pass yet, apparently due to a codegen bug in CoreLib I'm currently investigating).

Thanks

Tomas

src/coreclr/build-test.cmd Outdated Show resolved Hide resolved
if !__FailedToPrecompile! NEQ 0 (
echo Failed assemblies:
@echo Failed assemblies:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious; why did you add the @ here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this was needed to make the text display with echo turned off but I must admit I'm not 100% sure.

Copy link
Contributor

@nattress nattress left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

set __Command=!__Command! "!CORE_ROOT!\CoreRun.exe"
set __Command=!__Command! "!CORE_ROOT!\crossgen2\crossgen2.dll"
set __Command=!__Command! @"!__ResponseFile!"
set __Command=!__Command! !ExtraCrossGen2Args!
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we echo this in to the response file - it might make reproducing issues easier if it's all in a single file we can paste into VS debug pane.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, sadly this is tricky for several reasons:

  1. CoreRun.exe naturally cannot be part of the response file, that's the app to run.

  2. I need to double-check whether the app to run (crossgen2.dll) can be specified in a response file, probably yes.

  3. I'm not sure about CoreRun parser but for instance the managed command-line parser we're using in Crossgen2, R2RDump and SuperIlc has a different syntax for arguments on the command line vs. in the response file - e.g., file arguments mustn't be quoted in response files, there's just one argument per line etc. - so that emitting !ExtraCrossGen2Args! into the response file is potentially risky.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I meant just the ExtraCrossgen2Args, so when we run crossgen2 in VS we have all the args in one place and don't have to glue extras on potentially.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, yes, I need to double-check whether I can just output the extra args to the response file - in case of the managed parser, you would need to replace spaces (outside of quoted sections) with line breaks in order to make the parser work properly.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, so it seems to me that corerun doesn't suppor response files at all and the managed parser in Crossgen2 has exactly the problem I described i.o.w. the extra command-line args have to be constructed based on knowledge whether they are going to be put directly into the command line or into a response file. I think it will be useful to switch even Crossgen1 to use response files and unify the construction of extra args everywhere but I'd prefer to do that as a separate follow-up change if you don't mind.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally fine with that

Exit /b 1
call :TakeLock
set CrossGen2Status=0
if not exist "IL" (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quality of life improvement here for re-running crossgen each time 👍

@trylek trylek force-pushed the CompositeFrameworkScriptChanges branch from aa5df1a to 04cdc19 Compare April 15, 2020 15:19
@trylek trylek merged commit b9bc190 into dotnet:master Apr 15, 2020
@trylek trylek deleted the CompositeFrameworkScriptChanges branch April 15, 2020 19:39
@ghost ghost locked as resolved and limited conversation to collaborators Dec 9, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants