Consume renamed TargetFramework package

[ViktorHofer]

Depends on dotnet/arcade#8421

[ghost]

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

---

null

Author:	ViktorHofer
Assignees:	ViktorHofer
Labels:	area-Infrastructure-libraries
Milestone:	-

[ViktorHofer]

@safern @akoeplinger do you know if these failures are caused by my changes?

• Helix API does not contain an entry for OSX.1014.Amd64.Open
• Helix API does not contain an entry for Windows.10.Amd64.Client19H1.Ope

cc @dotnet/dnceng

[jonfortescue]

@ViktorHofer Not caused by your changes. Those queues were both deleted by the latest rollout due to the operating systems reaching end of life.

[garath]

Those queues were both deleted by the latest rollout due to the operating systems reaching end of life.

I believe the change for the OSX queue is happening in #64565

[safern]

The windows one is: #64699

[ViktorHofer]

Thanks all. I'll give this another try tomorrow. Meanwhile @safern or @ericstj would you mind reviewing and approving?

[akoeplinger]

@ViktorHofer looks like this runs into the same issue as the darc update PR, do you know why the System.Private.CoreLib.Generators.dll ends up in CORE_ROOT: #64376 (comment) ?

[ViktorHofer]

@akoeplinger the NativeAot leg on this PR isn't failing like in #64376 (comment), or am I overlooking something?

[akoeplinger]

@ViktorHofer yeah that's because #64715 added some handling so the nativeaot doesn't crash, but the underlying issue is still there (and affects the Mono llvm legs)

[ViktorHofer]

and affects the Mono llvm legs

So the Mono llvm legs should fail on my PR? If not, how can I check if the issue is happening on my PR?

EDIT: Ok I think I know what you mean but I have no idea how that could be caused by my changed. Looking...

[ViktorHofer]

Based on the time this started failing it could be dotnet/arcade@93656ab.

[ViktorHofer]

OK I have a lead.

[ViktorHofer]

@akoeplinger the fix is in the second commit. The actual issue existed since the CoreLib source generator was added to src/libraries. In 9580b7d @ericstj disabled binplacing source generators into the runtime path via a convention but CoreLib's source generator didn't comply to that convention as its parent directory was named "generators" instead of "gen".

This popped up with my change now because binplacing didn't work for non multi-targeting projects. Only projects with a <TargetFrameworks /> property were chosen as a binplace candidate and projects with a <TargetFramework /> property were ignored. I fixed that in dotnet/arcade@93656ab and hence the CoreLib generator started to fail (it's the only source generator that doesn't multi-target).

Would appreciate an approval of the second commit (it's just a simple rename). The first commit is already approved.

[MichalStrehovsky]

@akoeplinger the fix is in the second commit. The actual issue existed since the CoreLib source generator was added to src/libraries. In 9580b7d @ericstj disabled binplacing source generators into the runtime path via a convention but CoreLib's source generator didn't comply to that convention as its parent directory was named "generators" instead of "gen".

Ah, so indeed similar to #62836. I was puzzled what it would be a problem just now. I can sign off on the second commit.

[MichalStrehovsky]

Second commit LGTM

[ViktorHofer]

In total more than 3 hours for PR validation, that's crazy :O

[marek-safar]

In total more than 3 hours for PR validation, that's crazy :O

/cc @jkotas

[ericstj]

@akoeplinger Alexander Köplinger FTE the fix is in the second commit. The actual issue existed since the CoreLib source generator was added to src/libraries. In 9580b7d @ericstj Eric St. John FTE disabled binplacing source generators into the runtime path via a convention but CoreLib's source generator didn't comply to that convention as its parent directory was named "generators" instead of "gen".

Perhaps we should have stronger rules around project classification. IOW: make sure projects fall into a known convention, specify what they are, otherwise it's an error.

[ericstj]

In total more than 3 hours for PR validation, that's crazy :O

Seems this is the long-pole.

CoreCLR Pri0 Runtime Tests Run windows arm64 checked
Duration: 2h 41m 22s

https://dev.azure.com/dnceng/public/_build/results?buildId=1589294&view=logs&j=bb94f37e-a09b-5b51-09b2-ab78e7aa5103

[ViktorHofer]

CoreCLR Pri0 Runtime Tests Run windows arm64 checked
Duration: 2h 41m 22s

That leg doesn't start immediately as it depends on both the coreclr and libraries product build. Add 20 minutes to it and you are over the 3 hour mark.

Another long pole is the llvmfullaot leg which in total also takes more than 3 hours.

[MattGal]

CoreCLR Pri0 Runtime Tests Run windows arm64 checked
Duration: 2h 41m 22s

That leg doesn't start immediately as it depends on both the coreclr and libraries product build. Add 20 minutes to it and you are over the 3 hour mark.

Another long pole is the llvmfullaot leg which in total also takes more than 3 hours.

Just to pile on, since this thread keeps popping up and I had to hit unsubscribe, is that this is 4 days, 19 hours of machine time in Helix tests for the last build alone:

let JobIds = 
Jobs 
| where Source == "pr/public/dotnet/runtime/refs/pull/64670/merge" 
| extend props = parse_json(Properties)
| where props["BuildNumber"]=="20220203.14"
| project JobId;
WorkItems
| where JobId  in (JobIds)
| where Name != JobName
| extend runtime = Finished-Started
| summarize sum(runtime)

(gives 4.19:50:06.7330000)

[jkotas]

CoreCLR Pri0 Runtime Tests Run windows arm64 checked
Duration: 2h 41m 22s

It looks like we have hit low Win Arm64 Helix availability. Can we tell what caused it?

In the past week, this job executed in 84 minutes on average. There are jobs running for longer than this job or consuming more machine hours that this job. For example, the Mono llvmfullaot Pri0 Runtime Tests Run Linux arm64 release tool 142 minutes on average last week.

this is 4 days, 19 hours of machine time in Helix tests for the last build alone:

Yes, we know that each CI run in dotnet/runtime consumes days worth of build machine time and days worth of test machine time.

The lowest hanging fruit to improve CI machine costs is to eliminate idle build machines waiting for Helix jobs to finish. Tracked by dotnet/dnceng#1213