Polyfill the incremental generator ForAttributeWithMetadataName from roslyn (for LibraryImportGenerator).

[CyrusNajmabadi]

Followup to #70911

Note: I'm going to add EventSourceGenerator (also in this project) in a followup to this PR.

[ghost]

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Followup to #70911

Note: I'm going to add EventSourceGenerator (also in this project) in a followup to this PR.

Author:	CyrusNajmabadi
Assignees:	-
Labels:	area-System.Runtime.InteropServices
Milestone:	-

[joperezr]

cc @jkoritzinsky and @AaronRobinsonMSFT PTAL. For context, @CyrusNajmabadi is helping us add a much more performant way for source generators that depend on Attributes to run, and with these PRs he is onboarding generators one-by-one. Since you are the owners of the LibraryImportGenerator, do you mind taking a look at the changes and make sure it makes sense?

[jkoritzinsky]

Looks like we'll need to update the version of Roslyn that we test the LibraryImportGenerator against, but other than that LGTM

[CyrusNajmabadi]

Looks like we'll need to update the version of Roslyn that we test the LibraryImportGenerator against,

@jkoritzinsky why would we need to do that? (from my perspective, the point of polyfilling was to help the runtime team not have to take a dependency on a new roslyn version). Thanks!

[jkoritzinsky]

The incremental tests are failing as they need the change in the generator infra to support the (Modified, Added) state transition for the step tracking. We shouldn't need to test the version that LibraryImportGenerator builds against, just what the tests run against.

[AaronRobinsonMSFT]

We shouldn't need to test the version that LibraryImportGenerator builds against, just what the tests run against.

If we have to do this anyways, do we want the polyfill at all?

[CyrusNajmabadi]

@jkoritzinsky any thoughts on #71652 (comment) ?

@joperezr for context, Roslyn4 has an actual bug in it which can't be fixed in the polyfill layer. It's unclear to me how we intend to get that fix out to your layer such that you can consume it.

[jkoritzinsky]

Given that we should be able to consume the new Roslyn soon (within the month?) it might be okay to wait. But if we're causing enough perf issues to show up in VS traces, then it might be worthwhile to take the polyfill.

[jkoritzinsky]

I just took a quick look at things and found out something useful:

Arcade's minimum SDK version is 7.0.0 Preview 5. That means that we can update our Roslyn version that we ship the generators with (the MicrosoftCodeAnalysisVersion_4_X property) to 4.3.0-2.22307.7, which contains the bug fix for the Roslyn bug this PR is hitting.

This still won't let us use the new API Cyrus is polyfilling here (as that is going to be in the next preview).

I think it's definitely worthwhile to update the Roslyn version we're building these tests against to the version above, and I think it's probably worth it to polyfill the API if it will give us big performance improvements.

[CyrusNajmabadi]

@jkoritzinsky can you help with the change you want to see made here? I'm not entirely sure where i should make it. Thanks!

[jkoritzinsky]

Made the change! I figured out that the version I shared is the same as the most recent published version (built from the same commit), so I used the "public" package version instead of the one I shared previously.

[stephentoub]

Other than the failing tests (which sound like they're a known issue with us waiting on a new version?), LGTM.

[CyrusNajmabadi]

@jkoritzinsky NativeTypeWithConstructorAndFromNativeValueMethod_ReportsDiagnostic (not related to generators afaict) failed on CI. Running locally it passes. Do you know what the issue might be here?

[jkoritzinsky]

I’m not sure. I’ll take a look tomorrow. If it’s blocking, you can disable the test. We’re going to be rewriting that analyzer very soon, so the existing tests aren’t particularly useful.

[CyrusNajmabadi]

I’m not sure. I’ll take a look tomorrow. If it’s blocking, you can disable the test. We’re going to be rewriting that analyzer very soon, so the existing tests aren’t particularly useful

I can do that if you're sure. I am a bit concerned that this is due to the roslyn update, and that somehow may be breaking you in some way. I'd def love if someone can actually check to see what's going on so there isn't some unintentional regression.

[CyrusNajmabadi]

@jkoritzinsky can you help here? i think this is hte last day we can get this in. @joperezr as well.

[jkoritzinsky]

I'm heads-down on getting other work in for LibraryImportGenerator before the code-complete deadline. I don't think I'll have bandwidth to get to this today.

[CyrusNajmabadi]

@jkoritzinsky @joperezr is there someone else we can pull in? Today is the last day to get this in. So if we miss this it won't happen till the next release.

[stephentoub]

Today is the last day to get this in.

This is an important perf fix. It's fine to go beyond tomorrow for .NET 7. (But sooner the better.) If we don't get it in tomorrow, it'll just miss P7.

[jkoritzinsky]

The change LGTM

The test failure is due to the change in Roslyn to prefer nint over IntPtr when adding a method that has IntPtr in the signature. This failure is fine as we were using the "Preview" langversion so we understand that these sorts of changes happen when we update Roslyn versions.

[joperezr]

/azp run runtime

Restarting the runtime CI since the failing tests have now been removed, so just ensuring that we have a green CI before merging.

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[joperezr]

The remaining JIT failure is unrelated, so I'll go ahead and merge this.

[joperezr]

/backport to release/7.0-preview7

[github-actions]

Started backporting to release/7.0-preview7: https://github.com/dotnet/runtime/actions/runs/2666797473

[github-actions]

@joperezr backporting to release/7.0-preview7 failed, the patch most likely resulted in conflicts:

$ git am --3way --ignore-whitespace --keep-non-patch changes.patch

Applying: Polyfill the incremental generator ForAttributeWithMetadataName from roslyn (for LibraryImportGenerator).
Applying: Move common code to shared location
Using index info to reconstruct a base tree...
M	src/libraries/System.Text.RegularExpressions/gen/System.Text.RegularExpressions.Generator.csproj
A	src/libraries/System.Text.RegularExpressions/src/System/Collections/Generic/ValueListBuilder.Pop.cs
Falling back to patching base and 3-way merge...
CONFLICT (rename/rename): Rename src/libraries/System.Text.RegularExpressions/src/System/Collections/Generic/ValueListBuilder.Pop.cs->src/libraries/System.Private.CoreLib/src/System/Collections/Generic/ValueListBuilder.Pop.cs in HEAD. Rename src/libraries/System.Runtime.InteropServices/src/System/Collections/Generic/ValueListBuilder.Pop.cs->src/libraries/System.Private.CoreLib/src/System/Collections/Generic/ValueListBuilder.Pop.cs in Move common code to shared location
Auto-merging src/libraries/System.Private.CoreLib/src/System/Collections/Generic/ValueListBuilder.Pop.cs
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0002 Move common code to shared location
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
Error: The process '/usr/bin/git' failed with exit code 128

Please backport manually!