Report Requires diagnostics from dataflow analyzer #92724

sbomer · 2023-09-27T16:55:40Z

This moves the logic for generating diagnostics about RequiresUnreferencedCode,
RequiresAssemblyFiles, and RequiresDynamicCode attributes from the separate
Requires* analyzers into the DynamicallyAccessedMembersAnalyzer.

This includes logic for warning on access to attributed members, not the warnings
about mismatching attributes on base/override methods. The override validation
logic is still handled by the respective analyzers.

The DynamicallyAccessedMembersAnalyzer is now turned on by any of the settings
EnableTrimAnalyzer, EnableAotAnalyzer, or EnableSingleFileAnalyzer.

A new type RequiresAnalyzerContext is used to cache per-compilation state of
the analyzers, to avoid recomputing it for each trim analysis pattern that
might potentially produce a warning. The context only stores state for enabled
analyzers, based on the MSBuild property values.

There are a few minor differences in the warning behavior with this change:

Implicit calls to annotated base ctors warn because they are visible in the CFG
Unused local functions aren't discovered by the analysis, so don't warn
(matching linker/AOT behavior)

This moves the logic for generating diagnostics about RequiresUnreferencedCode, RequiresAssemblyFiles, and RequiresDynamicCode attributes from the separate Requires* analyzers into the DynamicallyAccessedMembersAnalyzer. This includes logic for warning on access to attributed members, not the warnings about mismatching attributes on base/override methods. The override validation logic is still handled by the respective analyzers. The DynamicallyAccessedMembersAnalyzer is now turned on by any of the settings EnableTrimAnalyzer, EnableAotAnalyzer, or EnableSingleFileAnalyzer. A new type RequiresAnalyzerContext is used to cache per-compilation state of the analyzers, to avoid recomputing it for each trim analysis pattern that might potentially produce a warning. The context only stores state for enabled analyzers, based on the MSBuild property values.

ghost · 2023-09-27T16:55:58Z

Tagging subscribers to 'linkable-framework': @eerhardt, @vitek-karas, @LakshanF, @sbomer, @joperezr, @marek-safar
See info in area-owners.md if you want to be subscribed.

Issue Details

This moves the logic for generating diagnostics about RequiresUnreferencedCode,
RequiresAssemblyFiles, and RequiresDynamicCode attributes from the separate
Requires* analyzers into the DynamicallyAccessedMembersAnalyzer.

This includes logic for warning on access to attributed members, not the warnings
about mismatching attributes on base/override methods. The override validation
logic is still handled by the respective analyzers.

The DynamicallyAccessedMembersAnalyzer is now turned on by any of the settings
EnableTrimAnalyzer, EnableAotAnalyzer, or EnableSingleFileAnalyzer.

A new type RequiresAnalyzerContext is used to cache per-compilation state of
the analyzers, to avoid recomputing it for each trim analysis pattern that
might potentially produce a warning. The context only stores state for enabled
analyzers, based on the MSBuild property values.

Author:	sbomer
Assignees:	-
Labels:	`linkable-framework`
Milestone:	-

sbomer · 2023-09-27T16:58:59Z

src/tools/illink/src/ILLink.RoslynAnalyzer/AnalyzerOptionsExtensions.cs

-			var tree = compilation.SyntaxTrees.FirstOrDefault ();
-			if (tree is null) {
-				return null;
-			}


This didn't seem necessary and required passing more context (Compilation) down into the warning logic, so I removed it.

sbomer · 2023-10-02T14:30:15Z

@vitek-karas @agocke PTAL

vitek-karas

Looks good - the only problem I see is the potentially problematic suppression of warnings due to RUC/RDC/RAF in presense of intrinsic handling of AOT specific behaviors. But I don't think we have such a case yet, so it doesn't block functionality... yet.

src/tools/illink/test/Mono.Linker.Tests.Cases/RequiresCapability/BasicRequires.cs

vitek-karas · 2023-10-11T15:34:24Z

src/tools/illink/src/ILLink.RoslynAnalyzer/TrimAnalysis/TrimAnalysisReflectionAccessPattern.cs

 		{
 			DiagnosticContext diagnosticContext = new (Operation.Syntax.GetLocation ());
-			foreach (var diagnostic in ReflectionAccessAnalyzer.GetDiagnosticsForReflectionAccessToDAMOnMethod (diagnosticContext, ReferencedMethod))
-				yield return diagnostic;
+			if (!OwningSymbol.IsInRequiresUnreferencedCodeAttributeScope (out _)) {


We could make it more similar to how this is done in the linker/AOT - there the DiagnosticContext is created with the information if some of the warnings should be disabled. Which is initialized by the IsInRequires...Scope methods.
The way it works here is probably a little bit more efficient, but it migth become a problem if we want to handle single-file and AOT warnings here as well. For example - eventually we would like to do some more clever processing around MakeGeneric ... solely AOT related stuff. That would need to happen inside the MethodCallPattern - and the current setup would not work for it (those warnings would be suppressed by putting a RUC on the scope - which is wrong)

I see what you mean. The analyzer approach for RequiresAttribute warnings below (foreach analyzer, requiresAnalyzer.CheckAndCreateRequiresDiagnostic) doesn't work so well with the way DiagnosticContext is defined in AOT since each analyzer only handles logic specific to that attribute.

I think a good way to unify them would be to change AOT's attribute handling to check ShouldSuppressAnalysisWarningsForRequires inside of the ReflectionMethodBodyScanner, in CheckAndReportRequires. Then we could adopt the same pattern in the analyzer.

I'll look into this as a follow-up.

src/tools/illink/src/ILLink.RoslynAnalyzer/TrimAnalysis/TrimAnalysisReflectionAccessPattern.cs

src/tools/illink/src/ILLink.RoslynAnalyzer/RequiresAnalyzerBase.cs

vitek-karas · 2023-10-12T11:08:26Z

src/tools/illink/src/ILLink.RoslynAnalyzer/RequiresUnreferencedCodeAnalyzer.cs

 			// Some RUC-annotated APIs are intrinsically handled by the trimmer
 			if (member is IMethodSymbol method && Intrinsics.GetIntrinsicIdForMethod (new MethodProxy (method)) != IntrinsicId.None) {
 				return true;


This is unfortunate - that said we have a similar hack in the trimmer. I guess improvement in the future - how to handle RUC and similar correctly with the intrinsics handling.

To make sure I understand the concern - the problem you're pointing out is that any AOT intrinsics in GetIntrinsicIdForMethod will now not warn for RUC which is wrong in general (the RUC diagnostics should only be relevant for trimming but not AOT intrinsics), right?

I'd agree with that. It's independent of this change, but something to keep an eye on as we add intrinsics.

That is more the point of the comment here: #92724 (comment)

This one is about the fact that we have a "External" way of suppressing RUC warnings on intrinsics. In here, we check if a method is intrinsic and then trust the other analyzer it will do its job instead. In linker we use this giant enum which describes which pattern the warning came from, and then based on that we suppress the "general" RUC processing and rely on the method body scanner instead.

NativeAOT is the only implementation where this weird approach is not used - there it's all part of the main "handle call" functionality (still not ideal where it is, but it's much better), and we don't need a way to determine if something will be handled by the intrinsics or other mechanics to decide if we should apply RUC logic or not.

This is definitely not something to fix in this PR though.

src/tools/illink/src/ILLink.RoslynAnalyzer/TrimAnalysis/TrimAnalysisFieldAccessPattern.cs

src/tools/illink/src/ILLink.RoslynAnalyzer/TrimAnalysis/TrimAnalysisReflectionAccessPattern.cs

- Clean up a few unused usings - Remove unused Instance field

And related cases involving implicit indexer references.

Extra warnings were being produced on some projects where the single-file analyzer was enabled, causing the DAM analyzer to light up. The dataflow warnings were missing a check of EnableTrimAnalzer to prevent these warnings. This also lifts the checking of MSBuild property values out into the compilation start action, so it only needs to be done per compilation. The context type has been renamed to reflect that it holds state relevant to all of the analyses handled by the dataflow logic now.

sbomer · 2023-10-21T00:40:15Z

Fixed a few more asserts that were hitting after merging in #93259, and added test coverage for the asserts and related cases that came to mind:

Ref property passed as out param
Delegate creation of an invocation operation

Also added a missing check for EnableTrimAnalyzer in the DAM analyzer.

And add missing checks in DAM analyzer

sbomer · 2023-10-23T20:16:51Z

Merging this to unblock follow-up work. @vitek-karas feel free to add more feedback if you have any after the last few fixes, and I'll address it in a new PR.

vitek-karas · 2023-10-26T13:24:15Z

Looks good - thanks!

sbomer added 3 commits September 26, 2023 21:05

Merge remote-tracking branch 'origin/main' into unifyAnalyzers

c4ce96b

Merge remote-tracking branch 'origin/main' into unifyAnalyzers

1207eb7

sbomer requested review from agocke and vitek-karas September 27, 2023 16:55

sbomer requested a review from marek-safar as a code owner September 27, 2023 16:55

ghost added the linkable-framework Issues associated with delivering a linker friendly framework label Sep 27, 2023

dotnet-issue-labeler bot added the area-Tools-ILLink .NET linker development as well as trimming analyzers label Sep 27, 2023

ghost assigned sbomer Sep 27, 2023

sbomer commented Sep 27, 2023

View reviewed changes

build-analysis bot mentioned this pull request Sep 27, 2023

[wasm] WBT - tests broken due to old skiasharp package reference #92743

Closed

vitek-karas approved these changes Oct 12, 2023

View reviewed changes

sbomer added 4 commits October 17, 2023 18:40

Add test for annotated value assigned to event

b84bf33

Use DiagnosticContext for consistency

9c286cf

Move field access handling into dataflow analyzer

3923a30

Remove unnecessary merge logic for reflection access pattern

ee37669

build-analysis bot mentioned this pull request Oct 20, 2023

[wasm] Runtime tests failing - _Vector2_3_4::Vector2_3_4Test.RunVector*Tests because of missing support for native libraries #93669

Closed

vitek-karas approved these changes Oct 20, 2023

View reviewed changes

src/tools/illink/src/ILLink.RoslynAnalyzer/TrimAnalysis/TrimAnalysisFieldAccessPattern.cs Outdated Show resolved Hide resolved

src/tools/illink/src/ILLink.RoslynAnalyzer/TrimAnalysis/TrimAnalysisReflectionAccessPattern.cs Show resolved Hide resolved

sbomer added 8 commits October 20, 2023 16:47

PR feedback

326ef23

- Clean up a few unused usings - Remove unused Instance field

Merge remote-tracking branch 'origin/main' into unifyAnalyzers

b85d8d7

Fix ExpectedWarnings for ILLink

f898cf1

Fix assert for ref property as out param

5e684d3

And related cases involving implicit indexer references.

Add test for deconstruction assignment to ref property

bc3c075

Fix delegate creation over invocation

a03a6fe

Fix ExpectedWarnings for illink

08c5d0c

build-analysis bot mentioned this pull request Oct 21, 2023

Intermittent build failure in AfterSourceBuild: "Could not write state file" #76488

Open

Add test coverage for EnableTrimAnalyzer checks

5ceaa31

And add missing checks in DAM analyzer

sbomer merged commit c345959 into dotnet:main Oct 23, 2023
62 checks passed

eerhardt mentioned this pull request Oct 25, 2023

Update SDK to 23523.1 dotnet/aspnetcore#51588

Merged

ghost locked as resolved and limited conversation to collaborators Nov 25, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Report Requires diagnostics from dataflow analyzer #92724

Report Requires diagnostics from dataflow analyzer #92724

sbomer commented Sep 27, 2023 •

edited

Loading

ghost commented Sep 27, 2023

sbomer Sep 27, 2023

sbomer commented Oct 2, 2023

vitek-karas left a comment

vitek-karas Oct 11, 2023

sbomer Oct 19, 2023

vitek-karas Oct 12, 2023

sbomer Oct 17, 2023

vitek-karas Oct 18, 2023

sbomer commented Oct 21, 2023

sbomer commented Oct 23, 2023

vitek-karas commented Oct 26, 2023

Report Requires diagnostics from dataflow analyzer #92724

Report Requires diagnostics from dataflow analyzer #92724

Conversation

sbomer commented Sep 27, 2023 • edited Loading

ghost commented Sep 27, 2023

sbomer Sep 27, 2023

Choose a reason for hiding this comment

sbomer commented Oct 2, 2023

vitek-karas left a comment

Choose a reason for hiding this comment

vitek-karas Oct 11, 2023

Choose a reason for hiding this comment

sbomer Oct 19, 2023

Choose a reason for hiding this comment

vitek-karas Oct 12, 2023

Choose a reason for hiding this comment

sbomer Oct 17, 2023

Choose a reason for hiding this comment

vitek-karas Oct 18, 2023

Choose a reason for hiding this comment

sbomer commented Oct 21, 2023

sbomer commented Oct 23, 2023

vitek-karas commented Oct 26, 2023

sbomer commented Sep 27, 2023 •

edited

Loading