Very sluggish VS 2019 and VS 2022 performance starting with DotNet 6 Preview 5 related to Source Generators #56702
Comments
dotnet-issue-labeler
bot
added
the
untriaged
|
I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label. |
|
Moving to dotnet/runtime as it appears to be an issue with the Json source generator. @chsienki for monitoring as this may be a case where we can leverage the V2 APIs for IDE perf |
|
Tagging subscribers to this area: @eiriktsarpalis, @layomia Issue DetailsVersion Used: Steps to Reproduce: It started with DotNet6 Preview 5, which made VS 2019 run out of memory pretty quickly (GC was happening so often, that basically after about 1 hour of typing in a few files, it became unusable due to constant GC pauses, and VS's Private Bytes being near 2.5GB). VS 2022 in this regard was very timely, but it did not improve things much. The performance is better compared with 2019, because GCs happen much more rarely and not as aggressively, but there are still very frequent hangs and slowdowns that happen more and more often as time goes on. This is actually happening while doing very mundane things - for example just editing a few 500-line files in an hour, perhaps renaming a few variables, etc. Nothing super-heavy like editing 100s of files, or refactoring whole namespaces, etc. So I investigated this and came to conclude that Evidence: Stack Traces First, in both VS 2019 and 2022, when I was examining stack traces of VS's threads during slowdowns, I saw references to Evidence: Memory and Managed Heap Snapshots When I attached another instance of VS to "debug" the problematic instance, and took managed heap snapshots, I noticed that the lion's share of allocated bytes was once again for types in Issue with Source Generators Now, according to every stack trace that I took of VS's threads when it was misbehaving, it always involved Also note that my solution does NOT use any source generators. And VS's option I presume it might have something to do with Source Generator support being automatically included for Expected Behavior: Actual Behavior: It does not seem to correlate with what file I edit (large or small), what text I edit (comment, type name, variable name, code), and which project the file comes from (one on which other projects in the solution depend, or the one on which no other projects depend).
|
|
@fitdev thanks for the feedback. We should minimize the amount of work done when the generator is present but there are no indicated serializable types, and also to make the execution of the generator as efficient as possible when there are types to generate code for. As a starting point - we are currently doing a lot of expensive type resolution every time the generator runs. We should defer this work until we are sure that there are types indicated for source generation, and also cache these type references for reuse whenever possible. One thought here is to observe the compilation instance, and if it's the same as the last one, we reuse some of the relevant metadata that we've computed previously. I'm currently prototyping this approach and will make a PR soon. As already mentioned we should also consider whether the v2 source generator APIs can help with some of these IDE perf considerations.
This suggests that we might have a scenario in VS where the source generators are invoked even when users explicitly opt out of using them. @chsienki, @KathleenDollard do we have settings specifying this behavior? I wasn't able to find any using Visual Studio Enterprise 2019 16.10.0 Preview 2.1. Can you provide a sample project to repro the issues you are facing here (sanitized if necessary)? This can help us further understand the issue here and help validate the fixes. Please include any further information about any configuration for your solution(s) & projects. |
|
@layomia It's extremely difficult to implement source generator caching in a way that is compatible with the semantics of the v1 (non-incremental) API. If the source generator contains a syntax receiver, this task should be treated as impossible; the source generator will not scale to large projects without migration to the v2 (incremental) API. There are two potential resolutions here:
|
Thanks @sharwell. I'll start planning for this. I consider this a .NET 6 effort since we want the source generator to be usable in IDE scenarios. cc @ericstj Update: before committing to reimplementing the generator with v2 APIs at this stage in .NET 6, I think it's better for me to first quantify how bad the problem is and what type of improvements we could see by just refactoring as I mentioned earlier. |
|
@maryamariyan as it looks like the logging generator has a similar pattern. |
@layomia Thank you for looking into this. Unfortunately I cannot provide even a sanitized solution for a repro, as it would involve several 1000 files. Basically I see this behavior with medium-large solutions. As I said, most of my projects are C#, some VB, using latest NET6 Preview 6 and latest VS 2022. I would eventually like to utilize JSON source generators of course, so it is a great effort, but not until such serious performance issues are addressed. And in any case there hsould be away to completely turn off the feature - either at the IDE level and/or at the project level via something like |
|
You can add a target to Directory.Build.Targets in your repo to workaround this: <Target Name="_RemoveTextJsonAnalyzer" BeforeTargets="CoreCompile">
<ItemGroup>
<Analyzer Remove="@(Analyzer)" Condition="'%(FileName)' == 'System.Text.Json.SourceGeneration'" />
</ItemGroup>
</Target> |
|
CC @RussKie as well for winforms generators with SyntaxReceivers |
|
We discussed a plan offline to mitigate the perf regression here. Here's a summary of the current thinking:
Concrete next steps for .NET 6:
|
This is just as applicable for VS 22, because there may be by now various third-party libraries shipping their own V1 source generators that would slow VS down. So there should be a way to disable source generators at a project / IDE level by either API version, assembly, or both. |
RussKie
added a commit
to dotnet/winforms
that referenced
this issue
Aug 10, 2021
RussKie
added a commit
to dotnet/winforms
that referenced
this issue
Aug 10, 2021
The multi-targeting strategy is only necessary for the NuGet package, correct? For the source generators contained in the ref-pack, we should just always use v2, as this is |
For NuGet packages I believe dotnet/roslyn#54108 is intended to provide a canonical way to do this. |
@PathogenDavid This approach will only work with environments that understand the new packaging conventions, which means it probably won't work with Roslyn 3.x / VS 2019. |
eerhardt
added a commit
to eerhardt/runtime
that referenced
this issue
Aug 24, 2021
This reduces the time spent in the background in VS running the source generator, since we only need to respond to methods that have the LoggerMessageAttribute on them. Contributes to dotnet#56702
|
Just wanted to say that the issue exists in latest VS 2022 Preview 3.1 and DotNet 6 Preview 7. VS is frequently blocked even when typing a mere xml doc comment - something completely unrelated to source generators, let alone Json Source Generator! Here's the sample stack trace during the "mini-hang" of one of VS's threads: |
Source generators using the V1 API (as JsonSourceGenerators does) do not distinguish one location in a source file from any other. All locations, including within comments, are equivalent and treated as relevant for them. |
|
Oh I see. Thank you for pointing this out. Well, I really hope for RTM you move all source generators to V2 and provide clear mechanism to disable all source generators (that may be coming from 3rd party nugets). Because in a solution with ~60 projects and a few thousand files this leads to very serious slowdowns, which is made even worse by the fact that this CPU time is basically wasted since the actual coding does not have anything to do with source generators, and thus ideally none of them should run in the first place. |
eerhardt
added a commit
that referenced
this issue
Aug 27, 2021
* Migrate LoggerMessageGenerator to IIncrementalGenerator This reduces the time spent in the background in VS running the source generator, since we only need to respond to methods that have the LoggerMessageAttribute on them. Contributes to #56702
github-actions bot
pushed a commit
that referenced
this issue
Aug 27, 2021
This reduces the time spent in the background in VS running the source generator, since we only need to respond to methods that have the LoggerMessageAttribute on them. Contributes to #56702
This was referenced Aug 27, 2021
Anipik
pushed a commit
that referenced
this issue
Aug 28, 2021
…ator (#58271) * Migrate LoggerMessageGenerator to IIncrementalGenerator This reduces the time spent in the background in VS running the source generator, since we only need to respond to methods that have the LoggerMessageAttribute on them. Contributes to #56702 * PR feedback * PR feedback Co-authored-by: Eric Erhardt <eric.erhardt@microsoft.com>
This was referenced Aug 30, 2021
|
Facing the same issue after moving to .net 6. I do not use any source generators. CPU usage routinely hits 100% after a few keystrokes. |
ghost
locked as resolved and limited conversation to collaborators
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Version Used:
Latest Official DotNet 6 Preview 6
Latest VS previews: VS 2019 16.11 Preview 3 and VS 2022 Preview 2
C#: 10 (
<LangVersion>preview</LangVersion>, includingglobal usings in a few places)Steps to Reproduce:
VS typing performance quickly degrades (hangs, up to several seconds pauses between characters typing, etc.) after working with several
csfiles for like 20 minutes or more in the context of a medium-sized DotNet 6 Preview 6 solution with maybe 70 projects, each with ~ 10 to 500 files.It started with DotNet6 Preview 5, which made VS 2019 run out of memory pretty quickly (GC was happening so often, that basically after about 1 hour of typing in a few files, it became unusable due to constant GC pauses, and VS's Private Bytes being near 2.5GB).
VS 2022 in this regard was very timely, but it did not improve things much. The performance is better compared with 2019, because GCs happen much more rarely and not as aggressively, but there are still very frequent hangs and slowdowns that happen more and more often as time goes on. This is actually happening while doing very mundane things - for example just editing a few 500-line files in an hour, perhaps renaming a few variables, etc. Nothing super-heavy like editing 100s of files, or refactoring whole namespaces, etc.
So I investigated this and came to conclude that
Microsoft.CodeAnalysiswas always the culprit.Evidence: Stack Traces
First, in both VS 2019 and 2022, when I was examining stack traces of VS's threads during slowdowns, I saw references to
Microsoft.CodeAnalysis.SourceGener*types all the time:Evidence: Memory and Managed Heap Snapshots
When I attached another instance of VS to "debug" the problematic instance, and took managed heap snapshots, I noticed that the lion's share of allocated bytes was once again for types in
Microsoft.CodeAnalysis. Some types had over a million instances! This was not the case prior to DotNet 6 Preview 5 and prior to Preview 5 VS 2019 was not running out of memory at all on the same solution.Issue with Source Generators
Now, according to every stack trace that I took of VS's threads when it was misbehaving, it always involved
Microsoft.CodeAnalysis.SourceGeneratedDocumentStateandSystem.Text.Json.SourceGenerationtypes.Also note that my solution does NOT use any source generators. And VS's option
Text Editor > VB/C# > Advanced > Enable all features in opened files from source generators (experimental)is tuned OFF.I presume it might have something to do with Source Generator support being automatically included for
System.Text.Jsonstarting with Preview 5 (a was noted in your blog post), and so obviously it is applicable to me as I am now on Preview 6.Expected Behavior:
VS 2019 does not run out of memory.
VS 2019 and VS 2022 do not slow down and become sluggish after typing a few characters / words.
Actual Behavior:
Both VS 2019 and 2022 are slowed down, with VS 2019 being slowed down to the point of being completely unusable (due to memory pressure and GC), while VS 2022 is usable but typing experience deteriorates very quickly and becomes often literally one or a few characters per second (i.e. >500ms pause after each character on average).
It does not seem to correlate with what file I edit (large or small), what text I edit (comment, type name, variable name, code), and which project the file comes from (one on which other projects in the solution depend, or the one on which no other projects depend).
The text was updated successfully, but these errors were encountered: