CfgAllPathValidator AreAllSuccessorsValid Stack Overflow on Windows and error MSB6006 in Linux Codespaces

[giuliohome]

Description

Stack Overflow on Windows
Tested on Windows 10 Enterprise and also on Windows 11 Enterprise 4vcpu 16BG ram

C:\Program Files\dotnet\sdk\8.0.200\Roslyn\Microsoft.CSharp.Core.targets(84,5): error : at SonarAnalyzer.CFG.Roslyn.
CfgAllPathValidator.b__7_0(SonarAnalyzer.CFG.Roslyn.BasicBlock)

Repro steps

Some code like https://github.com/dotnet/Open-XML-SDK/blob/main/test/DocumentFormat.OpenXml.Tests/ConformanceTest/WorkbookPr/GeneratedDocument.cs,
then

dotnet-sonarscanner begin ...
dotnet build 

Known workarounds

Deactivating from Warning to None the rules related to CfgAllPathValidator, found searching https://github.com/search?q=repo%3ASonarSource%2Fsonar-dotnet+CfgAllPathValidator&type=pullrequests

So S3604 and S2190 set to Action None in .sonar\conf\Sonar-cs.ruleset after dotnet-sonarscanner begin

The stack overflow is does not occur on Linux (e.g. Fedora 38). Of course this is related only to the client side, not to the openjava-17 server side.

Related information

• C#/VB.NET Plugins version latest
• Visual Studio version latest
• MSBuild / dotnet version latest
• SonarScanner for .NET version (if used) latest
• Operating System Win 11 Enterprise

dotnet --version
8.0.203

dotnet tool list -g
Package Id               Version      Commands
---------------------------------------------------------
dotnet-sonarscanner      6.2.0        dotnet-sonarscanner

java -version
java version "17.0.10" 2024-01-16 LTS
Java(TM) SE Runtime Environment (build 17.0.10+11-LTS-240)
Java HotSpot(TM) 64-Bit Server VM (build 17.0.10+11-LTS-240, mixed mode, sharing)

systeminfo | findstr /B /C:"OS Name" /B /C:"OS Version"
OS Name:                   Microsoft Windows 11 Enterprise
OS Version:                10.0.22631 N/A Build 22631

P.S. possibly related?

• Known issue of the working set in Open-XML-SDK on .NET Core? => see https://github.com/dotnet/Open-XML-SDK/tree/maintab=readme-ov-file#known-issues and System.IO.Compression: ZipArchiveEntry always stores uncompressed data in memory dotnet/runtime#1544

• Stack overflow in Rosyln CodeAnalysis? => see Stack overflow in Rosyln CodeAnalysis when ambiguous name is used dotnet/roslyn#71039

[martin-strecker-sonarsource]

Hello @giuliohome

I tried to reproduce the issue but I wasn't able to do so.

I created a .Net 8 console app and added the https://github.com/dotnet/Open-XML-SDK/blob/main/test/DocumentFormat.OpenXml.Tests/ConformanceTest/WorkbookPr/GeneratedDocument.cs file and <PackageReference Include="DocumentFormat.OpenXml" Version="3.0.2" />.
The project was compiled without errors with the latest version of our analyzer. I also tried this with .Net SDK versions 8.0.200, 8.0.202, and 8.0.203 but there was no difference.
I also cloned the OpenXML repo and analyzed it but there wasn't any error either.

Are you sure about the document in question? Can you provide a sample project with the error?

[giuliohome]

Hi, @martin-strecker-sonarsource, I understand that you are unable to reproduce the error using only that file, exactly as it appears in the original version. I acknowledge that.

Unfortunately, I cannot currently provide the full code (although I do have a deterministic reproduction of the issue). I'm not sure if I or someone else will be able to provide it later.

While you keep this on hold (or close it), here are a few hints:

The .cs file is generated from an .xlsx file, so consider that the bug may simply occur with a "different" but fundamentally similar template (more rows, different formatting). The issue lies in the fact that 1) the memory consumption while analyzing it (even when it does not throw an exception) could be quite excessive and 2) producing code smells from C# code generated from an .xlsx file by a Microsoft tool could be misleading when referring to the applicative project that simply uses a widely accepted library like the OpenXML SDK.

[martin-strecker-sonarsource]

Hi,

If the file causing the problems is auto-generated and you can control the code generation, it is recommended that such code be excluded from the analysis. To stop our analyzer from looking into such files, you can do either of these

• Place an // <auto-generated /> //<autogenerated /> or // generated by comment at the beginning
• Annotate the class in questions with a GeneratedCode attribute
• Name the file *.g.cs, *.generated.cs, or *.designer.cs

If you can not influence the code generation, you can use any of these options for file exclusions:
https://docs.sonarsource.com/sonarqube/latest/project-administration/analysis-scope/

It would still be helpful for us if you could help us create a reproducer. I suspect your generated file is huge but has a lot of repeating patterns. Can you give us some hints about these patterns, e.g., one giant method with simple statements, many small methods, or a huge method with a massive switch statement?

[giuliohome]

Okay, thank you!

That's incredible. On Windows, adding // <auto-generated /> at the beginning does fix the issue! I previously attempted using -d:sonar.exclusions="**/ExcelBaseTemplate.cs", but it didn't work.

I'm not sure if it optimizes memory usage on the Linux container as well, but I'll open a different issue if that's the problem.

Thanks!

[martin-strecker-sonarsource]

Hi,

I'm glad that helped you.

We take it very seriously when our analyzer breaks the build. Can you give us some more information about the generated code, so we can try reproducing the issue even though it is resolved for you? See also the second paragraph of #8977 (comment)

[giuliohome]

Thanks, I'll send you a repro, but I need to minimize it and delete any private info beforehand.

Can you give us some hints about these patterns, e.g., one giant method with simple statements, many small methods, or..

Yes, exactly. For example, GenerateWorksheetPart1Content with numerous Append calls.

P.S. (Please note that the situation on Linux may differ, and if that's the case, it would be more important from my perspective.)

[giuliohome]

A sample here: https://github.com/giuliohome-org/sonar-dotnet-issues-8977

[giuliohome]

I'm not going to open a different issue on Linux, but just look at what happens when you build it in Codespaces.

Codespaces

👋 Welcome to Codespaces! You are on our default image. 
   - It includes runtimes and tools for Python, Node.js, Docker, and more. See the full list here: https://aka.ms/ghcs-default-image
   - Want to use a custom image instead? Learn more here: https://aka.ms/configure-codespace

🔍 To explore VS Code to its fullest, search using the Command Palette (Cmd/Ctrl + Shift + P or F1).

📝 Edit away, run your app as usual, and we'll automatically make it available for you to access.

@giuliohome ➜ /workspaces/sonar-dotnet-issues-8977 (main) $ dotnet --version
8.0.201
@giuliohome ➜ /workspaces/sonar-dotnet-issues-8977 (main) $ dotnet build

Welcome to .NET 8.0!
---------------------
SDK Version: 8.0.201

Telemetry
---------
The .NET tools collect usage data in order to help us improve your experience. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.

Read more about .NET CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry

----------------
Installed an ASP.NET Core HTTPS development certificate.
To trust the certificate, view the instructions: https://aka.ms/dotnet-https-linux

----------------
Write your first app: https://aka.ms/dotnet-hello-world
Find out what's new: https://aka.ms/dotnet-whats-new
Explore documentation: https://aka.ms/dotnet-docs
Report issues and find source on GitHub: https://github.com/dotnet/core
Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli
--------------------------------------------------------------------------------------
MSBuild version 17.9.4+90725d08d for .NET
  Determining projects to restore...
  Restored /workspaces/sonar-dotnet-issues-8977/Generated.csproj (in 1.16 sec).
  Generated -> /workspaces/sonar-dotnet-issues-8977/bin/Debug/net8.0/Generated.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:24.36

And even if you run it and time it there:

@giuliohome ➜ /workspaces/sonar-dotnet-issues-8977 (main) $ mv output.xlsx original_output.xlsx 
@giuliohome ➜ /workspaces/sonar-dotnet-issues-8977 (main) $ time dotnet run
Hello, World!

real    0m3.044s
user    0m2.691s
sys     0m0.673s
@giuliohome ➜ /workspaces/sonar-dotnet-issues-8977 (main) $ cp output.xlsx original_output.xlsx 
@giuliohome ➜ /workspaces/sonar-dotnet-issues-8977 (main) $ 

Enters sonar-dotnet analyzer

@giuliohome ➜ /workspaces/sonar-dotnet-issues-8977 (main) $ dotnet add package SonarAnalyzer.CSharp 
  Determining projects to restore...
  Writing /tmp/tmpPgiPwT.tmp
info : X.509 certificate chain validation will use the fallback certificate bundle at '/usr/share/dotnet/sdk/8.0.201/trustedroots/codesignctl.pem'.
info : X.509 certificate chain validation will use the fallback certificate bundle at '/usr/share/dotnet/sdk/8.0.201/trustedroots/timestampctl.pem'.
info : Adding PackageReference for package 'SonarAnalyzer.CSharp' into project '/workspaces/sonar-dotnet-issues-8977/Generated.csproj'.
info :   GET https://api.nuget.org/v3/registration5-gz-semver2/sonaranalyzer.csharp/index.json
info :   OK https://api.nuget.org/v3/registration5-gz-semver2/sonaranalyzer.csharp/index.json 223ms
info :   GET https://api.nuget.org/v3/registration5-gz-semver2/sonaranalyzer.csharp/page/1.10.0/8.2.0.14119.json
info :   OK https://api.nuget.org/v3/registration5-gz-semver2/sonaranalyzer.csharp/page/1.10.0/8.2.0.14119.json 115ms
info :   GET https://api.nuget.org/v3/registration5-gz-semver2/sonaranalyzer.csharp/page/8.3.0.14607/9.5.0.73987.json
info :   OK https://api.nuget.org/v3/registration5-gz-semver2/sonaranalyzer.csharp/page/8.3.0.14607/9.5.0.73987.json 111ms
info :   GET https://api.nuget.org/v3/registration5-gz-semver2/sonaranalyzer.csharp/page/9.6.0.74858/9.23.0.88079.json
info :   OK https://api.nuget.org/v3/registration5-gz-semver2/sonaranalyzer.csharp/page/9.6.0.74858/9.23.0.88079.json 110ms
info : Restoring packages for /workspaces/sonar-dotnet-issues-8977/Generated.csproj...
info :   GET https://api.nuget.org/v3-flatcontainer/sonaranalyzer.csharp/index.json
info :   OK https://api.nuget.org/v3-flatcontainer/sonaranalyzer.csharp/index.json 134ms
info :   GET https://api.nuget.org/v3-flatcontainer/sonaranalyzer.csharp/9.23.0.88079/sonaranalyzer.csharp.9.23.0.88079.nupkg
info :   OK https://api.nuget.org/v3-flatcontainer/sonaranalyzer.csharp/9.23.0.88079/sonaranalyzer.csharp.9.23.0.88079.nupkg 6ms
info : Installed SonarAnalyzer.CSharp 9.23.0.88079 from https://api.nuget.org/v3/index.json to /home/codespace/.nuget/packages/sonaranalyzer.csharp/9.23.0.88079 with content hash /dwzO34zmW3xDv1i3QfhoyaKZ2aCBPWoRBwRw5Vnextq7nD2sWB+/6yjxn/Fa+Z3xhdsYoVbzIIvN5RvJ81dRw==.
info :   CACHE https://api.nuget.org/v3/vulnerabilities/index.json
info :   CACHE https://api.nuget.org/v3-vulnerabilities/2024.03.23.05.37.04/vulnerability.base.json
info :   CACHE https://api.nuget.org/v3-vulnerabilities/2024.03.23.05.37.04/2024.03.25.17.37.22/vulnerability.update.json
info : Package 'SonarAnalyzer.CSharp' is compatible with all the specified frameworks in project '/workspaces/sonar-dotnet-issues-8977/Generated.csproj'.
info : PackageReference for package 'SonarAnalyzer.CSharp' version '9.23.0.88079' added to file '/workspaces/sonar-dotnet-issues-8977/Generated.csproj'.
info : Generating MSBuild file /workspaces/sonar-dotnet-issues-8977/obj/Generated.csproj.nuget.g.props.
info : Writing assets file to disk. Path: /workspaces/sonar-dotnet-issues-8977/obj/project.assets.json
log  : Restored /workspaces/sonar-dotnet-issues-8977/Generated.csproj (in 586 ms).
@giuliohome ➜ /workspaces/sonar-dotnet-issues-8977 (main) $ time dotnet build
MSBuild version 17.9.4+90725d08d for .NET
  Determining projects to restore...
  Restored /workspaces/sonar-dotnet-issues-8977/Generated.csproj (in 262 ms).
/usr/share/dotnet/sdk/8.0.201/Roslyn/Microsoft.CSharp.Core.targets(84,5): error MSB6006: "csc.dll" exited with code 143. [/workspaces/sonar-dotnet-issues-8977/Generated.csproj]

Build FAILED.

/usr/share/dotnet/sdk/8.0.201/Roslyn/Microsoft.CSharp.Core.targets(84,5): error MSB6006: "csc.dll" exited with code 143. [/workspaces/sonar-dotnet-issues-8977/Generated.csproj]
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:01:52.20

real    1m52.657s
user    2m21.060s
sys     0m10.765s

Adding // <auto-generated /> does not help in this case.

[martin-strecker-sonarsource]

Thank you, @giuliohome, for the detailed re-producer and instructions. I opened PR #8984 with the code based on https://github.com/giuliohome-org/sonar-dotnet-issues-8977. The unit test is falling with the StackOverflow, so we can investigate and see what we can do about it. We will also take the performance problem into account while looking at it.

[giuliohome]

Thanks! Much appreciated @martin-strecker-sonarsource

[giuliohome]

@martin-strecker-sonarsource, I've seen that the PR has been merged. Thanks again! With all my admiration for the hard and competent work you did to deliver this fix. Your approach to problem-solving is truly inspiring.

Best Regards
Giulio