STJ source generator fails due to 260 character path length limit on Windows (with EmitCompilerGeneratedFiles=true)

[dallmair]

Description

When I'm building my project in VS, source generation fails with the following error:

CSC : error CS0016: Could not write to output file 'D:\Source\SomeDirectoryX\some-repository-abcdef\src\Host\obj\Debug\net8.0\generated\System.Text.Json.SourceGeneration\System.Text.Json.SourceGeneration.JsonSourceGenerator\UpdateSomethgMessageJsonSerializerContext.IReadOnlyCollectionSystemConfigurationInfo.g.cs' -- 'Could not find a part of the path 'D:\Source\SomeDirectoryX\some-repository-abcdef\src\Host\obj\Debug\net8.0\generated\System.Text.Json.SourceGeneration\System.Text.Json.SourceGeneration.JsonSourceGenerator\UpdateSomethgMessageJsonSerializerContext.IReadOnlyCollectionSystemConfigurationInfo.g.cs'.'

Just renaming one of the source directories to have a slightly shorter name suffices to make it work, so the problem is related to the length of the path. Of course, I can shorten the path on my side a bit, but there is also significant potential in the STJ source generator to reduce the risk of this issue happening:

1. The System.Text.Json.SourceGeneration appears twice in the directory structure. Once ought to be good enough.
2. Maybe it's possible to emit all JsonSerializerContext properties (like in this case IReadOnlyCollectionSystemConfigurationInfo) into a single file, or choose a different and shorter file name for those files.

This only happens with <EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles> in my Directory.Build.props file. When I comment this out, the generated code is not written to disk anymore, so it does not hit the path length limit and compilation succeeds.

Reproduction Steps

Put your C# project in a path with 100 characters. Set EmitCompilerGeneratedFiles to true. Compile on Windows.

(Alternatively, use a shorter path but longer property/type names.)

Expected behavior

The build should work fine.

As it is a path length issue, there will always be some limit. However, I was very surprised to hit this limit in a rather small project today. Expectation is to lift the limit such that an average developer does not encounter it in small projects anymore.

Actual behavior

The constant path overhead induced by SDK and STJ source generator is roughly 150 characters (obj\Debug\net8.0\generated\System.Text.Json.SourceGeneration\System.Text.Json.SourceGeneration.JsonSourceGenerator\FooJsonSerializerContext.Bar.g.cs), so even with some ordinary type/property names the path length of the generated files crosses the 260 character limit on Windows, which causes the build to fail.

Regression?

No response

Known Workarounds

Shorten the source repository root path. In my case, I can shorten the path by ~30 characters, at most. However, it's inconvenient and I should not need to worry as I'm just using 20-25% of the available length and the rest is used by the SDK/source generator.

Configuration

8.0 RC1

Other information

No response

[ghost]

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

When I'm building my project in VS, source generation fails with the following error:

CSC : error CS0016: Could not write to output file 'D:\Source\SomeDirectoryX\some-repository-abcdef\src\Host\obj\Debug\net8.0\generated\System.Text.Json.SourceGeneration\System.Text.Json.SourceGeneration.JsonSourceGenerator\UpdateSomethgMessageJsonSerializerContext.IReadOnlyCollectionSystemConfigurationInfo.g.cs' -- 'Could not find a part of the path 'D:\Source\SomeDirectoryX\some-repository-abcdef\src\Host\obj\Debug\net8.0\generated\System.Text.Json.SourceGeneration\System.Text.Json.SourceGeneration.JsonSourceGenerator\UpdateSomethgMessageJsonSerializerContext.IReadOnlyCollectionSystemConfigurationInfo.g.cs'.'

Just renaming one of the source directories to have a slightly shorter name suffices to make it work, so the problem is related to the length of the path. Of course, I can shorten the path on my side a bit, but there is also significant potential in the STJ source generator to reduce the risk of this issue happening:

1. The System.Text.Json.SourceGeneration appears twice in the directory structure. Once ought to be good enough.
2. Maybe it's possible to emit all JsonSerializerContext properties (like in this case IReadOnlyCollectionSystemConfigurationInfo) into a single file, or choose a different and shorter file name for those files.

This only happens with <EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles> in my Directory.Build.props file. When I comment this out, the generated code is not written to disk anymore, so it does not hit the path length limit and compilation succeeds.

Reproduction Steps

Put your C# project in a path with 100 characters. Set EmitCompilerGeneratedFiles to true. Compile on Windows.

(Alternatively, use a shorter path but longer property/type names.)

Expected behavior

The build should work fine.

As it is a path length issue, there will always be some limit. However, I was very surprised to hit this limit in a rather small project today. Expectation is to lift the limit such that an average developer does not encounter it in small projects anymore.

Actual behavior

The constant path overhead induced by SDK and STJ source generator is roughly 150 characters (obj\Debug\net8.0\generated\System.Text.Json.SourceGeneration\System.Text.Json.SourceGeneration.JsonSourceGenerator\FooJsonSerializerContext.Bar.g.cs), so even with some ordinary type/property names the path length of the generated files crosses the 260 character limit on Windows, which causes the build to fail.

Regression?

No response

Known Workarounds

Shorten the source repository root path. In my case, I can shorten the path by ~30 characters, at most. However, it's inconvenient and I should not need to worry as I'm just using 20-25% of the available length and the rest is used by the SDK/source generator.

Configuration

8.0 RC1

Other information

No response

Author:	dallmair
Assignees:	-
Labels:	area-System.Text.Json
Milestone:	-

[Sergio0694]

"Expectation is to lift the limit such that an average developer does not encounter it in small projects anymore."

Side note — I don't think average developers would set EmitCompilerGeneratedFiles in the first place.

[teo-tsirpanis]

@dallmair does the issue reproduce if you build with dotnet build?

[dallmair]

Side note — I don't think average developers would set EmitCompilerGeneratedFiles in the first place.

@Sergio0694 Hmm, I tend to agree and disagree at the same time.

Conceptually, I think your argument makes sense. However, I have a new hire here in my company who never heard of source generators and thinks it's all black magic that is going on. So in the spirit of "What are you not seeing? Look underneath." I showed that it's possible to open up the black box and how to do it. It's a great way to learn how systems work, after all. That is how we discovered the issue, and I think it's pretty common to dig deeper than what one already knows.

@teo-tsirpanis Indeed I forgot to even test this -- sorry! No, the issue does not reproduce with dotnet build, command-line build works fine and produces the expected result.

[danmoseley]

Not directly addressing it but does this help?

https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry#enable-long-paths-in-windows-10-version-1607-and-later

[Sergio0694]

@dallmair That certainly makes sense, but just FYI, it's likely easier to show them that you can simply expand the Project > References > Analyzers > Analyzer nodes in the solution explorer in VS, and see all generated files directly from there, with IntelliSense support and everything. It's much more convenient than emitting files to disk and looking at them from there 🙂

Granted, trying to open one from there that has a path that's too long will outright crash VS (I logged a bug, @sharwell is investigating already), but at least the actual build will not fail, and you'll be able to open files within the path length limit already.

[sharwell]

As it is a path length issue, there will always be some limit.

For dotnet/roslyn, we just made the result undefined if the user hasn't enabled long paths. I would recommend adopting a similar step in the build of the affected project.
dotnet/roslyn#68217

[teo-tsirpanis]

Not directly addressing it but does this help?

https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry#enable-long-paths-in-windows-10-version-1607-and-later

Setting the registry option might help indeed. I checked that both the MSBuild.exe and csc.exe files that come with Visual Studio have a manifest with long paths enabled.

Modern .NET supports long paths by itself since dotnet/corefx#3001.

[eiriktsarpalis]

I'm trying to understand if there's any action required here either from individual source generators or VS to get this to work in devices that don't have long paths enabled system-wide, or do we leave this as by-design behaviour? @sharwell should we transfer this issue to the roslyn repo perhaps?

[rhirano0715]

EmitCompilerGeneratedFiles=true is set in the following 3 and CS0016 occurs when I build with Visual Studio.

runtime/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/System.Text.Json.Tests.csproj

Line 4 in a485cbe

<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>

runtime/src/libraries/System.Text.Json/tests/System.Text.Json.SourceGeneration.Tests/System.Text.Json.SourceGeneration.Tests.targets

Line 4 in a485cbe

<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>

runtime/src/libraries/System.Text.Json/tests/System.Text.Json.SourceGeneration.Unit.Tests/System.Text.Json.SourceGeneration.Unit.Tests.targets

Line 4 in a485cbe

<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>

Are these EmitCompilerGeneratedFiles=true settings necessary?

[eiriktsarpalis]

Are these EmitCompilerGeneratedFiles=true settings necessary?

Building dotnet/runtime does require you to enable long paths.

[eiriktsarpalis]

Per #93111 (comment) I'm going to close this issue since it isn't clear to me if there's anything actionable on the libraries layer.