.NET 8: Repositories dependency info re-work

[mmitche]

Describe the Problem

Source-build works today by building a repository, gathering the versions of the outputs, and creating a property file with those versions. This property file is then fed to downstream repositories, which import it after their eng/Versions.props. The resulting behavior is that the downstream repo overrides all dependency versions specified in eng/Versions.props with the versions built from source. This behavior is not the same as the Maestro dependency-flow based approach. Maestro only updates properties for dependencies that are specified in eng/Version.Details.xml.

This causes the following difference: Say that dotnet/aspnetcore has a Microsoft.Net.Compilers.Toolset dependency. It codes a property for that dependency's version in eng/Versions.props as MicrosoftNetCompilersToolsetVersion, with an older version of the package.

• When building via source-build, roslyn builds first, and aspnetcore will pick up the new version of Microsoft.Net.Compilers.Toolset, even though it doesn't want to.
• When building officially, the non-latest version will be used.

This difference tends to cause build breaks in source-build. The version bump may be significant and require repo reaction. This is not ideal and not sustainable.

One option would be to only override those versions that are specified in eng/Version.Details.xml. This would more closely align the source-build and current official builds. The huge downside is that this will cause an explosion of ref packs. And in some cases, we would be building against old versions but actually executing against newer ones. Non-ideal.

In summary, the goals are:

• Reduce source-build build breaks
• Avoid a large increase in ref packs.
• Ensure that it is possible to freeze on old versions of a specific component if absolutely necessary

Describe the Solution

The solution is for repositories to:

• Track dependencies that come from within repos in source-build accurately within eng/Version.Details.xml files.
• Upgrade dependencies regularly via dependency flow mechanisms, or pin dependencies that should stay the same, with justification.
• Source-build switches to only overriding unpinned dependency versions specified in eng/Version.Details.xml.

Additional Context

Sub-issues:

• .NET 8: Tracking issue for repos specifying and regularly updating their dependencies. #2980
• .NET 8: Only flow dependencies specified in eng/Version.Details.xml #2981
• .NET 8: Generate report of non-explicit dependencies for each repo in source build #2982

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[mmitche]

/cc @MichaelSimons @tkapin @crummel

[MichaelSimons]

This work has been completed.