Build-up list of used dependencies from Version.Details.xml

[oleksandr-didyk]

Part of #2482

As part of building a project-specific PackageVersions.props file, a list of used dependencies should be built first. The list should not contain any manually maintained (pinned) versions, so simple re-use of Versions.props is not enough for the current use-case.

Since Darc already parses the dependency versions from the PackageVersions.props file, its logical can be re-used for completing the specified task.

[oleksandr-didyk]

Expected implementation time -> 5 working days

[MichaelSimons]

@crummel - what do you think about excluding pinned versions as suggested here? I initially thought we would include all version.details.xml dependencies. I can see why it would make sense to skip pinned dependencies though. On the other hand, if the dependency is pinned and is the only occurrence in the source-build graph, source-build would produce the pinned version.

[MichaelSimons]

Question: How does the dependency flow work today in regards to transitive dependencies? If I declare a dependency on A which in turn depends on B, do all A and B version properties get updated in a dependency flow? That is the best way to ensure cohesion.

[oleksandr-didyk]

Question: How does the dependency flow work today in regards to transitive dependencies? If I declare a dependency on A which in turn depends on B, do all A and B version properties get updated in a dependency flow? That is the best way to ensure cohesion.

I would have to check since I'm not sure, unless maybe @crummel would know

[premun]

Possible related: I created an isolated class recently for reading the XML that can be probably re-used. This library is also available in installer already.

https://github.com/dotnet/arcade-services/blob/main/src/Microsoft.DotNet.Darc/src/DarcLib/Helpers/VersionDetailsParser.cs

[oleksandr-didyk]

@crummel Chris Rummel FTE - what do you think about excluding pinned versions as suggested here? I initially thought we would include all version.details.xml dependencies. I can see why it would make sense to skip pinned dependencies though. On the other hand, if the dependency is pinned and is the only occurrence in the source-build graph, source-build would produce the pinned version.

Discussed offline - decided to exclude pinned versions for now, but the topic might need more discussion / investigation in the future.

Furthermore, the existing code for reading the Version.Details.xml file should be ideally re-used

[premun]

Can someone please confirm/correct my idea of how this works? I would like to be sure I understand the process:

1. Someone pins a version in their repo
2. We override it with the newest during source-build, fingers crossed it works
3. If source-build breaks, we have two options?
   1. Go back to the repo and bump there
   2. We can re-pin the package via a source-build patch

My questions:
Ad 2. - this is what the "let's exclude pinned versions" statements means?

Ad 3.2. - Is this a real option? I noticed patches like that. This also means we need to have that pinned version in sbrp, right?

[MichaelSimons]

Can someone please confirm/correct my idea of how this works? I would like to be sure I understand the process:

The source build behavior depends on where the pinning occurs. If the repo that is pinned is the first dependency on that source-repo it wins. Source-build only builds one version of a repo. The first one it finds when traversing the dependency graph starting at installer wins. By wins, I mean it would be the version we source-build so the repo which pins it should not have an issue because source build is building the same version it depends on. If it is not the first occurence, we would not override it with the newest, rather it would be overridden by the first version found in the dependency graph. This could be a newer version or an older version. This is why it is important to have coherency. Regardless is could be a newer or older version at which point #3 comes in play. The options are 3.i,3.ii - except re-pinning the package would require the use of a ref pack to avoid prebuilts or 3.iii which is to add a source-build patch that addresses the build issues to get the repo to build with the version source-build produces.

[MichaelSimons]

This was completed as part of dotnet/installer#15267.