Resolve package references to projects

[natemcmaster]

In project.json, dependencies could be either a project or a package reference. NuGet and compilation would resolve the package vs project based on some conventions (like folder name) and the global.json file. Futhermore, VS would automatically add new projects to the solution explorer when the were discovered.

In MSBuild, there is no equivalent feature. References are either ProjectReference or PackageReference. Swapping a package reference to a project is not always straight forward. In cases where the package is a transitive reference from other PackageReferences, you have to keep the PackageReference and set ExcludeAssets=All in order to avoid conflicts between ProjectReference and PackageReference. (See https://docs.microsoft.com/en-us/nuget/schema/msbuild-targets#replacing-one-library-from-a-restore-graph). It also requires manually adding new projects to the sln file so they appear in VS.

It would be nice to provide a way to resolve PackageReference to projects without requiring changes to csproj.

---

Some data on this:

• The ASP.NET team frequently needs this as we debug components across multiple solutions. We've attempted to automate this with scripting. (See this script in Kestrel). cc @pakrym
• ASP.NET core benchmark automation also had to implement a way to resolve package references as projects. Implementing this required writing code that hunts for PackageReference and edits the csproj file locally. See Restore source dependencies feature (#185) aspnet/Benchmarks#190 cc @mikeharder
• I've been asked about this several times from non-Microsoft devs
  • http://www.natemcmaster.com/blog/2017/01/19/project-json-to-csproj/#comment-3203817959
  • http://www.natemcmaster.com/blog/2017/02/01/project-json-to-csproj-part2/#comment-3206867094
  • http://www.natemcmaster.com/blog/2017/03/09/vs2015-to-vs2017-upgrade/#comment-3273066914
  • Twitter: https://twitter.com/digory/status/846119886426271744
• StackOverflow question: http://stackoverflow.com/questions/42773815/net-core-csproj-global-json-projects-equivalent

---

cc @davidfowl @dsplaisted

[dsplaisted]

This is possible to some degree today. If you have a ProjectReference, it will override any package references to the same package ID in the restore graph. You don't need the PrivateAssets=all PackageReference shown in the docs when want to replace the package with a project reference instead of a reference to a DLL.

Some caveats:

• For things to work in VS, you need to add the projects to your solution
• You may need to add feeds to your NuGet.config to match those that apply to the source code you're bringing in
• The package version for the project needs to be at least as high as any package dependencies on it require. So if the source code you're bringing in doesn't set the package version (ie if that's a parameter that's only passed in when you pack, for example), then you'll get downgrade warnings. This is a NuGet issue: Avoid downgrade warnings for Package vs Project checks NuGet/Home#4620

@CesarBS got this working for the ASP.NET benchmarks repo: aspnet/Benchmarks#190

[davidfowl]

@dsplaisted It's such a bad experience though. We need something comparable to what project.json had. It needs to be obvious and it needs to just work without the caveats listed. On top of that, it doesn't work well in VS if the projects aren't added to the solution (something xproj did automatically)

[cwe1ss]

Getting this feature back would be awesome!

With global.json, it was pretty easy to mistakenly commit the "wrong" global.json/sln file with the local debug paths. It would show up as a broken build on the CI server so it's not the end of the world. I'm just wondering if there's any chance of doing this without touching any of the committed files? Maybe the overrides could be stored in the .suo file or in a separate MySolution.sln.overrides file which could be added to the .gitignore file.

[fancyDevelopment]

@davidfowl I completly agree to you. The developer experience on dnx with project.json was great. especially in the case of debugging into a nuget package from own code. You had just to clone the git repository and set one path in global json. Had to make not a single Change to my project. I couldn't be easier. I can't understand why Microsoft has thrown away such a great feature when all this was already a release candidate and very close to a final release.

[mclark1129]

I also created a uservoice to restore this or similar tooling within VS.

https://visualstudio.uservoice.com/forums/121579-visual-studio-ide/suggestions/19071868-restore-global-json-functionality-in-vs2017

[tmitchel2]

I've been crying out for this for years, https://lernajs.io/ does something very similar (functionality, not implementation obviously) in the node.js world and it works really well.

[khirakawa]

For anyone who is looking to try the conditional reference solution accepted in this SO question to workaround this issue, note that conditional references aren't currently supported when doing nuget restores NuGet/Home#4996. Restoring nuget packages from VS GUI seems to work fine though.

[ToddThomson]

Here is a manual process that works for me:

1. When using debug configuration and source exists: Use project reference(s)

2. Otherwise utilize the package reference(s)
   Packages are created by CI with a custom msbuild target to generate the PackageVersion ( semver version 2 ) number when commits are pushed to the TFS Server.

3. When changing the project build configuration ( Debug <-> Release ) the project must be unloaded/reloaded ( there might be a better way, but I rarely need to move from the Debug configuration ).

4. TFS CI produces the Release build configuration packages with PackageVersion using:
   version.props imported from Directory.Build.props

Project>
  <PropertyGroup>
    <VersionPrefix>0.9.0</VersionPrefix>
    <VersionSuffix>ci</VersionSuffix>
  </PropertyGroup>
</Project>

file: Directory.Build.targets

<Project>
  
  <Target Name="SetVersion" BeforeTargets="Build" >
    <PropertyGroup>
      <BuildNumber>$([System.DateTime]::Now.ToString(yyyyMMdd-mmss))</BuildNumber>
      <PackageVersion Condition="'$(VersionSuffix)' != '' AND '$(BuildNumber)' != ''">$(VersionPrefix)-$(VersionSuffix)-$(BuildNumber)</PackageVersion>      
    </PropertyGroup>
  </Target>
  
</Project>

project csproj snippet

  <Choose>
    <When Condition="'$(Configuration)' == 'Debug' AND Exists('..\..\..\Acme')">
      <ItemGroup Condition="'$(Configuration)' == 'Debug'">
        <ProjectReference Include="..\..\Acme.Composition\Acme.Composition\Acme.Composition.csproj" />
      </ItemGroup>
    </When>
    <Otherwise>
      <ItemGroup>
        <PackageReference Include="Acme.Composition" Version="0.9.0-ci-*" />
      </ItemGroup>
    </Otherwise>
  </Choose>

[jnm2]

Is there a way to replace a package reference with a project reference when the package is referenced both directly and transitively? I'm not finding a way to suppress the transitive DLL reference, so the compiler finds every type both places and nothing compiles.

[natemcmaster]

You can't for a transitive package reference. You have to lift it to a direct package reference and add ExcludeAssets=All. https://docs.microsoft.com/en-us/nuget/schema/msbuild-targets#replacing-one-library-from-a-restore-graph

[jnm2]

I think that would have done the trick except that this is an old csproj due to needing winforms designers to work, so necessary package references are not transitively referenced through the replacement project reference.

[TFTomSun]

@natemcmaster you could do something similar to what I do to automatically treat packagereferences as projectreferences. Create a Directory.Build.targets in the root your repositories and add something like this:

<!--?xml version="1.0" encoding="utf-8"?-->
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <ItemGroup>
  
    <!-- Keep the identity of the  packagereference -->
    <ExtendedPackageReference Include="@(PackageReference)">
      <PackageName>%(Identity)</PackageName>
    </ExtendedPackageReference>
	
	<!-- Find all package references that are expected to be available as projects -> My convetion: they start with TomSun. -->
    <ExtendedPackageReference2 Include="@(ExtendedPackageReference -> StartsWith('TomSun.') )">
      <IsTomSunPackage>%(Identity)</IsTomSunPackage>
    </ExtendedPackageReference2>
	
	<!-- Filter them by mapping them to another itemGroup using the WithMetadataValue item function -->
    <TomSunPackageTemp Include="@(ExtendedPackageReference2 -> WithMetadataValue('IsTomSunPackage', True) )"/>
  
    <!-- Map back to the orginal packagereference identity -->
    <TomSunPackage Include="@(TomSunPackageTemp -> '%(PackageName)' )"/>
	
	<!-- Remove the TomSun. prefix -->
    <TomSunRepository Include="@(TomSunPackage -> Replace('TomSun.','') )"/>
	
	<!-- Reference them as the projects:  My convention here is that the project is located in [RepositoryNameWithoutTomSun]\[PackageName]\[PackageName].csproj -->
    <ProjectReference  Include="@(TomSunRepository -> '..\..\%(Identity)\TomSun.%(Identity)\TomSun.%(Identity).csproj' )"></ProjectReference>

	<!-- Remove the package references that are now referenced as projects -->
    <PackageReference Remove="@(TomSunPackage)"/>
</ItemGroup>
 
 <!-- Some logging ... -->
  <Target Name="LoggingTarget" BeforeTargets="Build">
    <Message Text="@(TomSunPackage)" Importance="high"></Message>
  </Target>
</Project>

The result is, that you can keep your packagereferences without any switch/case logic within the projects itself. They will be only converted to project references, based on conventions, when the Directory.Build.targets file is available in some parent directory. In my case the file is in the parent directory of all git repositories and is only available on my dev machine. I'm quite sure that you could extract such a behavior also into a build-nuget-package.

Of course your convention is different, but maybe this gives you an idea how u could solve it in your case. What is missing in this example is the check, whether the project is in the current solution. That check is technical possible, but i haven't take over the logic yet from my old target framework that was written for classic MSBuild project files.

Could be that this can be done easier now (starting from VS 2017) with targets and a task written in C#. But at least up to VS2015 it was necessary to do that transformation outside of the targets, otherwise the VS UI didn't show the references correctly.

[NitashEU]

@TFTomSun your solution helped me alot! You might consider to add <InProject>false</InProject>.

In VS2017 it showed me all the packages in the UI. Doing this in the first item was enough for me:

<ExtendedPackageReference Include="@(PackageReference)">
  <InProject>false</InProject>
  <PackageName>%(Identity)</PackageName>
</ExtendedPackageReference>

[ManniManfred]

Hi, my current workaround is the following, that checks the solution file.

<PropertyGroup>
	<SolutionContent Condition=" '$(SolutionPath)' != '' and Exists('$(SolutionPath)') ">$([System.IO.File]::ReadAllText($(SolutionPath)))</SolutionContent>
</PropertyGroup>
	
<ItemGroup Condition="'$(SolutionContent)' != ''">
	<MyPackageReferences Include="@(PackageReference)">
		<IsInSln>$(SolutionContent.Contains('$([System.String]::Copy('"%(Identity)"'))'))</IsInSln>
		<ProjectPath>%(ProjectPath)</ProjectPath>
	</MyPackageReferences>

	<ToUseProjectRef Include="@(MyPackageReferences -> WithMetadataValue('IsInSln', 'True') )"
				Exclude="@(MyPackageReferences -> WithMetadataValue('ProjectPath', '') )">
		<PackageName>%(Identity)</PackageName>
		<ProjectPath>%(ProjectPath)</ProjectPath>
	</ToUseProjectRef>
	
	<ProjectReference Include="@(ToUseProjectRef -> '%(ProjectPath)' )" />
	<PackageReference Remove="@(ToUseProjectRef)"/>
</ItemGroup>

And add the ProjectPath to package element:

    <PackageReference Include="GInfoNET.Libs.ErrorHandler" Version="1.0.0">
      <ProjectPath>..\..\ErrorHandler\GInfoNET.Libs.ErrorHandler.csproj</ProjectPath>
    </PackageReference>

I tried to get the project path dynamically from solution file, but without success :(

	<ToUseProjectRef Include="@(MyPackageReferences -> WithMetadataValue('IsInSln', 'True') )"
			 Exclude="@(MyPackageReferences -> WithMetadataValue('ProjectPath', '') )">
		<PackageName>%(Identity)</PackageName>
		<ProjectPathStartIndex>$([MSBuild]::Add(3, $(SolutionContent.IndexOf(',', %(ProjectNameIndex)))))</ProjectPathStartIndex>
		<ProjectPathEndIndex>$(SolutionContent.IndexOf('"', %(ProjectPathStartIndex)))</ProjectPathEndIndex>
		<ProjectPath>$(SolutionContent.Substring(%(ProjectPathStartIndex), $([MSBuild]::Subtract(%(ProjectPathEndIndex), %(ProjectPathStartIndex)))))</ProjectPath>
	</ToUseProjectRef>

[TFTomSun]

@NitashEU @ManniManfred My current solution determines the project path automatically using regex. It's now completely generic/common and easy to reuse.

  <PropertyGroup>
    <ReplacePackageReferences>true</ReplacePackageReferences>
  </PropertyGroup>

  <Choose>
    <When Condition="$(ReplacePackageReferences) AND '$(SolutionPath)' != '' AND '$(SolutionPath)' != '*undefined*' AND Exists('$(SolutionPath)')">
      
      <PropertyGroup>
        <SolutionFileContent>$([System.IO.File]::ReadAllText($(SolutionPath)))</SolutionFileContent>
        <SmartSolutionDir>$([System.IO.Path]::GetDirectoryName( $(SolutionPath) ))</SmartSolutionDir>
        <RegexPattern>(?&lt;="[PackageName]", ")(.*)(?=", ")</RegexPattern>
      </PropertyGroup>
      
      <ItemGroup>
        
        <!-- Keep the identity of the  packagereference -->
        <SmartPackageReference Include="@(PackageReference)">
          <PackageName>%(Identity)</PackageName>
          <InSolution>$(SolutionFileContent.Contains('\%(Identity).csproj'))</InSolution>
        </SmartPackageReference>

        <!-- Filter them by mapping them to another itemGroup using the WithMetadataValue item function -->
        <PackageInSolution Include="@(SmartPackageReference -> WithMetadataValue('InSolution', True) )">
          <Pattern>$(RegexPattern.Replace('[PackageName]','%(PackageName)') )</Pattern>
          <SmartPath>$([System.Text.RegularExpressions.Regex]::Match( '$(SolutionFileContent)', '%(Pattern)' ))</SmartPath>
        </PackageInSolution>
        
        <ProjectReference  Include="@(PackageInSolution -> '$(SmartSolutionDir)\%(SmartPath)' )"/>

        <!-- Remove the package references that are now referenced as projects -->
        <PackageReference Remove="@(PackageInSolution -> '%(PackageName)' )"/>
      </ItemGroup>

    </When>
  </Choose>

@ManniManfred
The reason why ur solution doesn't work is a known bug in MSBuild that has been shipped with Visual Studio 2017 Update 3:
dotnet/msbuild#2476

[ManniManfred]

@TFTomSun thanks for your solution

[tmitchel2]

Does anyone know if the https://github.com/dotnet/sourcelink project is meant to lend a hand with improving the experience here...?

[ChristopherHaws]

@tmitchel2 Not really. SourceLink allows you to embed the location of source files into the PDB, which is particularly useful when the link is to a public URL (such as a specific commit to a file on GitHub). Essentially, SourceLink provides the ability for IDE's such as Visual Studio to get the file for the code you are debugging into, but it does not load the project in your solution and allow you to modify it which is what this issue is about.

[TFTomSun]

@tmitchel2 to be a bit more specific, project references don't only ensure a nice debugging experience which u can indeed have with packagereferences too....but also think about refactorings over multiple projects...that wouldn't be possible with package references.therefore automatically replacing package references with project references when the source code of the referenced project is pulled still makes sense.

[ManniManfred]

Hi, I'm looking for a way to also include the .props and the .targets file of the "transfered" project.
I tried the following

...
        <!-- Filter them by mapping them to another itemGroup using the WithMetadataValue item function -->
        <PackageInSolution Include="@(SmartPackageReference -> WithMetadataValue('InSolution', True) )">
          <Pattern>$(RegexPattern.Replace('[PackageName]','%(PackageName)') )</Pattern>
          <SmartPath>$([System.Text.RegularExpressions.Regex]::Match( '$(SolutionFileContent)', '%(Pattern)' ))</SmartPath>
          <ProjectDir>$(SmartSolutionDir)\$([System.IO.Path]::GetDirectoryName(%(SmartPath)))</ProjectDir>
          <PropsToImport Condition="Exists('%(ProjectDir)\build\%(PackageName).props')">%(ProjectDir)\build\%(PackageName).props</PropsToImport>
          <TargetsToImport Condition="Exists('%(ProjectDir)\build\%(PackageName).targets')">%(ProjectDir)\build\%(PackageName).targets</TargetsToImport>
        </PackageInSolution>
        
        <ProjectReference Include="@(PackageInSolution -> '$(SmartSolutionDir)\%(SmartPath)' )"/>
        
        <!-- Remove the package references that are now referenced as projects -->
        <PackageReference Remove="@(PackageInSolution -> '%(PackageName)' )"/>

      </ItemGroup>

      <PropertyGroup>
        <AllPropsToImport>@(PackageInSolution->'%(PropsToImport)')</AllPropsToImport>
        <AllTargetsToImport>@(PackageInSolution->'%(TargetsToImport)')</AllTargetsToImport>
      </PropertyGroup>
      
    </When>
  </Choose>
  
  <Import Project="$(AllPropsToImport)" Condition="'$(AllPropsToImport)' != ''" />
  <Import Project="$(AllTargetsToImport)" Condition="'$(AllTargetsToImport)' != ''" />

but without success. The import statement failed.

[TFTomSun]

@ManniManfred
I'm quite sure that the Condition attribute is not supported in itemgroup meta data. As far as I know, the imports are processed first, before any properties or even itemgroups are evalulated. So i think it's technical not possible.

[dsplaisted]

@TFTomSun There are several passes of MSBuild evaluation. The first pass evaluates properties and also follows imports. So an import condition can depend on properties which were previously defined. However, items are evaluated in a subsequent phase, so imports can't depend on items.

There is some documentation on this under "Property and item evaluation order" here: https://docs.microsoft.com/en-us/visualstudio/msbuild/comparing-properties-and-items

The source code which implements this is here: https://github.com/Microsoft/msbuild/blob/master/src/Build/Evaluation/Evaluator.cs

[Simon-Gregory-LG]

It seems that Visual Studio 16.8 has regressed the experience of some of these discussed solutions, as the Manage NuGet window is considering the 'project version number' rather than the 'package version number' when resolving the installed version against the package feed.

It seems to come from a belief that projects and packages need to be treated the same in most cases. This completely misses the point that projects are completely out of scope when managing installed NuGet packages and so VS isn't currently honouring this:

NuGet/Home#10517

[softwarekamal]

Please any workaround for this. can you support switch between ProjectReference and PackageReference.
I find old Repo that do that See Here
But its not updated to Visual Studio 2022 yet.

Can you guys please vote to support that feature from following link:
https://developercommunity.visualstudio.com/t/Hybrid-ProjectReferences-and-PackageRefe/1609998?space=8&q=switch+projectreference&ftype=idea&entry=suggestion&dialog=comment

I wish you support that feature

[softwarekamal]

Edited,
Hello TFTomSun, I try your last solution.
1.I modify *.csproj and put the code #1151 (comment)
2.Then I add nuget package as reference and attach a Project to solution without reference it through GUI Add Reference. I think your script will do that csproj will do that.

Now Please what steps should I do during I still get nupkg version not Project Reference DLL, Should I modify csproj after add nuget package? or clean all references before apply script? Please any help for that.

[TFTomSun]

@softwarekamal just to clarify: The purpose of the script is being able to specify only PackageReferences in the csprojs and have an automatic resolution to ProjectReferences when a project with the specified package name exists in the current solution. The script can be either

• placed in the Directory.Build.targets file in the root of your repository (it will then automatically affect all csprojs that are on the same folder level or below the Directory.Build.targets file
• or it could also be made reusable across repositories by creating a nuget build SDK package. I provide such a nuget sdk package via nuget, if somebody is interested in it

important: The script is not intended to be copied into the csproj directly. The purpose is to keep the csprojs clean.

[softwarekamal]

Hello TFTomSun,
Thank you much everything is working now. I created Directory.Build.targets, And csproj is very clean.
There's little small trick here. Why there are still (Alert notification with Referenced DLL library?)

Would you mind to look I upload Solution *.rar https://anonfiles.com/h6e529Jdx4/TwoSolution_rar

Thank you much dear

[TFTomSun]

@softwarekamal I don't download and unpack files from unknown sources. But anyways, it looks like your projects are not using the sdk style. I'd suggest to migrate them to sdk style first.
follow the migration guide here:
https://docs.microsoft.com/de-de/dotnet/desktop/winforms/migration/?view=netdesktop-6.0

Afterwards replace the targetframework net5.0-windows with net48, if you want to keep building your project against the classic .NET Framework (4.8)

[softwarekamal]

Hello TFTomSun,
Thank you much dear for information. I have little problem that I still need to use .NET framework 4.8 because some libraries I use don't upgraded to new framework, can you look at following repo please, you can open it in visual studio directly https://github.com/softwarekamal/FooProject---DebugAndEdit

When you navigate to FooProject.cs you will find <TargetFrameworkVersion>v4.8</TargetFrameworkVersion> is exists. I don't know why the yellow notification icon still exists :(

Thank you much for great information dear.

[TFTomSun]

@softwarekamal you can keep building your project against net48 after the sdk migration. Please follow the steps mentioned in the link above to migrate to sdk style. Otherwise the given solution to dynamically switch between project and packagereferences probably won't work for you. At least I never tested it with the old style project format.

[weltkante]

@nagilson I don't see anything having happened to this issue recently warranting a close by the bot, maybe needs to be double checked? your tag changes seems to be the last thing that happened and they don't look like a reason to autoclose?

[nagilson]

Yeah, that seems like a bug with the bot. Thanks for pointing it out, reopened this. Hopefully it does not try to close it again 🙄

[dotnet-policy-service]

Thanks for creating this issue! We believe this issue is related to NuGet tooling, which is maintained by the NuGet team. Thus, we closed this one and encourage you to raise this issue in the NuGet repository instead. Don’t forget to check out NuGet’s contributing guide before submitting an issue!

If you believe this issue was closed out of error, please comment to let us know.

Happy Coding!

[dotnet-policy-service]

Thanks for creating this issue! We believe this issue is related to NuGet tooling, which is maintained by the NuGet team. Thus, we closed this one and encourage you to raise this issue in the NuGet repository instead. Don’t forget to check out NuGet’s contributing guide before submitting an issue!

If you believe this issue was closed out of error, please comment to let us know.

Happy Coding!

[japj]

I think this issue is at the cross intersection between NuGet and Dotnet Sdk,
which I guess was also the reason why it was tagged as both Area-Nuget and Area-NetSDK.
I would not want this to end up being ping-ponged between different repos without an actual resolution.

[baronfel]

Removing the NuGet label since their bot is a bit zealous :)

[japj]

Thanks, don’t forget to re-open the issue

[znakeeye]

Visual Studio seems to re-evaluate project references only when solution file changes. Thus, there is no good place to put the switching logic. E.g. BeforeCompile would not suffice. The provided solution works in most cases. It's as close we can get, but sometimes VS will complain about solution file being locked etc...

I think we are in fact asking for a Visual Studio feature. Somewhere in the UI you should be able to select "Use project X instead of package X, if available". Ideally, this setting would then be propagated to the .sln or .csproj file, and it would just work.