Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Apply source-build patches #58727

Merged
merged 17 commits into from
Mar 15, 2022
Merged

Apply source-build patches #58727

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
35d6ebc
lift version of Microsoft.CodeAnalysis.Common dependencies to previou…
lbussell Oct 21, 2021
a8ee96d
remove Microsoft.DotNet prebuilts
lbussell Oct 8, 2021
077fcca
Disable apphost build of 'csi', 'vbi' for source-build
dagood Oct 20, 2021
f3142bf
build a subset of roslyn projects
dseefeld Oct 22, 2021
f0d0e9a
change lifted package version names to refonly and set them if they a…
lbussell Jan 10, 2022
101eac5
remove unnecessary source-build conditions
lbussell Jan 10, 2022
912ab07
refactor CSharpSyntaxGenerator project to use Choose for targetframework
lbussell Jan 10, 2022
672e329
Revert "remove unnecessary source-build conditions"
lbussell Jan 12, 2022
8df52ca
Revert "build a subset of roslyn projects"
lbussell Jan 12, 2022
56dc22b
change RefOnly variable name prefixes back to SourceBuildLifted
lbussell Jan 20, 2022
3e570b5
update BuildBoss to allow new source build package version prefix
lbussell Jan 22, 2022
c91e00b
Merge branch 'main' into source-build-patches
lbussell Feb 7, 2022
b5bb06b
Merge branch 'main' into source-build-patches
lbussell Feb 15, 2022
35464c3
Merge branch 'main' into source-build-patches
lbussell Feb 19, 2022
caa111e
Revert changes for lifting some package versions for source-build
lbussell Feb 19, 2022
d3189c0
Merge branch 'main' into source-build-patches
lbussell Mar 3, 2022
ad01398
Merge branch 'main' into source-build-patches
lbussell Mar 10, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion eng/Tools.props
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<Project>

<ItemGroup>
<ItemGroup Condition="'$(DotNetBuildFromSource)' != 'true'">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ Why is Tools.props being included in sourcebuild? Can we exclude the file through some other means?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like it is included by Arcade: https://github.com/dotnet/arcade/blob/aa647dcfc48bd3a644625e549ff71b7d06ed0958/src/Microsoft.DotNet.Arcade.Sdk/tools/Tools.proj#L53

<!--
This package would normally be restored by the Arcade SDK, but it is not included during restore operations
if the -package flag is not also provided during the build. Roslyn separates the restore operation from the
Expand Down
26 changes: 15 additions & 11 deletions eng/Versions.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,17 +54,6 @@
<HumanizerCoreVersion>2.2.0</HumanizerCoreVersion>
<ICSharpCodeDecompilerVersion>7.1.0.6543</ICSharpCodeDecompilerVersion>
<MicrosoftBuildLocatorVersion>1.4.1</MicrosoftBuildLocatorVersion>
<!--
SourceBuild will requires that all dependencies also be source buildable. We are referencing a
version of MSBuild that is not SourceBuild compatible, which makes our build incompatible. Since we only
use these dependencies as reference assemblies, we can opt them out of this behavior by having their
version variable be prefixed with `RefOnly`. This will allow us to reference these libraries and remain
Source Build compatible.
-->
<RefOnlyMicrosoftBuildVersion>$(RefOnlyMicrosoftBuildPackagesVersion)</RefOnlyMicrosoftBuildVersion>
<RefOnlyMicrosoftBuildFrameworkVersion>$(RefOnlyMicrosoftBuildPackagesVersion)</RefOnlyMicrosoftBuildFrameworkVersion>
<RefOnlyMicrosoftBuildRuntimeVersion>$(RefOnlyMicrosoftBuildPackagesVersion)</RefOnlyMicrosoftBuildRuntimeVersion>
<RefOnlyMicrosoftBuildTasksCoreVersion>$(RefOnlyMicrosoftBuildPackagesVersion)</RefOnlyMicrosoftBuildTasksCoreVersion>
<NuGetVisualStudioContractsVersion>6.0.0-preview.0.15</NuGetVisualStudioContractsVersion>
<MicrosoftVisualStudioRpcContractsVersion>17.0.51</MicrosoftVisualStudioRpcContractsVersion>
<!--
Expand Down Expand Up @@ -274,6 +263,21 @@
<SystemCollectionsImmutableVersion>5.0.0</SystemCollectionsImmutableVersion>
<SystemReflectionMetadataVersion>5.0.0</SystemReflectionMetadataVersion>
<MicrosoftBclAsyncInterfacesVersion>6.0.0</MicrosoftBclAsyncInterfacesVersion>
<!--
SourceBuild will requires that all dependencies also be source buildable. We are referencing a
version of MSBuild that is not SourceBuild compatible, which makes our build incompatible. Since we only
use these dependencies as reference assemblies, we can opt them out of this behavior by having their
version variable be prefixed with `RefOnly`. This will allow us to reference these libraries and remain
Source Build compatible.
-->
<RefOnlyMicrosoftBuildVersion>$(RefOnlyMicrosoftBuildPackagesVersion)</RefOnlyMicrosoftBuildVersion>
<RefOnlyMicrosoftBuildFrameworkVersion>$(RefOnlyMicrosoftBuildPackagesVersion)</RefOnlyMicrosoftBuildFrameworkVersion>
<RefOnlyMicrosoftBuildRuntimeVersion>$(RefOnlyMicrosoftBuildPackagesVersion)</RefOnlyMicrosoftBuildRuntimeVersion>
<RefOnlyMicrosoftBuildTasksCoreVersion>$(RefOnlyMicrosoftBuildPackagesVersion)</RefOnlyMicrosoftBuildTasksCoreVersion>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ Is there a reason this needs to move to the bottom of the file?

<SourceBuildLiftedSystemCollectionsImmutableVersion Condition="'$(SourceBuildLiftedSystemCollectionsImmutableVersion)' == ''">$(SystemCollectionsImmutableVersion)</SourceBuildLiftedSystemCollectionsImmutableVersion>
<SourceBuildLiftedSystemReflectionMetadataVersion Condition="'$(SourceBuildLiftedSystemReflectionMetadataVersion)' == ''">$(SystemReflectionMetadataVersion)</SourceBuildLiftedSystemReflectionMetadataVersion>
<SourceBuildLiftedSystemRuntimeCompilerServicesUnsafeVersion Condition="'$(SourceBuildLiftedSystemRuntimeCompilerServicesUnsafeVersion)' == ''">$(SystemRuntimeCompilerServicesUnsafeVersion)</SourceBuildLiftedSystemRuntimeCompilerServicesUnsafeVersion>
<SourceBuildLiftedSystemTextEncodingCodePagesVersion Condition="'$(SourceBuildLiftedSystemTextEncodingCodePagesVersion)' == ''">$(SystemTextEncodingCodePagesVersion)</SourceBuildLiftedSystemTextEncodingCodePagesVersion>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ What is SourceBuildLifted prefix? Why do none of the existing naming approaches work? A comment should be added for future readers.

</PropertyGroup>
<PropertyGroup>
<UsingToolPdbConverter>true</UsingToolPdbConverter>
Expand Down
1 change: 1 addition & 0 deletions src/Interactive/csi/csi.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
<OutputType>Exe</OutputType>
<RootNamespace>CSharpInteractive</RootNamespace>
<TargetFrameworks>net6.0;net472</TargetFrameworks>
<UseAppHost Condition="'$(DotNetBuildFromSource)' == 'true'">false</UseAppHost>
</PropertyGroup>
<ItemGroup Label="Project References">
<ProjectReference Include="..\..\Compilers\Core\Portable\Microsoft.CodeAnalysis.csproj" />
Expand Down
1 change: 1 addition & 0 deletions src/Interactive/vbi/vbi.vbproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
<OutputType>Exe</OutputType>
<StartupObject>Sub Main</StartupObject>
<TargetFrameworks>net6.0;net472</TargetFrameworks>
<UseAppHost Condition="'$(DotNetBuildFromSource)' == 'true'">false</UseAppHost>
<RootNamespace></RootNamespace>
</PropertyGroup>
<ItemGroup Label="Project References">
Expand Down
1 change: 1 addition & 0 deletions src/Tools/BuildBoss/ProjectCheckerUtil.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,7 @@ private IEnumerable<string> GetAllowedPackageReferenceVersions(PackageReference
yield return $"$({name}Version)";
yield return $"$({name}FixedVersion)";
yield return $"$(RefOnly{name}Version)";
yield return $"$(SourceBuildLifted{name}Version)";
}
}

Expand Down
30 changes: 22 additions & 8 deletions ...s/Source/CompilerGeneratorTools/Source/CSharpSyntaxGenerator/CSharpSyntaxGenerator.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,27 @@
<!-- Make sure to reference the same version of Microsoft.CodeAnalysis.Analyzers as the rest of the build -->
<PackageReference Include="Microsoft.CodeAnalysis.Analyzers" Version="$(MicrosoftCodeAnalysisAnalyzersVersion)" PrivateAssets="all" />
</ItemGroup>
<ItemGroup>
<!-- If we're building the command line tool, we have to include some dependencies used for grammar generation -->
<Compile Include="..\..\..\..\..\Compilers\Core\Portable\Symbols\WellKnownMemberNames.cs" Link="Grammar\WellKnownMemberNames.cs" Condition="'$(TargetFramework)' != 'netstandard2.0'" />
<Compile Include="..\..\..\..\..\Compilers\CSharp\Portable\Declarations\DeclarationModifiers.cs" Link="Grammar\DeclarationModifiers.cs" Condition="'$(TargetFramework)' != 'netstandard2.0'" />
<Compile Include="..\..\..\..\..\Compilers\CSharp\Portable\Syntax\SyntaxKind.cs" Link="Grammar\SyntaxKind.cs" Condition="'$(TargetFramework)' != 'netstandard2.0'" />
<Compile Include="..\..\..\..\..\Compilers\CSharp\Portable\Syntax\SyntaxKindFacts.cs" Link="Grammar\SyntaxKindFacts.cs" Condition="'$(TargetFramework)' != 'netstandard2.0'" />

<PackageReference Include="Microsoft.CodeAnalysis.Common" Version="$(SourceGeneratorMicrosoftCodeAnalysisVersion)" PrivateAssets="all" Condition="'$(TargetFramework)' == 'netstandard2.0'" />
</ItemGroup>
<Choose>
<When Condition="'$(TargetFramework)' == 'netstandard2.0'">
<ItemGroup>
<PackageReference Include="Microsoft.CodeAnalysis.Common" Version="$(SourceGeneratorMicrosoftCodeAnalysisVersion)" PrivateAssets="all" />
</ItemGroup>
<ItemGroup Condition="'$(DotNetBuildFromSource)' == 'true'">
<PackageReference Include="System.Collections.Immutable" Version="$(SourceBuildLiftedSystemCollectionsImmutableVersion)" PrivateAssets="all" />
<PackageReference Include="System.Reflection.Metadata" Version="$(SourceBuildLiftedSystemReflectionMetadataVersion)" PrivateAssets="all" />
<PackageReference Include="System.Runtime.CompilerServices.Unsafe" Version="$(SourceBuildLiftedSystemRuntimeCompilerServicesUnsafeVersion)" PrivateAssets="all" />
<PackageReference Include="System.Text.Encoding.CodePages" Version="$(SourceBuildLiftedSystemTextEncodingCodePagesVersion)" PrivateAssets="all" />
</ItemGroup>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ Why do these need to be explicit? Aren't they inherited from Microsoft.CodeAnalysis.Common?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The packages are inherited from Microsoft.CodeAnalysis.Common. However, roslyn builds before runtime in the source-build tarball, so we don't have access to the "current" versions of the packages to build against. The SourceBuildLifted- versions point to the previously source-built versions of the packages (i.e. the last build's output) so that we don't try to restore versions of these packages we don't have. That is also why the RefOnly- prefix is not applicable here.

I have just realized that a better solution for this might be to add the packages to SBRP, so I will get working on that and hopefully this change won't be necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added the necessary packages to SBRP and verified that the changes to CSharpSyntaxGenerator.csproj will no longer be necessary in the context of a full source-build tarball build and bootstrap. I also reverted those changes in this PR.

</When>
<Otherwise>
<ItemGroup>
<!-- If we're building the command line tool, we have to include some dependencies used for grammar generation -->
<Compile Include="..\..\..\..\..\Compilers\Core\Portable\Symbols\WellKnownMemberNames.cs" Link="Grammar\WellKnownMemberNames.cs" />
<Compile Include="..\..\..\..\..\Compilers\CSharp\Portable\Declarations\DeclarationModifiers.cs" Link="Grammar\DeclarationModifiers.cs" />
<Compile Include="..\..\..\..\..\Compilers\CSharp\Portable\Syntax\SyntaxKind.cs" Link="Grammar\SyntaxKind.cs" />
<Compile Include="..\..\..\..\..\Compilers\CSharp\Portable\Syntax\SyntaxKindFacts.cs" Link="Grammar\SyntaxKindFacts.cs" />
</ItemGroup>
</Otherwise>
</Choose>
</Project>