Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use beta containers package to do multi-architecture publishing #15

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Use beta containers package to do multi-architecture publishing #15

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4c0703e
hook up new package and sidestep target import problems
baronfel Sep 3, 2024
a136575
do multi-arch publishes
baronfel Sep 3, 2024
81039b1
fix tarball export example for single-rid publishing
baronfel Sep 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/actions/publishAndLog/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ runs:
shell: bash
if: contains(fromJson(inputs.selfContained), true) && contains(fromJson(inputs.chunked), false)
continue-on-error: true
run: dotnet publish -p PublishProfile=${{ inputs.profile }} --self-contained true -p ContainerImageTags='"${{ inputs.variant }}-sc-atomic;latest-sc-atomic"' /bl
run: dotnet publish -p PublishProfile=${{ inputs.profile }} --self-contained true -p ContainerImageTag='"${{ inputs.variant }}-sc-atomic"' /bl
working-directory: ${{ inputs.cwd }}
env:
SDK_CONTAINER_DEBUG_REGISTRY_FORCE_CHUNKED_UPLOAD: "false"
Expand All @@ -43,7 +43,7 @@ runs:
shell: bash
continue-on-error: true
if: contains(fromJson(inputs.selfContained), false) && contains(fromJson(inputs.chunked), false)
run: dotnet publish -p PublishProfile=${{ inputs.profile }} --self-contained false -p ContainerImageTags='"${{ inputs.variant }}-fdd-atomic;latest-fdd-atomic"' /bl
run: dotnet publish -p PublishProfile=${{ inputs.profile }} --self-contained false -p ContainerImageTag='"${{ inputs.variant }}-fdd-atomic"' /bl
working-directory: ${{ inputs.cwd }}
env:
SDK_CONTAINER_DEBUG_REGISTRY_FORCE_CHUNKED_UPLOAD: "false"
Expand All @@ -59,7 +59,7 @@ runs:
shell: bash
continue-on-error: true
if: contains(fromJson(inputs.selfContained), true) && contains(fromJson(inputs.chunked), true)
run: dotnet publish -p PublishProfile=${{ inputs.profile }} --self-contained true -p ContainerImageTags='"${{ inputs.variant }}-sc-chunked;latest-sc-chunked"' /bl
run: dotnet publish -p PublishProfile=${{ inputs.profile }} --self-contained true -p ContainerImageTag='"${{ inputs.variant }}-sc-chunked"' /bl
working-directory: ${{ inputs.cwd }}
env:
SDK_CONTAINER_DEBUG_REGISTRY_FORCE_CHUNKED_UPLOAD: "true"
Expand All @@ -75,7 +75,7 @@ runs:
shell: bash
continue-on-error: true
if: contains(fromJson(inputs.selfContained), false) && contains(fromJson(inputs.chunked), true)
run: dotnet publish -p PublishProfile=${{ inputs.profile }} --self-contained false -p ContainerImageTags='"${{ inputs.variant }}-fdd-chunked;latest-fdd-chunked' /bl
run: dotnet publish -p PublishProfile=${{ inputs.profile }} --self-contained false -p ContainerImageTag='"${{ inputs.variant }}-fdd-chunked' /bl
working-directory: ${{ inputs.cwd }}
env:
SDK_CONTAINER_DEBUG_REGISTRY_FORCE_CHUNKED_UPLOAD: "true"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/containerize.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -424,7 +424,7 @@ jobs:
working-directory: ./src/sdk-container-demo
run: |
mkdir images \
&& dotnet publish -p PublishProfile=DefaultContainer -p ContainerArchiveOutputPath=images /bl
&& dotnet publish -t:PublishContainer -p ContainerArchiveOutputPath=images /bl -r linux-x64
- name: List tarballs
run: ls -l ./src/sdk-container-demo/images
- name: Load image into Docker
Expand Down
11 changes: 11 additions & 0 deletions Directory.Build.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
<Project>
<Target Name="_CheckContainersPackage" />
<PropertyGroup>
<_ContainersTargetsDir>$(PkgMicrosoft_NET_Build_Containers)/build/</_ContainersTargetsDir>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.NET.Build.Containers" Version="8.0.403"
GeneratePathProperty="true" />
</ItemGroup>
</Project>
Binary file added pkg/Microsoft.NET.Build.Containers.8.0.403.nupkg
View file
Open in desktop
Binary file not shown.
120 changes: 5 additions & 115 deletions src/sdk-container-demo/sdk-container-demo.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,128 +8,18 @@
<SelfContained>false</SelfContained>
<RuntimeIdentifiers>linux-x64;linux-arm64</RuntimeIdentifiers>
<PublishRepositoryUrl>true</PublishRepositoryUrl>
<PublishSingleFile>false</PublishSingleFile>
<!-- Shouldn't need to set this - it's inferred for single-RID publishing,
but the multi-RID publish currently does not infer it-->
<ContainerRepository Condition="'$(RuntimeIdentifier)' == ''" >$(AssemblyName)</ContainerRepository>
</PropertyGroup>

<PropertyGroup Label="Packaging properties">
<Description>A project that demonstrates publishing to various container registries using just
the .NET SDK</Description>
<Description>A project that demonstrates publishing to various container registries using just the .NET SDK</Description>
<Authors>Chet Husk</Authors>
<PackageProjectUrl>https://github.com/baronfel/sdk-container-demo</PackageProjectUrl>
<PackageDocumentationUrl>https://github.com/baronfel/sdk-container-demo/blob/main/README.md</PackageDocumentationUrl>
<PackageLicenseExpression>MIT</PackageLicenseExpression>
<Title>.NET SDK 8 Container Demo</Title>
<ContainerImageTags>latest;v1.1</ContainerImageTags>
</PropertyGroup>

<!-- Example of how you could push the same image to multiple registries via a single target-->
<ItemGroup>
<DestinationRegistry Include="sdk-container-demo.azurecr.io" />
<DestinationRegistry Include="ghcr.io" />
<SingleRegistryPublish
Include="$(MSBuildProjectFullPath)"
AdditionalProperties="ContainerRegistry=%(DestinationRegistry)" />
</ItemGroup>

<Target
Name="MultiRidBuild">
<ItemGroup>
<_rids Include="$(RuntimeIdentifiers)" />
<_InnerBuild Include="$(MSBuildProjectFullPath)"
AdditionalProperties="RuntimeIdentifier=%(_rids.Identity)"
PropertiesToUnset="RuntimeIdentifiers" />
</ItemGroup>
<MSBuild
Projects="@(_InnerBuild)"
Targets="Build"
BuildInParallel="true" />
</Target>

<Target
Name="MultiPush"
DependsOnTargets="Publish">
<MSBuild
Projects="@(SingleRegistryPublish)"
Targets="PublishContainer"
BuildInParallel="true" />
</Target>

<!-- crazy example of the things the SDK should be able to do for us eventually -->
<Target Name="MultiArchPublish">
<ItemGroup>
<!-- first, define the container tags you want to create. these will be the roots of the arch-specific manifests - what users will mostly be using. -->
<_TagsToPublish Include="$(ContainerImageTags)" />

<!-- next, define the architectures you want to publish for. we could have just used RuntimeIdentifiers from above, but I wanted to use golang-style tag parts,
so that these images would look and feel like other container images. -->
<_RIDItems Include="linux-x64" GolangArch="amd64"/>
<_RIDItems Include="linux-arm64" GolangArch="arm64"/>
</ItemGroup>

<!-- Ideally at this point we'd call some target that would map .NET RIDs to Golang architecture items, but this doesn't currently exist -->
<!-- <MapRidsToGolangArch RIDs="@(_RIDItems)">
<Output TaskParameter="RidsWithArch" ItemName="_RIDItems" />
</MapRidsToGolangArch> -->

<ItemGroup>
<!-- this next part is kind of gross - we're trying to construct an MSBuild property value like 'ContainerImageTags=<tagbase>-<arch>;<tagbase>-<arch>'
this means we need a map of each specific arch to the list of tags we'd like to push for that arch. the easiest way I found to do that is
* create unique items that 'splat' the RIDs across each of the tags
* batch by the RIDs - thus capturing all of the tags in the ItemGroup for that batch. we can now transform that ItemGroup to make our final property string -->

<_RIDSForTag Include="@(_TagsToPublish)" RID="%(_RIDItems.Identity)" GolangArch="%(_RIDItems.GolangArch)" />
<_TagsForRID Include="%(_RIDSForTag.RID)" ContainerTags="@(_RIDSForTag->'%(Identity)-%(GolangArch)', ';')" />

<!-- now that we have our mappings, we can build this project for each RID, passing the full set of tags to create -->
<_ContainerPublish
Include="$(MSBuildProjectFullPath)"
AdditionalProperties="RuntimeIdentifier=%(_TagsForRID.Identity);ContainerImageTags=%(_TagsForRID.ContainerTags)" />
</ItemGroup>

<MSBuild
Projects="@(_ContainerPublish)"
Targets="Publish;PublishContainer"
BuildInParallel="true" />

<Message Text="Created architecture-specific images, now creating manifest list" Importance="High" />

<!--
If we pushed to an actual registry we can make a multi-arch image here. We need the image name and the tags to do so.
However, the PublishContainer targt doesn't actually emit the name or tags so we have to reconstruct it.
This is a gap, we should fill it in the SDK.

NOTE: We can only create the multi-arch manifest if we pushed to a registry. This is a fundamental constraint, so that condition has been applied to all of this code
-->
<!-- Call ComputeContainerConfig on this project so that the properties it sets are visible -->
<MSBuild Projects="$(MSBuildThisFile)" Targets="ComputeContainerConfig" />

<!-- To make a manifest we need to know the name of the manifest (e.g. mcr.microsoft.com/dotnet/sdk:8.0) and the names of all of the
architecture-specific images that go into that manifest (e.g. mcr.microsoft.com/dotnet/sdk:8.0-arm64, mcr.microsoft.com/dotnet/sdk:8.0-amd64, etc).
Again, the easiest way to do that is to make add the 'parent' information to some other pience of data that already contains the tag/arch data.
-->

<ItemGroup Condition="'$(ContainerRegistry)' != ''">
<!-- We'll need to group by 'tag' here to get the manifest names (the nice tags). To make that easier, let's add the parent manifest name and the
'final' container name onto each of our already-RID-aware Tag items. -->
<_RIDSForTag>
<FullImageName>$(ContainerRegistry)/$(ContainerRepository):%(Identity)-%(GolangArch)</FullImageName>
<ParentManifestName>$(ContainerRegistry)/$(ContainerRepository):%(Identity)</ParentManifestName>
</_RIDSForTag>
</ItemGroup>

<ItemGroup Condition="'$(ContainerRegistry)' == ''">
<!-- We'll need to group by 'tag' here to get the manifest names (the nice tags). To make that easier, let's add the parent manifest name and the
'final' container name onto each of our already-RID-aware Tag items. -->
<_RIDSForTag>
<FullImageName>$(ContainerRepository):%(Identity)-%(GolangArch)</FullImageName>
<ParentManifestName>$(ContainerRepository):%(Identity)</ParentManifestName>
</_RIDSForTag>
</ItemGroup>

<!-- armed with that data, we can easily batch these Task calls. '%(_RIDSForTag.ParentManifestName)' filters _RIDSForTag down to all items that have the same ParentManifestName.
this means that we can refer to @(_RIDSForTag) here as only the Items for the images for this manifest! From there it's easy to splat the FullImageName onto the CLI calls. -->
<Exec Command="podman manifest create %(_RIDSForTag.ParentManifestName) @(_RIDSForTag->'%(FullImageName)', ' ')" />
<Message Text="Created manifest list %(_RIDSForTag.ParentManifestName)" Importance="High" />
<Exec Command="podman manifest push %(_RIDSForTag.ParentManifestName)" Condition="'$(ContainerRegistry)' != ''" />
<Message Text="Pushed manifest list %(_RIDSForTag.ParentManifestName) to $(ContainerRegistry)" Condition="'$(ContainerRegistry)' != ''" Importance="High" />
</Target>
</Project>
Loading