Create dotnet-dockerhub-mirror container registry

[markwilkie]

From @eerhardt :

My ideal end goal would be for the .NET Team to have a single “dotnet-dockerhub-mirror” container registry that mirrored images from dockerhub. This registry could be used by tests, builds, or other automation that needed images hosted on dockerhub, but we can’t reference dockerhub anymore because of the rate limiting issue. This single registry would be owned and maintained by the core engineering team.

Ideally, that registry would also:

Periodically update new images from dockerhub when a new image was pushed.
For example, when a new image to the docker.io/library/redis:7.2 tag is pushed to dockerhub, it would become available in “dotnet-dockerhub-mirror”
Have a self-service mechanism for adding new image tags to be mirrored
That way the Aspire team could add a new image by themselves without needing to ask someone to do it.
Similar to the mirroring packages from nuget.org

For now, I have set up an Aspire specific container registry in an Aspire specific Azure subscription and manually populated it with what we need. And will need to manually update it as we need.

I can help set up the single “dotnet-dockerhub-mirror” container registry, but I can’t do it alone because I don’t know how to create Azure resources in the core engineering team’s service tree. And I probably don’t have permission.

And @MichaelSimons comments:

1. This service will require a Docker Hub subscription as it would be the single point that would pull images. The A&D team already has one Greg has setup. When the time comes we can add your bot to this.

2. The eventual goal would be to get the Artifact Cache feature in ACR to meet our needs. Currently this feature as no built in support to update tags that have been cached. For example if redit:7.2 is in the cache it will never update this image when updates to the images/tag are made.

3. We may want to explorer limiting access to the public anonymous ACR (e.g. Vnet/AzDO build agents).

4. This service will need a cleanup mechanism to remove out of date content to keep the operating costs to a minimum.

5. We may want to consider if we should enable s360 vulnerability scanning on this ACR. The point being there is value in detecting usage of insecure/out of date images within our systems even for testing.

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

[baronfel]

+1-ing this to signal that the .NET SDK team would like this functionality as well - our CI builds test Container Registry HTTP API compatibility and part of that includes pulling and running the registry:2 image from Docker Hub, which is subject to rate-limiting.

[joeloff]

+1 Multiple SDK PRs impacted before RC2 branding/snapshots.

[nagilson]

Any updates to this? This is still a big problem for .NET.

[nohwnd]

Please prioritize fixing this, I am hitting the problem on my PR here, dotnet/sdk#43874

[lewing]

Bumping for pain