The Azure Container Registry (ACR) is central to image and artifact management within Azure.
ACR provides:
- Network-close registry access, providing the fastest and most reliable storage of images, close to your Azure deployments. For more info: Choosing a Docker Container Registry
- Integrated security to your Azure Active Directory accounts, enabling your company to control access to your collection of images
- Geo-replication, provides single registry/image/tag names, that can be used across multiple regions
- VNet and Firewall Rules, securing registries from public endpoints.
- Tasks for building, patching, testing container workloads
- Scheduled Tasks for automating tasks, such as automatically purging older artifacts
- Webhook Notifications for event based notifications of content pushed and/or deleted
- OS & Framework Patching, using ACR Tasks
- Helm Chart support, as the beginning or OCI Artifact support.
- Quarantine Pattern (private preview), providing secure by default registries
- Anonymous Pulls (preview), - aka public registries.
- Content Trust for assuring the delivery of an image from ACR to its destination.
To provide insight to our backlog, we wanted to provide the high level list of experiences we're enabling. While not a complete list, it should provide some insight to what's coming, what's a bit further out and the thought process behind our prioritization.
Containers have grown exponentially over the last few years. As a result, the ecosystem continues to grow. There are many great partners doing innovative and required work. Our goals for ACR is to work with, not compete with registry partners. We believe ACR can provide the best base experience in Azure, as ACR can integrate core capabilities required to run in Azure. While partners can build atop the OCI Distribution Spec.
As a result, we're working with partners to integrate with ACR, providing any necessary features they need to provide their additional capabilities. As an example, Web Hook notifications allow vulnerability scanning partners to respond to events, rather than scheduled jobs for detecting new/updated images requiring scanning. VNet & Firewall Rules enables core capabilities of Azure, other registries would be unable to provide.
As we look to our backlog, there's a long list of scenarios we're working to enable. They break down into 5 categories:
- Operational Health & Support
- Must Haves to enable basic registry access
- Must Haves, but possibly with partner integration
- Primitives that must be done within the core registry
- Innovative/differentiating features that require core registry changes
As a result, there are a set of features we know customers have come to expect from a registry, such as image vulnerability scanning and we are working to enable those, even if just links into the Azure Marketplace. While there are a set of features that require deeper integration, such as az acr login -r [yourregistry] enabling your individual Azure Identity to access your your collection of images, VNet & Firewall Rules or core performance work.
The following items are in our backlog. Some are in active development. The majority of our backlog items source from users, via uservoice.
- Vulnerability Scanning Updates
- Restrict Endpoint Access /VNet Support
- When will ACR VNet support GA
- Tokens & Repository Scoped Permissions (RBAC)
- Diagnostics & Monitoring
- Health Checks
- Auto Purge
- ACR Helm GA
- Performance & Scalability
- Container Build Caching with ACR Tasks and BuildX
- Helping with Prioritization
We've heard customers tell us that vulnerability scanning is table stakes for container registries. We agree. With partners like Aqua and TwistLock, customers can get the must-haves complete, today. ACR provides launch points to the Azure Marketplace, helping customers integrate vulnerability scanning.
Starting fall '19, Azure Security Center will start offering registry scanning and runtime protection. As with any new offering, it will take time for us to dial in the experiences.
We believe customers should have a choice for their security scanning solutions and will continue to provide extensibility, just as users can choose the security scanning of their users machines and their servers.
Customers have asked for more granular IP restrictions to their registry, in addition to authentication. As a shared registry API, this does present some challenges that we're incrementally addressing.
In March of 2019 we added preview support for VNet and Firewall rules through Service Endpoints. This allows customers to restrict their acr endpoint to requests from within the defined Azure VNet, or any allowed ip ranges.
The Service Endpoint approach to VNets puts firewall rules on the public endpoint, which limits the inbound traffic to network packets from reources defined as inside the VNet. However, the traffic does travel across the public network to be ingested.
Private Links is the next iteration of VNet & Firewall rules. With private endpoints, customers can assign private IPs to their resources, including ACR. This means that traffic from the VNet is not routed across the public network.
We plan to GA ACR VNet & Firewall rules when we complete the private link enhancements, and we add a new consumption pricing tier.
For container hosts that are not within the customers Azure VNet, such as multi-tenant services like Azure ML, ACI, Azure DevOps hosted build agents and ACR Tasks, the host is unable to pull the image. The host can only pull the image if it's in the VNet. We refer to this as the bootstrapping problem.
We are actively working with service teams to identify a secure solution that adherers to the promise that a customers VNet is secured to only the resources they place in that VNet.
When customers ask for GA support, they are typically asking when we'll be able to support them with their production workloads. ACR will support any customer by opening a support ticket. Although ACR VNet & Firewall rules are not yet GA, we will provide full support by opening a ticket: https://aka.ms/acr/support/create-ticket
One of the top requests has been UserVoice: Configure permissions at a repository level. We also have frequent requests for token based permissions, as service principals can provide access to other resources in Azure that customers may not be comfortable with. There's also a max number Service Principals and Managed Identities that can be provisioned within a subscription, and customers have faced challenges with AAD timeouts.
As customers use containers, and other artifacts, for their IoT deployment, the number of devices can grow to the millions.
To support the scale of IoT, ACR is implementing repository based RBAC, using tokens. This feature is under development, with a hopeful preview in the fall of 2019. In a subsequent release, tokens can be backed by service principals and AAD users, enabling RBAC with SPs, Managed Identities and Users.
Customers looking to troubleshoot why their push/pull operations are either failing or are too slow wind up contacting Azure Support, just to troubleshoot basic information. In the fall of '19, ACR will be shipping Diagnostics and Logs support.
Customers looking to test their connectivity to their ACR can use az acr health-check
As registries are filled with automated image builds, they wind up filling with tags, manifests and layers that never get used, or were deployed for a period of time, but no longer. Auto-purge-spec outlines how ACR will track image usage and move unused layers to a recycle bin, allowing subsequent purging. The feature will be configured and managed, with reasonable defaults, assuring you'll never lose anything you really wanted to keep.
As a step along this automated journey, ACR Scheduled Tasks and a new ACR image (mcr.microsoft.com/acr/acr-cli:0.1) can be used to automate image deletion.
Q: When will ACR Helm Support GA?
With the collaboration of Josh Dolitsky, of Chart Museum, we added Helm 2 support to ACR - az acr helm. As we watched the industry look for Yet Another Storage Solution (YASS) for new types like Singularity, CNAB and other new types, we realized we needed a more generalized solution. With help from people like Stephen Day and the OCI community, we focused on creating a generalized Artifacts store. In August of '19, the OCI TOB adopted OCI Artifacts.
With OCI Artifacts in place, we are working with the Helm Community to add Helm Registry support, allowing customers to use helm registry push|pull| across all OCI compliant Registries. As Helm 3 Registry support releases, and the community moves to Helm 3, our plan is to deprecate the Helm 2 acr support for, what we believe is a more native Helm 3 experience. We will continue to support az acr helm support for a period of transition to Helm 3. All Helm charts pushed to ACR with az acr helm will be fully compatible with Helm 3 registry support. Customers using ACR Helm should be comfortable with a smooth transition to Helm 3 and continue with az acr helm in the meantime.
As customers move from manual deployments to automation, we've seen a dramatic increase in usage. Some customers care using utilities like Watch Tower to automate docker pull, keeping your deployments up to date, while others are simply doing massive scaling.
When issuing docker pull a manifest is returned, listing the layers required. If the local host already has the layers, no additional requests are made. However, the registry must still respond with the manifest. ACR introduced a caching layer to cache & return manifest without having to hit storage. We should mention that an alternative approach would be to use Web Hooks to know when an image has been updated, rather than simply asking the registry: "do you have anything new, do you have anything new, do you have anything new...".
We've also been working with a few early adopter customers to scale thousands of nodes, each pulling large images. We recently scaled 1,000 nodes in 10 minutes with 11 terabytes being deployed. This performance work has been moved into the Premium SKU to enable high scale customers a solution to their larger needs. We continue to focus on concurrent throughput and individual pull performance scenarios.
When building locally, docker build benefits from image caching. This works as the machine is dedicated to the user, along with its disk. As users leverage cloud builds, they want the same build cache performance, without having to pay for dedicated compute, just waiting for builds. ACR released BuildX integration with ACR Tasks as a preview to first class integration. We hope to have something as simple as --cache as a parameter to tasks, soon.
Have a request, or wish we were doing something; Please provide your feedback and ranking to help us understand your needs and priority through UserVoice.