Using Bicep to deploy managed certificates with custom domains requires multiple runs

[johnnyreilly]

Please provide us with the following information:

This issue is a: (mark with an x)

• bug report -> please search issues before submitting
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Issue description

See discussion here: #607 (comment)

Azure Container Apps support managed certificates and custom domains. However, deploying them with Bicep is not straightforward. The following bicep results in an error message like this:

Creating managed certificate requires hostname '....' added as a custom hostname to a container app in environment 'caenv-appname-dev'

resource managedEnvironmentManagedCertificate 'Microsoft.App/managedEnvironments/managedCertificates@2022-11-01-preview' = {
  parent: managedEnvironment
  name: '${managedEnvironment.name}-certificate'
  location: location
  tags: tags
  properties: {
    subjectName: customDomainName
    domainControlValidation: 'CNAME'
  }
}

resource containerApp 'Microsoft.App/containerApps@2022-11-01-preview' = {
  //...
  properties: {
    configuration: {
      //...
      ingress: {
        //...
        customDomains: [
          {
            name: managedEnvironmentManagedCertificate.properties.subjectName
            certificateId: managedEnvironmentManagedCertificate.id
            bindingType: 'SniEnabled'
          }
        ]
        //...
      }
      //...
    }
    //...
  }
  //...
}

Steps to reproduce

Deploy an Azure Container App using Bicep with the above template additions

Expected behavior [What you expected to happen.]
It should deploy successfully.

Actual behavior [What actually happened.]

Creating managed certificate requires hostname '....' added as a custom hostname to a container app in environment 'caenv-appname-dev'

Screenshots
n/a

Additional context

I've written up the workaround way of deploying with Bicep here: https://johnnyreilly.com/azure-container-apps-bicep-managed-certificates-custom-domains

A potential fix is suggested here: #607 (comment) by @mdrakiburrahman

[mdrakiburrahman]

@dariagrigoriu - I've tested this extensively on 100+ Azure Container App envs - here's a summary of what the bugs are, and how a dev can easily fix it:

#607 (comment)

[johnnyreilly]

See update here from @vinisoto: #607 (comment)

We have fixed an issue that caused the step of creating the managed certificate to take a long time and then fail (although the certificate was indeed successfully created.

With this fix, the number of deployment steps is reduced from 3 to 2. We are designing ways to be able to accomplish the whole process in a single deployment template (e.g., removing the need of creating a container app before creating a managed certificate for certain type of domain ownership validation). We don't have an ETA to share at this time.

[KSemenenko]

there is no updates right?

[nikneem]

I learned that the team is aiming for a solution, but I'm not sure if any of it is testable or in preview yet. @anthonychu may have more info on this.

[groogiam]

I have not event been able to get this working with multiple runs. Even when using this solution https://johnnyreilly.com/azure-container-apps-bicep-managed-certificates-custom-domains I get a domain validation error when creating a custom domain even when it is set to disabled. It does not seem like there is anyway to deploy with bicep using managed certificates.

[nikneem]

That is why this issue is still open. It is not solved yet... I have a workaround that works for me but is not a solution I would use in production...

• Deploy your infra, including custom domain names, but leave out the managed certs here
• Again, deploy your infra, but now (since the domain names have been registered, including the domain names)
• Yet again deploy your infra, but now with hostnames that use the managed certs

If you did the three deployments (thus using different configurations) you can now deploy your app over and over again using only the third option of the list above.

Hope this helps for now, while waiting for a more solid solution ;)

[groogiam]

@nikneem Thanks for the info. I ran into a few issues with your procedure. See below for how I worked around. Thanks.

That is why this issue is still open. It is not solved yet... I have a workaround that works for me but is not a solution I would use in production...

• Deploy your infra, including custom domain names, but leave out the managed certs here

This step does not work. It gives you a domain validation error if even if you have the custom domains disabled. The problem is you need the verification id which is generated from the container app. So this step should be create the custom domain without the certs or any custom domains configured on the app.

• Again, deploy your infra, but now (since the domain names have been registered, including the domain names)

This should be add your TXT records to your DNS then deploy with domains disabled and no certs. You will likely need to go to the azure portal to get the validation records although they seem pretty standard so you only really need the domain validation id.

• Yet again deploy your infra, but now with hostnames that use the managed certs

If you did the three deployments (thus using different configurations) you can now deploy your app over and over again using only the third option of the list above.

Hope this helps for now, while waiting for a more solid solution ;)

[aaltotsky]

The solution involving multiple runs is not suitable for my situation. I am unable to disable the certificate on the working server through a redeployment. Is there a method to obtain a list of certificates? It is feasible to manually include them and utilize them in the Bicep script, but how can I determine which certificate corresponds to which?

[trylvis]

Any updates to this?

[JimGeersinga]

Still no update?

[KSemenenko]

Still no update?

dotnet/aspire#1498 (comment)

[rpetersson]

Update? Struggeling with the same issue. Currently click-opsing the certificates and refering to them from the iac code.

[KSemenenko]

We shut drop containers app. Now we use kubernetes air web apps.

[MrDeej]

Mine works after

azd config set alpha.aca.persistDomains on

[nikneem]

Is this alpha.aca.persistDomains on documented somewhere?

[edit]
Oh I see where this comes from... But still this does not help with initial deployments
[/edit]

[ryanheath]

azd config set alpha.aca.persistDomains on

fails deploying a container that does not use custom domain.

only one container is using a custom domain.

[adampaquette]

Updates?

[itpropro]

Any updates on this? We have a requirement for creating multiple container apps with certificates. This means the following steps are necessary:

1. Add a custom domain
   a. Add CNAME and TXT records
2. Add a managed certificate for that domain
   a. Custom domain with SNI needs to be configured on the Container App resource

As you can see, this is a circular dependency, as the outputs of the Container App are needed as an input in the DNS entries, which creates a situation, where we need at least 3 runs of the same Bicep template including deployment scripts to do the following

1. Check if it's the first run and leave the customDomain property empty
2. Check if it's the second run, create the records in the DNS Zone based on the required Container App outputs (customDomainVerificationId, fqdn) and add a custom domain with bindingType: 'Disabled'
3. Check, if the DNS records and the custom domain property exists, update the binding type to bindingType: 'SniEnabled' and create the managed certificate

The only way to fix this from my perspective would be to have customDomains as its own sub resource, so it can be deployed independently of the Container App. It's also needed to be able to create a customDomain configuration that can directly be used to create a managed certificate based on that afterwards.

As this issue is already over a year old, it would be great to get a timeline on when the implementation will be fixed so that it is possible to deploy a container app with custom domains (and certificates) in a single deployment.

[vRune4]

We are about to push our old on-premise production environment into Azure.

I've started testing custom domains thinking that this issue does not apply to me since I only have a handful custom domains. Because I can do this manually, right?

Which worked well until the next deployment: My manually added custom domain was, afaict, purged by my latest IaC deployment. 🐼

I'll probably have to do the triple deployment dance too. Having split my IaC into several parts (one for common/shared resources like the mgd env and then each service has its own IaC project with unique resources such as containerApps) I am a wee bit worried I won't be able to pull it off.

[trylvis]

Any updates on this? @vinisoto

[vinisoto]

Starting mid-January 2025, we will start rolling out a new bindingType to the customDomains property of the containerApp resource:

bindingType: 'auto'

What this means is that the platform will use a managed certificate that matches the custom domain of your app and will populate the manage certificate id without requiring an update to the container app.

This is an example of how the container app bicep template will potentially look like (final property names/values might change before January 2025). Be aware that this feature will be introduced as Public Preview

resource containerApp 'Microsoft.App/containerApps@2024-10-02-preview' = {
  name: containerAppName
  location: location
  properties: {
    managedEnvironmentId: resourceId('Microsoft.App/managedEnvironments', managedEnvName)
    configuration: {
      ingress: {
        external: true
        targetPort: 80
        allowInsecure: false
        traffic: [
          {
            latestRevision: true
            weight: 100
          }
        ]
        customDomains: [{
          name: customDomain
          bindingType: 'auto'
        }]
      }
    }
    template: {
      containers: [
        {
          name: containerAppName
          image: 'mcr.microsoft.com/k8se/quickstart:latest'
          resources: {
            cpu: json('1.0')
            memory: '2Gi'
          }
        }
      ]
      scale: {
        minReplicas: 0
        maxReplicas: 3
      }
    }
  }
}

With this, you can deploy the managed certificate and the container app in the same bicep template. Once the managed certificate domain verification process successfully completes, the platform will update your container app with the managed certificate

[nikneem]

Alrighty then, that looks like a great solution for the majority of users. I do think that there is a small group of users that really want to use their own cert, but for now I'm pretty sure you helped the majority of users here. Can we sign up for a (private) preview somewhere?

[rpetersson]

Great work! This is awesome!

[KSemenenko]

So good news!

[johnnyreilly]

Thanks - this is wonderful news!

[vRune4]

What this means is that the platform will use a managed certificate that matches the custom domain of your app and will populate the manage certificate id without requiring an update to the container app.

But will it allocate a fresh certificate?

My scenario:

• DNS updated with asuid.mycontainer IN TXT <mgd env's asuid>
• DNS updated with mycontainer IN A <mgd env's IP>
• No cert yet for mycontainer
• Redeploying mycontainer with bindingtype set to 'auto'

The DNS validation should pass, but will it start the certification request and (eventually) bind it?

On a related note: Pre-allocating a certificate. I've seen this process take 20+ minutes. Would be neat to kick it off before diverting my IN A DNS record (why would I point that at something that is incapable of replying for 20 minutes?). I know that going the "other way" works (I can divert IN A back and still keep my cert...), so it feels like the IN A record validation should not be a hard fail.

[KoblerS]

Starting mid-January 2025, we will start rolling out a new bindingType to the customDomains property of the containerApp resource:

bindingType: 'auto'

What this means is that the platform will use a managed certificate that matches the custom domain of your app and will populate the manage certificate id without requiring an update to the container app.

This is an example of how the container app bicep template will potentially look like (final property names/values might change before January 2025). Be aware that this feature will be introduced as Public Preview

resource containerApp 'Microsoft.App/containerApps@2024-10-02-preview' = {
  name: containerAppName
  location: location
  properties: {
    managedEnvironmentId: resourceId('Microsoft.App/managedEnvironments', managedEnvName)
    configuration: {
      ingress: {
        external: true
        targetPort: 80
        allowInsecure: false
        traffic: [
          {
            latestRevision: true
            weight: 100
          }
        ]
        customDomains: [{
          name: customDomain
          bindingType: 'auto'
        }]
      }
    }
    template: {
      containers: [
        {
          name: containerAppName
          image: 'mcr.microsoft.com/k8se/quickstart:latest'
          resources: {
            cpu: json('1.0')
            memory: '2Gi'
          }
        }
      ]
      scale: {
        minReplicas: 0
        maxReplicas: 3
      }
    }
  }
}

With this, you can deploy the managed certificate and the container app in the same bicep template. Once the managed certificate domain verification process successfully completes, the platform will update your container app with the managed certificate

This is great news!

Question, right now the property managedEnvironmentId is marked as deprecated, will this property still work in the new version since you mentioned it in your example?

[itpropro]

Starting mid-January 2025, we will start rolling out a new bindingType to the customDomains property of the containerApp resource:

bindingType: 'auto'

What this means is that the platform will use a managed certificate that matches the custom domain of your app and will populate the manage certificate id without requiring an update to the container app.

This is an example of how the container app bicep template will potentially look like (final property names/values might change before January 2025). Be aware that this feature will be introduced as Public Preview

resource containerApp 'Microsoft.App/containerApps@2024-10-02-preview' = {
  name: containerAppName
  location: location
  properties: {
    managedEnvironmentId: resourceId('Microsoft.App/managedEnvironments', managedEnvName)
    configuration: {
      ingress: {
        external: true
        targetPort: 80
        allowInsecure: false
        traffic: [
          {
            latestRevision: true
            weight: 100
          }
        ]
        customDomains: [{
          name: customDomain
          bindingType: 'auto'
        }]
      }
    }
    template: {
      containers: [
        {
          name: containerAppName
          image: 'mcr.microsoft.com/k8se/quickstart:latest'
          resources: {
            cpu: json('1.0')
            memory: '2Gi'
          }
        }
      ]
      scale: {
        minReplicas: 0
        maxReplicas: 3
      }
    }
  }
}

With this, you can deploy the managed certificate and the container app in the same bicep template. Once the managed certificate domain verification process successfully completes, the platform will update your container app with the managed certificate

Thanks for the update, just to clarify, does this mean that it will be possible to deploy everything in one step, so that the customDomainVerificationId and fqdn output fields of the container app resource can directly be used in the DNS zone resource and the certificate in the container app will automatically update?