azurerm_container_app_environment: Add support for workload_profile

[jiaweitao001]

• All related test passed.

TestAccContainerAppEnvironment_withWorkloadProfile
    === RUN   TestAccContainerAppEnvironment_withWorkloadProfile
    === PAUSE TestAccContainerAppEnvironment_withWorkloadProfile
    === CONT  TestAccContainerAppEnvironment_withWorkloadProfile
    --- PASS: TestAccContainerAppEnvironment_withWorkloadProfile (2083.54s)
    PASS

[pietersap]

Looking forward for the workload profiles support, thanks!

Is this a fix for azurerm_container_app_environment only? If yes, will there also be an accompanying PR to add the support to the azurerm_container_app resource, so we can deploy apps into the workload profile?

[jiaweitao001]

also be an accompanying PR to add the support to the azurerm_container_app resource, so we can deploy apps into the workload profile?

Hi @pietersap , there will be a separate PR for supporting workload profile in azurerm_container_app resource.

[tombuildsstuff]

hey @jiaweitao001

Thanks for this PR - I've left a comment inline, but by ignoring the presence of resources within the Resource Group, it appears that there's an issue that's not being surfaced?

Thanks!

[tombuildsstuff]

This indicates a bug in the Terraform Resource or Configuration being used, so this should (almost) never be used in test cases - why's this needed?

[jiaweitao001]

Hi @tombuildsstuff , when creating this container app environment resource, a NSG will be automatically created along with it. We won't be able to delete the RG without this line since there will be a hanging NSG.

[tombuildsstuff]

In which case, the API should be clearing that up during the deletion of the Container App Environment, since the API is creating it the API manages it's lifecycle (as other Resource Providers do) - would you mind opening an issue on Azure/azure-rest-api-specs / reaching out to the Service Team here so we can get this fixed?

[jiaweitao001]

This NSG was created to comply an Azure policy internally. It should not affect external users. Will remove this line.

[katbyte]

would this be more clear/explicit if

Suggested change

	* `maximum_count` - (Optional) The maximum number of containers that can be deployed in the Container App Environment.
	* `minimum_count` - (Optional) The minimum number of containers that can be deployed in the Container App Environment.
	* `maximum_container_count` - (Optional) The maximum number of containers that can be deployed in the Container App Environment.
	* `minimum_container_count` - (Optional) The minimum number of containers that can be deployed in the Container App Environment.

[jiaweitao001]

Sure. Will rename.

[davidkarlsen]

@katbyte @tombuildsstuff Can this be moved forward? Review-comments seem to have been met.
It's a very requested feature #22161 #21747
also - maybe those issues should not be marked as preview as feature is GA now.

[saarivirtajCGI]

Indeed, would be much appreciated! Been eagerly waiting for this to finally migrate back from AzApi resources.

[jackofallops]

Hi @jiaweitao001 - Thanks for this PR. I think it needs a little more design consideration and possibly some more information from the Service Team. The workload list appears to always contain a minimum of one workload specification (the Consumption item, added by default on creation) and can take multiple more of Dedicated types, specified by their SKUs in the property workloadProfileType. The docs suggest it's not possible to add additional Consumption profiles, so I think this needs to be reworked for the default Consumption profile and additional Dedicated Profiles? WDYT?

[jackofallops]

Can we set these to int, we need to maintain 32bit compatibility.

[jiaweitao001]

Changed.

[jackofallops]

This feels like there are missing Enums in the Spec, could you check with the service team?

[jiaweitao001]

According to the service team, this Consumption is the only possible value for Consumption block.

[jackofallops]

Can we have this new code in a new file? container_apps.go is already of significant siza and relates to the azurerm_container_app, so perhaps container_app_environment.go would be an appropriate separation?

[jiaweitao001]

Sure. Reorganized this part of code.

[jackofallops]

Can we have specific validation here? These I believe need to be specific values and casing based on the values that the API will accept (i.e. the allowed SKU names)

[jiaweitao001]

According to the service team, workload_profile_type does have a limited selection of SKUs, will specify the validation.

[davidkarlsen]

Is there anything to add here or is this complete to go?

[jackofallops]

Is there anything to add here or is this complete to go?

GH actions are only part of the process, the Acceptance Tests for this PR are currently queued on our CI server, assuming there's no issues highlighted there then I think this should be good to go.

[jackofallops]

Tests look good:

[mvanhil]

@jiaweitao001 Any reason why 'Consumption' is not added as a workload profile option?

Can we use this on the azure_container_app resource?

[bryan-brilivan]

@jiaweitao001 Any reason why 'Consumption' is not added as a workload profile option?

Can we use this on the azure_container_app resource?

If you read the discussion above, 'Consumption' is a default option and cannot be removed from the options and the focus is on adding a dedicated skus.

However I agree that we should be able to enable the workload profile without having to add dedicated skus.

Above setup can be done in the portal. I am thinking along the line of creating a flag that says enable_workflow_profile and making the workload_profile block optional.

[meizenga]

@jiaweitao001 Any reason why 'Consumption' is not added as a workload profile option?

Can we use this on the azure_container_app resource?

Agreed. Container App Environments run in a Consumption plan or in Workload Profile plan.
Within the Workload Profile you can either select your SKU's (like azurerm supports now) or run in the Consumption profile type/category.

Documentation of the workload profiles mentioning the Consumption profile type/category

Within AZ CLI you simply add the parameter --enable-workload-profiles and you can deploy in small subnet and still run in Consumption mode.

[mvanhil]

By allowing the "Consumption" option, you could instruct TF to just enable workload profiles, but without a dedicated compute.

[mvanhil]

I did some testing. When you deploy the following:

resource "azurerm_resource_group" "main" {
  name     = "rg-test"
  location = "westeurope"
}

resource "azurerm_virtual_network" "main" {
  name                = "vnet-test"
  resource_group_name = azurerm_resource_group.main.name
  location            = "westeurope"
  address_space       = ["10.50.50.0/24"]
}

resource "azurerm_subnet" "cae" {
  name                 = "subnet-test"
  resource_group_name  = azurerm_resource_group.main.name
  virtual_network_name = azurerm_virtual_network.main.name
  address_prefixes     = ["10.50.50.0/27"]

  delegation {
    name = "delegation"

    service_delegation {
      name    = "Microsoft.App/environments"
      actions = ["Microsoft.Network/virtualNetworks/subnets/join/action"]
    }
  }
}

resource "azurerm_container_app_environment" "main" {
  name                           = "cae-test"
  location                       = azurerm_resource_group.main.location
  resource_group_name            = azurerm_resource_group.main.name
  infrastructure_subnet_id       = azurerm_subnet.cae.id
  internal_load_balancer_enabled = true
  workload_profile {
    name                  = "caetest"
    workload_profile_type = "D4"
    minimum_count         = 0
    maximum_count         = 1
  }

  lifecycle {
    ignore_changes = [workload_profile]
  }
}

resource "azurerm_container_app" "example" {
  name                         = "aca-test"
  container_app_environment_id = azurerm_container_app_environment.main.id
  resource_group_name          = azurerm_resource_group.main.name
  revision_mode                = "Single"

  template {
    container {
      name   = "examplecontainerapp"
      image  = "mcr.microsoft.com/azuredocs/containerapps-helloworld:latest"
      cpu    = 0.25
      memory = "0.5Gi"
    }
  }
}

You get this:

Just delete the dedicated and you're good to go.
The example app is assigned to the Consumption profile.

[meizenga]

I had expected support for this:

workload_profile {
    name                  = "caetest"
    workload_profile_type = "Consumption"
  }

[HoussemDellai]

Great to see this feature added :)
Now we need to add the support to attach a container app azurerm_container_app into a workload profile !

[Mounsseff]

Hi, I get this issu:

(expected workload_profile.0.workload_profile_type to be one of ["D4" "D8" "D16" "D32" "E4" "E8" "E16" "E32"], got consumption )

we don't have the default consumption profile ?

[icornett]

Hi, I get this issu:

(expected workload_profile.0.workload_profile_type to be one of ["D4" "D8" "D16" "D32" "E4" "E8" "E16" "E32"], got consumption )

we don't have the default consumption profile ?

Consumption is enabled by default, you don't need to declare it

[aadje]

The ARM rest api does accept Consumption as a workload profile, which can be tested using the azapi_resource resource below. I think that adding Consumption to the ["D4" "D8" "D16" "D32" "E4" "E8" "E16" "E32"] validation list does make sense.

resource "azapi_resource" "container_app_env" {
  type      = "Microsoft.App/managedEnvironments@2023-05-01"
  name      = var.name
  location  = var.location
  parent_id = azurerm_resource_group.rg.id
  body = jsonencode({
    properties = {
      appLogsConfiguration = {
        destination = "log-analytics"
        logAnalyticsConfiguration = {
          customerId = var.workspace_id
          sharedKey  = var.workspace_key
        }
      }
      workloadProfiles = [
        {
          name                = "Consumption"
          workloadProfileType = "Consumption"
        }
      ]
    }
  })
}

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.