Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

APIM additional_location's zones was not able to unset, caused error when using additional_location which does not support zones #17515

Closed
1 task done
edwardccg opened this issue Jul 5, 2022 · 5 comments · Fixed by #23943

Comments

@edwardccg
Copy link

edwardccg commented Jul 5, 2022

Is there an existing issue for this?

  • I have searched the existing issues

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

1.2.4

AzureRM Provider Version

3.4.0

Affected Resource(s)/Data Source(s)

azurerm_api_management

Terraform Configuration Files

provider "azurerm" {
  features {}
}

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "<= 3.4.0"
    }
  }
}

resource "azurerm_resource_group" "example" {
  name     = "ed-resources"
  location = "Canada Central"
}

locals {
  apim_name = "ed-apim-tf"
}

resource "azurerm_public_ip" "apim_zone_public_ip" {
  name                    = "${lower(local.apim_name)}-public-ip"
  resource_group_name     = azurerm_resource_group.example.name
  location                = azurerm_resource_group.example.location
  allocation_method       = "Static"
  sku                     = "Standard"
  sku_tier                = "Regional"
  zones                   = [1,2,3]
  idle_timeout_in_minutes = 4
  ip_version              = "IPv4"
  domain_name_label       = lower(local.apim_name)
}

resource "azurerm_api_management" "api_management" {
  name                = local.apim_name
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  publisher_name      = "My Company"
  publisher_email     = "company@terraform.io"

  sku_name                  = "Premium_2"
  zones                     = [1,2]
  public_ip_address_id      = azurerm_public_ip.apim_zone_public_ip.id

  additional_location {
    location = "canadaeast"
    zones = []
  }
}

Debug Output/Panic Output

# azurerm_api_management.api_management will be created
  + resource "azurerm_api_management" "api_management" {
      + client_certificate_enabled    = false
      + developer_portal_url          = (known after apply)
      + gateway_disabled              = false
      + gateway_regional_url          = (known after apply)
      + gateway_url                   = (known after apply)
      + id                            = (known after apply)
      + location                      = "canadacentral"
      + management_api_url            = (known after apply)
      + name                          = "ed-apim-tf"
      + notification_sender_email     = (known after apply)
      + policy                        = (known after apply)
      + portal_url                    = (known after apply)
      + private_ip_addresses          = (known after apply)
      + public_ip_address_id          = (known after apply)
      + public_ip_addresses           = (known after apply)
      + public_network_access_enabled = true
      + publisher_email               = "company@terraform.io"
      + publisher_name                = "My Company"
      + resource_group_name           = "ed-resources"
      + scm_url                       = (known after apply)
      + sku_name                      = "Premium_2"
      + virtual_network_type          = "None"
      + zones                         = [
          + "1",
          + "2",
        ]

      + additional_location {
          + capacity             = (known after apply)
          + gateway_regional_url = (known after apply)
          + location             = "canadaeast"
          + private_ip_addresses = (known after apply)
          + public_ip_addresses  = (known after apply)
        }

      + hostname_configuration {
          + developer_portal {
              + certificate                     = (sensitive value)
              + certificate_password            = (sensitive value)
              + expiry                          = (known after apply)
              + host_name                       = (known after apply)
              + key_vault_id                    = (known after apply)
              + negotiate_client_certificate    = (known after apply)
              + ssl_keyvault_identity_client_id = (known after apply)
              + subject                         = (known after apply)
              + thumbprint                      = (known after apply)
            }

          + management {
              + certificate                     = (sensitive value)
              + certificate_password            = (sensitive value)
              + expiry                          = (known after apply)
              + host_name                       = (known after apply)
              + key_vault_id                    = (known after apply)
              + negotiate_client_certificate    = (known after apply)
              + ssl_keyvault_identity_client_id = (known after apply)
              + subject                         = (known after apply)
              + thumbprint                      = (known after apply)
            }

          + portal {
              + certificate                     = (sensitive value)
              + certificate_password            = (sensitive value)
              + expiry                          = (known after apply)
              + host_name                       = (known after apply)
              + key_vault_id                    = (known after apply)
              + negotiate_client_certificate    = (known after apply)
              + ssl_keyvault_identity_client_id = (known after apply)
              + subject                         = (known after apply)
              + thumbprint                      = (known after apply)
            }

          + proxy {
              + certificate                     = (sensitive value)
              + certificate_password            = (sensitive value)
              + default_ssl_binding             = (known after apply)
              + expiry                          = (known after apply)
              + host_name                       = (known after apply)
              + key_vault_id                    = (known after apply)
              + negotiate_client_certificate    = (known after apply)
              + ssl_keyvault_identity_client_id = (known after apply)
              + subject                         = (known after apply)
              + thumbprint                      = (known after apply)
            }

          + scm {
              + certificate                     = (sensitive value)
              + certificate_password            = (sensitive value)
              + expiry                          = (known after apply)
              + host_name                       = (known after apply)
              + key_vault_id                    = (known after apply)
              + negotiate_client_certificate    = (known after apply)
              + ssl_keyvault_identity_client_id = (known after apply)
              + subject                         = (known after apply)
              + thumbprint                      = (known after apply)
            }
        }

      + protocols {
          + enable_http2 = (known after apply)
        }

      + security {
          + enable_backend_ssl30                                = (known after apply)
          + enable_backend_tls10                                = (known after apply)
          + enable_backend_tls11                                = (known after apply)
          + enable_frontend_ssl30                               = (known after apply)
          + enable_frontend_tls10                               = (known after apply)
          + enable_frontend_tls11                               = (known after apply)
          + tls_ecdhe_ecdsa_with_aes128_cbc_sha_ciphers_enabled = (known after apply)
          + tls_ecdhe_ecdsa_with_aes256_cbc_sha_ciphers_enabled = (known after apply)
          + tls_ecdhe_rsa_with_aes128_cbc_sha_ciphers_enabled   = (known after apply)
          + tls_ecdhe_rsa_with_aes256_cbc_sha_ciphers_enabled   = (known after apply)
          + tls_rsa_with_aes128_cbc_sha256_ciphers_enabled      = (known after apply)
          + tls_rsa_with_aes128_cbc_sha_ciphers_enabled         = (known after apply)
          + tls_rsa_with_aes128_gcm_sha256_ciphers_enabled      = (known after apply)
          + tls_rsa_with_aes256_cbc_sha256_ciphers_enabled      = (known after apply)
          + tls_rsa_with_aes256_cbc_sha_ciphers_enabled         = (known after apply)
          + triple_des_ciphers_enabled                          = (known after apply)
        }

      + sign_in {
          + enabled = (known after apply)
        }

      + sign_up {
          + enabled = (known after apply)

          + terms_of_service {
              + consent_required = (known after apply)
              + enabled          = (known after apply)
              + text             = (known after apply)
            }
        }

      + tenant_access {
          + enabled       = (known after apply)
          + primary_key   = (sensitive value)
          + secondary_key = (sensitive value)
          + tenant_id     = (known after apply)
        }
    }

  # azurerm_public_ip.apim_zone_public_ip will be created
  + resource "azurerm_public_ip" "apim_zone_public_ip" {
      + allocation_method       = "Static"
      + domain_name_label       = "ed-apim-tf"
      + fqdn                    = (known after apply)
      + id                      = (known after apply)
      + idle_timeout_in_minutes = 4
      + ip_address              = (known after apply)
      + ip_version              = "IPv4"
      + location                = "canadacentral"
      + name                    = "ed-apim-tf-public-ip"
      + resource_group_name     = "ed-resources"
      + sku                     = "Standard"
      + sku_tier                = "Regional"
      + zones                   = [
          + "1",
          + "2",
          + "3",
        ]
    }

  # azurerm_resource_group.example will be created
  + resource "azurerm_resource_group" "example" {
      + id       = (known after apply)
      + location = "canadacentral"
      + name     = "ed-resources"
    }

Plan: 3 to add, 0 to change, 0 to destroy.

Expected Behaviour

Should be able to create APIM primary instance with 2 unit, 2 zone, secondary instance with 2 unit with no zone.

Actual Behaviour

Error from the additional_location block, probably due to the zones is enable in primary. additional_location block is trying to enable using the zones configuration from the primary but since the additional location is canadaeast which do not have availability zone, it failed with following error. Passing in [] or null couldn't help to unset the zones because that seems to evaluate as 'omitted' instead of replacing the zone setup for the additional location.

azurerm_resource_group.example: Creating...
azurerm_resource_group.example: Creation complete after 1s [id=/subscriptions/f822c2e3-2bb4-47ee-a834-4039a63c3502/resourceGroups/ed-resources]
azurerm_public_ip.apim_zone_public_ip: Creating...
azurerm_public_ip.apim_zone_public_ip: Creation complete after 5s [id=/subscriptions/f822c2e3-2bb4-47ee-a834-4039a63c3502/resourceGroups/ed-resources/providers/Microsoft.Network/publicIPAddresses/ed-apim-tf-public-ip]
azurerm_api_management.api_management: Creating...
╷
│ Error: creating/updating Api Management: (Service Name "ed-apim-tf" / Resource Group "ed-resources"): apimanagement.ServiceClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="NotSupported" Message="Subscription `f822c2e3-2bb4-47ee-a834-4039a63c3502` is not enabled for Availability Zone deployment in location `Canada East`. Supported Locations are `AUSTRALIAEAST,BRAZILSOUTH,CANADACENTRAL,CENTRALINDIA,CENTRALUS,CENTRALUSEUAP,EASTASIA,EASTUS,EASTUS2,EASTUS2EUAP,FRANCECENTRAL,GERMANYWESTCENTRAL,JAPANEAST,KOREACENTRAL,NORTHCENTRALUS,NORTHEUROPE,NORWAYEAST,QATARCENTRAL,SOUTHAFRICANORTH,SOUTHCENTRALUS,SOUTHEASTASIA,SWEDENCENTRAL,SWITZERLANDNORTH,UAENORTH,UKSOUTH,WESTEUROPE,WESTUS2,WESTUS3,ISRAELCENTRAL,ITALYNORTH,POLANDCENTRAL`."
│ 
│   with azurerm_api_management.api_management,
│   on main_issue.tf line 36, in resource "azurerm_api_management" "api_management":
│   36: resource "azurerm_api_management" "api_management" {
│ 
╵

Steps to Reproduce

terraform apply

Important Factoids

No response

References

No response

@edwardccg edwardccg added the bug label Jul 5, 2022
@github-actions github-actions bot removed the bug label Jul 5, 2022
@edwardccg
Copy link
Author

edwardccg commented Jul 7, 2022

Workaround solution in case anyone need this urgently.

In summary, we will have to

  1. create CAE location via az rest command, detail refer to below.
  2. After it’s created, add the additional_location block back into terraform so that code will match the Azure infra
  3. Run “preview” to make sure no changes so that terraform will update their statefile

To add location manually:

  1. Create a json (eg: “apim-cae-location.json”) with following contents
    NOTE: zones: [] is respected by the azurerm api here
{
  "properties": {
    "additionalLocations": [
      {
        "location": "canadaeast",
        "platformVersion": "stv2",
        "sku": {
          "name": "Premium",
          "capacity": 2
        },
        "zones": [],
        "publicIpAddressId": null,
        "virtualNetworkConfiguration": {
          "subnetResourceId": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/Microsoft.Network/virtualNetworks/{VNETName}/subnets/{SubnetName}"
        }
      }
    ]
  }
}
  1. After login with az cli with SPN, run following command to add the location manually.
az rest --method PATCH --header "Accept=application/json" \                                                                                                                                                     ─╯
--url 'https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.ApiManagement/service/{APIMInstanceName}?api-version=2021-08-01' \
--body @apim-cae-location.json
  1. Once the APIM instance is updated, the location will be added without problem

@edwardccg edwardccg changed the title APIM additional_location's zones was not able to unset, caused error when using location which does not support zones APIM additional_location's zones was not able to unset, caused error when using additional_location which does not support zones Jul 7, 2022
@AdamCoulterOz
Copy link
Contributor

AdamCoulterOz commented Feb 7, 2023

@manicminer can you please add the service/api-management tag to this? :-)

@rachelhan1024
Copy link

rachelhan1024 commented Feb 14, 2023

Hi @AdamCoulterOz Thanks for looking after this issue. We are building a premium APIM on Australia East with an additional location of Australia Southeast. This bug is blocking us from creating the additional location, as well as any following updates.

Please help to fix the issue. It is muchly appreciated.
Thank you

@domroutley
Copy link
Contributor

I am having the same issue, with UK South/UK West and Australia East/Australia Southeast.

It appears that the function that constructs the additional locations body elements to send to the API does not reference the additional location block argument of zones, but references the "root" or primary location zones argument.

I have raised #23943 that fixes this, and adds an acceptance test that fails without the change I have made.

@github-actions github-actions bot added this to the v3.82.0 milestone Nov 24, 2023
Copy link

github-actions bot commented May 1, 2024

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
6 participants