[AVM Module Issue]: wait_for_rbac_before_contact_operations running before rbac added

[monty124]

Check for previous/existing GitHub issues

• I have checked for previous/existing GitHub issues

Issue Type?

Bug

(Optional) Module Version

0.9.1

(Optional) Correlation Id

No response

Description

using similar code to issue 169 (however in a much larger project), this is not always reproduceable on every terraform apply, however at times the wait is running before the rbac completes and the following error is thrown. A subsequent apply then works and resources are deployed successfully

module "azure_keyvault" {
  source = "Azure/avm-res-keyvault-vault/azurerm"
  enable_telemetry = false
  name = var.KeyVaultResourceName
  tenant_id = data.azurerm_client_config.existing.tenant_id
  resource_group_name = data.azurerm_resource_group.existing.name
  location = data.azurerm_resource_group.existing.location
  legacy_access_policies_enabled = false
  sku_name = "standard"
  network_acls = {
    ip_rules = var.AllowedIPs
    bypass = "AzureServices"
    default_action = "Allow"
  }
  tags = var.tags
  purge_protection_enabled = true
  soft_delete_retention_days = 90

  role_assignments =  local.RBACUsers    
  contacts = {
    "contact" ={
    email = var.CertificateContactEmail
    }
  }
 wait_for_rbac_before_contact_operations = {
    create = "120s"
  }

}

wait is executing before rbac: and error

 Enter a value: yes

module.azure_keyvault.time_sleep.wait_for_rbac_before_contact_operations[0]: Creating...
... Creating...
module.azure_keyvault.time_sleep.wait_for_rbac_before_contact_operations[0]: Still creating... [10s elapsed]
module.azure_keyvault.azurerm_key_vault.this: Still creating... [10s elapsed]
... Creating...
module.azure_keyvault.time_sleep.wait_for_rbac_before_contact_operations[0]: Still creating... [20s elapsed]
module.azure_keyvault.azurerm_key_vault.this: Still creating... [20s elapsed]
... Creating...
module.azure_keyvault.azurerm_key_vault.this: Still creating... [30s elapsed]
module.azure_keyvault.time_sleep.wait_for_rbac_before_contact_operations[0]: Still creating... [40s elapsed]
module.azure_keyvault.azurerm_key_vault.this: Still creating... [40s elapsed]
module.azure_keyvault.time_sleep.wait_for_rbac_before_contact_operations[0]: Still creating... [50s elapsed]
module.azure_keyvault.azurerm_key_vault.this: Still creating... [50s elapsed]
module.azure_keyvault.time_sleep.wait_for_rbac_before_contact_operations[0]: Still creating... [1m0s elapsed]
module.azure_keyvault.azurerm_key_vault.this: Still creating... [1m0s elapsed]
module.azure_keyvault.time_sleep.wait_for_rbac_before_contact_operations[0]: Still creating... [1m10s elapsed]
module.azure_keyvault.azurerm_key_vault.this: Still creating... [1m10s elapsed]
module.azure_keyvault.time_sleep.wait_for_rbac_before_contact_operations[0]: Still creating... [1m20s elapsed]
module.azure_keyvault.azurerm_key_vault.this: Still creating... [1m20s elapsed]
module.azure_keyvault.time_sleep.wait_for_rbac_before_contact_operations[0]: Still creating... [1m30s elapsed]
module.azure_keyvault.azurerm_key_vault.this: Still creating... [1m30s elapsed]
module.azure_keyvault.time_sleep.wait_for_rbac_before_contact_operations[0]: Still creating... [1m40s elapsed]
module.azure_keyvault.azurerm_key_vault.this: Still creating... [1m40s elapsed]
module.azure_keyvault.time_sleep.wait_for_rbac_before_contact_operations[0]: Still creating... [1m50s elapsed]
module.azure_keyvault.azurerm_key_vault.this: Still creating... [1m50s elapsed]
module.azure_keyvault.time_sleep.wait_for_rbac_before_contact_operations[0]: Creation complete after 2m0s [id=2024-10-02T04:35:39Z]
module.azure_keyvault.azurerm_key_vault.this: Still creating... [2m0s elapsed]
module.azure_keyvault.azurerm_key_vault.this: Still creating... [2m10s elapsed]
module.azure_keyvault.azurerm_key_vault.this: Creation complete after 2m10s 
module.azure_keyvault.azurerm_role_assignment.this["user3"]: Creating...
... Creating...
module.azure_keyvault.azurerm_key_vault_certificate_contacts.this[0]: Creating...
module.azure_keyvault.azurerm_role_assignment.this["user6"]: Creation complete after 23s 
module.azure_keyvault.azurerm_role_assignment.this["user4"]: Creation complete after 26s 
╷
│ Error: checking for presence of existing Certificate Contacts (Key Vault <redacted>): keyvault.BaseClient#GetCertificateContacts: 
Failure responding to request: StatusCode=4 Message="Caller is not authorized to perform action on resource.\r\nIf role assignments, 
deny assignments or role definitions were changed recently, please observe propagation time.\r\nCaller: <redacted>/\r\nAction: 'Microsoft.KeyVault/vaults/certificatecontacts/write'\r\n
Resource: '<redacted>: null\r\nDecisionReason: null \r\nVault:
<redacted>\r\n" InnerError={"code":"ForbiddenByRbac"}
│
│   with module.azure_keyvault.azurerm_key_vault_certificate_contacts.this[0],
│   on .terraform\modules\azure_keyvault\main.tf line 83, in resource "azurerm_key_vault_certificate_contacts" "this":
│   83: resource "azurerm_key_vault_certificate_contacts" "this" {
│
│ checking for presence of existing Certificate Contacts (Key Vault "<redacted>"): keyvault.BaseClient#GetCertificateContacts: Failure responding to request: StatusCode=403 -- O
│ Message="Caller is not authorized to perform action on resource.\r\nIf role assignments, deny assignments or role definitions were changed recently, please observe propagation time.\r\nCaller:
│ <redacted>;iss=<redacted>\r\nAction: '<redacted>
'\r\nAssignment: (not found)\r\nDenyAssignmentId: null\r\nDecisionRe
│ InnerError={"code":"ForbiddenByRbac"}
╵

adding the following depends_on = [ azurerm_role_assignment.this ] to resource "time_sleep" "wait_for_rbac_before_contact_operations" seems to reliably fix this issue (so far in my testing!)

resource "time_sleep" "wait_for_rbac_before_contact_operations" {
  count = length(var.contacts) != 0 ? 1 : 0

  create_duration  = var.wait_for_rbac_before_contact_operations.create
  destroy_duration = var.wait_for_rbac_before_contact_operations.destroy
  triggers = {
    contacts = jsonencode(var.contacts)
  }
  depends_on = [ azurerm_role_assignment.this ]
}

[monty124]

I've been able to confirm with adding depends_on = [ azurerm_role_assignment.this ] I've not had this issue re-occur

[jayaprasad-github]

I have the same issue with the keyvault RBAC while accessing the secrets but depends on as mentioned in the above comment may not working as is in our case as we are assigning the roles as part of Keyvault creation module but we have a wait time to let the roles replicate in the backend.

This was a working code but not we are impacted with the same issue.

module "key_vault" {
source = "Azure/avm-res-keyvault-vault/azurerm"
version = ">= 0.5.0"
name = "kv-dev-uks-01"
location = data.azurerm_resource_group.location
resource_group_name = data.azurerm_resource_group.name
tenant_id = data.azurerm_client_config.current.tenant_id
public_network_access_enabled = false
soft_delete_retention_days = 10
enabled_for_disk_encryption = true
wait_for_rbac_before_secret_operations = {
create = "60s"
}
wait_for_rbac_before_key_operations = {
create = "60s"
}
enabled_for_deployment = true
purge_protection_enabled = true
private_endpoints = {
primary = {
private_dns_zone_resource_ids = [data.azurerm_private_dns_zone.vault_dns.id]
subnet_resource_id = data.azurerm_subnet.id
}
}
network_acls = {
bypass = "AzureServices"
default_action = "Allow"

}

role_assignments = {
#give the deployment user access to secrets
deployment_user_secrets = {
role_definition_id_or_name = "Key Vault Secrets Officer"
principal_id = data.azurerm_client_config.current.object_id
principal_type = "ServicePrincipal"
skip_service_principal_aad_check = false
}
#give the deployment user access to keys
deployment_user_keys = {
role_definition_id_or_name = "Key Vault Crypto Officer"
principal_id = data.azurerm_client_config.current.object_id
principal_type = "ServicePrincipal"
skip_service_principal_aad_check = false
}
user_managed_identity_keys = { #give the user assigned managed identity for the disk encryption set access to keys
role_definition_id_or_name = "Key Vault Crypto Officer"
principal_id = azurerm_user_assigned_identity.linuxvm_identity.principal_id
principal_type = "ServicePrincipal"
}
user_managed_identity_keys = { #give the user assigned managed identity for the disk encryption set access to keys
role_definition_id_or_name = "Key Vault Secrets Officer"
principal_id = azurerm_user_assigned_identity.linuxvm_identity.principal_id
principal_type = "ServicePrincipal"
}
}
}