Azure Log Analytics Failure: Terraform timeout waiting for 'Propagated' to return from API

[archmangler]

Terraform (and AzureRM Provider) Version

Terraform v0.11.11

Affected Resource(s)

• azurerm_log_analytics_workspace

Terraform Configuration Files

resource "azurerm_log_analytics_workspace" "oms" {
  name                     = "ommsecuritycenter"
  resource_group_name      = "${azurerm_resource_group.oms.name}"
  location                 = "${azurerm_resource_group.oms.location}"
  sku                      = "PerGB2018"
  retention_in_days        = "${var.retention_in_days}"
}

Debug Output

https://gist.github.com/archmangler/56aa86578c5f68df3ba5069df9323273

Panic Output

n/a

Expected Behavior

• The terraform azurerm provider should correct detect completion of the oms workspace deployment and mark it as successful.

Actual Behavior

• Instead it waits for "Propagated" to be returned by the API, until the 15 minute timeout and errors as follows:

Error: Error applying plan:

2019/01/24 18:33:49 [DEBUG] plugin: waiting for all plugin processes to complete...
1 error(s) occurred:

* module.securitycenter.azurerm_security_center_workspace.sc: 1 error(s) occurred:

* azurerm_security_center_workspace.sc: Error waiting: timeout while waiting for state to become 'Populated' (last state: 'Waiting', timeout: 15m0s)

• However the oms securitycenter work space is successfully created, when viewed in the portal
• and AzureRM API seems to be returning a 200 OK during the period azurerm is still creating the oms workspace
• Behaviour is intermittent with new deployments (newly created Azure tenant accounts)
• sometimes the provider will succeed, with the workspace successfully created, however when it fails due to the timeout many re-attempts (with no reconfiguration) are required to get it working finally.

Steps to Reproduce

1. terraform apply

Important Factoids

• Case raised with Azure Support resulted in a report that Azure API is behaving correctly (i.e returning 200 OK), but the terraform azure provider is not handling the OMS securitycenter workspace creation correctly in all cases.

References

• #0000

[archmangler]

Latest test result: With azurerm provider 1.23. the timeout can be extended to 30 mins, however the problem remains, confirming this is not a timeout issue, but an issue of Terraform provider expecting a parameter which is sometimes not returned by AzureRM following creation of the oms security center workspace.

[archmangler]

Any update?

[tombuildsstuff]

hey @archmangler

Thanks for opening this issue.

We're planning on adding support for custom timeouts to all resources as a part of 2.0 (which we're working on the dependencies for at the moment) - as such we should be able to tackle this then. As you've mentioned this timeout changed to 30m in version 1.23 of the Azure Provider - and it's helpful in the interim it could be possible to bump this timeout to an hour (as an example) - however this will ultimately be solved via custom timeouts which are being tracked in #417.

Rather than having multiple issues open tracking the same thing (since we plan to support this across all resources at once, as this is a behavioural change) I'm going to close this issue in favour of #171 - would you mind subscribing to that issue for updates?

Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!