azurerm_monitor_data_collection_rule, unable to set kind = "WorkspaceTransforms"

[eric-mark]

Is there an existing issue for this?

• I have searched the existing issues

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.6.3

AzureRM Provider Version

3.80

Affected Resource(s)/Data Source(s)

azurerm_monitor_data_collection_rule

Terraform Configuration Files

# Deploying our Log Analytics Workspace for Sentinel

resource "azurerm_log_analytics_workspace" "tf-law" {
  name                               = "${local.prefix}-log"
  location                           = azurerm_resource_group.tf-rg.location
  resource_group_name                = azurerm_resource_group.tf-rg.name
  sku                                = var.log_sku_mapping[var.env]
  retention_in_days                  = var.log_rention_days
  reservation_capacity_in_gb_per_day = var.reservation_capacity_in_gb[var.env]
  data_collection_rule_id            = "/subscriptions/${var.az_subscription_id_map[var.env]}/resourceGroups/${local.prefix}.rg/providers/Microsoft.Insights/dataCollectionRules/${local.prefix}.dcr" # this does not support azurerm_monitor_data_collection_rule.dcr_transformation.id

  tags = local.tags

  }
}


# Manages a Data Collection Rule for Log Analytics Workspace
resource "azurerm_monitor_data_collection_rule" "dcr_transformation" {
  name                = "${local.prefix}.dcr"
  resource_group_name = azurerm_resource_group.tf-rg.name
  location            = var.location_map[var.env]
  description         = "SIEM DCR Log Transformation"
  kind                = "WorkspaceTransforms"
  tags                = local.tags

  destinations {
    log_analytics {
      workspace_resource_id = azurerm_log_analytics_workspace.tf-law.id
      name                  = "${local.prefix}-log" #cannot use azurerm_log_analytics_workspace.tf-law.name due to race condition
    }
  }

  data_flow {
    streams       = ["Microsoft-Table-AWSCloudTrail"]
    destinations  = ["${local.prefix}-log"] #azurerm_log_analytics_workspace.tf-law.id does not work
    transform_kql = "source | extend CipherSuite = '' | extend EventVersion = '' | extend TlsVersion = ''"
  }

  data_flow {
    streams       = ["Microsoft-Table-AWSGuardDuty"]
    destinations  = ["${local.prefix}-log"] #azurerm_log_analytics_workspace.tf-law.id does not work
    transform_kql = "source | extend Partition = '' | extend SchemaVersion = ''"
  }

  data_flow {
    streams       = ["Microsoft-Table-AADNonInteractiveUserSignInLogs"]
    destinations  = ["${local.prefix}-log"] #azurerm_log_analytics_workspace.tf-law.id does not work
    transform_kql = "source | extend ConditionalAccessPolicies = '' | extend ConditionalAccessStatus = ''"
  }
}

Debug Output/Panic Output

│ Error: expected kind to be one of ["Linux" "Windows" "AgentDirectToStore"], got WorkspaceTransforms
│
│   with azurerm_monitor_data_collection_rule.dcr_transformation,
│   on data_collection_rule_law.tf line 7, in resource "azurerm_monitor_data_collection_rule" "dcr_transformation":
│    7:   kind                = "WorkspaceTransforms"

Expected Behaviour

In the example here, we must pass "kind" = "WorkspaceTransforms." If we do not pass this value, the DCR will not apply against the Log Analytics Workspace.

https://learn.microsoft.com/en-us/azure/azure-monitor/logs/tutorial-workspace-transformations-api

"resources": [
    {
        "type": "Microsoft.Insights/dataCollectionRules",
        "name": "[parameters('dataCollectionRuleName')]",
        "location": "[parameters('location')]",
        "apiVersion": "2021-09-01-preview",
        **"kind": "WorkspaceTransforms",**
        "properties": {
            "destinations": {
                "logAnalytics": [
                    {
                        "workspaceResourceId": "[parameters('workspaceResourceId')]",
                        "name": "clv2ws1"
                    }
                ]
            },

Actual Behaviour

We receive the following error: expected kind to be one of ["Linux" "Windows" "AgentDirectToStore"], got WorkspaceTransforms

It is not accepting the value required to transform the DCR within the Log Analytics Workspace.

Steps to Reproduce

terraform format
terraform validate
terraform plan
terraform apply

Important Factoids

n/a

References

n/a

[eric-mark]

Thank you, @teowa. We destroyed our DCR rule and re-applied it:

`# Manages a Data Collection Rule for Log Analytics Workspace
resource "azurerm_monitor_data_collection_rule" "dcr_transformation" {
name = "${local.prefix}.dcr"
resource_group_name = azurerm_resource_group.tf-rg.name
location = var.location_map[var.env]
description = "SIEM DCR Log Transformation"
kind = "WorkspaceTransforms" # bug-fixed #23868 -- you must destroy, to change the kind
tags = local.tags

destinations {
log_analytics {
workspace_resource_id = azurerm_log_analytics_workspace.tf-law.id
name = "${local.prefix}-log" # cannot use azurerm_log_analytics_workspace.tf-law.name due to race condition
}
}

data_flow {
streams = ["Microsoft-Table-AWSCloudTrail"]
destinations = ["${local.prefix}-log"] # azurerm_log_analytics_workspace.tf-law.id does not work
transform_kql = "source | extend CipherSuite = '' | extend EventVersion = '' | extend TlsVersion = ''"
}

data_flow {
streams = ["Microsoft-Table-AWSGuardDuty"]
destinations = ["${local.prefix}-log"] # azurerm_log_analytics_workspace.tf-law.id does not work
transform_kql = "source | extend Partition = '' | extend SchemaVersion = ''"
}

data_flow {
streams = ["Microsoft-Table-AADNonInteractiveUserSignInLogs"]
destinations = ["${local.prefix}-log"] # azurerm_log_analytics_workspace.tf-law.id does not work
transform_kql = "source | extend ConditionalAccessPolicies = '' | extend ConditionalAccessStatus = ''"
}
}`

Then we re-applied the Log Analytics Workspace association to the DCR rule and everything works.

Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.