Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Microsoft.Insights/diagnosticSettings@2021-05-01-preview - logAnalyticsDestinationType break PUT-GET consistency #22400

Open
teowa opened this issue Feb 1, 2023 · 1 comment
Open

Microsoft.Insights/diagnosticSettings@2021-05-01-preview - logAnalyticsDestinationType break PUT-GET consistency #22400

teowa opened this issue Feb 1, 2023 · 1 comment
Labels
Insights Service Attention Workflow: This issue is responsible by Azure service team.

Comments

@teowa
Copy link
Contributor

teowa commented Feb 1, 2023 •
edited
Loading

Background

Under some situation when set logAnalyticsDestinationType as Dedicated, the API returns null, and when set logAnalyticsDestinationType as null (or not set), the API returns AzureDiagnostics. This breaks PUT-GET consistency, see roundtrip_inconsistent_property. The diagnostic setting can be used for many target resource, it is hard to fully test all the situation. Below shows two cases:

  1. target KeyVault, PUT AzureDiagnostics or Dedicated, GET null
  2. target DataFactory (V2), PUT null, GET AzureDiagnostics. But PUT AzureDiagnostics or Dedicated can work well.

swagger:

azure-rest-api-specs/specification/monitor/resource-manager/Microsoft.Insights/preview/2021-05-01-preview/diagnosticsSettings_API.json

Lines 323 to 327 in e0a8a42

"logAnalyticsDestinationType": {
"type": "string",
"description": "A string indicating whether the export to Log Analytics should use the default destination type, i.e. AzureDiagnostics, or use a destination type constructed as follows: <normalized service identity>_<normalized category name>. Possible values are: Dedicated and null (null is default.)"
}
},

API flow 1, target KeyVault, PUT AzureDiagnostics or Dedicated, GET null

PUT https://management.azure.com/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.KeyVault/vaults/wtexpkeyvault/providers/microsoft.insights/diagnosticSettings/exp?api-version=2021-05-01-preview
x-ms-request-id: 34a48570-6b21-49fd-af95-f611c88ec97f
x-ms-correlation-request-id: 04fe903c-847d-427c-b23b-9ed8c53966b9

payload 
{
    "properties": {
        "logs": [
            {
                "category": "AuditEvent",
                "categoryGroup": null,
                "enabled": true,
                "retentionPolicy": {
                    "days": 0,
                    "enabled": false
                }
            },
            {
                "category": "AzurePolicyEvaluationDetails",
                "categoryGroup": null,
                "enabled": false,
                "retentionPolicy": {
                    "days": 0,
                    "enabled": false
                }
            }
        ],
        "metrics": [
            {
                "timeGrain": null,
                "enabled": true,
                "retentionPolicy": {
                    "days": 0,
                    "enabled": false
                },
                "category": "AllMetrics"
            }
        ],
        "workspaceId": "/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.OperationalInsights/workspaces/wt-testlaw",
        "logAnalyticsDestinationType": "AzureDiagnostics"
    }
}

200 OK

{
    "id": "/subscriptions/XXXX/resourcegroups/wt-diagnostic-resources/providers/microsoft.keyvault/vaults/wtexpkeyvault/providers/microsoft.insights/diagnosticSettings/exp",
    "type": "Microsoft.Insights/diagnosticSettings",
    "name": "exp",
    "location": null,
    "kind": null,
    "tags": null,
    "properties": {
        "storageAccountId": null,
        "serviceBusRuleId": null,
        "workspaceId": "/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.OperationalInsights/workspaces/wt-testlaw",
        "eventHubAuthorizationRuleId": null,
        "eventHubName": null,
        "metrics": [
            {
                "timeGrain": "PT1M",
                "category": "AllMetrics",
                "enabled": true,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            }
        ],
        "logs": [
            {
                "category": "AuditEvent",
                "categoryGroup": null,
                "enabled": true,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            },
            {
                "category": "AzurePolicyEvaluationDetails",
                "categoryGroup": null,
                "enabled": false,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            }
        ],
        "logAnalyticsDestinationType": "AzureDiagnostics"
    },
    "identity": null
}

GET https://management.azure.com/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.KeyVault/vaults/wtexpkeyvault/providers/microsoft.insights/diagnosticSettings/exp?api-version=2021-05-01-preview
x-ms-request-id: 1cb12d3e-463e-4a4d-9a8c-d1c963a2c4e3
x-ms-correlation-request-id: f430d3e4-c539-4cc5-b4be-78d3585e7322
200 OK

payload 
{
    "id": "/subscriptions/XXXX/resourcegroups/wt-diagnostic-resources/providers/microsoft.keyvault/vaults/wtexpkeyvault/providers/microsoft.insights/diagnosticSettings/exp",
    "type": "Microsoft.Insights/diagnosticSettings",
    "name": "exp",
    "location": null,
    "kind": null,
    "tags": null,
    "properties": {
        "storageAccountId": null,
        "serviceBusRuleId": null,
        "workspaceId": "/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.OperationalInsights/workspaces/wt-testlaw",
        "eventHubAuthorizationRuleId": null,
        "eventHubName": null,
        "metrics": [
            {
                "category": "AllMetrics",
                "enabled": true,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            }
        ],
        "logs": [
            {
                "category": "AuditEvent",
                "categoryGroup": null,
                "enabled": true,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            },
            {
                "category": "AzurePolicyEvaluationDetails",
                "categoryGroup": null,
                "enabled": false,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            }
        ],
        "logAnalyticsDestinationType": null
    },
    "identity": null
}

API flow 2, target DataFactory (V2), PUT null, GET AzureDiagnostics. But PUT AzureDiagnostics or Dedicated can work well

PUT https://management.azure.com/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.DataFactory/factories/wt-acctest/providers/microsoft.insights/diagnosticSettings/wt-datafac-DS?api-version=2021-05-01-preview
x-ms-request-id: d738f84e-cf08-449a-a370-7b328ed7e3bd
x-ms-correlation-request-id: f65996b6-3bf5-492f-bf1c-b6a7734d8bdd

payload 
{
    "properties": {
        "logs": [
            {
                "category": "SSISIntegrationRuntimeLogs",
                "categoryGroup": null,
                "enabled": true,
                "retentionPolicy": {
                    "days": 0,
                    "enabled": false
                }
            }
        ],
        "metrics": [
            {
                "timeGrain": null,
                "enabled": false,
                "retentionPolicy": {
                    "days": 0,
                    "enabled": false
                },
                "category": "AllMetrics"
            }
        ],
        "workspaceId": "/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.OperationalInsights/workspaces/wt-testlaw",
        "logAnalyticsDestinationType": ""
    }
}

200 OK

{
    "id": "/subscriptions/XXXX/resourcegroups/wt-diagnostic-resources/providers/microsoft.datafactory/factories/wt-acctest/providers/microsoft.insights/diagnosticSettings/wt-datafac-DS",
    "type": "Microsoft.Insights/diagnosticSettings",
    "name": "wt-datafac-DS",
    "location": null,
    "kind": null,
    "tags": null,
    "properties": {
        "storageAccountId": null,
        "serviceBusRuleId": null,
        "workspaceId": "/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.OperationalInsights/workspaces/wt-testlaw",
        "eventHubAuthorizationRuleId": null,
        "eventHubName": null,
        "metrics": [
            {
                "timeGrain": "PT1M",
                "category": "AllMetrics",
                "enabled": false,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            }
        ],
        "logs": [
            {
                "category": "SSISIntegrationRuntimeLogs",
                "categoryGroup": null,
                "enabled": true,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            }
        ],
        "logAnalyticsDestinationType": ""
    },
    "identity": null
}

GET https://management.azure.com/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.DataFactory/factories/wt-acctest/providers/microsoft.insights/diagnosticSettings/wt-datafac-DS?api-version=2021-05-01-preview

x-ms-request-id: f667dc88-c6e5-4eb1-ab12-e4685b271b42
x-ms-correlation-request-id: b438f808-33c0-4d7e-b93a-d1d82b4581a2
200 OK

payload 
{
    "id": "/subscriptions/XXXX/resourcegroups/wt-diagnostic-resources/providers/microsoft.datafactory/factories/wt-acctest/providers/microsoft.insights/diagnosticSettings/wt-datafac-DS",
    "type": "Microsoft.Insights/diagnosticSettings",
    "name": "wt-datafac-DS",
    "location": null,
    "kind": null,
    "tags": null,
    "properties": {
        "storageAccountId": null,
        "serviceBusRuleId": null,
        "workspaceId": "/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.OperationalInsights/workspaces/wt-testlaw",
        "eventHubAuthorizationRuleId": null,
        "eventHubName": null,
        "metrics": [
            {
                "category": "AllMetrics",
                "enabled": false,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            }
        ],
        "logs": [
            {
                "category": "SSISIntegrationRuntimeLogs",
                "categoryGroup": null,
                "enabled": true,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            },
            {
                "category": "ActivityRuns",
                "categoryGroup": null,
                "enabled": false,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            },
            {
                "category": "PipelineRuns",
                "categoryGroup": null,
                "enabled": false,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            },
            {
                "category": "TriggerRuns",
                "categoryGroup": null,
                "enabled": false,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            },
            {
                "category": "SandboxPipelineRuns",
                "categoryGroup": null,
                "enabled": false,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            },
            {
                "category": "SandboxActivityRuns",
                "categoryGroup": null,
                "enabled": false,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            },
            {
                "category": "SSISPackageEventMessages",
                "categoryGroup": null,
                "enabled": false,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            },
            {
                "category": "SSISPackageExecutableStatistics",
                "categoryGroup": null,
                "enabled": false,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            },
            {
                "category": "SSISPackageEventMessageContext",
                "categoryGroup": null,
                "enabled": false,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            },
            {
                "category": "SSISPackageExecutionComponentPhases",
                "categoryGroup": null,
                "enabled": false,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            },
            {
                "category": "SSISPackageExecutionDataStatistics",
                "categoryGroup": null,
                "enabled": false,
                "retentionPolicy": {
                    "enabled": false,
                    "days": 0
                }
            }
        ],
        "logAnalyticsDestinationType": "AzureDiagnostics"
    },
    "identity": null
}
@ghost ghost added the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Feb 1, 2023
@teowa teowa mentioned this issue Feb 1, 2023
Merged
@ankhyk ankhyk added Insights Service Attention Workflow: This issue is responsible by Azure service team. labels Feb 1, 2023
@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Feb 1, 2023
@teowa teowa mentioned this issue Feb 2, 2023
Closed
1 task
@teowa teowa mentioned this issue Jul 10, 2023
Closed
1 task
@teowa teowa mentioned this issue Feb 4, 2024
Closed
1 task
@LaurentLesle
Copy link

Impacting the Azure Verified Module project -> https://github.com/Azure/terraform-azurerm-avm-res-kusto-cluster/actions/runs/9090952291/job/24984653056?pr=27#step:3:228

We are using AzureDedicated to the destination type on a Kusto Explorer. Fun fact: the diagnostic profiles are sent to ADX* tables so the PUT is working but the get is null forcing Terraform to re-modify the service. AVM has a rule to get modules idempotent and we cannot achieve it dur to this issue.

Need a fix in the API to get a fix in the golang SDK to fix Terraform Azurerm.

@keisari-ch keisari-ch mentioned this issue Aug 6, 2024
Open
1 task
@stevendborrelli stevendborrelli mentioned this issue Aug 21, 2024
Open
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Insights Service Attention Workflow: This issue is responsible by Azure service team.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@LaurentLesle @ankhyk @teowa