`azurerm_kubernetes_cluster` - support migration of `outbound_type` #25021

stephybun · 2024-02-26T16:18:29Z

katbyte

LGTM 🛤️

<Actions> <action id="f410411e63aff4bb73a81c2aec1d373cf8a903e63b30dee2006b0030d8a94cc8"> <h3>Bump Terraform `azurerm` provider version</h3> <details id="1d9343c012f5434ac9fe8a98135bae3667b399259be16d9b14302ea3bd424a24"> <summary>Update Terraform lock file</summary> <p>changes detected:
	"hashicorp/azurerm" updated from "3.93.0" to "3.94.0" in file ".terraform.lock.hcl"</p> <details> <summary>3.94.0</summary> <pre>Changelog retrieved from:
	https://github.com/hashicorp/terraform-provider-azurerm/releases/tag/v3.94.0
FEATURES:

* **New Resource**: `azurerm_kubernetes_fleet_update_run` ([#24813](https://github.com/hashicorp/terraform-provider-azurerm/issues/24813))

ENHANCEMENTS:

* dependencies: updating to `v0.20240228.1142829` of `github.com/hashicorp/go-azure-sdk` ([#25081](hashicorp/terraform-provider-azurerm#25081 `servicefabric`: updating to use the transport layer from `hashicorp/go-azure-sdk` rather than `Azure/go-autorest` ([#25002](hashicorp/terraform-provider-azurerm#25002 `springcloud`: updating to API Version `2024-01-01-preview` ([#24937](hashicorp/terraform-provider-azurerm#24937 `securitycenter`: updating to use the transport layer from `hashicorp/go-azure-sdk` rather than `Azure/go-autorest` ([#25081](hashicorp/terraform-provider-azurerm#25081 Data Source: `azurerm_storage_table_entities` - support for `select` ([#24987](hashicorp/terraform-provider-azurerm#24987 Data Source: `azurerm_netapp_volume` - support for the `smb_access_based_enumeration` and `smb_non_browsable` properties ([#24514](hashicorp/terraform-provider-azurerm#24514 `azurerm_cosmosdb_account` - add support for the `minimal_tls_version` property ([#24966](hashicorp/terraform-provider-azurerm#24966 `azurerm_federated_identity_credential` - the federated credentials can now be changed without creating a new resource ([#25003](hashicorp/terraform-provider-azurerm#25003 `azurerm_kubernetes_cluster` - support for the `current_kubernetes_version` property ([#25079](hashicorp/terraform-provider-azurerm#25079 `azurerm_kubernetes_cluster` - private DNS is now allowed for the `web_app_routing` property ([#25038](hashicorp/terraform-provider-azurerm#25038 `azurerm_kubernetes_cluster` - migration between different `outbound_type`s is now allowed ([#25021](hashicorp/terraform-provider-azurerm#25021 `azurerm_mssql_database` - support for the `recovery_point_id` and `restore_long_term_retention_backup_id` properties ([#24904](hashicorp/terraform-provider-azurerm#24904 `azurerm_linux_virtual_machine` - support for the `automatic_upgrade_enabled`, `disk_controller_type`, `os_image_notification`, `treat_failure_as_deployment_failure_enabled`, and `vm_agent_platform_updates_enabled`properties ([#23394](hashicorp/terraform-provider-azurerm#23394 `azurerm_nginx_deployment` - support for the `automatic_upgrade_channel` property ([#24867](hashicorp/terraform-provider-azurerm#24867 `azurerm_netapp_volume` - support for the `smb_access_based_enumeration` and `smb_non_browsable` properties ([#24514](hashicorp/terraform-provider-azurerm#24514 `azurerm_netapp_pool` - support for the `encryption_type` property ([#24993](hashicorp/terraform-provider-azurerm#24993 `azurerm_role_definition` - upgrade to the API version `2022-05-01-preview` ([#25008](hashicorp/terraform-provider-azurerm#25008 `azurerm_redis_cache` - allow AAD auth for all SKUs ([#25006](hashicorp/terraform-provider-azurerm#25006 `azurerm_sql_managed_instance` - support for the `zone_redundant_enabled` property ([#25089](hashicorp/terraform-provider-azurerm#25089 `azurerm_spring_cloud_gateway` - support for the `application_performance_monitoring_ids` property ([#24919](hashicorp/terraform-provider-azurerm#24919 `azurerm_spring_cloud_configuration_service` - support for the `refresh_interval_in_seconds` property ([#25009](hashicorp/terraform-provider-azurerm#25009 `azurerm_synapse_workspace` - support for using the `user_assigned_identity_id` property within the `customer_managed_key` block ([#25027](hashicorp/terraform-provider-azurerm#25027 `azurerm_windows_virtual_machine` - support for the `automatic_upgrade_enabled`, `disk_controller_type`, `os_image_notification`, `treat_failure_as_deployment_failure_enabled`, and `vm_agent_platform_updates_enabled`properties ([#23394](https://github.com/hashicorp/terraform-provider-azurerm/issues/23394))

BUG FIXES:

* `azurerm_api_management_notification_recipient_email` - fixing an issue where response pages weren't iterated over correctly ([#25055](hashicorp/terraform-provider-azurerm#25055 `azurerm_api_management_notification_recipient_user` - fixing an issue where response pages weren't iterated over correctly ([#25055](hashicorp/terraform-provider-azurerm#25055 `azurerm_batch_pool` - fix setting the `extension.settings_json` property ([#24976](hashicorp/terraform-provider-azurerm#24976 `azurerm_key_vault_key` - `expiration_date` can be updated if newer date is ahead ([#25000](hashicorp/terraform-provider-azurerm#25000 `azurerm_pim_active_role_assignment` - fix an isue where the resource would disappear or fail to import after 45 days ([#24524](hashicorp/terraform-provider-azurerm#24524 `azurerm_pim_eligible_role_assignment` - fix an isue where the resource would disappear or fail to import after 45 days ([#24524](hashicorp/terraform-provider-azurerm#24524 `azurerm_recovery_services_vault` - validate that `use_system_assigned_identity` and `user_assigned_identity_id` cannot be set at the same time ([#24091](hashicorp/terraform-provider-azurerm#24091 `azurerm_recovery_vaults` will now create properly with `SystemAssigned,UserAssigned` identity ([#24978](hashicorp/terraform-provider-azurerm#24978 `azurerm_subscription` - fixing an issue where response pages weren't iterated over correctly ([#25055](https://github.com/hashicorp/terraform-provider-azurerm/issues/25055))


</pre> </details> <details> <summary>3.95.0</summary> <pre>Changelog retrieved from:
	https://github.com/hashicorp/terraform-provider-azurerm/releases/tag/v3.95.0
FEATURES:

* New Resource: `azurerm_container_app_custom_domain` ([#24421](hashicorp/terraform-provider-azurerm#24421 New Resource: `azurerm_data_protection_backup_instance_kubernetes_cluster` ([#24940](hashicorp/terraform-provider-azurerm#24940 New Resource: `azurerm_static_web_app` ([#25117](hashicorp/terraform-provider-azurerm#25117 New resource: `azurerm_static_web_app_custom_domain` ([#25117](hashicorp/terraform-provider-azurerm#25117 New resource: `azurerm_system_center_virtual_machine_manager_availability_set` ([#24975](hashicorp/terraform-provider-azurerm#24975 New Resource: `azurerm_workloads_sap_three_tier_virtual_instance` ([#24384](hashicorp/terraform-provider-azurerm#24384 New Resource: `azurerm_workloads_sap_single_node_virtual_instance` ([#24331](https://github.com/hashicorp/terraform-provider-azurerm/issues/24331))

ENHANCEMENTS:

* `dependencies`: updating to v0.20240229.1102109 of `github.com/hashicorp/go-azure-sdk` ([#25102](hashicorp/terraform-provider-azurerm#25102 `monitor`: updating to use the transport layer from `hashicorp/go-azure-sdk` rather than `Azure/go-autorest` [GH-#25102]
* `network`: updating to API Version `2023-09-01` ([#25095](hashicorp/terraform-provider-azurerm#25095 `azurerm_data_factory_integration_runtime_managed` - support for the `credential_name` property ([#25033](hashicorp/terraform-provider-azurerm#25033 `azurerm_linux_function_app` - support for the `description` property in the `ip_restriction` block ([#24527](hashicorp/terraform-provider-azurerm#24527 `azurerm_linux_function_app` - support for the `ip_restriction_default_action` and `scm_ip_restriction_default_action` properties ([#25131](hashicorp/terraform-provider-azurerm#25131 `azurerm_linux_function_app_slot` - support for the `description` property in the `ip_restriction` block ([#24527](hashicorp/terraform-provider-azurerm#24527 `azurerm_linux_function_app_slot` - support for the `ip_restriction_default_action` and `scm_ip_restriction_default_action` properties ([#25131](hashicorp/terraform-provider-azurerm#25131 `azurerm_linux_web_app` - support for the `description` property in the `ip_restriction` block ([#24527](hashicorp/terraform-provider-azurerm#24527 `azurerm_linux_web_app` - support for the `ip_restriction_default_action` and `scm_ip_restriction_default_action` properties ([#25131](hashicorp/terraform-provider-azurerm#25131 `azurerm_linux_web_app_slot` - support for the `description` property in the `ip_restriction` block ([#24527](hashicorp/terraform-provider-azurerm#24527 `azurerm_linux_web_app_slot` - support for the `ip_restriction_default_action` and `scm_ip_restriction_default_action` properties ([#25131](hashicorp/terraform-provider-azurerm#25131 `azurerm_mysql_flexible_server` - setting the `storage.size_gb` property to a smaller value now forces a new resource to be created ([#25074](hashicorp/terraform-provider-azurerm#25074 `azurerm_orbital_contact_profile` - changing the `channels` property no longer creates a new resource ([#25129](hashicorp/terraform-provider-azurerm#25129 `azurerm_private_dns_resolver_inbound_endpoint` - the `private_ip_address` property is no longer required when `private_ip_allocation_method` is `Dynamic` ([#25035](hashicorp/terraform-provider-azurerm#25035 `stream_analytics_output_blob` - support for the `blob_write_mode` property ([#25127](hashicorp/terraform-provider-azurerm#25127 `azurerm_windows_function_app` - support for the `description` property in the `ip_restriction` block ([#24527](hashicorp/terraform-provider-azurerm#24527 `azurerm_windows_function_app` - support for the `ip_restriction_default_action` and `scm_ip_restriction_default_action` properties ([#25131](hashicorp/terraform-provider-azurerm#25131 `azurerm_windows_function_app_slot` - support for the `description` property in the `ip_restriction` block ([#24527](hashicorp/terraform-provider-azurerm#24527 `azurerm_windows_function_app_slot` - support for the `ip_restriction_default_action` and `scm_ip_restriction_default_action` properties ([#25131](hashicorp/terraform-provider-azurerm#25131 `azurerm_windows_web_app` - support for the `description` property in the `ip_restriction` block ([#24527](hashicorp/terraform-provider-azurerm#24527 `azurerm_windows_web_app` - support for the `ip_restriction_default_action` and `scm_ip_restriction_default_action` properties ([#25131](hashicorp/terraform-provider-azurerm#25131 `azurerm_windows_web_app_slot` - support for the `description` property in the `ip_restriction` block ([#24527](hashicorp/terraform-provider-azurerm#24527 `azurerm_windows_web_app_slot` - support for the `ip_restriction_default_action` and `scm_ip_restriction_default_action` properties ([#25131](https://github.com/hashicorp/terraform-provider-azurerm/issues/25131))

BUG FIXES:

* Data Source: `azurerm_function_app_host_keys` - correctly set `event_grid_extension_key` by searching for the renamed property in the API response ([#25108](hashicorp/terraform-provider-azurerm#25108 `azurerm_app_service_public_certificate` - fix issue where certificate information was not being set correctly in the read ([#24943](hashicorp/terraform-provider-azurerm#24943 `azurerm_container_registry` - prevent recreation of the resource when the `georeplication.tags` are updated ([#24994](hashicorp/terraform-provider-azurerm#24994 `azurerm_firewall_policy_rule_collection_group` - fix issue where the client subscription ID was used to construct the `firewall_policy_id` ([#25145](hashicorp/terraform-provider-azurerm#25145 `azurerm_function_app_hybrid_connection` - fix issue where `SendKeyValue` was not populated in the API payload ([#23761](hashicorp/terraform-provider-azurerm#23761 `azurerm_orbital_contact_profile` - fix creation of the resource when `event_hub_uri` is not specified ([#25128](hashicorp/terraform-provider-azurerm#25128 `azurerm_recovery_services_vault` - prevent a panic when `immutability` is updated ([#25132](hashicorp/terraform-provider-azurerm#25132 `azurerm_storage_account` - fix issue where the queue encryption key type was set as the table encryption key type ([#25046](hashicorp/terraform-provider-azurerm#25046 `azurerm_web_app_hybrid_connection` - fix issue where `SendKeyValue` was not populated in the API payload ([#23761](hashicorp/terraform-provider-azurerm#23761 `azurerm_mssql_database` - fix incorrect error due to typo when using `restore_long_term_retention_backup_id` ([#25180](https://github.com/hashicorp/terraform-provider-azurerm/issues/25180))

DEPRECATIONS:

* Deprecated Resource: `azurerm_static_site` ([#25117](hashicorp/terraform-provider-azurerm#25117 Deprecated Resource: `azurerm_static_site_custom_domain` ([#25117](hashicorp/terraform-provider-azurerm#25117 `azurerm_kubernetes_fleet_manager` - the `hub_profile` property has been deprecated ([#25010](https://github.com/hashicorp/terraform-provider-azurerm/issues/25010))


</pre> </details> </details> <a href="https://infra.ci.jenkins.io/job/updatecli/job/azure/job/main/40/">Jenkins pipeline link</a> </action> </Actions> --- <table> <tr> <td width="77"> <img src="https://www.updatecli.io/images/updatecli.png" alt="Updatecli logo" width="50" height="50"> </td> <td> <p> Created automatically by <a href="https://www.updatecli.io/">Updatecli</a> </p> <details><summary>Options:</summary> <br /> <p>Most of Updatecli configuration is done via <a href="https://www.updatecli.io/docs/prologue/quick-start/">its manifest(s)</a>.</p> <ul> <li>If you close this pull request, Updatecli will automatically reopen it, the next time it runs.</li> <li>If you close this pull request and delete the base branch, Updatecli will automatically recreate it, erasing all previous commits made.</li> </ul> <p> Feel free to report any issues at <a href="https://github.com/updatecli/updatecli/issues">github.com/updatecli/updatecli</a>.<br /> If you find this tool useful, do not hesitate to star <a href="https://github.com/updatecli/updatecli/stargazers">our GitHub repository</a> as a sign of appreciation, and/or to tell us directly on our <a href="https://matrix.to/#/#Updatecli_community:gitter.im">chat</a>! </p> </details> </td> </tr> </table> --------- Co-authored-by: Jenkins Infra Bot (updatecli) <60776566+jenkins-infra-bot@users.noreply.github.com>

qaiserali · 2024-03-20T16:26:45Z

Hi,

This is still not working even using the latest version of azurerm.
I tried to migrate the outbound_type from loadBalancer to userDefinedRouting, but it still doesn't work, and the terraform is throwing the following error.

Kubernetes Cluster Name: "xxx-xxx-xx"): performing CreateOrUpdate: unexpected status 400 with response: { │ "code": "InvalidUserDefinedRoutingWithLoadBalancerProfile", │ "details": null, │ "message": "UserDefinedRouting and load balancer profile are mutually exclusive. Please refer to http://aka.ms/aks/outboundtype for more details", │ "subcode": "", │ "target": "networkProfile.loadBalancerProfile" │ }

Below is the code block of network_profile, and I don't have any settings related to load_balancer_profile in terraform
network_profile { dns_service_ip = var.network_dns_service_ip network_plugin = var.network_plugin network_plugin_mode = var.network_plugin_mode network_policy = var.network_policy outbound_type = var.outbound_type service_cidr = var.network_service_cidr pod_cidr = var.network_pod_cidr }

MartinForReal · 2024-04-03T08:58:58Z

internal/services/containers/kubernetes_cluster_resource.go

@@ -2158,6 +2157,10 @@ func resourceKubernetesClusterUpdate(d *pluginsdk.ResourceData, meta interface{}
 			ebpfDataPlane := d.Get(key).(string)
 			existing.Model.Properties.NetworkProfile.NetworkDataplane = pointer.To(managedclusters.NetworkDataplane(ebpfDataPlane))
 		}
+
+		if key := "network_profile.0.outbound_type"; d.HasChange(key) {
+			existing.Model.Properties.NetworkProfile.OutboundType = pointer.To(managedclusters.OutboundType(d.Get(key).(string)))


@stephybun we also need to reset LoadBalancerProfile if outbound type is not loadbalancer and rest NATGatewayProfile if outbound type is not managedNATGateway

@MartinForReal feel free to open a PR, please make sure to include a test case to demonstrate the use case that is meant to be supported. If you're unable to I'd recommend opening an issue with the details so that maintainers or members in the community can take a look into this if there's free bandwidth.

github-actions · 2024-05-04T02:03:13Z

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

support migration of outbound types

96bd665

stephybun added enhancement service/kubernetes-cluster labels Feb 26, 2024

github-actions bot added documentation size/S labels Feb 26, 2024

katbyte approved these changes Feb 26, 2024

View reviewed changes

stephybun merged commit 3f85dee into main Feb 27, 2024
32 checks passed

stephybun deleted the f/remove-forcenew-outboundtype branch February 27, 2024 06:20

github-actions bot added this to the v3.94.0 milestone Feb 27, 2024

stephybun added a commit that referenced this pull request Feb 29, 2024

Update for #25079 #25038 #25021

c387e60

rizkybiz pushed a commit to rizkybiz/terraform-provider-azurerm that referenced this pull request Feb 29, 2024

support migration of outbound types (hashicorp#25021)

39ea2f0

rizkybiz pushed a commit to rizkybiz/terraform-provider-azurerm that referenced this pull request Feb 29, 2024

Update for hashicorp#25079 hashicorp#25038 hashicorp#25021

55b1b16

stephybun mentioned this pull request Apr 3, 2024

Support outbound type migration in aks service #22854

Closed

MartinForReal reviewed Apr 3, 2024

View reviewed changes

github-actions bot added the waiting-response label Apr 3, 2024

github-actions bot locked as resolved and limited conversation to collaborators May 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`azurerm_kubernetes_cluster` - support migration of `outbound_type` #25021

`azurerm_kubernetes_cluster` - support migration of `outbound_type` #25021

stephybun commented Feb 26, 2024

katbyte left a comment

qaiserali commented Mar 20, 2024

MartinForReal Apr 3, 2024 •

edited

Loading

stephybun Apr 3, 2024

github-actions bot commented May 4, 2024

azurerm_kubernetes_cluster - support migration of outbound_type #25021

azurerm_kubernetes_cluster - support migration of outbound_type #25021

Conversation

stephybun commented Feb 26, 2024

katbyte left a comment

Choose a reason for hiding this comment

qaiserali commented Mar 20, 2024

MartinForReal Apr 3, 2024 • edited Loading

Choose a reason for hiding this comment

stephybun Apr 3, 2024

Choose a reason for hiding this comment

github-actions bot commented May 4, 2024

`azurerm_kubernetes_cluster` - support migration of `outbound_type` #25021

`azurerm_kubernetes_cluster` - support migration of `outbound_type` #25021

MartinForReal Apr 3, 2024 •

edited

Loading