Support for AKS API Server VNet Integration

[zioproto]

Is there an existing issue for this?

• I have searched the existing issues

Description

AKS API Server VNet Integration. The product is still in preview but let's track it in a GitHub issue so we are ready to merge a PR as soon is promoted to GA.

Resources:

• API Server VNET Integration AKS#2729
• https://learn.microsoft.com/en-us/azure/aks/api-server-vnet-integration
• https://www.youtube.com/watch?v=54y986U1uYM&t=494s

Status:

• Product still in preview.
• Implementation in the Terraform provider is available. https://registry.terraform.io/providers/hashicorp/azurerm/3.61.0/docs/resources/kubernetes_cluster#vnet_integration_enabled

New or Affected Resource(s)/Data Source(s)

azurerm_kubernetes_cluster

Potential Terraform Configuration

variable "vnet_integration_enabled" {
  type        = bool
  default     = false
  description = "(Optional) Should API Server VNet Integration be enabled? For more details please visit Use API Server VNet Integration."
}

resource "azurerm_kubernetes_cluster" "main" {
   [..CUT..]
   vnet_integration_enabled = var.vnet_integration_enabled
   [..CUT..]

}

References

No response

[richshadman]

Any chance of this possibly getting this reprioritized and reclassified?

I am trying to create a cluster that is entirely within our private network and would love to make use of this feature. Right now trying to set the subnet results in the following error:

I would argue that this is actually a bug in the current version of the module as the module allows setting a subnet here:
(https://github.com/Azure/terraform-azurerm-aks/blob/main/main.tf#L262)

I think instead of adding a new variable, it would be better to simply set this to true if the subnet is set.

@lonegunmanb @zioproto would it be possible to reclassify as a bug and make the suggested change?

local environment information:
Terraform v1.8.5
on windows_amd64

• provider registry.terraform.io/azure/azapi v1.13.1
• provider registry.terraform.io/hashicorp/azuread v2.52.0
• provider registry.terraform.io/hashicorp/azurerm v3.109.0
• provider registry.terraform.io/hashicorp/null v3.2.2
• provider registry.terraform.io/hashicorp/tls v4.0.5

[ryan-grenz-evelyn]

@richshadman I am hitting the same problem now, though my problem is because of KMS keyvault network access being set to Private. I'm pretty sure a week or so ago when I last deployed the cluster with the same settings the Azure API didn't enforce the vnet integration requirement, but it certainly does now.

Anyway, I would really like to have this switch available and it looks like the @bislydev commit above would add it?

[zioproto]

@richshadman you are right. I identified a mistake on our side, we merged the var.api_server_subnet_id variable that is actually part of the preview feature API Server VNET Intergration, in PR #381

Please bare with us until the feature is declared as GA. Then me and @lonegunmanb will update this module as soon as possible.

[zekena2]

Any info when will it be GA?

[ddezoysa]

@richshadman @lonegunmanb @zioproto When this will be in GA? Please help.

[joemiao-exos]

Preview features should be included in aks module. It's been really painful not being able to manage some preview features but others.

For example,

I have to run the following command to enable API Server Vnet Integration outside of the cluster.

az aks update --name xxx --resource-group xxx --enable-apiserver-vnet-integration --apiserver-subnet-id "xxxx"

The real pain comes when I run a tf plan on the aks module after ANY change, it shows a difference in TF and remote infra.
And I have no way to ignore this change as vnet_integration_enabled is not a supported input in this aks module.
I have no way around it.

      ~ api_server_access_profile {
          - vnet_integration_enabled = true -> null
            # (2 unchanged attributes hidden)
        }