AKS: support setting cluster outbound IPs when using standard loadbalancer

[landro]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

In June, Azure added preview support for using Standard sku load balancers in AKS clusters. Support for configuring load balancer sku was added to this provider in 73f6b2b.

The feature is now in GA, and Azure has added the additional possibility of configuring what LoadBalancerProfile to use (what IPs or IP prefixes to use for outbound traffic) in the latest version of the containerservice API (github.com/Azure/azure-sdk-for-go/services/containerservice/mgmt/2019-08-01/containerservice).

Support for setting IPs or IP prefixes should therefor be added to this provider.

While at it, one should also add the possibility to just configure the number of managed outbound IPs the cluster should provision.

New or Affected Resource(s)

• azurerm_kubernetes_cluster

Potential Terraform Configuration

resource "azurerm_kubernetes_cluster" "default" {
...
 network_profile {
    load_balancer_sku = "standard"
    # one of the following
    load-balancer-outbound-ips = [ip1, ip2] 
    load-balancer-outbound-ip-prefixes = [ippref1, ippref2]
    load-balancer-managed-outbound-ip-count = 4
  }
}

References

• https://docs.microsoft.com/en-us/azure/aks/load-balancer-standard#optional---provide-your-own-public-ips-or-prefixes-for-egress

[evenh]

I’ve started work on this feature

[stromvirvel]

I'm struggeling with this topic aswell, but I'm not sure if this goes to the same issue. I'm fine with creating a random static IP address, but somehow I want to export that IP address. After switching the LB SKU from basic to standard, Azure created a static IP address and assigned it to the LB. But there seem no way to access that IP address.

See: https://www.terraform.io/docs/providers/azurerm/r/kubernetes_cluster.html#attributes-reference

[PirateBread]

Having some related issues here.

1. Randomly assigned public IP is not ideal but it's OK as long as we can retrieve this somewhere BUT there are no attributes which reference this IP so we can't use it to build our NSG rules for example.

2. We already have static public IPS which we attach to our ingresses, however this only applies to inbound traffic. It doesn't seem possible for example to set this as the default IP for outbound traffic?

Overall it's quite disappointing because this used to work, some time back before all the VMSS/ Standard SKU changes came through. Our Ingress IP was the only public IP our clusters had so it was quite easy to manage.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!