feat: Add support for networking ip_family which enables support for IPV6

[bryantbiggs]

Description

• add support for networking ip_family which enables support for IPV6
• add policy for AmazonEKS_CNI_IPv6_Policy for nodes using VPC CNI to connect when IPV6 is enabled
  • propagate cluster_ip_family down to sub-modules to be able to determine which CNI policy should be used
  • all sub-modules now utilize the iam_role_attach_cni_policy variable to control whether the CNI policy should be attached or not
• bump Terraform AWS provider min version required to support ip_family attribute
  • r/aws_eks_cluster: Add ip_family argument hashicorp/terraform-provider-aws#22485
• update eks-managed-node-group example to demonstrate/validate use of IPV6

Note: IPV6 can only be enabled on new clusters, not existing. So the addition of the IPV6 rule on the security groups is non-breaking (does not affect current clusters either)

Motivation and Context

• Closes Support creating IPv6 EKS clusters  #1746

Breaking Changes

• No

How Has This Been Tested?

• I have tested and validated these changes using one or more of the provided examples/* projects
  • Validated using eks-managed-node-group example

[stevehipwell]

@bryantbiggs this looks great to me, now that the AWS provider v3.72.0 do you anticipate this can be released soon?

[bryantbiggs]

@stevehipwell yes! I just wanted to have at least one deploy to ensure it works as advertised. Just launched a cluster this morning and everythings looking good

[bryantbiggs]

@stevehipwell just trying to figure out why the CoreDNS plugin is not registering as healthy under IPV6 - working through this now https://aws.amazon.com/blogs/containers/amazon-eks-launches-ipv6-support/ - let me know if you have any insight (see eks-managed-node-group example in this PR for the configs currently used)

[stevehipwell]

@bryantbiggs I assume you're using the latest cordns version (v1.8.4), are you using the add-in or manually managing it? I don't think there was anything in the IPv6 announcement about coredns specifically.

Have you looked at the bootstrap options to see if there is an issue with the old logic?

[mKeRix]

@bryantbiggs We observed a while back that when launching a new cluster, compute nodes need to be available before the CoreDNS plugin will launch successfully. Since we manage the cluster addons outside of the module we solved this by simply putting a depends_on = [module.eks] in our case, as that will force it to run after the nodegroup in the module finished creating. Is it possible that you're hitting the same wall? You could try launching the cluster without the addon first and then add it back to see if that works.

Note that this happens on IPv4 clusters as well, so it's not related to your changes here.

[bryantbiggs]

@bryantbiggs I assume you're using the latest cordns version (v1.8.4), are you using the add-in or manually managing it? I don't think there was anything in the IPv6 announcement about coredns specifically.

Have you looked at the bootstrap options to see if there is an issue with the old logic?

Ya, the addons are all pulling the latest versions that are specified in the announcement. I assume there are no changes in the bootstrap logic (I hope) - I would presume this is all handled in the bootstrap.sh script, I didn't see any mention in the docs/announcements

[bryantbiggs]

@bryantbiggs We observed a while back that when launching a new cluster, compute nodes need to be available before the CoreDNS plugin will launch successfully. Since we manage the cluster addons outside of the module we solved this by simply putting a depends_on = [module.eks] in our case, as that will force it to run after the nodegroup in the module finished creating. Is it possible that you're hitting the same wall? You could try launching the cluster without the addon first and then add it back to see if that works.

Note that this happens on IPv4 clusters as well, so it's not related to your changes here.

Yes, this works today in the eks-managed-node-group - the addons are the last bit and the node groups in that example all come up; the CoreDNS plugin takes the longest but it eventually turns healthy all in one terraform apply. I may tear down the cluster and have another go because I did have to update the VPC settings a bit (add egress only gateway and assign IPV6 on launch) and update the node security group for IPV6 egress

[bryantbiggs]

Between this module and MSK, talk about longest turn around time to test out changes 😅 🕐

[bryantbiggs]

ah, might be a permissions issue https://docs.aws.amazon.com/eks/latest/userguide/cni-iam-role.html#cni-iam-role-create-ipv6-policy

You must attach an IPv6 IAM policy to your node IAM or CNI IAM role. Between the two, we recommend that you attach it to a CNI IAM role. For more information, see Step 1: (Optional) Create IAM policy for IPv6 and Step 2: Create the Amazon VPC CNI plugin IAM role.

[bryantbiggs]

Policy addition fixed it - should be all set for review now

[antonbabenko]

This PR is included in version 18.1.0 🎉

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.