ip_masq with aliased ip addresses

[chrislovecnm]

When we have aliased ip addresses do we need ip_masq enabled for network policies?

https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/blob/master/modules/private-cluster/masq.tf#L22

I am seeing this and I am wondering if we do not need this when we have aliased ip subnets for the services and pods.

[chrislovecnm]

/cc @morgante

[chrislovecnm]

@aaron-lane after researching this, this is a bug. We do not and should not enable ip masq when using aliased ip addresses.

[morgante]

@chrislovecnm The specific reasons for having this are two-fold:

1. Alias IPs are often not routable from on-prem, so pods calling on-prem services need to have their calls NATed
2. As far as I know, ip_masq is required to allow network policies to work. Do you have information to the contrary?

I would be open to adding an additional flag which disables ip-masq.

[chrislovecnm]

As far as I know, ip_masq is required to allow network policies to work. Do you have information to the contrary?

I verified with a couple different engineers that we do not need it. Do you have documentation to the contrary? Yes, having an additional flag for it would be great. I agree with you on the first use case that we need to provide that support.

The reason this is a big inconvenience and a bit of a chicken and egg problem is that you cannot create the vpc, subnet, and private cluster from the same TF. You have to be on a bastion, or open up the api server endpoint (which I would not recommend). To be on a bastion, you have to have the subnet. Hence the chicken and egg problem. The base issues is that the kubectl call cannot communicate with the private server in order to install ip masq.

Thanks

[morgante]

I verified with a couple different engineers that we do not need it. Do you have documentation to the contrary?

It looks like that requirement might be out of date.

I definitely see your use case. I suggest we add a configure_ip_masq variable which defaults to false (this will be a breaking change) and include it for that block.

Can you file a PR for this? If not we'll have someone handle it on our end.

[chrislovecnm]

@morgante to clarify. Change this

terraform-google-kubernetes-engine/autogen/masq.tf

Lines 22 to 24 in d8e7850

	resource "kubernetes_config_map" "ip-masq-agent" {
	count = "${var.network_policy ? 1 : 0}"

To use a new variable:

count = "${var.configure_ip_masq ? 1 : 0}"

[morgante]

Yes.