Add dataplane-v2 provisioning support

[ryan-atkins]

Fixes #656

The datapath_provider was released in terraform-provider-google-beta v3.41.0

• adds datapath_provider var for beta clusters, defaults to "DATAPATH_PROVIDER_UNSPECIFIED"
• as-is, would need to set network_policy = false whenever desiring to use the "ADVANCED_DATAPATH" provider due to module currently setting CALICO as the default network_policy_provider.

Could use suggestions on how best to handle introducing this and better working with the network_policy config.

[comment-bot-dev]

Thanks for the PR! 🚀
✅ Lint checks have passed.

[bharathkkb]

Thanks for the PR @ryan-atkins

as-is, would need to set network_policy = false whenever desiring to use the "ADVANCED_DATAPATH" provider due to module currently setting CALICO as the default network_policy_provider

It makes sense to automatically set that for the user if they are explicitly enabling dataplanev2. We should also document this in the variable description. For implementation we would need to check if ADVANCED_DATAPATH is set and use that to control the network_policy behavior in places below.

terraform-google-kubernetes-engine/autogen/main/cluster.tf.tmpl

Line 145 in 4a85321

disabled = ! var.network_policy

terraform-google-kubernetes-engine/autogen/main/main.tf.tmpl

Line 82 in 4a85321

cluster_network_policy = var.network_policy ? [{

[github-actions]

This PR is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days

[xaocon]

@bharathkkb, are you working on this already? If not would you feel like maybe doing some mentoring around it? I'd like to see it implemented but I'm still pretty new with Terraform and don't know much about this module. It looks like it's pretty close already but I may need some guidance around network policy.

[bharathkkb]

Hi @xaocon
Feel free to pick this up and happy to help support. I had two comments above, one was just hard coding the example. The other was to automatically set network_policy = false depending on if datapath_provider is ADVANCED_DATAPATH

[keskiju]

I'm not quite sure if network_policy should be automatically disabled if datapath_provider is set to ADVANCED_DATAPATH. I guess the following configuration is perfectly valid for the google_container_cluster:

  datapath_provider = "ADVANCED_DATAPATH"

  network_policy {
    enabled = true
    provider = "PROVIDER_UNSPECIFIED"
  }

I would assume that with these settings it would automatically choose CILIUM as the network policy provider. You can even leave the network_policy.provider unset since PROVIDER_UNSPECIFIED is the default value for it there. I wonder why this module uses CALICO as the default network_policy.provider instead of PROVIDER_UNSPECIFIED. There would be no problem if PROVIDER_UNSPECIFIED was the default.

[sharkymcdongles]

I'm not quite sure if network_policy should be automatically disabled if datapath_provider is set to ADVANCED_DATAPATH. I guess the following configuration is perfectly valid for the google_container_cluster:

  datapath_provider = "ADVANCED_DATAPATH"

  network_policy {
    enabled = true
    provider = "PROVIDER_UNSPECIFIED"
  }

I would assume that with these settings it would automatically choose CILIUM as the network policy provider. You can even leave the network_policy.provider unset since PROVIDER_UNSPECIFIED is the default value for it there. I wonder why this module uses CALICO as the default network_policy.provider instead of PROVIDER_UNSPECIFIED. There would be no problem if PROVIDER_UNSPECIFIED was the default.

I think you might be incorrect as when I try to deploy a cluster using ADVANCED_DATAPATH and network policy true I get:

Error: googleapi: Error 400: Enabling NetworkPolicy for clusters with DatapathProvider=ADVANCED_DATAPATH is not allowed., badRequest

Also in the documentation here: https://cloud.google.com/kubernetes-engine/docs/how-to/dataplane-v2?hl=it

Warning: You cannot enable the NetworkPolicy feature in the same request that creates a cluster.

So perhaps it is possible to enable it after the fact given the warning here, but it seems it cannot be done on first deploy.

[ryan-atkins]

Sorry ignored this for awhile!
With #809 being merged, I believe that resolves needing to touch the network_policy here as the default now is false.
Updated the example and reran integration tests 👍

[bharathkkb]

Thanks @ryan-atkins

[gw0]

Why is this so complicated? Wouldn't it make much more sense to extend network_policy.provider by adding dataplane-v2 (or cilium)?

[morgante]

@gw0 This was the simplest non-breaking fix, but we could probably re-evaluate interaction with network_policy_provider.