Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow setting network tags on default node pool #1118

Open
tomasgareau opened this issue Jan 7, 2022 · 1 comment
Open

Allow setting network tags on default node pool #1118

tomasgareau opened this issue Jan 7, 2022 · 1 comment
Labels
bug Something isn't working P3 medium priority issues triaged Scoped and ready for work

Comments

@tomasgareau
Copy link
Contributor

I'm trying to bring up a safer-cluster in a Shared VPC network roughly based on the terraform-example-foundation/3-networks approach.

In this VPC network, access to private Google APIs is configured with:

  • DNS entries that point *.googleapis.com to private.googleapis.com (for resources without an external IP address, i.e., my safer-cluster nodes)
  • a firewall rule that allows egress to private.googleapis.com (i.e., 199.36.153.8/30) for any resources that have the network tag allow-google-apis

Like in the terraform-example-foundation-app, I've added the allow-google-apis network tag to the node_pools_tags variable.

What I'd expect to happen is:

  1. The google_container_cluster.primary resource gets created
  2. This creates a default node pool, which immediately gets deleted (as configured by the safer-cluster module)
  3. The node pool I configured now gets created (with the appropriate network tags)
  4. My cluster is ready to go

Instead, however, this seems to be happening:

  1. The google_container_cluster.primary resource gets created
  2. The default node pool gets created and tries to register itself with the control plane
  3. The default node pool cannot register itself with the control plane
  4. The create operation finally times out and the cluster errors out with "Error waiting for creating GKE cluster: All cluster resources were brought up, but: only 0 nodes out of 1 have registered"

In my firewall logs, I can see that the default pool is trying to reach private.googleapis.com (i.e., DNS is working as expected). However, since I cannot add the allow-google-apis network tag to this pool, this egress gets denied:

image

This may be related to #305. Is there a way to add network tags to this pool so it can finish creating successfully (and get immediately deleted, haha)? Or is there an alternative recommended approach here?

@morgante
Copy link
Contributor

The fix here would be to add the network tags (from the first pool) in the cluster config, like we do for service account.

Happy to review a PR.

@morgante morgante added bug Something isn't working P3 medium priority issues triaged Scoped and ready for work labels Jan 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working P3 medium priority issues triaged Scoped and ready for work
Projects
None yet
Development

No branches or pull requests

2 participants