-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add optional firewall support #452
Comments
Your second rule definitely sounds like a good addition (assuming we have variables to control it). For your first rule, what will break if you don't have this rule? Will GKE itself break, or just some applications? I'm reticent to override firewall rules when many organizations have alternative/preferred egress paths. |
The TF examples above were more or less pulled out of my current usage, for upstreaming it most would be controlled by variables and opt in e.g. firewall_create_intra_cluster_egress_rule = false # Create 1. if true
firewall_allow_ingress_webhook_ports = [] # Create 2. if non empty w/given ports
firewall_priority = 2000 Regarding 1, my scenario was with a VPC with a Spinning up a GKE cluster with Specifically i was seeing e.g. DNS breaks since you can't hit UDP/53 to the kube-dns pods, the kube-dns pods themselves manage to work fine because VPC firewalls always allow access to the metadata server. Example of getting I/O timeouts during DNS resolution
I did raise a support case last summer regarding the intranode/egress stuff, i don't think that got anywhere unfortunately. Ideally GKE itself creates a Egress rule for internal cluster traffic just like how it manages the Ingress rules. This rule workaround is similar to those Ingress rules managed by GKE because it restricts the destination to the cluster CIDR + cluster specific network tags. In fact we could probably restrict it to just the pod + master CIDR blocks and it should work fine (no need for node/service ranges) |
Got it, in that case I think both firewall rules make sense (provided they can be disabled). Happy to accept a PR. |
1000+ for this; kubernetes/kubernetes#79739 (comment) — this comment fairly accurately makes explicit what problem people (including me) run into any time a new cluster is provisoined. |
Consider adding a opt in method for this module to create common/useful firewalls related to the cluster. I tend to always have to create 2 firewall rules due to running in a slightly more restricted VPC or for certain workloads (Admission Controllers).
Explict Egress rule for cluster traffic.
GKE already creates a
Ingress
rule for the cluster and then relies on the fact there is a implicit allow-all-egress rule for VPCs by default. That assumption doesn't work when you are running locked down VPCs that create adefault-deny-all
egress rule (also w/ intranode visibility enabled)Allow non 443 ports for GKE Master -> nodes due to admission controllers
This is for when cluster workloads include webhooks/admission controllers that run on non 443 ports. Without this you will end up seeing timeouts and pods are blocked from being created.
kubernetes/kubernetes#79739
Examples: 9443 (Elasticsearch Operator elastic/cloud-on-k8s#896), 15017 (Istio 1.5+ istio/istio#19532)
The text was updated successfully, but these errors were encountered: