Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow avoiding cluster-internal rerouting for LoadBalancer services by only setting hostname in status field #641

Open
simon-wessel opened this issue Mar 11, 2024 · 5 comments
Labels
enhancement New feature or request

Comments

@simon-wessel
Copy link

simon-wessel commented Mar 11, 2024

Is your feature request related to a problem? Please describe.
Kube proxy creates "shortcuts" for external IPs of LoadBalancer services. When a pod in the cluster connects to a IP that is set as the external IP of a LoadBalancer service, the traffic will not leave the cluster and instead be directly routed to that service. Therefore the Netscaler ADC is not part of the traffic and any rules/settings that are configured will be "bypassed".

Describe the solution you'd like
There are long discussions over at kubernetes (here and here) if this behaviour is intransparent for the user and poses problems or risks for those who want to use features of the load balancer (firewalls/logging/auth/...).

I would like to kindly request the option to change the default behavior when needed. Other ingress controllers have by now implemented a workaround to not set .status.loadBalancer.ingress[].ip, but instead only .status.loadbalancer.ingress[].hostname. This could be configured using an annotation.

As far as I know it is currently not possible for NetsScaler LoadBalancer services to not have the .status.loadBalancer.ingress[].ip field set after provisioning. Please correct me if I am wrong.

Describe alternatives you've considered
The topic has gained enough traction that a KEP has been introduced and there is a new feature in alpha state in Kubernetes 1.29. However many users are not yet using that version or may not want to use the feature in an Alpha state. Also even if the Kubernetes Alpha feature is enabled, the ingress controller still needs to set the .status.loadBalancer.ingress[].ipMode field. The support for this field could also be implemented while working on this issue.

Additional context
Steps to reproduce:

  1. Create LoadBalancer service as described here.
  2. Create a pod in the same cluster and send traffic to the external address of the LoadBalancer service.
  3. Monitor the Netscaler ADC to see that requests do not hit the ADC.
@subashd subashd added the enhancement New feature or request label Mar 12, 2024
@arijitr-citrix arijitr-citrix self-assigned this Mar 14, 2024
@simon-wessel simon-wessel changed the title Allow avoiding IPVS for LoadBalancers by only setting hostname in status field Allow avoiding cluster-internal rerouting for LoadBalancer services by only setting hostname in status field Mar 15, 2024
@simon-wessel
Copy link
Author

Correction: According to the comments in the kubernetes tickets the traffic will stay in the cluster no matter if IPVS or iptables is used.

If have updated the title and issue description.

@simon-wessel
Copy link
Author

Update: Even with the 1.29 Alpha feature the Ingress controller would need to support setting the new ipMode in the status. This is described in this blog article.

@arijitr-citrix I see you assigned yourself the issue. Do you see this implemented in the foreseeable future?

@arijitr-citrix arijitr-citrix removed their assignment Mar 25, 2024
@arijitr-citrix
Copy link
Collaborator

Hi @simon-wessel As we are stacked into current commits, we need more information to pick this task. I request you to fill out Requirement Gathering Questionnaire. We can then check based on the urgency.

@simon-wessel
Copy link
Author

Hi @arijitr-citrix I have filled out the Questionnaire as requested.

@lukasboettcher
Copy link

Attached is a patch for triton bundled with the quay.io/netscaler/netscaler-k8s-ingress-controller:2.1.4 image. @arijitr-citrix please have a look.

lb_status_patch.txt

This patch allows a user to use a hostname instead of an ip in the LoadBalancerIngress with the service.citrix.com/loadbalancer-force-hostname annotation or set the ipMode with the service.citrix.com/loadbalancer-ip-mode annotation on a service of type LoadBalancer. Both of which would fix this cluster-internal routing issue.

For anyone interested, you can patch the /usr/src/triton/kubernetes/kubernetes.py file in the aforementioned image to get this functionality.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants