Feature/publish-ingress

[Revolution1]

What this PR does / why we need it:

This is an old painful problem, you can't easily get perfect web app loadbalancing with ingress-nginx on AWS.

ingress-nginx only supports been put behined a Netowrk/Classic Elastic Loadbalancer,
but Classic ELB does not support websocket,
Network ELB does not pass a l7 protocol header to let nginx do the force-ssl-redirection.

So I decided to add a feature called "publish ingress".
Like "publish service", "publish ingress" is to let the controller automatically get lb address from another ingress provisioned by another ingress controller, which, in this scinario, is the alb-ingress-controller.

Then you can get perfect HTTP+HTTPS(terminated by ALB)+Websocket support.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Which issue/s this PR fixes

some of them:
#5231 #3378 #5051 #69 #2000

How Has This Been Tested?

Wrote unit test and passed.
Built a image and deployed on my own test cluster(EKS+alb-ingress-controller). Works well.

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I've read the CONTRIBUTION guide
• I have added tests to cover my changes.
• All new and existing tests passed.
• Works well on my test environment.

[k8s-ci-robot]

Welcome @Revolution1!

It looks like this is your first PR to kubernetes/ingress-nginx 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/ingress-nginx has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @Revolution1. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Revolution1
To complete the pull request process, please assign bowei
You can assign the PR to them by writing /assign @bowei in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Revolution1]

/assign @bowei
Not sure if ingress-nginx team like this idea (had raised a feature-requesting issue before)
Still got some document to write.

BTW, I saw some code related to pulish-service in the nginx templating files, I don't know if publish ingress should cover that part too.

[aledbf]

@Revolution1 thank you for the pull request. That said I am against this feature.
If the main Kubernetes project adds support for ALBs then support for this is transparent and no changes required. Also, you are adding a dependency on another ingress controller in front of ingress-nginx.

Why not just use the ALB ingress controller?

[Revolution1]

@aledbf
Actually I evaluate these approches.

1. contribute to kubernetes:

• comparing to relatively small project like ingress-controller, bringing new feature to kubernetes takes ages.
• The need for this feature is quite urgent.
• Upgrading components from k8s main project is like upgrading the kernel of your system, to some production environment, that's impossible.
• If main project team does this, that would be so great. but...

2. contribute to alb-ingress-comtroller:

• their team has a simmilar proposal that can merge multiple k8s ingress def into one ALB instance, but their design and the concept is very complicated, the dev process are very slow too.

3. add publish-ingress to ingress-nginx:

• coding is not hard to do.
• did some investigation of issues in this repo, seems their some others want this feature too.
• Ingress-nginx already have a simmilar feature publish-service
• Is "adding a dependency on another ingress controller" so bad? I don't understand.

Q: Why not just use the ALB ingress controller
A: alb-ingress-controller only supports "one ingress one alb instance", which cost $20 per instance per month.

Q: Why not do some work around, like deploy alb first then put the address to --publish-address?
A: I really hate to do this sequential manual work. We are in a age of "helm install everything", the dumb workaroud is not acceptable to me.

[aledbf]

ingress-nginx already have a simmilar feature publish-service

This is not comparable. The flag publish-service only allows you to specify a kubernetes service, not an external dependency.

Is "adding a dependency on another ingress controller" so bad? I don't understand.

Yes, it is bad.

Q: Why not do some work around, like deploy alb first then put the address to --publish-address?

You should use this approach, not only requires no code changes but you can get this working in less than five minutes.

[aledbf]

Also, if you really want to follow the approach proposed in the pull request, you don't need to upstream the change to use it.

[Revolution1]

For public cloud service, service load balancer and ingress both need the cloud provider to provision the external loaadbalancer for you.

Service and ingress both are built-in api resources of kubernetes.
Don't know for what reason ingress is called "external dependency" but service loadbalancer is not.

Why?

• Is cloud crontroller manager a external denpendency? (need it to provision you service loadbalancer)
• Is metrics server a external dependency? (I see there is code HPA related code in this repo)
• Is prometheus operator a external dependency? (service monitor in the chart)

Why? Why another ingress controller here is called external dependency and it is so bad?

[Revolution1]

Also, if you really want to follow the approach proposed in the pull request, you don't need to upstream the change to use it.

Okay, great!
👍