Gslb CRD and Controller

[ytsarev]

First iteration of Gslb CRD and Controller.

Important points

• Gslb instantiates GslbResolver(abstracted coredns deployment)
• Gslb does not instantiate Ingress objects, instead it gets the list of Ingresses with gslb=true label attached. This way we will decouple Gslb from standard Ingress creation. It will give us freedom to e.g. use helm charts with minimum modifications( if we stick to original plan then we would have to replace Ingresses with Gslb objects in charts which might be painful from maintenance standpoint)
• Currently we just reflect collected gslb labeled Ingresses in Gslb.Status. Later on we will use this information for a real action, e.g update resolver and external dns

[donovanmuller]

Think we just need to discuss how Ingress resources relate to Gslb resources.
The reason the initial design had two distinct Ingress concepts:

• one for the usual ingress for a service within a single cluster (cluster specific ingress). I.e. the Ingress resource you would create under normal circumstances
• and another separate/independant Ingress resource that will be created by the GSLB controller specifically to handle ingress for the GSLB host

The reason for the separation is:

• it is a common in some environments to have a cluster specific host (e.g. servicea.dc-1.host.com) that allows clients to deliberately connect to a service in a specific cluster (e.g. dc-1). This would be the usual Ingress resource created
• exposing a service via Gslb should be a conscious/explicit decision, where the team has taken into account that their services in fact can support running in a multi datacenter scenario. I.e. not all services would be exposed via GSLB
• the GSLB service's host (e.g. servicea.cloud.host.com) will typically differ from the cluster specific host (e.g. servicea.dc-1.host.com). Expressing this as annotations etc. becomes complex if using the cluster specific Ingress vs. using the details in the Gslb resource to generate a Ingress specifically for GSLB traffic
• given that you can specify multiple hosts's and service backend's in an Ingress, does that mean that all hosts's in an Ingress should be exposed via GSLB? If not how would you specify only certain hosts's? Also, in the case of multiple service backend's, does the health of the GSLB service depend on the Pod health of all the Pods for all backend services? It becomes quite difficult where to draw the line in how much you support from a GSLB perspective vs. what is available for a Ingress resource
• it gives a nice separation between ingress for local traffic and GSLB traffic and deleting/editing the local Ingress does not necessarily affect GSLB traffic
• it also guards against other ingress solutions (who do not use Ingress resource to describe an ingress endpoint) not being compatible with Oh My GSLB. Like Istio Ingress Gateway, Ambassador API Gateway, Treafik IngressRoute etc.

The GSLB Ingress resource being created specifically, with a one to one relationship with a Gslb resource, by the controller seemed to solve many of the issues above and was seen as almost an implementation detail.

We also initially considered using annotation's to upgrade a Ingress to support GSLB but with the above considerations and the potential of the annotations becoming more and more to add specific Gslb meaning etc. it seemed to make more sense to just use a Gslb resource as the only resource that a user would interact with to expose a service.

Happy to debate though 👍

[ytsarev]

@donovanmuller thanks a lot for extensive comment! Now I get it, Gslb is not going to replace standard Ingresses but sit next to them. Makes total sense! I will update the controller logic and api schema.
Regarding the problem of which pod health to monitor I think we need to introduce something like NodeLabelSelector in Gslb resource to explicitly being able to control the health check targets

[donovanmuller]

Regarding the label selector, it could be a option but if we agree that the service backends in the Gslb resource are explicitly added for the GSLB exposed service, then by extension we can assume that all Pods for the referenced backend servers should be used when considering the health of the GSLB service.

Ie. because it's now a choice and by adding the backend services/paths, we are basically agreeing that all Pods Endpoints backed by the services should all be included as part of the health checks.

Make sense? Agree?

[ytsarev]

@donovanmuller ok, so instead of LabelSelector we about to rely on Gslb Ingress ServiceName and make the recursive pod checks. It makes sense but this way we will health check exclusively frontend /exposed services only. It is more simple and straightforward meanwhile Readiness check of the frontend pods should be good enough to detect potential backend problem.

[ytsarev]

Closing in favour of #5 . Not force pushing to #4 to keep and log useful contextual discussion