-
Notifications
You must be signed in to change notification settings - Fork 451
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature request: suggest to provide a helm chart for gateway-api and related CRDs #1590
Comments
Seems reasonable, I guess. This should also optionally install the validating webhook? |
Yes, I think so, this requirement is coming from that I am working on a Contour based Microservices API Gateway, we need to keep the k8s cluster with gateway-api installed, and I think it is a better way to make community to provide an official helm chart and maintained it by community. |
I'm hesitant, but open to discussing. If there are some clear and strong motivators to do this I think it could be worth discussing at some length in our next community meeting. |
I'm not sure a Helm chart is a good fit for Gateway API. Here's what Helm's docs have to say about CRDs:
https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#some-caveats-and-explanations I understand and appreciate their hesitancy to support updating or deleting CRDs and I think more broadly that highlights how careful we need to be when installing and updating CRDs. I'm hoping that the role of CRD management will largely be handled by the cluster provider in the future. For example, on GKE, Gateway API CRDs can be installed and upgraded automatically for you: https://cloud.google.com/kubernetes-engine/docs/how-to/deploying-gateways#enable-gateway. |
Yes, this is helm chart design that can't upgrade and delete CRDs, In our practice, we manually manage CRDs in private K8s cluster, maybe we can request some feedback from end users for this. |
I didn't even think about the CRD install issue, I think that makes this probably a -1 from me. |
Eventhough Helm does not support it properly, in reality a lot of/majority of OSS tools do manage them through Helm though. |
Helm's CRD management system means that you need to manage them via another tool after the initial install, so I don't see much that a chart would provide beyond a small reduction in install friction via a dependency (absent first party support for kustomize URL dependencies in Helm itself). In practice, from what I've seen in the wild, I'd expect many clusters to have a CD system (Argo or similar) to ensure kustomize URL dependencies are deployed. |
As per our comments, close it, when we have more requirement, let's reopen. |
@wilsonwu, could this issue be reopened? |
Yes, reopened. |
I don't think anything has changed though - I don't see us creating a Helm chart for Gateway API CRDs any time soon unless something has changed about Helm and CRDs I haven't heard of? |
@youngnick
I think this comment is more relevant than taken. |
The additional problem that we have on top of the usual CRD management problems is that it's possible (maybe even likely) that multiple implementations that require the CRDs will be installed in the same cluster, and they would then need different versions of the Helm chart (if we provided one). I worry that we could end up with implementations having their CRDs downgraded while they're running, because another implementation was installed in the cluster. This is different to all the examples @lucasfcnunes mentioned in that all of those CRDs are owned by only a single implementation. I'm not saying that we shouldn't do something to make this experience better, just that I don't think the solution can be "make a Helm chart" without addressing some of the concerns that arise from being a shared set of CRDs rather than a set of CRDs closely tied to a single implementation. |
We just don't use Helm would actually protect us from downgrading (installing over current resources) as each resource is tied to a single release. Helm never takes over when the resources don't match the release's annotations and labels. |
I'm +1 for a community owned helm chart install for the CRDs and I want to pitch another item that would be nice to include: Ability to generate Gateway API related objects (HTTPRoutes, TLSRoutes, etc) Why?
To me it seems like there's a couple options:
Today, I'm working around this limitation with: https://github.com/jimangel/unofficial-gateway-api-helm That does not include the CRDs, but in my opinion, I think the project could have some generic templating of Gateway API spec objects AND a I think the trade-off would be documentation around upgrades / life-cycle. The alternative is more custom YAML and more custom solutions like I've already started. It would be great to have a centralized solution for the Gateway API spec to increase adoption. The helm chart could be positioned as a secondary option, as in, "Gateway API project suggests installing via kustomize..." and then below have the helm options for advanced users and/or spec templating. |
Just for my own personal clarity on where people are coming from here and what's actually being suggested, I wanna know what people's expected commitment level is here: Is anyone in this thread specifically interested in owning and maintaining this long term? Or is this more of a request to have the current project as it stands take on the responsibility? |
I still have the same concerns about CRD management directly via Helm, but will be the first to admit that Gateway API CRD management is nowhere near as easy as it could/should be. I've created #2678 to discuss an alternative installation method that could also be bundled in a Helm chart. |
@shaneutt I already have the burden of this with https://github.com/jimangel/unofficial-gateway-api-helm; and without a community chart, I would imagine we'll have more fragmented solutions as everyone makes their own "wedge" to solve this problem vs. a community standard. I would gladly help maintain and support a solution, assuming there's one we all agree on. I kind of thread-jacked a bit with my rant on objects beyond CRDs, and honestly, I think that's going to be an increasingly bigger problem. I would love to see other OSS helm charts adopt support for Gateway API objects, similar to how we see with ingress today, but there's not even published guidance or standards around that.
These concerns are valid. As we chatted about, it's not an issue for external secrets operator or cert manager. But, to be fair, those are the more than only CRDs. Given the "out-of-tree" spec for Gateway API, there needs to be a way to unify it with an implementation AND there needs to be a way to unify it with your workloads. Today, there is no standard and it's causing a lot of friction (especially for anyone trying to automate their clusters at scale using GitOps + OSS helm charts/kustomize). I would be open to the discussion in #2678 for an alternative to install CRDs (defining the "official" way for other implementations/charts to include the Gateway API). Then my remaining gripe for "how can we make gateway api included in all the helm charts as a standard, like ingress" could be solved per workload. If we define what that "official" way is, I'm happy to help write the docs and guidance so others can adopt this model. It's less about helm and more about the fragmented state of things today and the friction caused by Gateway API being out of tree with no defined guidance on how other helm charts / OSS workloads can integrate seamlessly with BYOgateway API integration. |
This has been stale for some time, if someone wants to champion this please let us know as that's what it will take to move it forward. |
@shaneutt Hello. I just create a poc of crds-only helm chart and it works. See the draft pr #2951 for more detail. If the general direction is alright, I can finish the remaining todos and get the pr merged. Edited: The draft pr is on hold and awaiting further discussion. Sorry I didn't read through all the context, i.e. the complex requirements we want it to achieve. See envoyproxy/gateway#1721 (comment) for an example to incorperate crds installation in iac/gitops. |
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten |
👋 hey all, checking in because I've been MIA on paternity leave. I'll be back in office mid-June, happy to help see this to completion or help @networkhermit when I return. /remove-lifecycle rotten |
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
I can see folks are still interested here, so I'll unstale this for now. I'm also adding this as "help-wanted", but this will definitely require some close work with myself and other maintainers, so it's definitely not a good first issue. Solving this properly will require more work on recommendations for CRD version management, which is a very poorly defined area at the moment. /help-wanted |
What would you like to be added:
I'd like to have a helm chart to easily install and upgrade gateway-api.
Why this is needed:
A usage standard, I can contribute it.
The text was updated successfully, but these errors were encountered: