-
Notifications
You must be signed in to change notification settings - Fork 693
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add tooling to customize the all-in-one manifests #2406
Comments
The tool would need to take into account the cluster role needed to create |
We discussed offline:
|
A good first step could be to document how to tweak the existing manifests. It came as a question again: https://discuss.elastic.co/t/difficulty-installing-eck-in-a-single-namespace/225891. |
Regarding this discuss forum comment. If the configuration the Makefile in |
That's also an option yes. I would be in favor of removing all those files that we have a hard time maintaining in sync with each other. I thought we already did that actually when removing the namespace/global operators concept. |
Just tested on a fresh k8s cluster and our single namespace make target seems to work fine
This gives you a restricted single namespace deployment of ECK. I have not double checked all the RBAC rules whether they are as minimal as possible, but I think this is a start. |
I just saw this issue and maybe my perspective as a user can help (and I also want to +1 the necessity of this 😄 ): This is a much neede feature for us, as we are using multiple elasticsearch-clusters within our kubernetes-cluster. And we also have several clusters where we need it each with different namespaces. So our reqruiement is to be very flexible in terms of which namespaces the operator needs to watch. Deploying the all-in-one solution however gives the operator a lot of permissions (if I understand it correctly, it can almost do anything in any namespace in the cluster) - this is far form a least-privilige approach and does not feel right from a security perspective, so it is not the right option for us. S in order to use the namespace version, we currently have to check out this repository and run the A helm chart would fit best into our tool-stack, as we could simply set a version for the operator once and then configure the namespaces to be watched in several I think the bottomline requirement here is basically: Download a version of the operator once, then be able to configure it for many environments - all within the project's repository. |
Based on the upcoming 1.1.0 release, possible ways of customizing the operator installation are as follows:
The available tools for generating customized manifests using the above parameters are:
Hand-rolled templatingPros
Cons
HelmPros
Cons
KustomizePros
Cons
Other toolsMost of these tools are fairly new (therefore, not widely in use yet) and require extra tooling to be installed. How other popular projects handle installationIstio
cert-manager
Knative
KubeDB
CockroachDB
|
I wanted to highlight that one big blocker for Helm is this issue: Helm does not support upgrading our CRDs api version. See also this issue in our repo. Hence if you have a Helm-managed ECK setup, you cannot upgrade ECK if the upgrade includes a change in CRD versioning. You'll have to delete all resources first. Which is pretty bad :( According to the issue it looks like it has been resolved recently, so probably worth double-checking. |
Indeed, Helm does not support updating or deleting CRDs. Based on discussions about this very issue in the past, as I understand it, the reasons we have been reluctant to use Helm as an official distribution method are:
1 and 2 are probably the reasons why most other projects in the Kubernetes ecosystem provide the option to use raw manifests in place of Helm as well. Our main issue right now is the difficulty and complexity of generating manifests for the different use cases that users have. I think a good way of addressing this problem is to have a tool similar to The workflow would be:
|
I would to add, that the tiller isn't needed with helm3. But because we apply
Well, if it's
|
Some users would like to run the operator in a particular namespace, with restricted RBAC access:
Issue #374 and this design doc mentions a CLI tool to help generate the correct manifest according to one's need:
Detailing every manual patch to be made in a documentation page seems quite complicated, so a CLI/script/helm/kustomize seems more appropriate to me. We could also rely on it to customize the E2E tests manifests.
The text was updated successfully, but these errors were encountered: