Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Role, rolebinding and secret for injector-leader-elector are missing namespace #682

Closed
thechristschn opened this issue Jan 15, 2022 · 3 comments · Fixed by #683
Closed
Labels
bug Something isn't working injector Area: mutating webhook service

Comments

@thechristschn
Copy link
Contributor

Describe the bug
The role, rolebinding and secret resources used by the injector-leader-elector are always deployed to the current namespace instead of a given namespace.

This affects role vault-agent-injector-leader-elector-role, rolebinding vault-agent-injector-leader-elector-binding and secret vault-injector-certs. These resources are only deployed if injector.replicas is set to more than 1.

While the secret doesn't seem to be important because it will be created in the correct namespace if missing, the role and rolebinding are more important, because the serviceAccount doesn't get the needed permissions to elect a leader. Or maybe to even create the needed certificates.

To Reproduce
Steps to reproduce the behavior:

  1. Install chart to a specific namespace with injector.replicas: 2
  2. Check the logs of the injector. It goes into a CrashLoopBackOff because permissions are missing.
Using internal leader elector logic for webhook certificate management
E0115 13:07:49.773888       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.2/tools/cache/reflector.go:167: Failed to watch *v1.Secret: failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:vault-injector:vault-injector-agent-injector" cannot list resource "secrets" in API group "" in the namespace "vault-injector"

Expected behavior
All namespaced resources should be deployed to the given namespace and the leader-elector/sidecar-injector works.

Environment

  • Kubernetes version: 1.21
  • vault-helm version: 0.18.0

Chart values:

# all default except from:
injector.replicas: 2
@tvoran
Copy link
Member

tvoran commented Jan 19, 2022

Hi @thechristschn, I'm not seeing the same behavior you're describing when doing a helm install to a non-default namespace (with injector.replicas > 1); can you share all the values you set on a chart deployment, and the helm command used to deploy?

@tvoran tvoran added the injector Area: mutating webhook service label Jan 19, 2022
@thechristschn
Copy link
Contributor Author

Hello @tvoran,

sorry, I missed a essential point on how to reproduce this bug, because I oversimplified the way we deploy the injector (and didn't knew it changes the behavior).
We render the helm chart with helm template and apply it afterwards with kubectl. We have additional processing on the helm charts, so we can't just use helm install.

So the commands to reproduce the issue:

kubectl create ns vault
helm repo add hashicorp https://helm.releases.hashicorp.com
helm template vault hashicorp/vault --set injector.replicas=2 -n vault | kubectl apply -f -

If I use helm install -n vault ... it would work as expected, because this also sets the cluster-context for the helm command to the given namespace and everything is applied into that namespace, if not specified otherwise. But this isn't the case if the chart is just rendered and applied afterwards.

But I think the namespace should be rendered correctly to the files if a namespace is provided for the template command. And all other namespaced resources already have metadata.namespace: {{ .Release.Namespace }} in the template (at least all resources we use).

The workaround would be to add the namespace to the kubectl command, but I think the chart behavior should be consistent for all resources.

@tvoran
Copy link
Member

tvoran commented Feb 8, 2022

Thanks for detailing your use case; definitely helps us understand what you were running into.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working injector Area: mutating webhook service
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants