-
Notifications
You must be signed in to change notification settings - Fork 194
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use existing service account for Helm deployment #4155
Comments
This ask seems reasonable - we'll look at doing it for the next release. |
Are you aware that you can run a single instance of the operator in a multitenant mode now? See Credential scope. Doing this might be significantly easier for you to manage/orchestrate than running the operator in multitenant mode, while still achieving your user-multitenancy goals? See also: https://azure.github.io/azure-service-operator/guide/authentication/#using-multiple-operators-with-a-single-credential-per-operator
|
I'm not quite sure if that is something that would solve our problem, or maybe I just don't fully understand it. We have just recently migrated to workload identities cluster-wide, so in our multi-tenant deployment we use that for the operator in the tenant namespace. With the user assigned identity, we can then be certain that this tenant namespace can only create resources in a specific resource group that was created for this particular tenant. |
Is my understanding correct that right now you're using the If the objective you have is that every namespace in your cluster needs their own identity to access anything in Azure, and each namespace needs a different identity, you can instead accomplish that by installing the operator once in a namespace such as
Note that there's no secret details specified in that command. This means that ASO is watching every namespace, but has no permissions with Azure to do anything (because no identities are configured). Then in each tenant namespace, the users can create a secret named The key points here being:
You might still want the ability to configure the ServiceAccount of the single installation of ASO, but you won't need to do it in every namespace just in the Does that make sense? |
You are correct, I followed the instructions for multiple operator multitenancy deployment here: https://azure.github.io/azure-service-operator/guide/authentication/multitenant-deployment/ The objective is that every namespace in the cluster needs their own identity to access anything in Azure, and each namespace needs a different identity, so I tried your suggested deployment. But, then I ran into issues with the default ASO service account. From the log, I can see:
And this is the secret used by the operator itself:
Seems to me that the operator finds the secret in the tenant namespace, but is using the default service account and workload identity to authenticate, and this fails. What am I missing here to get it to work? |
You should be able to do this with a single operator instance and per-namespace secrets as I described above. Just making sure the pattern is in alignment w/ what I'm expecting, you created an You had said:
I assume that means your
If this is the case, did you miss this step for allowing the ASO pod to use that identity to authenticate with Azure?
The error you got looks to me like ASO used the right identity, but was denied from exchanging its serviceaccount identity for an Azure token because no FederatedIdentityCredential allowed it. |
Yes, your assumptions are right. The |
Yes I think even if you can use this (IMO simpler) multitenant strategy there's still value in letting you control the ServiceAccount. It's just that you only need it once not N times. Either way we'll leave this issue open and investigate making SA configurable in the next release. |
Now it works as per your suggestions. My mistake was that I used the wrong subject when I created the federated credential. Once I got that right and referenced the service operator service account everything worked fine. Thanks! |
Glad to hear it worked! We will pursue making SA configurable in the Helm chart too. |
Describe the current behavior
Currently, there is no option to specify an existing service account when you deploy ASO with the Helm chart. In a multi tenant setup, this is particularly bad from a security perspective, because it requires that
ClusterRoleBinding
can be created when deploying the tenant ASO operator. With this requirement, Helm deployments in the tenant namespace can use this and elevate privileges to gain cluster wide admin permissions.Describe the improvement
Add an option to use existing service account for Helm deployment to avoid trying to create
ClusterRoleBinding
.Additional context
The tenant namepaces we have are locked down and are not allowed to do stuff on a cluster level. For CI/CD we use Flux, and for security, we use OPA Gatekeeper (Azure Policies). Now, we have a constraint that requires that if you deploy with Flux and use a
HelmRelease
, there must be a service account specified in theHelmRelease
, so for that reason we have default service accounts in each tenant namespace that are able to do everything in that namespace, plus manage CRDs on a cluster level.The text was updated successfully, but these errors were encountered: