Support private repositories (ACR) for the docker container

[jamesc]

I'm trying to integrate with ACR which is only a private repository. It looks like there is no ability to provide an ImagePullSecrets into the Pod descriptor. The canonical example is at https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#create-a-pod-that-uses-your-secret

[jamesc]

For AKS+ACR it looks like we can work around this using the service principal created for AKS - https://docs.microsoft.com/en-us/azure/container-registry/container-registry-auth-aks#grant-aks-access-to-acr

So this isn't a showstopper for the AKS usecase but could impact other private docker repository usage

[asymmetric]

@jamesc I wonder if a better approach would be tying the ImagePullSecret to a ServiceAccount, as explained here.

The idea being that since it's possible for a cluster admin to do this, we can avoid adding an additional key to the Habitat type and therefore reduce bloat.

[jamesc]

@asymmetric I think that's a much better idea. Feels like that is just documentation then ? I'll test this on a new cluster in Azure and see if works.

[asymmetric]

@jamesc Did you get a chance to test this?

I'm going to close this as I think there's nothing to be done on our side, but let me know if things don't work and we can reopen.

[irvingpop]

I think this is worth reopening, as it affects pulling containers from any private registry (even dockerhub)

[indradhanush]

@irvingpop Hello. I read through the issue and was unable to understand why would we need to reopen it at the moment. 🤔
Could you please clarify a bit more?

Nevertheless, it appears that there's an unanswered question from above and I'll do my best:

Feels like that is just documentation then ?

The approach of using a ServiceAccount mentioned in the above commment already appears to link to a page that documents the process. Besides, it feels that it's better suited to Kubernetes documentation than the operator. I hope it helps. :)

[irvingpop]

You're probably right that the ServiceAccount route is the "right way", and used by experienced k8s users.

But for newer users, most of the examples out there use imgPullSecret on the deployment. It's not a great experience for users who are learning kubernetes and the habitat-operator together.

[jtimberman]

Hello!

I encountered trouble trying to use private Docker Hub repositories for the applications I'm deploying using the habitat-operator. We're using namespaces to segregate our applications, and we were unable to use the default namespace's service accounts, default and habitat-operator, and had to create those in the namespace, and create the imagePullSecrets in the namespace as well. This is awkward and means if we need to update the secret for some reason (e.g., the Docker Hub account is compromised), we have to roll that out across all the namespaces, rather than one place.

For example, this goes in the Kubernetes resource manifest for every application:

---
apiVersion: v1
data:
  .dockerconfigjson: "base64 encoded string of the file here"
kind: Secret
metadata:
  name: registry-secret
  namespace: APPNAME
type: kubernetes.io/dockerconfigjson
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: APPNAME
imagePullSecrets:
- name: registry-secret
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: habitat-operator
  namespace: APPNAME
imagePullSecrets:
- name: registry-secret

This is inefficient from a long term maintenance perspective. Am I missing something in this? Is there an easier way to accomplish this when using namespaces? I think it would be a nicer user experience to be able to specify the imagePullSecrets on the Habitat resource, and have that get passed into the Pods, or be able to use the secret and service accounts at the default namespace level.