toolkit: ARM64

[moikot]

I'm looking for a solution for reconciling the state of my cluster with Git repo. I want, literally, to provision my cluster, install some essential components, like parts of FluxCD toolkit and some secret management tool Hashicorp vault, unseal the vault, point to source-controller to the sources, and everything else should be installed automatically based on the configuration I keep in Git.

I know that I can install source-controller, kustomize-controller, and helm-controller (since all of my services are using Helm) and create CRDs describing the resources I want to monitor and reconcile. The problem is that CRDs like Kustomization and HelmRelease have to be present in the cluster before the reconciliation process could start. I can potentially create one Kustomization CRD pointing to a GIT repo with all other HelmReleases needed for the rest of my services and I hope kustomize-controller will create CRDs and helm-controller reconcile them.

I have to mention here that my cluster is running on arm64 SBCs and the problem with kustomize-controller is that it relies on kustomize and kubectl pulled as binaries and although kubectl can be pulled for arm64, kustomize fails to provide such option. I can easily produce multi-arch images for source-controller and helm-controller but not for kustomize-contoller.

I've just started thinking if I can implement a very stupid controller which just takes all YAML files from a folder in a git repo and applies them to a Kubernetes cluster. Or maybe I'm missing some other option here?

[stefanprodan]

I can potentially create one Kustomization CRD pointing to a GIT repo with all other HelmReleases needed for the rest of my services and I hope kustomize-controller will create CRDs and helm-controller reconcile them.

The tk bootstrap command does the initial setup for you.

I have to mention here that my cluster is running on arm64 SBCs and the problem with kustomize-controller is that it relies on kustomize

We need to drop the kustomize binary and rewrite the controller parts that are shelling out to it. Fixed in fluxcd/kustomize-controller#92

[stefanprodan]

GitHub doesn't have support for multi-arch images and we're moving away from Docker Hub.

[moikot]

I agree that Docker Hub is becoming less and less attractive. And I might be wrong, but it seems like manifest list v2 going to be rolled out quite soon on GHCR, probably at the same time when the build push action v2 is released.

[stefanprodan]

@moikot can you please test the latest gotk release on ARM64?

gotk install --arch=arm64 

[moikot]

@stefanprodan I can confirm that all four controllers are working just fine in my arm64 cluster. Thanks a lot for making it possible!

[stefanprodan]

Thank you for testing it, much appreciate 👍