Kubernetes clusters using the GitOps tool Flux.
The Git repository is the driving the state of the Kubernetes clusters.
The awesome Flux SOPS integration is used to encrypt secrets with gpg.
The following apps are installed on the clusters.
| Software | Purpose |
|---|---|
| Flux2 | GitOps Tool managing the cluster |
| NGINX Ingress Controller | Cluster Ingress controller |
| Rook-Ceph | Cloud native distributed block storage for Kubernetes |
| MetalLB | Bare metal LoadBalancer |
| Cert-Manager | Letsencrypt certificates with Cloudflare DNS |
| ExternalDNS | Configure Cloudflare DNS Servers |
| kube-vip | Virtual IP Load-Balancer for Control Plane High Availability |
| Kube-Prometheus Stack | Prometheus & Exporters to monitor the cluster |
| Grafana | Monitoring & Logging Dashboard |
| Alertmanager | Monitoring Alerts |
| Grafana Loki | Log aggregation system |
| System Upgrade Controller | Automated k3s upgrades |
| Descheduler | Evicts pods to optimize scheduling |
| Kubernetes Metrics Server | Source of container resource metrics for Kubernetes |
While most of my infrastructure and workloads are self-hosted I do rely upon the cloud for certain key parts of my setup. This saves me from having to worry about two things. (1) Dealing with chicken/egg scenarios and (2) services I critically need whether my cluster is online or not.
| Service | Use | Cost |
|---|---|---|
| 1Password | Secrets with External Secrets | ~€80/yr |
| NetCaup | Domain | ~€5/yr |
| GitHub | Hosting this repository and continuous integration/deployments | Free |
| Total: ~€7/mo |
| Name | Device | CPU | OS Disk | Data Disk | RAM | OS | Purpose |
|---|---|---|---|---|---|---|---|
| kube-01 | Dell 5090 | i5-10500T | 256 GB NVME | 128 GB SSD | 32 GB | Debian 12 | k8s control-plane, worker |
| kube-02 | Dell 5090 | i5-10500T | 256 GB NVME | 128 GB SSD | 32 GB | Debian 12 | k8s control-plane, worker |
| kube-03 | Dell 5090 | i5-10500T | 256 GB NVME | 128 GB SSD | 32 GB | Debian 12 | k8s control-plane, worker |
| Device | Purpose |
|---|---|
| Lenovo M920q (opnsense) | Network - Router |
| MikroTik CRS328-24P-4S+ | Network - Switch |
| Cyberpower USV OR600ERM | UPS |
Renovate Bot makes sure the components are never outdated.
It creates PullRequests when Helm charts or Docker images have newer versions available and even keeps Flux and k3s up-to-date.
Big shout out to k8s@home and everyone from awesome-home-kubernetes for the inspiration ❤️