tristian-gitops

Pull-based GitOps-Kubernetes (fluxcd/flux2-managed) on top of Proxmox VE cluster + Unraid VMs @Home

A highly opinionated setup on deploying HA k3s cluster with Ansible and Terraform backed by Flux and SOPS.

👀 Overview

Stacks:

• cert-manager - SSL certificates - with Cloudflare DNS challenge
• calico - CNI (container network interface)
• echo-server - REST Server Tests (Echo-Server) API (useful for debugging HTTP issues)
• flux - GitOps tool for deploying manifests from the cluster directory
• hajimari - start page with ingress discovery
• kube-vip - layer 2 load balancer for the Kubernetes control plane
• local-path-provisioner - default storage class provided by k3s
• metallb - bare metal load balancer
• reloader - restart pods when Kubernetes configmap or secret changes
• reflector - mirror configmaps or secrets to other Kubernetes namespaces
• system-upgrade-controller - automate upgrading k3s
• traefik - ingress controller

For provisioning:

• Ubuntu - this is a pretty universal operating system that supports running all kinds of home related workloads in Kubernetes
• Ansible - this will be used to provision the Ubuntu operating system to be ready for Kubernetes and also to install k3s
• Terraform - in order to help with the DNS settings this will be used to provision an already existing Cloudflare domain and DNS settings

📝 Prerequisites

🔧 Tools

Tool	Purpose
ansible	Preparing Ubuntu for Kubernetes and installing k3s
direnv	Exports env vars based on present working directory
flux	Operator that manages your k8s cluster based on your Git repository
age	A simple, modern and secure encryption tool (and Go library) with small explicit keys, no config options, and UNIX-style composability.
go-task	A task runner / simpler Make alternative written in Go
ipcalc	Used to verify settings in the configure script
jq	Used to verify settings in the configure script
kubectl	Allows you to run commands against Kubernetes clusters
sops	Encrypts k8s secrets with Age
terraform	Prepare a Cloudflare domain to be used with the cluster

🛠️ Complement

Tool	Purpose
helm	Manage Kubernetes applications
kustomize	Template-free way to customize application configuration
pre-commit	Runs checks pre git commit
gitleaks	Scan git repos (or files) for secrets
prettier	Prettier is an opinionated code formatter.

📂 Repository structure

The Git repository contains the following directories under cluster and are ordered below by how Flux will apply them.

• base directory is the entrypoint to Flux
• crds directory contains custom resource definitions (CRDs) that need to exist globally in your cluster before anything else exists
• core directory (depends on crds) are important infrastructure applications (grouped by namespace) that should never be pruned by Flux
• apps directory (depends on core) is where your common applications (grouped by namespace) could be placed, Flux will prune resources here if they are not tracked by Git anymore

cluster
├── apps
│   ├── default
│   ├── networking
│   └── system-upgrade
├── base
│   └── flux-system
├── core
│   ├── cert-manager
│   ├── metallb-system
│   ├── namespaces
│   └── system-upgrade
└── crds
    └── cert-manager

🤝 Thanks

Big shout out to all the authors and contributors to the projects awesome-home-kubernetes