Skip to content

cschmittiey/k8s-homelab

Repository files navigation

A logo that merges the kubernetes logo and a house.

My Home Operations Repository :octocat:

... managed with Flux, Renovate, and GitHub Actions 🤖


📖 Overview

This is a mono repository for my home infrastructure and Kubernetes cluster. I try to adhere to Infrastructure as Code (IaC) and GitOps practices using tools like Kubernetes, Flux, Renovate, and GitHub Actions.


⛵ Kubernetes

There is a template over at onedr0p/cluster-template that I used to create this repository. It is a good starting point for creating a new repository for your own cluster. Also consider joining the Home Operations discord!

Installation

My Kubernetes cluster is deployed with Talos. This is a semi-hyper-converged cluster - workloads and block storage are sharing the same available resources on my nodes, while I have a separate truenas server using ZFS for NFS/SMB shares, bulk file storage and backups.

Core Components

a screenshot of the output of kubefetch. talos is my k8s distro, 2 nodes for now, longhorn for storage, flux for gitops, and cilium for cni. shoutout to my screen reader users. stay cool out there.

  • cert-manager: Creates SSL certificates for services in my cluster.
  • cilium: Internal Kubernetes container networking interface.
  • cloudflared: Enables Cloudflare secure access to certain ingresses.
  • external-dns: Automatically syncs ingress DNS records to a DNS provider.
  • SOPS: encrypted Kubernetes secrets using age
  • ingress-nginx: Kubernetes ingress controller using NGINX as a reverse proxy and load balancer.
  • sops: Managed secrets for Kubernetes and Terraform which are commited to Git.
  • spegel: Stateless cluster local OCI registry mirror.
  • volsync: Backup and recovery of persistent volume claims.

GitOps

Flux watches the clusters in my kubernetes folder (see Directories below) and makes the changes to my clusters based on the state of my Git repository.

The way Flux works for me here is it will recursively search the kubernetes/apps folder until it finds the most top level kustomization.yaml per directory and then apply all the resources listed in it. That aforementioned kustomization.yaml will generally only have a namespace resource and one or many Flux kustomizations (ks.yaml). Under the control of those Flux kustomizations there will be a HelmRelease or other resources related to the application which will be applied.

Renovate watches my entire repository looking for dependency updates, when they are found a PR is automatically created. When some PRs are merged Flux applies the changes to my cluster.

Directories

This Git repository contains the following directories under Kubernetes.

📁 kubernetes
├── 📁 apps           # applications
├── 📁 bootstrap      # bootstrap procedures
├── 📁 flux           # core flux configuration
└── 📁 templates      # re-useable components

Networking

I'm using Cilium as my CNI, pod-gateway for routing traffic through VPNs when needed, and cloudflared to get my services exposed without needing to port forward. (although i do anyways, lol)


☁️ Cloud Dependencies

While most of my infrastructure and workloads are self-hosted I do rely upon the cloud for certain key parts of my setup. This saves me from having to worry about two things. (1) Dealing with chicken/egg scenarios and (2) services I critically need whether my cluster is online or not.

The alternative solution to these two problems would be to host a Kubernetes cluster in the cloud and deploy applications like Vault, Vaultwarden, ntfy, and Gatus. However, maintaining another cluster and monitoring another group of workloads is a lot more time and effort than I am willing to put in.

Service Use Cost
Cloudflare DNS, Argo Tunnels Free
GitHub Hosting this repository and continuous integration/deployments Free
NextDNS My router DNS server which includes AdBlocking ~$20/yr
Total: ~$20/yr

🌐 DNS

I'm using external-dns to manage my dns records in cloudflare. Works great!


🔧 Hardware

Device Count Disk Size Ram Operating System Purpose
Intel NUC7i5BNK 1 1TB SSD 32GB Talos getting started node
Intel NUC8i7BEH 1 1TB SSD 32GB Talos thank u klipper ❤️
Homebuilt PC (Ryzen 5600G) 1 2TB SSD 32GB Talos the real workhorse
HP T740 Thin Client 2 1TB SSD varies Talos what a steal for $50!

🤝 Gratitude and Thanks

Special thanks goes out to onedr0p for his awesome template and readme which i have shamelessly stolen and tweaked.

Thanks to all the people who donate their time to the Home Operations Discord community. Be sure to check out kubesearch.dev for ideas on how to deploy applications or get ideas on what you may deploy.