Kubernetes

K3s, a thin distribution, and a platform layer built on top.

Why K3s

The cluster runs K3s, the lightweight, fully conformant Kubernetes distribution. On a fleet of mixed ARM and x86 single-board computers and mini-PCs, the choice is simple: K3s ships as a single binary, supports multi-arch out of the box, and strips out everything I'd otherwise have to disable.

Node Configuration

Every node is provisioned from Git with a repeatable first-boot flow:

  • DietPi as the base OS: minimal, consistent, automated.
  • K3s data directory on local NVMe/SSD, never on the boot SD card.
  • The bundled servicelb, traefik, and local-storage components are disabled because the platform layer replaces them.
  • Storage prerequisites installed per node: open-iscsi, lvm2, nfs-common, nvme-cli.
  • Upstream DNS pointed at the two AdGuard Home appliances.
One control-plane, on purpose

The control-plane is a single Lenovo ThinkCenter M900. Etcd high availability is on the roadmap, not the critical path: for a single home cluster, a fast rebuild beats a rare outage.

The Platform Layer

Everything that makes the cluster useful is a layer on top, deployed and reconciled by ArgoCD:

Layer Tool Purpose
Ingress + Mesh Istio (Gateway API + ambient) All HTTP/HTTPS/TCP traffic via istio-gateway; ambient mTLS between workloads
Load balancing MetalLB L2 pool for bare-metal services
Storage Longhorn Replicated block storage for every persistent volume
Certificates cert-manager Wildcard TLS via Cloudflare DNS-01
Secrets Vault + External Secrets Operator Central secret store, synced into the cluster
GitOps ArgoCD Every application reconciled from Git
Observability Prometheus · Grafana · Loki Metrics and logs, retained and dashboarded
Registry Zot Self-hosted OCI registry for cluster images

More details in Services and Architecture.