Homelab notes: a tiny k3s cluster that survives me

I keep a small k3s cluster at home on three second-hand mini PCs. It exists for one reason: to be broken. Production clusters at work are not the place to learn what happens when you drain the wrong node, and a homelab where breakage is expected removes all the fear from experimenting.

This post is the English-only kind of note — lab logs don’t always get translated. The system handles that gracefully: no Bulgarian version of this file exists, so no Bulgarian URL exists either.

Bootstrap

k3s makes the control plane a one-liner per node:

# first node
curl -sfL https://get.k3s.io | sh -s - server --cluster-init

# remaining nodes
curl -sfL https://get.k3s.io | K3S_URL=https://10.0.0.10:6443 \
  K3S_TOKEN=$(cat token) sh -s - server

Three servers, embedded etcd, and the cluster tolerates losing any single machine — which matters, because I will unplug one to use the outlet for a vacuum cleaner.

GitOps or it didn’t happen

Everything that runs in the cluster is declared in a Git repository and reconciled by Flux. The workflow for changing anything is a pull request to myself:

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: grafana
  namespace: monitoring
spec:
  interval: 10m
  chart:
    spec:
      chart: grafana
      sourceRef:
        kind: HelmRepository
        name: grafana
  values:
    persistence:
      enabled: true

The payoff came the day an SSD died. The replacement node joined the cluster, Flux reconciled for twenty minutes, and everything — dashboards, DNS, the works — came back without me restoring anything by hand.

A homelab is not infrastructure. It is a save point for your curiosity.

What it taught me that work could not

• etcd quorum is visceral once you have personally killed two of three nodes and watched the API server stop answering.
• Resource limits matter when your “cloud” totals 24 GB of RAM — a single unbounded Java service can starve the whole lab.
• Backups are a workflow, not a feature. Velero snapshots to a NAS, restore drills monthly, or it’s all theater.

If you have been thinking about one: two or three used mini PCs, k3s, Flux, and a cheap switch. Start there, break things, write down what broke.