Skip to content

adampetrovic/home-ops

BranchesTags

Repository files navigation

Home Operations Repository

A comprehensive Kubernetes-native home infrastructure platform

🚀 GitOps • 🔒 Security-First • 🤖 Fully Automated

License GitHub last commit Kubernetes Talos

📖 Overview

This repository contains the complete infrastructure-as-code (IaC) configuration for my home operations platform. Built on modern cloud-native principles, it demonstrates enterprise-grade practices scaled down for home use, featuring:

  • 🏗️ Kubernetes-Native Architecture: Built on Talos Linux for immutable infrastructure
  • ⚡ GitOps Workflow: Managed by Flux CD for declarative, Git-driven deployments
  • 🔐 Zero-Trust Security: Comprehensive authentication, authorization, and secrets management
  • 🤖 Full Automation: From hardware provisioning to application deployment
  • 📊 Complete Observability: Metrics, logs, traces, and alerting across the stack
  • 🏠 Smart Home Integration: IoT, automation, and media management platform

🎯 Core Principles

  • Infrastructure as Code: Everything defined declaratively in Git
  • GitOps: Git as the single source of truth for cluster state
  • Security by Design: Zero-trust networking, encrypted secrets, automated updates
  • Cloud-Native: Kubernetes-first, microservices architecture
  • Observability: Comprehensive monitoring and alerting
  • Automation: Minimal manual intervention required

🏗️ Infrastructure

Cluster Architecture

The platform runs on a high-availability Kubernetes cluster powered by Talos Linux:

Component Details
OS Talos Linux v1.12.3 - Immutable, API-driven Linux
Kubernetes v1.35.1 - Latest stable Kubernetes
CNI Cilium - eBPF-based networking and security
Ingress Envoy Gateway - Gateway API-based ingress
Nodes 3x Control Plane + 2x Workers
High Availability Virtual IP, distributed etcd, automated failover

🖥️ Hardware Specifications

Device Count CPU Cores RAM OS Data Purpose
Intel NUC12WSHi7 2 i7-1265P 12 (16 threads) 64GB 1TB SSD 1TB NVMe Kubernetes Control Plane
Intel NUC11PAHi7 1 i7-1165G7 4 (8 threads) 64GB 1TB SSD 1TB NVMe Kubernetes Control Plane
Intel NUC11PAHi7 1 i7-1165G7 4 (8 threads) 64GB 1TB SSD 1TB NVMe Kubernetes Worker Node
Minisforum MS-01 1 i9-13900H 14 (20 threads) 96GB 1TB NVMe 2TB NVMe Kubernetes Worker Node
Synology RS1219+ 1 Atom C2538 - 4GB - 6×16TB NAS Storage
Synology DVA1622 1 Atom C3508 - 4GB - 2×4TB NVR/Security Cameras
UniFi UXG-Pro 1 - - - - Gateway/Router
UniFi US-48-500W 1 - - - - 48-Port PoE Switch
APC SMC1000I-2UC 1 - - - - UPS Power Management

🌐 Network Topology

  • Management VLAN (VLAN 80): 10.0.80.0/21 - Kubernetes nodes
  • Trusted VLAN (VLAN 10): 10.0.10.0/24 - Home devices, secondary k8s interfaces
  • Cluster Networking:
    • Pod CIDR: 10.69.0.0/16
    • Service CIDR: 10.96.0.0/16
    • LoadBalancer VIP: 10.0.80.99

🚀 Applications

The platform hosts 70+ applications across multiple categories:

🏠 Home Automation

📺 Media Management

🛠️ Productivity & Tools

🗄️ Database & Storage

  • CloudNative-PG - PostgreSQL operator
  • PgAdmin - PostgreSQL administration
  • Dragonfly - In-memory data store, drop-in Redis replacement
  • Garage - Distributed S3-compatible object storage

🔐 Security & Authentication

🌐 Networking & DNS

  • Cilium - eBPF-based CNI and security
  • Envoy Gateway - Gateway API-based ingress (Internal + External gateways)
  • Cloudflared - Secure tunnels to Cloudflare
  • External DNS - Automatic DNS record management (Cloudflare for external, UniFi for internal)
  • CoreDNS - Cluster DNS with conditional forwarding for internal domains
  • AdGuard Home - Network-wide ad blocking
  • Multus - Multiple network interfaces
  • NTPd - Network time protocol server
  • SMTP Relay - Outbound email service
  • Echo Server - HTTP echo/debug server

📊 Observability & Monitoring

💾 Storage Management

  • Rook-Ceph - Distributed block and object storage
  • OpenEBS - Local persistent volumes
  • VolSync - Volume backup and synchronization
  • Snapshot Controller - Volume snapshot management
  • Kopia - Backup repository web UI

⚙️ System Services

🤖 CI/CD & Automation

  • Actions Runner Controller - Self-hosted GitHub Actions runners
    • Enables image pre-pulling to Talos nodes via talosctl
    • Scales 0-3 runners dynamically based on workflow demand
    • Authenticates via GitHub App with cluster-admin and os:admin permissions

🏛️ Architecture

GitOps Workflow

graph TD
    A[Developer] -->|Git Push| B[GitHub Repository]
    B -->|Webhook| C[Flux CD]
    C -->|Pull Changes| B
    C -->|Apply Manifests| D[Kubernetes Cluster]
    D -->|Sync Status| C
    E[Renovate Bot] -->|Dependency Updates| B
    F[External Secrets] -->|Fetch Secrets| G[1Password]
    F -->|Create K8s Secrets| D
Loading

Flux CD continuously monitors the Git repository and automatically applies changes to the cluster:

  1. Source Controller - Monitors Git repositories and Helm charts
  2. Kustomize Controller - Applies Kustomize configurations
  3. Helm Controller - Manages Helm releases
  4. Image Automation - Automatically updates container images

Security Architecture

graph TD
    A[Internet] -->|HTTPS| B[Cloudflare]
    B -->|Cloudflare Tunnel| C[Envoy Gateway]
    A -->|HTTPS| C
    C -->|Ext Auth| D[Authelia]
    D -->|LDAP Auth| E[LLDAP]
    D -->|Authorized| F[Application]
    G[External Secrets] -->|API| H[1Password Connect]
    G -->|K8s Secrets| F
Loading
  • Zero-Trust Network: All traffic encrypted and authenticated
  • Multi-Factor Authentication: TOTP, WebAuthn, and Duo support
  • Secrets Management: Encrypted at rest with SOPS, fetched from 1Password
  • Certificate Management: Automated TLS with Let's Encrypt
  • Network Policies: Microsegmentation with Cilium

Storage Strategy

graph TD
    A[Applications] -->|RWO Volumes| B[Rook-Ceph RBD]
    A -->|RWX Volumes| C[Rook-Ceph FS]
    A -->|Local Volumes| D[OpenEBS LocalPV]
    B -->|Backup| E[VolSync]
    C -->|Backup| E
    E -->|Kopia| F[NFS Repository]
    E -->|Restic| G[Cloudflare R2]
    H[NAS] -->|NFS| A
Loading
  • Distributed Storage: Rook-Ceph across all nodes for redundancy
  • Local Storage: OpenEBS for high-performance local volumes
  • Network Storage: NFS mounts from Synology NAS
  • Backup Strategy: Dual-storage — Kopia hourly to NFS, Restic daily to Cloudflare R2 for off-site disaster recovery

Networking Deep Dive

  • CNI: Cilium with eBPF for high-performance networking (deployed without kube-proxy)
  • Load Balancing: Cilium LB IPAM for bare-metal LoadBalancer services
  • Ingress: Envoy Gateway with dual Gateways (internal + external) using the Kubernetes Gateway API
  • DNS: AdGuard Home for network-wide filtering, UniFi for internal DNS records, Cloudflare for external DNS, CoreDNS with conditional forwarding for in-cluster resolution
  • Multi-Homing: Multus CNI for additional network interfaces (IoT VLAN access)
  • Tunnel: Cloudflared for secure external access through Cloudflare

🔧 Operations & Automation

Task Automation

The repository includes comprehensive Taskfile automation:

# Cluster operations
task talos:generate           # Generate Talos configuration
task talos:apply              # Apply Talos configuration
task talos:bootstrap          # Bootstrap new cluster
task talos:fetch-kubeconfig   # Generate talos kubeconfig
task talos:upgrade            # Upgrade Talos on a node (requires: node=<ip>)
task talos:upgrade-rollout    # Rolling Talos upgrade on all nodes
task talos:upgrade-k8s        # Upgrade Kubernetes version (requires: node=<ip> to=<version>)
task talos:reboot-node        # Reboot node (requires: IP=<ip>)
task talos:nuke               # Reset nodes to maintenance mode (DESTRUCTIVE!)

# Volume backup operations
task volsync:check            # Check volsync repo (requires: app=<name>)
task volsync:debug            # Debug restic (requires: app=<name>)
task volsync:list             # List snapshots (requires: app=<name>)
task volsync:unlock           # Unlock restic repository (requires: app=<name>)
task volsync:snapshot         # Create snapshot (requires: app=<name>)
task volsync:restore          # Restore from snapshot (requires: app=<name>)
task volsync:cleanup          # Delete volume populator PVCs

# Kubernetes operations
task k8s:delete-failed-pods   # Delete pods with failed status

Upgrade Procedures

  • Talos OS: Automated rolling upgrades via Tuppr or manual via task talos:upgrade node=<ip>
  • Kubernetes: Automated via Tuppr or manual coordinated upgrades following compatibility matrix
  • Applications: Automated via Renovate bot + Flux CD
  • Full documentation: See docs/UPGRADE.md

Disaster Recovery

Complete cluster rebuild capability:

  1. Hardware Reset: PXE boot into Talos maintenance mode
  2. Cluster Bootstrap: Automated via bootstrap scripts
  3. Backup Restoration: VolSync automatically restores from Kopia (NFS) or Restic (R2) snapshots
  4. Full documentation: See docs/BOOTSTRAP.md

📁 Repository Structure

📁 kubernetes/
├── 📁 apps/              # Application deployments organized by namespace
│   ├── 📁 actions-runner-system/ # Self-hosted GitHub Actions runners
│   ├── 📁 automation/    # Home automation stack
│   ├── 📁 cert-manager/  # Certificate management
│   ├── 📁 database/      # Database services
│   ├── 📁 default/       # Default namespace apps (atuin, memos, etc.)
│   ├── 📁 external-secrets/ # Secrets management with 1Password
│   ├── 📁 flux-system/   # Flux operator and instance
│   ├── 📁 kube-system/   # Core cluster services (cilium, metrics, etc.)
│   ├── 📁 media/         # Media management applications
│   ├── 📁 network/       # Networking and DNS services
│   ├── 📁 observability/ # Monitoring and logging
│   ├── 📁 openebs-system/ # OpenEBS storage
│   ├── 📁 rook-ceph/     # Rook-Ceph distributed storage
│   ├── 📁 security/      # Authentication and security
│   ├── 📁 storage/       # Garage object storage
│   ├── 📁 system-upgrade/ # Automated Talos/K8s upgrades (Tuppr)
│   └── 📁 volsync-system/ # Volume backup services
├── 📁 components/        # Reusable Kustomize components
│   ├── 📁 authelia-proxy/ # Authelia ext-auth security policy
│   ├── 📁 common/        # Common configurations
│   └── 📁 volsync/       # VolSync components
└── 📁 flux/              # Flux system configuration
    └── 📁 cluster/       # Cluster-wide configurations

📁 talos/                 # Talos Linux configuration
├── talconfig.yaml        # Node definitions (managed by talhelper)
├── talenv.yaml           # Talos environment vars
├── talsecret.yaml        # Talos secrets (encrypted)
├── 📁 clusterconfig/     # Generated cluster configs (do not edit)
└── 📁 patches/           # Configuration patches
    ├── 📁 controller/    # Controller-specific patches
    └── 📁 global/        # Global patches

📁 bootstrap/             # Initial cluster bootstrapping
├── helmfile.yaml         # Helmfile for bootstrapping
└── resources.yaml.j2     # Template for resources

📁 scripts/               # Helper scripts
└── 📁 lib/               # Script libraries

📁 docs/                  # Documentation
├── BOOTSTRAP.md          # Bootstrap procedures
├── NODE-REPLACEMENT.md   # Node replacement guide
└── UPGRADE.md            # Upgrade procedures

📁 .taskfiles/            # Task automation scripts
├── 📁 Kubernetes/        # Kubernetes tasks
├── 📁 Talos/             # Talos tasks and scripts
└── 📁 VolSync/           # VolSync tasks and templates

Taskfile.yaml             # Main task definitions

Application Organization

Each application follows a consistent structure:

app-name/
├── app/                     # Application manifests
│   ├── helmrelease.yaml     # Helm chart configuration
│   ├── kustomization.yaml   # Kustomize configuration
│   ├── externalsecret.yaml  # Secret management (if needed)
│   └── configs/             # Additional config files (optional)
└── ks.yaml                  # Flux Kustomization

🚀 Getting Started

Prerequisites

  • Hardware: Minimum 3x bare-metal servers or VMs with 16GB+ RAM
  • Network: VLAN-capable switch and router/firewall
  • DNS: Domain name with Cloudflare DNS management (external), UniFi gateway for internal DNS
  • Secrets: 1Password account for secrets management
  • Tools: talosctl, kubectl, flux, task, age (for SOPS)

Quick Start

  1. Fork this repository and customize for your environment
  2. Configure secrets: Set up SOPS age key and 1Password Connect
  3. Prepare hardware: Install Talos Linux on your nodes
  4. Bootstrap cluster: 
    ./scripts/bootstrap-cluster.sh
  5. Monitor deployment: Applications will automatically deploy via GitOps

Configuration Areas

Key files to customize for your environment:

  • talos/talconfig.yaml - Hardware and network configuration
  • kubernetes/components/common/vars/cluster-settings.yaml - Cluster-wide configuration
  • kubernetes/components/common/vars/cluster-secrets.sops.yaml - Encrypted secrets

☁️ Cloud Dependencies

Service Purpose Cost
1Password Secrets management via External Secrets ~$100/year
Cloudflare DNS, CDN, R2 storage, and secure tunnels Free
GitHub Source control and CI/CD Free
Total ~$8/month

🤝 Community & Inspiration

This repository builds upon the excellent work of the k8s-at-home community. Special thanks to:

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

⭐ If you find this repository helpful, please consider giving it a star!

🐛 Report Bug • 💡 Request Feature • 💬 Discussions

About

HomeOps driven by Kubernetes and GitOps using Flux

Topics

kubernetes talos renovate gitops k8s-at-home kubesearch

Resources

Readme

License

MIT license
Activity

Stars

25 stars

Watchers

0 watching

Forks

2 forks
Report repository

Contributors

Languages

Generated from onedr0p/cluster-template