System Overview¶

Architectural overview of the fzymgc-house self-hosted Kubernetes cluster.

High-Level Architecture¶

graph TB
    subgraph Internet["Internet"]
        Users["Users"]
        CF["Cloudflare"]
    end

    subgraph Hardware["Hardware Layer"]
        TPI_A["TuringPi Alpha"]
        TPI_B["TuringPi Beta"]
    end

    subgraph Kubernetes["Kubernetes Layer (k3s)"]
        CP["Control Plane<br/>tpi-alpha-1,2,3"]
        Workers["Workers<br/>tpi-alpha-4, tpi-beta-1-4"]
    end

    subgraph Platform["Platform Services"]
        Vault["Vault<br/>Secrets"]
        Authentik["Authentik<br/>SSO/OIDC"]
        Traefik["Traefik<br/>Ingress"]
        ArgoCD["ArgoCD<br/>GitOps"]
    end

    subgraph Apps["Applications"]
        Grafana["Grafana"]
        VM["VictoriaMetrics"]
        Other["Other Apps"]
    end

    Users --> CF --> Traefik
    Traefik --> Authentik
    Traefik --> Apps
    TPI_A --> CP
    TPI_B --> Workers
    CP --> Platform
    Workers --> Apps
    Vault --> Platform
    Vault --> Apps
    ArgoCD --> Apps

Three-Layer Architecture¶

Layer	Location	Tool	Purpose
1. Cluster Deployment	`ansible/`	Ansible	Node configuration, k3s installation
2. Infrastructure Config	`tf/`	Terraform	Vault, Authentik, Grafana setup
3. Application Deployment	`argocd/`	ArgoCD	GitOps-driven app manifests

graph LR
    subgraph L1["Layer 1: Ansible"]
        A1["Node Config"]
        A2["k3s Install"]
        A3["CNI Setup"]
    end

    subgraph L2["Layer 2: Terraform"]
        T1["Vault Config"]
        T2["Authentik Setup"]
        T3["Grafana Config"]
    end

    subgraph L3["Layer 3: ArgoCD"]
        K1["Helm Charts"]
        K2["Kustomize"]
        K3["Raw Manifests"]
    end

    A1 --> A2 --> A3 --> T1 --> T2 --> T3 --> K1 & K2 & K3

Layer 1: Ansible¶

Handles physical node configuration and k3s installation.

Role	Purpose
`k3s-server`	Control plane deployment
`k3s-agent`	Worker node deployment
`kube-vip`	API server HA with VIP
`calico`	CNI networking
`longhorn-disks`	Storage preparation

Layer 2: Terraform¶

Configures infrastructure services via HCP Terraform.

Module	Purpose
`cluster-bootstrap`	Initial secrets and RBAC
`vault`	Secrets management policies
`authentik`	Identity provider configuration
`grafana`	Dashboards and datasources
`cloudflare`	DNS records and tunnels
`core-services`	Shared service configuration
`hcp-terraform`	HCP Terraform workspace management

Layer 3: ArgoCD¶

GitOps-driven application deployment.

Pattern	Use Case
Helm charts	Complex applications (Grafana, Vault)
Kustomize	Environment overlays
Raw manifests	Simple resources

Hardware Architecture¶

TuringPi 2 Cluster¶

Two TuringPi 2 boards with RK1 compute modules:

graph TB
    subgraph Alpha["TuringPi Alpha"]
        A1["tpi-alpha-1<br/>Control Plane"]
        A2["tpi-alpha-2<br/>Control Plane"]
        A3["tpi-alpha-3<br/>Control Plane"]
        A4["tpi-alpha-4<br/>Worker"]
    end

    subgraph Beta["TuringPi Beta"]
        B1["tpi-beta-1<br/>Worker"]
        B2["tpi-beta-2<br/>Worker"]
        B3["tpi-beta-3<br/>Worker"]
        B4["tpi-beta-4<br/>Worker"]
    end

    VIP["kube-vip<br/>192.168.20.140"]
    A1 & A2 & A3 --> VIP

Board	Nodes	Role	Count
Alpha	1-3	Control Plane	3
Alpha	4	Worker	1
Beta	1-4	Workers	4

Hardware Specifications¶

Component	Specification
Compute	Turing RK1 (RK3588, 8-core ARM)
Memory	32GB per node
Storage	NVMe SSD per node
Network	1GbE per node
OS	Armbian 25.08

High Availability¶

Component	HA Strategy
API Server	kube-vip VIP (`192.168.20.140`)
etcd	Embedded in k3s (3-node quorum)
Storage	Longhorn distributed replication
Ingress	MetalLB load balancing

GitOps Deployment Flow¶

sequenceDiagram
    participant Dev as Developer
    participant GH as GitHub
    participant HCP as HCP Terraform
    participant Argo as ArgoCD
    participant K8s as Kubernetes

    Dev->>GH: Push to feature branch
    Dev->>GH: Create PR
    GH->>HCP: Trigger speculative plan
    HCP-->>GH: Post plan results
    Dev->>GH: Merge PR
    GH->>HCP: Trigger apply (tf/ changes)
    HCP->>K8s: Apply Terraform changes
    GH->>Argo: Webhook notification
    Argo->>GH: Pull latest manifests
    Argo->>K8s: Sync applications
    K8s-->>Argo: Report sync status

Deployment Triggers¶

Change Type	Automation
`tf/` changes	HCP Terraform apply
`argocd/` changes	ArgoCD sync
`ansible/` changes	Manual playbook run

Secret Management Flow¶

sequenceDiagram
    participant Vault as HashiCorp Vault
    participant ESO as External Secrets Operator
    participant K8s as Kubernetes Secret
    participant Pod as Application Pod

    Vault->>Vault: Store secret at path
    ESO->>Vault: Authenticate via K8s auth
    Vault-->>ESO: Return secret data
    ESO->>K8s: Create/update Secret
    K8s-->>Pod: Mount as env/volume

    Note over ESO,K8s: Automatic refresh on TTL

Secret Paths¶

Category	Vault Path	Example
Cluster services	`secret/fzymgc-house/cluster/*`	`cluster/authentik`
Infrastructure	`secret/fzymgc-house/infrastructure/*`	`infrastructure/bmc`
Applications	`secret/fzymgc-house/apps/*`	`apps/grafana`

Authentication Flow¶

sequenceDiagram
    participant User
    participant App as Application
    participant Traefik
    participant Authentik
    participant Vault

    User->>App: Access protected resource
    App->>Traefik: Request forwarded
    Traefik->>Authentik: Forward auth check
    Authentik-->>User: Redirect to login
    User->>Authentik: Authenticate
    Authentik->>Authentik: Validate credentials
    Authentik-->>User: Issue OIDC token
    User->>App: Request with token
    App->>Traefik: Forward with auth
    Traefik->>Authentik: Validate token
    Authentik-->>Traefik: Token valid
    Traefik-->>App: Allow request
    App-->>User: Return resource

Authentication Methods¶

Method	Use Case
OIDC	Web applications (Grafana, ArgoCD)
Forward Auth	Services without native SSO
LDAP	Legacy application support
Certificate	Machine-to-machine auth

Network Architecture¶

graph TB
    subgraph External["External"]
        CF["Cloudflare Tunnel"]
        Internet["Internet"]
    end

    subgraph Cluster["Cluster Network"]
        MetalLB["MetalLB<br/>145-149, 155-159"]
        Traefik["Traefik Ingress"]
        Calico["Calico CNI"]
    end

    subgraph Pods["Pod Network"]
        PodCIDR["10.42.0.0/16"]
    end

    subgraph Services["Service Network"]
        SvcCIDR["10.43.0.0/16"]
    end

    Internet --> CF --> Traefik
    MetalLB --> Traefik
    Traefik --> Calico --> Pods
    Pods <--> SvcCIDR

Network	CIDR	Purpose
Cluster nodes	`192.168.20.0/24`	Physical node network
Pod network	`10.42.0.0/16`	Kubernetes pods
Service network	`10.43.0.0/16`	Kubernetes services
MetalLB pools	`192.168.20.145-149`, `192.168.20.155-159`	Load balancer IPs
API VIP	`192.168.20.140`	kube-vip HA

See Network Reference for detailed configuration.

Security Architecture¶

graph TB
    subgraph Identity["Identity Layer"]
        Authentik["Authentik SSO"]
        OIDC["OIDC Provider"]
    end

    subgraph Secrets["Secrets Layer"]
        Vault["HashiCorp Vault"]
        ESO["External Secrets"]
    end

    subgraph Network["Network Layer"]
        Calico["Calico Policies"]
        Traefik["TLS Termination"]
    end

    subgraph Audit["Audit Layer"]
        Loki["Loki Logs"]
        VM["VictoriaMetrics"]
    end

    Authentik --> OIDC --> Services
    Vault --> ESO --> Services
    Calico --> Services
    Traefik --> Services
    Services --> Loki & VM

Domain	Solution	Purpose
Identity	Authentik	SSO via OIDC, LDAP
Secrets	Vault + ESO	Dynamic secret injection
Network	Calico	Pod network policies
Ingress	Traefik	TLS termination, routing
Audit	Loki + VM	Centralized logging/metrics

Design Principles¶

GitOps-First¶

All cluster state is defined in Git. ArgoCD ensures the cluster matches the repository.

Single source of truth: Git repository
Declarative: Desired state, not imperative commands
Automated sync: Changes apply automatically
Audit trail: Git history tracks all changes

Secrets in Vault¶

No secrets stored in Git. All sensitive data lives in Vault.

Dynamic secrets: Generated on-demand with TTL
Kubernetes auth: Pods authenticate via service accounts
Automatic rotation: ESO refreshes secrets automatically
Audit logging: All access logged in Vault

SSO via Authentik¶

Single sign-on for all services with consistent identity.

OIDC standard: Industry-standard authentication
Forward auth: Protect services without native SSO
Centralized policies: One place for access control
MFA support: Additional security when needed

Observability by Default¶

Every service is monitored and logged from deployment.

Metrics: VictoriaMetrics collects from all pods
Logs: Loki aggregates all container logs
Dashboards: Grafana provides visibility
Alerts: Alertmanager notifies on issues

Key Design Decisions¶

Why k3s over k8s?¶

Factor	k3s	Full k8s
Resource usage	~512MB RAM	~2GB RAM
Installation	Single binary	Many components
ARM support	Native	Requires tuning
Updates	Simple	Complex
Trade-off	Less customizable	More flexible

Decision: Resource efficiency on ARM hardware outweighs customization needs.

Why Vault over Sealed Secrets?¶

Factor	Vault	Sealed Secrets
Dynamic secrets	Yes	No
Rotation	Automatic	Manual re-seal
Audit	Built-in	None
UI	Yes	No
Complexity	Higher	Lower

Decision: Dynamic secrets and audit logging justify additional complexity.

Why Authentik over Other SSO?¶

Factor	Authentik	Keycloak	Authelia
Resource usage	Medium	High	Low
Features	Full IdP	Full IdP	Forward auth only
Terraform provider	Yes	Yes	No
UI/UX	Modern	Dated	Minimal
Learning curve	Medium	Steep	Low

Decision: Balance of features, resource usage, and Terraform support.

Why ArgoCD over Flux?¶

Factor	ArgoCD	Flux
UI	Full dashboard	None (Weave add-on)
Multi-tenancy	Built-in	Basic
App-of-Apps	Native	Requires setup
Notifications	Extensive	Basic
Learning curve	Lower	Higher

Decision: UI and multi-tenancy support improve operations.

Topic	Link
Network details	Network Reference
Secrets management	Secrets Reference
Services catalog	Services Reference
Technology stack	Technologies Reference