Skip to content

Services Catalog

Inventory of the services that run in the fzymgc-house cluster. Every entry names the manifest that defines it. An ArgoCD Application in argocd/cluster-app/templates/ delivers each in-cluster service, unless the entry names another source. secret/ prefixes every Vault path below.

Service URL Namespace Category
Vault vault.fzymgc.house vault Platform
Keycloak id.fzymgc.house keycloak Platform
ArgoCD argocd.fzymgc.house argocd Platform
K8s OIDC RBAC Internal cluster-scoped Platform
oauth2-proxy Internal oauth2-proxy Platform
Temporal temporal.fzymgc.house temporal Application
Mealie mealie.fzymgc.house mealie Application
Miniflux miniflux.fzymgc.house miniflux Application
Octopus octopus.fzymgc.house octopus Application
Dolt SQL Server doltdb.fzymgc.house dolt Application
Karakeep karakeep.fzymgc.house karakeep Application
LiteLLM llm.fzymgc.house (LLM /v1, MCP /<server>/mcp, admin /ui) litellm Application
Agent Memory (Engram) llm.fzymgc.house/engram/mcp agent-memory Application
Kubernetes MCP llm.fzymgc.house/kubernetes/mcp kubernetes-mcp Application
ClickHouse MCP llm.fzymgc.house/clickhouse_ro/mcp clickhouse-mcp Application
Firewalla MCP llm.fzymgc.house/firewalla_ro/mcp firewalla-mcp Application
UniFi Network MCP llm.fzymgc.house/unifi_ro/mcp unifi-mcp Application
Life agent Internal life-agent Application
Headroom headroom.fzymgc.house (Tailscale-internal) headroom-agents Application
hl-assets Uploader drop.fzymgc.net (write), i.hl-assets.dev (read, edge only) hl-assets Application
dns-control dns-control.fzymgc.house dns-control Application
Traefik traefik.k8s.fzymgc.house (dashboard) traefik Infrastructure
Longhorn longhorn.fzymgc.house longhorn-system Infrastructure
cert-manager Internal cert-manager Infrastructure
External Secrets Operator Internal external-secrets Infrastructure
External DNS Internal external-dns Infrastructure
Cloudflared N/A (outbound) cloudflared Infrastructure
Tailscale Internal none (node hosts) Infrastructure
NATS Internal nats Infrastructure
Mosquitto mqtt.fzymgc.house:8883 mosquitto Infrastructure
CNPG pg-main.fzymgc.house postgres Infrastructure
Valkey Internal valkey Infrastructure
Gateway API Internal kube-system Infrastructure
Kubernetes Replicator Internal kube-system Infrastructure
Reloader Internal kube-system Infrastructure
System Upgrade Controller Internal system-upgrade Infrastructure
Velero Internal velero Infrastructure
HyperDX (ClickStack) hyperdx.fzymgc.house clickstack Observability
ClickStack Operators Internal clickstack-operators Observability
Vector Internal vector Observability
OTel Scraper otel-gateway.fzymgc.house otel-scraper Observability
kube-state-metrics Internal kube-state-metrics Observability
Uptime Kuma status.fzymgc.net uptime-kuma Observability
Hubble UI hubble.fzymgc.house hubble Observability
HCP Terraform Operator Internal hcp-terraform GitOps
Actions Runner Controller Internal arc-systems, arc-runners GitOps
Argo CD Image Updater Internal argocd GitOps
Renovate renovate-wh.fzymgc.net (webhook) renovate GitOps
Property Value
URL vault.fzymgc.house
Alt URLs vault-0.fzymgc.house, vault-1.fzymgc.house, vault-2.fzymgc.house (one per pod)
Namespace vault
Chart hashicorp/vault 0.34.1
Ingress Type TCP passthrough (TLS ends at Vault)
Auth Method OIDC (Keycloak)
Manifests argocd/app-configs/vault/
Runbook Vault operations
Property Value
URL id.fzymgc.house
Namespace keycloak
Operator Keycloak operator CRDs and controller 26.6.4 (argocd/app-configs/keycloak-operator/)
Instances 2
Ingress Type Traefik IngressRoute. The admin-allowlist Middleware limits /admin and /realms/master to LAN and Tailscale sources. The Cloudflare tunnel delivers the public path straight to keycloak-service
Auth Method Native (the identity provider)
Database CNPG main, database keycloak, through the keycloak-pooler-rw Pooler
Vault Path fzymgc-house/cluster/keycloak, fzymgc-house/cluster/postgres/users/main-keycloak
Realm Config tf/keycloak/
Property Value
URL argocd.fzymgc.house
Namespace argocd
Install tf/cluster-bootstrap/argocd.tf (Helm release argo-cd, image quay.io/argoproj/argocd:v3.2.1)
Ingress Type Helm-managed Ingress. The tunnel also serves argocd-wh.fzymgc.net for webhooks
Auth Method OIDC (Keycloak, through the Dex connector)
Vault Path fzymgc-house/cluster/argocd, fzymgc-house/cluster/argocd/oidc
Applications argocd/cluster-app/templates/, one file per Application
Property Value
URL Internal only
Namespace Cluster-scoped
Purpose Kubernetes RBAC bindings for Keycloak OIDC groups
Auth Method OIDC (Keycloak)
Manifests argocd/app-configs/k8s-oidc-rbac/
Keycloak Group ClusterRole Access
k8s-admins cluster-admin Full cluster access
k8s-developers edit Edit resources in namespaces
k8s-viewers view Read-only access

Keycloak issues OIDC tokens with group claims. ClusterRoleBindings map those groups to the ClusterRoles above. k3s validates the tokens with its --kube-apiserver-arg=oidc-* flags.

Property Value
URL Internal only
Namespace oauth2-proxy
Chart oauth2-proxy/oauth2-proxy 10.7.0
Auth Method OIDC (Keycloak)
Session Store the valkey ValkeyCluster (see Valkey)
Vault Path fzymgc-house/cluster/oauth2-proxy/oidc, fzymgc-house/cluster/valkey
Manifests argocd/app-configs/oauth2-proxy/

The shared forward-auth provider. middlewares.yaml defines one Traefik forwardAuth Middleware per consumer, each with an allowed_groups gate: temporal-users, longhorn-users, hubble-users and traefik-dashboard-users.

Property Value
URL temporal.fzymgc.house
Namespace temporal
Chart temporal 1.7.0 (server), temporal-worker-controller 0.30.0, temporal-webhook-receiver 3.2.0
Ingress Type Traefik IngressRoute. The tunnel serves discord-wh.fzymgc.net for the webhook receiver
Auth Method Forward-auth (oauth2-proxy, group temporal-users)
Vault Path fzymgc-house/cluster/postgres/users/main-temporal, fzymgc-house/cluster/clickstack (otel_ingest_api_key), fzymgc-house/cluster/ghcr/pull-secret
Database CNPG main: temporal (default store), temporal_visibility (visibility store)
Workers chart from ghcr.io/fzymgc-house/charts, source fzymgc-house/temporal-workers. Argo CD Image Updater moves the worker image
Manifests argocd/app-configs/temporal-server/, argocd/app-configs/temporal-workers/
Node gateway Deployment/temporal-node-gateway in the namespace temporal: Envoy with an OPA sidecar, two replicas, behind Service/temporal-node-gateway-vip on 192.168.250.146:7233 and [fddb:f665:73f7:250::146]:7233. The resolver Pis dial the IPv4 literal. Envoy ends the mTLS of the resolver Pis and admits only a client certificate with a URI SAN under spiffe://fzymgc.house/dns-node-worker/. OPA binds each node to the Temporal namespace dns-control and to its own task queue. Manifests in argocd/app-configs/temporal-server/node-gateway/
Temporal namespaces workflows for the cluster workers, dns-control for dns-control and the node workers (retention 7 days)

Components:

  • temporal-server runs the frontend, history, matching and worker services.
  • temporal-web is the UI for workflow visibility.
  • temporal-admintools holds the CLI for namespace management.
  • temporal-worker-controller manages worker deployments through CRDs.
  • temporal-webhook-receiver turns inbound webhooks into workflow signals.
Property Value
URL mealie.fzymgc.house
Alt URL mealie.k8s.fzymgc.house
Namespace mealie
Image ghcr.io/mealie-recipes/mealie:v3.25.1
Ingress Type Traefik IngressRoute
Auth Method OIDC (Keycloak, native login)
Database CNPG main, database mealie
Vault Path fzymgc-house/cluster/mealie, fzymgc-house/cluster/mealie/oidc, fzymgc-house/cluster/litellm-keys/mealie, fzymgc-house/cluster/postgres/users/main-mealie
Manifests argocd/app-configs/mealie/
Property Value
URL miniflux.fzymgc.house
Alt URL miniflux.k8s.fzymgc.house
Namespace miniflux
Image miniflux/miniflux:2.3.3
Ingress Type Traefik IngressRoute
Auth Method OIDC (Keycloak, OAUTH2_PROVIDER=oidc)
Database CNPG main, database miniflux
Vault Path fzymgc-house/cluster/miniflux, fzymgc-house/cluster/miniflux/oidc, fzymgc-house/cluster/postgres/users/main-miniflux
Manifests argocd/app-configs/miniflux/

RSS reader. The homelab:miniflux skill drives it from an agent session.

Property Value
URL (admin UI) octopus.fzymgc.house (internal)
Webhook (public) octopus-wh.fzymgc.net/api/github/webhook
Namespace octopus
Ingress Type Traefik IngressRoute (UI), Cloudflare tunnel webhook_services (webhook)
Auth Method Keycloak OIDC for the UI (OIDC_DISCOVERY_URL), GitHub App credentials for the API, HMAC signature on the webhook
Image ghcr.io/seanb4t/octopus and ghcr.io/seanb4t/octopus-migrate (fork, pinned by digest)
Database CNPG main, database octopus. A dedicated Qdrant (qdrant/qdrant:v1.19.1) holds embeddings
Vault Path fzymgc-house/cluster/octopus, fzymgc-house/cluster/octopus-oidc, fzymgc-house/cluster/litellm-keys/octopus, fzymgc-house/cluster/postgres/users/main-octopus, fzymgc-house/cluster/ghcr/pull-secret
Runbook Octopus
Manifests argocd/app-configs/octopus/

Self-hosted AI pull request review. LLM calls and embeddings go to the LiteLLM Service in the cluster. Reviews use the octopus-review lane, utility calls use the octopus-utility lane and embeddings use the octopus-embed lane. The two chat lanes allow only providers with zero data retention (ZDR).

Property Value
URL doltdb.fzymgc.house (routed VIP 192.168.250.147 and fddb:f665:73f7:250::147, Service dolt-vip)
Remote API doltdb-remote.fzymgc.house (Traefik IngressRoute to port 50051)
Namespace dolt
Image dolthub/dolt-sql-server:2.3.3
Ports 3306 (MySQL wire protocol), 9418
Ingress Type Routed LoadBalancer dolt-vip, dual-stack. The ClusterIP Service dolt serves the in-cluster scraper
Auth Method MySQL native (password)
Vault Path fzymgc-house/cluster/dolt
Manifests argocd/app-configs/dolt/

Version-controlled SQL database with a MySQL-compatible interface, for Beads, Gastown and AI agents. It gives branch and merge workflows on structured data.

Property Value
URL karakeep.fzymgc.house
Alt URL karakeep.k8s.fzymgc.house
Namespace karakeep
Image ghcr.io/karakeep-app/karakeep:0.33.2, with Meilisearch v1.41.0 and browserless/chrome
Ingress Type Traefik IngressRoute
Auth Method OIDC (Keycloak)
Vault Path fzymgc-house/cluster/karakeep, fzymgc-house/cluster/karakeep/oidc, fzymgc-house/cluster/litellm-keys/karakeep
Manifests argocd/app-configs/karakeep/

Bookmark manager with AI tagging through LiteLLM, full-text search in Meilisearch, and page archiving through browserless Chrome.

Property Value
LLM URL https://llm.fzymgc.house/v1 (OpenAI-compatible: /v1/chat/completions, /v1/embeddings, /v1/models; per-consumer virtual key in Authorization: Bearer)
MCP URL https://llm.fzymgc.house/<server>/mcp, twelve routes, admission by virtual key in x-litellm-api-key: Bearer <key> (see MCP Gateway Clients)
Admin UI https://llm.fzymgc.house/ui (generic OIDC to Keycloak SSO, client in tf/keycloak/litellm.tf)
Namespace litellm
Ingress Type Traefik IngressRoute. The host root is published with a denylist of excluded prefixes (/login, /register, /authorize, /token, the root .well-known documents) and a priority-100 allow for the per-server /.well-known/oauth-protected-resource/<server>/mcp documents
Auth Method LLM and key-scoped MCP: a per-consumer virtual key (/key/generate; budgets, rate limits, object_permission.mcp_servers, or a litellm_unified_access_group for a key that Terraform owns). Identity MCP (engram, kubernetes, dns): the client’s own Keycloak token, relayed to the upstream (oauth_passthrough), obtained by the client through RFC 9728 discovery, RFC 7591 dynamic registration and PKCE
Vault Path fzymgc-house/cluster/litellm (master key, OpenRouter keys, shared upstream and MCP credentials), fzymgc-house/cluster/litellm-keys/master, fzymgc-house/cluster/litellm/oidc, fzymgc-house/cluster/litellm-valkey, fzymgc-house/cluster/postgres/users/main-litellm; each consumer’s own virtual key on fzymgc-house/cluster/litellm-keys/<consumer>
State CNPG main, database litellm (keys, spend, budgets). Auth cache in the litellm-valkey ValkeyCluster in the litellm namespace
Chart litellm-helm 1.105.0-rc.3, image pinned by digest (argocd/app-configs/litellm-chart/; secrets and ingress in argocd/app-configs/litellm/)
Runbooks LiteLLM (operator), MCP Gateway Clients (clients)

The unified LLM and MCP data plane (ADR 2026-08-15). One OpenAI-compatible proxy fronts OpenRouter, Google Gemini and the MCP upstreams. Every consumer authenticates with its own virtual key. The key carries a USD budget, a rate limit and, for MCP, the set of routes it can reach. Spend is attributed per key alias.

LLM lanes (proxy_config.model_list in values.yaml; the lane name is what a client sends as model): the consumer lanes octopus-embed, octopus-review, octopus-utility, engram-embed, engram-summarize, gemini-embed and gpt-4o-mini; the passthrough wildcards openrouter/* and openrouter-zdr/* (any OpenRouter slug, the second with zero data retention enforced by the OpenRouter key of the lane); and the operator-only zdr-probe-embed. A key’s models list is its allow-list, and LiteLLM checks it before aliases resolve, so every string a consumer sends must appear on its key. See Minting and scoping virtual keys.

MCP routes (proxy_config.mcp_servers, twelve entries in three shapes): context7, exa, firecrawl and fal (SaaS, LiteLLM injects the upstream key) and firewalla_ro, clickhouse_ro and clickhouse_rw (in-cluster, LiteLLM injects the upstream bearer) overwrite the client’s Authorization. engram, kubernetes and dns relay the client’s own Keycloak token (oauth_passthrough: true) and hold no upstream credential. deepwiki (SaaS) and unifi_ro (in-cluster) hold no upstream credential and no relay flag. Tool names carry the server name as a prefix (context7-resolve-library-id). clickhouse_rw is the only route that can change state, and only a key scoped to it can reach it.

Identity routes and discovery: an unauthenticated call to engram or kubernetes answers 401 with WWW-Authenticate: Bearer resource_metadata=…/.well-known/oauth-protected-resource/<server>/mcp. That document is the upstream’s own RFC 9728 metadata, relayed by LiteLLM with only resource rewritten. It names Keycloak as the authorization server. The client registers itself against Keycloak (the loopback Trusted Hosts policy is the sole control) and completes PKCE. kubernetes-mcp validates the relayed token against the realm’s JWKS with audience https://llm.fzymgc.house/kubernetes/mcp, and the k3s API server authorizes it as oidc:<email>.

Observability: LiteLLM emits GenAI metrics, logs and spans over OTLP to cs-otel-collector (ClickStack): gen_ai.client.operation.duration, gen_ai.client.response.duration, gen_ai.client.token.usage and gen_ai.usage.cost, labelled by metadata.user_api_key_alias. The cost metric is a cumulative histogram that HyperDX cannot total, so the spend tiles and the spend alert sum litellm.cost.total on the LiteLLM spans instead. The collector scrubs key material (sk-[REDACTED]) from the Traefik log, the uvicorn log and the span attributes.

Client configuration (claude-code and codex):

// LLM: base URL https://llm.fzymgc.house/v1 , api key = your virtual key (api_key on cluster/litellm-keys/claude-code)
// MCP servers (mcpServers) — eight key-scoped routes share one header shape:
"context7": { "type": "http", "url": "https://llm.fzymgc.house/context7/mcp", "headers": { "x-litellm-api-key": "Bearer <your-virtual-key>" } },
"exa": { "type": "http", "url": "https://llm.fzymgc.house/exa/mcp", "headers": { "x-litellm-api-key": "Bearer <your-virtual-key>" } },
// ... firecrawl, fal, deepwiki, firewalla_ro, clickhouse_ro (clickhouse_rw only on a key scoped to it)
// identity routes: the same x-litellm-api-key header and NO static Authorization — Claude Code obtains the
// Keycloak token itself; add them with `claude mcp add … --callback-port <port>` (see the client runbook)
"kubernetes": { "type": "http", "url": "https://llm.fzymgc.house/kubernetes/mcp", "headers": { "x-litellm-api-key": "Bearer <your-virtual-key>" } }
Property Value
MCP URL llm.fzymgc.house/engram/mcp (through LiteLLM, the client’s Keycloak token relayed)
Console URL engram.fzymgc.house (LAN and Tailscale only; Traefik IngressRoute, vault-issuer TLS; the root redirects to /ui)
Namespace agent-memory
Chart ghcr.io/seanb4t/charts/engram, pinned by targetRevision in argocd/cluster-app/templates/agent-memory.yaml. Renovate moves it
Image ghcr.io/seanb4t/engram (private GHCR package), pinned by version tag in argocd/app-configs/agent-memory/values.yaml. Argo CD Image Updater moves the pin, and the pull request waits for a review. See engram
Auth Method MCP: OAuth JWT from Keycloak, issuer https://id.fzymgc.house/realms/fzymgc, audience mcp-public, validated by engram. Console: OIDC auth code through the Keycloak engram-ui confidential client, with its own ENGRAM_UI_ISSUER
Vault Path fzymgc-house/cluster/agent-memory, fzymgc-house/cluster/engram-ui, fzymgc-house/cluster/engram-ui/oidc, fzymgc-house/cluster/litellm-keys/agent-memory, fzymgc-house/cluster/clickstack, fzymgc-house/cluster/agent-memory/b2, fzymgc-house/cluster/ghcr/pull-secret
Storage Qdrant (qdrant-data PVC, longhorn-encrypted), collection memory_v4. A CronJob uploads Qdrant snapshots to B2 (fzymgc-k8s-backups, memory/qdrant-snapshots), and a dedicated Velero schedule backs the namespace up daily at 04:00
Manifests argocd/cluster-app/templates/agent-memory.yaml (values), argocd/app-configs/agent-memory/

Persistent vector-memory store for the AI agent harness. It wraps Qdrant with an MCP server (seanb4t/engram). Embeddings go through the LiteLLM engram-embed lane to gemini-embedding-2. Summaries go through the engram-summarize lane. Typed decisions (memory.decisions, provider Jev through the LiteLLM /openrouter pass-through) and the jev search ranker are on. They attach advisory verdicts to engram spine-review consolidate pairs and rerank every search_memory and search_discovery call. Record text leaves the cluster for TypeSafe without zero data retention. The 2026-09-24 ADR accepts that. Audit capture (memory.search.rerankAudit) is on for the Jev grading window, and off is the steady state. Every reranked search writes one log line with the query text and the candidate ids, ranks and scores to ClickStack. Record content never appears in the line, and no new data leaves the cluster. This query shows the newest audit lines:

SELECT Timestamp, LogAttributes['surface'] AS surface, LogAttributes['outcome'] AS outcome,
LogAttributes['query'] AS query
FROM default.otel_logs
WHERE ServiceName = 'engram' AND Body = 'search rerank audit'
ORDER BY Timestamp DESC LIMIT 5
Property Value
MCP URL llm.fzymgc.house/kubernetes/mcp (through LiteLLM, the client’s Keycloak token relayed)
Namespace kubernetes-mcp
Image ghcr.io/containers/kubernetes-mcp-server:v0.0.66 (pinned by digest)
Auth Method require_oauth = true, audience https://llm.fzymgc.house/kubernetes/mcp, validated against the realm’s JWKS. The k3s API server authorizes the caller as oidc:<email>
Mode read_only = true
Manifests argocd/app-configs/kubernetes-mcp/

The Deployment carries a checksum annotation of its ConfigMap, so a configuration change rolls the pod. A test in tools/litellm-verify/ recomputes the value.

Property Value
MCP URL llm.fzymgc.house/clickhouse_ro/mcp and llm.fzymgc.house/clickhouse_rw/mcp (through LiteLLM, a virtual key scoped to the route)
Namespace clickhouse-mcp
Image ghcr.io/clickhouse/mcp-clickhouse:0.6.0 (pinned by digest)
Deployments clickhouse-mcp-ro (ClickHouse user mcp_readonly) and clickhouse-mcp-rw
Vault Path fzymgc-house/cluster/clickstack, fzymgc-house/cluster/litellm
Manifests argocd/app-configs/clickhouse-mcp/

Queries the ClickStack telemetry tables (default.otel_logs, otel_metrics_*, otel_traces). See ClickStack.

Property Value
MCP URL llm.fzymgc.house/firewalla_ro/mcp (through LiteLLM, a virtual key scoped to firewalla_ro)
Namespace firewalla-mcp
Image ghcr.io/fzymgc-house/firewalla-mcp, pinned by version tag in the kustomization.yaml of the app. Argo CD Image Updater moves the pin
Auth Method Client: LiteLLM virtual key scoped to firewalla_ro (x-litellm-api-key). Gateway to server: a static bearer that LiteLLM injects. Server to MSP: a personal access token
Vault Path fzymgc-house/cluster/firewalla-mcp (msp_id, token, box_id); the bearer the server validates is firewalla_mcp_auth_token on fzymgc-house/cluster/litellm
Source github.com/fzymgc-house/firewalla-mcp (Go, modelcontextprotocol/go-sdk)
Tools Read only: boxes, devices, alarms, flows, rules, target lists
Manifests argocd/app-configs/firewalla-mcp/

Read-only MCP server over the Firewalla MSP API v2 (docs.firewalla.net), so that an agent can triage Firewalla state over MCP instead of SSH.

Property Value
MCP URL llm.fzymgc.house/unifi_ro/mcp (through LiteLLM, a virtual key scoped to unifi_ro)
Namespace unifi-mcp
Image ghcr.io/sirkirby/unifi-network-mcp:0.38.1 (pinned by digest, Renovate moves the pin)
Auth Method Client: LiteLLM virtual key scoped to unifi_ro (x-litellm-api-key). Gateway to server: none. Only the litellm namespace reaches the pod. Server to controller: the UniFi local account automation-readonly
Vault Path fzymgc-house/cluster/unifi-mcp (username, password), a copy of fzymgc-house/infrastructure/unifi/readonly
Source github.com/sirkirby/unifi-mcp (Python, package unifi-network-mcp)
Tools Lazy mode: unifi_tool_index and unifi_execute reach every read tool. Every create, update and delete tool is gated off by policy
Manifests argocd/app-configs/unifi-mcp/

Read-only MCP server over the UniFi Network API of the UCG-Fiber at unifi.fzymgc.house, so that an agent can triage devices, clients, firewall policies and events without the controller UI. The design is in UniFi Network MCP behind LiteLLM.

Property Value
URL Internal (no ingress)
Namespace life-agent
Images ghcr.io/seanb4t/life-agent-hermes (pinned by version tag), ghcr.io/seanb4t/life-agent-sandbox and ghcr.io/seanb4t/life-agent-smokescreen (pinned by the digest of the tag main). All three are signed with keyless cosign. Argo CD Image Updater moves the pins
Chat surface Discord, one permitted user and one channel
Vault Path fzymgc-house/cluster/life-agent, fzymgc-house/cluster/litellm-keys/life-agent, fzymgc-house/cluster/ghcr/pull-secret
Source github.com/seanb4t/life-agent
Runbook Life agent
Manifests argocd/app-configs/life-agent/, argocd/app-configs/life-agent-gateway/

The personal Hermes agent: the Hermes gateway, its shell sandbox and two smokescreen egress proxies. Sean speaks to it in Discord.

Property Value
URL https://headroom.fzymgc.house (Tailscale-internal, not on public DNS)
Namespace headroom-agents
Surface Anthropic /v1/messages
Upstream https://api.anthropic.com (direct)
Run mode --mode cache (preserves Anthropic prefix-cache hits)
Image ghcr.io/fzymgc-house/headroom-otel:0.38.0 (pinned by digest)
Vault Path fzymgc-house/cluster/clickstack (otel_ingest_api_key), fzymgc-house/cluster/ghcr/pull-secret
Runbook Headroom
Manifests argocd/app-configs/headroom-agents/

One single-replica instance compresses LLM context for Claude Code on the operator laptop.

Property Value
URL https://drop.fzymgc.net behind Cloudflare Access over the cloudflared-main tunnel. The tunnel dials the ClusterIP http://hl-assets-uploader.hl-assets.svc.cluster.local:8080 with no Traefik hop
Namespace hl-assets
Workloads Deployment/hl-assets-uploader (replicas: 1, no autoscaler), Service/hl-assets-uploader (ClusterIP 8080)
Image ghcr.io/fzymgc-house/hl-assets-uploader (private GHCR package, dual-arch), pinned by version tag in the kustomization.yaml of the app. Argo CD Image Updater moves the pin
Source apps/hl-assets-uploader/ (Go module, cgo against libheif and libwebp)
Auth Method The pod reads the Cloudflare Access headers without verifying them, and records an unauthenticated upload as such. The CiliumNetworkPolicy admits TCP/8080 from the cloudflared namespace only. Service-token client classes: share.fish, the curl one-liner, the iOS Shortcut, and the uptime monitor (its own pair)
Vault Paths fzymgc-house/cluster/hl-assets/r2 (Secret hl-assets-r2), fzymgc-house/cluster/hl-assets/salt (Secret hl-assets-salt), fzymgc-house/cluster/hl-assets/access-token (the client-class pairs), fzymgc-house/cluster/hl-assets/monitor-token (the monitor’s pair), fzymgc-house/cluster/clickstack (Secret hl-assets-otlp-headers), fzymgc-house/cluster/ghcr/pull-secret
Egress CiliumNetworkPolicy/hl-assets-egress: DNS, TCP/443 to world, and the ClickStack collector on 4318. Port-scoped, not host-scoped
Storage None. Request-scoped spill on a disk-backed emptyDir at /spill, unlinked at creation. The root filesystem is read-only
Operator surface Three takedown verbs on the same binary, run through kubectl exec: unshare (reversible), purge (irreversible, needs a confirmation token) and republish (the restore path). See Taking a link down
Monitoring Two Uptime Kuma monitors, a keyword canary on i.hl-assets.dev/_health/canary and an authenticated /readyz probe, both page the cluster-edge Pushover app. Four R2 usage notification policies watch operation volume. See Monitoring
Runbooks hl-assets Uploader, apps/hl-assets-uploader/PORTABILITY.md (relocation checklist)
Manifests argocd/app-configs/hl-assets/

The write path for hl-assets.dev. A client POSTs a file. The pod strips metadata, transcodes HEIC to WebP, derives a content-addressed object key, writes to R2 and returns the public URL in one request. No client holds an R2 credential.

The read path is not this service. Cloudflare’s edge serves shared links from an R2 custom domain with no origin in a public GET, so this pod has no effect on an existing link. See the split read/write ADR and the edge-policy ADR.

Endpoints:

Path Purpose
POST /api/v1/upload The only functional route. Multipart body
GET /healthz Liveness. The process is alive
GET /readyz Readiness. It makes no network call, so a dependency outage does not restart a healthy process
GET /statusz Object-store reachability, in-flight reservation total, encoder versions. It calls R2, so no kubelet probe reads it

Upload contract:

  • 201 Created is a fresh upload. The body is a JSON object with the public URL. An X-Token header carries the capability for a later takedown. The pod issues it once, in a header only, with no re-issue path.
  • 200 OK is a deduplication hit. The same URL comes back, with no X-Token, and the pod does not record the uploader’s filename.
  • Both carry an X-HL-Assets-Dedup header on both content types.
  • Accept: text/plain returns the bare URL plus one newline.
  • Every refusal is structured JSON with a code from a closed enum. Video and camera RAW get 415, and the body names the detected type.
  • A non-POST request to the upload path gets 404 with Allow: POST.

scripts/share.fish drives the live upload path through Cloudflare Access and prints the public URL. The Go suite in the image build holds the contract above. tools/hl-assets-verify/ covers the operational items.

The salt is not rotatable. Rotating it raises no error and silently stops deduplication. See docs/reference/secrets.md.

Related: the uploader-service ADR (the write path’s decisions) and the credential-isolation ADR (what the R2 token can reach).

Property Value
URL https://dns-control.fzymgc.house (Traefik IngressRoute, Certificate dns-control-public-tls)
Namespace dns-control
Workloads Deployment/dns-control (replicas: 1), Service/dns-control (ClusterIP 8080)
Image ghcr.io/fzymgc-house/dns-control (private GHCR package, dual-arch), pinned by version tag in the kustomization.yaml of the app and in the values.yaml of external-dns. Argo CD Image Updater moves both pins
Source apps/dns-control/ (Go binary with an embedded Svelte UI)
Auth Method Native OIDC in the binary against the Keycloak client dns-control (tf/keycloak/dns_control.tf). Every /api/* route requires the client role dns-reader, which the group dns-readers grants. The operations actions under /api/ops/ also require dns-operator, which the group dns-operators grants
Vault Paths fzymgc-house/cluster/dns-control/oidc (oidc_client_secret, written by Terraform), fzymgc-house/infrastructure/dns/pdns-api (api_key), fzymgc-house/cluster/clickstack (dns_control_reader_password, and the OTLP header Secret dns-control-otlp-headers), fzymgc-house/cluster/ghcr/pull-secret, fzymgc-house/cluster/dns-control (nats_creds, mounted at /etc/dns-control/nats/dns-control.creds), fzymgc-house/cluster/postgres/users/main-dns-control (username and password, rendered into the database URL by the ExternalSecret dns-control-db with sslmode=verify-full)
Egress CiliumNetworkPolicy/dns-control-default: kube-dns on 53, Keycloak on 8443, ClickStack on 8123 and 4318, the pdns API of 192.168.20.6 and 192.168.20.7 on 8081, the CNPG pods of the cluster main on 5432, the NATS servers on 4222 and the Temporal frontend on 7233. No world egress
Storage None in the namespace. The record index lives in memory, and dns-control rebuilds it at each start. Velero excludes the namespace. The audit rows of the operations actions live in the CNPG database dns_control
Operations Temporal namespace dns-control (task queue dns-control), NATS account DNS (stream DNS_EVENTS), CNPG database dns_control (table ops_audit)
Node worker dns-node-worker on each resolver Pi, installed by the Ansible role dns-node-worker. The unit dns-node-worker.service runs the worker as dns-worker and polls the Temporal queue dns-node-<node> through the node gateway. The unit dns-node-worker-pki.service runs Vault Agent, which renews the client certificate. The binary is the GitHub release dns-node-worker-v<version> of this repository
Runbooks dns-control
Manifests argocd/cluster-app/templates/dns-control.yaml, argocd/app-configs/dns-control/

The control plane over the resolver pair. It serves Search, Name detail, Query log and Health. It reads both pdns nodes by IP address and ClickHouse as dns_control_reader, and changes no record. It runs the cache flush, the list refresh, the drain and the undrain of a node through the node workers. It does not read the Terraform state of main-cluster-dns-records until issue #2413 adds a scoped token.

Property Value
Dashboard traefik.k8s.fzymgc.house (forward-auth, group traefik-dashboard-users; argocd/app-configs/service-ingresses/)
Namespace traefik
Chart traefik and traefik-crds, pinned by targetRevision in argocd/cluster-app/templates/traefik.yaml
VIP 192.168.250.145 and fddb:f665:73f7:250::145 (Service traefik-vip, Cilium LB-IPAM, announced over BGP, externalTrafficPolicy: Local)
Entrypoints web (80), websecure (443, HTTP/3), postgres (5432, TCP passthrough for CNPG)
Vault Path fzymgc-house/cluster/clickstack (OTLP export to the ClickStack collector)
Manifests argocd/cluster-app/templates/traefik.yaml, argocd/app-configs/traefik/
Property Value
URL longhorn.fzymgc.house
Namespace longhorn-system
Install tf/cluster-bootstrap/longhorn.tf (Helm release longhorn)
Ingress Type Traefik IngressRoute (argocd/app-configs/service-ingresses/)
Auth Method Forward-auth (oauth2-proxy, group longhorn-users)
Vault Path fzymgc-house/cluster/longhorn/crypto-key, fzymgc-house/cluster/longhorn/b2
Node disks ansible/roles/longhorn-disks/
Property Value
URL Internal only
Namespace cert-manager
Install tf/cluster-bootstrap/cert-manager.tf (Helm release cert-manager)
ClusterIssuers vault-issuer (internal, Vault PKI), cloudflare-acme-issuer (Let’s Encrypt, DNS-01)
Vault Path fzymgc-house/cluster/cert-manager, fzymgc-house/infrastructure/pki/fzymgc-ica1-ca
Manifests argocd/app-configs/cert-manager-config/
Property Value
URL Internal only
Namespace external-secrets
Install tf/cluster-bootstrap/external-secrets.tf (Helm release external-secrets-operator)
Auth Method Vault Kubernetes auth
ClusterSecretStore vault
Property Value
URL Internal only
Namespace external-dns
Chart external-dns 1.22.0
Provider pdns (the PowerDNS authoritative tier on the resolver pair), TXT registry
Sources traefik-proxy, service, gateway-httproute, ingress
Vault Path fzymgc-house/infrastructure/dns/pdns-api
Manifests argocd/app-configs/external-dns/

Publishes A and AAAA records for the Lab names under fzymgc.house. An IngressRoute gets a record only when it carries the external-dns.alpha.kubernetes.io/target annotation. See DNS records and the PowerDNS authoritative tier.

Property Value
URL N/A (outbound tunnel)
Namespace cloudflared
Tunnel fzymgc-house-main, ingress rules in tf/cloudflare/tunnel.tf
Vault Path fzymgc-house/cluster/cloudflared/tunnels/fzymgc-house-main
Manifests argocd/app-configs/cloudflared-main/
Runbook Cloudflare

Public ingress. The tunnel serves id.fzymgc.house, the *.fzymgc.net services and the <service>-wh.fzymgc.net webhook hosts (argocd, discord, octopus, renovate).

Property Value
URL Internal only
Namespace None. The subnet routers run on the node hosts tpi-alpha-4 and tpi-beta-4 and on the NAS
Roles ansible/roles/k8s-tailscale/, ansible/roles/nas-tailscale/ (ansible/tailscale-subnet-routers-playbook.yml)
Auth Method OAuth (Tailscale API)
Vault Path fzymgc-house/infrastructure/k8s/tailscale-auth
Policy tf/tailscale/ (ACL, DNS)

The subnet routers advertise the routes in k8s_tailscale_advertise_routes. nas_tailscale_advertise_routes must stay equal to it.

Property Value
URL Internal only
Namespace nats
Chart nats 2.14.6
Auth Method NKey (Ed25519 signatures), account JWTs
Vault Path fzymgc-house/cluster/nats, fzymgc-house/cluster/mosquitto (bridge credentials)
Storage 10Gi per node (longhorn-encrypted)
Cluster Size 3 replicas
Manifests argocd/app-configs/nats/

Features:

  • JetStream persistence with Raft consensus across the 3-node cluster
  • TLS for client and cluster connections
  • Account-based multi-tenancy. A PostSync Job pushes the accounts
Account Purpose Consumers
SYS System monitoring and administration nats-box, monitoring
SERVICES Cluster service communication Temporal, Mosquitto bridge
DNS The events of dns-control on the stream DNS_EVENTS dns-control
Port Protocol Purpose
4222 NATS Client connections
1883 MQTT MQTT listener (the Mosquitto bridge target)
7777 HTTP Prometheus metrics
8222 HTTP Monitoring endpoint

See NATS Operations for key management and administration.

Property Value
URL mqtt.fzymgc.house:8883 (routed VIP 192.168.250.148 and fddb:f665:73f7:250::148, Service mosquitto-vip)
Namespace mosquitto
Image eclipse-mosquitto:2.1.2-alpine
Ingress Type Routed LoadBalancer mosquitto-vip, dual-stack
Auth Method Username and password (mosquitto_passwd)
Password File /mosquitto/config/passwd, from the Vault key passwd (pre-hashed)
Ports 1883 (plain, cluster-internal), 8883 (TLS)
Bridge nats-bridge to nats.nats.svc:1883, TLS with the NATS CA
Vault Path fzymgc-house/cluster/mosquitto
Manifests argocd/app-configs/mosquitto/

MQTT broker for IoT devices and Home Assistant.

Property Value
URL pg-main.fzymgc.house (Traefik TCP passthrough on the postgres entrypoint, port 5432)
Namespace postgres (operator in cnpg-system)
Chart cloudnative-pg 0.29.0
Auth Method PostgreSQL native (TLS plus password)
Cluster Name main
Instances 3
Storage 50Gi per instance (postgres-storage class)
PostgreSQL ghcr.io/cloudnative-pg/postgresql:18.1-standard-bullseye
Backups Barman Cloud plugin to s3://fzymgc-k8s-backups/postgres/barman/main (Backblaze B2), a base backup every 4 hours, WAL archiving, 15-day retention
Vault Path fzymgc-house/cluster/postgres/users/<role>, fzymgc-house/cluster/postgres/b2
Manifests argocd/app-configs/cnpg/
Database Owner Application
keycloak keycloak Keycloak
litellm litellm LiteLLM
mealie mealie Mealie
miniflux miniflux Miniflux
octopus octopus Octopus
temporal temporal Temporal (default store)
temporal_visibility temporal Temporal (visibility store)
dns_control dns_control dns-control (the audit store of the operations actions)

Connection:

postgres://user:password@main-rw.postgres.svc.cluster.local:5432/database?sslmode=require

Connection pooling for Keycloak. Keycloak connects through a CNPG Pooler (PgBouncer) instead of main-rw, so the pooler absorbs a primary switchover and Keycloak keeps its JDBC socket. This pairs with primaryUpdateMethod: switchover.

Property Value
Pooler keycloak-pooler-rw (namespace postgres)
Service DNS keycloak-pooler-rw.postgres.svc.cluster.local:5432
Instances 2
Type / poolMode rw / session
Auth auto-provisioned (cnpg_pooler_pgbouncer role plus user_search)
  • The pool mode is session, not transaction. Transaction pooling breaks the session-scoped state that Keycloak relies on (advisory locks during Liquibase migrations, prepared statements in the JDBC pool).
  • The client-to-pooler hop requires TLS (client_tls_sslmode: require, encrypt only). The pooler presents the cluster server certificate postgres-main-server-tls, whose SAN covers only main-r, main-ro and main-rw, so a client cannot use sslmode=verify-full against the pooler name.
  • CNPG does not pause PgBouncer on a switchover. For a planned switchover with zero connection errors, set spec.pgbouncer.paused: true on the Pooler first, switch over, then set it back to false.
  • Do not set conn_max_age or conn_health_checks on a pooled consumer. The session-mode pooler already stabilizes the socket, and app-side persistent connections double-pool and pin session slots.
Property Value
URL Internal only
Namespace valkey (operator in valkey-operator-system)
Chart valkey-operator 0.7.0
Cluster ValkeyCluster/valkey, one shard, 2 replicas, TLS from vault-issuer
Consumers oauth2-proxy (sessions). LiteLLM runs its own litellm-valkey cluster in the litellm namespace
Vault Path fzymgc-house/cluster/valkey
Manifests argocd/app-configs/valkey/
Property Value
URL Internal only
Namespace kube-system
Version CRDs v1.6.2 (cilium_gateway_api_crd_version in ansible/roles/cilium/defaults/main.yml)
Purpose Kubernetes Gateway API resources for Cilium

The Cilium Ansible role installs the CRDs, and Cilium uses them. No application on the cluster routes traffic through them.

Property Value
URL Internal only
Namespace kube-system
Source mittwald/kubernetes-replicator
Purpose Replicate Secrets and ConfigMaps across namespaces
Manifests argocd/app-configs/kubernetes-replicator/

Add an annotation to the source Secret or ConfigMap:

replicator.v1.mittwald.de/replicate-to: "namespace-a,namespace-b"

Or use a regular expression:

replicator.v1.mittwald.de/replicate-to-matching: "app-.*"
Property Value
URL Internal only
Namespace kube-system
Chart stakater/reloader 2.2.17
Replicas 2
Purpose Roll Deployments when a ConfigMap or Secret changes

Add an annotation to the Deployment:

reloader.stakater.com/auto: "true"

Or name the resources:

configmap.reloader.stakater.com/reload: "my-configmap"
secret.reloader.stakater.com/reload: "my-secret"
Property Value
URL Internal only
Namespace system-upgrade
Version v0.20.1
Purpose Rolling k3s upgrades through Plan CRDs
Manifests argocd/app-configs/system-upgrade/
Plan Target Version
k3s-server Control plane nodes v1.36.0+k3s1
k3s-agent Worker nodes v1.36.0+k3s1

The plans hold the only pin for the k3s version of the cluster. A bump to the pin upgrades the server nodes first, one at a time, then the agent nodes. The controller cordons and drains each node during its upgrade. Renovate proposes each bump in one attended pull request. The procedure is in docs/operations/k3s.md.

Property Value
URL Internal only
Namespace velero
Chart vmware-tanzu/velero 12.1.0
Purpose Kubernetes backup and disaster recovery
Vault Path fzymgc-house/cluster/velero/b2
Manifests argocd/app-configs/velero/

The backup strategy is exclude-only. Every namespace is backed up unless it is listed.

Schedule Frequency TTL Scope
daily-backup Daily at 02:00 30 days Core resources
weekly-full-backup Sunday at 03:00 90 days Extended resources, including NetworkPolicies
agent-memory-pvc-backup Daily at 04:00 30 days The agent-memory namespace, with the CSI data mover

Excluded namespaces:

  • Kubernetes core: kube-system, kube-node-lease, kube-public, default
  • Networking: traefik
  • Storage: longhorn-system
  • Operators: cert-manager, external-secrets, cnpg-system, external-dns, valkey-operator-system
  • Backup: velero
  • Ephemeral: arc-systems, arc-runners, system-upgrade, hcp-terraform, renovate
  • Telemetry: vector, clickhouse-mcp, firewalla-mcp
Property Value
URL hyperdx.fzymgc.house
ClickHouse ch.fzymgc.house (HTTP), ch-native.fzymgc.house (native protocol, TCP passthrough)
Namespace clickstack
Ingress Type Traefik IngressRoute
Auth Method OIDC (Keycloak SSO with explicit endpoints; client in tf/keycloak/clickstack.tf)
Vault Path fzymgc-house/cluster/clickstack, fzymgc-house/cluster/clickstack/oidc, fzymgc-house/cluster/clickstack-watchdog
OTel Endpoint 192.168.250.149:4317 (routed VIP, Service otel-collector-vip), plaintext with bearer token
Alerts argocd/app-configs/clickstack-alerts/ bootstraps the alert set, paging through Pushover (fzymgc-house/cluster/pushover)
Manifests argocd/app-configs/clickstack/, argocd/app-configs/clickstack-chart/
Runbook ClickStack

The observability stack: logs, metrics and traces in ClickHouse, with HyperDX as the UI. The tables are default.otel_logs, otel_metrics_* and otel_traces, with 30 days of history.

Property Value
URL Internal only
Namespace clickstack-operators
Chart clickhouse.github.io/ClickStack-helm-charts/clickstack-operators 1.1.0
Purpose MongoDB and ClickHouse operators for the ClickStack stack

Bundles the MongoDB Community Operator and the Altinity ClickHouse operator. It reconciles the custom resources in the clickstack namespace, so it must exist before the clickstack Application.

Property Value
URL Internal only
Namespace vector
Chart helm.vector.dev/vector 0.58.0 (app 0.58.0-distroless-libc)
Controller DaemonSet (role: Agent), tolerates the control-plane taint
Purpose Ship pod logs straight to ClickHouse (otel_logs)
Vault Path fzymgc-house/cluster/clickstack
Manifests argocd/cluster-app/templates/monitoring-vector.yaml, argocd/app-configs/monitoring-vector/

Vector does not interpolate environment variables in its configuration, so a secret reaches the configuration as a file: mount the Secret key under /etc/vector-secrets (extraVolumes) and reference it as SECRET[clickhouse.<key>]. Check a change with a real vector run, not vector validate. validate does not resolve the secrets backend and reports a false authentication failure.

Property Value
URL otel-gateway.fzymgc.house (OTLP gRPC ingress on 4317 for off-cluster emitters)
Namespace otel-scraper
Chart opentelemetry-collector 0.174.2
Controllers otel-scrape (Deployment: Prometheus scrape targets and the k8scluster receiver), otel-node (DaemonSet: hostmetrics and kubeletstats per node), otel-network (Deployment, three replicas: syslog from the UniFi gateway, switches and APs on the VIP 192.168.250.150)
Vault Path fzymgc-house/cluster/clickstack (otel_ingest_api_key), fzymgc-house/infrastructure/pki/fzymgc-ica1-ca
Manifests argocd/app-configs/monitoring-otel-scraper/

The three collectors export OTLP gRPC to cs-otel-collector.clickstack.svc.cluster.local:4317. See ClickStack for token rotation.

Property Value
URL Internal only
Namespace kube-state-metrics
Chart kube-state-metrics 8.4.2
Purpose Kubernetes object state as Prometheus metrics, scraped by otel-scrape
Manifests argocd/app-configs/monitoring-kube-state-metrics/
Property Value
URL status.fzymgc.net (Cloudflare tunnel, behind Cloudflare Access, tf/cloudflare/access.tf)
Alt URL status.fzymgc.house redirects to status.fzymgc.net, except /api/push/, which the LAN reaches directly
Namespace uptime-kuma
Image louislam/uptime-kuma:2.5.4
Storage 1Gi Longhorn PVC (SQLite)
Monitors tf/uptime-kuma/
Manifests argocd/app-configs/uptime-kuma/

Service uptime monitoring with a status page.

Property Value
URL hubble.fzymgc.house
Namespace hubble (ingress); the upstream Service hubble-ui is in kube-system
Ingress Type Traefik IngressRoute (cross-namespace Service reference to hubble-ui.kube-system.svc:80)
Auth Method Forward-auth (oauth2-proxy, group hubble-users)
Storage None (read-only flow view from hubble-relay)
Manifests argocd/app-configs/hubble-ui/
Runbook Hubble UI

Cilium network flow dashboard. It shows live pod-to-pod traffic, filters by namespace, verdict and L7 attributes, and draws a service map. Use it during a CiliumNetworkPolicy rollout to make sure that the intended flows pass and the others drop.

Property Value
URL Internal only
Namespace hcp-terraform
Chart hashicorp/hcp-terraform-operator 2.12.1
Purpose HCP Terraform workspace and agent pool management through CRDs
Vault Path fzymgc-house/cluster/hcp-terraform, fzymgc-house/infrastructure/pki/fzymgc-ica1-ca
Manifests argocd/app-configs/hcp-terraform-operator/
Runbook HCP Terraform

Watches Workspace and AgentPool CRDs and drives HCP Terraform to plan and apply the workspaces.

Property Value
URL Internal only
Controller Namespace arc-systems
Runners Namespace arc-runners
Chart gha-runner-scale-set-controller 0.14.2
Scale Set fzymgc-house-cluster-runners for https://github.com/fzymgc-house
Vault Path fzymgc-house/cluster/github
Manifests argocd/app-configs/arc-controller/, argocd/app-configs/arc-runners/

Autoscaling GitHub Actions runner scale sets. The controller in arc-systems manages the runner pods in arc-runners.

Property Value
URL Internal only
Namespace argocd
Chart argo/argocd-image-updater 1.3.1
Purpose Opens the pin pull requests of first-party images
Vault Path fzymgc-house/cluster/argocd-image-updater, fzymgc-house/cluster/ghcr/pull-secret
Manifests argocd/cluster-app/templates/argocd-image-updater.yaml, argocd/app-configs/argocd-image-updater/
Runbook Argo CD Image Updater

It reads GHCR every two minutes. For each new first-party image, it opens a pull request against main that moves the pin of that image.

Property Value
Webhook (public) renovate-wh.fzymgc.net
Namespace renovate
Chart mend-renovate-ce 14.6.2
Scope Autodiscovery filter fzymgc-house/*,seanb4t/*,holomush/*
Vault Path fzymgc-house/cluster/renovate, fzymgc-house/cluster/ghcr/pull-secret
Manifests argocd/app-configs/renovate/

Dependency update bot. It opens the version-bump pull requests in the repositories above.

Service Purpose Management
Cloudflare DNS, tunnels, Access, R2, WAF Terraform (tf/cloudflare)
HCP Terraform Infrastructure automation Terraform workspaces, the operator above
GitHub Source control, Actions Web UI
Let’s Encrypt Public TLS certificates cert-manager
Tailscale Tailnet coordination Terraform (tf/tailscale)
Method Description Configuration
OIDC Direct OpenID Connect login A Keycloak client in tf/keycloak/
Forward-auth A Traefik Middleware proxies the request to the shared oauth2-proxy argocd/app-configs/oauth2-proxy/middlewares.yaml
Virtual key A LiteLLM key with a budget, a rate limit and a route allow-list tf/litellm/
Certificate mTLS client certificate Vault PKI
None No authentication Internal services only
Type Description TLS Handling
Traefik IngressRoute Native Traefik CRD Traefik ends TLS
TCP passthrough Raw TCP proxy (IngressRouteTCP with HostSNI) The backend ends TLS
Helm-managed Ingress defined in Helm values Varies by chart
Cloudflare tunnel Public path through cloudflared Cloudflare ends TLS
Routed LoadBalancer A VIP in the routed VIP prefixes, dual-stack, announced over BGP The Service handles TLS
  1. Create the Kubernetes manifests in argocd/app-configs/<service>/ and an Application in argocd/cluster-app/templates/.
  2. Configure the ingress (IngressRoute or Ingress). For a name that clients resolve on the LAN, add the external-dns.alpha.kubernetes.io/target annotation (see DNS records).
  3. Set up authentication:
    • OIDC: create a Keycloak client in tf/keycloak/.
    • Forward-auth: add a Middleware to argocd/app-configs/oauth2-proxy/middlewares.yaml with an allowed_groups=<app>-users gate.
  4. Add secrets to Vault and update the Vault policy (tf/vault/).
  5. Create an ExternalSecret for the Kubernetes Secret.
  6. Update this catalog and the secrets reference.