Octopus (Self-Hosted PR Review)

Octopus is a self-hosted AI code-review tool. It receives GitHub pull-request webhooks, indexes the repository into a vector store, reviews diffs with an LLM, and posts inline severity-rated comments.

This cluster runs a fork (seanb4t/octopus) carrying a stack of four env-gated patches (see Fork state). LLM and embedding traffic is routed through LiteLLM at llm.fzymgc.house (reached by way of the headroom-apps compression proxy), so private-repo content never leaves the cluster’s governed egress and inherits the OpenRouter ZDR guarantee on the lanes that enforce it.

See ADRs: hl-6nm (split model routing), hl-78b (fork-and-contribute), hl-msh (dedicated statefuls), hl-8g4 (two-hostname split). Epic: hl-0we.

Architecture

Piece	Detail
Image	ghcr.io/seanb4t/octopus (web) + ghcr.io/seanb4t/octopus-migrate, multi-arch, digest-pinned from the fork stack tip
Web	One Deployment in octopus ns; in-process pg-boss review worker (ENABLE_REVIEW_WORKERS=true); dnsConfig ndots:1 (alpine/musl + fzymgc.house search-domain trap)
Schema	octopus-migrate initContainer runs prisma db push --accept-data-loss — octopus is schema-sync, not migration-based (the lone shipped migration cannot migrate deploy from scratch)
Vector store	Dedicated qdrant-octopus (own Longhorn-encrypted PVC); code_chunks at 1024-dim (the octopus-embed lane, bge-m3; EMBEDDING_DIM=1024)
Database	Dedicated octopus DB on the main CNPG cluster (pg-boss queue + app data)
LLM/embeddings	OPENAI_BASE_URL=http://headroom-apps.headroom-apps.svc.cluster.local:8787/v1 — headroom-apps compresses and forwards to LiteLLM at https://llm.fzymgc.house/v1, which validates the octopus virtual key (OPENAI_API_KEY, Vault property litellm_api_key); internal CA trusted via NODE_EXTRA_CA_CERTS=/etc/ssl/fzymgc/fullchain.crt (mounted from the fzymgc-ica1-ca configmap — node needs the root; ca.crt alone fails UNABLE_TO_GET_ISSUER_CERT)
Auth (UI)	Keycloak SSO (Better Auth genericOAuth; provider id intentionally stays authentik for backward compatibility with existing linked accounts — see tf/keycloak/octopus.tf) + magic-link; built-in social logins hidden (NEXT_PUBLIC_DISABLE_SOCIAL_LOGINS, baked). ADMIN_EMAILS gates platform admin and must match the Keycloak-surfaced email
Webhook (public)	octopus-wh.fzymgc.net/api/github/webhook via the Cloudflare tunnel webhook_services entry; HMAC-verified (github_webhook_secret)
Admin UI (internal)	octopus.fzymgc.house (router-hosts, LAN/Tailscale only)
GitOps	ArgoCD Application octopus → argocd/app-configs/octopus/

Review flow

1. GitHub sends a pull_request webhook to octopus-wh.fzymgc.net/api/github/webhook (HMAC-verified).
2. Octopus enqueues a pg-boss process-review job; the in-process worker retrieves relevant code_chunks (dense + sparse RRF) from Qdrant.
3. The review model is called through LiteLLM → OpenRouter (or-deepseek-v4-pro by default, an alias on the octopus key for openrouter/deepseek/deepseek-v4-pro).
4. Findings post as inline comments with severity. Findings at/above the org’s checkFailureThreshold (default critical) submit the review as REQUEST_CHANGES; otherwise COMMENT.

Repo indexing (Create Index / Sync) embeds in batches of EMBEDDING_BATCH_ITEMS (64 here — see Embedding pipeline limits) through LiteLLM’s octopus-embed lane (bge-m3, 1024-dim, an OpenRouter-served model that replaced the decommissioned seattle ollama box).

GitHub App (octopus-fzymgc, app id 3985029)

• Setup URL must be set to https://octopus.fzymgc.house/api/github/callback with “Redirect on update” enabled. Without it, GitHub never redirects back after an install, the signed-state callback never fires, and organizations.githubInstallationId is never written — repos will not sync.
• Always connect installations from octopus (“Connect GitHub” per org), never from GitHub’s settings page (that path has no state and cannot link).
• One octopus org ↔ one GitHub installation (githubInstallationId is unique). A user may own up to 3 octopus orgs; create one per GitHub account/org to be reviewed.
• The webhook secret cannot be read back from GitHub. To rotate: PATCH /app/hook/config (app JWT) + vault kv patch … github_webhook_secret=… with the same value, then force the ESO refresh. Webhook 401s = the two sides disagree.

Model configuration

Models live in the Octopus DB (available_models + org defaults), not GitOps. All are registered with provider=openai — required so octopus’s ai-router uses the OpenAI client (→ LiteLLM) instead of falling back to Anthropic (unregistered, non-gpt-prefixed model IDs default to the Anthropic client and fail).

Every string octopus sends must be on the octopus virtual key’s models list, and the key’s aliases map each legacy string onto the LiteLLM lane that serves it (the lane names are model_name entries in argocd/app-configs/litellm-chart/values.yaml):

Model string (available_models)	Model (LiteLLM lane)	Category	Role
or-deepseek-v4-pro	openrouter/* — aliased to openrouter/deepseek/deepseek-v4-pro	llm	org default review model
or-minimax-m3, or-gemma-4-31b, or-gemma-4-31b-free	openrouter/* — aliased to the matching OpenRouter slug	llm	selectable
ollama/bge-m3 (also bge-m3, and octopus’s hard-coded text-embedding-3-large default)	octopus-embed	embedding	org default embed model (1024-dim)

The full key body, aliases included, is in LiteLLM → Minting and scoping virtual keys → octopus. Avoid model IDs containing codex (octopus routes them to the OpenAI Responses API, which LiteLLM does not serve for this consumer). Do not set an org Anthropic key.

Fork state (seanb4t/octopus)

The fork is a thin staging ground for upstreamable patches (ADR hl-78b). Branches are stacked — each is one upstream-PR-sized feature — and the tip builds the cluster image:

Branch (each stacked on the previous)	Adds	Cluster env
master (tracks upstream)	—	—
feat/configurable-embedding-dim	EMBEDDING_DIM (Qdrant vector size), QDRANT_API_KEY support, db:deploy script, migrator image stage, multi-arch GHCR CI	EMBEDDING_DIM=1024
feat/generic-oidc-auth	Better Auth genericOAuth provider (works with any OIDC IdP) + NEXT_PUBLIC_DISABLE_SOCIAL_LOGINS login-button gating	OIDC_DISCOVERY_URL, OIDC_PROVIDER_ID=authentik, OIDC_CLIENT_ID/SECRET (Vault octopus-oidc, written by tf/keycloak/octopus.tf)
feat/configurable-embedding-batch	EMBEDDING_BATCH_ITEMS (indexer + embeddings-client batch size)	EMBEDDING_BATCH_ITEMS=64
feat/optional-review-branding	DISABLE_REVIEW_BRANDING (omit the “Reviewed by Octopus Review” footer)	DISABLE_REVIEW_BRANDING=true
feat/file-recency-metadata	INDEX_FILE_RECENCY (+ optional INDEX_FILE_RECENCY_WINDOW) — per-file lastModifiedAt chunk payload via a bare blob-less history scan at index time	INDEX_FILE_RECENCY=true
feat/doc-aware-analysis	Structured analysis-chunk selection (doc cap, path diversity, recency) replacing the blind scroll, + ## Documentation Accuracy analysis section + doc-skeptical prompt framing	—
feat/recency-aware-review-context	Age annotations + stale-doc demotion in review context (both the webhook and local-review paths)	—
feat/configurable-max-orgs (tip)	MAX_OWNED_ORGS_PER_USER env-gate for the per-user owned-org cap (default 3; <= 0 = unlimited)	MAX_OWNED_ORGS_PER_USER=0

Image build: the fork’s .github/workflows/build-image.yaml triggers on every stack branch + master, and bakes build-time (NEXT_PUBLIC_*) values: GITHUB_APP_SLUG=octopus-fzymgc, OIDC_PROVIDER_ID=authentik, OIDC_PROVIDER_NAME=Authentik, DISABLE_SOCIAL_LOGINS=true. The cluster Deployment pins web + migrate digests from the tip branch’s build.

Adding fork work: branch from the current tip, add the new branch to the CI trigger list, push (build runs), then repoint the Deployment digests in a cluster PR.

Rebasing on upstream: rebase the stack bottom-up (embedding-dim onto master, then each branch onto its rebased parent), force-push the whole stack, rebuild from the tip, repoint digests.

Upstream contribution plan (hl-0we.4, P4): each stack branch is one upstream PR. Additionally file as issues/docs PRs:

• Self-hosting docs: GitHub App Setup URL + Redirect on update are required for installation linking; schema init is prisma db push (not db:migrate).
• Bug: duplicate-org creation — the (app)/layout.tsx auto-createOrgForUser can run twice around its own redirect (no “already has default org” guard).
• Bug: stale current_org_id cookie pointing at a deleted org hard-loops every page to /complete-profile (pages lack the layout’s orgs[0] fallback).
• Self-host polish: no-op the pubby realtime SaaS (api.pubby.dev) and ElasticSearch sync-log writers when unconfigured (currently constant unhandled-rejection noise; cosmetic — indexing/reviews work without them).

Cluster-specific, never upstreamed: hl-0we.17 (P3) — replace pubby with NATS and ES logging with ClickHouse on a separate private-fork track.

Embedding pipeline limits

Hard-won operating envelope — change any of these and indexing breaks again:

Layer	Setting	Why
octopus batch size	EMBEDDING_BATCH_ITEMS=64	upstream default 512 × ~24k-char chunks made each embeddings request 40–60 s on the old self-hosted bge-m3; 64 keeps every response small and each request short, and the octopus-embed lane’s settings in values.yaml are sized for that batch
encoding_format	must pass through to the lane’s upstream	openai-node silently requests base64; stripping the param makes the SDK mis-decode float arrays 4:1 (1024 → 256 dims, Qdrant rejects). Do not add a drop-encoding_format transformation anywhere on the path

The previous gateway’s 2 MiB response-buffer limit (raised to 32 MiB by a Gateway-scoped policy), its llm-embeddings route timeouts and retry policy, and the seattle ollama box’s launchd tuning no longer apply: the gateway and the box are both decommissioned, and the lane’s timeout and retry behaviour is LiteLLM’s, configured per lane in values.yaml. The retired rows are in this file’s git history.

Operations

Rotate the LiteLLM virtual key

In-place regeneration is not available on this estate; the procedure is revoke and re-mint under the same alias (octopus), with a short outage, and it is documented once in LiteLLM → Rotating a virtual key. In outline: /key/delete the old key, /key/generate the replacement with the same body (the alias, the models list and the aliases map from that key’s section), then write the new value to the property octopus reads:

vault kv patch secret/fzymgc-house/cluster/octopus litellm_api_key="<new key>"
kubectl -n octopus annotate externalsecret octopus-app force-sync="$(date +%s)" --overwrite

octopus’s Deployment carries reloader.stakater.com/auto: "true", so the pod restarts on its own once ESO writes the Secret — confirm with kubectl -n octopus rollout status deploy/octopus rather than restarting by hand (the manual-restart caveat in the runbook is fovea’s, not octopus’s).

Update the fork image

# after the fork tip builds:
docker buildx imagetools inspect ghcr.io/seanb4t/octopus:<sha> | grep Digest
# repoint both digests in argocd/app-configs/octopus/deployment.yaml, PR + merge

Backfill file-recency metadata

lastModifiedAt is stamped on chunks at index time (gated by INDEX_FILE_RECENCY). Repos indexed before the recency rollout read as “age unknown” (no age annotations, no stale-doc demotion) until re-indexed: Cancel + re-index from the repo page. The selection/prompt improvements (doc-aware context, ## Documentation Accuracy) apply regardless; only age-awareness needs the re-index.

Add another repository / GitHub org

Same-account repos: adjust the installation’s repository selection (GitHub side), then Sync in octopus. New GitHub account/org: create a new octopus org (org switcher → Create Organization) → Connect GitHub from that org → install → it links + syncs.

Rollback

kubectl -n octopus scale deploy/octopus --replicas=0
# data is isolated to the octopus DB + qdrant-octopus PVC — no blast radius into
# engram/agent-memory. Uninstall the GitHub App to stop webhooks entirely.

Troubleshooting

Symptom	Check
Webhook deliveries 401	Webhook secret mismatch (App vs Vault) — rotate both sides (see GitHub App section)
Repos don’t appear after install	Installed from GitHub’s settings page (no state) or Setup URL missing — organizations.githubInstallationId null; reconnect via octopus’s Connect GitHub
Index stuck “Indexing…” forever, pod idle	The fire-and-forget index task died without updating status (its progress logs go to the unconfigured pubby/ES, not stdout) — check Failed to index repo in pod logs, then Cancel + re-index
Embed 503/504s	See Embedding pipeline limits; watch the LiteLLM — spend by key alias and litellm-model-latency tiles on the LiteLLM — LLM & key usage HyperDX dashboard filtered to key alias octopus, and the octopus-embed rows in otel_traces
Qdrant “expected dim 1024, got 256”	encoding_format is being stripped en route to ollama (see limits table)
Redirect loop to /complete-profile	Stale current_org_id browser cookie pointing at a deleted org — clear the cookie (do not submit the form: it creates a duplicate org)
Failed to trigger event / sync log to ES log spam	Unconfigured pubby/ElasticSearch — cosmetic (hl-0we.17 tracks the NATS/ClickHouse replacement)
Review never posts	kubectl -n octopus logs deploy/octopus ([reviewer] lines); confirm the model string is on the octopus key’s models list (a missing string is a LiteLLM 403, not a model error) and available_models has it with provider=openai
Image pull fails	ghcr-pull-secret ExternalSecret in octopus ns (private seanb4t image)
ExternalSecret SecretSyncedError after a spec change	ArgoCD SSA spec.data lag — delete the ExternalSecret (ArgoCD self-heals it fresh); deleting only the target Secret is not enough

References

• Secrets: docs/reference/secrets.md (secret/fzymgc-house/cluster/octopus, …/octopus-oidc)
• Manifests: argocd/app-configs/octopus/, Application argocd/cluster-app/templates/octopus.yaml
• Keycloak OIDC app: tf/keycloak/octopus.tf
• Spec/plan: docs/engineering/specs/2026-06-06-octopus-self-host-design.md, docs/engineering/plans/2026-06-06-octopus-self-host.md