Skip to content
fzymgc-house Cluster Docs

Self-Host Octopus PR Review (BYOK via agentgateway) — Design

  • Bead: hl-0we
  • Date: 2026-06-06
  • Status: Design (brainstorm → spec)
  • Depends on: hl-0sr (LiteLLM → agentgateway migration) for the LLM/embedding gateway endpoint
  • Upstream: octopusreview/octopus (MIT)

1. Problem & motivation

Section titled “1. Problem & motivation”

We want automated AI code review on fzymgc-house/selfhosted-cluster PRs. The repo is private, unlicensed, 0 stars, so the hosted free-for-OSS tiers of both Qodo Merge (star threshold) and Octopus cloud (public + OSI-licensed requirement) are unavailable. The realistic path is self-hosting Octopus with BYOK, routing its LLM and embedding traffic through the cluster’s own governed LLM gateway (agentgateway, built in hl-0sr) so private-repo content never leaves the cluster’s controlled egress and inherits the OpenRouter ZDR guarantee.

Goals

Section titled “Goals”
  • Octopus runs in-cluster (GitOps via ArgoCD), reviewing PRs on this repo (and optionally other fzymgc-house repos) with inline severity-rated comments.
  • LLM (reviews) and embeddings both flow through agentgateway: reviews via an OpenRouter frontier model (ZDR-enforced), embeddings via self-hosted bge-m3.
  • Minimal, upstreamable code change — fork only as a staging ground for an upstream PR.

Non-goals

Section titled “Non-goals”
  • Multi-tenant / multi-org operation (single org, single admin).
  • GitLab/Bitbucket providers, billing (Stripe), realtime UI (Pubby), full-text search (Elasticsearch), object storage (R2), transactional email — all disabled.
  • Replacing the existing dev-flow/pr-review-toolkit review skills; Octopus is an additional, always-on webhook reviewer.

2. Grounded facts (Rule 7)

Section titled “2. Grounded facts (Rule 7)”

Verified against octopusreview/octopus@master (.env.example, docker-compose.yml, /docs/self-hosting, and source: apps/web/lib/{embeddings,qdrant,ai-client,provider}.ts).

  • Required runtime stack is minimal: web (Next.js/Bun) + Postgres 15+ + Qdrant. Everything else in .env.example (Redis, Elasticsearch, Pubby, R2, Stripe, Resend, social OAuth) is optional/cloud-only. Job queue is pg-boss (rides on Postgres — no broker). The review worker runs in-process when ENABLE_REVIEW_WORKERS=true.
  • GitHub integration is a GitHub App: env GITHUB_APP_ID, GITHUB_APP_PRIVATE_KEY, GITHUB_WEBHOOK_SECRET, GITHUB_STATE_SECRET, NEXT_PUBLIC_GITHUB_APP_SLUG. Webhook endpoint: POST https://<host>/api/github/webhook. Permissions: Pull requests r/w, Contents r, Checks r/w. Events: pull_request, pull_request_review. GitHub OAuth (GITHUB_CLIENT_ID/SECRET) is used separately for user login.
  • Secrets at rest: OCTOPUS_DATA_KEY (openssl rand -hex 32) encrypts OAuth tokens and per-org AI provider keys; decoupled from BETTER_AUTH_SECRET.
  • Embeddings client honors OPENAI_BASE_URL with no code change: embeddings.ts does new OpenAI({ apiKey: process.env.OPENAI_API_KEY! }) with no baseURL passed; the official openai Node SDK falls back to process.env.OPENAI_BASE_URL. So embeddings (and any OpenAI-SDK review path) redirect to agentgateway via env alone.
  • Models are DB-configurable: ai-client.ts resolves review/embed models as repo.{reviewModelId,embedModelId} → org.{defaultModelId,defaultEmbedModelId} → availableModel.isPlatformDefault → hardcoded (claude-sonnet-4-6 / text-embedding-3-large). There is an availableModel registry (category: llm|embedding, modelId, isActive, isPlatformDefault) — models are added in the admin UI/DB, not code.
  • Only hard-coded blocker for bge-m3: qdrant.ts has const VECTOR_SIZE = 3072; (text-embedding-3-large), reused across 7 collections (code_chunks, knowledge_chunks, review_chunks, chat_chunks, flowchart_chunks, feedback_patterns, docs_chunks). bge-m3 is 1024-dim → this constant must become configurable.
  • qdrant.ts QdrantClient is constructed with { url } only — it ignores QDRANT_API_KEY. Wiring it is a one-line optional hardening.

Open plan-time verification

Section titled “Open plan-time verification”

One code path was not located: how a model string maps to a provider/SDK on the review (chat) call. Risk: if claude-* model IDs hard-route to the Anthropic SDK (api.anthropic.com), OPENAI_BASE_URL will not redirect reviews. Mitigation (baked into this design): register an OpenAI-category availableModel whose modelId is an agentgateway alias (mapped to an OpenRouter slug) and set it as the org/repo review model, so the review call takes the openai-SDK path. Confirm at plan time by reading the review chat-completion call site (likely apps/web/lib/review*).

3. Architecture

Section titled “3. Architecture”

Three in-cluster components plus a thin, upstreamable fork.

GitHub PR event                          you (LAN/Tailscale)
      │ webhook, signature-verified             │ admin UI + OAuth login
      ▼ octopus-wh.fzymgc.net (PUBLIC)          ▼ octopus.fzymgc.house (INTERNAL, router-hosts)
Cloudflare Tunnel ───────────────►  octopus web (Svc octopus.octopus:3000)  ◄─── Traefik IngressRoute
                                                 │
                          ┌──────────────────────┼──────────────────────┐
                          ▼                       ▼                      ▼
                   CNPG Postgres               Qdrant               agentgateway
                 (db "octopus" on main)   (qdrant/qdrant,      (OpenAI-compat, in-cluster)
                  pg-boss queue + data      1024-dim PVC)      ├─ reviews → OpenRouter (ZDR)
                                                               └─ embeddings → Ollama bge-m3

3.1 Components

Section titled “3.1 Components”
ComponentSpecReuses
octopus webOne Deployment, image ghcr.io/seanb4t/octopus, ENABLE_REVIEW_WORKERS=true, ENABLE_INTERNAL_CLI=false; ClusterIP Service octopus.octopus.svc:3000; env from ESO secret.argocd/app-configs/litellm/ raw-kustomize layout
QdrantPlain qdrant/qdrant Deployment + Longhorn-encrypted PVC; ClusterIP qdrant-octopus.octopus.svc:6333; EMBEDDING_DIM=1024.minimal raw-kustomize
PostgresCNPG Database CR octopus + managed role; conn string composed in ESO template → main-rw.postgres.svc.cluster.local:5432/octopus. Requires editing argocd/app-configs/cnpg/postgres-cluster.yaml to add the octopus role to managed.roles (every existing user — authentik/grafana/tandoor/litellm — appears there).cnpg/db-tandoor.yaml, cnpg/users-tandoor.yaml, cnpg/postgres-cluster.yaml
Webhook (public)Internet-facing octopus-wh.fzymgc.net via a new webhook_services entry (Terraform webhook_domain=fzymgc.net + webhook_suffix=-wh); the Cloudflare tunnel routes it to the in-cluster octopus origin. GitHub App webhook URL = https://octopus-wh.fzymgc.net/api/github/webhook.webhook_services (today only argocd-wh.fzymgc.net) in tf/cloudflare/
UI (internal)Traefik IngressRoute Host(octopus.fzymgc.house) with router-hosts.fzymgc.house/enabled: "true" (LAN/Tailscale-only DNS, not internet-reachable) + octopus-tls Certificate (vault-issuer). BETTER_AUTH_URL/NEXT_PUBLIC_APP_URL = https://octopus.fzymgc.house.tandoor/ingress.yaml (router-hosts + separate vault-issuer Certificate); dolt/ingress.yaml (router-hosts annotation only)
SecretsESO ExternalSecrets from fzymgc-house/cluster/octopus (app secrets) and fzymgc-house/cluster/postgres/users/main-octopus (DB creds).litellm/agent-memory ESO
ArgoCD appargocd/cluster-app/templates/octopus.yaml, project core-services, automated prune+selfHeal, ServerSideApply, CreateNamespace.tandoor.yaml template

Hostname split (resolves the public-vs-internal tension): the webhook must be internet-reachable and is secured by the GitHub HMAC signature, so it lives on the public octopus-wh.fzymgc.net. The admin UI stays off the internet on the internal octopus.fzymgc.house (router-hosts), protected by Octopus’s own Better Auth. Both hostnames resolve to the same octopus Service. Better Auth’s single canonical URL is the internal host (UI links + GitHub OAuth login callback); the GitHub App’s webhook URL is the public -wh host (its handler validates the signature independently of any Better Auth session). Exact tunnel service_url target for the webhook (direct Service vs. via Traefik websecure, as argocd-wh does) is a plan-time confirmation.

3.2 LLM/embedding wiring

Section titled “3.2 LLM/embedding wiring”
  • OPENAI_API_KEY = an agentgateway virtual key (from Vault); OPENAI_BASE_URL = in-cluster agentgateway OpenAI-compatible endpoint http://agentgateway.agentgateway.svc.cluster.local:3000/v1 (name/namespace/port per the hl-0sr plan; re-confirm against the merged agentgateway manifests at go-live). No tunnel hop.
  • In the admin UI, register two OpenAI-category availableModels:
    • review model → agentgateway alias mapped to an OpenRouter frontier model (ZDR alias);
    • embed model → agentgateway alias for bge-m3 (the same embedder VK engram uses). Set them org-default (and/or per-repo).
  • EMBEDDING_DIM=1024 so Qdrant collections match bge-m3.

3.3 The fork (fork-and-contribute)

Section titled “3.3 The fork (fork-and-contribute)”
  • Fork octopusreview/octopus → seanb4t/octopus. The cluster GHCR pull-secret (secret/fzymgc-house/cluster/ghcr/pull-secret) is a seanb4t account-scoped PAT with read:packages, which covers seanb4t/* and fzymgc-house/* alike (verified by pulling a private seanb4t image) — no new credential needed.
  • Patch on a branch, designed to be upstream-mergeable (env-gated, backward-compatible):
    • apps/web/lib/qdrant.ts: const VECTOR_SIZE = Number(process.env.EMBEDDING_DIM ?? 3072);
    • (optional) qdrant.ts QdrantClient: add apiKey: process.env.QDRANT_API_KEY.
  • Build apps/web/Dockerfile → ghcr.io/seanb4t/octopus, multi-arch amd64+arm64 (cluster has ARM tpi-* nodes — main build risk for a Next.js standalone image; validate the arm64 build early).
  • Contribute upstream: open a PR adding the configurable embedding dimension (+ optional QDRANT_API_KEY wiring). Run the fork image until merged, then switch the Deployment to the upstream stock image → maintenance tax → ~0.

4. Data flow

Section titled “4. Data flow”
  1. GitHub sends a pull_request webhook to /api/github/webhook (HMAC-verified with GITHUB_WEBHOOK_SECRET).
  2. Octopus enqueues a pg-boss process-review job; the in-process worker fetches the diff, clones/indexes changed files, and upserts bge-m3 embeddings into Qdrant (code_chunks).
  3. The review engine retrieves relevant chunks (dense + sparse RRF) and calls the review model through agentgateway → OpenRouter.
  4. Findings post as inline comments with severity; Critical → REQUEST_CHANGES. A Check Run reports status.

5. Security & data handling

Section titled “5. Security & data handling”
  • ZDR preserved: all model/embedding traffic egresses only through agentgateway; reviews hit OpenRouter under the ZDR alias, embeddings stay in-cluster (Ollama bge-m3). No direct api.openai.com / api.anthropic.com calls.
  • Webhook path is intentionally unauthenticated at the proxy layer and secured by GitHub signature — it is the only Octopus surface exposed to the internet (octopus-wh.fzymgc.net). The admin UI is not internet-exposed (internal octopus.fzymgc.house via router-hosts) and is additionally protected by Octopus’s own Better Auth (GitHub OAuth login + ADMIN_EMAILS pin). No blanket Authentik forward-auth (it would break the webhook, and the UI is already off-internet).
  • Secrets via Vault/ESO only; OCTOPUS_DATA_KEY + BETTER_AUTH_SECRET generated and stored in Vault, never committed. Qdrant not exposed beyond the cluster; optional QDRANT_API_KEY.
  • New Vault path secret/fzymgc-house/cluster/octopus documented in docs/reference/secrets.md.

6. Sequencing (phased, reversible)

Section titled “6. Sequencing (phased, reversible)”

Can proceed now (independent of hl-0sr):

  • P1: fork seanb4t/octopus, apply patch, multi-arch image CI → GHCR; open upstream PR.
  • P2: CNPG — add octopus to managed.roles in cnpg/postgres-cluster.yaml + db-octopus.yaml (Database CR) + users-octopus.yaml (role-credential ESO); plain Qdrant Deployment + PVC; namespace + ESO scaffolding.
  • P3: register the GitHub App (webhook URL https://octopus-wh.fzymgc.net/api/github/webhook, perms, events); store App creds + OAuth creds in Vault.
  • P4: author all manifests (Deployment/Service + internal UI IngressRoute/cert + secrets/ kustomization + ArgoCD app); register octopus in the webhook_services Terraform variable (HCP Terraform workspace, tf/cloudflare/) so octopus-wh.fzymgc.net resolves and the tunnel routes it to the octopus origin.

Blocks on hl-0sr cutover:

  • P5: set OPENAI_BASE_URL + the agentgateway VK; register the OpenAI-category alias models; go-live sync. Until then, the app can be staged suspend/scaled-to-0 or pointed at the current LiteLLM endpoint as a temporary bring-up (LiteLLM is OpenAI-compatible) and repointed at cutover.

7. Testing & validation

Section titled “7. Testing & validation”
  • Image: arm64 + amd64 build succeed; container boots, bun run db:migrate applies cleanly.
  • Wiring: a test PR triggers a webhook (delivery 200 in GitHub App advanced settings), a process-review job runs, Qdrant code_chunks is created at dim 1024, an inline comment posts.
  • ZDR: confirm via agentgateway logs/metrics that review + embedding calls traverse the gateway (provider/model labels present) and no direct provider egress occurs.
  • Rollback: suspend/scale-to-0 the Deployment; uninstall the GitHub App; data isolated to the dedicated DB + Qdrant PVC (no blast radius into engram/agent-memory).

8. Risks & mitigations

Section titled “8. Risks & mitigations”
RiskMitigation
Review call hard-routes claude-* to Anthropic SDK, ignoring OPENAI_BASE_URLUse an OpenAI-category alias model for reviews; confirm call site at plan time
arm64 build of the Next.js standalone image failsValidate buildx arm64 early in P1; pin node base; fall back to amd64-node-scheduling if needed
agentgateway VK cannot be model-scoped (hl-0sr T5 governance gap)Accept for single-tenant homelab; one VK reaches any model
Upstream PR not mergedKeep running the fork image; patch is 1–2 lines, low rebase cost
hl-0sr slipsStage on LiteLLM (OpenAI-compatible) as interim backend; repoint at cutover
Webhook service_url target (direct Service vs. via Traefik) unproven for this appMirror the committed argocd-wh webhook_services entry at plan time; verify GitHub webhook delivery returns 200 in P3

9. Decisions

Section titled “9. Decisions”
  • D1: Self-host (cloud free tiers unavailable for a private/unlicensed repo).
  • D2: Route LLM + embeddings through agentgateway (ZDR, central governance) — depends on hl-0sr.
  • D3: Reviews via OpenRouter frontier model; embeddings via self-hosted bge-m3 (1024-dim).
  • D4: Fork-and-contribute via seanb4t/octopus → ghcr.io/seanb4t/octopus; upstream the EMBEDDING_DIM patch; revert to stock image once merged.
  • D5: Dedicated Qdrant (plain qdrant/qdrant Deployment) + dedicated CNPG database — isolated from engram/agent-memory.
  • D6: GitHub App only; single org; Better Auth (GitHub OAuth + ADMIN_EMAILS); no blanket forward-auth. Two hostnames, one Service: public octopus-wh.fzymgc.net (webhook_services, webhook only, signature-secured) + internal octopus.fzymgc.house (router-hosts, UI/OAuth, off-internet). Better Auth canonical URL = the internal host.
  • D7: Disable Redis/Elasticsearch/Pubby/R2/Stripe/email; ENABLE_INTERNAL_CLI=false.