Agentgateway generic OpenRouter passthrough lane

Status: Design (pre-plan) Bead: hl-rpm9 Date: 2026-06-20 Deciders: Sean Brandt

Overview

Add a dedicated, transparent OpenRouter passthrough host to agentgateway (openrouter-gw.fzymgc.house) so in-cluster clients can call any OpenRouter OpenAI-compatible endpoint with any model slug and any request parameters, without per-model alias registration. The lane keeps OpenTelemetry usage/token labels (via the Detect route type) and offers a client-selectable Zero-Data-Retention (ZDR) variant for callers that need the privacy guarantee.

This complements — does not replace — the existing curated llm-gw.fzymgc.house lane (ADR hl-hv3, hl-6nm), which stays the blessed, alias-gated path for the cluster’s standard workloads (octopus, engram, reviews).

Motivation

The curated openrouter backend on llm-gw.fzymgc.house already forwards the request’s model field unchanged and already passes arbitrary params through (it carries no ai.overrides). What it lacks is endpoint coverage: the llm-chat HTTPRoute only matches /v1/chat/completions and /v1/models, and llm-embeddings only /v1/embeddings. There is no path for /v1/responses, /v1/messages (Anthropic-style), /v1/messages/count_tokens, or future OpenRouter endpoints, and no single lane intended for “send whatever OpenRouter accepts.”

The goal is a lane where a client configured exactly as if it were talking to https://openrouter.ai/api/v1 works unchanged — only the base host swaps to the in-cluster gateway, gaining the cluster virtual-key gate and OTel pipeline.

Decisions

Dedicated host openrouter-gw.fzymgc.house, not a catch-all on the curated llm-gw host. Clean isolation: no Gateway-API longest-prefix interplay with the curated routes, and a clear mental model (“this host is raw OpenRouter”). Cost: one cert SAN + one Gateway listener + one HostMapping alias (all in-repo; see Components).
Detect route type for the non-ZDR lane (routes: { "*": Detect }), not Passthrough. Detect forwards the body as-is but still extracts the model field and usage for telemetry/rate-limiting, preserving the cluster’s OTel provider/model/token labels. Passthrough would go dark on telemetry for no added benefit at this layer.
Transparent 1:1 path mirroring. Under Detect/Passthrough, agentgateway ignores pathPrefix and forwards the client’s path verbatim (set_default_path early-returns). Therefore the contract is: clients set their base URL to https://openrouter-gw.fzymgc.house/api/v1 (mirroring OpenRouter’s real /api/v1 base). A client POST /api/v1/responses is forwarded to openrouter.ai/api/v1/responses. The generic backend omits pathPrefix (it would be ignored anyway).
ZDR is a typed-route sibling backend — it cannot be a *: Detect catch-all. ai.overrides injects zdr: true into the parsed request body; a forward-as-is lane has no parsed body to inject into. The ZDR backend therefore enumerates typed RouteTypes so pathPrefix + suffix reconstruction and the ZDR override both apply. This mirrors the existing openrouter-zdr isolation accepted in ADR hl-0eb (ZDR is backend-global; no per-model selector).

Typed set = Completions + Embeddings — OpenRouter’s OpenAI-compatible surface (/api/v1/chat/completions, /api/v1/embeddings). Messages and AnthropicTokenCount are deliberately excluded: those are Anthropic’s native /v1/messages surface, which OpenRouter’s OpenAI-compatible API does not expose; routing them through an OpenAI provider would silently translate to /chat/completions. Responses is a verify-and-add item (only include it once OpenRouter is confirmed to serve /api/v1/responses).

No double-prefix. For typed RouteTypes the upstream suffix is the provider’s fixed canonical string per type — openai::path_suffix: Completions → /chat/completions, Embeddings → /embeddings — appended to pathPrefix: /api/v1, giving /api/v1/chat/completions and /api/v1/embeddings. The routes map key is used only for ends-with detection of the RouteType, not as the upstream suffix — so the client’s /api/v1/chat/completions matches the /v1/chat/completions key (ends-with) yet reconstructs to a single /api/v1/..., never /api/v1/v1/.... Both lanes therefore work from the same client base URL (.../api/v1).
Lane selection uses two separate HTTPRoutes, not two rules in one route, header-selected by the x-agentgateway-backend: openrouter-generic-zdr request header. Both routes match PathPrefix "/" on the same host; the ZDR route adds the header match and is chosen by Gateway API’s header-count precedence tiebreaker (more matching headers wins on a path tie), not by rule list-order. This is the exact, proven in-cluster pattern of llm-chat / llm-chat-zdr (two HTTPRoutes sharing a path; the ZDR one adds the header) — a single HTTPRoute with two identical-path rules would be ambiguous/unreachable under naive first-match, so it is avoided. A request without the header matches only the generic route; a request with it matches both and the header tiebreaker selects ZDR.

Architecture

openrouter-gw.fzymgc.house   (NEW Gateway listener; SAN on agentgateway-tls;
                              HostMapping alias -> 192.168.20.155)
  client base_url: https://openrouter-gw.fzymgc.house/api/v1
        │
        │  vkey gate (AgentgatewayPolicy apiKeyAuthentication, Strict;
        │             targets BOTH routes below)
        ▼
  Two HTTPRoutes on the same host, same path "/", header-count tiebreaker:

  HTTPRoute: openrouter-generic        (PathPrefix "/", no header)
     └─► openrouter-generic            routes: { "*": Detect }
              pathPrefix ignored → forwards /api/v1/* verbatim → openrouter.ai
              (transparent proxy; usage telemetry scraped)

  HTTPRoute: openrouter-generic-zdr    (PathPrefix "/" + header
                                        x-agentgateway-backend: openrouter-generic-zdr)
     └─► openrouter-generic-zdr        routes: { /v1/chat/completions: Completions,
                                                 /v1/embeddings: Embeddings }
              pathPrefix /api/v1 + fixed suffix; ai.overrides zdr=true

A request without the ZDR header matches only openrouter-generic. A request with the header matches both routes (both PathPrefix "/"); the ZDR route’s extra header match wins the Gateway API precedence tiebreaker.

Both backends target host: openrouter.ai:443, authenticate with the existing agentgateway-openrouter secret, and initiate TLS (tls: {}).

Components

New file: argocd/app-configs/agentgateway/openrouter-generic.yaml (registered in kustomization.yaml). Holds both backends, the HTTPRoute, and the vkey policy for the new route.

AgentgatewayBackend/openrouter-generic

spec:
  ai:
    provider:
      openai: {}              # no model pin → model from request
      host: openrouter.ai
      port: 443
      # pathPrefix omitted: Detect ignores it; client sends /api/v1/* verbatim
  policies:
    auth: { secretRef: { name: agentgateway-openrouter } }
    tls: {}
    ai:
      routes:
        "*": Detect           # forward as-is + scrape usage telemetry

AgentgatewayBackend/openrouter-generic-zdr

spec:
  ai:
    provider:
      openai: {}
      host: openrouter.ai
      port: 443
      pathPrefix: /api/v1     # typed RouteTypes apply prefix + suffix
  policies:
    auth: { secretRef: { name: agentgateway-openrouter } }
    tls: {}
    ai:
      routes:
        /v1/chat/completions: Completions
        /v1/embeddings: Embeddings
        # /v1/responses: Responses  # add once OpenRouter /api/v1/responses confirmed
      overrides:
        - field: provider
          value: { zdr: true, require_parameters: true }

HTTPRoute/openrouter-generic — parentRef Gateway agentgateway sectionName: openrouter-gw, hostname openrouter-gw.fzymgc.house, one rule: match: PathPrefix "/" (no header filter) → backend openrouter-generic.
HTTPRoute/openrouter-generic-zdr — same parentRef/host, one rule: match: PathPrefix "/" plus header x-agentgateway-backend: openrouter-generic-zdr → backend openrouter-generic-zdr. Separate route (not a second rule) so Gateway API’s header-count precedence selects it on the path tie — the proven llm-chat-zdr pattern.
AgentgatewayPolicy/openrouter-generic-apikey — apiKeyAuthentication mode Strict, secretRef: agentgateway-vkeys, targetRefs: [HTTPRoute openrouter-generic, HTTPRoute openrouter-generic-zdr] (both routes, so the vkey gate cannot be bypassed via the ZDR header). A dedicated policy keeps the new lane self-contained and independently revertible, rather than extending the shared llm-apikey targetRefs.

Edited files:

gateway.yaml — add listener openrouter-gw (HTTPS:443, hostname openrouter-gw.fzymgc.house, TLS Terminate agentgateway-tls, allowedRoutes: { namespaces: { from: Same } }).
certificate.yaml — add openrouter-gw.fzymgc.house to dnsNames (cert-manager re-issues via the existing Cloudflare DNS-01 issuer). Note: this name becomes visible in public CT logs, consistent with the existing llm-gw/mcp-gw SAN trade-off.
hostmapping.yaml — add openrouter-gw.fzymgc.house to aliases and a tag. This is the actual DNS mechanism; the Gateway’s router-hosts annotation is a no-op (the operator watches Ingress/Service, not Gateway).
kustomization.yaml — add openrouter-generic.yaml to resources.

Reused without change:

agentgateway-openrouter auth secret, agentgateway-vkeys virtual keys, agentgateway-tls cert secret.
llm-max-buffer policy — already Gateway-scoped (targetRefs: Gateway agentgateway), so the new listener inherits the 32Mi buffer automatically.

Data flow

Client (base URL https://openrouter-gw.fzymgc.house/api/v1, header Authorization: Bearer <virtual-key>) sends e.g. POST /api/v1/responses with { "model": "anthropic/claude-opus-4.7", ... }.
Gateway terminates TLS, the vkey policy enforces a valid virtual key (Strict).
HTTPRoute openrouter-generic rule 1 (no special header) → backend openrouter-generic.
Detect reads model for telemetry, leaves body/path untouched, injects the agentgateway-openrouter upstream Authorization, initiates TLS, forwards to openrouter.ai/api/v1/responses.
Response streams back; OTel records provider=openrouter, model, token usage.

ZDR path: client adds header x-agentgateway-backend: openrouter-generic-zdr; the request matches both routes on the / path, and the ZDR route wins on the header-count tiebreaker → backend openrouter-generic-zdr. The typed RouteType parses the body, ai.overrides sets provider.zdr=true + require_parameters=true, the upstream path reconstructs as pathPrefix + fixed canonical suffix (e.g. /api/v1/chat/completions), and it forwards.

Non-goals (YAGNI)

No gateway-side retry on the generic lane (interactive clients retry themselves; the embeddings retry policy exists only for octopus batch indexing).
No model alias map — native slugs only; aliasing is the curated lane’s job.
No new virtual keys / Vault paths — reuse agentgateway-vkeys. If a separate quota/identity for this lane is wanted later, it is an additive follow-up.
No Realtime/websocket support in v1 (would require Passthrough and loses telemetry; revisit if a concrete need appears).
The curated llm-gw lane and its routes/aliases are untouched.

Testing / verification plan

Pre-merge (static):

kubectl --context fzymgc-house apply --dry-run=server -k argocd/app-configs/agentgateway — validates the new backends/routes/policy and the routes enum values against the deployed v1.3.0-alpha.1 CRD.
Confirm cert-manager re-issues agentgateway-tls with the new SAN before the listener serves traffic.
Confirm the Gateway-scoped llm-max-buffer policy is inherited by the new openrouter-gw listener (no sectionName on its targetRef → all listeners).

Post-deploy (live, gated):

DNS resolves openrouter-gw.fzymgc.house → 192.168.20.155 (HostMapping).
Non-ZDR: curl https://openrouter-gw.fzymgc.house/api/v1/chat/completions with a native slug + virtual key → 200 from OpenRouter (transparent path).
GET /api/v1/models returns OpenRouter’s full catalog (a passthrough bonus).
Missing/invalid virtual key on both routes → 401/403 (the ZDR header must not bypass the vkey gate).
Route selection (addresses review #1): a request with the x-agentgateway-backend: openrouter-generic-zdr header lands on the ZDR backend, and one without it lands on the generic backend — confirm via OTel/backend labels that the header-count tiebreaker actually selects the ZDR route on the shared / path. This is the riskiest assumption; verify before relying on ZDR.
ZDR path reconstruction (addresses review #2): confirm a ZDR /api/v1/chat/completions request reaches OpenRouter at /api/v1/chat/completions (single prefix, not /api/v1/v1/...) and returns 200 — a double-prefix would surface as a 404.
ZDR enforcement: response/telemetry confirms provider.zdr was applied; a ZDR request for a model that requires params errors closed rather than serving without ZDR.
ClickStack: confirm Detect emits provider/model/token labels for the generic lane (the reason Detect was chosen over Passthrough), including for streamed (stream: true) responses.

Open risks

Detect telemetry completeness for streamed (SSE) responses and for non-chat endpoints (/v1/responses) — verify token usage is captured, not just request metadata.
Streaming under Detect — confirm stream: true responses pass through cleanly (expected, since the body is forwarded as-is).
Header-count precedence on a shared / path — the ZDR route selection relies on Gateway API’s header tiebreaker between two same-path routes. Proven in-cluster for llm-chat/llm-chat-zdr (which share a specific path); this design shares the / catch-all path. Behaviorally identical under the spec, but must be live-verified (see verification plan) before ZDR is trusted.
ZDR-on-typed coverage — only the enumerated endpoints (Completions, Embeddings) get the override. An un-enumerated path sent with the ZDR header falls to the default Completions type and is forwarded (parsed as chat), not rejected — so a client mis-using the ZDR header on a non-chat, non-embedding endpoint could get a malformed-but-forwarded request rather than a clean error. Documented as a known bound; clients should send the ZDR header only on chat/embeddings traffic.

References

ADR hl-hv3 — agentgateway as unified LLM/MCP data plane (replaces LiteLLM).
ADR hl-6nm — split model routing (OpenRouter reviews + in-cluster embeddings).
ADR hl-0eb — ZDR via ai.overrides (backend-global; supersedes hl-env).
Deployed CRD: agentgateway v1.3.0-alpha.1 AgentgatewayBackend.routes enum = [AnthropicTokenCount, Completions, Detect, Embeddings, Messages, Models, Passthrough, Realtime, Responses]; keys match ends-with; * wildcard; default Completions. (verified against the live CRD; bead hl-rpm9)
agentgateway upstream behavior (verified via deepwiki against agentgateway/agentgateway, recorded in bead hl-rpm9):
- set_default_path early-returns for Detect/Passthrough → pathPrefix ignored, client path forwarded verbatim (basis for the 1:1 proxy contract).
- typed RouteTypes build the upstream path from pathPrefix + a fixed openai::path_suffix(route_type) (Completions → /chat/completions, Embeddings → /embeddings) — the routes key is only an ends-with detector, so no double-prefix.
- Detect still reads the model field for telemetry.
Existing patterns: llm-routes.yaml (routes), llm-backends.yaml (backends), llm-policies.yaml (openrouter-zdr backend + llm-chat-zdr separate-route header selection — the proven header-count-tiebreaker precedent — llm-apikey vkey gate, llm-max-buffer Gateway-scoped buffer), hostmapping.yaml (DNS), certificate.yaml (SAN).
Bead hl-rpm9 — grounding traces (probe/deepwiki/CRD) and design-review rounds.