Skip to content

Design: the OpenRouter ZDR guardrail

See openspec/changes/openrouter-zdr-guardrail/proposal.md, section Why, for the measurements and for the decisions that Sean made on 2026-10-04 and 2026-10-05. The current state that shapes the approach:

  • Every OpenRouter lane and the decisions route read one variable, OPENROUTER_LANE_API_KEY. The litellm-secrets ExternalSecret in argocd/app-configs/litellm/secrets.yaml builds it from the Vault property openrouter_api_key at secret/fzymgc-house/cluster/litellm. An operator created that key by hand, and nothing in this repository can reproduce it.
  • If a remoteRef names a property that Vault does not hold, the whole ExternalSecret fails. The pods keep the values that they loaded at start.
  • If an extraEnvVars entry is missing, os.environ/<NAME> resolves to None without an error. Only the lanes that name the variable fail, and only at request time.
  • The LiteLLM Deployment carries reloader.stakater.com/auto: "true". When litellm-secrets changes, Stakater Reloader restarts the proxy.
  • On LiteLLM v1.104.0-rc.1, the provider block of a request replaces the provider block of the lane. The proposal gives the four requests that show it.
  • The workspace main-cluster-litellm runs on the HCP agent with the Vault run role tfc-litellm and the policy terraform-litellm. That policy reads cluster/litellm-keys/* only, and it has no write grant.
  • tf/backblaze sets the pattern for a generated provider credential. An operator seeds a provisioning key at infrastructure/backblaze/terraform. The workspace reads it with an ephemeral read, and writes each generated key to its own Vault path with vault_kv_secret_v2.
  • The OpenRouter provider OpenRouterTeam/openrouter v0.3.19, the newest release on 2026-10-05, manages openrouter_workspace, openrouter_guardrail and openrouter_api_key. It needs a management key. It cannot set the default guardrail of a workspace: the resource openrouter_workspace_default_guardrail merged upstream on 2026-09-29 (#473), and no release held it on 2026-10-05. It cannot assign a guardrail to one API key either: issue #130 and the pull request #617 that fixes it were open on 2026-10-05.
  • The default guardrail of a workspace applies to all traffic of that workspace. Per the provider documentation, a guardrail of a member or a key combines with it and can only narrow it. The zdr field of a request can add ZDR. It cannot remove the ZDR of a guardrail.
  • On 2026-10-05 the account held six workspaces. The lane key openrouter_api_key is the key litellm in the workspace fzymgc-litellm, which has no default guardrail. The management API lists each workspace with its default_guardrail_id, and the guardrail API takes the five enforce_zdr_* fields.

This change adds no address and no prefix, so docs/reference/network.md does not change.

Goals:

  • OpenRouter enforces ZDR for each request of the five ZDR lanes, whatever LiteLLM forwards.
  • Code holds the guardrail, so a change in the OpenRouter dashboard shows in a plan. Until a provider release can manage it, the management API sets it (D-9).
  • No person handles the ZDR key. Terraform creates it, Vault holds it, and ESO delivers it.
  • The lanes without ZDR keep their behavior and their key.

Non-Goals:

  • The plain key openrouter_api_key. It stays a hand-made key in the workspace fzymgc-litellm.
  • The decisions route /openrouter/alpha/decisions. It keeps OPENROUTER_LANE_API_KEY. engram’s re-ranker sends memory text through it, and whether a guardrail applies to that API is a separate question.
  • scripts/seed-litellm-vault.sh. It writes cluster/litellm, not the new paths.
  • An upstream fix in LiteLLM. With the guardrail, the precedence of the two provider blocks no longer decides ZDR.
  • A budget, a model list or a provider list on the guardrail.

D-1. A workspace of its own holds the ZDR key, and its default guardrail enforces ZDR

Section titled “D-1. A workspace of its own holds the ZDR key, and its default guardrail enforces ZDR”

Terraform creates the workspace litellm-zdr, and its default guardrail enforces ZDR. The ZDR key lives in that workspace and in no other. So the rule covers the key without an assignment, and the plain key in the workspace fzymgc-litellm stays outside it. A new key after a rotation is in the same workspace, so the rule covers it too.

Alternative: a guardrail assigned to the ZDR key. Rejected for now. The provider cannot write the assignment (issue #130). A terraform_data resource that calls the API is the workaround in that issue, and it is code we would own for a gap the provider will close.

Alternative: the default guardrail of fzymgc-litellm, the workspace of the plain key. Rejected. Then openrouter/* and gpt-4o-mini would lose their providers without ZDR, and karakeep, mealie and the claude-code key choose such models on purpose.

Alternative: a LiteLLM guardrail on the ZDR lanes, in addition to the OpenRouter guardrail. Rejected. LiteLLM can attach a guardrail to named lanes through policies: with condition.model. But the built-in guardrails read the content of the messages, and none of them reads or removes a request parameter such as provider. A guardrail that does is a custom Python class that runs as an async_pre_call_hook. Sean ruled on 2026-08-16 that ZDR is a property of the lane, set in configuration, with no pre-call hook. A second layer in LiteLLM changes only the error that a caller sees, because OpenRouter enforces the rule after LiteLLM.

Alternative: additional_drop_params: ["provider"] on each ZDR lane. Rejected. LiteLLM drops the parameter after it merges the request with the lane, so it also drops the ZDR rule of the lane.

D-2. Terraform in tf/litellm creates the key, and Sean seeds only the management key

Section titled “D-2. Terraform in tf/litellm creates the key, and Sean seeds only the management key”

Sean decided this on 2026-10-05. tf/litellm already owns the LiteLLM keys and runs with Vault access. The OpenRouter provider reads the management key with an ephemeral read, so the management key never enters state. The ZDR key does enter state, because the provider returns it as the attribute key of openrouter_api_key. The state of an HCP workspace is encrypted, and tf/backblaze stores its generated B2 keys the same way.

The workspace applies only through a merge, so a rotation is a pull request too. A local zdr_key_version feeds a terraform_data resource, and the key carries lifecycle { replace_triggered_by = [terraform_data.zdr_key_version] }. A new value replaces the key. The keys of tf/litellm follow the same idea with key_wo_version.

Alternative: a module and an HCP workspace of their own, tf/openrouter. Rejected by Sean on 2026-10-05. It needs a workspace entry, a Vault JWT role and an operator apply of tf/hcp-terraform.

Alternative: a script that calls the management API once. Rejected by Sean on 2026-10-05. No code holds the guardrail afterwards.

D-3. The guardrail enforces ZDR for all five model groups and holds nothing else

Section titled “D-3. The guardrail enforces ZDR for all five model groups and holds nothing else”

openrouter-zdr/* accepts any model, so the guardrail sets enforce_zdr_anthropic, enforce_zdr_openai, enforce_zdr_google, enforce_zdr_xai and enforce_zdr_other. For the OpenAI, Anthropic and Google groups, OpenRouter then removes the first-party endpoints and keeps Azure, Bedrock and Vertex. The budget of each consumer lives on its LiteLLM virtual key, so the guardrail holds no budget, no model list and no provider list.

D-4. The Vault paths follow the existing patterns

Section titled “D-4. The Vault paths follow the existing patterns”
Path Property Writer Reader
infrastructure/openrouter/terraform management_key Sean, by hand tf/litellm, ephemeral
cluster/litellm/openrouter-zdr api_key tf/litellm the litellm-secrets ExternalSecret

The first path matches infrastructure/backblaze/terraform. The second is a Terraform-owned subpath of cluster/litellm, like cluster/litellm/oidc that tf/keycloak writes. tf/CLAUDE.md forbids a Terraform write to the bare path cluster/litellm, which the operator owns. The comment in tf/vault/policy-litellm.tf forbids a grant on that bare path too. A grant on a subpath does not reach the bare path, because a KV path grant has no wildcard.

The blocks still apply to a request without a provider block, and require_parameters: true still limits a lane to providers that support each parameter. The lane change edits only api_key.

D-6. The probe lane moves with octopus-embed

Section titled “D-6. The probe lane moves with octopus-embed”

The comment on zdr-probe-embed requires the same model and the same key as octopus-embed, so that the lane is the only difference between the probe and its comparison.

D-7. The names follow the rule for provider keys

Section titled “D-7. The names follow the rule for provider keys”

The variable is OPENROUTER_ZDR_LANE_API_KEY. A standard name such as OPENROUTER_API_KEY opens the built-in pass-through route /openrouter/* to every workload key. docs/operations/litellm.md, section Provider keys use names that LiteLLM does not read, gives the rule.

D-8. A security review of the provider comes before its first use

Section titled “D-8. A security review of the provider comes before its first use”

The management key can create keys and spend on the account. The provider is young: five releases on 2026-09-22 and 6 stars on 2026-10-05. A review of v0.3.19 (commit f521db3) on 2026-10-05 found:

  • The management key goes only to https://openrouter.ai/api/v1, in the Authorization header. The provider sends nothing to another host, runs no process and writes no file.
  • A refresh keeps the attribute key of openrouter_api_key, because the API returns it only at creation and the read code does not touch it. An import leaves it empty.
  • At TF_LOG=DEBUG the provider logs whole response bodies, and the body of a new key holds the key. Never set TF_LOG on main-cluster-litellm. On 2026-10-05 the workspace had only the five TFC_VAULT_* variables.
  • Releases ship from bot pull requests with no human review. The version is pinned exactly, the lock file holds the hashes, and a bump needs a new read of the source.
  • A replacement deletes the old key, so the key carries create_before_destroy.

D-9. Until a provider release can set it, the management API sets the default guardrail

Section titled “D-9. Until a provider release can set it, the management API sets the default guardrail”

Sean decided this on 2026-10-05, after terraform validate showed that v0.3.19 has no openrouter_workspace_default_guardrail. After the apply of pull request 2, an operator sends one PATCH /api/v1/guardrails/<default_guardrail_id> with the five enforce_zdr_* fields. The output openrouter_zdr_default_guardrail_id of tf/litellm gives the id. When a release ships the resource, a pull request pins it and adds the resource with an import block. Its plan shows the import and no change.

Alternative: wait for the release. Rejected by Sean on 2026-10-05. The release had no date, and the gap stays open until then.

Alternative: openrouter_guardrail in the workspace, assigned to the key once through the API. Rejected. A key rotation drops the assignment, and a person has to repeat it.

  • [The key exists before the guardrail, from the apply of pull request 2 until the API call] → No lane uses the key before pull request 3, and pull request 3 merges only after the proof of Migration Plan step 3.
  • [A person changes the default guardrail before the import] → No plan shows it until the import. The acceptance command of openspec/changes/openrouter-zdr-guardrail/proposal.md shows a lost ZDR rule. Run it after each change to the OpenRouter account.
  • [The provider loses the attribute key at a refresh] → The next plan would write an empty value to Vault. A task makes sure that the second plan after the apply shows no change.
  • [The write grant reaches beyond its path] → Each grant names one exact path, with paired data and metadata entries, as tf/vault/policy-backblaze.tf does.
  • [The property is missing when pull request 3 merges, and the whole ExternalSecret fails] → A task makes sure that the property exists before the merge. The pods keep their loaded values.
  • [The new key fails, and the five ZDR lanes fail] → The other lanes keep working. The way back is a revert of pull request 3.
  • [A later model of a ZDR lane has no ZDR endpoint] → The lane answers HTTP 404 “No endpoints found matching your data policy”. Pick a model with ZDR endpoints.
  • [A person edits the guardrail in the dashboard after the import] → The next plan of main-cluster-litellm shows the change and the apply reverts it.

Each step names its way back.

  1. Pull request 1 adds the grants to terraform-litellm. main-cluster-vault applies on merge. Way back: revert the pull request.

  2. Sean creates a management key in the OpenRouter dashboard (Settings > Management Keys) and writes it:

    Terminal window
    export VAULT_ADDR=https://vault.fzymgc.house
    printf '%s' "$OPENROUTER_MANAGEMENT_KEY" | vault kv put \
    secret/fzymgc-house/infrastructure/openrouter/terraform management_key=-

    Way back: delete the key in the dashboard and run vault kv metadata delete secret/fzymgc-house/infrastructure/openrouter/terraform. Nothing else reads that path.

  3. Pull request 2 adds the provider and the resources. Its speculative plan creates four resources and changes nothing else. main-cluster-litellm applies on merge. Then an operator sets the default guardrail of the workspace (D-9):

    Terminal window
    MK="$(vault kv get -field=management_key secret/fzymgc-house/infrastructure/openrouter/terraform)"
    GID="$(curl -s https://openrouter.ai/api/v1/workspaces -H "Authorization: Bearer $MK" \
    | jq -r '.data[] | select(.slug == "litellm-zdr") | .default_guardrail_id')"
    curl -s -X PATCH "https://openrouter.ai/api/v1/guardrails/$GID" \
    -H "Authorization: Bearer $MK" -H 'Content-Type: application/json' \
    -d '{"enforce_zdr_anthropic":true,"enforce_zdr_openai":true,"enforce_zdr_google":true,
    "enforce_zdr_xai":true,"enforce_zdr_other":true}' \
    | jq -c '(.data // .) | {name, zdr: [.enforce_zdr_anthropic, .enforce_zdr_openai,
    .enforce_zdr_google, .enforce_zdr_xai, .enforce_zdr_other]}'

    The reply must show five times true. Then the operator proves the key at OpenRouter:

    Terminal window
    ZDR_KEY="$(vault kv get -field=api_key secret/fzymgc-house/cluster/litellm/openrouter-zdr)"
    curl -s https://openrouter.ai/api/v1/chat/completions \
    -H "Authorization: Bearer $ZDR_KEY" -H 'Content-Type: application/json' \
    -d '{"model":"deepseek/deepseek-v4.1-flash","max_tokens":256,
    "provider":{"only":["StreamLake"],"zdr":false},
    "messages":[{"role":"user","content":"Reply with the word OK."}]}' \
    | jq -c '{provider, error: .error.message}'

    The reply must not name StreamLake. A 404 with “No endpoints found matching your data policy” is the expected answer. Way back: revert the pull request. The destroy removes the key, the workspace with its guardrail, and the Vault path, and no lane uses them yet.

  4. Pull request 3 merges. ArgoCD syncs litellm, the ExternalSecret writes the new variable, and the proxy restarts. Way back: revert the squash commit with git revert <sha> on a branch, and merge that pull request.

  5. An operator runs the acceptance command of openspec/changes/openrouter-zdr-guardrail/proposal.md and the plain requests of the spec. If a plain request fails, take the way back of step 4.

To rotate the ZDR key later, raise zdr_key_version in tf/litellm in a pull request. The merge replaces the key (D-2). Vault gets the new value, ESO refreshes the Secret within five minutes, and Reloader restarts the proxy.