Design: the OpenRouter ZDR guardrail
Context
Section titled “Context”See openspec/changes/openrouter-zdr-guardrail/proposal.md, section Why, for the measurements and for the decisions that Sean made on
2026-10-04 and 2026-10-05. The current state that shapes the approach:
- Every OpenRouter lane and the decisions route read one variable,
OPENROUTER_LANE_API_KEY. Thelitellm-secretsExternalSecret inargocd/app-configs/litellm/secrets.yamlbuilds it from the Vault propertyopenrouter_api_keyatsecret/fzymgc-house/cluster/litellm. An operator created that key by hand, and nothing in this repository can reproduce it. - If a
remoteRefnames a property that Vault does not hold, the whole ExternalSecret fails. The pods keep the values that they loaded at start. - If an
extraEnvVarsentry is missing,os.environ/<NAME>resolves toNonewithout an error. Only the lanes that name the variable fail, and only at request time. - The LiteLLM Deployment carries
reloader.stakater.com/auto: "true". Whenlitellm-secretschanges, Stakater Reloader restarts the proxy. - On LiteLLM v1.104.0-rc.1, the
providerblock of a request replaces theproviderblock of the lane. The proposal gives the four requests that show it. - The workspace
main-cluster-litellmruns on the HCP agent with the Vault run roletfc-litellmand the policyterraform-litellm. That policy readscluster/litellm-keys/*only, and it has no write grant. tf/backblazesets the pattern for a generated provider credential. An operator seeds a provisioning key atinfrastructure/backblaze/terraform. The workspace reads it with an ephemeral read, and writes each generated key to its own Vault path withvault_kv_secret_v2.- The OpenRouter provider
OpenRouterTeam/openrouterv0.3.19, the newest release on 2026-10-05, managesopenrouter_workspace,openrouter_guardrailandopenrouter_api_key. It needs a management key. It cannot set the default guardrail of a workspace: the resourceopenrouter_workspace_default_guardrailmerged upstream on 2026-09-29 (#473), and no release held it on 2026-10-05. It cannot assign a guardrail to one API key either: issue #130 and the pull request #617 that fixes it were open on 2026-10-05. - The default guardrail of a workspace applies to all traffic of that workspace. Per the provider
documentation, a guardrail of a member or a key combines with it and can only narrow it. The
zdrfield of a request can add ZDR. It cannot remove the ZDR of a guardrail. - On 2026-10-05 the account held six workspaces. The lane key
openrouter_api_keyis the keylitellmin the workspacefzymgc-litellm, which has no default guardrail. The management API lists each workspace with itsdefault_guardrail_id, and the guardrail API takes the fiveenforce_zdr_*fields.
This change adds no address and no prefix, so docs/reference/network.md does not change.
Goals / Non-Goals
Section titled “Goals / Non-Goals”Goals:
- OpenRouter enforces ZDR for each request of the five ZDR lanes, whatever LiteLLM forwards.
- Code holds the guardrail, so a change in the OpenRouter dashboard shows in a plan. Until a provider release can manage it, the management API sets it (D-9).
- No person handles the ZDR key. Terraform creates it, Vault holds it, and ESO delivers it.
- The lanes without ZDR keep their behavior and their key.
Non-Goals:
- The plain key
openrouter_api_key. It stays a hand-made key in the workspacefzymgc-litellm. - The decisions route
/openrouter/alpha/decisions. It keepsOPENROUTER_LANE_API_KEY. engram’s re-ranker sends memory text through it, and whether a guardrail applies to that API is a separate question. scripts/seed-litellm-vault.sh. It writescluster/litellm, not the new paths.- An upstream fix in LiteLLM. With the guardrail, the precedence of the two
providerblocks no longer decides ZDR. - A budget, a model list or a provider list on the guardrail.
Decisions
Section titled “Decisions”D-1. A workspace of its own holds the ZDR key, and its default guardrail enforces ZDR
Section titled “D-1. A workspace of its own holds the ZDR key, and its default guardrail enforces ZDR”Terraform creates the workspace litellm-zdr, and its default guardrail enforces ZDR. The ZDR key
lives in that workspace and in no other. So the rule covers the key without an assignment, and the
plain key in the workspace fzymgc-litellm stays outside it. A new key after a rotation is in the
same workspace, so the rule covers it too.
Alternative: a guardrail assigned to the ZDR key. Rejected for now. The
provider cannot write the assignment (issue #130). A terraform_data resource that calls the API
is the workaround in that issue, and it is code we would own for a gap the provider will close.
Alternative: the default guardrail of fzymgc-litellm, the workspace of the plain key. Rejected.
Then openrouter/* and gpt-4o-mini would lose their providers without ZDR, and karakeep, mealie
and the claude-code key choose such models on purpose.
Alternative: a LiteLLM guardrail on the ZDR lanes, in addition to the OpenRouter guardrail. Rejected.
LiteLLM can attach a guardrail to named lanes through policies: with condition.model. But the
built-in guardrails read the content of the messages, and none of them reads or removes a request
parameter such as provider. A guardrail that does is a custom Python class that runs as an
async_pre_call_hook. Sean ruled on 2026-08-16 that ZDR is a property of the lane, set in
configuration, with no pre-call hook. A second layer in LiteLLM changes only the error that a
caller sees, because OpenRouter enforces the rule after LiteLLM.
Alternative: additional_drop_params: ["provider"] on each ZDR lane. Rejected. LiteLLM drops the
parameter after it merges the request with the lane, so it also drops the ZDR rule of the lane.
D-2. Terraform in tf/litellm creates the key, and Sean seeds only the management key
Section titled “D-2. Terraform in tf/litellm creates the key, and Sean seeds only the management key”Sean decided this on 2026-10-05. tf/litellm already owns the LiteLLM keys and runs with Vault
access. The OpenRouter provider reads the management key with an ephemeral read, so the management
key never enters state. The ZDR key does enter state, because the provider returns it as the
attribute key of openrouter_api_key. The state of an HCP workspace is encrypted, and
tf/backblaze stores its generated B2 keys the same way.
The workspace applies only through a merge, so a rotation is a pull request too. A local
zdr_key_version feeds a terraform_data resource, and the key carries
lifecycle { replace_triggered_by = [terraform_data.zdr_key_version] }. A new value replaces the
key. The keys of tf/litellm follow the same idea with key_wo_version.
Alternative: a module and an HCP workspace of their own, tf/openrouter. Rejected by Sean on
2026-10-05. It needs a workspace entry, a Vault JWT role and an operator apply of tf/hcp-terraform.
Alternative: a script that calls the management API once. Rejected by Sean on 2026-10-05. No code holds the guardrail afterwards.
D-3. The guardrail enforces ZDR for all five model groups and holds nothing else
Section titled “D-3. The guardrail enforces ZDR for all five model groups and holds nothing else”openrouter-zdr/* accepts any model, so the guardrail sets enforce_zdr_anthropic,
enforce_zdr_openai, enforce_zdr_google, enforce_zdr_xai and enforce_zdr_other. For the
OpenAI, Anthropic and Google groups, OpenRouter then removes the first-party endpoints and keeps
Azure, Bedrock and Vertex. The budget of each consumer lives on its LiteLLM virtual key, so the
guardrail holds no budget, no model list and no provider list.
D-4. The Vault paths follow the existing patterns
Section titled “D-4. The Vault paths follow the existing patterns”| Path | Property | Writer | Reader |
|---|---|---|---|
infrastructure/openrouter/terraform |
management_key |
Sean, by hand | tf/litellm, ephemeral |
cluster/litellm/openrouter-zdr |
api_key |
tf/litellm |
the litellm-secrets ExternalSecret |
The first path matches infrastructure/backblaze/terraform. The second is a Terraform-owned subpath
of cluster/litellm, like cluster/litellm/oidc that tf/keycloak writes. tf/CLAUDE.md forbids
a Terraform write to the bare path cluster/litellm, which the operator owns. The comment in
tf/vault/policy-litellm.tf forbids a grant on that bare path too. A grant on a subpath does not
reach the bare path, because a KV path grant has no wildcard.
D-5. The lanes keep their provider blocks
Section titled “D-5. The lanes keep their provider blocks”The blocks still apply to a request without a provider block, and require_parameters: true
still limits a lane to providers that support each parameter. The lane change edits only api_key.
D-6. The probe lane moves with octopus-embed
Section titled “D-6. The probe lane moves with octopus-embed”The comment on zdr-probe-embed requires the same model and the same key as octopus-embed, so
that the lane is the only difference between the probe and its comparison.
D-7. The names follow the rule for provider keys
Section titled “D-7. The names follow the rule for provider keys”The variable is OPENROUTER_ZDR_LANE_API_KEY. A standard name such as OPENROUTER_API_KEY opens the
built-in pass-through route /openrouter/* to every workload key. docs/operations/litellm.md,
section Provider keys use names that LiteLLM does not read, gives the rule.
D-8. A security review of the provider comes before its first use
Section titled “D-8. A security review of the provider comes before its first use”The management key can create keys and spend on the account. The provider is young: five releases on
2026-09-22 and 6 stars on 2026-10-05. A review of v0.3.19 (commit f521db3) on 2026-10-05 found:
- The management key goes only to
https://openrouter.ai/api/v1, in theAuthorizationheader. The provider sends nothing to another host, runs no process and writes no file. - A refresh keeps the attribute
keyofopenrouter_api_key, because the API returns it only at creation and the read code does not touch it. An import leaves it empty. - At
TF_LOG=DEBUGthe provider logs whole response bodies, and the body of a new key holds the key. Never setTF_LOGonmain-cluster-litellm. On 2026-10-05 the workspace had only the fiveTFC_VAULT_*variables. - Releases ship from bot pull requests with no human review. The version is pinned exactly, the lock file holds the hashes, and a bump needs a new read of the source.
- A replacement deletes the old key, so the key carries
create_before_destroy.
D-9. Until a provider release can set it, the management API sets the default guardrail
Section titled “D-9. Until a provider release can set it, the management API sets the default guardrail”Sean decided this on 2026-10-05, after terraform validate showed that v0.3.19 has no
openrouter_workspace_default_guardrail. After the apply of pull request 2, an operator sends one
PATCH /api/v1/guardrails/<default_guardrail_id> with the five enforce_zdr_* fields. The output
openrouter_zdr_default_guardrail_id of tf/litellm gives the id. When a release ships the
resource, a pull request pins it and adds the resource with an import block. Its plan shows the
import and no change.
Alternative: wait for the release. Rejected by Sean on 2026-10-05. The release had no date, and the gap stays open until then.
Alternative: openrouter_guardrail in the workspace, assigned to the key once through the API.
Rejected. A key rotation drops the assignment, and a person has to repeat it.
Risks / Trade-offs
Section titled “Risks / Trade-offs”- [The key exists before the guardrail, from the apply of pull request 2 until the API call] → No lane uses the key before pull request 3, and pull request 3 merges only after the proof of Migration Plan step 3.
- [A person changes the default guardrail before the import] → No plan shows it until the import.
The acceptance command of
openspec/changes/openrouter-zdr-guardrail/proposal.mdshows a lost ZDR rule. Run it after each change to the OpenRouter account. - [The provider loses the attribute
keyat a refresh] → The next plan would write an empty value to Vault. A task makes sure that the second plan after the apply shows no change. - [The write grant reaches beyond its path] → Each grant names one exact path, with paired data and
metadata entries, as
tf/vault/policy-backblaze.tfdoes. - [The property is missing when pull request 3 merges, and the whole ExternalSecret fails] → A task makes sure that the property exists before the merge. The pods keep their loaded values.
- [The new key fails, and the five ZDR lanes fail] → The other lanes keep working. The way back is a revert of pull request 3.
- [A later model of a ZDR lane has no ZDR endpoint] → The lane answers HTTP 404 “No endpoints found matching your data policy”. Pick a model with ZDR endpoints.
- [A person edits the guardrail in the dashboard after the import] → The next plan of
main-cluster-litellmshows the change and the apply reverts it.
Migration Plan
Section titled “Migration Plan”Each step names its way back.
-
Pull request 1 adds the grants to
terraform-litellm.main-cluster-vaultapplies on merge. Way back: revert the pull request. -
Sean creates a management key in the OpenRouter dashboard (Settings > Management Keys) and writes it:
Terminal window export VAULT_ADDR=https://vault.fzymgc.houseprintf '%s' "$OPENROUTER_MANAGEMENT_KEY" | vault kv put \secret/fzymgc-house/infrastructure/openrouter/terraform management_key=-Way back: delete the key in the dashboard and run
vault kv metadata delete secret/fzymgc-house/infrastructure/openrouter/terraform. Nothing else reads that path. -
Pull request 2 adds the provider and the resources. Its speculative plan creates four resources and changes nothing else.
main-cluster-litellmapplies on merge. Then an operator sets the default guardrail of the workspace (D-9):Terminal window MK="$(vault kv get -field=management_key secret/fzymgc-house/infrastructure/openrouter/terraform)"GID="$(curl -s https://openrouter.ai/api/v1/workspaces -H "Authorization: Bearer $MK" \| jq -r '.data[] | select(.slug == "litellm-zdr") | .default_guardrail_id')"curl -s -X PATCH "https://openrouter.ai/api/v1/guardrails/$GID" \-H "Authorization: Bearer $MK" -H 'Content-Type: application/json' \-d '{"enforce_zdr_anthropic":true,"enforce_zdr_openai":true,"enforce_zdr_google":true,"enforce_zdr_xai":true,"enforce_zdr_other":true}' \| jq -c '(.data // .) | {name, zdr: [.enforce_zdr_anthropic, .enforce_zdr_openai,.enforce_zdr_google, .enforce_zdr_xai, .enforce_zdr_other]}'The reply must show five times
true. Then the operator proves the key at OpenRouter:Terminal window ZDR_KEY="$(vault kv get -field=api_key secret/fzymgc-house/cluster/litellm/openrouter-zdr)"curl -s https://openrouter.ai/api/v1/chat/completions \-H "Authorization: Bearer $ZDR_KEY" -H 'Content-Type: application/json' \-d '{"model":"deepseek/deepseek-v4.1-flash","max_tokens":256,"provider":{"only":["StreamLake"],"zdr":false},"messages":[{"role":"user","content":"Reply with the word OK."}]}' \| jq -c '{provider, error: .error.message}'The reply must not name StreamLake. A 404 with “No endpoints found matching your data policy” is the expected answer. Way back: revert the pull request. The destroy removes the key, the workspace with its guardrail, and the Vault path, and no lane uses them yet.
-
Pull request 3 merges. ArgoCD syncs
litellm, the ExternalSecret writes the new variable, and the proxy restarts. Way back: revert the squash commit withgit revert <sha>on a branch, and merge that pull request. -
An operator runs the acceptance command of
openspec/changes/openrouter-zdr-guardrail/proposal.mdand the plain requests of the spec. If a plain request fails, take the way back of step 4.
To rotate the ZDR key later, raise zdr_key_version in tf/litellm in a pull request. The merge
replaces the key (D-2). Vault gets the new value, ESO refreshes the Secret within five minutes, and
Reloader restarts the proxy.