engram sends record text to TypeSafe Jev for consolidate verdicts and search reranking, accepting standard retention without zero data retention
Date: 2026-09-24 Status: Accepted Deciders: Sean Brandt
Context
Section titled “Context”engram 0.20.0 adds typed decisions. A typed decision is a question that a model answers with probabilities. engram asks the questions through OpenRouter’s Decisions API, which routes the model typesafe/jev-1.13 to TypeSafe on the US West Coast. Two features use it:
engram spine-review consolidateattaches an advisory relation verdict to each candidate pair. It sends up to 1500 characters of each record (its summary, then the head of its content). It runs only when an operator starts it.- The search reranker (
memory.search.ranker: jev) runs on everysearch_memoryandsearch_discoverycall. It sends the query and up to 100 candidate records, 600 characters each. - Console query understanding (
memory.search.understanding: jev) was added in engram 0.22.0. It runs when someone types a query on the console/searchpage. It sends only the query text. It suggests filter chips, and results change only when the user accepts a chip.
The provider does not train on inputs. It keeps inputs under its standard retention, and zero data retention is not confirmed. The 2026-07-10 ADR accepted a limited-retention posture for the Gemini embedder. This decision adds a second upstream that receives record text.
The engram retrieval eval that closed seanb4t/engram#605 did not measure the Jev reranker. It measured only that lexical ranking beats vector-only order on paraphrased queries (MRR 0.817 against 0.579).
Decision
Section titled “Decision”Turn on both features in argocd/cluster-app/templates/agent-memory.yaml:
| Value | Setting |
|---|---|
memory.decisions.provider |
jev |
memory.decisions.baseURL |
https://llm.fzymgc.house (amended 2026-10-10, was https://llm.fzymgc.house/openrouter) |
memory.decisions.path |
/v1/systemone (added 2026-10-10) |
memory.decisions.apiKeySecret |
memory-mcp-litellm / litellm_api_key |
memory.search.ranker |
jev |
memory.search.understanding |
jev |
The calls go through LiteLLM with engram’s own virtual key. The model and the timeouts keep the binary defaults: typesafe/jev-1.13, 10 s per decision call and 2 s per rerank. The amendment below changes the route.
Rationale
Section titled “Rationale”- Jev never changes a record. A consolidate verdict is advisory. A failed or slow rerank falls back to lexical order, and the search still succeeds.
- Neither feature has a training risk, because the provider does not train on inputs. The remaining exposure is the retention window, which is the same class of exposure the 2026-07-10 ADR accepted.
- The cost is small. A probe on 2026-09-24 used 315 input tokens and cost $0.000013. A full rerank is about 15k tokens, or roughly $0.0006 per search.
- Search is the feature that recall uses most. Turning on the reranker gives a live comparison with lexical ranking that the eval did not provide.
Alternatives considered
Section titled “Alternatives considered”- Decisions only, with lexical search. This sends record text only during consolidate runs. It was not chosen, because it gives no evidence on whether Jev ranks better than lexical.
- Neither feature. This keeps the current posture. It was not chosen, because the retention exposure is bounded and both features fail safe.
Consequences
Section titled “Consequences”- Record text from the spine leaves the cluster on every search. The volume is up to 100 records of 600 characters per call.
- Each search takes up to 2 s longer when TypeSafe is slow. A timeout returns lexical order.
- The original route recorded $0 spend in LiteLLM. The amendment below fixes that.
- Console query text leaves the cluster on every console search. It contains no record content, scope names or tag vocabulary.
- To turn off the reranker, remove
memory.search.ranker. To turn off query understanding, setmemory.search.understandingtooff. To turn off all Jev calls, also removememory.decisions. No change needs a migration.
Sign-off: human approval of the pull request that lands this ADR is the sign-off for the data-residency decision.
Amendment 2026-10-10: the native LiteLLM lane to TypeSafe
Section titled “Amendment 2026-10-10: the native LiteLLM lane to TypeSafe”Sean decided on 2026-10-10 that all access to TypeSafe goes through LiteLLM and that no workload calls the TypeSafe API directly. engram now calls LiteLLM’s native decisions route, POST https://llm.fzymgc.house/v1/systemone. The lane typesafe/jev-1.13 in the LiteLLM model_list forwards the call to the TypeSafe API as typesafe/jev-1.13.0, with a TypeSafe key that only the LiteLLM pod holds.
- Record text now travels from engram to LiteLLM to TypeSafe. OpenRouter no longer carries it. The sender is the same cluster, and the destination is the same provider.
- TypeSafe applies its standard retention, and zero data retention is not confirmed. The posture above is unchanged.
- The route goes through the normal LiteLLM model pipeline. The key allow-list, the budgets and the spend tracking apply. LiteLLM prices the lane from the input tokens ($0.042 per million), so the spend dashboards show the cost under the key alias
engram. - The
/openrouter/alpha/decisionspass-through and its route grant stay for theoctopuskey.