Skip to content

engram sends record text to TypeSafe Jev for consolidate verdicts and search reranking, accepting standard retention without zero data retention

Date: 2026-09-24 Status: Accepted Deciders: Sean Brandt

engram 0.20.0 adds typed decisions. A typed decision is a question that a model answers with probabilities. engram asks the questions through OpenRouter’s Decisions API, which routes the model typesafe/jev-1.13 to TypeSafe on the US West Coast. Two features use it:

  • engram spine-review consolidate attaches an advisory relation verdict to each candidate pair. It sends up to 1500 characters of each record (its summary, then the head of its content). It runs only when an operator starts it.
  • The search reranker (memory.search.ranker: jev) runs on every search_memory and search_discovery call. It sends the query and up to 100 candidate records, 600 characters each.
  • Console query understanding (memory.search.understanding: jev) was added in engram 0.22.0. It runs when someone types a query on the console /search page. It sends only the query text. It suggests filter chips, and results change only when the user accepts a chip.

The provider does not train on inputs. It keeps inputs under its standard retention, and zero data retention is not confirmed. The 2026-07-10 ADR accepted a limited-retention posture for the Gemini embedder. This decision adds a second upstream that receives record text.

The engram retrieval eval that closed seanb4t/engram#605 did not measure the Jev reranker. It measured only that lexical ranking beats vector-only order on paraphrased queries (MRR 0.817 against 0.579).

Turn on both features in argocd/cluster-app/templates/agent-memory.yaml:

Value Setting
memory.decisions.provider jev
memory.decisions.baseURL https://llm.fzymgc.house (amended 2026-10-10, was https://llm.fzymgc.house/openrouter)
memory.decisions.path /v1/systemone (added 2026-10-10)
memory.decisions.apiKeySecret memory-mcp-litellm / litellm_api_key
memory.search.ranker jev
memory.search.understanding jev

The calls go through LiteLLM with engram’s own virtual key. The model and the timeouts keep the binary defaults: typesafe/jev-1.13, 10 s per decision call and 2 s per rerank. The amendment below changes the route.

  • Jev never changes a record. A consolidate verdict is advisory. A failed or slow rerank falls back to lexical order, and the search still succeeds.
  • Neither feature has a training risk, because the provider does not train on inputs. The remaining exposure is the retention window, which is the same class of exposure the 2026-07-10 ADR accepted.
  • The cost is small. A probe on 2026-09-24 used 315 input tokens and cost $0.000013. A full rerank is about 15k tokens, or roughly $0.0006 per search.
  • Search is the feature that recall uses most. Turning on the reranker gives a live comparison with lexical ranking that the eval did not provide.
  • Decisions only, with lexical search. This sends record text only during consolidate runs. It was not chosen, because it gives no evidence on whether Jev ranks better than lexical.
  • Neither feature. This keeps the current posture. It was not chosen, because the retention exposure is bounded and both features fail safe.
  • Record text from the spine leaves the cluster on every search. The volume is up to 100 records of 600 characters per call.
  • Each search takes up to 2 s longer when TypeSafe is slow. A timeout returns lexical order.
  • The original route recorded $0 spend in LiteLLM. The amendment below fixes that.
  • Console query text leaves the cluster on every console search. It contains no record content, scope names or tag vocabulary.
  • To turn off the reranker, remove memory.search.ranker. To turn off query understanding, set memory.search.understanding to off. To turn off all Jev calls, also remove memory.decisions. No change needs a migration.

Sign-off: human approval of the pull request that lands this ADR is the sign-off for the data-residency decision.

Amendment 2026-10-10: the native LiteLLM lane to TypeSafe

Section titled “Amendment 2026-10-10: the native LiteLLM lane to TypeSafe”

Sean decided on 2026-10-10 that all access to TypeSafe goes through LiteLLM and that no workload calls the TypeSafe API directly. engram now calls LiteLLM’s native decisions route, POST https://llm.fzymgc.house/v1/systemone. The lane typesafe/jev-1.13 in the LiteLLM model_list forwards the call to the TypeSafe API as typesafe/jev-1.13.0, with a TypeSafe key that only the LiteLLM pod holds.

  • Record text now travels from engram to LiteLLM to TypeSafe. OpenRouter no longer carries it. The sender is the same cluster, and the destination is the same provider.
  • TypeSafe applies its standard retention, and zero data retention is not confirmed. The posture above is unchanged.
  • The route goes through the normal LiteLLM model pipeline. The key allow-list, the budgets and the spend tracking apply. LiteLLM prices the lane from the input tokens ($0.042 per million), so the spend dashboards show the cost under the key alias engram.
  • The /openrouter/alpha/decisions pass-through and its route grant stay for the octopus key.