Disable the Headroom semantic response cache on both instances

Date: 2026-06-23 Status: Accepted Decision: hl-k1am Deciders: Sean Brandt

Context

Headroom ships an LRU+TTL semantic response cache that can return a cached completion for near-duplicate prompts. For agentic traffic (Claude Code sessions) and in-cluster app traffic (mealie, karakeep, octopus), prompt near-duplicates exist but a cached response may be stale relative to the current conversation state. Separately, Headroom’s --mode cache run-mode is a distinct feature (freezing prior turns to preserve provider prefix-cache), easily confused with the response cache.

Decision

Launch both instances with --no-cache; the semantic response cache is disabled. The --mode cache run-mode (agents instance) is retained and is orthogonal — it freezes prior turns for Anthropic prefix-cache preservation, not response deduplication.

Rationale

• The LRU+TTL response cache risks returning a stale completion on a near-duplicate agentic prompt where conversation state has evolved — especially hazardous for tool-calling agents where mid-conversation state matters.
• --mode cache (agents) addresses prefix-cache economics; it is a different cache layer from the response-deduplication cache that --no-cache disables. Conflating them would enable the wrong cache type.
• A per-app revisit path is preserved: --no-cache can be removed per-instance after measuring whether cache hits are safe for that traffic pattern.

Alternatives Considered

• Disable the response cache on both instances, --no-cache (chosen): eliminates stale-completion risk; the agents instance’s --mode cache still handles prefix-cache optimization orthogonally; can be re-enabled per-app after measurement.
• Enable the response cache (rejected): could eliminate redundant upstream calls for repeated prompts, but risks stale completions for near-duplicate agentic prompts where context has changed.

Consequences

• Positive: eliminates the stale-completion correctness risk for agentic and tool-calling traffic (resolves open question #3); establishes a clear separation between run mode (prefix-cache) and response cache (deduplication).
• Negative: forgoes any response-level deduplication savings initially.
• Neutral: per-app revisit is explicitly noted as a future option after measurement.