Self-Hosted Memory Layer for Claude Code — Bake-Off Design (B vs C)

Bead: hl-73s
Date: 2026-05-30
Status: Design (pending design-reviewer gate)
Author: Sean Brandt (with Claude Opus 4.8)

1. Problem & goals

Stand up a fully self-hosted, correctable memory layer subordinate to Claude Code, replacing claude-mem. Claude Code remains the runtime; the memory layer attaches via an MCP server plus lifecycle hooks and does not own the agent loop.

Hard constraints (non-negotiable):

No cloud egress. No mem0 SaaS. No cloud LLM for fact extraction or embeddings. Local Ollama only. No data leaves local machines.
Reuse existing infra where it fits: k3s, in-cluster LiteLLM, Vault/ExternalSecrets, and Mac-mini Ollama. Neither candidate uses CNPG PostgreSQL; Qdrant is deployed fresh for the eval (there is no in-cluster Qdrant today — see §2).
Correctable: a specific stored fact must be reliably deletable and editable. This is the primary reason for leaving claude-mem.
Subordinate to Claude Code: MCP + hooks, not an agent runtime.

Success = a one-week throwaway eval that produces a clean go/no-go decision against pre-defined pass/fail thresholds, killable cleanly if memory proves not worth it.

2. Verification findings (grounded; see bead hl-73s notes)

Infrastructure reality (stale-assumption flag). The cluster repo inventory shows the “existing infra” is split: PostgreSQL (CNPG main, PG18.1) and LiteLLM run in k3s, but Qdrant and Ollama are NOT in the cluster — Ollama runs on the Mac mini, and the cluster’s only LLM path is LiteLLM → OpenRouter (cloud). Consequently mem extraction/embeddings must reach local Ollama on the Mac mini, fronted by in-cluster LiteLLM, never the OpenRouter-backed routes; and Candidate C’s Qdrant is deployed fresh to k3s for the eval rather than reusing an existing in-cluster Qdrant (there is none).
mem0 is excluded from the bake-off. Verified against the mem0 repo: graph memory was removed from the OSS SDK (v2.0.0); temporal reasoning and memory decay are Platform-only. The current V3 algorithm is single-pass ADD-only — add() no longer emits UPDATE/DELETE, so contradictory facts accumulate (issue #4956). The #4573 audit found 97.8% junk across 10,134 entries, worst with a tiny local extraction model (gemma2:2b); dominant junk was system-prompt restating, hallucinated profiles, and (2.1%) secret/PII leakage into the vector store. mem0’s correctability (delete/update by id, history) is fine, but operated under our constraints (local + infer=false) it degrades to “a worse version of Candidate C.” It is therefore not a contender.
Temporal storage vs retrieval: mem0 OSS stores created_at/updated_at + history but cannot filter/rank on time (Platform-only). Graphiti, by contrast, makes bi-temporal the core data model in OSS.
Graphiti (Zep OSS, Apache-2.0): bi-temporal knowledge graph; each edge carries valid_at/invalid_at/created_at/expired_at; automatic edge invalidation on contradiction (resolve_edge_contradictions) — the supersession mem0 OSS lacks, not paywalled. Hybrid retrieval (semantic + BM25 + graph BFS), no LLM-in-loop reranking. Graph DB mandatory (FalkorDB/Neo4j/Kuzu). Local Ollama via OpenAIGenericClient base_url. Official MCP server. Peer-reviewed (94.8% DMR > MemGPT; +18.5% LongMemEval; −90% latency vs baseline).
Go stack for Candidate C verified current: official modelcontextprotocol/go-sdk v1.4.0+ (stdio + streamable-HTTP, struct→JSON-Schema inference, net/http handler); qdrant/go-client v1.18.2 (gRPC full CRUD); embeddings via LiteLLM OpenAI-compatible /v1/embeddings.

3. Decisions (from brainstorming)

Decision	Choice
Deployment target	All-in-cluster (k3s), throwaway `mem-eval` namespace via ArgoCD
LLM/embedding path	mem server → in-cluster LiteLLM → Mac-mini Ollama (`:11434`)
Cloud-egress guardrail	LiteLLM virtual key allowlisted to `ollama/*` routes only (structural, not convention)
Extraction model (B)	Capable local model (`qwen2.5:14b` class) — never a tiny model
Embedding model (both)	One model, fixed dims (e.g. `nomic-embed-text`, 768d)
Candidate B store	FalkorDB (Redis-based, lightest)
Candidate C store	Throwaway Qdrant deployed to k3s
Candidate C language	Go, build-your-own (official go-sdk + qdrant/go-client)
Memory scope	Project-default + opt-in global tier; namespace by stable repo identity, with workspace/worktree/repo-path/base-dir as provenance metadata
Ingestion trigger	Agent-controlled `store_memory` during the session (primary), nudged by a Stop hook; plus a measured firehose baseline (external transcript replay) to quantify junk delta. SessionEnd is read-only (verified) — cannot host propose/confirm.
Eval feed	Real Claude Code sessions across multiple repos, ~1 week, both backends fed the same sessions

4. Shared foundation (identical for B and C)

4.1 Data path & guardrail

Claude Code (Mac mini) ──MCP/https──> memory server (k3s)
                                          │
                  extraction(B) + embeddings ──> LiteLLM (k3s) ──> Ollama (Mac mini :11434)

LiteLLM config gains a wildcard route model_name: "ollama/*" → litellm_params.model: "ollama_chat/*" (chat) plus an embeddings route, both with api_base = Mac-mini LAN endpoint (verified: LiteLLM supports provider-prefix wildcard routing).
mem servers authenticate with a LiteLLM virtual key whose models allowlist contains only ollama/* (or, if wildcard-in-key proves unsupported, the enumerated names ollama/qwen2.5:14b + ollama/nomic-embed-text). LiteLLM rejects any request for a model not in the key’s allowlist (verified); since the key contains zero OpenRouter models, cloud egress is structurally impossible regardless of wildcard support.
Mac-mini Ollama: OLLAMA_HOST=0.0.0.0; :11434 restricted to the cluster’s egress source IPs via the Mac-mini packet filter (pf) / LAN ACL (exact rule is a plan-time detail).

4.2 Namespace & provenance model

Namespace key = stable repo identity (git common-dir / remote slug), identical across all jj worktrees/workspaces, so a project’s memory does not fragment across the 11 workspaces.
Tiers: project:<repo> (default) and global (opt-in durable cross-project facts).
Provenance metadata on every memory: repo, workspace, worktree_path, base_dir, source (user-said | agent-inferred | imported), category, created_at.
B: group_id = tier key; provenance in episode/entity metadata. C: Qdrant payload fields scope, repo, workspace, worktree, base_dir, filtered at query time.

4.3 Claude Code wiring (both registered in parallel: `mem-b`, `mem-c`)

MCP over streamable-HTTP: claude mcp add --transport http ….
SessionStart hook (verified shape): resolve project:<repo> + global, query memory, emit recency-ordered current facts via
```
{"hookSpecificOutput": {"hookEventName": "SessionStart", "additionalContext": "<memories>"}}
```
(There is no systemPrompt field; additionalContext is the mechanism.)
Ingestion = agent-controlled (corrected from SessionEnd). SessionEnd hooks are read-only — they cannot inject candidate memories back for confirmation — so propose/confirm is implemented as: Claude calls the store_memory tool during the session, guided by the CLAUDE.md rules below + a SessionStart reminder; a Stop hook optionally nudges an end-of-work capture pass (block-with-reason), gated to fire at most once per session. Exact Stop-hook wiring is a plan-time detail.
CLAUDE.md memory instruction: store decisions/preferences/conventions/gotchas; NEVER store transient state, secrets, timestamps, or system-prompt restating (the #4573 junk taxonomy encoded as negative rules).

4.4 Deployment & secrets

Two sibling ArgoCD Applications — mem-b and mem-c — both targeting the mem-eval namespace. Each Application carries its own resources-finalizer so deleting either cascades cleanly (per the per-Application finalizer lesson — a single finalizer on one app would orphan the other’s resources on teardown).
Secrets (LiteLLM key, Qdrant API key, FalkorDB password) via Vault + ExternalSecrets, per repo convention.

5. Candidate B — Graphiti temporal graph (Python, deploy-as-is)

Components: FalkorDB + official Graphiti Python MCP server (containers).
Capabilities: automatic supersession (bi-temporal edge invalidation), point-in-time truth, relationship recall, hybrid search.
Tools: add_episode, search_facts, search_nodes, get_episodes, delete_entity_edge, delete_episode, clear_graph, get_status.
Correctability: delete edge/episode by UUID; “edit” = invalidate-old + add-new (history preserved).
Accepted costs: seconds-per-episode async ingestion (selective ingestion only), eventual-consistency lag, local-GPU burn, SEMAPHORE_LIMIT kept low.

6. Candidate C — Go explicit-store (build-your-own, ~300 LOC)

Components: throwaway Qdrant (k3s) + Go MCP server (modelcontextprotocol/go-sdk, streamable-HTTP) + qdrant/go-client + small LiteLLM /v1/embeddings client. Single static binary → distroless image.
Tools: store_memory(content, scope, category, tags), search_memory(query, scope, k), get_memory(id), update_memory(id, …) (re-embeds), delete_memory(id), list_memories(scope), delete_all(scope).
Capabilities: zero junk by construction (deliberate writes), instant writes (~ms), trivially correctable (a Qdrant point you own), recency via created_at payload sort.
Accepted costs: no automatic memory formation (hook + discipline does it), no native supersession (explicit update/delete), “temporality” is created_at sorting.

7. Throwaway eval plan

7.1 Stand-up (minimal)

k8s namespace mem-eval holds both stacks (FalkorDB+Graphiti, Qdrant+Go server); LiteLLM ollama/* routes + allowlisted key; both MCP servers + hooks registered.
Logical memory scopes (NOT k8s namespaces) within the stores: a datestamped eval-2026-05 scope (group_id / Qdrant payload scope) for the curated run, and a separate eval-firehose scope for the measured baseline. Both are trivially droppable at teardown (clear_graph / drop-collection / delete_all).

7.2 Feed

Real Claude Code sessions across multiple repos for ~1 week, both backends fed the same sessions (apples-to-apples). Curated ingestion primary.
Firehose baseline runs on Candidate B only (the extraction candidate), via an external post-session transcript replay into the eval-firehose scope — measuring Graphiti’s extraction junk rate vs. curated. (A firehose on C is not meaningful: C has no extraction LLM, so auto-storing every turn measures raw-volume noise, not extraction junk.)

7.3 Signals & pass/fail (defined up front)

#	Signal	PASS	FAIL (kill)
1	Curated junk rate (manual audit of sample)	< 10%	> 25%
1b	Firehose junk rate (baseline)	record the delta	—
2	Supersession correctness (B)	≥ 90%	< 70%
3	Deleted facts stay deleted	0 reappearances	any reappearance
4	Recall usefulness (should-remember moments)	≥ 70%	< 50%
5	Correction effort	< 30s, 1 deterministic call	multi-step / flaky
6	Write→searchable latency	B seconds / C ms (record)	B queue backs up beyond a workday
7	Cloud egress (LiteLLM logs)	*0 non-`ollama/` calls**	any cloud call

Measurement methodology (Signals 2 & 4 need ground truth):

Signal 4 (recall usefulness): maintain a running should-remember log during the week — whenever Claude lacks context it demonstrably should have had, log a (query, expected-memory) pair. Pre-seed with a small scripted probe set so the denominator isn’t empty. At week end, replay the log against each store and score hits.
Signal 2 (supersession, B): inject a scripted set of N known contradiction pairs (employer/version/decision changes) during the eval; verify the old fact is invalidated and the new one is returned as current.
Signal 1 (junk): manual audit of a fixed random sample (or the full set) per scope at week end, classified against the #4573 junk taxonomy.

7.4 Kill criteria

Neither passes junk + recall → memory layer not worth it; stop, keep disciplined CLAUDE.md notes.
B fails if supersession < 70%, or latency unusable, or junk > 25% despite curation → graph extraction doesn’t earn Neo4j/ops.
C fails only if recall < 50% (too sparse); its junk/correctability pass by construction.
Teardown: delete both ArgoCD Applications (mem-b and mem-c; finalizers cascade), drop Qdrant collection + FalkorDB data, revoke LiteLLM key, remove hooks + MCP registrations.

7.5 End-of-week decision

Pick winner (B, C, both for different scopes, or neither), or extend with tuning. Default lean: C is the likely keeper (zero-junk, instant, owned, correctable); B is the “is temporal-graph worth the ops?” experiment — graduates to Neo4j only if supersession + relationship recall earn it.

8. Recommendation & case against

Recommendation: run both in parallel (cheap; same sessions). Lean toward C as keeper, B as the graph experiment.

Case against: two stacks + hooks + LiteLLM routing for a week is real setup cost; a modest need may be better served by disciplined CLAUDE.md notes or markdown. Memory layers have a poor track record — the eval must keep “neither” a live outcome.

9. Out of scope

Production hardening / HA of the chosen system (separate follow-up after a winner is picked).
Neo4j migration for B (only if B wins and graph earns it).
Importing historical claude-mem data (eval uses fresh sessions).
Multi-user / team memory (single-operator for now).

10. Open questions for reviewer

Are the §7.3 thresholds calibrated right, or too lenient/strict for a one-week sample?
Is the qwen2.5:14b-class extraction model the right capability/latency trade for B on an M4 Max?
Is the Stop-hook capture nudge worth the per-stop friction, or should ingestion be purely agent-discretionary (tool + CLAUDE.md only)?