Upgrade engram embedding model to Qwen3-Embedding-8B (4096-dim) via OpenRouter ZDR
Date: 2026-06-13 Status: Superseded by 2026-07-10-engram-gemini-embedder-retention-posture Decision: hl-57x5 Deciders: Sean Brandt
Context
Section titled “Context”engram’s embedder ran against an in-cluster ollama bge-m3 lane (1024-dim, :11435) inherited from a prior embedding-SPOF mitigation (hl-92xp). Migrating off the contended local lane forced a model-selection decision: retain bge-m3 via cloud, or upgrade. The choice fixes engram’s Qdrant collection vector dimensionality (1024 vs 4096) and the embedding provider contract for all future memory writes.
Decision
Section titled “Decision”Upgrade engram’s embedder to qwen/qwen3-embedding-8b (4096-dim) via OpenRouter ZDR, populating a fresh memory_v2 Qdrant collection via engram reindex before a verified cutover.
Rationale
Section titled “Rationale”- Qwen3-8B leads MMTEB retrieval (70.88) while tied for cheapest ($0.010/1M tok) — quality and cost are not in tension here (+16.3 pts over bge-m3 54.60 at the same price).
- The 1024→4096 dimension change is a breaking Qdrant schema change; once accepted, all future engram memory writes are pinned to 4096-dim until the next deliberate migration.
- OpenRouter multi-provider routing (3 ZDR upstreams: Nebius/DeepInfra/SiliconFlow) reduces single-provider outage risk without in-cluster serving infrastructure.
- gemini-embedding-001 rejected (15× cost, −3.17 retrieval pts); openai/text-embedding-3-large strictly dominated (MMTEB 59.27 at 13× cost).
Alternatives Considered
Section titled “Alternatives Considered”- qwen/qwen3-embedding-8b via OpenRouter ZDR (4096-dim) — CHOSEN. MMTEB-R 70.88 (highest), $0.010/1M (tied cheapest), 3-provider ZDR fan-out. Cost: requires re-embed + new collection; heavier per-request latency (tolerable at interactive low-QPS).
- baai/bge-m3 via OpenRouter ZDR (1024-dim, keep) — rejected. No rebuild/hazard, same $0.010/1M, but MMTEB-R 54.60 (worst); foregoes +16.3 pts at equal cost.
- gemini-embedding-001 (3072-dim) — rejected. First-party stability, MMTEB-R 67.71, but 15× cost ($0.150/1M) and −3.17 pts vs Qwen; single-provider.
- openai/text-embedding-3-large (3072-dim) — rejected. Strictly dominated: MMTEB-R 59.27 (lowest cloud option) at $0.130/1M (13×).
Consequences
Section titled “Consequences”Positive:
- Largest retrieval-quality jump available at negligible cost delta (~141 points, re-embed is fractions of a cent).
- Cloud-hosted; eliminates GPU contention with octopus batch indexing on the shared seattle ollama lane.
- ZDR preserved via a dedicated backend with structured field override (per ADR hl-0eb).
Negative:
- memory_v2 is a hard 4096-dim contract — future model changes require another reindex + collection rebuild.
- OpenRouter becomes the sole embed path (accepted SPOF, tracked in hl-73s.23).
- Rollback after PR-B cutover requires seattle :11435 to still be running.
Neutral:
- engram’s litellm.url is unchanged; the swap is a gateway backend/route change + chart values.