Declare telemetry disposable; greenfield table cutover

Date: 2026-07-05 Status: Accepted Decision: hl-r9en Deciders: Sean Brandt

Context

Upstream ClickStack seed schema churns roughly twice a month with no retrofit path — every seed is CREATE TABLE IF NOT EXISTS, so existing deployments drift from upstream improvements permanently (e.g. the 2.29 metrics-PK change applies only to fresh tables). The replicated rebuild (hl-uj69) additionally requires re-creating every table with a Replicated engine. A data policy was needed, not just a topology.

Decision

Telemetry (logs, metrics, traces, sessions) is declared disposable. The rebuild cutover drops and recreates all otel_* tables rather than migrating data; MongoDB (dashboards, alerts, source configs) and all URLs survive untouched. Drop-and-rebootstrap becomes the standing pattern for adopting future upstream schema changes.

Rationale

• No upstream migration tooling exists, and building one couples us to internals that churn ~2x/month.
• Disposable-data status is what makes drop-and-rebootstrap reusable on every future chart bump, not just this rebuild.
• The state that actually matters operationally (dashboards/alerts) lives in MongoDB and is preserved.

Alternatives Considered

• Tables-only greenfield (chosen): minutes-long window; no tooling to maintain; repeatable.
• Migrate/preserve telemetry through the schema change (rejected): requires tooling upstream does not provide; blocks the reusable pattern; preserves data explicitly declared low-value.
• Full-stack rebuild incl. MongoDB (rejected): hours-long observability gap and alert re-provisioning risk for no benefit over tables-only.

Consequences

• Positive: repeatable schema-evolution pattern; no migration tooling; MongoDB state and URLs preserved.
• Negative: all pre-cutover telemetry history permanently lost at each rebootstrap.
• Neutral: 14-30d retention going forward means the precedent recurs at low cost.