Skip to content
fzymgc-house Cluster Docs

Per-file recency via bare blob-less git clone at index time (octopus fork)

Date: 2026-06-08 Status: Accepted Decision: hl-0d4 Deciders: Sean Brandt

Context

Section titled “Context”

Octopus chunk payloads carry only indexedAt (the indexing-run timestamp, identical for every chunk), giving no signal for when individual files last changed. The GitHub full-index flow is REST-only and has never cloned a repo; adding commit-history metadata requires a new git invocation on that path. The recency scan must not block or fail an index run, must not transmit file contents, and must authenticate consistently with existing codebase conventions.

Decision

Section titled “Decision”

Perform a bare blob-less clone (--bare --filter=blob:none) at index time, run git log --format=COMMIT:%cI --name-status in one pass to build a Map<path, lastCommitISO>, and write the result as lastModifiedAt: string | null on every chunk payload. The scan runs in a finally-guarded temp directory; any failure yields a warning via OnLog without blocking the index.

Rationale

Section titled “Rationale”
  • Blob-less clone transfers no file contents — cost is proportional to commit-history depth, not repository size, making it viable for large repos.
  • git is already in the runner image (apps/web/Dockerfile:50) and execFileAsync is already used by the indexer; no new runtime dependency.
  • Auth via GIT_CONFIG_* env headers mirrors the existing Bitbucket/GitLab clone convention and avoids embedding tokens in git URLs (which surface in error logs).
  • Graceful failure (warn + continue) is structurally required so a network or git error never aborts an index run.

Alternatives Considered

Section titled “Alternatives Considered”
  • Bare blob-less clone + one-pass git log (chosen): commits/trees only; rename tracking via R<score> entries; one mechanism for all three providers. Weakness: net-new clone infrastructure on the GitHub REST-only path.
  • REST-only — GitHub Commits API per file: no git binary needed, but one API call per file → rate-limit exhaustion on large repos; GitHub-only.
  • Full clone: simple but transfers all file contents; prohibitively expensive for large repos.

Consequences

Section titled “Consequences”
  • Positive: per-file recency signals across all three index providers; lastModifiedAt: string | null becomes the canonical chunk-payload recency contract; --shallow-since window caps history cost.
  • Negative: the GitHub indexing path gains a git network operation it previously lacked; incremental (webhook) re-indexing cannot re-run the scan — it stamps the push head-commit timestamp or null.
  • Neutral: Qdrant payloads are schemaless — additive field, no collection migration; pre-existing chunks read as “age unknown” until a full re-index.