The drop.fzymgc.net Access edge and its origin trust: four ingest surfaces behind Cloudflare Access, with origin-side JWT verification deliberately declined under D-70 and a measured A3 verdict on header forgeability

Date: 2026-08-08 Status: Accepted Deciders: Sean Brandt

Context

Milestone v1.5’s write path needs a public surface. The split read/write ADR fixed the architecture (public GET terminates at the Cloudflare edge over R2’s custom domain), the edge policy ADR fixed what the read path does, and the credential-isolation ADR fixed the R2 workload credential. What none of them fixed is how a caller reaches the uploader pod at all: its write endpoint, its authentication, and what the origin may trust. This record fixes that.

One hostname now carries two very different surfaces. drop.fzymgc.net serves a browser SSO drop page (a human behind Keycloak) and a scripted API (service tokens in share.fish, a curl one-liner, and an iOS Shortcut). There are four ingest surfaces in total. And the service behind it has no user model of its own — Access plus Keycloak already are the account system, so the pod records who the edge says rather than authenticating anyone itself. Whether that is safe is the whole question this record answers.

The discipline this project earned in Phase 1 applies here too: every claim carries either its committed evidence file or an explicit argument by construction label, and a residual that was deliberately accepted is written as accepted, never as mitigated. The phase was measured end to end on 2026-08-14; where a measurement exists it is cited by filename.

Decision

Publish the uploader at drop.fzymgc.net through a Cloudflare Tunnel that dials the ClusterIP directly, gate it with a new Access application and two policies, mint three service tokens and alarm their expiry, and — the subversive part — decline to verify Cf-Access-Jwt-Assertion at the origin, relying instead on the edge plus a compensating Cilium ingress rule.

Nine decisions, D-68 through D-76, each with its reversibility rating carried through from the phase record:

Decision	What it fixes	Reversibility
D-68	The hostname is drop.fzymgc.net — one var.internal_services map entry yields both the tunnel ingress rule and the proxied CNAME, so they cannot drift apart	costly — the string is baked into clients
D-69	The tunnel dials the ClusterIP directly; no Traefik IngressRoute	reversible
D-70	The origin does not verify Cf-Access-Jwt-Assertion; headers stay read-and-not-verified	reversible — adding verification later is additive
D-71	A compensating Cilium ingress rule admits only the tunnel connector’s namespace on :8080	reversible
D-72	A new browser policy admits exactly two email domains, decoupled from the shared one	reversible
D-73	Split credential custody — terminal clients read Vault, the iOS pair lives in the Shortcut	costly for the iOS credential
D-74	Silent token expiry is alarmed with a fixed one-week lead time	reversible
D-75	The web toolchain is pnpm@11.8.0 via corepack, matching the rest of the repo	costly
D-76	ROADMAP SC#1 amended; status and content-type are vacuous discriminators for this phase	reversible

D-68 — the hostname, and what was never viable

hl-assets.dev was never viable, and must not be narrowed to make it work. That zone stamps Content-Security-Policy: sandbox zone-wide (kind = "zone", expression = "true" — tf/cloudflare/rulesets-hl_assets_dev.tf:231,253,272), the deliberate fail-closed backstop for Phase 2 SC#3. A bare sandbox blocks all script execution and forces an opaque origin, which would kill the same-origin CF_Authorization cookie the whole browser design depends on. fzymgc.house was rejected for reintroducing split-horizon DNS. So the page lives on a fresh hostname in a zone without that ruleset.

D-69 — no Traefik hop

cloudflared dials the hl-assets-uploader.hl-assets.svc.cluster.local:8080 ClusterIP directly. This is the estate’s existing Access pattern (status.fzymgc.net → uptime-kuma) and it costs no new manifest — the kustomization set-equalities survive untouched.

D-70 — the declined verification, and exactly what the exemption covers

The Go service does not verify Cf-Access-Jwt-Assertion. This overrides the research recommendation and is legitimate under Cloudflare’s own guidance, which makes validation optional for applications connected through Cloudflare Tunnel — which this one is.

State precisely what that exemption does and does not cover. It is an argument about whether a request reached the origin via Access at all: the tunnel is trusted transport, so if you were behind the tunnel, you passed a policy. It says nothing about whether the contents of a client-controllable header survive the trip. That gap is A3, and it is the subject of a live measurement below — because the audit ledger’s trust now rests on a measurement rather than on a mechanism.

Consequences that follow and were honoured: no ACCESS_TEAM_DOMAIN, ACCESS_AUD or ACCESS_JWT_REQUIRED env vars; internal/config/config.go’s variable slice stays at its 18 names; no JWKS dependency and no new request-path network I/O.

D-71 — the compensating ingress rule

D-70’s trust in the edge is only sound if nobody outside the tunnel connector can reach the pod while pretending the edge sent them. Cilium’s default is the problem: a policy that specifies an ingress section — including an empty one — puts its endpoints into ingress default-deny, and omitting the key leaves ingress unrestricted. D-71 fulfils Phase 3’s D-45 and adds an explicit ingress key to CiliumNetworkPolicy/hl-assets-egress naming the cloudflared-main pod and the probe path on :8080. A pod inside cloudflared’s namespace is the only in-cluster caller that can reach :8080.

D-72 — the deliberately narrower browser policy

Browser access uses a new cloudflare_zero_trust_access_policy, not a reuse of docs_allow. It admits exactly two email_domain values — fzymgc.email and fuzzymagic.com — narrower than docs_allow’s four, and deliberately decoupled so a future widening of the docs site never silently widens the uploader.

D-73 — split credential custody

The two terminal clients read their service token from Vault at secret/fzymgc-house/cluster/hl-assets/access-token. The iOS Shortcut’s pair lives in 1Password as the authoritative copy and is pasted into the Shortcut at setup, because Shortcuts cannot read 1Password at run time. Accepted with eyes open: the secret lives inside the Shortcut body and syncs through iCloud. This buys custody and rotation bookkeeping, not runtime secrecy (see accepted residual (b)).

D-74 — silent expiry is alarmed

Silent token expiry is alarmed via a cloudflare_notification_policy with the Expiring Access Service Token type. The lead time is a fixed one week (not expressible in Terraform for this alert type), and the verification harness independently gates at now+30d — neither substitutes for the other. The alert type takes no filters, so it fires for every Access service token in the account (see accepted residual (c)).

D-75 — pnpm toolchain

apps/hl-assets-uploader/web/ uses pnpm@11.8.0 via corepack, matching website/package.json’s exact pin and the pnpm/action-setup in .github/workflows/docs.yml — one JS toolchain across the repo. The Dockerfile web stage spends the seam Phase 3 opened, and the corepack fetch runs on both native multi-arch runners (D-48).

D-76 — the amended success criterion

ROADMAP SC#1 was amended: it no longer asserts on content-type. It now requires both a location: header prefixed https://fzymgc.cloudflareaccess.com/cdn-cgi/access/login/ and a www-authenticate: Cloudflare-Access header, via curl -i and never -L, with a transport-success guard first. Status and content-type are vacuous for this phase: the drop page is text/html — exactly like the Access login form — so those two axes cannot discriminate. The two headers are the only non-vacuous discriminators, and both were measured live on status.fzymgc.net.

Origin trust — the record’s centre

The boundary being defended is: what may the uploader record as “who did this”, given that the pod never authenticates anyone itself?

The answer has three parts.

Part 1 — the edge is the attestation boundary. drop.fzymgc.net is reachable only through the cloudflared-main tunnel, and only requests that passed an Access policy arrive over it. That is an argument by construction claim about transport: the tunnel does not carry cleartext from the public internet, so a request that reached the origin travelled through the edge. It is not a claim that the client-controllable identity headers were generated by the edge.

Part 2 — the compensating ingress rule closes the pod from every other caller. D-71’s Cilium ingress admits only the tunnel connector’s namespace on :8080, so no in-cluster workload outside cloudflared can reach the origin pretending to be the edge. Establishment is by the manifest, argocd/app-configs/hl-assets/networkpolicy.yaml, which now carries an ingress key naming cloudflared-main and the probe path.

Part 3 — A3, measured. The residual the decline leaves open is whether a forged client-controllable header survives the trip. A3 was probed live on 2026-08-14 against the deployed origin, recorded in a3-forged-header.observed.txt:

Arm	Probed with	Recorded identity	Recorded identity_source
Arm 1 (service token)	forged Cf-Access-Authenticated-User-Email: forged-arm1@example.invalid beside CF-Access-Client-Id + CF-Access-Client-Secret	"unauthenticated"	"none"
Arm 2 (browser cookie)	forged email beside CF_Authorization	not measured	not measured

Arm 1 came back TRUE for forgeability prevention: the edge stripped the client’s forged email header, and the record resolved to identity='unauthenticated', identity_source='none' — the forged value did not reach the record, and neither did the service-token client id. That is a measurement of current behaviour, not a documented guarantee: Cloudflare’s documentation corpus does not address inbound header stripping in either direction.

Arm 2 is not measured, and the measurement is declined — see accepted residual (e). It needs a real browser session with an authentic CF_Authorization cookie, which no headless probe in this phase held, and the operator-supplied cookie is rejected by Cloudflare with auth_status NONE. This is the authenticated-insider case — the one a family member with a valid session could exploit by forging a different email over their own cookie. Because a forged email outranks a genuine service-token client id in the origin’s own precedence, the append-only audit ledger is forgeable by an authenticated caller unless the edge strips the email as it did in arm 1. That the edge demonstrably strips on the service-token path is encouraging but is not evidence of the cookie path. The cleanup of the forged probe object and record is owed to Phase 6, item OPS-01 (filed in deferred-items.md).

This paragraph read “It is an OPEN finding” until 2026-08-15. It was amended, not deleted, when the residual was accepted: the risk is unchanged and still stated in full above — what changed is that the measurement is now formally declined rather than pending, with a reopening condition attached. ai-forged-header still FAILS closed on the unmeasured arm rather than reporting an unearned pass, so nothing here converts an absent measurement into a green one.

The three source comments that predicted Phase 5 would “put JWT verification in front” were corrected in place in the same pass as this ADR (see Consequences).

Residual mitigations — ESTABLISHED versus OWED

ESTABLISHED

Each cites the evidence that established it or names the manifest that carries it.

1. The ingress rule admits only the tunnel connector’s namespace. D-71 — argocd/app-configs/hl-assets/networkpolicy.yaml now carries the ingress key naming cloudflared-main and the probe path on :8080. argument by construction over the manifest.
2. The browser policy admits exactly two email domains. D-72 — cloudflare_zero_trust_access_policy in tf/cloudflare/access.tf with fzymgc.email + fuzzymagic.com. argument by construction.
3. The three service tokens are per-client-class, never shared. D-73 and D-86’s lineage from L-3 — ios_*, cli_*, curl_* pairs, each in its own service-token resource. argument by construction over tf/cloudflare/access.tf.
4. The binding cookie is enabled, and the forged email is stripped on the service-token path. Evidence: a3-forged-header.observed.txt (arm 1), 2026-08-14.
5. No new request-path network I/O was introduced by D-70 — /readyz keeps its measured property of making no network call by type. argument by construction.

OWED — each names the phase and gate that discharges it

Written as obligations, never as aspirations — an auditor cannot distinguish the two unless the document does. Dispositioned 2026-08-15 (Phase 6, plan 06-09). A disposition below names what discharged it; where something is built but not yet exercised, that is said rather than rounded up to closed.

1. Takedown and the tombstone verb. The write path’s takedown surface and the tombstone verb are owed. Discharged by: Phase 6, OPS-01 — CLOSED 2026-08-16 (plan 06-11), mechanism complete and rehearsed end to end.

   • Built: three verbs on the uploader binary — unshare (06-01), purge and republish (06-06) — reached through a second store.Takedown interface, so the request path’s store.Store still cannot delete or list.
   • Built: the store.Tombstone shape (06-04), whose retained and dropped field sets are pinned byte-for-byte by a committed golden fixture. The record survives at its own key carrying state=purged; only its contents are erased.
   • Documented: the procedure, the decision table and the edge-purge step (06-09).
   • What was outstanding, and for how long. At the 2026-08-15 disposition this item read NOT closed: the end-to-end rehearsal on a live object had not run, and no verb in this phase had touched a real bucket. That was correct and is kept here rather than deleted — a reader should be able to see the obligation and its discharge, not only the discharge. The stated closing condition was “it closes when 06-10 produces its transcript”. 06-10 did not produce one: it halted at its own <precondition>, because the image carrying the verbs had never been rebuilt, merged and synced, and nobody owned that step. Plan 06-11 owned it.
   • What closed it, 2026-08-16. Two transcripts, both non-empty and committed, both under .planning/milestones/ws-image-host-r2-2026-09-19/milestones/v1.5-phases/06-operations-documentation/: 06-rehearsal-unshare-republish.observed.txt and 06-rehearsal-purge.observed.txt.
   • The pod they ran against. hl-assets-uploader-86b789f97d-q5bjx, whose imageID resolved to sha256:0bd2dc3e41b69fc9e789a2750d39a7e184c253caa155f150d236fea7fd34f86c — the OCI index digest pinned in argocd/app-configs/hl-assets/deployment.yaml — with the binary inside answering -version with 0.2.0, equal to apps/hl-assets-uploader/VERSION. Both readings are in 06-pod-digest.observed.txt, taken before any rehearsal stage ran.
   • What the run proved. All three verbs touched a real bucket, on a disposable object uploaded for the purpose. unshare was proven reversible by bytes, not by a state flag: the republished object’s SHA-256 equals the one the URL served beforehand. purge deleted the public object and the retained original — both now 404 — and left a tombstone whose key set is byte-equal to tombstone.golden.json’s eight audit fields, carrying no content-describing field and no management token. OPS-01’s “rehearsed end to end” is therefore satisfied by measurement rather than by declaration.

2. The monitoring and cost-alerting sweep. The drop surface’s monitoring and cost-alerting posture is owed. Discharged by: Phase 6 — both halves declared; live on merge.

   • Cost alerting: four cloudflare_notification_policy resources (06-05), the cross product of anomaly and cost-ceiling thresholds against Class A and Class B operations. The product literals were established by probing the account’s own validator, with negative and ordering controls, rather than guessed. Residual (d) below does not apply: the alarms are declared in Terraform, not left as dashboard state.
   • Monitoring: two Uptime Kuma monitors (06-07) — a keyword canary on the read path and an authenticated /readyz probe on the write path — both routed to cluster-edge explicitly.
   • Additionally: three recurring operational items measured weekly by a scheduled workflow (06-08), each of which previously failed silently.
   • Apply ordering, stated so nothing here reads as live: tf/cloudflare and tf/uptime-kuma are VCS-connected HCP workspaces that apply only on merge to main. At the time of writing none of these resources exists in the account. They are declared, which is what this obligation asked for; they are not yet firing, and must not be described as though they were.

Accepted residuals — named, dated, given their reopening condition

Each is deliberately accepted and stated as accepted, never as mitigated.

1. (a) A pod inside the tunnel connector’s namespace could forge identity headers. Accepted 2026-08-08. The cloudflared namespace runs one workload, and narrowing the ingress selector to the connector’s pod label would add a second thing to drift, diverging from all three in-repo precedents. Reopens if: a second workload lands in cloudflared without re-narrowing the selector.

2. (b) The iOS service-token pair lives inside the Shortcut body and syncs through iCloud. Accepted 2026-08-08, D-73. This buys custody and rotation bookkeeping, not runtime secrecy. Reopens if: the Shortcut definition is shared or a device is compromised with the body intact.

3. (c) The expiry notification has no filters, so it fires for every service token in the account rather than only this phase’s three. Accepted 2026-08-08. Today a superset of one group, which is harmless. Reopens if: unrelated service tokens are minted and their expiry alarms become noise.

4. (d) If the notification grant was declined, the dashboard-created alert is untracked state invisible to drift detection — the same class as the estate’s existing unmanaged alert. Accepted 2026-08-08. Reopens if: the dashboard-created alert is later relied on as drift-tracked.

5. (e) A3 arm 2 is not measured: the edge’s stripping of a client-forged Cf-Access-Authenticated-User-Email from an authenticated browser-cookie caller is unverified, so the append-only ledger is forgeable by an authenticated caller. Accepted 2026-08-14 by the operator, verbatim: “closed as an ACCEPTED RESIDUAL, not a defect. The checkpoint asks us to measure a Cloudflare-owned behaviour, which repo rule afja1qt48h puts out of scope — the same reasoning that rejected the scale-Keycloak-to-zero break-glass form in this phase. Unlike break-glass there is NO owned substitute: D-70 leaves the Go service with no defence of its own, so the property is the vendor’s by construction.”

   What is retained rather than waived: arm 1 IS measured PASS (a3-forged-header.observed.txt — the forged email did not reach the record on the service-token path), and the ai-forged-header gate FAILS closed on the unmeasured arm rather than reporting an unearned pass. The measurement is additionally blocked from the Cloudflare side today (auth_status NONE on the supplied cookie; signature, audience, IP, expiry and app-duration all verified clean).

   This is an acceptance, not an impossibility: a lever exists and this ADR names it. Reopens if: the ledger’s integrity becomes load-bearing — at which point verify Cf-Access-Jwt-Assertion in the Go service, which re-opens PORT-02 in both directions and adds a JWKS dependency, and which would make the property ours to measure.

6. (f) The one-time management token is written to the user’s own local state file. Accepted 2026-08-08, recorded here 2026-08-15 with the reopening condition it had been missing. share.fish writes it to $XDG_STATE_HOME/hl-assets/tokens.tsv at mode 600 (share.fish:107,180-182, documented at docs/operations/hl-assets-uploader.md:109-117). The token is a takedown capability for exactly one object and can never be reissued, so discarding it loses the capability entirely on the only surface that can capture it. Reopens if: the file is ever synced off-device, the tokens become capabilities over more than the single object they were issued for, or a rotation/expiry path is added that makes retention unnecessary.

7. (g) The uptime monitor’s Access client_secret is unredacted in Terraform plan output and state. Accepted 2026-08-15 (Phase 6, 06-07). MEASURED, not assumed: the uptimekuma_monitor_http.headers attribute is not marked sensitive at the pinned provider version, read with basic_auth_pass — which is marked sensitive — as a firing control in the same call, so the absence is a real absence rather than a reading of the wrong object. The credential is therefore rendered in the main-cluster-uptime-kuma plan output and stored unredacted in its state, which is materially worse than the Pushover credentials sitting beside it.

   Written as accepted, never as mitigated, because neither alternative exists: Cloudflare Access authenticates on the CF-Access-Client-Id / CF-Access-Client-Secret request headers specifically and does not accept Basic authentication, and the provider ships no write-only variant of any attribute at this version. What bounds it — and does not change the disposition — is that the exposed credential is this monitor’s own client class and nothing else’s, which is precisely what a fourth service token on its own Vault path bought. The same exposure was accepted from the writing side in the Cloudflare workspace. Reopens if: the provider gains a write-only or sensitive headers variant, or Access gains an authentication mode whose credential attribute is marked sensitive — at which point this becomes a mitigation that was declined rather than one that was unavailable.

Abstentions

The phase gate’s abstentions from 05-08-SUMMARY.md were recorded as abstentions here so a reader does not read a green where none was held. Both have since been resolved (updated 2026-08-15); the original text is kept below its correction so the record shows what was believed and when.

• ai-grants — RESOLVED 2026-08-14 by plan 05-10 (gap G-05-3). This section previously said the id stayed red because “the bootstrap token cannot self-read its own permission groups”. That diagnosis was wrong. The bootstrap token is ACCOUNT-owned, and /user/tokens/* serves user-owned tokens only, so it answered 1000 Invalid API Token structurally — regardless of which permission groups were held. No grant could ever have turned that endpoint family green. read_permission_group_ids now reads /accounts/{account_id}/tokens/*, and a live run on the same unchanged credential reports PASS ai-grants (ai-grants-account-family.observed.txt, 2026-08-14T23:48:57Z, passed=5 failed=0). The waiver at 05-08-SUMMARY.md:76 is retired along with its diagnosis.
• ai-token-upload — RESOLVED. The scheme-less PUBLIC_BASE_URL defect it recorded is fixed; evidence in public-base-url-scheme.observed.txt.

Neither was ever a claim of an established control. Both were genuine measurement gaps at the time of writing; both have since been measured.

Alternatives Considered

• Origin-side JWT verification (the research recommendation) — declined under D-70. Cloudflare’s own guidance makes validation optional for Tunnel-connected applications, and turning it on would add a JWKS dependency and re-open the PORT-02 env-set gate in both directions. Trust instead rests on the edge plus the compensating ingress rule, with A3 measured rather than assumed.
• Narrowing the hl-assets.dev CSP ruleset to make the drop page work there — rejected. expression = "true" is the deliberate fail-closed backstop for Phase 2 SC#3; narrowing it weakens a zone-wide control to serve one page.
• A Traefik IngressRoute in front of the pod — rejected. Adds a hop and a manifest for no access benefit; the estate’s established Access pattern dials the ClusterIP directly.
• Reusing docs_allow for browser access — rejected. It carries four domains and would silently widen the uploader whenever the docs site widens.
• A shared service token — rejected outright. One token per client class (L-3) means revoking one family device’s credential leaves the terminal clients working; a shared token defeats that.

Consequences

Positive:

• The phase’s decisions and residuals now survive outside .planning/, every claim carrying its evidence file by name or an explicit construction label.
• A3’s verdict is a measurement on the default branch, not a mechanism argument: on the service-token path the edge stripped the forged email and the record says identity='unauthenticated'. A trusted reader of the audit ledger can now tell measured from intended without re-running anything.
• D-71 closes the pod from every in-cluster caller except the tunnel connector, so the safety of declining JWT verification does not depend on the network policy doing nothing.

Negative:

• Arm 2 of A3 is unmeasured. The authenticated-insider forge is an OPEN residual, not a discharge — recorded as finding (1) above and filed for Phase 6 cleanup.
• The accepted residuals are real: a pod in cloudflared could forge headers, the iOS secret syncs through iCloud, and the expiry alert fires for every token in the account.
• The three stale source comments predicting Phase 5 would verify JWT were corrected in place; the false sentence no longer exists on the default branch.

Sign-off: decision, date (2026-08-08) and deciders (Sean Brandt) are recorded above. Human approval of the pull request that lands this ADR is the sign-off for the accepted residuals, for the open A3 finding, and for the three comment corrections.