Keycloak DCR Traefik-Deny Removal (Approach G) Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Unblock agentgateway-proxied RFC 7591 Dynamic Client Registration (DCR) by removing the self-defeating Traefik deny-client-registration rule + middleware, leaving Keycloak’s loopback Trusted-Hosts policy as the sole DCR control.

Architecture: Pure GitOps manifest change in argocd/app-configs/keycloak/. Deleting the priority-1000 /clients-registrations IngressRoute rule + its ipAllowList Middleware lets that path fall through to the existing bare Host(id.fzymgc.house) route → keycloak-service:8443 (ICA1 ServersTransport). agentgateway’s DCR hairpin then reaches Keycloak; the loopback Trusted-Hosts policy (host-sending=false, source-IP irrelevant) governs registration. No agentgateway / CoreDNS / TLS-trust change.

Tech Stack: Traefik v3 IngressRoute/Middleware CRDs, Kustomize, ArgoCD (GitOps, targetRevision: HEAD), yamllint.

Spec: docs/engineering/specs/2026-06-29-keycloak-dcr-traefik-deny-fix-design.md Bead: hl-thsf.1 · Follow-ups: hl-thsf.2, hl-g3xy

Grounding: All line references are against origin/main (commit a1e2bc48 / PR #1447). The local working copy MUST be reset onto main@origin before editing (jj new main@origin); a stale copy shows a 36-line ingress-route.yaml with no deny rule. Verify the file is 78 lines before starting.

Implementation Tasks

Task 1: Remove the Traefik DCR deny (rule + middleware)

Files:

Modify: argocd/app-configs/keycloak/ingress-route.yaml (remove lines 12-29 rule+comment, lines 39-50 Middleware+comment)
Step 1: Pre-flight — confirm the tree is current

Run: wc -l argocd/app-configs/keycloak/ingress-route.yaml && rg -c "deny-client-registration" argocd/app-configs/keycloak/ingress-route.yaml Expected: 78 lines and 2 matches. If you see 36/0, STOP — reset onto main@origin first (jj new main@origin).

Step 2: Delete the priority-1000 deny route rule and its comment

Remove this exact block (the comment lines 12-19 and the rule lines 20-29):

    # Deny anonymous Dynamic Client Registration on the PUBLIC (LAN-reachable) route.
    # agentgateway proxies DCR to keycloak-service INTERNALLY (pod->pod, bypassing
    # Traefik — confirmed: Keycloak sees the agentgateway pod IP), so this rule only
    # blocks LAN/external-direct /clients-registrations. Paired with the dcr Job's
    # host-sending-registration-request-must-match=false: the Keycloak-side source
    # check is off (no stable pod value to whitelist), and this Traefik deny is what
    # keeps the anonymous-DCR surface cluster-internal per ADR hl-xktj. Higher priority
    # than the bare Host rule so it wins for this path. [hl-thsf]
    - match: Host(`id.fzymgc.house`) && PathPrefix(`/realms/fzymgc/clients-registrations`)
      kind: Rule
      priority: 1000
      middlewares:
        - name: deny-client-registration
      services:
        - name: keycloak-service
          port: 8443
          scheme: https
          serversTransport: keycloak-internal-tls

After removal, the routes: list contains only the bare Host(id.fzymgc.house) rule.

Step 3: Delete the deny-client-registration Middleware and its comment

Remove this exact block (the --- separator at line 39, the comment lines 40-41, and the Middleware lines 42-50):

---
# Deny-all middleware: ipAllowList of an unreachable range → every real client gets 403.
# Attached only to the /clients-registrations path rule above. (Traefik v3 — ipAllowList.)
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: deny-client-registration
  namespace: keycloak
spec:
  ipAllowList:
    sourceRange:
      - 127.0.0.1/32

The ServersTransport keycloak-internal-tls block (and its comment) MUST remain unchanged — the bare Host route still references it.

Step 4: Verify no dangling references to the removed middleware

Run: rg -n "deny-client-registration" argocd/app-configs/keycloak/ingress-route.yaml Expected: no matches (exit 1). The only remaining repo reference is the prose mention in dcr-trusted-hosts-job.yaml:18, fixed in Task 2.

Step 5: Render and lint

Run: kubectl kustomize argocd/app-configs/keycloak > /dev/null && echo KUSTOMIZE_OK Expected: KUSTOMIZE_OK (no Kustomize error). Run: yamllint -c .yamllint.yaml argocd/app-configs/keycloak/ingress-route.yaml Expected: no errors.

Step 6: Sanity-check the rendered output no longer contains the Middleware

Run: kubectl kustomize argocd/app-configs/keycloak | rg -c "kind: Middleware|deny-client-registration" Expected: 0.

Step 7: Commit

Commit using VCS-appropriate commands per references/vcs-preamble.md. Message: fix(keycloak): remove self-defeating Traefik DCR deny — unblock agentgateway DCR [hl-thsf.1]

Task 2: Correct the now-false DCR comments in the trusted-hosts Job

Files:

Modify: argocd/app-configs/keycloak/dcr-trusted-hosts-job.yaml:11-22 (comment block only — no functional kcadm change)

The loopback TRUST MODEL (trusted-hosts=["localhost","127.0.0.1"], host-sending=false, client-uris-must-match=true) is correct and stays. Only the false network-path claims are corrected: the “(no SNAT)” parenthetical (contradicts bpf_masquerade=true), the “agentgateway reaches keycloak-service INTERNALLY” claim (it hairpins via the public issuer URL — see spec root cause), and the “LAN-direct DCR is denied at the public Traefik route” claim (that deny is removed in Task 1).

Step 1: Replace the false-claim portion of the comment block

Replace this exact text (lines 11-22):

# The source-host check is DISABLED (host-sending=false) because under Cilium (no
# SNAT) Keycloak sees the ephemeral agentgateway POD IP, which is unwhitelistable
# (pod-CIDR + CoreDNS reverse both ruled out). Trust is instead gated by
# client-uris-must-match=true: the client's redirect URIs must match a trusted-host,
# and MCP loopback redirects (http://localhost:*/callback, http://127.0.0.1:*) match
# the loopback trusted-hosts. Validated: anonymous DCR with a loopback redirect
# returns a distinct client_id (keycloak direct test, two distinct ids observed).
# LAN-direct DCR is denied at the public Traefik route (deny-client-registration
# middleware, ingress-route.yaml); agentgateway reaches keycloak-service INTERNALLY
# so it is unaffected. Keycloak REJECTS host-sending=false + client-uris-must-match=false
# ("at least one ... must be enabled"); client-uris=true satisfies that while keeping
# the source check off.

with:

# The source-host check is DISABLED (host-sending=false) because the source IP
# Keycloak sees is unreliable: agentgateway's DCR proxy uses the PUBLIC issuer URL
# (agentgateway v1.3.1 mcp/auth.rs client_registration derives the endpoint from
# auth.issuer), so its DCR hairpins out to Traefik and arrives masqueraded
# (Cilium bpf.masquerade=true) — NOT as a stable pod IP. Trust is instead gated by
# client-uris-must-match=true: the client's redirect URIs must match a trusted-host,
# and MCP loopback redirects (http://localhost:*/callback, http://127.0.0.1:*) match
# the loopback trusted-hosts. Validated: anonymous DCR with a loopback redirect
# returns a distinct client_id (keycloak direct test, two distinct ids observed).
# This loopback Trusted-Hosts policy is now the SOLE DCR control: the prior Traefik
# deny (deny-client-registration middleware) was removed in hl-thsf.1 because it
# also 403'd agentgateway's own (hairpinned) DCR. LAN-direct DCR is therefore
# governed by this policy too (loopback redirect URIs accepted; others rejected);
# restoring cluster-internal-only DCR is tracked in hl-thsf.2. Keycloak REJECTS
# host-sending=false + client-uris-must-match=false ("at least one ... must be
# enabled"); client-uris=true satisfies that while keeping the source check off.

Step 2: Lint

Run: yamllint -c .yamllint.yaml argocd/app-configs/keycloak/dcr-trusted-hosts-job.yaml Expected: no errors.

Step 3: Verify the kcadm args are untouched (comment-only change)

Run: rg -n 'trusted-hosts"=|host-sending-registration-request-must-match"=|client-uris-must-match"=' argocd/app-configs/keycloak/dcr-trusted-hosts-job.yaml Expected: unchanged — trusted-hosts"=["localhost","127.0.0.1"], host-sending-registration-request-must-match"=["false"], client-uris-must-match"=["true"].

Step 4: Render the whole app one more time

Run: kubectl kustomize argocd/app-configs/keycloak > /dev/null && echo KUSTOMIZE_OK Expected: KUSTOMIZE_OK.

Step 5: Commit

Commit using VCS-appropriate commands per references/vcs-preamble.md. Message: docs(keycloak): correct false DCR network-path comments post-deny-removal [hl-thsf.1]

Validation (post-merge, operator-run — the Phase 1 gate)

Not part of the implementation commits. After the PR merges and ArgoCD syncs the keycloak app (and the keycloak-dcr-trusted-hosts Job re-runs BeforeHookCreation):

(b) Two POST /register via mcp-gw.fzymgc.house/.well-known/oauth-authorization-server/client-registration → distinct client_ids (real DCR). Expected PASS.
(c) Full MCP client (Claude Code / engram) DCR + PKCE + list_memory green against id.fzymgc.house.
(a) PRM discovery …/.well-known/oauth-protected-resource/mcp/engram resolves to the Keycloak realm; AS-metadata …/oauth-authorization-server returns 200.
(d) Unauth /mcp/engram → 401 + WWW-Authenticate.
REVISED (retires the old “LAN-direct → 403” gate): LAN-direct DCR with a loopback redirect URI now succeeds (201); with a non-loopback redirect URI is rejected by Keycloak.

On pass: bd note hl-thsf "Phase 1 exit-gate: DCR via mcp-gw returns distinct client_ids (Approach G)", then close hl-thsf.1.

Follow-ups (not in this plan)

hl-thsf.2 — A1+trust: restore cluster-internal DCR (CoreDNS rewrite + ICA1 in gateway pod) and re-add the network deny.
hl-g3xy — upstream agentgateway: DCR/metadata backendRef override.
ADR hl-xktj amendment — fold in (a) the #1447 ClusterIP→loopback trust-model shift and (b) this deny removal. Captured via /capture-adrs post-plan-review.