Skip to content
fzymgc-house Cluster Docs

Preserve real client IPs on Traefik Service (externalTrafficPolicy: Local)

Date: 2026-07-02 Status: Accepted Decision: hl-1e4w Deciders: Sean (with Claude)

Context

Section titled “Context”

Traefik’s LoadBalancer Service had no externalTrafficPolicy set (defaulted to Cluster). Under Cilium’s kube-proxy replacement this SNATs cross-node external traffic before it reaches any Middleware, so IP-based access control (ipAllowList) evaluates a SNAT’d address instead of the real client IP. This surfaced as Keycloak’s admin-allowlist Middleware wrongly denying legitimate LAN/Tailscale clients (hl-ba3a). The fix is a Service-wide setting, affecting every app behind Traefik, not just Keycloak.

Decision

Section titled “Decision”

Set externalTrafficPolicy: Local on Traefik’s Service so IP-based Middlewares see real client addresses, accepting that this is a cluster-wide traffic-policy change rather than a Keycloak-scoped one.

Rationale

Section titled “Rationale”
  • Matches the exact remedy already documented in this repo for an identical SNAT pattern on the dolt LoadBalancer Service (argocd/app-configs/dolt/network-policies.yaml)
  • MetalLB L2 mode degrades gracefully (per-node VIP announcement gated on a Ready backend pod) rather than black-holing traffic

Alternatives Considered

Section titled “Alternatives Considered”
  • externalTrafficPolicy: Local (chosen): stops SNAT at the source; MetalLB L2 mode natively supports it
  • Recover client IP via X-Forwarded-For / PROXY protocol (rejected): Cilium SNAT operates at L3/L4 and adds no L7 header; PROXY protocol between MetalLB/Cilium and Traefik is unconfigured — materially larger change for the same outcome
  • Drop IP-based gating for /admin entirely, rely solely on Cloudflare Access (rejected): Cloudflare Access never sees LAN-direct traffic to the internal VIP — would leave that path completely ungated, a regression from defense-in-depth

Consequences

Section titled “Consequences”
  • Positive: IP-allowlist Middlewares cluster-wide now see real client CIDRs, not SNAT’d addresses
  • Negative: traffic handling changes for every Traefik-fronted app, not just Keycloak; nodes without a Ready Traefik pod stop receiving traffic for the VIP
  • Neutral: follow-up worth doing — audit other Traefik-fronted services that may implicitly depend on Cluster-policy SNAT behavior