Hubble UI Operations

Operator guide for the Cilium Hubble UI dashboard exposed at https://hubble.fzymgc.house.

Access

• Auth: shared oauth2-proxy forward-auth via Traefik, gated to Keycloak group hubble-users (ADR hl-1fia)
• Required group: hubble-users (flat group, no admin tier — see ADR hl-1fia)
• Add a user: managed in tf/keycloak/oauth2_proxy.tf (keycloak_group.dashboard_users["hubble"]). Keycloak users are not created in Terraform (see tf/keycloak/users.tf) — resolve the target user with a data "keycloak_user" block by username, then bind them with a keycloak_user_groups resource referencing keycloak_group.dashboard_users["hubble"].id (mirrors the existing operator_dashboards binding). Keycloak UI changes are NOT durable — Terraform reconciliation will revert them.

Architecture

Component	Location
Hubble UI service	hubble-ui.kube-system.svc:80 (deployed by Cilium chart)
TLS certificate	hubble-tls Secret in hubble namespace (issued by vault-issuer)
Forward-auth middlewares	oauth-errors + oauth-auth-hubble in the oauth2-proxy namespace (argocd/app-configs/oauth2-proxy/middlewares.yaml)
IngressRoute	hubble-ui IngressRoute in hubble namespace, host hubble.fzymgc.house (argocd/app-configs/hubble-ui/ingress-route.yaml)
oauth2-proxy	Shared Deployment in the oauth2-proxy namespace, backed by the Keycloak oauth2-proxy confidential client (tf/keycloak/oauth2_proxy.tf)
ArgoCD Application	hubble-ui (under argocd/cluster-app/templates/hubble-ui.yaml)

Common queries

The UI supports filtering live flow data by namespace, source/destination pod, verdict (forwarded vs dropped), HTTP method, and L4 port. Useful operator queries:

• Dropped flows in any namespace — set Verdict filter to DROPPED. Catches policy mismatches and connectivity issues.
• Flows to/from a specific namespace — set Namespace filter. Use when validating a new CiliumNetworkPolicy rollout (PR3+).
• Service Map view — visualizes pod-to-pod traffic patterns. Useful for understanding actual cross-namespace dependencies before writing policies.

Debugging

If https://hubble.fzymgc.house returns 502 or stays at the Keycloak login:

1. Check the IngressRoute and Middlewares exist:

   kubectl get ingressroute -n hubble
   kubectl get middleware -n oauth2-proxy

2. Confirm cert is Ready:

   kubectl get certificate -n hubble hubble-tls

3. Confirm the upstream Hubble UI service:

   kubectl get svc -n kube-system hubble-ui
   kubectl get pod -n kube-system -l k8s-app=hubble-ui

4. Test forward-auth directly against the shared oauth2-proxy:

   curl -sI "http://oauth2-proxy.oauth2-proxy.svc/oauth2/auth?allowed_groups=hubble-users"

   Expected: 401/403 (unauthenticated) — the oauth-errors middleware turns that into a redirect to /oauth2/sign_in in the real request path.

Removing access

To revoke a user, remove their keycloak_user_groups binding to keycloak_group.dashboard_users["hubble"] in tf/keycloak/oauth2_proxy.tf (or the per-user file that binds them). Open the PR; HCP Terraform applies on merge. Membership changes propagate on the user’s next oauth2-proxy session refresh (cookie-backed; not instantaneous like a live forward-auth check).