Skip to content

Hubble UI Operations

The Cilium Hubble UI is exposed at https://hubble.fzymgc.house behind the shared oauth2-proxy. This page states what exists, who can reach it, and how to debug the path.

Component Location
Hubble UI Service hubble-ui.kube-system.svc:80, deployed by the Cilium chart (cilium_hubble_enabled in ansible/roles/cilium/defaults/main.yml)
TLS certificate the hubble-tls Certificate in the hubble namespace, issued by vault-issuer (argocd/app-configs/hubble-ui/certificate.yaml)
IngressRoute hubble-ui in the hubble namespace, host hubble.fzymgc.house (argocd/app-configs/hubble-ui/ingress-route.yaml)
Forward-auth middlewares oauth-errors and oauth-auth-hubble in the oauth2-proxy namespace (argocd/app-configs/oauth2-proxy/middlewares.yaml)
oauth2-proxy the shared Deployment in the oauth2-proxy namespace, backed by the Keycloak oauth2-proxy confidential client (tf/keycloak/oauth2_proxy.tf)
ArgoCD Application hubble-ui (argocd/cluster-app/templates/hubble-ui.yaml)

The middleware order in the IngressRoute matters. oauth-errors must precede oauth-auth-hubble, so that oauth2-proxy’s 401 and 403 become the sign-in flow.

Access is gated to the Keycloak group hubble-users. The group is flat, with no admin tier (ADR). It is keycloak_group.dashboard_users["hubble"] in tf/keycloak/oauth2_proxy.tf.

To add a user, resolve them with a data "keycloak_user" block by username, as tf/keycloak/users.tf does for the operator, then bind them with a keycloak_user_groups resource that references keycloak_group.dashboard_users["hubble"].id, in the shape of keycloak_user_groups.operator_dashboards. A change made in the Keycloak UI does not last. Terraform reverts it on its next apply.

To revoke a user, remove their keycloak_user_groups binding and open a pull request. HCP Terraform applies on merge. The change takes effect on the user’s next oauth2-proxy session refresh, not on the next request, because the session is cookie-backed.

The UI filters live flows by namespace, source or destination pod, verdict, HTTP method and L4 port.

  • Dropped flows in any namespace: set the Verdict filter to DROPPED. It catches policy mismatches and connectivity problems.
  • Flows to or from one namespace: set the Namespace filter. Use it to check a new CiliumNetworkPolicy.
  • The Service Map view shows pod-to-pod traffic. Use it to read the real cross-namespace dependencies before you write a policy.

If https://hubble.fzymgc.house returns 502 or stays at the Keycloak login:

  1. Make sure that the IngressRoute and the middlewares exist:

    Terminal window
    kubectl get ingressroute -n hubble
    kubectl get middleware -n oauth2-proxy
  2. Make sure that the certificate is Ready:

    Terminal window
    kubectl get certificate -n hubble hubble-tls
  3. Make sure that the upstream Service and pod exist:

    Terminal window
    kubectl get svc -n kube-system hubble-ui
    kubectl get pod -n kube-system -l k8s-app=hubble-ui
  4. Test forward-auth against the shared oauth2-proxy:

    Terminal window
    curl -sI "http://oauth2-proxy.oauth2-proxy.svc/oauth2/auth?allowed_groups=hubble-users"

    Expect 401 or 403 for an unauthenticated request. In the real request path, the oauth-errors middleware turns that into a redirect to /oauth2/sign_in.