Skip to content

Network Reference

Network configuration and addressing for the fzymgc-house cluster.

Three places define what this page describes. tf/unifi declares the UCG-Fiber gateway and the UniFi switching fabric under it: the subnet and the gateway address of every VLAN leg, the firewall zone of each leg, and the switch ports. The gateway routes every leg and serves no DHCP. The fabric can veto a DHCP server: DHCP Guard drops offers from any server that is not on a network’s trusted list. See UniFi. The resolver pair sandiego and portland owns its leg addresses, the floating resolver addresses and the Kea DHCP pools, all declared in dns_vlan_interfaces in ansible/inventory/host_vars/sandiego.yml and portland.yml. The cluster’s own addressing (Cilium, Traefik) is in argocd/.

Network CIDR Purpose
Trusted-Main Supernet 192.168.16.0/20 One covering CIDR for “Main, wired or wireless” in policies, for example the Keycloak admin allowlist. It covers Main and MainWireless. The unassigned 16-19 and 28-31 blocks are reserved for trusted-Main use. A network numbered inside this /20 gains trusted-policy access, so never number an untrusted network here
Main 192.168.20.0/22 Trusted LAN, VLAN 3000, one L2 segment (.20.0 to .23.255) for the wired clients and the SSID kessel-run. Gateway 192.168.20.1. Resolver VIP .20.5, sandiego .20.6, portland .20.7. The Home Assistant VM ha .20.220. The printer office-printer .21.81, a Kea reservation below the pool. Kea pool 192.168.22.20 to 192.168.23.249, so a client can hold any address in the /22, not only .20.0/24
MainWireless 192.168.24.0/22 Trusted LAN, VLAN 3001, the same trust policy as Main. No SSID puts a client on it. Gateway 192.168.24.1. Resolver VIP .24.5, sandiego .24.6, portland .24.7. Kea pool 192.168.24.200 to 192.168.27.254. Static addresses sit below the pool. Not advertised on the tailnet, because wireless clients need no inbound path
Core 192.168.40.0/22 Untagged native infrastructure LAN. Gateway 192.168.40.1. Holds the two resolvers (sandiego .40.6, portland .40.7), their Core VIP .40.5, heimdall .40.20, calgary .40.21, the NAS containers nas-support .40.32 and vault-unseal-nas .40.34, macvlan children of the NAS interface vlan1, which holds no address, and the Turing Pi BMCs tpi-alpha-bmc .40.40 and tpi-beta-bmc .40.41, untagged on uplinks whose native network is Core. Kea pool 192.168.41.10 to 192.168.43.249. Declared as network-core in the tailnet policy and advertised on the tailnet
IoT 192.168.218.0/24 Trusted IoT LAN, VLAN 3020. Gateway 192.168.218.1. Resolver VIP .218.5, sandiego .218.6, portland .218.7. Kea pool 192.168.218.101 to 192.168.218.223
Lab 192.168.200.0/22 Trusted lab LAN, VLAN 1000. Gateway 192.168.200.1. Resolver VIP .200.5, sandiego .200.6, portland .200.7. Static space .200.140 to .200.159: the k3s node Lab legs .141 to .144 and .151 to .154, and free addresses .140, .145 to .149 and .155 to .159. Kea pool 192.168.200.160 to 192.168.201.19. Lab is not on the tailnet
Guest 192.168.79.0/24 Untrusted segment, VLAN 3100. Gateway 192.168.79.1. Kea on the pair serves its DHCPv4 and advertises the pair’s VIP 192.168.79.5 as the only DNS server, with no search domain. The pair holds .79.6 (sandiego) and .79.7 (portland) with trusted: false, so the public recursor process pdns-recursor@public answers this segment from public DNS only. Kea pool 192.168.79.20 to 192.168.79.249 (subnet id 6), with a 600-second lease and no DNS update. Not inside the trusted-Main supernet
Telework 192.168.217.0/24 Untrusted segment, VLAN 3200. Gateway 192.168.217.1. The same arrangement as Guest: Kea DHCPv4, VIP 192.168.217.5, sandiego .217.6, portland .217.7, trusted: false. Kea pool 192.168.217.20 to 192.168.217.249 (subnet id 7), with a 600-second lease and no DNS update. Not inside the trusted-Main supernet
Node Network 192.168.20.0/24 Physical node addresses, a subset of Main
Pod Network 10.245.0.0/16, fd00:10:245::/48 Kubernetes pods, from Cilium cluster-pool IPAM. Cilium does not use the k3s cluster-cidr IPv4 value 10.42.0.0/16
Service Network 10.43.0.0/16, fd00:10:43::/112 Kubernetes Services
Routed VIPs 192.168.250.0/24 LoadBalancer VIPs from Cilium LB-IPAM. The nodes announce each VIP over BGP to the UCG-Fiber. No leg holds an address in it, so no client is on-link with a VIP. Not inside the trusted-Main supernet. 192.168.250.N mirrors fddb:f665:73f7:250::N
Routed VIPs, IPv6 fddb:f665:73f7:250::/64 The IPv6 match of the routed VIPs. A routed prefix, not a leg, so it takes no ULA leg index

Kea on the pair serves DHCPv4 on all seven legs. No device serves DHCPv6. A client forms its IPv6 addresses by SLAAC from the router advertisements of the gateway and of the pair. See IPv6 ULA Addressing.

Adding a leg to dns_vlan_interfaces is not the same as serving it. Five consumers read that structure: the systemd-networkd legs, the nftables accept sets, the listen addresses of the two recursor processes, keepalived’s vrrp_instance blocks and the RA announcer’s configuration. Each one needs its own converge and its own adoption step. A leg declared but not converged everywhere presents as a total DNS failure on that segment. The DNS runbook has the per-consumer checks.

One SSID per wireless VLAN. The UniFi access points broadcast them, and tf/unifi/wlans.tf declares them. A client joins a VLAN by SSID and by nothing else.

SSID VLAN Security
kessel-run Main (3000) WPA2/WPA3
kessel-run-iot IoT (3020) WPA2 only, PMF off
kessel-run-g Guest (3100) Enhanced Open (OWE) with a captive portal, wireless clients isolated from each other
kessel-run-tw Telework (3200) WPA3 only

The UniFi controller chooses the Wi-Fi channels. On 2.4 GHz it uses only channels 1, 6 and 11. ap-office serves no 2.4 GHz. Three 802.15.4 networks share the band with Wi-Fi. A Zigbee network keeps the channel that it forms on. So the ZHA network forms last, after this table holds the Thread and Hue channels and after a ZHA energy scan from the location of the coordinator. When you read or set a channel, write it in its row.

Network Radio 802.15.4 channel
Thread (MyHome16) The Apple TV, the HomePod, the Aqara Hub M3 and the ZBT-2 25 (2475 MHz)
Hue Zigbee The Hue Bridge Pro 15 (2425 MHz)
ZHA Zigbee The SLZB-06MG24 Not formed. Thread holds 25 and Hue holds 15. Channel 20 (2450 MHz) is the free channel between Wi-Fi channels 6 and 11

What the tailnet advertises, and what it deliberately does not

Section titled “What the tailnet advertises, and what it deliberately does not”

Advertised: 192.168.40.0/22 (Core), and the routed VIP prefixes 192.168.250.0/24 and fddb:f665:73f7:250::/64. Nothing else, and no prefix inside the trusted-Main supernet 192.168.16.0/20. The list is k8s_tailscale_advertise_routes in ansible/roles/k8s-tailscale/defaults/main.yml, and the tailnet policy tailscale/policy.hujson auto-approves the same three routes for tag:nas and tag:subnet-router. Main is not advertised, because every host a remote operator needs has a Core address, a routed VIP or a tailnet node of its own, and a Main route on a client that sits on Main duplicates an on-link prefix.

Core is advertised so that an off-LAN operator can reach the resolvers, heimdall and the Core VIP. Without it a remote session reaches 192.168.20.1 and nothing on 192.168.40.0/22, which reads as “the resolver is down” when the truth is “this vantage cannot see it”.

Four constraints hold, and each has a reason that is not style:

  • The gateway is never a subnet router. Only tag:nas and tag:subnet-router appear in the policy’s autoApprovers.routes. The Firewalla, the gateway before 2026-10-01, failed silently and intermittently as a subnet router. See the tailscale router subnet route breakage report. No advertiser sits on Core, so Core traffic is forwarded from the advertiser through the UCG-Fiber to Core by ordinary L3 routing.
  • Every advertiser announces the same set. truenas-scale (primary) and the tpi-*-4 standbys form a failover group, and Tailscale serves one at a time. A route the primary announces and the standbys do not disappears on failover, silently. To add a route, add it to k8s_tailscale_advertise_routes and nas_tailscale_advertise_routes, and converge both roles. No Main prefix is advertised, so a tailnet client reaches no macvlan child of the NAS on Main, such as precipice at 192.168.22.109. The Core containers nas-support and vault-unseal-nas are reachable through the Core route even while truenas-scale is primary, because the NAS holds no Core address and forwards to them through the gateway. A host never reaches its own macvlan children.
  • Only the routed VIP prefix is advertised in IPv6. The VIPs publish AAAA records, and a tailnet client can try the ULA before IPv4. fddb:f665:73f7:250::/64 holds the VIPs and no leg. It is the only ULA route on the tailnet. The rest of fddb:f665:73f7::/48 stays off, because each ULA route widens what the tailnet reaches and needs its own decision. Lab is not on the tailnet.
  • No source-bound IPv6 proof uses a tailnet vantage. A source-bound proof runs dig -b from a host that holds a real address in the leg’s own /64. Subnet-route SNAT replaces the source address of a tailnet client, so a probe from a tailnet client looks on-link when it is not.

The /80 boundary is weaker than the /27 boundary. The Lab Kea pool starts at .160, so no DHCP client gets an address inside the /27. The Lab RA gives clients SLAAC addresses in fddb:f665:73f7:4::/64, and nothing keeps them out of the /80. A SLAAC address lands inside the /80 only if its interface identifier starts with aaaa, a chance of 1 in 65,536 for each address. Privacy addresses rotate, so over time a Lab client can land inside the /80.

Unique Local Addresses (ULA) give the estate stable internal IPv6 addressing that does not depend on the ISP prefix.

Prefix CIDR Purpose
ULA Base fddb:f665:73f7::/48 Site-local IPv6 prefix

Each VLAN has a /64 from the ULA prefix:

VLAN IPv6 Subnet Gateway ULA address
Main (3000) fddb:f665:73f7:1::/64 fddb:f665:73f7:1::1
Guest (3100) fddb:f665:73f7:2::/64 fddb:f665:73f7:2::1
IoT (3020) fddb:f665:73f7:3::/64 fddb:f665:73f7:3::1
Lab (1000) fddb:f665:73f7:4::/64 fddb:f665:73f7:4::1
Telework (3200) fddb:f665:73f7:5::/64 fddb:f665:73f7:5::1
Core (untagged) fddb:f665:73f7:6::/64 fddb:f665:73f7:6::1
MainWireless (3001) fddb:f665:73f7:7::/64 fddb:f665:73f7:7::1

ULA is routed between all seven legs. The UCG-Fiber holds ::1 in the ULA /64 of each leg, so it has a connected route to each prefix and forwards ULA traffic between them under its zone policies. Each address is the ipv6.aliases value of the network in tf/unifi/networks.tf. The gateway holds it only while the network takes a delegated prefix (ipv6.interface_type = "pd"). With the type none, no leg has a ULA gateway address.

Each leg also takes a global /64 from the /56 that FiOS delegates. The gateway picks the /64 of each leg itself, so the global prefix of a leg does not hold its ULA index. Guest takes a global /64 too, but a Guest client has no IPv6 path to the internet, because the captive portal authorizes a client over IPv4 only.

Guest and Telework are untrusted. The zone firewall of the gateway fences their traffic to the other legs in both families: Guest is in the zone Hotspot and Telework is in the zone Telework. The gateway redirects no DNS query on any leg. See Zone firewall in the UniFi runbook.

This table is the allocating authority for ULA indices. Allocate a new leg here, in this table, and nowhere else. Indices 1 to 7 are in use, in client addresses and in dns_vlan_interfaces on both resolvers, so renumbering one invalidates live addresses. A new leg takes 8. Allocate here first, then add the leg to ula_leg_index in ansible/inventory/group_vars/all.yml, never the reverse. The inventory is a consumer of this table, not a second authority. Every ULA address in Ansible derives from ula_leg_prefix.<leg>, which that file builds from ula_site_prefix and the index. A leg allocated without reading this table is a /64 collision with a live broadcast domain, and nothing in the estate reports it.

fddb:f665:73f7:250::/64 is the routed VIP prefix, not a leg, and it takes no index. No leg holds an address in it. The UCG-Fiber learns a host route for each VIP in it over BGP from the nodes. Its fourth hextet reads 250 to match the IPv4 block 192.168.250.0/24. A new leg still takes index 8. The variables are lb_vip_prefix_ipv4 and lb_vip_prefix_ipv6 in ansible/inventory/group_vars/all.yml.

The resolver pair holds a floating address and one address per node on every leg. The last hextet mirrors the IPv4 last octet, so an address reads the same in both stacks: ::5 VIP, ::6 sandiego, ::7 portland. ::1 is the gateway, held by the UCG-Fiber on every leg (see VLAN Subnets above).

VLAN ULA /64 VIP (::5) sandiego (::6) portland (::7)
Main (3000) fddb:f665:73f7:1::/64 fddb:f665:73f7:1::5 fddb:f665:73f7:1::6 fddb:f665:73f7:1::7
IoT (3020) fddb:f665:73f7:3::/64 fddb:f665:73f7:3::5 fddb:f665:73f7:3::6 fddb:f665:73f7:3::7
Lab (1000) fddb:f665:73f7:4::/64 fddb:f665:73f7:4::5 fddb:f665:73f7:4::6 fddb:f665:73f7:4::7
Core (untagged) fddb:f665:73f7:6::/64 fddb:f665:73f7:6::5 fddb:f665:73f7:6::6 fddb:f665:73f7:6::7
MainWireless (3001) fddb:f665:73f7:7::/64 fddb:f665:73f7:7::5 fddb:f665:73f7:7::6 fddb:f665:73f7:7::7
Guest (3100) fddb:f665:73f7:2::/64 fddb:f665:73f7:2::5 fddb:f665:73f7:2::6 fddb:f665:73f7:2::7
Telework (3200) fddb:f665:73f7:5::/64 fddb:f665:73f7:5::5 fddb:f665:73f7:5::6 fddb:f665:73f7:5::7

Guest and Telework carry trusted: false, a kea_subnet_id and a kea_pool. Kea sends no DNS update and no search domain on either leg. The pair announces RA with RDNSS on both legs. What a dual-stack client on those legs learns, and what it gets when it asks, is in the DNS runbook.

The suffix of every address above is in dns_vlan_interfaces in ansible/inventory/host_vars/sandiego.yml and portland.yml. The /64 of each leg comes from ula_leg_prefix in ansible/inventory/group_vars/all.yml.

The digit repetition is a readability hazard, and the eye is not the control

Section titled “The digit repetition is a readability hazard, and the eye is not the control”

The /64 index and the node suffix sit in different hextets and use the same small integers, so several addresses repeat a digit for two unrelated reasons:

  • fddb:f665:73f7:6::6 is Core (index 6) on sandiego (suffix 6). The two sixes mean different things.
  • fddb:f665:73f7:7::7 is MainWireless (index 7) on portland (suffix 7).
  • fddb:f665:73f7:5::7 looks like a portland address and is portland’s own address on the Telework leg (index 5). It appears once in the estate, on the telework entry in ansible/inventory/host_vars/portland.yml. A reviewer who deletes it as a typo deletes a live address.

Do not check an address by eye. Parse it with ip_network, test containment in the leg’s /64, and compare the exact value against the tables above.

Each k3s node holds one static Lab ULA on end0.1000, under the prefix fddb:f665:73f7:4:aaaa::. The suffix is the last octet of the IPv4 Lab address of the node. netplan sets the address statically, and the leg keeps accept-ra: false, so it takes no SLAAC address. The rule at priority 6000 sends replies from the Lab /64 to table 1000. That table holds the on-link Lab /64 route and a default route via fddb:f665:73f7:4::1 on end0.1000. A reply to a Lab destination leaves on-link. A reply to any other destination leaves through ::1. The prefix is tp2_node_lab_ula_prefix in ansible/roles/tp2-bootstrap-node/defaults/main.yml, derived from ula_leg_prefix.lab in ansible/inventory/group_vars/all.yml.

Suffixes First address Last address Holder
::141-::144 fddb:f665:73f7:4:aaaa::141 fddb:f665:73f7:4:aaaa::144 tpi-alpha-1 to tpi-alpha-4
::151-::154 fddb:f665:73f7:4:aaaa::151 fddb:f665:73f7:4:aaaa::154 tpi-beta-1 to tpi-beta-4

The resolver pair announces the ULA prefixes. The gateway announces them too, because its router advertisement carries every prefix on the bridge of a network, and the ULA ::1 is on that bridge.

dns-common-ra-announce.service on sandiego and portland runs radvd against a configuration rendered from dns_vlan_interfaces, one interface stanza per leg. Each stanza advertises that leg’s ULA /64 with:

Directive Value Why
AdvPreferredLifetime 3600 Finite, so the prefix can be deprecated. Under RFC 4862 section 5.5.3(e) a host retires a prefix only by watching these count down. With infinite there is no withdrawal path
AdvValidLifetime 7200 Finite, and at the two-hour clamp of section 5.5.3(e), so a reduction is honored rather than discarded
AdvDefaultLifetime 0 The resolvers are not routers. A zero router lifetime lets a non-router announce a prefix while the UCG-Fiber stays the sole default gateway
AdvRDNSSLifetime 1800 The RFC 8106 RDNSS option that names the leg’s vip6 as the recursive resolver

The values are dns_common_ra_announce_*_seconds in ansible/roles/dns-common/defaults/main.yml. The template is ansible/roles/dns-common/templates/dns-common-ra-announce.conf.j2. Clients configure themselves through SLAAC.

The announcement of the pair does not depend on the gateway. The gateway holds the ULA ::1 of a leg only while the network takes a delegated prefix. With ipv6_interface_type = "none", the bridge of the network holds a link-local address only.

RDNSS has two sources on every leg, and they must agree. The pair’s RA carries the leg’s vip6. The gateway’s RA carries the servers of dhcp_v6_server.dns_servers on the network in tf/unifi/networks.tf, and each leg states its own vip6 there and nothing else. RFC 8106 section 5.3.1 leaves multi-source ordering to the host, so two sources that disagree are a coin flip rather than a fallback. The gateway runs no DHCPv6 service, so no leg has a DHCP-sourced DNS server. See Network fields in the UniFi runbook.

Node IP Address Lab leg (end0.1000) Role
tpi-alpha-1 192.168.20.141 192.168.200.141 Control plane (etcd)
tpi-alpha-2 192.168.20.142 192.168.200.142 Control plane (etcd)
tpi-alpha-3 192.168.20.143 192.168.200.143 Control plane (etcd)
Node IP Address Lab leg (end0.1000) Board
tpi-alpha-4 192.168.20.144 192.168.200.144 alpha
tpi-beta-1 192.168.20.151 192.168.200.151 beta
tpi-beta-2 192.168.20.152 192.168.200.152 beta
tpi-beta-3 192.168.20.153 192.168.200.153 beta
tpi-beta-4 192.168.20.154 192.168.200.154 beta

Each node’s Lab leg routes Lab-sourced traffic through table 1000 (rule priority 6000), and the Kubernetes InternalIP stays on the Main address. See Lab service VLAN.

The k3s node-ip of each node lists the IPv4 address above first and then the Main ULA fddb:f665:73f7:1:aaaa::<host octet> (host_address and host_address_ipv6 in the node’s host_vars).

Address Purpose Provider
192.168.250.145, fddb:f665:73f7:250::145 Traefik, Service traefik/traefik-vip Cilium LB-IPAM pool routed-vips, announced over BGP
192.168.250.146, fddb:f665:73f7:250::146 Temporal gateway for the node workers on the resolver Pis, Service temporal/temporal-node-gateway-vip on TCP 7233. The workers dial the IPv4 literal. The network policy admits the Pis only, at 192.168.40.6 and 192.168.40.7 on Core Cilium LB-IPAM pool routed-vips, announced over BGP
192.168.250.147, fddb:f665:73f7:250::147 Dolt SQL on TCP 3306, Service dolt/dolt-vip Cilium LB-IPAM pool routed-vips, announced over BGP
192.168.250.148, fddb:f665:73f7:250::148 MQTT over TLS on TCP 8883, Service mosquitto/mosquitto-vip Cilium LB-IPAM pool routed-vips, announced over BGP
192.168.250.149, fddb:f665:73f7:250::149 OTLP on TCP 4317 and 4318, Service clickstack/otel-collector-vip Cilium LB-IPAM pool routed-vips, announced over BGP
192.168.250.150, fddb:f665:73f7:250::150 Syslog on UDP 514 from the UniFi gateway, switches and APs, Service otel-scraper/network-collector-vip. The devices send to the IPv4 address. The network policy admits the Lab address of the gateway, 192.168.200.1, and Core, 192.168.40.0/22, only Cilium LB-IPAM pool routed-vips, announced over BGP
192.168.250.254, fddb:f665:73f7:250::254 Kubernetes API on TCP 6443, Service kube-system/kubernetes-api. The three servers announce it Cilium LB-IPAM pool api-vip, announced over BGP

The gateway holds a blackhole route for 192.168.250.0/24 and for fddb:f665:73f7:250::/64 (tf/unifi/static-routes.tf). It drops the traffic for a VIP that no node announces, so that traffic does not leave through the default route to the WAN.

Terraform in tf/cloudflare is the source of truth for every public record in every zone of the account. See Cloudflare. Three classes of name reach the cluster from the internet, all through the Cloudflare Tunnel:

Name What it reaches
id.fzymgc.house Keycloak, for the Cloudflare Access OIDC back-channel
<service>-wh.fzymgc.net Webhook receivers (webhook_services in tf/cloudflare/variables.tf)
<service>.fzymgc.net Internal services behind Cloudflare Access (internal_services)

The webhook and Access names live under fzymgc.net so that they never collide with the internal fzymgc.house view. The rest of the fzymgc.house zone is mail, GitHub Pages and off-site hosts, declared in tf/cloudflare/dns-fzymgc_house.tf.

The internal resolvers answer these names under fzymgc.house with the routed VIPs. Each name has an A record and an AAAA record. See DNS records for the writer of each record.

Record Type Target
Traefik-routed hosts, for example id, argocd and *.k8s A, AAAA Traefik VIPs (192.168.250.145, fddb:f665:73f7:250::145)
doltdb A, AAAA Dolt VIPs (192.168.250.147, fddb:f665:73f7:250::147)
mqtt A, AAAA Mosquitto VIPs (192.168.250.148, fddb:f665:73f7:250::148)

Names under k8s.fzymgc.house (for example vault.k8s, karakeep.k8s, traefik.k8s) are Traefik-routed hosts like any other. They do not pass through Cloudflare.

Pattern Resolution
<svc>.<ns>.svc.cluster.local Service ClusterIP
<pod-ip-dashed>.<ns>.pod.cluster.local Pod IP
<svc>.<ns>.svc Short form (within cluster)

The resolver pair sandiego and portland answers client DNS on every VLAN. Client DNS follows a fixed chain, not a flat forward:

client :53
-> the advertised resolver address on the client's VLAN (VIP .5, or .6/.7 per node)
-> pdns-recursor@main on the resolver pair <-- the client-facing tier, per-VLAN listeners
|- the policy zones (RPZ): block lists, allow list, the entries of dns-control
|- forward zone: fzymgc.house -> PowerDNS on the loopback listener
'- everything else -> dnscrypt-proxy on 127.0.0.1:5300 -> a relay -> a public resolver
-> PowerDNS (authoritative for fzymgc.house and the reverse zones)
client :53 on Guest or Telework
-> the leg VIP or node address
-> pdns-recursor@public on the resolver pair <-- no path to PowerDNS
|- the withheld zone: fzymgc.house and the reverse zones -> NXDOMAIN
'- every other name -> dnscrypt-proxy on 127.0.0.1:5300 -> a relay -> a public resolver

The client-facing tier is PowerDNS Recursor, in two processes on each node. Every block rule and allow rule of it is a response policy zone (RPZ), a DNS zone whose records are rules. What the chain returns differs by trust. On the five trusted legs it resolves fzymgc.house internally. Guest and Telework (trusted: false) use the second chain, so a client on those legs gets NXDOMAIN for a .house name. See the ADR and the client view in the DNS runbook.

The addresses are in Resolver addresses on each ULA leg above for IPv6. The IPv4 last octet mirrors it (.5 VIP, .6 sandiego, .7 portland). keepalived holds the floating addresses, so the VIP answers from whichever node owns it.

PowerDNS holds the curated zone. dns-control and external-dns write records through its API, and lightningstream reconciles the two nodes. The loopback listener is dns_authoritative_listen (127.0.0.1:1053) in ansible/inventory/group_vars/dns_hosts.yml. See the PowerDNS authoritative tier.

The UCG-Fiber keeps a small DNS role beside the pair. Its dnsmasq answers on the .1 address of every leg and forwards each query to the Core VIP 192.168.40.5, which is the IPv4 DNS server of both WANs. So a client on a trusted leg that carries a hand-set .1 nameserver gets the internal view. The zone firewall blocks DNS to the gateway from Guest and Telework. The gateway serves no DHCP, produces no curated zone and redirects no DNS query. See Zone firewall in the UniFi runbook. The design of the pair is in docs/engineering/specs/2026-07-07-authoritative-house-resolver-design.md.

Internet
|
v
Cloudflare Edge (WAF, DDoS protection)
|
v
Cloudflare Tunnel (encrypted)
|
v
cloudflared pod (namespace: cloudflared)
|
v
Traefik (namespace: traefik), or Keycloak directly for id.fzymgc.house
|
v
Backend Services
Internal Client
|
v
DNS: <name>.fzymgc.house -> Traefik VIP (192.168.250.145 or fddb:f665:73f7:250::145)
|
v
UCG-Fiber: BGP route to the Lab address of a node with a ready Traefik endpoint
|
v
Traefik (namespace: traefik)
|
v
Backend Services

Traefik is deployed by argocd/cluster-app/templates/traefik.yaml. Its Service is the dual-stack traefik-vip LoadBalancer on the routed VIP prefixes with externalTrafficPolicy: Local, so every Middleware below sees the real client address.

Name Port Protocol Purpose
web 80 HTTP Redirects to websecure
websecure 443 HTTPS TLS termination
traefik 8080 HTTP Dashboard, reached only through the traefik-internal Service and its IngressRoute
Name Namespace Type Purpose
oauth-auth-<app> (one per dashboard: temporal, hubble, traefik-dashboard, longhorn) oauth2-proxy ForwardAuth Per-route Keycloak-backed SSO, gated by ?allowed_groups=<app>-users
oauth-errors oauth2-proxy Errors Converts oauth2-proxy’s 401 and 403 into a redirect to /oauth2/sign_in
admin-allowlist keycloak IPAllowList Restricts id.fzymgc.house /admin and /realms/master to traffic that reaches Traefik directly from the trusted set: 192.168.16.0/20 (the trusted-Main supernet), 100.64.0.0/10 (Tailscale CGNAT), and fddb:f665:73f7:1::/64 and fddb:f665:73f7:7::/64 (the Main and MainWireless ULA /64s). The Middleware uses the default RemoteAddrStrategy. The Cloudflare tunnel path never reaches this Middleware. A Cloudflare Access Application gates that path (tf/cloudflare/access.tf). See argocd/app-configs/keycloak/ingress-route.yaml
offnet-marker keycloak Headers Sets X-Fzymgc-Offnet: 1 on every id.fzymgc.house request that reaches Traefik from outside the trusted set. If the header is present, Keycloak’s passkey path forces OTP (tf/keycloak/authentication.tf). A priority-2 route in the keycloak IngressRoute bypasses this Middleware for ClientIP matches on the same CIDRs as admin-allowlist. The two lists change together. On the Cloudflare tunnel path Keycloak reads the Cf-Connecting-IP and Cf-Ray headers instead
llm-hsts litellm Headers HSTS on llm.fzymgc.house
miniflux-greader-stripprefix miniflux StripPrefix Strips /api/greader.php for the Google Reader API of Miniflux
redirect-to-cloudflare uptime-kuma RedirectRegex Sends status.fzymgc.house to status.fzymgc.net

To list what is in force:

Terminal window
kubectl get middlewares -A
Item Value
Default certificate The secret wildcard-fzymgc-house-tls in the traefik namespace, served for any host without its own certificate
cloudflare-acme-issuer ClusterIssuer, Let’s Encrypt through the Cloudflare DNS-01 solver (argocd/app-configs/cert-manager-config/cloudflare-acme-issuer.yaml), for public names
vault-issuer ClusterIssuer, the internal Vault CA (vault-certmanager-issuer.yaml), for in-cluster hops

Cilium LB-IPAM assigns each LoadBalancer VIP from the pool routed-vips, and the Cilium BGP control plane announces the VIP to the UCG-Fiber. The objects are in argocd/app-configs/cilium-lb/:

Object File Role
CiliumLoadBalancerIPPool routed-vips ip-pool.yaml The ranges of the routed VIP prefixes, below the reserved .254 and ::254. It serves only the namespaces in its serviceSelector
CiliumBGPClusterConfig ucg-fiber bgp.yaml One BGP instance on every node, with one peer per family at the Lab gateway
CiliumBGPPeerConfig ucg-fiber-v4 and ucg-fiber-v6 bgp.yaml The timers, graceful restart, the family and the advertisement label of each peer
CiliumBGPAdvertisement lb-vips bgp.yaml Announces the LoadBalancer addresses of every Service

A Service gets a VIP only when it sets loadBalancerClass: io.cilium/bgp-control-plane, because the cilium role sets defaultLBServiceIPAM to none. The gateway side is the FRR file in tf/unifi/bgp.tf. See BGP in the UniFi runbook and The LoadBalancers in the Lab runbook.

The cluster uses Cilium for policy enforcement (policyEnforcementMode: default, cilium_policy_enforcement_mode in ansible/roles/cilium/defaults/main.yml):

  • Standard Kubernetes NetworkPolicy (networking.k8s.io/v1). Cilium honors these as L3 and L4 rules.
  • CiliumNetworkPolicy (cilium.io/v2). Used when a rule needs identity awareness (FQDN, ServiceAccount) or L7 awareness.
  • CiliumClusterwideNetworkPolicy (cilium.io/v2). Cluster-scoped. The repository declares none.

Cilium evaluates policies per pod identity. A pod that no policy selects defaults to allow-all, which is the standard Kubernetes NetworkPolicy semantics.

The repository declares CiliumNetworkPolicies in nine namespaces. dolt alone carries five and litellm two, so the namespace count is not the policy count.

Namespace CNPs Coverage Manifest
clickhouse-mcp 1 ingress 8000 from the fronting gateways; egress ClickHouse 8123, kube-dns argocd/app-configs/clickhouse-mcp/networkpolicy.yaml
clickstack 1 OTLP ingest 4317 and 4318, ClickHouse 8123, collector 8888, kube-dns, world 443 argocd/app-configs/clickstack/networkpolicy.yaml
dolt 5 SQL 3306, Remote API 50051, Prometheus 9418, L4-only DNS egress, kube-apiserver egress argocd/app-configs/dolt/network-policies.yaml
firewalla-mcp 1 ingress 8080; egress kube-dns, world 443 argocd/app-configs/firewalla-mcp/networkpolicy.yaml
headroom-agents 1 ingress 8787; egress kube-dns, OTLP 4318, world 443 argocd/app-configs/headroom-agents/networkpolicy.yaml
hl-assets 1 ingress 8080; egress kube-dns, OTLP 4318, world 443 argocd/app-configs/hl-assets/networkpolicy.yaml
kubernetes-mcp 1 ingress 8080 from litellm only; egress kube-dns, kube-apiserver, keycloak 8443 for JWKS and issuer discovery. Without that egress the provider never builds and every call answers 401 temporarily_unavailable argocd/app-configs/kubernetes-mcp/networkpolicy.yaml
litellm 2 proxy ingress 4000 from Traefik, the HCP Terraform agents, life-agent, octopus, karakeep and mealie; valkey ingress from the proxy argocd/app-configs/litellm/networkpolicy.yaml

To compare the declared set with what is in force:

Terminal window
kubectl get ciliumnetworkpolicies -A

No policy carries an L7 rules.dns block, and every namespace that needs egress to named hosts uses the coarse toEntities: world rule rather than toFQDNs. That is a decision, not an oversight: toFQDNs fails on most of the nodes in either DNS-interception mode. See DNS L7 visibility in the Cilium runbook.

Crown-jewel namespace coverage is tracked under the post-migration hardening initiative (docs/engineering/specs/2026-05-12-cilium-post-migration-hardening-design.md).

The cluster runs Cilium kube-proxy replacement. kube-proxy runs on no node. The values are in ansible/roles/cilium/defaults/main.yml and ansible/roles/k3s-common/defaults/main.yml.

Setting Value Source
kube-proxy-replacement true (Socket LB) cilium_kube_proxy_replacement
enable-host-legacy-routing false (eBPF native routing for the host network namespace) cilium_bpf_host_legacy_routing
enable-bpf-masquerade true (preserves source NAT for pod-to-external) cilium_bpf_masquerade
routing-mode tunnel, VXLAN on port 8473 cilium_routing_mode, cilium_tunnel_protocol, cilium_tunnel_port
KUBERNETES_SERVICE_HOST / _PORT 127.0.0.1 / 6444, kube-apiserver on a server and the k3s client load balancer on an agent, so no node dials the API VIP cilium_k8s_service_host, cilium_k8s_service_port
disable-kube-proxy: true on all 3 control-plane nodes k3s_disable_kube_proxy, rendered into /etc/rancher/k3s/config.yaml
devices end0, end0.1000 and end0.3000 cilium_devices
  1. A pod opens a connection to a Service ClusterIP.
  2. Cilium socket LB intercepts at the socket layer, before iptables, and rewrites the destination to a backend pod IP through the eBPF Service map.
  3. The packet flows directly to the backend pod, across nodes through the VXLAN tunnel when needed.

There are no KUBE-SVC-* iptables chains. The only KUBE-* chain on each node is KUBE-KUBELET-CANARY, which kubelet manages for its own iptables health check.

  • Service routing issues: see Cilium KPR.
  • Service map state: cilium-dbg service list from any agent.
  • Real-time policy verdicts: cilium-dbg monitor filtered by endpoint ID.
Setting Value
Tunnel name fzymgc-house-main (tunnel_name in tf/cloudflare/variables.tf)
Namespace cloudflared (argocd/app-configs/cloudflared-main/)
Replicas 2

tf/cloudflare/tunnel.tf declares the ingress rules and the DNS records that point at the tunnel:

  • id.fzymgc.house to https://keycloak-service.keycloak.svc.cluster.local:8443.
  • <service>-wh.fzymgc.net to each webhook receiver in webhook_services.
  • <service>.fzymgc.net to each service in internal_services, behind Cloudflare Access.
  • <service>.fzymgc.net to each SSH bastion in ssh_services, through the Access browser terminal.
  • Anything else answers http_status:404.
Port Protocol Service
443 HTTPS All web services (through Cloudflare)
6443 HTTPS Kubernetes API (internal only)
Service Port Protocol
Traefik 80/443 HTTP/HTTPS
Vault 8200 HTTPS
Keycloak 8443 HTTPS
Grafana 3000 HTTP
ArgoCD 80/443 HTTP/HTTPS
VictoriaMetrics 8428 HTTP
Loki 3100 HTTP
Dolt 3306 MySQL

Run the ssh command with SSH to the console of the UCG-Fiber:

Terminal window
kubectl get ciliumloadbalancerippools routed-vips
kubectl get svc -A | rg LoadBalancer
ssh root@192.168.40.1 "vtysh -c 'show ip route bgp' -c 'show ipv6 route bgp'"
Terminal window
kubectl get ingressroutes -A
kubectl get ingressroutetcps -A
kubectl get middlewares -A
Terminal window
# From within cluster
kubectl run -it --rm debug --image=busybox -- nslookup vault.vault.svc
# From a LAN client
dig vault.fzymgc.house
Terminal window
# Standard Kubernetes NetworkPolicies
kubectl get networkpolicies -A
kubectl describe networkpolicy <name> -n <namespace>
# CiliumNetworkPolicies (cilium.io/v2)
kubectl get ciliumnetworkpolicies -A
kubectl describe ciliumnetworkpolicy <name> -n <namespace>
# Check policy verdicts via Hubble Relay (cilium-cli)
hubble observe --verdict DROPPED
hubble observe --pod <namespace>/<pod>
# Check enforcement state on a specific node's cilium agent
kubectl exec -n kube-system <cilium-pod> -c cilium-agent -- \
cilium-dbg endpoint list