Firewalla Network Configuration

Who owns the Firewalla’s network configuration, where it actually lives, how to change it without losing the change, and which of its behaviours are wrong by default.

This is the config surface. For the collector, host telemetry and prefix-delegation triage, see Firewalla OTel Collector; for boot and /extdata, Router Boot Ordering; for the DNS serving plane, DNS.

The three writers, and which one wins

Three things write the Firewalla’s network state, and they do not agree about what exists.

Writer	Owns	Where it keeps it
FireRouter	interface addressing, routing tables, dnsmasq generation	redis db 1, sysdb:networkConfig
FireMain	policy objects derived from the above — ipsets, iptables chains, flow accounting	redis db 0
The Firewalla app (phone/cloud, closed source)	the operator’s model of the network	its own cloud state, pushed as a whole config

The app wins, and it wins by forgetting. Every app save rebuilds the entire network config from the app’s own model and POSTs it whole. Any key the app does not model is absent from that payload — not preserved, not merged, simply gone. FireRouter then applies exactly what it was handed, correctly, and the field disappears.

This is measurable rather than theoretical. A static ipv6 added to the Lab network survived a FireRouter reapply and was dropped by the next app save of that network, one rename later (.planning/spikes/008-lab-ula-static-apply/, 2026-09-19):

Time	Writer	Lab object carries ipv6?
21:36:47	us, via /v1/config/set	yes
21:41:56	app, Lab → Lab2	no
21:42:10	app, Lab2 → Lab	no

So: anything held only in FireRouter’s model is transient across app saves. A field we want durably must be re-asserted by something that watches for the overwrite. It is the same shape as the UniFi controller’s port_overrides — a closed writer behind a full-replace API deletes every field it does not model, and no diff, plan or error shows it.

Where the configuration actually is

There is no config.json. /home/pi/.router/config/config.json does not exist on this appliance; a plausible-looking path is not the store. ncm.saveConfig() writes to redis:

# FireRouter's client is db 1 — db 0 is Firewalla's. Without -n 1 the key reads as absent.
ssh pi@192.168.20.1 "redis-cli -n 1 --raw GET sysdb:networkConfig | jq '.interface.vlan[\"bond0.1000\"]'"

# The same object, served: identical to the persisted one.
ssh pi@192.168.20.1 "curl -s localhost:8837/v1/config/active | jq '.interface.vlan[\"bond0.1000\"]'"

/home/pi/.router/config/ does exist and holds generated per-service config — dhcp/conf/, dhcpcd6/, dnsmasq/. Those are outputs, not the source.

Every past config is on the box. Redis db 0 holds history:networkConfig, a zset of whole configs scored by epoch — 31 entries spanning a month, as of 2026-09-19. This is the audit log: it records exactly what each writer posted, so “did the app drop my key” is a direct read rather than an inference.

ssh pi@192.168.20.1 '
redis-cli --raw ZRANGE history:networkConfig -4 -1 WITHSCORES | paste - - |
while IFS=$(printf "\t") read -r cfg score; do
  printf "%s  " "$(date -d @"${score%.*}" "+%F %T")"
  printf "%s\n" "$cfg" | jq -c ".interface.vlan[\"bond0.1000\"] | keys"
done'

The config is secret-bearing — WiFi passwords, the mesh key, AP and WireGuard keys; 57 such values in the 2026-09-19 reading. A fetched copy is a credential. Encrypt it at rest (age to the SOPS recipient) or delete it; never leave it in a scratch directory and never commit it.

Changing it safely

The only write endpoint is POST /v1/config/set on localhost:8837, and it takes the whole config. There is no PATCH, no per-interface write, no merge — service/routes/interface.js is a lone GET /:intf. The app posts whole configs too.

Two properties keep a whole-config write narrow:

• pl.reapply diffs per plugin instance and skips unchanged ones, so one added key reapplies one interface and its dependents. It ignores meta.name, which is why renaming a network does not reapply it — but a rename still posts a whole config, which is how the app’s overwrite lands.
• ncid is an optimistic lock. The body’s ncid must equal the persisted one or /set returns ncid not match. A save that landed between your read and your write makes the write fail rather than clobber. Do not send ignoreNcid to get past it — re-read and re-apply.

Use a transaction. transactionOp: "append" with a transID applies the config and auto-reverts after 120 seconds unless a transactionOp: "commit" follows; "revert" reverts on demand. A failed apply rolls back on its own. This is the app’s own safety net and it makes an experimental change self-limiting — if the session dies mid-check, the box restores itself.

The worked procedure, with checks and both exits, is .planning/spikes/008-lab-ula-static-apply/run.sh (prepare / apply / app-check / rollback). Its shape:

1. GET /v1/config/active → modify the one field → diff it, and look at the diff.
2. POST /set with transactionOp: "append" and a fresh transID. Non-200 means nothing was applied.
3. Run the checks inside the 120 s window.
4. POST /set with transactionOp: "commit" on success, "revert" on failure.

Never hand-edit redis, and never restart firerouter to pick up a change. The API is the only supported path; it validates, applies, persists and publishes in one place, and the transaction is only available through it.

What happens when an interface is reapplied

Worth knowing before changing anything, because it explains what survives and what does not.

reapplyIpv6Settings() is flushIP(6) → applyIpv6Settings() → changeRoutingTables(), and flushIP(6) does ip -6 addr flush and then toggles disable_ipv6 1→0. That toggle takes the device down and up internally, and the kernel deletes every single-nexthop route on the device with it.

The consequence: a route cannot be added out-of-band and expected to last. ip -6 route add … dev bond0.X is gone at the next reapply, with no address to recreate it. Addressing is the only durable lever, and .routing in the config model carries WAN default policy only — there is no static-route field.

Reapply triggers include an app save, a WAN prefix re-delegation (EVENT_PD_CHANGE), and an explicit POST /v1/config/apply_current_config.

IPv6 specifics

A static address on a LAN is a real field, unexposed

isIPv6Enabled() is ipv6 || ipv6DelegateFrom || ipv6PassthroughFrom, so "ipv6": ["fddb:…::1/64"] on a LAN interface enables and applies on its own, alongside a delegated GUA. The app does not expose it for LANs (firewalla/firewalla#6857) — which is exactly why it strips it.

constructor: mode means an added address changes the RA

Legs with a generated *_v6.conf advertise with:

dhcp-range=tag:bond0.3000,::,constructor:bond0.3000,slaac,86400
enable-ra
ra-param=bond0.3000,200,3600

dnsmasq’s constructor: builds the advertised prefix from every global-scope address on the interface — and a ULA is global scope. So adding a ULA to one of those legs makes the Firewalla advertise that prefix at 86400 s. Where the resolver pair already announces it (D-12) that is a second RA source for the same prefix, and the last RA to arrive sets the lifetime.

Which legs have a _v6.conf is not uniform. As of 2026-09-19: Core, Main, MainWireless and IoT do. Lab has no dhcp6 section at all, so the Firewalla sends no RA there and Lab clients have no GUA and no v6 default router from it — only the pair’s RA. Check before assuming:

ssh pi@192.168.20.1 "ls /home/pi/.router/config/dhcp/conf/"

ULA forwarding is allowed, with one subtlety

FW_FORWARD carries -s fc00::/7 … -j FW_ULA_LOCAL_ONLY, and that chain RETURNs only for WAN in/out, otherwise FW_ACCEPT_DEFAULT. Inter-LAN ULA is accepted; no rule needs adding.

The subtlety: for a source /64 the appliance does not hold, that ACCEPT short-circuits the rest of FW_FORWARD, so per-network policy is bypassed until both legs’ /64s are in their c_net_<uuid13>_set6 ipsets. FireMain populates those from each network’s addresses and does so on its own within about a minute of an address being added — so the window closes by itself, but it exists.

Static ULA addresses are held by a reconciler

The router-ula-reconciler role keeps a static ipv6 on each leg listed in router_ula_reconciler_addresses. One redis-cli subscribed to firerouter.iface_change_applied runs under ula-reconcile.service. On each event it reads the leg. If the address is missing, it reads /v1/config/active, adds the field, and posts the config back with that config’s ncid. If the address is present, it does nothing. The list is the five trusted legs: Main, IoT, Lab, Core and MainWireless. Guest and Telework are not in it. On the legs that run dnsmasq in constructor: mode the Firewalla also advertises the ULA prefix at 86400 s beside the resolver pair’s 7200 s, which is accepted: the pair can no longer deprecate a ULA prefix on those legs.

ssh pi@192.168.20.1 "systemctl is-active ula-reconcile; sudo journalctl -u ula-reconcile -n 20 --no-pager"

The loop sends a Uptime Kuma push after each Redis event and every 240 seconds while idle. The monitor waits 300 seconds between pushes and sends a Pushover alert when they stop. The router uses the Traefik VIP for the /api/push/ path on status.fzymgc.house. That path reaches Kuma directly. Other paths still redirect to Cloudflare Access. The router sends the push token in the request path. The existing Firewalla Vault Agent reads it from Vault and writes a mode 0600 file. The listener reads that file for each push.

The diagnostic emitter publishes firewalla.ula_reconcile.up each minute. The value is 1 when the unit holds both the script and its Redis subscriber. The value is 0 if either process is absent. The ClickStack alert fires on 0 or a missing gauge. If the gauge is missing, compare it with firewalla.diag.heartbeat before you diagnose the unit. The Kuma status page is https://status.fzymgc.net/status/router-ula.

If you deploy this change, wait for the Vault policy and Kuma Terraform applies first. Then wait for ArgoCD to sync the direct push route. Run the router-ula-reconciler and otel-collector Ansible tags after those steps. Stop and start ula-reconcile.service to make sure that both alerts change state and recover.

To reproduce an app save without the app, post the active config with the field removed. The address must return within a few seconds.

Events worth subscribing to

FireRouter publishes on redis pub/sub after applying config:

Channel	Fires
firerouter.iface_change_applied	after each interface reapply
firerouter.change_applied	after each config apply
firerouter.wan_conn_changed, firerouter.wan_state_changed, firerouter.apc_change_applied	WAN and AP-controller events

This is pub/sub, not keyspace notification — notify-keyspace-events is empty on this box and irrelevant to it. There is no scripts.d hook directory in FireRouter; post_main.d is Firewalla-side and runs at boot and service restart, which is not every reapply. Anything that must react to a config change subscribes here.

ssh pi@192.168.20.1 "redis-cli PUBSUB CHANNELS 'firerouter*'"

Rules that are wrong by default

1. redis-cli without -n 1 reads the wrong database and reports FireRouter’s config key as absent. The absence is the tool, not the box.
2. The app’s save is a full replace. Treat any field it does not model as transient. Before relying on one, test the round-trip: add it, make an unrelated edit in the app, save, and read the newest history:networkConfig entry.
3. A route without an address does not survive. See What happens when an interface is reapplied.
4. NRestarts stays 0 across a supervised restart — it counts systemd’s restart-on-failure only. Use MainPID and ExecMainStartTimestamp to tell whether something actually restarted.
5. An app field change is also an adoption event. Setting a per-network DHCPv6 DNS server made FireMain rewrite bond0.NNNN_v6.conf and restart firerouter_dhcp 22 s later, with no converge from us — and in doing so it adopted a file already staged in that conf-dir.
6. A process is expensive on this box. During the livelock the router is triaged for, process creation is the scarce resource. Prefer one long-lived process to a cron or a poll loop that spawns; prefer no process at all.

Reading list

• .planning/spikes/007-firerouter-static-ipv6-baseline/ — the read-only probe behind most of this page; run.sh reproduces every reading, ra_probe.py is an rdisc6 stand-in for hosts without ndisc6.
• .planning/spikes/008-lab-ula-static-apply/ — the gated write, and the app round-trip result.
• Firewalla OTel Collector — telemetry, memory model, recovery.
• DNS — the serving plane, dead-prefix deprecation, the client-side runbook.
• Network Reference — the ULA allocation table, which is the allocating authority for /64 indices.