Curated DNS records and Kea reservations as control-plane data
The curated host entries of fzymgc.house are data that dns-control holds, not configuration that
Terraform applies. One Temporal workflow projects each change onto pdns, onto the Kea reservation
files of both resolver nodes and onto two S3 sinks. Sean decided the model on 2026-09-29 after
issue #2527, which explored why a live write followed by a capture pull request gave one fact two
appliers. This design records the decisions and their alternatives.
Context
Section titled “Context”See proposal.md for the motivation. The pieces this design builds on already exist:
apps/dns-controlruns in k3s with a database on the shared CNPGmaincluster, anops_audittable with an append-only trigger, a lineage index that reads pdns RRset comments as writer marks, a leg file at/config/legs.json, and anOpsWorkflowthat orders the two nodes from that leg file (Phase 4 D-17).cmd/dns-node-workerruns on each Pi as userdns-workerwithStateDirectory/var/lib/dns-worker, registers fixed activities on its own node queue, and holds alist writeactivity that writes a file with a temporary file and a rename. Every payload across the Temporal boundary is a protobuf message built with buf and checked by protovalidate (Phase 4 D-25).- Kea 3.2 on each Pi runs in hot-standby with a unix control socket that only root and
_keareach, and an HTTP control listener on127.0.0.1:8000with basic authentication. The Kea role renderssubnet4[].reservationsinline fromtf/dns-records/hosts.ymlby address arithmetic. tf/dns-recordsreadshosts.yml(37 entries, 21 with a MAC) inhosts.tfand derives the A, AAAA, CNAME and PTR resources. Terraform 1.15 and themmianl/powerdns2.5.0 provider. The provider keeps a foreign comment whencommentsis unset.tf/rustfsmanages one bucket on the NAS RustFS through theaminueza/minioprovider.tf/backblazemanages buckets and application keys and stores each key in Vault. The NAS S3 endpoint is192.168.20.203, nameds3-direct.nas.fzymgc.housein/etc/hostson the Pis and in no DNS zone.docs/reference/network.mdis the addressing authority. This change allocates no new address or prefix.- The dns-control network policy names every egress by namespace or address and forbids the
Cilium DNS-name key. It holds no
worldentity today.
Goals / Non-Goals
Section titled “Goals / Non-Goals”Goals:
- One writer for a curated record and its reservation, and one store that both project from.
- A change that completes with no pull request, no merge and no second applier.
- A rebuild path for a Pi and for the entry set that needs neither the cluster nor git.
- The smallest change to the Kea role and to the node worker that gives the above.
Non-Goals:
- Moving legs, listeners, client groups or any other topology into the control plane. SEED-001 stays open for those.
- Moving the
records.tffamilies, the DDNS resources or the zones out of Terraform. - A hosts database for Kea, or Kea’s
reservation-addcommand. Both put a database on the read path of a Pi. - Batching several changes into one export. The queue is serial and an unchanged set writes nothing, so one export per change is enough until the audit rows show bursts.
- A review step after a change is live. The audit objects and the Changes screen are the record.
Decisions
Section titled “Decisions”D-1: The entry is the unit of truth, and pdns and Kea are projections
Section titled “D-1: The entry is the unit of truth, and pdns and Kea are projections”Sean ruled on 2026-08-18 that a device’s reservation and its A record are one literal. Under the
old model the literal was a line in hosts.yml. Under this model it is an entry row. pdns holds
the RRsets and the Kea fragment holds the reservation, and both are derived from the row by the
workflow. The lineage index compares pdns with the row and reports drift.
Alternative: pdns as the truth for records and the database as the truth for MACs only. That keeps the invariant in the workflow instead of the data, and it gives the reconcile no target. Rejected.
The write API gains an entry shape beside the RRset shape of WRT-01. An entry write produces the A, AAAA, CNAME and PTR RRsets and the reservation in one plan. The RRset shape stays for TXT and CNAME names that are not host entries.
D-2: One ChangeWorkflow on the existing serial queue
Section titled “D-2: One ChangeWorkflow on the existing serial queue”The workflow runs five activities in order: pdns write, reservation write on the passive node,
reservation write on the active node, export to each sink, and audit finalise plus event. It
reuses the readOrder of OpsWorkflow for the node order and the single-worker task queue of
WRT-05. Each activity has its own retry policy. The pdns write and the reservation writes are
idempotent by content, so a retry after a crash is safe. A failure in the reservation write does
not roll back pdns: the audit row records the partial state, the Changes screen shows it, and a
retry of the workflow finishes it. Temporal’s history is the record of a half-done change.
Alternative: one activity that does everything. Rejected, because a retry would repeat the pdns write and the export for a Kea failure.
D-3: The reservation write is a file per leg and a config-reload over the HTTP listener
Section titled “D-3: The reservation write is a file per leg and a config-reload over the HTTP listener”The activity input is ReservationWriteInput { repeated LegReservations legs }, where each leg
carries its name and repeated Reservation { hw_address, ip_address, hostname }. dns-control
places an entry in a leg from ipv4_cidr in the leg file, so the worker does no arithmetic and
the Kea subnet id never reaches dns-control. The worker writes
/var/lib/dns-worker/kea/reservations-<leg>.json for every leg in the input, with the
writeAtomic helper of the list write, and then posts config-reload to 127.0.0.1:8000 with
the kea-ctrl basic-authentication credential. /etc/kea is 0750 root:_kea, so the worker
cannot read the copy that the Kea role writes. The worker role reads the same Vault entry
(secret/fzymgc-house/infrastructure/dns/kea-ha, fields ctrl_auth_user and
ctrl_auth_password) on the control node and writes its own copy under
/etc/dns-node-worker/kea/ at 0640 root:dns-worker, with the mode set at the write and a verify
task that reads it back as the worker user.
The template renders "reservations": <?include "/var/lib/dns-worker/kea/reservations-<leg>.json"?>
in each subnet block. The Phase 3 comment in the template that rejects an include file argues
against a generated intermediate between two readers of one file. There is now one writer and no
second reader, so the objection no longer applies, and the comment is rewritten to say why.
Alternative: the unix control socket. It needs root or _kea, and _kea in the worker’s
SupplementaryGroups gives the worker every Kea command. The HTTP listener exists for
kea-shell and the credential is already on the box. Chosen.
Alternative: config-set with the whole configuration. That makes the worker the owner of the
Kea configuration and collides with Ansible. Rejected.
Rollback for a reservation change: apply the previous plan of the entry, or on a node run
scripts/dns-playbook.sh --limit <node> --tags kea after removing the file, which reseeds it from
the export.
D-4: The export is a dual write to a NAS bucket and a B2 bucket, decided by Sean on 2026-09-29
Section titled “D-4: The export is a dual write to a NAS bucket and a B2 bucket, decided by Sean on 2026-09-29”One activity, change.export_sink, runs once per sink inside its own workflow change.export,
each run with its own retry policy and each idempotent: the activity reads the current object’s
sha256 from its metadata and skips the write when it matches. The object entries.json carries
the sha256 as user metadata. The change row stays immutable after it is final, so the result of
each sink lives in the side table change_exports, keyed by change and sink. A sink with no row is
pending, and the Changes screen reads it as pending: <sink>. The export workflow is a child of
the change workflow with the parent close policy abandon, so a sink that is down never holds the
change lock.
Sean rejected git as the sink on 2026-09-29: the ruleset on main requires a pull request and
its bypass only covers the pull request rule, so a bot cannot land on main, and a bot branch or
a second repository adds a GitHub App for nothing the audit rows do not already give. Sean
rejected RustFS tiered storage on 2026-09-29 because a tiered object is unreadable while the NAS
is down. Sean rejected RustFS S3 Tables for the audit on 2026-09-29 because the append-only
trigger in CNPG is the audit’s guarantee and Iceberg cannot give it.
The buckets: tf/rustfs adds minio_s3_bucket dns-control-export with prevent_destroy and
minio_s3_bucket_versioning Enabled, plus a minio_iam_user, a minio_iam_policy scoped to
that bucket and a vault_kv_secret_v2 for the key, per the RustFS runbook. tf/backblaze adds a
b2_bucket with versioning and a b2_application_key scoped to it, stored in Vault beside
cluster_backups. The dns-control ExternalSecret adds both keys.
Network: s3-direct.nas.fzymgc.house has no DNS record by design, so the Deployment gets a
hostAliases entry for 192.168.20.203, and the network policy gets toCIDRSet for that
address on 443. B2’s addresses are not fixed and the policy forbids the DNS-name key, so the
policy gets toEntities: [world] on TCP 443, the shape the ClickStack policy uses for its S3
backups. The header comment of the policy is rewritten to name the two reasons.
D-5: The Kea role seeds from the export and never overwrites
Section titled “D-5: The Kea role seeds from the export and never overwrites”The role reads entries.json on the control node with the aws command and the scoped key of
each sink from Vault, the NAS bucket first and B2 when the NAS does not answer, checks the sha256
of the bytes against the object metadata, slices the set per trusted leg by the address
arithmetic of the subnet block, and writes each file with force: false. A node with a file keeps it. A fresh node gets the export. kea_reservations_path
and the include_vars task go away. kea-dhcp4 -t in the role’s verify stays and now covers the
includes.
Alternative: seed empty arrays and let the worker fill them. Rejected because a reserved device on a fresh active node would get a pool address until the first change.
D-6: The Terraform handoff is removed with destroy = false and a mark adoption run
Section titled “D-6: The Terraform handoff is removed with destroy = false and a mark adoption run”Terraform 1.15 supports removed { from = ...; lifecycle { destroy = false } }. The handoff
pull request adds one removed block per resource family (hosts, hosts_aaaa, host_aliases,
the PTR resources of reverse.tf), deletes hosts.yml and hosts.tf, and cuts the
host_entries and canonical_ptr_map outputs and the canonical_ptr read in
reverse-ddns.tf. The apply removes the resources from state and touches no zone. Before the
merge, the store already holds the import of hosts.yml and the export diff is empty. After the
apply, one workflow run marks every curated RRset with writer=dns-control change=<uuid> and
writes nothing else, so the dig readings before and after are equal.
Rollback: git revert of the handoff pull request. The apply then imports nothing, because the
resources are gone from state, so the rollback is terraform import of each resource from the
export, with the import commands generated by dns-control entries export --format tf-import.
That command ships in the same pull request as the removed blocks.
The mark grammar becomes writer=dns-control change=<uuid>. The parser accepts the old grammar
with pr= until no RRset carries it, then the acceptance is removed.
D-7: Placement and validation live in dns-control, not in the worker
Section titled “D-7: Placement and validation live in dns-control, not in the worker”The worker validates only the shape of each reservation. dns-control validates that an address sits in exactly one trusted leg, that a MAC is unique across entries, and that the write-policy filter accepts the hostname and every alias. The worker never reads the leg file.
Risks / Trade-offs
Section titled “Risks / Trade-offs”- [A half-done change: pdns written, reservation not] → The audit row and the Changes screen
show the state, the workflow retries, and
dns_list_driftreports the name until Kea matches. - [Kea refuses the reload because one included file is malformed] → The worker validates each
reservation before the write, and a refused write leaves the old file.
kea-dhcp4 -truns in the role’s verify. - [The credential of the HTTP listener widens: the worker can send any command] → The
restrict-commandslist of that listener is checked in the design task, andconfig-reload,status-getand the lease commands are the set it allows. - [B2 egress opens
world:443for dns-control] → The policy comment names the reason, and the only world call in the binary is the B2 client. - [The NAS object and the B2 object disagree] → Both carry the sha256, the audit row holds the expected one, and the seed task refuses a file whose sha256 does not match the audit object it reads beside it.
- [A person edits pdns by hand] → The lineage reconcile reports the name as drift. Sean ruled on 2026-09-14 against a drift detector for Terraform-owned inventory. This reconcile is different: it compares two stores the repository owns, which is the case the ruling allowed.
- [The lineage index attributes fewer names to Terraform] → This is a gain. Issue #2413 shrinks
to the
records.tffamilies.
Migration Plan
Section titled “Migration Plan”- Merge the buckets, credentials, ExternalSecret,
hostAliasesand network policy. Nothing reads them yet. - Merge dns-control with the store, the workflow and the export activities. The store is empty.
- Merge the node worker and the Kea role. Converge the passive node, then the active node. The
seed reads an export the store produced from a dry-run import of
hosts.yml, so Kea renders the same reservations as before. - Import
hosts.ymlinto the store for real. Run the export diff. Run the lineage reconcile and make sure that no curated name shows drift. - Merge the handoff. Read the plan: every curated record removed from state,
0 to destroy. After the apply, run the mark adoption, then thedigreadings. - Merge the docs and the planning amendments.
Rollback per step is in the decision that introduces it. Steps 1 to 4 are reversible by revert with no live effect.
Open Questions
Section titled “Open Questions”- The name of the B2 bucket and its region.
tf/backblazefixes the convention at implementation time, and nothing in the specs depends on the name. - Whether the Changes screen gets a “retry export now” button or the operator waits for the workflow’s retry. The retry policy decides the wait, and the button can come later.
This file is the copy of the design of the OpenSpec change curated-records-as-control-plane-data.
The change directory holds the proposal, the specs and the tasks.