Flux kustomize-controller SSA ownership — per-object dispositions, 2026-09-07
Captured against context fzymgc-house (server v1.36.0+k3s1) with two full
kubectl get … --show-managed-fields -o json sweeps — one over every namespaced API resource
enumerated by kubectl api-resources --verbs=list --namespaced=true (204 kinds, -A), one over
every cluster-scoped resource from the --namespaced=false counterpart (57 kinds) — projected with
jq, and edited with kubectl patch --type=json. No kubectl apply was used at any point.
Why this file exists: Flux is uninstalled from this cluster and its kustomize-controller
cannot re-acquire field ownership, so the audit that found this residue has no second customer and a
re-runnable tool would have no one to serve. What is worth keeping is the disposition reached for
each object and the one piece of method knowledge that has repeatedly been lost — the
--show-managed-fields trap below. This is a dated record of work performed on a specific day, not
standing operational prose, so it lives in evidence/ rather than in a runbook under
docs/operations/.
Where the planning artifacts cited below live. Several passages cite a phase planning artifact
by bare filename — 04-CONTEXT.md, 04-RESEARCH.md, 04-01-PLAN.md. Every one of them lives in
.planning/workstreams/cluster-health/phases/04-vestigial-and-dead-references/. That directory is a
planning record kept in the repository and is deliberately not published to the documentation site,
so a reader on the site should expect to open those artifacts from a checkout rather than follow a
link from here.
The mechanic, and the trap
Section titled “The mechanic, and the trap”The substance of this section is already written down in-tree, at
argocd/app-configs/velero/backup-schedule.yaml:74-90, where it was recorded on 2026-08-24 after the
same manager was found holding an includedResources allow-list on both Velero Schedules. It is
restated here because that comment is attached to one manifest and this residue spans 55 objects.
Server-side apply lets a manager delete only the fields it owns. Ownership is recorded per field
path, in each object’s metadata.managedFields, and it does not expire when the manager stops
running. ArgoCD therefore cannot remove a field it never declared, and — this is the part that makes
the defect invisible — it reports Synced the whole time, because from its own point of view every
field it declares does match. A manifest can be true in git and false in the cluster for months.
kubectl patch --type=json is not an apply. It is a plain update, which bypasses SSA ownership
entirely, and that is precisely why it can delete a field a departed manager owns while an ArgoCD
sync never can. Where git already agrees with the intended end state, the patch converges the cluster
toward the manifest rather than away from it, which is the exemption the Velero comment argues and
this phase inherits.
Removing the last field in a manager’s entry collapses the entry. Removing a field the manager co-owns with a living manager does not: the entry survives carrying the co-owned paths, and removing those paths would destroy real configuration. That distinction is the whole reason this phase clears ownership in two narrow steps rather than one blunt one — see Correction: the entry does not collapse on the objects this tracer chose below.
The trap: kubectl get -o json STRIPS managedFields unless --show-managed-fields is passed.
An audit that omits the flag returns a clean, confident zero. Stated adjacent to every command in
this file that could be invalidated by it: each of the two sweeps in the transcript below was taken
with --show-managed-fields, and each per-object re-read after a patch was taken with
--show-managed-fields. The flag’s presence is itself asserted, not assumed — the transcript records
how many objects in the two dumps carry a non-empty managedFields array, and a zero there
invalidates every other number in this file.
This trap has now fired three times in this repository: once in the original 2026-08-24 SSA audit;
once in the argocd/app-configs/velero/backup-schedule.yaml investigation, which is why the comment
there ends with a note about it; and once — most instructively — inside the 04-CONTEXT.md paragraph
that was written to warn about it, whose own measured figures were taken without the flag. Always
include a known-affected object as a negative control.
Inventory at sweep start, verbatim
Section titled “Inventory at sweep start, verbatim”The VEST-01 gate is read two-sided. Query (a) counts objects still carrying a
kustomize-controller entry in managedFields, and its eventual target is near zero — but an empty
result from a broken query reads identically to a genuine zero, and for this particular query the
broken form is one forgotten flag away. The inverted-predicate control immediately below it counts
objects owned by argocd-controller over the same two dumps with the same traversal; a large
non-zero control proves the traversal can see field ownership at all.
=== capture start: 2026-09-07T19:40:05Z ===--- (0) PRECONDITION: objects in the two dumps carrying a non-empty managedFields array ---12024--- (a) objects field-owned by kustomize-controller ---55--- (a-inverted) CONTROL: objects field-owned by argocd-controller ---904--- (b) breakdown of (a): cluster-scoped rows ---20--- (b) breakdown of (a): the three Namespaces carrying NO Flux label ---Namespace/-/cert-managerNamespace/-/external-secretsNamespace/-/metallb--- (c) API surface swept ---namespaced kinds enumerated: 204cluster-scoped kinds enumerated: 57[VERIFIED: kubectl, 2026-09-07T19:40:41Z] — (a) returned 55 objects field-owned by
kustomize-controller, and its inverted control returned 904 objects field-owned by
argocd-controller, over dumps in which 12,024 objects carry a non-empty managedFields array.
The 55 breaks down as 20 cluster-scoped rows and 35 namespaced ones; the three label-less Namespaces
cert-manager, external-secrets and metallb are present by name, which is the structural proof
that no Flux-label pre-filter was applied. The full 55-line inventory was frozen to
/tmp/flux-inventory.txt and the two dumps to /tmp/flux-ns.json and /tmp/flux-cl.json before
the first patch, because the selector every previous audit used is erased by the very patch this
phase applies.
Correction: the inventory is 55 objects, not 35
Section titled “Correction: the inventory is 55 objects, not 35”Both the origin todo (2026-08-24-clear-vestigial-flux-ssa-field-ownership.md) and 04-CONTEXT.md
state the in-scope set as 35 objects. Measured today it is 55. Three independent causes, each
of which under-reports on its own:
- The sweep was namespaced-only. The origin method enumerated resources with
kubectl api-resources --verbs=list --namespaced=true, which cannot see a cluster-scoped carrier. There are 20 of them —ClusterRole×4,ClusterRoleBinding×4,Namespace×8,VolumeSnapshotClass×2,ClusterIssuer×1,CustomResourceDefinition×1. This accounts for the whole of the 35 → 55 gap. - A Flux-label pre-filter is blind to three Namespaces. Using
-l kustomize.toolkit.fluxcd.io/nameas the audit’s source of truth missesNamespace/-/cert-manager,Namespace/-/external-secretsandNamespace/-/metallb, which carrykustomize-controllerfield ownership while carrying no Flux label at all. A label-filtered sweep of both halves returns 52, not 55. The selector is also self-erasing — it is simultaneously the audit’s query and the thing being removed — which is why the capture above had to precede the first patch. - Object-level counting over-reports the field-level problem by roughly six times. For most of
the 55, the only paths
kustomize-controllersolely owns are the two labels it wrote,metadata.labels.kustomize.toolkit.fluxcd.io/nameand/namespace. SSA tracks ownership per field path, not per object, so “55 objects affected” and “55 objects carrying real orphaned configuration” are very different claims, and only the first is true.
Causes 1 and 2 make the number too small; cause 3 makes it sound larger than it is. Both directions were live in the upstream artifacts at once.
Correction: the entry does not collapse on the objects this tracer chose
Section titled “Correction: the entry does not collapse on the objects this tracer chose”04-01-PLAN.md Task 1 predicts, for both tracer objects, that “the entry [will] collapse entirely
on both, because the two labels were its only content”, and its acceptance criteria assert that
neither object carries a kustomize-controller entry in managedFields after the patch. Measured by
--dry-run=server before either live patch, and confirmed by re-read after: both entries survive.
The plan’s own research already said so. 04-RESEARCH.md §RQ-5.5 row 30
(Issuer/cnpg-system/selfsigned-issuer, co-owned paths 1) and row 55
(VolumeSnapshotClass/-/longhorn-snap, co-owned paths 3) both carry entry collapses on value-patch alone: **no**. The labels were not the entry’s only content: kustomize-controller
also appears as an owner of spec.selfSigned on the Issuer and of deletionPolicy, driver and
parameters.type on the VolumeSnapshotClass — and on all four of those paths it shares ownership
with the live argocd-controller. Removing them is expressly prohibited: it would delete real
configuration a living manager depends on.
Entry survival is the expected outcome, not a failure, and the gate was reading the wrong
property. VEST-01’s Done-when as amended on 2026-09-07 is no live object has a field whose sole
owner is kustomize-controller — not no object carries an entry. Measured over the frozen
pre-patch dumps, of the 55 objects:
- 3 carry a
kustomize-controllerentry whose entire content is the two Flux labels. Only these three collapse on the value patch —ClusterRole/-/plugin-barman-cloud,ServiceAccount/kube-system/replicator-kubernetes-replicatorandServiceAccount/vault/tf-runner. - 40 carry at least one path the entry shares with a live manager. On these the entry can never be emptied by any patch that respects live configuration, and both tracer objects are among them.
- 40 have zero sole-Flux paths once the two labels are gone — i.e. the value patch alone discharges the Done-when for them, entry still present.
- 15 retain a sole-Flux field after the value patch. That is the set the narrow entry strip in 04-02 Task 4 is scoped to, less the four objects excluded by ruling — see Predicted residual, derived pre-sweep below.
The two 40s are different sets of the same size, and the two 15s likewise: co-ownership and
surviving-sole-ownership are independent properties, and an object can have both (the replicator
Deployment co-owns 20 paths with argocd-controller and k3s and solely retains spec.strategy).
The assertion actually made below is therefore the sole-ownership one, which is both the stronger
claim and the one the phase is accountable for. See 04-01-SUMMARY.md § Deviations.
Per-object dispositions
Section titled “Per-object dispositions”Sole ownership is computed by walking every managedFields entry’s fieldsV1 tree to its leaf paths
and set-differencing the kustomize-controller paths against the union of every other manager’s.
Where the post-patch entry state column reads survives, the entry remains but holds only paths a
live manager co-owns — which satisfies the amended Done-when. The first two rows were written by 04-01
from its tracer slice; the remaining 53 were filled by 04-02 Task 1 — 42 objects it swept, and 11 it
deliberately excluded, each row naming which exclusion applies and why. All 55 inventory objects are
now accounted for.
The co-owned paths and post-patch entry columns for those 53 rows are measured after the
patch, by re-reading each object with --show-managed-fields and re-running the same leaf-path
set-difference. They are not transcribed from 04-RESEARCH.md §RQ-5.5’s pre-patch table, and on two
objects the two disagree: Deployment/cnpg-system/barman-cloud reads 38 co-owned paths here
against §RQ-5.5’s 31, and Deployment/kube-system/replicator-kubernetes-replicator 22 against 20.
The live reading is the authoritative one — it is the state the entry strip in Task 4 actually acts
on. The cause of the gap was not chased down, because it does not change any disposition: both
objects were entry survives before and after, and the property the phase is accountable for is the
sole-ownership column, which agrees with the prediction exactly.
| Object | Flux label | Sole-Flux fields beyond the labels | Co-owned paths | Action taken | Post-patch kustomize-controller entry |
|---|---|---|---|---|---|
Issuer/cnpg-system/selfsigned-issuer |
yes (2) | none | 1 — spec.selfSigned, co-owned with argocd-controller |
D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding spec.selfSigned only; sole-Flux paths: 0 |
VolumeSnapshotClass/-/longhorn-snap |
yes (2) | none | 3 — deletionPolicy, driver, parameters.type, all co-owned with argocd-controller |
D-04 step 1 value patch, identical shape, no namespace flag; --dry-run=server first, then live |
survives, holding those 3 paths only; sole-Flux paths: 0 |
Certificate/cnpg-system/barman-cloud-client |
yes (2) | none | 9 — co-owned with argocd-controller, cert-manager-certificates-issuing, cert-manager-certificates-key-manager, cert-manager-certificates-readiness, cert-manager-certificates-trigger | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 9 co-owned paths; sole-Flux paths: 0 |
Certificate/cnpg-system/barman-cloud-server |
yes (2) | none | 10 — co-owned with argocd-controller, cert-manager-certificates-issuing, cert-manager-certificates-key-manager, cert-manager-certificates-readiness, cert-manager-certificates-trigger | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 10 co-owned paths; sole-Flux paths: 0 |
Certificate/longhorn-system/longhorn-tls |
yes (2) | none | 5 — co-owned with argocd-controller, cert-manager-certificates-issuing, cert-manager-certificates-key-manager, cert-manager-certificates-readiness, cert-manager-certificates-trigger | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 5 co-owned paths; sole-Flux paths: 0 |
Certificate/postgres/postgres-main-server |
yes (2) | none | 5 — co-owned with argocd-controller, cert-manager-certificates-issuing, cert-manager-certificates-key-manager, cert-manager-certificates-readiness, cert-manager-certificates-trigger | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 5 co-owned paths; sole-Flux paths: 0 |
Certificate/postgres/postgres-main-streaming-replica-client |
yes (2) | none | 6 — co-owned with argocd-controller, cert-manager-certificates-issuing, cert-manager-certificates-key-manager, cert-manager-certificates-readiness, cert-manager-certificates-trigger | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 6 co-owned paths; sole-Flux paths: 0 |
Certificate/vault/vault-api-tls |
yes (2) | none | 7 — co-owned with argocd-controller, cert-manager-certificates-issuing, cert-manager-certificates-key-manager, cert-manager-certificates-readiness, cert-manager-certificates-trigger | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 7 co-owned paths; sole-Flux paths: 0 |
Cluster/postgres/main |
yes (2) | none | 9 — co-owned with argocd-controller, kubectl-cnpg, manager | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 9 co-owned paths; sole-Flux paths: 0 |
ClusterIssuer/-/fzymgc-house-issuer |
yes (2) | none | 1 — co-owned with Terraform, cert-manager-clusterissuers | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 1 co-owned path; sole-Flux paths: 0 |
ClusterRole/-/k8s-full-secrets-abilities-with-labels |
yes (2) | none | 1 — co-owned with argocd-application-controller, argocd-controller | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 1 co-owned path; sole-Flux paths: 0 |
ClusterRole/-/plugin-barman-cloud |
yes (2) | none | 0 — none | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
collapsed — the two labels were its entire content; sole-Flux paths: 0 |
ClusterRole/-/replicator-kubernetes-replicator |
yes (2) | none | 1 — co-owned with argocd-application-controller, argocd-controller | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 1 co-owned path; sole-Flux paths: 0 |
ClusterRole/-/velero |
yes (2) | none | 1 — co-owned with argocd-application-controller, argocd-controller | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 1 co-owned path; sole-Flux paths: 0 |
ClusterRoleBinding/-/plugin-barman-cloud-binding |
yes (2) | none | 4 — co-owned with argocd-application-controller, argocd-controller | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 4 co-owned paths; sole-Flux paths: 0 |
ClusterRoleBinding/-/replicator-kubernetes-replicator |
yes (2) | none | 2 — co-owned with argocd-application-controller, argocd-controller | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 2 co-owned paths; sole-Flux paths: 0 |
ClusterRoleBinding/-/velero |
yes (2) | none | 2 — co-owned with argocd-application-controller, argocd-controller | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 2 co-owned paths; sole-Flux paths: 0 |
CustomResourceDefinition/-/objectstores.barmancloud.cnpg.io |
yes (2) | none | 6 — co-owned with argocd-controller, k3s | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 6 co-owned paths; sole-Flux paths: 0 |
Deployment/cnpg-system/barman-cloud |
yes (2) | spec.template.metadata.creationTimestamp |
38 — co-owned with argocd-controller, k3s | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 38 co-owned paths; sole-Flux paths: 1 — D-07: the value is re-established by the API server and ownership does not move, so the Task 4 entry strip is the route |
Deployment/kube-system/replicator-kubernetes-replicator |
yes (2) | spec.strategy, spec.template.metadata.creationTimestamp |
22 — co-owned with argocd-controller, k3s | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 22 co-owned paths; sole-Flux paths: 2 — D-07: the value is re-established by the API server and ownership does not move, so the Task 4 entry strip is the route |
ExternalSecret/postgres/main-backup-storage-credentials |
yes (2) | none | 10 — co-owned with argocd-controller, external-secrets | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 10 co-owned paths; sole-Flux paths: 0 |
ExternalSecret/postgres/main-superuser-credentials |
yes (2) | none | 10 — co-owned with argocd-controller, external-secrets | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 10 co-owned paths; sole-Flux paths: 0 |
ExternalSecret/velero/cloud-credentials |
yes (2) | none | 9 — co-owned with argocd-controller, external-secrets | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 9 co-owned paths; sole-Flux paths: 0 |
IngressRoute/longhorn-system/longhorn-dashboard |
yes (2) | none | 2 — co-owned with argocd-controller, operator | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 2 co-owned paths; sole-Flux paths: 0 |
IngressRoute/traefik/traefik-dashboard |
yes (2) | none | 2 — co-owned with argocd-controller, operator | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 2 co-owned paths; sole-Flux paths: 0 |
IngressRouteTCP/vault/vault |
yes (2) | none | 3 — co-owned with argocd-controller, operator | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 3 co-owned paths; sole-Flux paths: 0 |
IPAddressPool/metallb/default |
yes (2) | none | 1 — co-owned with Terraform, argocd-controller, controller | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 1 co-owned path; sole-Flux paths: 0 |
L2Advertisement/metallb/default |
yes (2) | none | 1 — co-owned with Terraform, argocd-controller | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 1 co-owned path; sole-Flux paths: 0 |
Namespace/-/cnpg-system |
yes (2) | spec |
0 — none | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 0 co-owned paths; sole-Flux paths: 1 — D-07: the value is re-established by the API server and ownership does not move, so the Task 4 entry strip is the route |
Namespace/-/postgres |
yes (2) | spec |
0 — none | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 0 co-owned paths; sole-Flux paths: 1 — D-07: the value is re-established by the API server and ownership does not move, so the Task 4 entry strip is the route |
Namespace/-/traefik |
yes (2) | spec |
0 — none | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 0 co-owned paths; sole-Flux paths: 1 — D-07: the value is re-established by the API server and ownership does not move, so the Task 4 entry strip is the route |
Namespace/-/vault |
yes (2) | spec |
0 — none | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 0 co-owned paths; sole-Flux paths: 1 — D-07: the value is re-established by the API server and ownership does not move, so the Task 4 entry strip is the route |
Namespace/-/velero |
yes (2) | spec |
2 — co-owned with argocd-controller | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 2 co-owned paths; sole-Flux paths: 1 — D-07: the value is re-established by the API server and ownership does not move, so the Task 4 entry strip is the route |
ObjectStore/postgres/barman-object-store-main |
yes (2) | none | 6 — co-owned with argocd-controller, manager | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 6 co-owned paths; sole-Flux paths: 0 |
Schedule/velero/daily-backup |
yes (2) | none | 3 — co-owned with argocd-controller, velero-server | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 3 co-owned paths; sole-Flux paths: 0 |
Schedule/velero/weekly-full-backup |
yes (2) | none | 3 — co-owned with argocd-controller, velero-server | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 3 co-owned paths; sole-Flux paths: 0 |
Service/cnpg-system/barman-cloud |
yes (2) | none | 10 — co-owned with argocd-controller, manager | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 10 co-owned paths; sole-Flux paths: 0 |
Service/vault/vault-0 |
yes (2) | none | 8 — co-owned with argocd-controller | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 8 co-owned paths; sole-Flux paths: 0 |
Service/vault/vault-1 |
yes (2) | none | 8 — co-owned with argocd-controller | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 8 co-owned paths; sole-Flux paths: 0 |
Service/vault/vault-2 |
yes (2) | none | 8 — co-owned with argocd-controller | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 8 co-owned paths; sole-Flux paths: 0 |
ServiceAccount/cnpg-system/plugin-barman-cloud |
yes (2) | none | 2 — co-owned with argocd-controller | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 2 co-owned paths; sole-Flux paths: 0 |
ServiceAccount/kube-system/replicator-kubernetes-replicator |
yes (2) | none | 0 — none | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
collapsed — the two labels were its entire content; sole-Flux paths: 0 |
ServiceAccount/velero/velero |
yes (2) | none | 1 — co-owned with argocd-controller | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 1 co-owned path; sole-Flux paths: 0 |
VolumeSnapshotClass/-/longhorn-bak |
yes (2) | none | 3 — co-owned with argocd-controller | D-04 step 1 value patch: JSON-Patch remove of both metadata.labels.kustomize.toolkit.fluxcd.io/* keys, --dry-run=server first, then live |
survives, holding 3 co-owned paths; sole-Flux paths: 0 |
Certificate/traefik/wildcard-fzymgc-house-tls |
yes (2) | spec.dnsNames, spec.issuerRef.kind, spec.issuerRef.name, spec.secretName, spec.usages |
0 — none | EXCLUDED — D-01 / ADR hl-0gol exemption. Not patched, not stripped. The cluster’s default TLS certificate and the highest-blast-radius object in the estate |
untouched — entry present, both Flux labels present; this is the one object on which VEST-01’s Done-when is not met by design |
Secret/vault/vault-token |
yes (2) | data.VAULT_TOKEN, type |
0 — none | DELETED by 04-03 (D-06). Never patched or stripped here; cleared by removal of the whole object. See docs/operations/evidence/2026-09-07-vault-tf-runner-decommission.md for the pre-flight, revoke-branch and deletion transcripts with their controls |
no entry — the object no longer exists. Absent at the closing gate. The credential outcome is recorded in the decommission file and is deliberately not restated here |
ServiceAccount/vault/tf-runner |
yes (2) | none beyond the two labels | 0 — none | DELETED by 04-03 (D-06). Reads like an ordinary sweep object and is not one: never value-patched, so its two label paths were still solely owned right up to deletion. See docs/operations/evidence/2026-09-07-vault-tf-runner-decommission.md |
no entry — the object no longer exists. Named in every exclusion list for exactly that reason: a sweep that had swallowed it would have made the D-06 unit read as two objects rather than three |
ClusterRoleBinding/-/vault-tf-runner |
yes (2) | roleRef, subjects |
0 — none | DELETED by 04-03 (D-06). Bound a ServiceAccount used by 0 pods to a ClusterRole that does not exist; removed rather than repaired, which is a strict reduction in privilege. See docs/operations/evidence/2026-09-07-vault-tf-runner-decommission.md |
no entry — the object no longer exists. Deleted binding-first, before the ServiceAccount it named |
Namespace/-/cert-manager |
NO | spec |
0 — none | EXCLUDED from the sweep — carries no Flux label at all, so a label patch has nothing to remove. Flux-owned through spec alone; belongs to the residual, not the sweep |
entry present holding f:spec; D-07 — namespaceStrategy.PrepareForUpdate copies spec.finalizers unconditionally, so no value patch can release it. Task 4 strips the entry |
Namespace/-/external-secrets |
NO | spec |
0 — none | EXCLUDED from the sweep — no Flux label. As above | as above; Task 4 strips the entry |
Namespace/-/metallb |
NO | spec |
0 — none | EXCLUDED from the sweep — no Flux label. As above | as above; Task 4 strips the entry |
ConfigMap/default/fzymgc-root-ca |
yes (2) | none | 1 — co-owned with argocd-controller |
DEFERRED to Task 2 — it is a mittwald replication source and its patch is the D-02 propagation experiment | see ## Replicator propagation (D-02) |
ConfigMap/default/fzymgc-ica1-ca |
yes (2) | none | 2 — co-owned with argocd-controller |
DEFERRED to Task 2 — replication source, D-02 experiment | see ## Replicator propagation (D-02) |
Secret/default/fzymgc-root-ca |
yes (2) | data.ca.crt |
0 — none | DEFERRED to Task 2 — replication source, D-02 experiment. The data path is never removed: the replicator deletes from every replica any key no longer on the source |
entry survives on data.ca.crt; Task 4 strips the entry, which releases the path without touching the value |
Secret/default/fzymgc-ica1-ca |
yes (2) | data.ca.crt, data.fullchain.crt |
0 — none | DEFERRED to Task 2 — replication source, D-02 experiment. data paths never removed, same reason |
entry survives on both data.* paths; Task 4 strips the entry |
The first two rows are 04-01’s tracer slice, chosen to take one from each half of the inventory: a
namespaced object, and a cluster-scoped one from the half the origin audit’s --namespaced=true
sweep could not see at all.
Both are ArgoCD-tracked and neither had anything sole-Flux beyond the two labels, so the value patch
could not reach live configuration on either even had a path been mis-specified.
=== tracer patch: 2026-09-07T19:42:34Z ===--- (a) PATCHED Issuer/cnpg-system/selfsigned-issuer ---flux label keys: 0 sole-Flux field paths: 0 kustomize-controller entry: present (spec.selfSigned, co-owned)--- (a) PATCHED VolumeSnapshotClass/-/longhorn-snap ---flux label keys: 0 sole-Flux field paths: 0 kustomize-controller entry: present (deletionPolicy, driver, parameters.type, co-owned)--- (a-inverted) CONTROL: IngressRoute/traefik/traefik-dashboard, not yet swept ---flux label keys: 2 sole-Flux field paths: 2 kustomize-controller entry: present[VERIFIED: kubectl --show-managed-fields, 2026-09-07T19:42:34Z] — both patched objects read 0
Flux label keys and 0 sole-Flux field paths, while the untouched control object
IngressRoute/traefik/traefik-dashboard still reads 2 and 2. Zero on both sides would have
been a broken query rather than a clean cluster; the control is what distinguishes them. Neither
patch removed a managedFields entry — no entry is removed anywhere in 04-01. The narrow entry
strip is scoped to the measured residual and runs in 04-02 Task 4, behind its measurement gate.
Predicted residual, derived pre-sweep
Section titled “Predicted residual, derived pre-sweep”This section is written before the sweep, from the frozen pre-patch dumps, and is the reference 04-02 Task 3 checks its measured post-sweep residual against. Deriving it rather than transcribing it is the point: a prediction copied out of a planning document tests only that two documents agree, while a prediction computed from the capture tests that the cluster is where the phase thinks it is. A mismatch at 04-02 Task 3 means the measurement basis moved again and is a stop-and-surface condition, not something to reconcile by editing either side.
The computation is /tmp/sole-owner.jq, run as
jq -s -r -f /tmp/sole-owner.jq --arg m kustomize-controller /tmp/flux-ns.json /tmp/flux-cl.json. It
walks every managedFields entry’s fieldsV1 tree to its leaf paths and set-differences the paths
owned by $m against the union of every other manager’s, emitting one line per object with a
non-empty remainder. The manager is a parameter, not a literal. That is what makes the control
possible: the target run passes kustomize-controller and the control run passes
argocd-controller, and the two invocations differ in exactly that one argument. A program with the
manager baked in cannot be controlled, because the control silently re-runs the target query and its
non-zero result proves nothing. 04-02 Tasks 3 and 4 run this same file unchanged, so the prediction
below and the measurement it is later compared against are provably the same computation.
=== prediction derived: 2026-09-07T19:44:00Z ===--- (a) objects with >=1 sole-kustomize-controller field path (pre-sweep) ---52--- (a-inverted) CONTROL: same program, --arg m argocd-controller ---801--- (b) of (a), objects still sole-owning a field once the two Flux labels are removed ---15--- (c) of (b), after the four exclusions ruled for this phase ---11[VERIFIED: kubectl + jq, 2026-09-07T19:44:00Z] — the target run returned 52 objects, the
inverted control 801, both non-empty and different, which is what proves --arg m is actually
read. Subtracting the two Flux label paths the sweep removes leaves 15; the four exclusions below
leave the predicted residual of 11, frozen to /tmp/flux-residual-predicted.txt. The derivation
agrees object-for-object with the residual ruled in 04-01-PLAN.md, so there is no disagreement to
record and no halt.
The 11 predicted residual objects, with the field path that survives:
| Object | Surviving sole-Flux path | Why the value patch cannot clear it |
|---|---|---|
Namespace/-/cert-manager |
spec |
namespaceStrategy.PrepareForUpdate unconditionally copies spec.finalizers from the old object, and f:spec marks a key that always exists |
Namespace/-/cnpg-system |
spec |
as above |
Namespace/-/external-secrets |
spec |
as above |
Namespace/-/metallb |
spec |
as above |
Namespace/-/postgres |
spec |
as above |
Namespace/-/traefik |
spec |
as above |
Namespace/-/vault |
spec |
as above |
Namespace/-/velero |
spec |
as above |
Secret/default/fzymgc-root-ca |
data.ca.crt |
removing the value propagates an empty CA to ~212 replicas before selfHeal restores it — a cluster-wide TLS-trust outage window |
Secret/default/fzymgc-ica1-ca |
data.ca.crt, data.fullchain.crt |
as above |
Deployment/kube-system/replicator-kubernetes-replicator |
spec.strategy |
re-defaulted byte-identically inside the same request, so structured-merge-diff sees no change and ownership does not move |
Four objects retain a sole-Flux field after the value patch and are deliberately NOT in the 11:
Certificate/traefik/wildcard-fzymgc-house-tls— sole-ownsspec.dnsNames,spec.issuerRef.kind,spec.issuerRef.name,spec.secretNameandspec.usages. The D-01 / ADRhl-0golexemption, and the one object on which VEST-01’s Done-when is not met by design. This is the “1” in 54 of 55.Secret/vault/vault-token— sole-ownsdata.VAULT_TOKENandtype. Cleared by deletion in 04-03, not by patch.ClusterRoleBinding/-/vault-tf-runner— sole-ownsroleRefandsubjects. Cleared by deletion in 04-03.Deployment/cnpg-system/barman-cloud— its only surviving sole-Flux path isspec.template.metadata.creationTimestamp, which the API server neither stores nor serves; a JSON-Patchremoveagainst it returns 422. Excluded by thephantom-narrowingruling: a path absent from the served object is not a field the object has. The judgement is recorded here rather than left silent, 04-02 Task 4 records the 422 verbatim, and the object is cleared anyway as a side effect of the entry strip, so the narrowing leaves nothing behind.
A third object 04-03 deletes does not appear in the derivation at all, and its absence is the easy
one to misread. ServiceAccount/vault/tf-runner has nothing sole-Flux beyond the two labels, so it
drops out of the prediction the moment the labels are subtracted and reads like an ordinary sweep
object. It is not one: 04-02 excludes it from the sweep because 04-03 owns it, so it is never
value-patched, and its two label paths are therefore still solely owned at 04-02’s measurement
time — it surfaces in the measured residual whenever 04-03 has not yet run, and 04-03 is a same-wave
sibling. Left out of 04-02’s exclusion list it turns the measurement gate into a coin flip on run
order. The D-06 group is three objects — the Secret, the ServiceAccount and the ClusterRoleBinding
— and every exclusion list in this phase names all three.
Where the arithmetic lands. The value patch alone reaches 39 of 55 — that is the
value-patch-alone figure, measured at the cluster after the sweep, and it is the only number in this
paragraph that describes the patch on its own. 04-03’s three deletions take it to 42
(vault-token and vault-tf-runner, which the patch could not have cleared, plus tf-runner, which
it never touches). The narrow entry strip in 04-02 Task 4 then takes the remaining 11 predicted
residual objects plus the phantom barman-cloud — 12 in all — bringing the total to 54 of 55.
The single object left is the ADR-exempt wildcard certificate. Nothing in 04-01 removed a
managedFields entry.
42 is a composite, never a value-patch-alone figure, and must not be quoted as one. It is also,
coincidentally, the size of the Task 1 sweep work set, which is how the two get conflated. The
§RQ-5.4 correction at the foot of this file measures the 39 and convicts 04-06-PLAN.md of restating
42 as the value-patch-alone figure; read the two passages together so they cannot drift apart again.
Correction: hl-e4ny is not a live handoff target
Section titled “Correction: hl-e4ny is not a live handoff target”The origin todo instructs that this work be “folded into hl-e4ny”, and ADR
hl-0gol-exclude-orphaned-vestigial-flux-wildcard-cert-from-new-argoc.md defers the wildcard
certificate to the same identifier. Beads are retired in this repository — bd reports no database
here — so both references are dead links, recorded here rather than acted on. The ADR’s substantive
ruling stands on its own merits independent of the bead it names: Certificate/traefik/wildcard-fzymgc-house-tls
is left alone, and is the single exemption to VEST-01’s Done-when.
Sweep transcript
Section titled “Sweep transcript”D-04 step 1, run across the whole inventory. The list is the source of truth, not a selector. The
42-object work set is /tmp/flux-inventory.txt (55 rows, frozen by 04-01 before the first patch)
minus thirteen objects in five groups:
| Removed from the sweep | Count | Why |
|---|---|---|
Certificate/traefik/wildcard-fzymgc-house-tls |
1 | D-01 / ADR hl-0gol exemption — the cluster’s default TLS certificate, deliberately deferred |
Secret/vault/vault-token, ServiceAccount/vault/tf-runner, ClusterRoleBinding/-/vault-tf-runner |
3 | Owned by 04-03, a wave-2 sibling running in parallel. All three, not one — the D-06 unit is a dead triple |
Namespace/-/cert-manager, Namespace/-/external-secrets, Namespace/-/metallb |
3 | Carry no Flux label at all, so a label patch has nothing to remove. Flux-owned through spec alone; they belong to the residual |
ConfigMap/default/fzymgc-{root,ica1}-ca, Secret/default/fzymgc-{root,ica1}-ca |
4 | The mittwald replication sources — deferred to Task 2, whose patch is the D-02 experiment |
Issuer/cnpg-system/selfsigned-issuer, VolumeSnapshotClass/-/longhorn-snap |
2 | Already cleared by the 04-01 tracer |
55 − 13 = 42. The resolved work set was written to /tmp/flux-worked.txt as one tab-separated
<kind>.<group> / name / namespace-or-dash triple per line, and it — not a kind-wide query — is what
the gate below iterates. A kind-wide assertion could not express “the objects this task is
responsible for”: ConfigMap and Secret still legitimately carry Flux labels on the four Task 2
sources and on ~212 replicator copies, and ServiceAccount still carries them on an object 04-03
owns, so a kind-wide zero would fail on a correctly executed sweep.
Bare kinds were resolved to <kind>.<group> from each object’s apiVersion in the frozen dumps, so
that Cluster.postgresql.cnpg.io and Issuer.cert-manager.io cannot bind to another installed API
group of the same kind name.
Every patch names exactly two paths, both under /metadata/labels/:
kubectl --context fzymgc-house patch <kind>.<group> <name> [-n <ns>] --type=json -p '[ {"op":"remove","path":"/metadata/labels/kustomize.toolkit.fluxcd.io~1name"}, {"op":"remove","path":"/metadata/labels/kustomize.toolkit.fluxcd.io~1namespace"}]'Nothing under /spec, /data or /metadata/managedFields appears in any patch in this task.
/spec/strategy, /spec/finalizers and /spec/template/metadata/creationTimestamp were all
deliberately left out: §RQ-2 measured all three classes and the API server re-establishes every one
of them, so their removal buys no ownership — and on the CA objects a data removal would buy a
cluster-wide TLS-trust outage. Each object was patched --dry-run=server -o name first and the
result read before the live run.
=== sweep start: 2026-09-07T20:43:53Z ===work set (from /tmp/flux-inventory.txt minus 13 excluded objects): 42patch (identical on every object, two paths, both under /metadata/labels/): [{"op":"remove","path":"/metadata/labels/kustomize.toolkit.fluxcd.io~1name"},{"op":"remove","path":"/metadata/labels/kustomize.toolkit.fluxcd.io~1namespace"}]--- per-object: dry-run=server first, then live ---certificate.cert-manager.io cnpg-system/barman-cloud-client dry-run: certificate.cert-manager.io/barman-cloud-client | live: certificate.cert-manager.io/barman-cloud-clientcertificate.cert-manager.io cnpg-system/barman-cloud-server dry-run: certificate.cert-manager.io/barman-cloud-server | live: certificate.cert-manager.io/barman-cloud-servercertificate.cert-manager.io longhorn-system/longhorn-tls dry-run: certificate.cert-manager.io/longhorn-tls | live: certificate.cert-manager.io/longhorn-tlscertificate.cert-manager.io postgres/postgres-main-server dry-run: certificate.cert-manager.io/postgres-main-server | live: certificate.cert-manager.io/postgres-main-servercertificate.cert-manager.io postgres/postgres-main-streaming-replica-client dry-run: certificate.cert-manager.io/postgres-main-streaming-replica-client | live: certificate.cert-manager.io/postgres-main-streaming-replica-clientcertificate.cert-manager.io vault/vault-api-tls dry-run: certificate.cert-manager.io/vault-api-tls | live: certificate.cert-manager.io/vault-api-tlscluster.postgresql.cnpg.io postgres/main dry-run: Warning: Retention policies specified in .spec.backup.retentionPolicy are only used by the in-tree barman-cloud support, which is not being used in this cluster. Please use a backup plugin and migrate this configuration to the plugin configurationcluster.postgresql.cnpg.io/main | live: Warning: Retention policies specified in .spec.backup.retentionPolicy are only used by the in-tree barman-cloud support, which is not being used in this cluster. Please use a backup plugin and migrate this configuration to the plugin configurationcluster.postgresql.cnpg.io/mainclusterissuer.cert-manager.io -/fzymgc-house-issuer dry-run: clusterissuer.cert-manager.io/fzymgc-house-issuer | live: clusterissuer.cert-manager.io/fzymgc-house-issuerclusterrole.rbac.authorization.k8s.io -/k8s-full-secrets-abilities-with-labels dry-run: clusterrole.rbac.authorization.k8s.io/k8s-full-secrets-abilities-with-labels | live: clusterrole.rbac.authorization.k8s.io/k8s-full-secrets-abilities-with-labelsclusterrole.rbac.authorization.k8s.io -/plugin-barman-cloud dry-run: clusterrole.rbac.authorization.k8s.io/plugin-barman-cloud | live: clusterrole.rbac.authorization.k8s.io/plugin-barman-cloudclusterrole.rbac.authorization.k8s.io -/replicator-kubernetes-replicator dry-run: clusterrole.rbac.authorization.k8s.io/replicator-kubernetes-replicator | live: clusterrole.rbac.authorization.k8s.io/replicator-kubernetes-replicatorclusterrole.rbac.authorization.k8s.io -/velero dry-run: clusterrole.rbac.authorization.k8s.io/velero | live: clusterrole.rbac.authorization.k8s.io/veleroclusterrolebinding.rbac.authorization.k8s.io -/plugin-barman-cloud-binding dry-run: clusterrolebinding.rbac.authorization.k8s.io/plugin-barman-cloud-binding | live: clusterrolebinding.rbac.authorization.k8s.io/plugin-barman-cloud-bindingclusterrolebinding.rbac.authorization.k8s.io -/replicator-kubernetes-replicator dry-run: clusterrolebinding.rbac.authorization.k8s.io/replicator-kubernetes-replicator | live: clusterrolebinding.rbac.authorization.k8s.io/replicator-kubernetes-replicatorclusterrolebinding.rbac.authorization.k8s.io -/velero dry-run: clusterrolebinding.rbac.authorization.k8s.io/velero | live: clusterrolebinding.rbac.authorization.k8s.io/velerocustomresourcedefinition.apiextensions.k8s.io -/objectstores.barmancloud.cnpg.io dry-run: customresourcedefinition.apiextensions.k8s.io/objectstores.barmancloud.cnpg.io | live: customresourcedefinition.apiextensions.k8s.io/objectstores.barmancloud.cnpg.iodeployment.apps cnpg-system/barman-cloud dry-run: deployment.apps/barman-cloud | live: deployment.apps/barman-clouddeployment.apps kube-system/replicator-kubernetes-replicator dry-run: deployment.apps/replicator-kubernetes-replicator | live: deployment.apps/replicator-kubernetes-replicatorexternalsecret.external-secrets.io postgres/main-backup-storage-credentials dry-run: externalsecret.external-secrets.io/main-backup-storage-credentials | live: externalsecret.external-secrets.io/main-backup-storage-credentialsexternalsecret.external-secrets.io postgres/main-superuser-credentials dry-run: externalsecret.external-secrets.io/main-superuser-credentials | live: externalsecret.external-secrets.io/main-superuser-credentialsexternalsecret.external-secrets.io velero/cloud-credentials dry-run: externalsecret.external-secrets.io/cloud-credentials | live: externalsecret.external-secrets.io/cloud-credentialsingressroute.traefik.io longhorn-system/longhorn-dashboard dry-run: ingressroute.traefik.io/longhorn-dashboard | live: ingressroute.traefik.io/longhorn-dashboardingressroute.traefik.io traefik/traefik-dashboard dry-run: ingressroute.traefik.io/traefik-dashboard | live: ingressroute.traefik.io/traefik-dashboardingressroutetcp.traefik.io vault/vault dry-run: ingressroutetcp.traefik.io/vault | live: ingressroutetcp.traefik.io/vaultipaddresspool.metallb.io metallb/default dry-run: ipaddresspool.metallb.io/default | live: ipaddresspool.metallb.io/defaultl2advertisement.metallb.io metallb/default dry-run: l2advertisement.metallb.io/default | live: l2advertisement.metallb.io/defaultnamespace -/cnpg-system dry-run: namespace/cnpg-system | live: namespace/cnpg-systemnamespace -/postgres dry-run: namespace/postgres | live: namespace/postgresnamespace -/traefik dry-run: namespace/traefik | live: namespace/traefiknamespace -/vault dry-run: namespace/vault | live: namespace/vaultnamespace -/velero dry-run: namespace/velero | live: namespace/veleroobjectstore.barmancloud.cnpg.io postgres/barman-object-store-main dry-run: objectstore.barmancloud.cnpg.io/barman-object-store-main | live: objectstore.barmancloud.cnpg.io/barman-object-store-mainschedule.velero.io velero/daily-backup dry-run: schedule.velero.io/daily-backup | live: schedule.velero.io/daily-backupschedule.velero.io velero/weekly-full-backup dry-run: schedule.velero.io/weekly-full-backup | live: schedule.velero.io/weekly-full-backupservice cnpg-system/barman-cloud dry-run: service/barman-cloud | live: service/barman-cloudservice vault/vault-0 dry-run: service/vault-0 | live: service/vault-0service vault/vault-1 dry-run: service/vault-1 | live: service/vault-1service vault/vault-2 dry-run: service/vault-2 | live: service/vault-2serviceaccount cnpg-system/plugin-barman-cloud dry-run: serviceaccount/plugin-barman-cloud | live: serviceaccount/plugin-barman-cloudserviceaccount kube-system/replicator-kubernetes-replicator dry-run: serviceaccount/replicator-kubernetes-replicator | live: serviceaccount/replicator-kubernetes-replicatorserviceaccount velero/velero dry-run: serviceaccount/velero | live: serviceaccount/velerovolumesnapshotclass.snapshot.storage.k8s.io -/longhorn-bak dry-run: volumesnapshotclass.snapshot.storage.k8s.io/longhorn-bak | live: volumesnapshotclass.snapshot.storage.k8s.io/longhorn-bak--- result ---patches succeeded: 42patches returning 422 / rejected: 0=== sweep end: 2026-09-07T20:44:08Z ===[VERIFIED: kubectl patch --type=json, dry-run then live, 2026-09-07T20:44:08Z] — 42 of 42 patches
succeeded and 0 returned a 422. Re-reading every object in the work set afterwards:
0 still carry a kustomize.toolkit.fluxcd.io/ label key, against a work set of 42 — and the
paired control, kubectl -n traefik get certificate -l kustomize.toolkit.fluxcd.io/name, still
returns 1 hit. That control is what separates a clean sweep from a selector that has stopped
matching anything; the one hit it returns is the ADR-exempt wildcard certificate, which independently
re-reads as 2 Flux labels and 1 kustomize-controller entry. The exemption is observable, not
merely asserted.
Both CA Secrets were untouched by this task and re-read with every key intact:
Secret/default/fzymgc-root-ca data["ca.crt"] = 1000 bytes; Secret/default/fzymgc-ica1-ca
data["ca.crt"] = 1076 and data["fullchain.crt"] = 2076.
What the sweep did and did not move
Section titled “What the sweep did and did not move”Measured live over the 42, with --show-managed-fields:
=== post-sweep ownership state: 2026-09-07T20:45 ===objects still carrying a Flux label key 0 / 42kustomize-controller entry COLLAPSED (labels were all it held) 2 / 42kustomize-controller entry SURVIVES (co-owns live fields) 40 / 42objects with a sole-Flux field path remaining 7 / 42Entry survival on 40 of 42 objects is the correct outcome, not a failure, and it is the reason
this phase’s Done-when is ownership-scoped. Those entries hold only paths a living manager also
owns — rules on the ClusterRoles, spec.addresses on the MetalLB pool, spec.entryPoints on the
IngressRoutes, the cert-manager status paths on all six Certificates. Removing them would delete live
configuration, which this plan’s prohibitions forbid. What matters is that on 33 of those 40 the
sole-Flux set is now empty: ArgoCD can reach every field on them.
Only two entries collapsed — ClusterRole/-/plugin-barman-cloud and
ServiceAccount/kube-system/replicator-kubernetes-replicator — the only objects in the work set whose
entire entry content was the two Flux labels. 04-01 measured three such objects across the whole
inventory; the third is ServiceAccount/vault/tf-runner, which is excluded here because 04-03 owns
it. Two, not three, is therefore the expected number for a work set that correctly excludes it.
The 7 objects retaining a sole-Flux path are exactly the D-07 class, and each is on its way to Task 4 rather than to another value patch:
| Object | Surviving sole-Flux path | Why no value patch can clear it |
|---|---|---|
Namespace/-/cnpg-system |
spec |
namespaceStrategy.PrepareForUpdate copies spec.finalizers from the old object unconditionally, and f:spec marks a key that always exists |
Namespace/-/postgres |
spec |
as above |
Namespace/-/traefik |
spec |
as above — and kustomize-controller is its sole manager, so the strip empties managedFields entirely |
Namespace/-/vault |
spec |
as above |
Namespace/-/velero |
spec |
as above |
Deployment/kube-system/replicator-kubernetes-replicator |
spec.strategy, spec.template.metadata.creationTimestamp |
SetDefaults_Deployment re-defaults strategy byte-identically inside the same request, so structured-merge-diff sees no change and ownership does not move; creationTimestamp is a phantom the API server never serves |
Deployment/cnpg-system/barman-cloud |
spec.template.metadata.creationTimestamp |
phantom only — see ### Phantom-path narrowing |
Adding the three label-less Namespaces (excluded from the sweep, spec-owned) and the two default/
CA Secrets (Task 2, data.*-owned) gives the 11-object predicted residual, plus the phantom
barman-cloud. That is the set Task 3 re-measures and Task 4 strips.
Per D-07, the re-defaulting is the disposition for the value on all of these — the value comes back and there is nothing to fight. What D-07 does not do is discharge an ownership-scoped Done-when, which is why the narrow entry strip exists.
Replicator propagation (D-02)
Section titled “Replicator propagation (D-02)”The four default/ CA objects — ConfigMap/fzymgc-root-ca, ConfigMap/fzymgc-ica1-ca,
Secret/fzymgc-root-ca, Secret/fzymgc-ica1-ca — are mittwald kubernetes-replicator sources
annotated replicator.v1.mittwald.de/replicate-to: "*". They were held back from the sweep above
because their patch is the D-02 experiment: does the replicator propagate a label removal, or
does it only copy labels on create?
This is the one place in the phase where a wrong patch is a cluster-wide outage rather than a cosmetic error. Two constraints were absolute:
- No
datapath was removed from any of the four, by any mechanism.configmaps.go:217-223deletes from each replica any key that was previously replicated and is no longer on the source, so adataremoval here would propagate an empty CA to every namespace before ArgoCD’s selfHeal restored the source. The Flux ownership of thosedata.*paths is released by the Task 4 entry strip instead, which touches no value. replicator.v1.mittwald.de/strip-labelswas not set, to any value. Its guard isif !ok && stripLabels != "true", whereokis the map-presence boolean — so setting it to"false"disables label copying exactly as thoroughly as"true"would, and would strip every label from every replica. Removing the two Flux labels from the sources is the narrow, correct action; the annotation’s absence is asserted on all four sources below.
=== D-02 propagation experiment: 2026-09-07T20:49:09Z ===--- BEFORE: label carriers cluster-wide, taken before the four source patches ---ConfigMaps carrying kustomize.toolkit.fluxcd.io/name: 108Secrets carrying kustomize.toolkit.fluxcd.io/name: 111TOTAL: 219--- BEFORE controls ---ConfigMaps carrying replicator.v1.mittwald.de/replicated-from-version: 106replica ConfigMap/argocd/fzymgc-root-ca data["ca.crt"] bytes: 749--- CORRECTION control: replica managedFields read both ways, before any patch ---kubectl -n argocd get cm fzymgc-root-ca -o json | jq ".metadata.managedFields | length" -> 0kubectl -n argocd get cm fzymgc-root-ca --show-managed-fields -o json | jq -c "[...]" -> [{"m":"replicator","op":"Update","t":"2025-08-25T18:47:19Z"}]--- PATCH: exactly four default/ sources, exactly two label paths each, no data path --- [{"op":"remove","path":"/metadata/labels/kustomize.toolkit.fluxcd.io~1name"},{"op":"remove","path":"/metadata/labels/kustomize.toolkit.fluxcd.io~1namespace"}]default/configmap fzymgc-root-ca dry-run: configmap/fzymgc-root-ca | live: configmap/fzymgc-root-cadefault/configmap fzymgc-ica1-ca dry-run: configmap/fzymgc-ica1-ca | live: configmap/fzymgc-ica1-cadefault/secret fzymgc-root-ca dry-run: secret/fzymgc-root-ca | live: secret/fzymgc-root-cadefault/secret fzymgc-ica1-ca dry-run: secret/fzymgc-ica1-ca | live: secret/fzymgc-ica1-ca--- AFTER (immediate) ---patched at: 2026-09-07T20:49:26ZConfigMaps carrying kustomize.toolkit.fluxcd.io/name: 92Secrets carrying kustomize.toolkit.fluxcd.io/name: 107TOTAL: 199--- AFTER (t+60s, stable) ---read at: 2026-09-07T20:50:52ZConfigMaps carrying kustomize.toolkit.fluxcd.io/name: 0Secrets carrying kustomize.toolkit.fluxcd.io/name: 3TOTAL: 3--- AFTER controls: a zero above is ALSO what a cluster-wide CA wipe looks like ---CONTROL 1 replicas still exist -> ConfigMaps with replicated-from-version: 106 (Secrets: 108)CONTROL 2 replica CA bytes -> ConfigMap/argocd/fzymgc-root-ca data["ca.crt"]: 749 bytes, replicated-keys: ca.crtCONTROL 2b other-namespace spot -> vault/fzymgc-root-ca -> ca.crt bytes: 749 flux label keys: 0 velero/fzymgc-root-ca -> ca.crt bytes: 749 flux label keys: 0 longhorn-system/fzymgc-root-ca -> ca.crt bytes: 749 flux label keys: 0 cnpg-system/fzymgc-root-ca -> ca.crt bytes: 749 flux label keys: 0--- SOURCES after: labels gone, data intact, no strip-labels annotation ---default/configmap fzymgc-root-ca flux labels: 0 strip-labels annotation: 0 data: ca.crt=749Bdefault/configmap fzymgc-ica1-ca flux labels: 0 strip-labels annotation: 0 data: ca.crt=806B fullchain.crt=1555Bdefault/secret fzymgc-root-ca flux labels: 0 strip-labels annotation: 0 data: ca.crt=1000Bdefault/secret fzymgc-ica1-ca flux labels: 0 strip-labels annotation: 0 data: ca.crt=1076B fullchain.crt=2076B=== D-02 experiment end: 2026-09-07T20:50:57Z ===--- residual label carriers, named: 3 Secrets, and what each one is ---vault/tfstate-default-authentik-tf managers=[HashiCorp] kustomize-controller entries: 0vault/tfstate-default-vault-tf managers=[HashiCorp] kustomize-controller entries: 0vault/vault-token managers=[kustomize-controller] kustomize-controller entries: 1in the frozen 55-object inventory: only Secret/vault/vault-token[VERIFIED: kubectl, 2026-09-07T20:50:57Z] — the replicator propagates a label removal, and it
does so within seconds.
| Reading | Value |
|---|---|
Before, ConfigMaps carrying kustomize.toolkit.fluxcd.io/name |
108 |
| Before, Secrets carrying the same | 111 |
| Before, total | 219 — four sources, 212 replicas, and the three pre-existing vault label carriers named below. Per kind: 108 ConfigMaps = 2 sources + 106 replicas; 111 Secrets = 2 sources + 106 replicas + 3 carriers. §RQ-1’s total of 219 is matched; its hedged “about 215 replicas” was three high, and the decomposition here is the measured one. |
| After, immediately (t+0s) | 199 — propagation already in flight |
| After, t+15s | 3 |
| After, t+30s / t+45s / t+60s | 3, 3, 3 — stable |
| Wait actually used | 60 s of polling at 15 s intervals. The 30 m --resync-period ceiling was never needed |
The mechanism explains the speed: the informer’s UpdateFunc runs the same replication path as a
create, the staleness short-circuit compares resourceVersion rather than content, and removing a
label bumps resourceVersion — so the guard does not fire and a full Update() carrying the shrunk
label map runs immediately. resourceCopy.Labels = labelsCopy is a whole-map replacement, not a
merge, so a key absent from the source is absent from the replica after the write.
212 replicas cleared from four edits. Every one of them was a derived copy holding no Flux field ownership at all — only a copied label — so this is a cosmetic cleanup of the estate rather than an ownership change. But it means the label-selector audit that started this whole workstream now returns near-zero cluster-wide, which is precisely why an ownership-scoped gate replaced it.
Both controls, because a zero is also what a CA wipe looks like
Section titled “Both controls, because a zero is also what a CA wipe looks like”A count of zero label carriers is exactly what a cluster-wide CA wipe would produce. Only these readings distinguish the two, and both were taken after the patches:
| Control | Before | After | Reads |
|---|---|---|---|
Replicas still exist — ConfigMaps carrying replicator.v1.mittwald.de/replicated-from-version |
106 | 106 | unchanged; Secrets likewise 108. The replicas were not deleted |
Replicas still carry CA bytes — ConfigMap/argocd/fzymgc-root-ca data["ca.crt"] |
749 B | 749 B | unchanged, replicated-keys: ca.crt still set |
| Same, spot-checked in four more namespaces | — | 749 B in vault, velero, longhorn-system, cnpg-system, each with 0 Flux label keys |
the label went, the CA stayed |
The four sources themselves re-read with every data key intact: ConfigMap/default/fzymgc-root-ca
ca.crt = 749 B; ConfigMap/default/fzymgc-ica1-ca ca.crt = 806 B and fullchain.crt = 1555 B;
Secret/default/fzymgc-root-ca ca.crt = 1000 B; Secret/default/fzymgc-ica1-ca ca.crt = 1076 B
and fullchain.crt = 2076 B. No replica was hand-patched, and none needed to be.
The three label carriers that remain, and why two of them are not in scope
Section titled “The three label carriers that remain, and why two of them are not in scope”The after count is 3, not 0, and all three are Secrets in vault:
| Object | Field managers | kustomize-controller entries |
Disposition |
|---|---|---|---|
Secret/vault/vault-token |
kustomize-controller |
1 | In the 55-object inventory. Excluded from this plan — 04-03 owns it (D-06) and clears it by deletion |
Secret/vault/tfstate-default-authentik-tf |
HashiCorp |
0 | Not in the inventory. Carries the two Flux labels but no Flux field ownership at all |
Secret/vault/tfstate-default-vault-tf |
HashiCorp |
0 | as above |
The two tfstate-* Secrets are the clearest illustration in this whole record of why VEST-01 is an
ownership gate and not a label gate. They carry kustomize.toolkit.fluxcd.io/name: apps — they
would show up in any label-selector audit — and their only field manager is HashiCorp, the
Terraform Vault state backend. There is nothing for ArgoCD to be blocked from, because
kustomize-controller owns no field on them. A label-based Done-when would have demanded two
pointless patches on objects a live manager writes; the ownership-based one correctly leaves them
alone. They are recorded here so a later reader who runs the old selector and sees 2 does not
mistake it for unfinished work.
Correction: a replica’s managedFields is not empty
Section titled “Correction: a replica’s managedFields is not empty”04-CONTEXT.md states that a spot-checked replica, ConfigMap/argocd/fzymgc-root-ca, has empty
managedFields. Measured both ways immediately before the four patches:
$ kubectl -n argocd get cm fzymgc-root-ca -o json | jq '.metadata.managedFields | length'0
$ kubectl -n argocd get cm fzymgc-root-ca --show-managed-fields -o json \ | jq -c '[.metadata.managedFields[]|{m:.manager,op:.operation,t:.time}]'[{"m":"replicator","op":"Update","t":"2025-08-25T18:47:19Z"}]The replica has one entry — manager: replicator, operation: Update. The “empty” reading is
the --show-managed-fields trap, firing for the third time in this repository and, this time,
inside the document written to warn about it. 04-CONTEXT.md states the trap in one paragraph and
then reports a figure taken without the flag in the next.
The correction strengthens D-02 rather than weakening it: the replica’s labels are solely owned by
the replicator manager through a plain Update, so the replicator’s next full Update() carrying
the shrunk label map removes them with no ownership conflict at all — which is exactly the behaviour
measured above.
The narrow entry strip
Section titled “The narrow entry strip”The value patch of the sweep removed the two Flux labels from every in-scope object. On most of the
inventory that is the whole job: the kustomize-controller entry either collapsed to nothing or
survived only because the dead manager co-owns live configuration with a living one, and a co-owned
entry is not a defect — releasing it would mean deleting configuration ArgoCD is actively managing.
On twelve objects the entry survived for a different reason: a field path the dead manager owns alone, which no value patch can release. Those twelve are what this step strips, and nothing else.
The operator’s ruling, stated so a later reader can see the step was narrow by design. On
2026-09-07 the operator ruled strip-residual-only plus phantom-narrowing. The
managedFields-entry removal that D-04 declined is re-admitted, scoped to the objects whose
sole-Flux residual provably survives the value patch — nothing more. The blanket second step
§RQ-5.4 recommends would strip 55 entries, including roughly 40 where kustomize-controller
co-owns live fields with argocd-controller or k3s; that breadth is exactly what D-04 was right to
decline, and re-admitting it in the narrow form is not a reversal of that judgement. The authorised
set is the file /tmp/flux-residual-comparable.txt, which the measurement gate produced only after
proving it equals the residual prediction frozen before the first patch — an exact set match in both
directions, symmetric difference empty.
The twelve, and why each was irreducible
Section titled “The twelve, and why each was irreducible”| # | Object | Sole-kustomize-controller path that survived the value patch |
Why no value patch can release it |
|---|---|---|---|
| 1 | Deployment/kube-system/replicator-kubernetes-replicator |
spec.strategy, spec.template.metadata.creationTimestamp |
§RQ-2b: spec.strategy is re-defaulted byte-identically by the API server on removal, so ownership does not move. …creationTimestamp is a phantom (§RQ-2a). |
| 2 | Namespace/-/cert-manager |
spec |
§RQ-2c: Namespace.spec is re-established by the API server (spec.finalizers: ["kubernetes"]); removing it changes nothing and releases nothing. |
| 3 | Namespace/-/cnpg-system |
spec |
as above |
| 4 | Namespace/-/external-secrets |
spec |
as above |
| 5 | Namespace/-/metallb |
spec |
as above |
| 6 | Namespace/-/postgres |
spec |
as above |
| 7 | Namespace/-/traefik |
spec |
as above |
| 8 | Namespace/-/vault |
spec |
as above |
| 9 | Namespace/-/velero |
spec |
as above |
| 10 | Secret/default/fzymgc-ica1-ca |
data.ca.crt, data.fullchain.crt |
§RQ-1: these are the CA bytes. The mittwald replicator deletes from every replica any key no longer present on the source, so a value patch here is a cluster-wide CA wipe across ~212 objects. Prohibited by name. |
| 11 | Secret/default/fzymgc-root-ca |
data.ca.crt |
as above |
| 12 | Deployment/cnpg-system/barman-cloud |
spec.template.metadata.creationTimestamp |
§RQ-2a: the path is not in the served object at all. A JSON-Patch remove on it is a 422. See the phantom-narrowing record below. |
Rows 1–11 are the eleven objects in /tmp/flux-residual-comparable.txt. Row 12 is
Deployment/cnpg-system/barman-cloud, excluded from the residual comparison by the
phantom-narrowing ruling and stripped here anyway, so the narrowing leaves nothing behind.
The patch shape, and the guard on every one of them
Section titled “The patch shape, and the guard on every one of them”Each removal is a two-operation JSON Patch — a test on /metadata/managedFields/<index>/manager
with value kustomize-controller, then a remove of /metadata/managedFields/<index>:
[{"op":"test","path":"/metadata/managedFields/0/manager","value":"kustomize-controller"}, {"op":"remove","path":"/metadata/managedFields/0"}]managedFields is an array and its ordering is not a stable property of the object. The index was
therefore resolved fresh from that same object immediately before its own patch, twice — once
before the server dry-run and again before the live write — never transcribed from an earlier read.
The test operation is what makes that discipline enforceable rather than merely intended: a stale
index fails the whole patch loudly with The request is invalid instead of silently removing a
different manager’s entry.
=== entry strip: 2026-09-07T22:39:47Z ===authorised set: /tmp/flux-residual-comparable.txt (11) + Deployment/cnpg-system/barman-cloud (phantom-narrowed) = 12
--- Deployment/kube-system/replicator-kubernetes-replicator fresh index=0 patch: [{"op":"test","path":"/metadata/managedFields/0/manager","value":"kustomize-controller"},{"op":"remove","path":"/metadata/managedFields/0"}] dry-run: deployment.apps/replicator-kubernetes-replicator (exit 0) live (fresh index=0): deployment.apps/replicator-kubernetes-replicator (exit 0) managers after: ["argocd-controller","k3s"]--- Namespace/-/cert-manager fresh index=0 patch: [{"op":"test","path":"/metadata/managedFields/0/manager","value":"kustomize-controller"},{"op":"remove","path":"/metadata/managedFields/0"}] dry-run: namespace/cert-manager (exit 0) live (fresh index=0): namespace/cert-manager (exit 0) managers after: []--- Namespace/-/cnpg-system fresh index=0 patch: [{"op":"test","path":"/metadata/managedFields/0/manager","value":"kustomize-controller"},{"op":"remove","path":"/metadata/managedFields/0"}] dry-run: namespace/cnpg-system (exit 0) live (fresh index=0): namespace/cnpg-system (exit 0) managers after: ["argocd-controller"]--- Namespace/-/external-secrets fresh index=0 patch: [{"op":"test","path":"/metadata/managedFields/0/manager","value":"kustomize-controller"},{"op":"remove","path":"/metadata/managedFields/0"}] dry-run: namespace/external-secrets (exit 0) live (fresh index=0): namespace/external-secrets (exit 0) managers after: []--- Namespace/-/metallb fresh index=0 patch: [{"op":"test","path":"/metadata/managedFields/0/manager","value":"kustomize-controller"},{"op":"remove","path":"/metadata/managedFields/0"}] dry-run: namespace/metallb (exit 0) live (fresh index=0): namespace/metallb (exit 0) managers after: []--- Namespace/-/postgres fresh index=0 patch: [{"op":"test","path":"/metadata/managedFields/0/manager","value":"kustomize-controller"},{"op":"remove","path":"/metadata/managedFields/0"}] dry-run: namespace/postgres (exit 0) live (fresh index=0): namespace/postgres (exit 0) managers after: ["argocd-controller"]--- Namespace/-/traefik fresh index=0 patch: [{"op":"test","path":"/metadata/managedFields/0/manager","value":"kustomize-controller"},{"op":"remove","path":"/metadata/managedFields/0"}] dry-run: namespace/traefik (exit 0) live (fresh index=0): namespace/traefik (exit 0) managers after: []--- Namespace/-/vault fresh index=0 patch: [{"op":"test","path":"/metadata/managedFields/0/manager","value":"kustomize-controller"},{"op":"remove","path":"/metadata/managedFields/0"}] dry-run: namespace/vault (exit 0) live (fresh index=0): namespace/vault (exit 0) managers after: ["argocd-controller","argocd-controller"]--- Namespace/-/velero fresh index=0 patch: [{"op":"test","path":"/metadata/managedFields/0/manager","value":"kustomize-controller"},{"op":"remove","path":"/metadata/managedFields/0"}] dry-run: namespace/velero (exit 0) live (fresh index=0): namespace/velero (exit 0) managers after: ["argocd-controller"]--- Secret/default/fzymgc-ica1-ca fresh index=0 patch: [{"op":"test","path":"/metadata/managedFields/0/manager","value":"kustomize-controller"},{"op":"remove","path":"/metadata/managedFields/0"}] dry-run: secret/fzymgc-ica1-ca (exit 0) live (fresh index=0): secret/fzymgc-ica1-ca (exit 0) managers after: ["argocd-controller"]--- Secret/default/fzymgc-root-ca fresh index=0 patch: [{"op":"test","path":"/metadata/managedFields/0/manager","value":"kustomize-controller"},{"op":"remove","path":"/metadata/managedFields/0"}] dry-run: secret/fzymgc-root-ca (exit 0) live (fresh index=0): secret/fzymgc-root-ca (exit 0) managers after: ["argocd-controller"]--- Deployment/cnpg-system/barman-cloud fresh index=0 patch: [{"op":"test","path":"/metadata/managedFields/0/manager","value":"kustomize-controller"},{"op":"remove","path":"/metadata/managedFields/0"}] dry-run: deployment.apps/barman-cloud (exit 0) live (fresh index=0): deployment.apps/barman-cloud (exit 0) managers after: ["argocd-controller","k3s"]
=== stripped ok: 12 failed: 0 ===The RED arm, run live once
Section titled “The RED arm, run live once”A guard nobody has seen fail is not a guard, and every assertion above rests on this one having
worked. The negative control was therefore run live, not as a dry-run, against an object that had
already been stripped — Deployment/kube-system/replicator-kubernetes-replicator, whose
managedFields index 0 now holds argocd-controller. Aimed there, the patch is precisely the
accident the guard exists to prevent: without the test operation it would have deleted a living
manager’s entry.
=== RED arm (negative control), live, 2026-09-07T22:40:06Z ===target: Deployment/kube-system/replicator-kubernetes-replicator, already stripped.managers before RED: ["argocd-controller","k3s"]The guard is aimed at index 0, which now holds argocd-controller, NOT kustomize-controller.If the test op were absent, this patch would delete a LIVING manager's entry.
$ kubectl -n kube-system patch deployment replicator-kubernetes-replicator --type=json -p '[{"op":"test","path":"/metadata/managedFields/0/manager","value":"kustomize-controller"},{"op":"remove","path":"/metadata/managedFields/0"}]'The request is invalid: the server rejected our request due to an error in our requestexit: 1
managers after RED: ["argocd-controller","k3s"]No value moved
Section titled “No value moved”The three dry-run classes of §RQ-5.4 predicted that removing the entry touches nothing. That was re-asserted live, against readings taken immediately before the strip:
before strip after stripSecret/default/fzymgc-root-ca data.ca.crt bytes 1000 1000Secret/default/fzymgc-ica1-ca data.ca.crt bytes 1076 1076Secret/default/fzymgc-ica1-ca data.fullchain.crt bytes 2076 2076Namespace/traefik spec.finalizers ["kubernetes"] ["kubernetes"]Namespace/cert-manager spec.finalizers ["kubernetes"] ["kubernetes"]Deployment/kube-system/replicator-kubernetes-replicator managers ["kustomize-controller","argocd-controller","k3s"] ["argocd-controller","k3s"]ConfigMap/argocd/fzymgc-root-ca data["ca.crt"] bytes 749 749Deployment/kube-system/replicator-kubernetes-replicator rollout generation 2 / observedGeneration 2 / readyReplicas 1 — unchanged, no rolloutThe replica reading is the one that matters most: ConfigMap/argocd/fzymgc-root-ca is one of the
~212 objects the mittwald replicator maintains from default/, and a CA wipe on the source would
show up there. It did not move a byte.
The three objects 04-03 owns — Secret/vault/vault-token, ServiceAccount/vault/tf-runner and
ClusterRoleBinding/-/vault-tf-runner — were all still alive after this strip. None of them was
patched, asserted on, or otherwise touched by this plan.
Phantom-path narrowing: Deployment/cnpg-system/barman-cloud
Section titled “Phantom-path narrowing: Deployment/cnpg-system/barman-cloud”Deployment/cnpg-system/barman-cloud was excluded from the residual comparison by the operator’s
phantom-narrowing ruling. The ruling rests on a measurement, recorded here so the judgement is
auditable rather than silent. Captured before the object was stripped, while the phantom
ownership was still readable:
$ kubectl --context fzymgc-house -n cnpg-system patch deployment barman-cloud --type=json \ -p '[{"op":"remove","path":"/spec/template/metadata/creationTimestamp"}]' --dry-run=serverThe request is invalid: the server rejected our request due to an error in our request(exit 1)
$ kubectl --context fzymgc-house -n cnpg-system get deploy barman-cloud -o json \ | jq -c '.spec.template.metadata | keys'["labels"]
$ kubectl --context fzymgc-house -n cnpg-system get deploy barman-cloud --show-managed-fields -o json \ | jq -c '[.metadata.managedFields[] | select(.manager=="kustomize-controller") | .fieldsV1["f:spec"]["f:template"]["f:metadata"]]'[{"f:creationTimestamp":{},"f:labels":{"f:app":{}}}]The three readings together are the whole argument. Flux’s entry claims
f:spec.f:template.f:metadata.f:creationTimestamp; the served object’s spec.template.metadata
carries a labels key and nothing else; and a JSON-Patch remove on the claimed path is rejected
because there is no such member to remove. A path the API server neither stores nor serves is not
a field the object has. Counting it as a live sole-Flux residual would have made the residual set
disagree with the frozen prediction over a field that does not exist — which is what the narrowing
excludes, and the only thing it excludes.
The narrowing leaves nothing behind: barman-cloud was stripped anyway as the twelfth object above,
and afterwards carries no kustomize-controller entry at all. The exclusion is from the comparison,
not from the strip.
[VERIFIED: kubectl, 2026-09-07T22:41:14Z] — 12 of 12 authorised objects stripped, 0 failures.
Every patch carried a test index guard on /metadata/managedFields/<index>/manager with the index
resolved fresh immediately before both the dry-run and the live write; the guard’s RED arm was run
live once and rejected with The request is invalid, leaving
["argocd-controller","k3s"] intact on the object it was aimed at. The ADR hl-0gol-exempt
Certificate/traefik/wildcard-fzymgc-house-tls still carries its kustomize-controller entry and
both Flux labels. No value moved on any of the twelve.
The closing gate over a full cluster re-capture, the residual table, and the phase findings are
04-06’s and are deliberately not written here.
Closing gate, verbatim
Section titled “Closing gate, verbatim”The measurement VEST-01 closes on. Taken against a fresh full-cluster re-capture — not against
04-02’s post-strip dumps — because the three objects 04-03 owns were deleted between the two, and a
gate reusing a stale capture would report a residual of four for a cluster that holds one. The
re-capture enumerated the same 204 namespaced and 57 cluster-scoped kinds, both halves with
--show-managed-fields, and yielded 12,546 objects, 12,184 of them carrying a non-empty
managedFields array. That corpus size is printed before any count below is believed: a capture
taken without the flag serves zero managedFields, and every reading downstream of it is vacuously
clean.
=== closing gate: 2026-09-08T00:51:46Z ===corpus: 12546 objects captured (204 namespaced kinds -A, 57 cluster-scoped), 12184 with managedFields
--- (a) objects with a sole kustomize-controller field ---1Certificate/traefik/wildcard-fzymgc-house-tls metadata.labels.kustomize.toolkit.fluxcd.io/name,metadata.labels.kustomize.toolkit.fluxcd.io/namespace,spec.dnsNames,spec.issuerRef.kind,spec.issuerRef.name,spec.secretName,spec.usages
--- (a-inverted) CONTROL: same program, --arg m argocd-controller ---801
--- objects still carrying ANY kustomize-controller managedFields entry ---38
--- value preservation, re-read live at gate time ---Secret/default/fzymgc-root-ca data keys: ca.crt ca.crt bytes: 1000Secret/default/fzymgc-ica1-ca data keys: ca.crt,fullchain.crt ca.crt: 1076 fullchain.crt: 2076Namespace/traefik spec.finalizers: ["kubernetes"]Deployment/kube-system/replicator-kubernetes-replicator managers: ["argocd-controller","k3s"] (2)[VERIFIED: kubectl + jq, 2026-09-08T00:51:46Z] — the set of objects holding a field path whose
sole owner is kustomize-controller is exactly one object,
Certificate/traefik/wildcard-fzymgc-house-tls, and that object is required to be present rather
than absent. ADR hl-0gol exempts it deliberately, so an empty result here would have meant the
cluster’s default TLS certificate had been patched — a worse outcome wearing a better number. The
inverted control ran the same program file, /tmp/sole-owner.jq, with only --arg m changed, and
returned 801 objects for argocd-controller. The two readings are unequal and the control is
large, so the 1 is a property of the cluster rather than of a traversal that returns empty for every
manager. Stderr was suppressed on neither invocation. The closing property therefore holds at 54 of
55: of the 55 objects in the frozen pre-patch inventory, 54 now have no field whose sole owner is
kustomize-controller, and the 55th is the one ADR-locked exemption. Nothing was written to the
cluster by this gate; every interaction was a read.
Three of the 55 are absent from the live cluster entirely rather than merely cleared — 04-03 deleted
them. They count among the 54 because the property no live object has a field whose sole owner is
kustomize-controller is satisfied vacuously by an object that no longer exists, and because
deletion was the disposition the operator ruled for that unit. Their rows are in ## Per-object dispositions; the deletion transcripts are in
docs/operations/evidence/2026-09-07-vault-tf-runner-decommission.md.
The 38 objects still carrying a kustomize-controller entry are the co-owned ones the narrow ruling
deliberately preserved, plus the exempt certificate. Every one of them holds only paths a live manager
also owns, so ArgoCD can reconcile them; that is the difference between an entry that survives and a
field that is stuck. The count fell from 41 to 38 for exactly the three deletions and for no other
reason.
The sole-ownership program, embedded
Section titled “The sole-ownership program, embedded”Every count in this file comes from one program, and it is reproduced here in full because D-05
declined a committed script: Flux is uninstalled, kustomize-controller cannot re-acquire ownership,
and a tool under scripts/ would have no second customer to serve. What has to survive is the
method, not an executable.
The trap first, because it invalidates everything below it: kubectl get -o json strips
managedFields unless --show-managed-fields is passed. A capture taken without the flag serves an
empty managedFields array on every object, so the program below returns zero for every manager and
the cluster reads perfectly clean. Assert the corpus size — the count of objects carrying a non-empty
managedFields array — before believing any ownership number derived from a dump. That is why the
closing gate above prints 12184 next to 12546 rather than jumping straight to the 1.
The program takes its manager from --arg m and hardcodes none. That parameterisation is
load-bearing, not stylistic: it is the only thing that makes the inverted run a genuine control. With
the manager baked in as a literal, the “control” silently re-runs the target query and its non-zero
result proves nothing about the target. Because $m is a parameter, the target run and the control
run differ in exactly one argument to one file, so a traversal that is broken for all managers
returns zero on both sides and is distinguishable from a cluster that is actually clean. Suppress
stderr on neither invocation; a silenced jq compile error defaults to a count of zero, which reads
exactly like success.
# Leaf-path sole-ownership over managedFields. Reproduce verbatim; the manager is a parameter.cat > /tmp/sole-owner.jq <<'JQ'# Leaf-path sole-ownership set-difference over managedFields.## Invocation: jq -s -r -f /tmp/sole-owner.jq --arg m <manager> dump1.json dump2.json## The target manager is read from $m and is NOT hardcoded. That is deliberate and# load-bearing: it is what makes an inverted-predicate control possible. The target# invocation passes the dead manager, the control invocation passes a different one,# and the two runs differ in exactly that one parameter. A program with the manager# baked in cannot be controlled, because the "control" silently re-runs the target# query and its non-zero result proves nothing.## Emits one line per object that has at least one field path owned by $m and by no# other manager: Kind/namespace-or-dash/name<TAB>path,path,...
def leaves($p): to_entries[] as $e | ($p + [$e.key]) as $q | if ($e.value | type) == "object" and ($e.value | length) > 0 then ($e.value | leaves($q)) else $q end;
# All leaf field paths owned by $m ($want true) or by every manager except $m ($want false).def owned($obj; $want): [ ($obj.metadata.managedFields // [])[] | select(if $want then .manager == $m else .manager != $m end) | (.fieldsV1 // {}) | leaves([]) | map(sub("^f:"; "")) | join(".") ] | unique;
.[].items[]| . as $o| (owned($o; true) - owned($o; false)) as $sole| select($sole | length > 0)| "\($o.kind)/\($o.metadata.namespace // "-")/\($o.metadata.name)\t\($sole | join(","))"JQ
# Capture BOTH halves of the estate, each with --show-managed-fields. The namespaced half# alone misses every cluster-scoped object and under-reports this inventory by 20.K="kubectl --context fzymgc-house"NS=$($K api-resources --verbs=list --namespaced=true -o name | tr '\n' ',' | sed 's/,$//')CL=$($K api-resources --verbs=list --namespaced=false -o name | tr '\n' ',' | sed 's/,$//')$K get $NS -A --show-managed-fields -o json > /tmp/flux-ns.json$K get $CL --show-managed-fields -o json > /tmp/flux-cl.json
# Corpus floor FIRST: a zero here means the flag was dropped, not that the cluster is clean.jq -s '[.[].items[] | select((.metadata.managedFields // []) | length > 0)] | length' \ /tmp/flux-ns.json /tmp/flux-cl.json
# Target, then the control — same file, only --arg m changes. Never redirect stderr.jq -s -r -f /tmp/sole-owner.jq --arg m kustomize-controller /tmp/flux-ns.json /tmp/flux-cl.jsonjq -s -r -f /tmp/sole-owner.jq --arg m argocd-controller /tmp/flux-ns.json /tmp/flux-cl.jsonResidual
Section titled “Residual”One object, by ruling rather than by oversight.
| Object | Sole-kustomize-controller paths |
Why it remains |
|---|---|---|
Certificate/traefik/wildcard-fzymgc-house-tls |
metadata.labels.kustomize.toolkit.fluxcd.io/name, metadata.labels.kustomize.toolkit.fluxcd.io/namespace, spec.dnsNames, spec.issuerRef.kind, spec.issuerRef.name, spec.secretName, spec.usages |
ADR hl-0gol exemption (D-01). The cluster’s default TLS certificate and the highest-blast-radius object in the estate. Never patched, never stripped, deliberately left whole |
The closing property is 54 of 55. Of the 55 objects in the frozen pre-patch inventory, 54 now
have no field whose sole owner is kustomize-controller — 51 cleared in place by the value patch and
the narrow entry strip, three removed outright by 04-03 — and the 55th is the row above. The
exemption is asserted present: a residual of zero would not have been a better result, it would
have meant the wildcard certificate was patched in violation of hl-0gol. This is the shape of gate
that a count alone cannot express, which is why the closing assertion is a set relation in both
directions.
Thirty-eight objects still carry a kustomize-controller managedFields entry. That is not a
residual. Every one of those entries holds only paths a live manager — argocd-controller or k3s —
also owns, and a co-owned path is a path ArgoCD can reconcile. The property VEST-01 is accountable
for is sole ownership, not entry presence; releasing those entries would have meant deleting live
configuration another controller is actively managing, which is precisely the breadth D-04 was right
to decline.
Findings
Section titled “Findings”A green ArgoCD sync is not evidence of ownership. This whole defect class was invisible to
Synced, and had to be: you may only delete a field you own, so ArgoCD cannot remove a field it never
declared, and it reports Synced the entire time it is failing to converge that field. The Velero
includedResources allow-list at argocd/app-configs/velero/backup-schedule.yaml:74-90 sat true in
git and false in the cluster for months on exactly this mechanism. Sync status answers “does the
declared state match?”; it does not answer “who may write this field?”, and only the second question
detects a dead manager.
SSA ownership is per field path, never per object. Every count here walks each managedFields
entry’s fieldsV1 tree to its leaves and set-differences one manager’s leaf paths against the union
of all others’. Object-level counting materially over-reports the problem: 55 objects carried a
kustomize-controller entry and 52 held at least one solely-owned path, but only 15 held anything
solely owned beyond the two Flux labels, and only 11 of those survived this phase’s ruled
exclusions. So the object-level headline over-states the actionable field-level residual by roughly
5× against the 11 and roughly 3.5× against the 15; 04-01-SUMMARY.md records the same relation with
the shorthand “roughly six-fold”. The exact ratio matters less than the direction, which is always the
same: counting objects makes the problem look bigger and more uniform than it is, and hides that most
of those entries hold nothing but two labels.
The value patch alone does not reach the closing property; the narrow entry strip is what closes
it. Measured against the post-sweep, pre-strip capture: after the two Flux labels were removed from
every in-scope object, 16 of the 55 still held a solely-owned field path — 39 of 55 achieved.
Counterfactually crediting ServiceAccount/vault/tf-runner, which was excluded from the sweep under
D-06 and whose only sole paths were those two labels, gives 40 of 55; further discounting the phantom
path on Deployment/cnpg-system/barman-cloud gives 41 of 55. The narrow entry strip over 12 objects
took it to 51, and 04-03’s three deletions to 54 of 55. Three independent mechanics are why the
value patch cannot get there on its own: namespaceStrategy.PrepareForUpdate copies Namespace.spec
forward unconditionally, spec.strategy is re-defaulted byte-identically inside the same request, and
the default/ CA data.* paths cannot be removed at all without a cluster-wide replicator fan-out
that would delete the keys from ~212 replicas.
Flux is uninstalled and cannot re-acquire ownership, which is why D-05 declined a re-runnable
tool. kustomize-controller does not run in this cluster; nothing re-applies these labels, and
nothing re-claims these paths. The audit that found this residue has no second customer, so what is
worth keeping is the disposition reached for each object and the one piece of method knowledge that
has repeatedly been lost — the --show-managed-fields trap. That is a dated record, not standing
operational prose, which is why it lives in evidence/ and why the program is embedded above rather
than committed under scripts/.
Correction: §RQ-5.4’s summary line disagrees with its own document’s measurements
Section titled “Correction: §RQ-5.4’s summary line disagrees with its own document’s measurements”04-RESEARCH.md §RQ-5.4 closes with the sentence that the restated Done-when “is achievable on 54 of
55 objects (the 55th being the ADR-exempt wildcard certificate)” — written as the consequence of
declining the entry strip, i.e. as what the value patch alone delivers. That is false, and the
same document measures it false. §RQ-2 records that namespaceStrategy.PrepareForUpdate copies
Namespace.spec forward unconditionally and that spec.strategy is re-defaulted byte-identically
inside the same request, so neither releases on a value patch; §RQ-5.5’s own per-object table marks
the CA Secrets’ data.* paths unreachable without a replicator fan-out. Measured at the cluster after
the sweep, the value patch alone reached 39 of 55 — 40 if ServiceAccount/vault/tf-runner is
credited counterfactually, 41 if the phantom path on barman-cloud is discounted. 54 of 55 is
reachable, but only with the narrow entry strip, which is exactly the step the sentence was
written to do without.
04-06-PLAN.md inherited the error and restated the value-patch-alone figure as 42 of 55. That
number is also not an achievement count: 42 is the size of the Task 1 sweep work set — the objects
04-02 patched — which is a different quantity that happens to sit in the same range. Both figures
are recorded here so a later reader comparing the plan against this file sees the discrepancy rather
than silently picking one.
This is the fifth falsified premise in this phase, and the second found inside a research artifact rather than a source todo. The durable lesson is where it was found: not in a measurement, but in a long document’s own summary line. §RQ-2 and §RQ-5.5 are correct; §RQ-5.4’s closing sentence drifted from them while summarising them, and it drifted in the flattering direction — reporting a harder property as already achieved. Read the section, not the summary of the section, whenever the summary is what a decision is about to rest on.
Correction: hl-e4ny remains a dead cross-reference at phase close
Section titled “Correction: hl-e4ny remains a dead cross-reference at phase close”Restated at close because this file now outlives the phase that produced it. Beads are retired in
this repository (D-03), so neither this record nor any downstream agent should treat hl-e4ny as a
live work handoff — there is nothing on the other end of it. ADR hl-0gol’s substantive ruling, which
is what exempts the wildcard certificate in ## Residual above, stands on its own merits entirely
independently of the bead identifier it happens to name.