DNS Gate Suite Audit — test_dns_common.py

Audit of tools/dns-role-gates/tests/test_dns_common.py (107 test functions, 142 collected, 5,381 lines) against the tiers that can replace it.

The tiers

Tier	Mechanism	State
Static quality	ansible-lint, profile: production	enforcing (a1dce9aa8)
Config grammar	daemon validate:	wired for all five daemons
Invariants on the real system	dns-common/tasks/verify.yml	4 criteria: C2-shape, NFR-03, T-02-17, PDNS-09
Those three, pre-ship	molecule	proven (65259a6cb)
Our own shell scripts	—	no tier; see category D

Verdicts

A — META: tests of the harness, or of verify.yml’s shape. Delete. (28)

These assert facts about test scaffolding and YAML formatting, not about the estate. Nothing in the running system changes if they go.

the_named_role_files_are_reachable_through_a_floored_enumeration, the_criterion_block_is_exactly_one_block_with_exactly_one_always, the_three_accumulators_are_initialised_outside_the_criterion_block, the_stopping_point_tracker_is_never_cacheable, the_forwarding_capture_carries_the_canonical_triple, the_nfr_03_sequence_appears_in_the_canonical_order, the_value_assert_compares_the_reading_against_the_zero_literal, every_assert_in_the_criterion_block_is_quiet, the_enter_facts_and_pass_key_appends_each_sit_on_one_physical_line, the_roll_up_criterion_order_equals_this_registry_as_a_set, the_fixture_play_reaches_the_role_through_include_role_with_tasks_from, the_fixture_play_declares_no_role_value_of_its_own, the_fixture_play_runs_locally_and_gathers_nothing, the_fixture_play_reads_the_real_inventory_files_rather_than_restating_them, the_fixture_play_carries_both_mandatory_headers, the_variable_file_corpus_is_at_least_the_size_measured_on_this_tree, the_declaration_scanner_actually_detects_the_override, both_verify_overrides_are_distinct_names, the_verify_block_has_exactly_one_evaluation_path_for_the_forwarding_value, the_template_corpus_is_exactly_the_six_this_plan_authors, the_verify_block_asserts_the_ruleset_is_loaded_not_merely_that_it_is_enabled, the_on_node_leg_count_assertion_agrees_with_the_offline_constant, the_offline_leg_count_agrees_with_the_transcribed_leg_addresses, the_role_actually_includes_the_diagnostics_task_file, the_firewall_render_is_preceded_by_the_peer_symmetry_assert, the_asymmetric_leg_set_actually_breaks_the_render, the_apply_script_template_exists, the_resolved_comment_records_a_posture_not_a_command

the_enter_facts_and_pass_key_appends_each_sit_on_one_physical_line is a line-formatting assertion shipped as an estate gate. It is the clearest single example of the drift.

B — Not ours. Delete. (4)

Test	Owner
every_template_renders_identically_with_and_without_trim_blocks	Jinja2
the_rendered_ruleset_parses_under_a_real_nft	nftables — and molecule now runs nft --check properly
the_sysctl_template_carries_all_three_mandatory_headers	lefthook spdx-header-ansible
every_template_carries_all_three_mandatory_headers	lefthook spdx-header-ansible

C — Already covered, and covered better. Delete. (~8)

Corrected after reading the assertion. T-02-17’s that: clause is only:

- (dns_common_nftables_enabled | bool) == (dns_common_nftables_table_present | bool)
- not (dns_common_nftables_table_present | bool) or 'policy drop' in dns_common_nftables_table_value

That is presence and default-deny, and nothing about rule content. The first draft of this audit put ~40 rendered-ruleset tests here on the assumption that “the kernel’s ruleset agrees with the guard” subsumed them. It does not. Those tests moved to D1.

What T-02-17 and C2-shape genuinely do subsume:

• the_nftables_input_chain_defaults_to_drop — T-02-17’s second clause, against the kernel
• the_inventory_itself_carries_the_asserted_number_of_legs — C2-shape
• the_network_file_count_equals_the_vlan_entry_count_on_both_hosts — C2-shape
• the_netdev_count_equals_the_tagged_entry_count_on_both_hosts — C2-shape
• the_parent_vlan_references_equal_the_netdev_device_names — C2-shape
• both_resolvers_declare_the_same_vlan_leg_set — converge.yml peer assert, via hostvars
• every_resolver_declares_a_peer_that_exists — same
• the_role_installs_an_include_that_makes_the_rendered_ruleset_loadable — T-02-17’s fail_msg walks the operator through exactly this; the include’s effect is observed, not asserted

D — Ours, uncovered. Port before deleting. (~25)

D1 — rule content and security scoping. (~45) T-02-17 does not reach any of these. Port to verify.yml as asserts over nft list ruleset output — the kernel’s own text, which is not reimplementing a grammar, it is reading the parser’s output.

The policy is ours (we decide ssh is never accepted unsourced). Only the grammar is nftables’. Asserting our policy against the loaded ruleset keeps the first and drops the second.

• ssh_is_never_accepted_unsourced
• ssh_is_never_permitted_on_a_tagged_leg
• the_election_protocol_is_never_accepted_unscoped
• no_tailnet_dns_accept_is_ever_written_unscoped
• the_dhcpv6_client_port_is_scoped_to_link_local_sources
• echo_request_is_never_in_the_unscoped_type_sets
• neither_ping_rule_carries_a_source_scope

D2 — config values NFR-03 does not reach. NFR-03 checks ip_forward only; the drop-in sets four keys.

• the_sysctl_dropin_sets_exactly_the_four_multi_homing_keys
• each_sysctl_key_carries_the_value_the_segmentation_model_requires
• the_sysctl_dropin_filename_sorts_after_debians_own_99_sysctl_conf
• the_resolved_dropin_gives_up_the_loopback_dns_port
• the_role_never_writes_an_etc_hosts_entry
• the_only_resolv_conf_write_is_the_gated_d16_render

D3 — cross-role coupling. No single role’s verify.yml can see these.

• the_kea_ha_port_agrees_across_the_peer_urls_and_the_nftables_accept
• the_tailnet_range_contains_both_recorded_node_addresses
• no_verify_override_is_declared_in_any_variable_file_under_ansible

D4 — our own shell scripts. Not config; different rule applies. (13)

dns-common-network-apply.sh.j2, dns-common-revert.sh.j2, dns-common-ra-watch.sh.j2 are application code we wrote. Per the standing rule, app code is test-driven and that is the target. These tests stay — but they currently assert by parsing the Jinja template text, which is the same weak instrument this audit is retiring everywhere else.

• the_handover_is_one_on_node_transaction
• the_apply_action_set_equals_the_manifest
• the_apply_and_revert_action_sets_are_inverses
• the_revert_only_action_set_equals_its_manifest
• the_revert_stops_a_triggering_socket_before_its_service
• a_failed_socket_stand_down_does_not_attempt_the_dependent_service_stop
• the_revert_masks_the_socket_activated_units_rather_than_only_disabling_them
• the_apply_lifts_every_mask_the_revert_applies_before_it_severs_the_connection
• the_network_manager_is_masked_not_only_disabled
• the_resolved_posture_is_inert
• the_enable_half_survives_in_network_yml
• every_rsync_opt_survives_the_modules_argv_round_trip
• the_composed_remote_command_is_sudo_rsync_and_never_sudo_alone

Recommended home: bats, executing the rendered script, or molecule running the apply/revert cycle against the container. Both observe behaviour instead of re-reading the template.

Tally

Verdict	Count
A — meta, delete	28
B — not ours, delete	4
C — covered better elsewhere, delete	~8
D1–D3 — port to verify.yml, then delete	~45
D4 — keep, move to bats/molecule	13

Roughly 40 of 107 delete outright, ~45 port to verify.yml, 13 relocate to bats.

The assertions largely survive — they move to a tier that reads the running kernel instead of a rendered string. What goes is the 28 meta-tests, the 4 not-ours, and the ~8 genuinely duplicated. What shrinks dramatically is pytest, not coverage.

Open questions

1. Does T-02-17’s guard cover the full rule inventory? RESOLVED: no. Presence and policy drop only. ~40 tests moved from C to D1. Checking this rather than assuming it is the difference between porting the security assertions and silently dropping them.
2. Does molecule verify need a separate invocation? main.yml is converge-then-verify, so converge already runs the verify tier. Confirm molecule verify is not therefore redundant.
3. The other four roles. kea’s verify.yml has 10 asserts and is far richer than dns-common’s 4. Per-role coverage varies; this audit does not transfer by analogy.