Turing Pi boards
The two Turing Pi 2 boards, tpi-alpha and tpi-beta, each hold four k3s nodes and one BMC. This
page covers the uplink of a board, the legs of its nodes and the address of its BMC. It also
holds the steps that move a board between networks.
The uplink and the legs
Section titled “The uplink and the legs”Each board carries an onboard switch that joins the BMC and the four nodes to one uplink. The switch has no VLAN filtering: it passes a tagged frame untouched, and it puts every untagged frame on the native network of the uplink. So the native network of the uplink decides where the BMC lands, and each node picks its own network by tagging.
| Item | Value |
|---|---|
tpi-alpha uplink |
switch-office port 12, profile TpiUplink |
tpi-beta uplink |
switch-office port 11, profile TpiUplink |
TpiUplink |
Core native, Main and Lab tagged, every other network excluded (tf/unifi/port-profiles.tf) |
| Node legs | end0.3000 for Main and end0.1000 for Lab, rendered by tp2-bootstrap-node. end0 holds no address |
| BMC | Untagged, so on Core: tpi-alpha-bmc at 192.168.40.40 and tpi-beta-bmc at 192.168.40.41, from their dns-control entries |
The group_vars of each board set tp2_node_main_tagged: true. With the default false, the
template renders Main on end0 untagged, which is the shape for an uplink that is native Main.
The two values change together with the port, in the same pull request.
A port reaches TpiUplink from another profile, never from its own network keys. The provider
does not clear the profile of a port that moves to its own keys, so the controller keeps the old
profile and ignores the keys (UniFi).
The BMC API
Section titled “The BMC API”The BMC presents a self-signed certificate. Log in with a JSON POST and pin the public key:
host=tpi-beta-bmc.fzymgc.housepin=$(echo | openssl s_client -connect "$host:443" 2>/dev/null | openssl x509 -pubkey -noout | openssl pkey -pubin -outform der | openssl dgst -sha256 -binary | openssl enc -base64)export BMC_PASS=$(vault kv get -field=password secret/fzymgc-house/infrastructure/bmc/tpi-beta)tok=$(curl -sS --insecure --pinnedpubkey "sha256//$pin" -H 'Content-Type: application/json' -d "{\"username\":\"root\",\"password\":\"$BMC_PASS\"}" "https://$host/api/bmc/authenticate" | jq -r .id)curl -sS --insecure --pinnedpubkey "sha256//$pin" -H "Authorization: Bearer $tok" "https://$host/api/bmc?opt=get&type=about" | jq .The alpha password is at bmc/tpi-alpha under the same mount. Both boards run an upstream
development build of the BMC firmware, which type=about reports.
Move a board to a native Core uplink
Section titled “Move a board to a native Core uplink”The order matters. Cilium’s VLAN filter on end0 drops a tagged frame for a VLAN device that
Cilium does not own, and a restart of the agent never adopts an unlisted device.
-
Make sure that
cilium_devicesof theciliumrole listsend0.3000, and converge the role (k3s-playbook.yml --tags cilium). Cilium selects a listed device when it appears, so the list can lead the netplan change. -
Create or change the dns-control entry of the BMC with its Core address.
-
In one pull request, set
tp2_node_main_tagged: truein the group_vars of the board and give the uplink port the profileTpiUplink. Merge it and confirm themain-cluster-unifirun. At that moment the untagged Main leg of each node stops, and the four nodes goNotReady. -
For each node, through its Lab address, render and apply the tagged Main leg:
Terminal window cd ansibleuv run ansible-playbook -i inventory/hosts.yml bootstrap-nodes-playbook.yml --tags network --limit tpi-beta-4 -e ansible_host=192.168.200.154The node is
Readyagain within about a minute. Take the next node when it is. -
Read the port back from the controller. It holds the profile id of
TpiUplink, not ofAllMainDefault.
The Lab leg is the management path during the move, because it is tagged and does not change. If
a node is lost, the BMC serial console (tpi uart -n <slot>) is the recovery path.
The BMC keeps its old lease
Section titled “The BMC keeps its old lease”After the flip, the BMC still holds its Main lease, which is now on the wrong network, until its DHCP client renews. That is up to six hours. To force it, put a throwaway container on Core with a temporary address inside the old subnet. The container then shares a segment with the BMC. Send the DHCP client of the BMC the renew signal. Kea refuses the old lease on the Core leg, so the client falls back to discovery and takes the Core reservation.
T=.claude/skills/nas-throwaway-lxc/scripts/throwaway_lxc.pyuv run $T up bmcfix --existing-vlan 1 --packages sshpassuv run $T exec bmcfix -- ip addr add 192.168.23.230/22 dev eth0export BMC_PASS=$(vault kv get -field=password secret/fzymgc-house/infrastructure/bmc/tpi-beta)printf '%s\n' "$BMC_PASS" | uv run $T exec bmcfix --stdin -- sshpass -d 0 ssh -o StrictHostKeyChecking=no root@192.168.23.229 'kill -USR1 $(pidof udhcpc)'uv run $T downThe BMC answers on its Core address within seconds. Do not bounce the uplink port for this. The bridge port of the BMC on the onboard switch never goes down, so the client sees no link change.
tpi-alpha-bmc refuses password login over SSH and accepts the operator’s key only, so the
password path above fails on it with Permission denied (publickey,keyboard-interactive). For
that board, give the container an SSH server and the operator’s public keys, and reach the BMC
from a workstation with agent forwarding. The temporary address is a /32 with one host route
to the BMC, because a /22 on the container pulls every Main-subnet reply onto the Core segment,
where the workstation is not.
T=.claude/skills/nas-throwaway-lxc/scripts/throwaway_lxc.pyuv run $T up jump --existing-vlan 1 --packages openssh-serverssh-add -L | uv run $T exec jump --stdin -- sh -c 'mkdir -p /root/.ssh && cat > /root/.ssh/authorized_keys && chmod 700 /root/.ssh && chmod 600 /root/.ssh/authorized_keys'uv run $T exec jump -- sh -c 'ip addr add 192.168.23.230/32 dev eth0; ip route add 192.168.22.140/32 dev eth0 src 192.168.23.230'ssh -A root@<the Core address of the container> # then: ssh root@192.168.22.140, then: kill -USR1 $(pidof udhcpc)uv run $T downA rebuilt node
Section titled “A rebuilt node”A rebuilt node boots with the untagged DHCP configuration of its image, so it takes a Core lease
until tp2-bootstrap-node renders the tagged configuration. Core has Kea, and the role connects
by the address you pass it, so pass that Core address as ansible_host for the first run.