Node lifecycle
How a folly node goes from a NetBox device to an attested Talos node with a SPIRE identity, and back out, with the system that owns each step.
A folly node is an entry in clusters/folly/inventory/nodes.yaml, which terraform/netbox/folly applies into NetBox as a kubernetes-node device. fleetd on spore attests it and drives each run. Identity lists the statuses and who writes each, and Hardware inventory the nodes file.
At a glance#
- The owner adds the node to the nodes file in a pull request. Atlantis creates a
planneddevice with no MAC address. - The owner connects the machine to its node port. Firmware PXE loads signed iPXE from spore, then signed HookOS.
- HookOS proves its TPM identity and boot state to fleetd. fleetd records the MAC against the device's serial.
- fleetd checks every approved EK, then issues the LDevID and LAK certificates. Without an approved binding, the machine waits.
- The signed confirmation triggers admission. fleetd checks the reported switch port, starts a Tinkerbell
provisionWorkflow and renders the Talos configuration. - fleetd returns an install code wrapped to the TPM. The Workflow wipes the system disk, installs Talos and stores the code in META.
- Talos boots from disk and fetches its configuration from
/talos/configwith the UUID and code. The configuration supplies its static address. - After the complete response, fleetd deletes the configuration and sets the device
active. - fleetd approves the worker's initial kubelet certificate only when its per-node bootstrap token and requested node name match. Kubernetes readiness remains separate.
- fleetd lists the
active, attested node inspire-nodes.json, and spore'sspire-server-fleet-entriesregisters the node's SPIRE entry,spiffe://lolwtf.ca/node/<name>, from it. The SPIRE agent then attests by LDevID and serves SVIDs, since each workload entry is parented to that node entry. Identity has the entries.
An unknown serial also waits. A machine must prove every EK that its inventory entry approves before fleetd issues an identity.
Actions after installation#
The owner requests an action through NetBox. Rufio uses the node's AMT address to boot it into HookOS when needed.
| Request | Result |
|---|---|
desired_action: provision | Reinstalls an eligible node. HookOS attestation obtains the install code. |
desired_action: attest | Boots HookOS and attests the node. |
desired_action: wipe | Erases the disk. |
Status decommissioning | Starts retirement: wipes and erases the entire disk, powers off the node and sets it offline. |
Retirement also cordons the Kubernetes node, evicts its pods and deletes its cluster records before fleetd marks it offline.
A provision for an unsupported node, such as a TPM 1.2 node, wipes the disk and leaves it in HookOS. Control-plane installation additionally requires homelab.fleetd.controlPlanes; it is disabled by default.
See Identity for the credential terms and Tinkerbell for the Workflow components.
Who owns what#
| Component | What it owns | Where it lives |
|---|---|---|
| Nodes file | Each node's name, serial, model, class, switch port, address, BMC type and approved EK binding | clusters/folly/inventory/nodes.yaml |
terraform/netbox/folly | The device, its address and cable, its declared custom fields, the custom field definitions, fleetd's webhook | Atlantis on offsite |
| NetBox | Status, desired_action, verdicts, MACs | capsule |
| fleetd | Attestation, the LDevID and LAK, install codes, renewal, Hardware, Machine, Workflow, the rendered config, retirement | spore |
| Boot plane | dnsmasq, nginx, the signed iPXE and HookOS, at SPORE_BOOTSTRAP_IP | spore |
| Tinkerbell | The Tink server and Rufio, which run each Workflow | spore |
| SPIRE | The server, the agents, and tpm-reattest | server on spore, agents on the folly nodes |
| UniFi network | Node ports, each the Kubernetes network untagged, and that network's DHCP pool for netbooting machines | terraform/network/unifi/folly/bootstrap.tf, k8s.tf |
Trust#
- Firmware checks images against its Secure Boot database, which also contains Microsoft certificates. After the event-log separator, the node's class picks a policy profile in
apps/fleetd/attest/policy/:talosaccepts only the lab's signing authorities, andubuntualso accepts Microsoft's UEFI CA and Canonical's Master CA. Intake requires HookOS. No profile pins an exact image digest. - The LDevID key stays in the TPM. fleetd signs every LDevID with spore's YubiKey CA, and the bundle keeps the LDevIDs capsule's and the software CA issued trusted until they renew.
- A provision's wipe waits for the boot's pass and the install code, so a refused machine keeps its disk.
- The running node renews its LDevID with a quote that its LAK signs. A
decommissioningorofflinedevice gets no renewal. - An authenticated quote that misses the policy quarantines a worker: fleetd refuses renewal and cordons it. SPIRE registration removal and existing credential expiry are separate. A control-plane failure records a failed verdict and alerts without a cordon; fleet-owned registration eligibility can change. Identity has the rules.
Where to go next#
- Security engineering brief: purchase, receiving, trust boundaries and off-the-shelf alternatives.
- Technical design: component relationships, sequence diagrams, tradeoffs, evidence and threats.
- Prepare a machine, Provision a node, Reprovision a node, Retire a node
- Identity, fleetd, Tinkerbell, NetBox
Linked from 5
- IdentitySPIRE on spore gives each folly node with a TPM 2.0 an identity rooted in its TPM, by LDevID or EK hash, gives its pods SPIFFE IDs, and lets a pod reach GCP with no key file.
- PlatformThe shared systems that the apps and hosts depend on, and what applies each layer from git.
- DIY hardware-to-workload identityA proposed signed bootstrap image, provisioning-time tool, narrow identity service and SPIRE stack, without Tinkerbell or a full fleetd daemon.
- Folly node lifecycle and trust designThe technical design, tradeoffs, evidence and threat boundaries for node provisioning, trusted boot and workload identity at folly.
- Hardware inventoryThe nodes file that declares each folly Kubernetes node, the code that reads it, the OpenTofu root that applies it into NetBox, and the check that compares NetBox with the machines.