infra_wiki

Remote access

The Tailscale tailnet that reaches both sites' private addresses, and the grants that decide who reaches what.

Remote access is a Tailscale tailnet that the owner uses to reach both sites' private addresses from off the LAN. OpenTofu declares the tailnet, and Atlantis applies it.

Parts#

PartJobWhere it runs
ConnectorRoutes tailnet traffic into the site, and advertises its LAN networks, K8S_NODE_CIDR and LB_RANGEOne in each cluster
policy.hujsonHolds the grants, and approves the Connectors' routesTailscale
devices.tfAuthorizes each device, sets its key expiry and assigns its tagsTailscale
CI identityLets the nixos-deploy workflow join as tag:ci with a GitHub OIDC tokenGitHub Actions
forge OAuth clientEnrolls forge as tag:lab-hostforge

The Kubernetes nodes run no Tailscale client (nix/system/tailscale-disable.nix), so tailnet traffic reaches each site through its Connector. folly's LAN networks are Management and the Anchor network, and offsite's is Default.

OpenTofu stores the forge client's secret in 1Password. After the Atlantis apply, copy it into forge's SOPS file as tailscale-auth-key.

Grants#

SourceReaches
The ownerEvery tagged device, the owner's own devices, both sites' LAN networks, the offsite-bell subnet, and both clusters' node subnets and load-balancer addresses (VIPs)
Every tailnet membertag:kthx-ingress, the kthx tailnet Ingress, on TCP port 443, and Tailscale SSH to their own devices after a fresh sign-in check
tag:follyoffsite's node subnet, VIPs and Default network
tag:offsitefolly's node subnet and VIPs
tag:citag:pi4 devices on TCP port 22
tag:gcpNothing

The Connectors carry tag:k8s, tag:k8s-<site>, and tag:folly or tag:offsite. autoApprovers accepts routes from tag:k8s-folly and tag:k8s-offsite without review. devices.tf also gives tag:offsite to desktop-g7i75ls, which no host sheet covers. offsite-bell is an ipset in policy.hujson, and no UniFi root declares its subnet.

Rules#

  • The CI identity accepts only nixos-deploy.yaml runs on main.
  • The tailnet requires device approval. A new device waits until Atlantis applies its entry in devices.tf.
  • policy.hujson holds literal subnets. After a topology change, edit its autoApprovers, ipsets and tests. Otherwise new routes wait for approval, the grants deny the new range, and the policy apply fails.

Where it lives#

  • terraform/network/tailscale/: the tailnet settings, policy and device map, the CI identity in github_actions.tf and the forge client in oauth_clients.tf
  • terraform/network/tailscale/fleet.tf.json: the tailnet name
  • clusters/base/networking/tailscale/: the tailscale-operator HelmRelease
  • clusters/<site>/networking/tailscale-connectors/: each site's Connector
  • nix/hosts/forge.nix: forge's tailscale-auth-key secret

Edit on GitHub

Linked from 5
  • HostsEvery host in the homelab, with its role, site, hardware, OS and status, and how to reach it.
  • NetBoxThe record of every machine in the lab, on capsule, reachable only over the tailnet.
  • NetworkThe two sites, their UniFi gateways and networks, and the tunnel, routes, ingress and tailnet that connect them.
  • Ingress and DNSThe Gateways, DNS records, certificates, tunnels and lab resolvers that give cluster services a name and a path from the LAN, tailnet or internet.
  • GlossaryThe lab-specific terms that the wiki pages use, what each means, and the page that defines it.

↑↓ move ↵ open esc close