Ownership and security
Which system owns each part of kthx, the identities and federation each part uses, and how built images are signed and admitted.
OpenTofu, Flux and the kthx engine each own part of kthx. Built apps defines the nouns.
Who owns what#
| Part | Owner | Source of truth |
|---|---|---|
| Vessels, tunnels and signing keys | OpenTofu, applied by Atlantis | terraform/ |
| The engine, quick sites, Target RBAC and the image policy | Flux | clusters/ |
| App namespaces, releases, DNS records and Datastores | The engine, through Target APIs | Its database |
| Engine settings, such as Targets and build routes | The engine | Its database, exported from the console's Settings page |
The engine deletes only namespaces labelled app.kubernetes.io/managed-by: spindrift. Target RBAC cannot touch Secrets, so Datastore credentials reach Apps through External Secrets.
Identities#
| Caller | Credential | Can |
|---|---|---|
| The owner | A console passkey, enrolled once with KTHX_ENGINE_ENROLMENT_TOKEN | Every console and MCP action |
An agent on /mcp | An agent token, valid 90 days | Every action except minting tokens, replacing the engine settings, and connecting or probing a Target |
| kthx on GitHub | The private GitHub App kthx-bot (ID 4576122), installed on jonpulsifer only | Push branches, open pull requests and run workflows in connected repositories |
| A quick-site owner | Write access to the sites bucket, and a Google ID token or tailnet login that the site's marker names | Deploy and delete that site, and call its owner routes |
| The engine, at quick sites | A projected ServiceAccount token with audience kthx, sent to /api/engine/* on the control host | List and read every quick site with its owner login, and reserve and release names under the quick-site zone. It carries no login, so it opens no site and cannot delete every site. |
| Rowbutt | An agent token from Secret mate-kthx-agent, which mate's pod holds, and a sandbox's federated token for [email protected] | What an agent on /mcp can, without approval, and every quick site it deploys |
Secrets#
clusters/offsite/apps/kthx-engine/secret.sops.yaml holds the engine's secrets under their KTHX_ENGINE_ names.
| Secret | Also in | If missing |
|---|---|---|
| Enrolment token | Nobody can enrol a passkey | |
| Vercel and Cloudflare tokens | Their Targets cannot connect | |
| GitHub App key | 1Password and nix/secrets/bosun.sops.yaml. Update all three before you delete an old key. | kthx has no GitHub App until the console creates one |
KTHX_ENGINE_GITHUB_WEBHOOK_SECRET | The engine refuses every GitHub webhook |
Quick-site trust#
Quick sites trust Tailscale-User-Login only on kthx.<tailnet>, where KTHX_ADMIN_LOGINS can delete every site and free its name. Names the engine reserves for built apps stay taken. Network policies kthx and kthx-gateway admit only the Tailscale proxy, Gateway, kubelet and namespace pods. Rate limits key reliably only on traffic that arrives through Cloudflare.
The console shares an origin with kthx's /api and /cli on kthx.lolwtf.ca, so kthx's same-origin write guard trusts console script.
The engine's read of quick sites includes each owner login. That login is no secret inside the cluster: any cluster-admin, Rowbutt's sandbox included, can mint the engine's token or read kthx-db. Keeping it from agents is hygiene, and it is no security boundary.
Federation#
The engine runs as ServiceAccount kthx-engine/kthx-engine on offsite, with no GCP or cluster key.
- GCP: The
offsiteprovider of thefml-poolworkload identity pool lets it impersonatekthx-engine-controller@bluenose(terraform/gcp/projects/bluenose/iam.tf). - folly: The API server accepts its token as user
federated:system:serviceaccount:kthx-engine:kthx-engine(clusters/folly/talos/patches.tf). - kthx: The quick-site server accepts a token from offsite's issuer with audience
kthxand subjectsystem:serviceaccount:kthx-engine:kthx-engine(clusters/offsite/apps/kthx/helm-release.yaml). It reads the issuer's keys fromhttps://oidc.lolwtf.ca/offsite, and neither API server accepts that audience. The engine sends kthx only that projected token, set byserviceAccount.token.kthxAudienceinclusters/offsite/apps/kthx-engine/helm-release.yaml. Its other tokens never reach kthx. - Vercel and Cloudflare: none. The engine uses stored tokens.
Quick sites run as kthx-server@bluenose, which reaches only bucket bluenose-kthx.
Supply chain#
- The engine verifies each Build's provenance, then signs the image digest with a KMS key in the trusted-builds GCP project.
- Kyverno policy
kthx-engine-verify-imagesrejects an App pod without that signature. It matches pods labelledkthx.dev/deployon both clusters (clusters/base/platform/kthx-engine-policy/verify-images.yaml). - Kyverno reads the image signature from GHCR. A private package without registry access prevents pod admission.
- On Cloud Run, the Vessel's Binary Authorization policy admits only attested images.
github-actionsbuilds reach GCP through thehomelabworkload identity pool as the workflowkthx-engine-build.ymlonmain.
Linked from 5
- kthxThe lab's hosting product on the offsite cluster, which the owner and agents use to serve a directory at <name>.kthx.dev or to build a repository and deploy it to a cluster or a cloud.
- How changes shipThe path each kind of change takes from a pull request to the running system, and the GitOps rule that no one changes live state by hand.
- PKIThe FML certificate chain, the Kubernetes cluster CAs and token signers, the cluster OIDC issuers, workload identity federation and the hardware DevID CAs.
- Connect an agent to kthxMint an agent token in the kthx console and connect an MCP client, Rowbutt or the owner's workstation to the kthx engine, which serves each console command as a tool.
- Install kthxInstall the kthx engine on the offsite cluster, from the OpenTofu roots to the first passkey.