Built apps
A kthx built app is a repository or archive that kthx builds into a signed artifact and deploys to Targets on the clusters, GCP, Vercel or Cloudflare.
A built app is an App that kthx builds from a GitHub repository or an uploaded archive, signs, and deploys. The owner and agents manage them in the console and over MCP.
Concepts#
| Noun | Meaning |
|---|---|
| App | A source and the Components deployed from it |
| Component | A service, website or job in an App |
| Vessel | A cluster, GCP project, Vercel team or Cloudflare account, created by OpenTofu |
| Target | One runtime on a Vessel, where a Component runs |
| Build | One run of a build route. Its Artifact is an image digest or a file tree. |
| Deploy | An Artifact and config applied to a Target |
| Datastore | A Postgres or Valkey instance for Apps |
| Function | A JavaScript fetch handler on Workers or Cloud Run, with no Build or Deploy |
Targets#
| Target | Vessel | A Deploy creates |
|---|---|---|
kubernetes | folly or offsite | A spindrift-app HelmRelease or an Argo CD Application that installs into app-<app> |
cloudrun | bluenose | A Cloud Run service or job |
static | bluenose | A Firebase Hosting release |
vercel | Vercel team | A prebuilt deployment |
cloudflare-pages | Cloudflare account | A Pages deployment |
A Target's console page lists each missing prerequisite and the OpenTofu code that adds it, which kthx can open as a pull request. Disconnecting a Target stops deploys to it and leaves its workloads running.
Deleting an App deletes its namespace. Removing a static Component spends its Firebase Hosting site id, so a redeploy needs a new App or Component name.
Builds#
A file in each scope of a repository states the scope's kind and build, and it wins over detection. kthx reads kthx.yaml, or spindrift.yaml when a scope has none. Connecting a repository writes kthx.yaml into a scope that has neither. apps/kthx-engine/src/domain/detection/kthx-file.ts holds the schema.
A Build uses the first route, in Settings rank order, whose SLSA (Supply-chain Levels for Software Artifacts) build level meets the Target's minimum, 2 by default. An App can pin one route.
| Route | Level | Runs on |
|---|---|---|
github-actions | 2 | .github/workflows/kthx-engine-build.yml |
cloud-build | 3 | Cloud Build |
in-cluster | 1 | A Job in spindrift-build on offsite. The route's nodeSelector and tolerations put it on hotrod, the sandbox node. |
bosun | 2 | Bosun, which is parked |
Addresses and config#
A Component's reach is none, private or public. Ingress and DNS covers the records and zones. auth: proxy adds oauth2-proxy, which admits only the owner's GitHub login.
<app>.lolwtf.dev shows a status page until a Component serves it. kthx writes the record of an App on a zone apex once, and cannot change or delete it.
When the engine has the KTHX_ENGINE_KTHX_* variables, a name an App mints in the quick sites' zone is reserved for the App in kthx first: when its vanity name is set or its zone changes, and before each Deploy. A name a quick site or another App holds refuses the edit or rejects the Deploy.
Only deleting the App releases names: those its placed Components still mint, and none while a retained workload serves them. A name the App reserved through an earlier edit or a deleted Component stays reserved. Two Apps of the same name with a Component of the same name mint one canonical name, so the second one's Deploys are rejected.
apps/kthx-engine/src/commands/apps/kthx-names.ts holds the rule.
kthx writes config values to Secret Manager in bluenose, and each Target reads them from there (Secrets).
Function env values are write-only, but Cloud Run shows them to bluenose readers. Workers Functions need the Cloudflare token scopes in terraform/network/cloudflare/kthx-engine.tf.
Linked from 13
- BosunA Go daemon that keeps a warm pool of one-job GitHub Actions runners in Cloud Hypervisor microVMs. No host runs it.
- clankerbankerA public pay-per-call API at clankerbanker.ca, a kthx built app on offsite, that sells fortunes, dice rolls, storage and model answers to bots for USDC over x402.
- kthxThe lab's hosting product on the offsite cluster, which the owner and agents use to serve a directory at <name>.kthx.dev or to build a repository and deploy it to a cluster or a cloud.
- Ownership and securityWhich system owns each part of kthx, the identities and federation each part uses, and how built images are signed and admitted.
- hotrodAn HP ProDesk 600 G6 mini PC that is the offsite cluster's sandbox node, the worker that runs the untrusted work, on Talos Linux.
- Cloud accountsThe Google Cloud projects, Cloudflare zones and Google Workspace domain that OpenTofu declares, and what each one is for.
- How changes shipThe path each kind of change takes from a pull request to the running system, and the GitOps rule that no one changes live state by hand.
- Ingress and DNSThe Gateways, DNS records, certificates, tunnels and lab resolvers that give cluster services a name and a path from the LAN, tailnet or internet.
- WorkloadsWhat runs on the folly and offsite Kubernetes clusters, the files that list it, and the workloads that have their own page.
- GlossaryThe lab-specific terms that the wiki pages use, what each means, and the page that defines it.
- Connect an agent to kthxMint an agent token in the kthx console and connect an MCP client, Rowbutt or the owner's workstation to the kthx engine, which serves each console command as a tool.
- Install kthxInstall the kthx engine on the offsite cluster, from the OpenTofu roots to the first passkey.
- Operate PostgresFind, connect to, inspect and restart the CloudNativePG Postgres databases with the kubectl cnpg plugin, and check which ones have a backup.