docs: document master/agent phase 1 and add master stack example

This commit is contained in:
2026-10-12 15:04:43 +02:00
parent 05610391e3
commit 4f092eaf79
4 changed files with 79 additions and 0 deletions
+16
View File
@@ -11,6 +11,8 @@ See `README.md` for user-facing docs. This file is for working ON the code.
go test ./... # all tests, fast (no docker required)
go test -run TestFoo ./internal/api # single test
go build ./cmd/server # produces ./server
go build ./cmd/master ./cmd/agent # new master/agent binaries (phase 1: inventory)
DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/ # Postgres tests
docker build -t package-updater:dev . # multi-stage, builds golang:1.26-alpine
# Local smoke test (builds from Dockerfile, runs against /var/run/docker.sock):
@@ -32,6 +34,13 @@ curl -sH "Authorization: Bearer $UPDATER_API_KEY" \
- `internal/selfupdate` — flush-then-exec wrapper for updating ourselves (NOT wired in live, see gotcha)
- `internal/metrics` — Prometheus collectors
- `internal/logging` — slog JSON, request-id context propagation
- `cmd/master`, `cmd/agent` — master/agent architecture (spec `2026-10-12-master-agent-design.md`), phase 1 = inventory only. `cmd/server` keeps doing updates until phase 3.
- `internal/protocol` — agent↔master WebSocket messages
- `internal/pin` — master key pair, self-signed cert, fingerprint-pinned TLS client
- `internal/agenttoken`— per-agent tokens (`pua_` prefix, only the SHA-256 hash is stored)
- `internal/imageref` — split `repo:tag@digest`
- `internal/agent/...` — inventory collection, Docker event debounce, connection loop
- `internal/master/...`— Postgres store, agent hub, change broker, OIDC auth, htmx UI
## Design intent (do not break without discussion)
@@ -73,12 +82,19 @@ curl -sH "Authorization: Bearer $UPDATER_API_KEY" \
with a permission error against a worker-only node. This is an operator/deployment
concern (point `DOCKER_HOST` at a manager, or schedule the updater on a manager),
not something the code can detect or work around.
- **WebSocket read limit is raised to 1 MiB on both sides.** coder/websocket defaults to 32 KiB, which a full snapshot from a busy machine exceeds — symptom is an agent stuck in a reconnect loop.
- **The master's key lives in Postgres (`master_identity`).** Wiping the database means every agent needs a new `MASTER_KEY`.
- **SSE responses must flush through every wrapper.** `web.events` uses `http.ResponseController`; a middleware that wraps `ResponseWriter` without `Unwrap()` silently breaks live updates. Nginx needs `X-Accel-Buffering: no` (set by the handler).
- **The "stateless" and "label gate" design intents above describe `cmd/server` only.** The master is stateful by design and gates updates per service in the UI (spec 2026-10-12).
- **The master does OIDC discovery once at startup and exits if the issuer is unreachable.** It does not retry; Swarm restarts the container, so a short Authentik outage during deploy resolves itself.
## References
- Design spec: `docs/superpowers/specs/2026-05-22-package-updater-design.md` (489 lines, authoritative)
- Implementation plan: `docs/superpowers/plans/2026-05-22-package-updater-implementation.md`
- Consumer-side CI integration: `gitea-action/`
- Master/agent spec: `docs/superpowers/specs/2026-10-12-master-agent-design.md`
- Phase 1 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-1-inventory.md`
## Conventions
+15
View File
@@ -46,6 +46,21 @@ Two things to get right when running in Swarm mode:
mounted. This is a deployment concern the service cannot detect or work
around.
## Master/agent (in progress)
A master with one agent per machine is replacing the single-instance webhook
model; see the [design spec](docs/superpowers/specs/2026-10-12-master-agent-design.md).
Phase 1 delivers a live inventory: every Compose stack, Swarm service and
container on every machine, behind Authentik login. Updates still go through
the existing `/update` webhook until phase 3.
- Master: `updater-master` in the same image, example stack in `deploy/master.stack.yml`.
At startup it does OIDC discovery against `OIDC_ISSUER` and exits if the issuer
is unreachable; Swarm then restarts it.
- Agents: add one per machine on the master's **Agents** page; it shows a
ready-to-run compose file with the agent's token and the master's key
fingerprint.
## Quick start
1. Build and push the image (e.g. via your own CI).
+41
View File
@@ -0,0 +1,41 @@
# Example Swarm stack for the package-updater master (phase 1: inventory only).
# Deploy on the manager: docker stack deploy -c master.stack.yml package-updater-master
#
# Before deploying:
# 1. In Postgres: CREATE USER package_updater ...; CREATE DATABASE package_updater OWNER package_updater;
# 2. In Authentik: create an OAuth2/OpenID provider + application with redirect URI
# https://updater.example.com/auth/callback, and a group "updater-admins".
# 3. docker secret create pu_oidc_client_secret - ; docker secret create pu_database_url -
services:
master:
image: gitea.shcizo.se/shcizo/package-updater:latest
entrypoint: ["/bin/sh", "-c", "DATABASE_URL=$$(cat /run/secrets/pu_database_url) OIDC_CLIENT_SECRET=$$(cat /run/secrets/pu_oidc_client_secret) exec /usr/local/bin/updater-master"]
environment:
PUBLIC_URL: https://updater.example.com
AGENT_URL: wss://10.0.0.5:8090/agent
OIDC_ISSUER: https://auth.example.com/application/o/package-updater/
OIDC_CLIENT_ID: package-updater
ALLOWED_GROUP: updater-admins
secrets:
- pu_database_url
- pu_oidc_client_secret
ports:
# Agents connect here directly over the internal network, not via the proxy.
- target: 8090
published: 8090
mode: host
networks:
- proxy
deploy:
placement:
constraints: [node.role == manager]
secrets:
pu_database_url:
external: true
pu_oidc_client_secret:
external: true
networks:
proxy:
external: true
@@ -29,6 +29,13 @@
- The agent detects changes by re-running the full inventory after Docker events settle and diffing it against the last one sent, rather than re-inspecting individual items. Same messages on the wire, much less code.
- The envelope is `{type, payload}`. The `id` field arrives in phase 2 together with jobs.
- Only the tables phase 1 needs are created. `desired_ref`, `available_*`, `last_poll_error`, `image_versions` and `update_jobs` arrive in later phases' migrations.
- The agents column is named `swarm_manager` (spec: `is_swarm_manager`), and services carry a single unique `location_key` column instead of one unique index per kind.
- Templates and static assets live under `internal/master/web/` (embedded there), not a top-level `web/` directory.
- The master has no `/metrics` or `/version` endpoint in phase 1; only `/healthz`.
- The overview shows all Swarm services in one "Swarm" group, not one group per cluster.
- The session cookie gets the `Secure` flag only when `PUBLIC_URL` starts with `https://`, so a local http setup works.
- The agent runs a full re-inventory every 60 seconds in addition to Docker events, because a Swarm replica dying on another node emits no event on the manager running the agent.
- Known limitation: the master files Swarm services under the cluster ID the agent reports in its hello. An agent whose token leaks can therefore overwrite or remove the overview rows of any Swarm cluster. This matches the spec's statement that a leaked token can feed false inventory but not command machines; phase 2 must not route Swarm jobs based only on the cluster ID an agent claims.
## Review Focus