Files
package-updater/CLAUDE.md
shcizo b1d6e5e705 fix: review fixes for webhook and auto mode (#8)
Clear available_* only when the deployed ref equals it, drop the unused
poll source, stop leaking internal errors to CI, use the request-scoped
logger, validate X-Request-ID, re-read jobs once after the timeout,
report refused when no target ref exists, and tidy UI, deploy and docs.
2026-10-12 20:48:43 +02:00

127 lines
8.0 KiB
Markdown

# package-updater — Claude context
Master/agent fleet updater. A master (UI behind OIDC, Postgres) shows every Compose
stack, Swarm service and container on every machine and starts updates; one agent per
machine (`/var/run/docker.sock` mounted) reports its inventory and runs the jobs.
Both are in one image: `updater-master` and `updater-agent`.
See `README.md` for user-facing docs. This file is for working ON the code.
## Commands
```bash
go test ./... # all tests, fast (no docker, no postgres)
go test -race ./internal/agent/... ./internal/master/... ./internal/e2e/
go test -run TestFoo ./internal/master/hub # single test
go build ./cmd/master ./cmd/agent # produce ./master and ./agent
docker build -t package-updater:dev . # multi-stage, builds golang:1.26-alpine
# Postgres tests (store), against any throwaway database:
DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
```
## Architecture (one-line per package)
- `cmd/master` — wiring only: config → store → hub → jobs → web + webhook, two listeners (UI, agents over pinned TLS)
- `cmd/agent` — wiring only: config → docker client → inventory → job runner → master connection
- `internal/protocol` — the WebSocket messages both sides speak (`Version`, `JobStart`, `JobResult`, ...)
- `internal/pin` — master key pair, certificate and fingerprint pinning
- `internal/agenttoken` — agent token generation and hashing
- `internal/imageref` — image reference parsing (repo, tag, digest)
- `internal/logging` — slog JSON, request-id context propagation
- `internal/agent/config` — env-var loading; fails fast on missing `MASTER_URL`/`MASTER_KEY`/`AGENT_TOKEN`
- `internal/agent/inventory` — reads Compose, Swarm and plain containers from the Docker daemon
- `internal/agent/conn` — dial, hello, snapshot, job messages, reconnect with backoff; one writer goroutine per session
- `internal/agent/updater` — job `Runner` (single FIFO worker, outbox until acked), Compose and Swarm executors, override file, mount-root check, registry auth
- `internal/master/config` — env-var loading
- `internal/master/store` — Postgres access and migrations
- `internal/master/hub` — agent connections: auth, per-connection writer, routing, job reconcile on hello
- `internal/master/jobs` — decides whether a service may be updated and starts a job on the right agent
- `internal/master/webhook` — `POST /update`: records the available version, starts jobs for auto services, waits for them
- `internal/master/web`, `webauth` — HTMX UI behind OIDC login, SSE signals
- `internal/master/events` — in-process publish/subscribe for UI refresh signals
- `internal/master/model` — domain types shared by the above
## Design intent (do not break without discussion)
- **Single FIFO worker per agent by design** (`internal/agent/updater/runner.go`).
Two `docker compose` calls against the same stack race.
- **One queued or running job per service**, enforced by the unique index
`update_jobs_one_active` (not by application code).
- **The agent writes exactly one file**, `<stack dir>/compose.override.yaml`, and only
if it is absent or starts with the header line. It refuses the job when any other
auto-loaded override name exists.
- **Every config file and the override path must be inside a read-write bind mount of
the agent's own container whose source equals its destination.** Mounts are read per
job (`ContainerInspect(SELF_ID)`) and checked after symlink resolution. This replaces
the old `STACKS_ROOT`.
- **Self-protection on the master**: a service is refused when its image repo equals the
repo of `AGENT_IMAGE` or the row has the `self` flag. The agent does not re-check.
- **Unknown message types are logged and ignored; malformed messages close the connection.**
- **Reconcile before routable**: the hub fails a reconnecting agent's jobs it does not
list in `known_jobs`, and only then accepts jobs for that connection.
- **The webhook never creates services; new services always start `manual`.** The upsert
does not touch `update_mode`.
- **Auth**: agents by token over pinned TLS; people by OIDC; CI by the `UPDATER_API_KEY`
bearer token on `POST /update` only. Every state-changing UI request is a POST with a
CSRF token.
## Gotchas
- **Image matching is case-sensitive.** The tricky case is `localhost:5000/foo:v1` where
the first colon is a port, not a tag. See `internal/imageref`.
- **Path safety uses `filepath.Rel` + ".." prefix check**, NOT `strings.HasPrefix`.
Prevents the `/foo-evil` vs `/foo` confusion. See `internal/agent/updater/pathcheck.go`.
- **Go 1.26.3** (`go.mod`). Dockerfile pins `golang:1.26-alpine`. Bumping one
without the other has caused a fix commit already.
- **`<summary>` C# convention from global CLAUDE.md does not apply here** — this
is Go. Use idiomatic GoDoc (`// FuncName does X.`).
- **Compose loads only one override file.** Of `compose.override.yml`,
`compose.override.yaml`, `docker-compose.override.yml`, `docker-compose.override.yaml`
only the first that exists is read. Hence the refusal described above.
- **Stacks started with explicit `-f` do not load the override on a manual `up`.** The
agent's own jobs always pass it explicitly.
- **Override entries are never pruned.** A service deleted from the stack file keeps its
`image:` line in `compose.override.yaml`.
- **There is no `STACKS_ROOT`.** Stack directories must be mounted read-write at the host
path (`/path:/path`), and `SELF_ID` must resolve to the agent's own container. On Docker
Desktop/OrbStack bind sources are translated, so Compose jobs fail there.
- **`job.start` carries no paths.** The agent reads working dir and config files from its
own Docker daemon at execution time.
- **`protocol.Service.ConfigFiles` is the raw comma-joined label string**, so `Service`
stays comparable with `==` (`inventory.Diff` depends on it).
- **Swarm "succeeded" means the daemon accepted the new spec**, not that tasks converged.
- **Swarm updates need manager-node API access** for the agent doing the update.
- **The image's default entrypoint is `updater-agent`.** The master overrides it with
`updater-master`.
- **The webhook waits via a broker signal plus a store re-read, with a fallback poll**
(the broker drops signals for slow subscribers). Marking and starting jobs runs on
`context.WithoutCancel`, so a client that disconnects cannot leave a queued job no agent
knows about; only the waiting stops. The master never fails or cancels a timed-out job.
- **A webhook job's target is `<image_repo>:<request tag>`**, built from the service's own
repo, never from the request's image string. With an explicit target the "follows no tag"
refusal does not apply; removed, self, built and container still do.
- **`available_*` is cleared by `FinishJob`** on success only when the job's `to_ref`
(as created, without `@digest`) equals `available_ref` and `available_at <= job.created_at`
(that version is now running). A manual Update of the followed tag does not clear a
badge for a different tag.
- **Job output is capped at `protocol.MaxJobOutput` (128 KiB, tail).** NUL bytes are
stripped by the hub because Postgres text rejects them.
## References
- Master/agent spec: `docs/superpowers/specs/2026-10-12-master-agent-design.md` (authoritative)
- Phase 1 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-1-inventory.md`
- Phase 2 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md`
- Phase 3 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-3-webhook.md`
- Original single-instance design (historical):
`docs/superpowers/specs/2026-05-22-package-updater-design.md`
- Consumer-side CI integration: `gitea-action/` (calls the master's `POST /update`)
## Conventions
- Conventional Commits (`feat:`, `fix:`, `docs:`, `ci:`, `build:`, `chore:`).
- One package = one responsibility; interfaces defined at consumer site
(`jobs.Store`, `jobs.Hub`, `conn.Jobs`, `hub.Store`) for testability.
- Table-driven tests with `stretchr/testify`. No mocks beyond hand-written fakes.