133 lines
6.6 KiB
Markdown
133 lines
6.6 KiB
Markdown
# package-updater
|
|
|
|
Updates self-hosted Docker Compose and Docker Swarm services from one place. A
|
|
**master** (web UI behind OIDC login, state in Postgres) shows every Compose stack,
|
|
Swarm service and container on every machine, and starts updates when someone clicks
|
|
*Update*. One **agent** per machine reports its inventory and runs the jobs. Both are in
|
|
the same image: `updater-master` and `updater-agent` (the default entrypoint).
|
|
|
|
Design: [master/agent spec](docs/superpowers/specs/2026-10-12-master-agent-design.md).
|
|
Phase 2 plan: [jobs and manual update](docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md).
|
|
|
|
> **Webhook:** the old `POST /update` endpoint is gone. It returns in phase 3 on the
|
|
> master; until then the Gitea action in `gitea-action/` has nothing to call.
|
|
|
|
## What an update does
|
|
|
|
Clicking *Update* on a service re-pulls the service's **current tag** (`repo:tag`). The
|
|
agent asks its Docker daemon for the tag's digest and deploys `repo:tag@digest`, so what
|
|
runs is exactly what was pulled. The log streams to the browser while it runs; job
|
|
history is kept in Postgres. A service without a tag (`repo@digest`), a plain container,
|
|
and the updater's own services cannot be updated, and the UI says why. A bare image
|
|
(`image: postgres`, no tag and no digest) counts as `postgres:latest`; only a reference
|
|
with a digest and no tag (`repo@sha256:...`) "follows no tag". Compose services with a `build:` section are shown but never updated
|
|
("built from source (build:)"). One update at a
|
|
time per service, and one job at a time per agent.
|
|
|
|
## Deploying the master
|
|
|
|
Use [`deploy/master.stack.yml`](deploy/master.stack.yml) as a starting point (Swarm stack;
|
|
needs Postgres and an OIDC provider). The master listens for the UI on `PORT` (default
|
|
`8080`, put it behind your reverse proxy) and for agents on `AGENT_PORT` (default `8090`,
|
|
pinned TLS, reached directly on the internal network).
|
|
|
|
## Adding agents
|
|
|
|
Open the master's **Agents** page and add one. It shows a ready-to-run compose file with
|
|
the agent's token (shown once) and the master's key fingerprint; the same layout is in
|
|
[`deploy/agent.compose.yml`](deploy/agent.compose.yml).
|
|
|
|
| Variable | Required | Default | Purpose |
|
|
|---|---|---|---|
|
|
| `MASTER_URL` | yes | — | `wss://host:port/agent` |
|
|
| `MASTER_KEY` | yes | — | Master key fingerprint (`sha256:...`) shown in the UI. |
|
|
| `AGENT_TOKEN` | yes | — | Token shown once when the agent was added. |
|
|
| `SELF_ID` | no | hostname | Container ID or name of the agent itself; used to read its mounts. |
|
|
| `UPDATE_TIMEOUT` | no | `5m` | Per-job timeout (Go duration). |
|
|
| `DOCKER_CONFIG` | no | — | Directory with a `config.json` holding registry credentials. Only inline `auths` entries work; `credsStore`/`credHelpers` are not available in the container. |
|
|
| `LOG_LEVEL` | no | `info` | `debug` / `info` / `warn` / `error`. |
|
|
|
|
`docker compose` runs inside the agent with only `PATH`, `HOME` and the `DOCKER_*`
|
|
variables. Variables a stack uses for interpolation must come from the stack's own `.env`
|
|
file or an `--env-file` (which the agent honours), never from the agent container's
|
|
environment.
|
|
|
|
### Registry credentials
|
|
|
|
With `DOCKER_CONFIG` set, Compose jobs pull with the same credentials as `docker compose`.
|
|
For Swarm jobs the agent passes the matching registry's credentials with the service
|
|
update (same as `docker service update --with-registry-auth`), so Swarm stores them in the
|
|
cluster and hands them to the nodes that pull the image. Only the entry for the image's
|
|
registry host is ever sent.
|
|
|
|
### Mounting stack directories
|
|
|
|
There is no `STACKS_ROOT`. The agent may only touch Compose stacks in directories that
|
|
are bind-mounted into its own container **read-write at the same path as on the host**
|
|
(Compose labels hold host paths). Add one line per directory:
|
|
|
|
```yaml
|
|
volumes:
|
|
- /var/run/docker.sock:/var/run/docker.sock
|
|
- /home/shcizo/self-hosted:/home/shcizo/self-hosted
|
|
```
|
|
|
|
Read-only mounts and mounts whose source differs from the destination are refused with a
|
|
clear message. On Docker Desktop and OrbStack bind sources are translated host paths, so
|
|
Compose jobs fail there; that is a development-only limitation.
|
|
|
|
## How Compose updates work, and their limits
|
|
|
|
The agent never edits your files. It writes one file next to the stack's base file,
|
|
`compose.override.yaml`, with a header line and an `image:` entry per updated service
|
|
(`repo:tag@digest`), then runs `docker compose pull` and `up -d` for that service with
|
|
the override passed explicitly.
|
|
|
|
- The agent only writes `compose.override.yaml` if it is absent or starts with its header
|
|
line, and **refuses the job** if any other auto-loaded override exists
|
|
(`compose.override.yml`, `docker-compose.override.yml`, `docker-compose.override.yaml`):
|
|
Compose loads only one of them, so ours would silently disable yours.
|
|
- Stacks you start with explicit `-f` files do not load the override on a manual
|
|
`docker compose up`; the agent's own jobs always pass it.
|
|
- **The override is never pruned.** If you delete a service from the stack file, its
|
|
`image:` entry stays in `compose.override.yaml`, and a manual `docker compose up -d`
|
|
would recreate it. Remove the entry by hand.
|
|
|
|
**Check before enabling.** List the config files of every running stack to spot existing
|
|
override files and stacks started with explicit `-f`:
|
|
|
|
```bash
|
|
docker ps --format '{{.Label "com.docker.compose.project.config_files"}}' | sort -u
|
|
```
|
|
|
|
## Swarm updates
|
|
|
|
The agent updates the service with `ServiceUpdate` and the pinned image. "Succeeded"
|
|
means the daemon accepted the new spec, not that the tasks converged; the inventory shows
|
|
`degraded` until they do. The agent doing the update must talk to a manager node (run it
|
|
on a manager, or point `DOCKER_HOST` at one).
|
|
|
|
## Upgrading from the single-instance updater
|
|
|
|
This release removes `cmd/server` and changes the image's default entrypoint to
|
|
`updater-agent`. A running old-style updater (default entrypoint, `UPDATER_API_KEY`) will
|
|
crash-loop the next time its image is pulled. Anything that pulls `:latest` for the old
|
|
instance triggers this: its own `/update` webhook, Watchtower or Shepherd.
|
|
|
|
Pinning means two things: pin the old instance to the last pre-phase-2 tag **and** remove
|
|
its `se.shcizo.auto-update` label, so nothing updates it to the new image. Or do not tag a
|
|
release until phase 3 moves `POST /update` to the master.
|
|
|
|
Master and agents must be on this release together: a phase-1 agent connects but every
|
|
job sent to it fails "agent too old for updates".
|
|
|
|
## Development
|
|
|
|
```bash
|
|
go test ./...
|
|
go test -race ./internal/agent/... ./internal/master/... ./internal/e2e/
|
|
go build ./cmd/master ./cmd/agent
|
|
docker build -t package-updater:dev .
|
|
DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
|
|
```
|