shcizo/package-updater
1
0
Fork 0
Code Issues 8 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

docs: master/agent phase 2 — jobs and manual update (#7)

Browse Source
This commit is contained in:
shcizo
2026-10-12 18:48:53 +02:00
parent b392e11b37
commit b7e25bb1fb
3 changed files with 160 additions and 164 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+84 -95
View File
@@ -1,121 +1,110 @@
# package-updater
Webhook-driven Docker Compose service updater. Fills the gap between Watchtower (polling, no CI integration) and full GitOps (Argo CD, Flux) for a self-hosted, single-host environment.
Updates self-hosted Docker Compose and Docker Swarm services from one place. A
**master** (web UI behind OIDC login, state in Postgres) shows every Compose stack,
Swarm service and container on every machine, and starts updates when someone clicks
*Update*. One **agent** per machine reports its inventory and runs the jobs. Both are in
the same image: `updater-master` and `updater-agent` (the default entrypoint).
**Trigger flow:**
Design: [master/agent spec](docs/superpowers/specs/2026-10-12-master-agent-design.md).
Phase 2 plan: [jobs and manual update](docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md).
1. Gitea workflow builds and pushes a new image to your registry.
2. Workflow calls `POST /update` on this service with the image name.
3. Service finds the matching Compose-managed container(s) on the host via Docker labels.
4. Runs `docker compose pull` + `up -d` for the relevant service(s).
> **Webhook:** the old `POST /update` endpoint is gone. It returns in phase 3 on the
> master; until then the Gitea action in `gitea-action/` has nothing to call.
See [design spec](docs/superpowers/specs/2026-05-22-package-updater-design.md) and [implementation plan](docs/superpowers/plans/2026-05-22-package-updater-implementation.md) for full design and rationale.
## What an update does
## How it finds the right stack
Clicking *Update* on a service re-pulls the service's **current tag** (`repo:tag`). The
agent asks its Docker daemon for the tag's digest and deploys `repo:tag@digest`, so what
runs is exactly what was pulled. The log streams to the browser while it runs; job
history is kept in Postgres. A service without a tag (`repo@digest`), a plain container,
and the updater's own services cannot be updated, and the UI says why. One update at a
time per service, and one job at a time per agent.
The service queries the Docker socket and reads the labels Compose itself attaches to every container:
## Deploying the master
- `com.docker.compose.project`
- `com.docker.compose.service`
- `com.docker.compose.project.working_dir`
- `com.docker.compose.project.config_files`
Use [`deploy/master.stack.yml`](deploy/master.stack.yml) as a starting point (Swarm stack;
needs Postgres and an OIDC provider). The master listens for the UI on `PORT` (default
`8080`, put it behind your reverse proxy) and for agents on `AGENT_PORT` (default `8090`,
pinned TLS, reached directly on the internal network).
A container is eligible for update only if it has **both**:
## Adding agents
- An image name matching the request (tag-agnostic), AND
- The opt-in label `se.shcizo.auto-update=true`.
Defense in depth: a valid bearer token AND the opt-in label must both be present before any container is touched.
## Swarm mode
Set `MODE=swarm` to update Docker Swarm services instead of Compose stacks. The
service runs `docker service update --image` via the Docker API instead of
shelling out to `docker compose`.
Two things to get right when running in Swarm mode:
- **The opt-in label goes on the service, not the container/task.** Swarm mode
reads `Service.Spec.Labels`, so add it with
`docker service update --label-add se.shcizo.auto-update=true <service>` or
set it under `deploy.labels` in the stack file — a plain `labels:` entry on
the service (container-level) is not visible to Swarm mode's discovery.
- **The updater must talk to a Swarm manager.** `docker service update`
requires manager API access, so either point `DOCKER_HOST` at a manager node
or schedule the updater container itself on a manager with the socket
mounted. This is a deployment concern the service cannot detect or work
around.
## Master/agent (in progress)
A master with one agent per machine is replacing the single-instance webhook
model; see the [design spec](docs/superpowers/specs/2026-10-12-master-agent-design.md).
Phase 1 delivers a live inventory: every Compose stack, Swarm service and
container on every machine, behind Authentik login. Updates still go through
the existing `/update` webhook until phase 3.
- Master: `updater-master` in the same image, example stack in `deploy/master.stack.yml`.
At startup it does OIDC discovery against `OIDC_ISSUER` and exits if the issuer
is unreachable; Swarm then restarts it.
- Agents: add one per machine on the master's **Agents** page; it shows a
ready-to-run compose file with the agent's token and the master's key
fingerprint.
## Quick start
1. Build and push the image (e.g. via your own CI).
2. Copy `docker-compose.example.yml` to `/home/shcizo/self-hosted/package-updater/docker-compose.yml`.
3. Create `.env` next to it: `UPDATER_API_KEY=$(openssl rand -hex 32)`.
4. Point your reverse proxy (NPM/Traefik/Caddy) at `package-updater:8080`. NPM should handle TLS.
5. `docker compose up -d`.
6. Add the opt-in label `se.shcizo.auto-update: "true"` to each service you want auto-updated.
7. Use the [Gitea composite action](gitea-action/README.md) in your repos to call `/update` after a build.
## Configuration
All via environment variables.
Open the master's **Agents** page and add one. It shows a ready-to-run compose file with
the agent's token (shown once) and the master's key fingerprint; the same layout is in
[`deploy/agent.compose.yml`](deploy/agent.compose.yml).
| Variable | Required | Default | Purpose |
|---|---|---|---|
| `UPDATER_API_KEY` | **yes** | — | Bearer token. Service refuses to start without it. |
| `STACKS_ROOT` | no | `/home/shcizo/self-hosted` | Required parent for any stack eligible to update. |
| `PORT` | no | `8080` | HTTP listen port. |
| `LOG_LEVEL` | no | `info` | `debug` / `info` / `warn` / `error`. |
| `MASTER_URL` | yes | — | `wss://host:port/agent` |
| `MASTER_KEY` | yes | — | Master key fingerprint (`sha256:...`) shown in the UI. |
| `AGENT_TOKEN` | yes | — | Token shown once when the agent was added. |
| `SELF_ID` | no | hostname | Container ID or name of the agent itself; used to read its mounts. |
| `UPDATE_TIMEOUT` | no | `5m` | Per-job timeout (Go duration). |
| `OPT_IN_LABEL` | no | `se.shcizo.auto-update` | Label name to check; value must equal `"true"`. |
| `MODE` | no | `compose` | `compose` or `swarm`. Selects the update mechanism for the whole deployment; not mixed per-request. |
| `DOCKER_CONFIG` | no | — | Directory with a `config.json` holding registry credentials. Only inline `auths` entries work; `credsStore`/`credHelpers` are not available in the container. |
| `LOG_LEVEL` | no | `info` | `debug` / `info` / `warn` / `error`. |
## Endpoints
### Mounting stack directories
| Endpoint | Auth | Purpose |
|---|---|---|
| `POST /update` | Bearer token | Trigger pull + restart for matching services |
| `GET /healthz` | none | Liveness + Docker socket reachability |
| `GET /version` | none | Build info |
| `GET /metrics` | none | Prometheus exposition |
There is no `STACKS_ROOT`. The agent may only touch Compose stacks in directories that
are bind-mounted into its own container **read-write at the same path as on the host**
(Compose labels hold host paths). Add one line per directory:
`/healthz`, `/version`, and `/metrics` are intentionally unauthenticated — they're internal-network only behind the reverse proxy.
```yaml
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /home/shcizo/self-hosted:/home/shcizo/self-hosted
```
## Observability
Read-only mounts and mounts whose source differs from the destination are refused with a
clear message. On Docker Desktop and OrbStack bind sources are translated host paths, so
Compose jobs fail there; that is a development-only limitation.
- **Logs**: JSON to stdout, picked up by Promtail/Alloy → Loki.
- **Metrics**: Prometheus exposition on `/metrics`. Notable: `package_updater_update_jobs_total{project,service,status}`, `package_updater_last_update_timestamp{project,service}`, `package_updater_docker_ping_up`.
## How Compose updates work, and their limits
The agent never edits your files. It writes one file next to the stack's base file,
`compose.override.yaml`, with a header line and an `image:` entry per updated service
(`repo:tag@digest`), then runs `docker compose pull` and `up -d` for that service with
the override passed explicitly.
- The agent only writes `compose.override.yaml` if it is absent or starts with its header
line, and **refuses the job** if any other auto-loaded override exists
(`compose.override.yml`, `docker-compose.override.yml`, `docker-compose.override.yaml`):
Compose loads only one of them, so ours would silently disable yours.
- Stacks you start with explicit `-f` files do not load the override on a manual
`docker compose up`; the agent's own jobs always pass it.
- **The override is never pruned.** If you delete a service from the stack file, its
`image:` entry stays in `compose.override.yaml`, and a manual `docker compose up -d`
would recreate it. Remove the entry by hand.
**Check before enabling.** List the config files of every running stack to spot existing
override files and stacks started with explicit `-f`:
```bash
docker ps --format '{{.Label "com.docker.compose.project.config_files"}}' | sort -u
```
## Swarm updates
The agent updates the service with `ServiceUpdate` and the pinned image. "Succeeded"
means the daemon accepted the new spec, not that the tasks converged; the inventory shows
`degraded` until they do. The agent doing the update must talk to a manager node (run it
on a manager, or point `DOCKER_HOST` at one).
## Upgrading from the single-instance updater
This release removes `cmd/server` and changes the image's default entrypoint to
`updater-agent`. A running old-style updater (default entrypoint, `UPDATER_API_KEY`) will
crash-loop the next time its image is pulled. Pin it to the last pre-phase-2 tag first.
Master and agents must be on this release together: a phase-1 agent connects but every
job sent to it fails "agent too old for updates".
## Development
```bash
go test ./...
go build ./cmd/server
go test -race ./internal/agent/... ./internal/master/... ./internal/e2e/
go build ./cmd/master ./cmd/agent
docker build -t package-updater:dev .
DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
```
## Known v1 gaps
These are tracked in the spec's section 2 and section 15 as deliberate out-of-scope:
- **Self-update wiring**: `internal/selfupdate.Wrapped` exists and is unit-tested but is not wired into the live queue. The HTTP response flush ordering for self-replacement is a future enhancement; for now, expect to manually rerun `docker compose up -d` on the host if pushing a new image of `package-updater` itself causes a mid-response interruption.
- **No rollback**: Compose's "keep old container if new fails to start" is the only safety net.
- **Single host only in Compose mode**. Swarm mode (`MODE=swarm`) is the
multi-node path, but only from a manager node's point of view — the updater
itself still needs manager API access (see "Swarm mode" above).
- **No per-repo API keys**: a single shared bearer token is used.