shcizo/package-updater
1
0
Fork 0
Code Issues 8 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat: master/agent phase 3 — webhook and auto mode (#8) (#22)
Docker Build / docker (push) Successful in 38s
Details

Browse Source 
## Summary
- `POST /update` on the master (outside OIDC, bearer `UPDATER_API_KEY`; unset → 503). Matches `image` against `image_repo` on all agents, marks `available_*` on every match, starts webhook-triggered jobs for `auto` services targeting `<image_repo>:<tag>` (followed tag when no tag is given), and waits for them up to `UPDATE_TIMEOUT` (default 5m). 404 on no match, 500 if any auto job failed or timed out, otherwise 200.
- Manual/auto toggle per service on the overview and service page (`POST /services/{id}/mode`, CSRF); auto is blocked for self, `build:` and container services with the reason shown. "Update available" badge and an "Available" row.
- `FinishJob` clears `available_*` only when the successful job was created for that exact ref, after it was reported. Migration `0003_webhook.sql`, phase-3 plan doc, README / gitea-action README / CLAUDE.md / deploy example updated. `action.yml` unchanged.

Closes #8

## Behaviour changes for CI
- Mixed results now return 500 (the old 207 is gone); no match returns 404 instead of 200.
- With a `tag`, Compose services get pinned to `<image_repo>:<tag>` via `compose.override.yaml`; the tag must be a pushed tag.
- `busy` (job already active) and `refused` (not updatable) are reported but return 200.
- The reverse proxy read timeout must exceed `UPDATE_TIMEOUT`.

## Test plan
- [x] `go test ./...` and `go test -race ./internal/agent/... ./internal/master/... ./internal/e2e/`
- [x] `go test -tags integration ./internal/master/store/` against throwaway Postgres
- [x] UI verified in browser (mode toggle, disabled reason, badge via SSE, Available row)
- [ ] Manual: webhook with manual service → `marked` + badge
- [ ] Manual: webhook with auto service → waits, `succeeded`, badge cleared, override pinned
- [ ] Manual: wrong token 401, unknown image 404, offline agent 500
- [ ] Gitea action pointed at the master's `/update` in a real CI run

Reviewed-on: #22
This commit was merged in pull request #22.
This commit is contained in:
shcizo
2026-10-12 20:03:22 +00:00
parent 417ef315c2
commit 41a619b316
27 changed files with 2015 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+90 -8
View File
@@ -8,9 +8,7 @@ the same image: `updater-master` and `updater-agent` (the default entrypoint).
Design: [master/agent spec](docs/superpowers/specs/2026-10-12-master-agent-design.md).
Phase 2 plan: [jobs and manual update](docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md).
> **Webhook:** the old `POST /update` endpoint is gone. It returns in phase 3 on the
> master; until then the Gitea action in `gitea-action/` has nothing to call.
Phase 3 plan: [webhook and auto mode](docs/superpowers/plans/2026-10-12-master-agent-phase-3-webhook.md).
## What an update does
@@ -20,9 +18,9 @@ runs is exactly what was pulled. The log streams to the browser while it runs; j
history is kept in Postgres. A service without a tag (`repo@digest`), a plain container,
and the updater's own services cannot be updated, and the UI says why. A bare image
(`image: postgres`, no tag and no digest) counts as `postgres:latest`; only a reference
with a digest and no tag (`repo@sha256:...`) "follows no tag". Compose services with a `build:` section are shown but never updated
("built from source (build:)"). One update at a
time per service, and one job at a time per agent.
with a digest and no tag (`repo@sha256:...`) "follows no tag". Compose services with a
`build:` section are shown but never updated ("built from source (build:)"). One update at
a time per service, and one job at a time per agent.
## Deploying the master
@@ -31,6 +29,89 @@ needs Postgres and an OIDC provider). The master listens for the UI on `PORT` (d
`8080`, put it behind your reverse proxy) and for agents on `AGENT_PORT` (default `8090`,
pinned TLS, reached directly on the internal network).
| Variable | Required | Default | Purpose |
|---|---|---|---|
| `UPDATER_API_KEY` | no | — | Bearer token for `POST /update`. Without it the webhook answers `503` and the master logs `webhook_disabled`. |
| `UPDATE_TIMEOUT` | no | `5m` | How long one `POST /update` waits for its auto jobs (Go duration). |
The master's other settings (`DATABASE_URL`, `PUBLIC_URL`, `AGENT_URL`, `OIDC_*`,
`ALLOWED_GROUP`, `AGENT_IMAGE`, `PORT`, `AGENT_PORT`, `LOG_LEVEL`) are shown in
[`deploy/master.stack.yml`](deploy/master.stack.yml).
## Webhook (CI)
CI calls the master after pushing an image. The Gitea action in
[`gitea-action/`](gitea-action/README.md) does this; any HTTP client works.
```
POST /update
Authorization: Bearer <UPDATER_API_KEY>
Content-Type: application/json
{"image": "registry.example.com/myapp", "tag": "1.5.0"}
```
`image` is matched against the image repo of every service on every agent (tag and digest
in the request are ignored for matching; matching is case-sensitive). `tag` is optional.
For every match the master records the version as *available* (the overview shows an
"update available" badge), including services that cannot be updated automatically
(updater's own, `build:` and plain containers), so the badge shows for them too. Services
in **manual** mode stop there. Services in **auto**
mode also get an update job, and the request waits for those jobs to finish, at most
`UPDATE_TIMEOUT`.
**`tag` semantics.** The target of an auto job is `<service's image repo>:<tag>`, built
from the service's own repo and never from the request's `image` string. The tag must be
one that has been pushed. Compose services get pinned to it through
`compose.override.yaml`, as in any update. Omit `tag` to re-pull the tag the service
already follows (a bare image counts as `:latest`); a service that follows no tag
(`repo@digest`) is then not touched and is reported as `refused` ("no tag given and the
service follows no tag"), manual or auto.
With a `tag`, the "follows no tag" refusal does not apply; the other refusals (the
updater's own services, `build:` services, plain containers) still do.
Response (`200` or `500`):
```json
{"request_id": "…", "image": "registry.example.com/myapp", "tag": "1.5.0", "matched": 2,
"results": [
{"service_id": 12, "service": "myapp/web", "kind": "compose", "mode": "auto",
"status": "succeeded", "job_id": 42, "job_url": "https://updater.example.com/jobs/42",
"to_ref": "registry.example.com/myapp:1.5.0", "duration_ms": 8210},
{"service_id": 15, "service": "myapp/worker", "kind": "compose", "mode": "manual",
"status": "marked", "to_ref": "registry.example.com/myapp:1.5.0", "duration_ms": 3}]}
```
| `status` | Meaning | Counts as failure |
|---|---|---|
| `succeeded` | Auto job finished successfully. | no |
| `failed` | Auto job failed (for example agent offline), or the version could not be recorded. `error` says why. | yes |
| `timeout` | Job still running when `UPDATE_TIMEOUT` passed. The master leaves it running. | yes |
| `marked` | Manual service; the version was recorded only. | no |
| `busy` | The service already has a queued or running job; `job_id` is that job, which is not waited for. | no |
| `refused` | The service may not be updated, or there is no target (no `tag` given and the service follows no tag). `error` has the reason. | no |
| Code | When |
|---|---|
| `200` | No result is `failed` or `timeout`. |
| `400` | Body is not JSON, `image` is missing, or `tag` is not a valid Docker tag. |
| `401` | Missing or wrong bearer token. |
| `404` | No service uses the image (catches typos in CI configuration). |
| `500` | At least one result is `failed` or `timeout`. |
| `503` | `UPDATER_API_KEY` is not set on the master. |
The webhook never creates services: the first deployment of a stack is done by hand. A
new service always starts in **manual** mode; switch it with the *Make auto* button on the
overview or the service page. Auto is refused for the updater's own services, `build:`
services and plain containers.
> **Warning:** auto mode deploys whatever CI pushes, without anyone looking at it. Using
> it on critical services is your own risk.
The request is open for as long as the jobs run, so the reverse proxy in front of the
master must allow a read timeout longer than `UPDATE_TIMEOUT`. If the client disconnects,
marking and starting jobs still complete; only the waiting stops.
## Adding agents
Open the master's **Agents** page and add one. It shows a ready-to-run compose file with
@@ -115,8 +196,9 @@ crash-loop the next time its image is pulled. Anything that pulls `:latest` for
instance triggers this: its own `/update` webhook, Watchtower or Shepherd.
Pinning means two things: pin the old instance to the last pre-phase-2 tag **and** remove
its `se.shcizo.auto-update` label, so nothing updates it to the new image. Or do not tag a
release until phase 3 moves `POST /update` to the master.
its `se.shcizo.auto-update` label, so nothing updates it to the new image. The master
serves `POST /update` again (see [Webhook](#webhook-ci)); point the Gitea action's
`endpoint` there and set `UPDATER_API_KEY` on the master.
Master and agents must be on this release together: a phase-1 agent connects but every
job sent to it fails "agent too old for updates".