docs: webhook contract, auto mode and phase 3 plan (#8)

This commit is contained in:
2026-10-12 20:43:11 +02:00
parent 71d66846f2
commit f25a7a7779
5 changed files with 274 additions and 16 deletions
+18 -5
View File
@@ -37,6 +37,7 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
- `internal/master/store` — Postgres access and migrations
- `internal/master/hub` — agent connections: auth, per-connection writer, routing, job reconcile on hello
- `internal/master/jobs` — decides whether a service may be updated and starts a job on the right agent
- `internal/master/webhook` — `POST /update`: records the available version, starts jobs for auto services, waits for them
- `internal/master/web`, `webauth` — HTMX UI behind OIDC login, SSE signals
- `internal/master/events` — in-process publish/subscribe for UI refresh signals
- `internal/master/model` — domain types shared by the above
@@ -59,7 +60,10 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
- **Unknown message types are logged and ignored; malformed messages close the connection.**
- **Reconcile before routable**: the hub fails a reconnecting agent's jobs it does not
list in `known_jobs`, and only then accepts jobs for that connection.
- **Auth**: agents by token over pinned TLS; people by OIDC. Every state-changing UI
- **The webhook never creates services; new services always start `manual`.** The upsert
does not touch `update_mode`.
- **Auth**: agents by token over pinned TLS; people by OIDC; CI by the `UPDATER_API_KEY`
bearer token on `POST /update` only. Every state-changing UI
request is a POST with a CSRF token.
## Gotchas
@@ -89,7 +93,16 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
- **Swarm "succeeded" means the daemon accepted the new spec**, not that tasks converged.
- **Swarm updates need manager-node API access** for the agent doing the update.
- **The image's default entrypoint is `updater-agent`.** The master overrides it with
`updater-master`. There is no `POST /update` until phase 3.
`updater-master`.
- **The webhook waits via a broker signal plus a store re-read, with a fallback poll**
(the broker drops signals for slow subscribers). Marking and starting jobs runs on
`context.WithoutCancel`, so a client that disconnects cannot leave a queued job no agent
knows about; only the waiting stops. The master never fails or cancels a timed-out job.
- **A webhook job's target is `<image_repo>:<request tag>`**, built from the service's own
repo, never from the request's image string. With an explicit target the "follows no tag"
refusal does not apply; removed, self, built and container still do.
- **`available_*` is cleared by `FinishJob`** on success when `available_at <= job.created_at`
(a job created after the suggestion deployed the newest version).
- **Job output is capped at `protocol.MaxJobOutput` (128 KiB, tail).** NUL bytes are
stripped by the hub because Postgres text rejects them.
@@ -98,10 +111,10 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
- Master/agent spec: `docs/superpowers/specs/2026-10-12-master-agent-design.md` (authoritative)
- Phase 1 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-1-inventory.md`
- Phase 2 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md`
- Original single-instance design (historical, its webhook is gone until phase 3):
- Phase 3 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-3-webhook.md`
- Original single-instance design (historical):
`docs/superpowers/specs/2026-05-22-package-updater-design.md`
- Consumer-side CI integration: `gitea-action/` (calls `POST /update`, which no longer
exists until phase 3)
- Consumer-side CI integration: `gitea-action/` (calls the master's `POST /update`)
## Conventions
+84 -5
View File
@@ -8,9 +8,7 @@ the same image: `updater-master` and `updater-agent` (the default entrypoint).
Design: [master/agent spec](docs/superpowers/specs/2026-10-12-master-agent-design.md).
Phase 2 plan: [jobs and manual update](docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md).
> **Webhook:** the old `POST /update` endpoint is gone. It returns in phase 3 on the
> master; until then the Gitea action in `gitea-action/` has nothing to call.
Phase 3 plan: [webhook and auto mode](docs/superpowers/plans/2026-10-12-master-agent-phase-3-webhook.md).
## What an update does
@@ -31,6 +29,86 @@ needs Postgres and an OIDC provider). The master listens for the UI on `PORT` (d
`8080`, put it behind your reverse proxy) and for agents on `AGENT_PORT` (default `8090`,
pinned TLS, reached directly on the internal network).
| Variable | Required | Default | Purpose |
|---|---|---|---|
| `UPDATER_API_KEY` | no | — | Bearer token for `POST /update`. Without it the webhook answers `503` and the master logs `webhook_disabled`. |
| `UPDATE_TIMEOUT` | no | `5m` | How long one `POST /update` waits for its auto jobs (Go duration). |
The master's other settings (`DATABASE_URL`, `PUBLIC_URL`, `AGENT_URL`, `OIDC_*`,
`ALLOWED_GROUP`, `AGENT_IMAGE`, `PORT`, `AGENT_PORT`, `LOG_LEVEL`) are shown in
[`deploy/master.stack.yml`](deploy/master.stack.yml).
## Webhook (CI)
CI calls the master after pushing an image. The Gitea action in
[`gitea-action/`](gitea-action/README.md) does this; any HTTP client works.
```
POST /update
Authorization: Bearer <UPDATER_API_KEY>
Content-Type: application/json
{"image": "registry.example.com/myapp", "tag": "1.5.0"}
```
`image` is matched against the image repo of every service on every agent (tag and digest
in the request are ignored for matching; matching is case-sensitive). `tag` is optional.
For every match the master records the version as *available* (the overview shows an
"update available" badge). Services in **manual** mode stop there. Services in **auto**
mode also get an update job, and the request waits for those jobs to finish, at most
`UPDATE_TIMEOUT`.
**`tag` semantics.** The target of an auto job is `<service's image repo>:<tag>`, built
from the service's own repo and never from the request's `image` string. The tag must be
one that has been pushed. Compose services get pinned to it through
`compose.override.yaml`, as in any update. Omit `tag` to re-pull the tag the service
already follows (a bare image counts as `:latest`); a service that follows no tag
(`repo@digest`) is then not touched, and an auto service is reported as `refused`.
With a `tag`, the "follows no tag" refusal does not apply; the other refusals (the
updater's own services, `build:` services, plain containers) still do.
Response (`200` or `500`):
```json
{"request_id": "…", "image": "registry.example.com/myapp", "tag": "1.5.0", "matched": 2,
"results": [
{"service_id": 12, "service": "myapp/web", "kind": "compose", "mode": "auto",
"status": "succeeded", "job_id": 42, "job_url": "https://updater.example.com/jobs/42",
"to_ref": "registry.example.com/myapp:1.5.0", "duration_ms": 8210},
{"service_id": 15, "service": "myapp/worker", "kind": "compose", "mode": "manual",
"status": "marked", "to_ref": "registry.example.com/myapp:1.5.0", "duration_ms": 3}]}
```
| `status` | Meaning | Counts as failure |
|---|---|---|
| `succeeded` | Auto job finished successfully. | no |
| `failed` | Auto job failed (for example agent offline), or the version could not be recorded. `error` says why. | yes |
| `timeout` | Job still running when `UPDATE_TIMEOUT` passed. The master leaves it running. | yes |
| `marked` | Manual service; the version was recorded only. | no |
| `busy` | The service already has a queued or running job; `job_id` is that job, which is not waited for. | no |
| `refused` | The service may not be updated (`error` has the reason). | no |
| Code | When |
|---|---|
| `200` | No result is `failed` or `timeout`. |
| `400` | Body is not JSON, `image` is missing, or `tag` is not a valid Docker tag. |
| `401` | Missing or wrong bearer token. |
| `404` | No service uses the image (catches typos in CI configuration). |
| `500` | At least one result is `failed` or `timeout`. |
| `503` | `UPDATER_API_KEY` is not set on the master. |
The webhook never creates services: the first deployment of a stack is done by hand. A
new service always starts in **manual** mode; switch it with the *Make auto* button on the
overview or the service page. Auto is refused for the updater's own services, `build:`
services and plain containers.
> **Warning:** auto mode deploys whatever CI pushes, without anyone looking at it. Using
> it on critical services is your own risk.
The request is open for as long as the jobs run, so the reverse proxy in front of the
master must allow a read timeout longer than `UPDATE_TIMEOUT`. If the client disconnects,
marking and starting jobs still complete; only the waiting stops.
## Adding agents
Open the master's **Agents** page and add one. It shows a ready-to-run compose file with
@@ -115,8 +193,9 @@ crash-loop the next time its image is pulled. Anything that pulls `:latest` for
instance triggers this: its own `/update` webhook, Watchtower or Shepherd.
Pinning means two things: pin the old instance to the last pre-phase-2 tag **and** remove
its `se.shcizo.auto-update` label, so nothing updates it to the new image. Or do not tag a
release until phase 3 moves `POST /update` to the master.
its `se.shcizo.auto-update` label, so nothing updates it to the new image. The master
serves `POST /update` again (see [Webhook](#webhook-ci)); point the Gitea action's
`endpoint` there and set `UPDATER_API_KEY` on the master.
Master and agents must be on this release together: a phase-1 agent connects but every
job sent to it fails "agent too old for updates".
+9 -2
View File
@@ -1,4 +1,4 @@
# Example Swarm stack for the package-updater master (phase 2: inventory and manual updates).
# Example Swarm stack for the package-updater master (phase 3: inventory, updates and the CI webhook).
# Deploy on the manager: docker stack deploy -c master.stack.yml package-updater-master
#
# Before deploying:
@@ -6,19 +6,24 @@
# 2. In Authentik: create an OAuth2/OpenID provider + application with redirect URI
# https://updater.example.com/auth/callback, and a group "updater-admins".
# 3. docker secret create pu_oidc_client_secret - ; docker secret create pu_database_url -
# 4. Optional, enables POST /update (without it the endpoint answers 503):
# docker secret create pu_updater_api_key -
# The reverse proxy must allow a read timeout longer than UPDATE_TIMEOUT (default 5m).
services:
master:
image: gitea.shcizo.se/shcizo/package-updater:latest
entrypoint: ["/bin/sh", "-c", "DATABASE_URL=$$(cat /run/secrets/pu_database_url) OIDC_CLIENT_SECRET=$$(cat /run/secrets/pu_oidc_client_secret) exec /usr/local/bin/updater-master"]
entrypoint: ["/bin/sh", "-c", "DATABASE_URL=$$(cat /run/secrets/pu_database_url) OIDC_CLIENT_SECRET=$$(cat /run/secrets/pu_oidc_client_secret) UPDATER_API_KEY=$$(cat /run/secrets/pu_updater_api_key) exec /usr/local/bin/updater-master"]
environment:
PUBLIC_URL: https://updater.example.com
AGENT_URL: wss://10.0.0.5:8090/agent
OIDC_ISSUER: https://auth.example.com/application/o/package-updater/
OIDC_CLIENT_ID: package-updater
ALLOWED_GROUP: updater-admins
# UPDATE_TIMEOUT: 5m
secrets:
- pu_database_url
- pu_oidc_client_secret
- pu_updater_api_key
ports:
# Agents connect here directly over the internal network, not via the proxy.
- target: 8090
@@ -35,6 +40,8 @@ secrets:
external: true
pu_oidc_client_secret:
external: true
pu_updater_api_key:
external: true
networks:
proxy:
@@ -0,0 +1,148 @@
# Master/Agent Phase 3 — Webhook and Auto Mode Implementation Plan
> This plan documents what was built for issue #8 and where it deviates from the spec. Code in "Frozen contracts" is final; tasks give the contract and the test cases.
**Goal:** CI calls `POST /update` on the master after pushing an image. The master records the pushed version as *available* on every service that uses the image. Services in `auto` mode are updated at once and the request waits for the result, so CI sees a broken deploy. People switch a service between `manual` and `auto` in the UI.
**Architecture:** A new package `internal/master/webhook` is an `http.Handler` mounted on the master's mux outside the OIDC login; it does its own bearer-token check. It looks up services by image repo, writes `available_*` on each match, starts a job through `jobs.Service.Start` for the `auto` ones and waits for them. Waiting subscribes to the in-process broker before any job exists, reacts to `events.Jobs`, re-reads each pending job from Postgres on every signal and on a fallback ticker, and gives up at `UPDATE_TIMEOUT`. `FinishJob` clears `available_*` in the transaction that records a successful deploy.
**Tech Stack:** unchanged. No new dependencies.
**Spec:** `docs/superpowers/specs/2026-10-12-master-agent-design.md`, "Webhook" section. Phase 2 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md` (its conventions still apply).
## Global Constraints
- `go test ./...` passes with no Docker daemon and no Postgres. Postgres tests carry `//go:build integration` and need `DATABASE_URL`. Run `go test -race ./internal/agent/... ./internal/master/... ./internal/e2e/`.
- Interfaces are declared in the consuming package. Hand-written fakes, `stretchr/testify`, table-driven where there is more than one case.
- Every exported identifier gets a GoDoc comment. Conventional Commits; every commit message ends with ` (#8)`.
- The token (`UPDATER_API_KEY`) never appears in logs.
- Every state-changing UI request is a POST carrying the CSRF token (the mode toggle included).
- The HTTP server for the UI has no `WriteTimeout`, so a webhook request can wait up to `UPDATE_TIMEOUT`.
## Deliberate deviations from the spec in this phase
- **No `image_versions` table and no `available_digest` column.** Both belong to phase 4 (version history and the version picker). Phase 3 stores only `available_ref`, `available_source` (`webhook`; `poll` is reserved for phase 4) and `available_at` on `services`.
- **The old `207` is gone.** A request with both a working and a failed service answers `500` as a whole; there is no partial-success code. The Gitea action fails on any 4xx/5xx, so CI surfaces it.
- **`busy` and `refused` are not failures.** A service that already has an active job (`busy`, with that job's ID, not waited for) or may not be updated (`refused`, with the reason) does not make the request fail. Only `failed` and `timeout` do. A manual match is `marked`, also not a failure.
- **A webhook with a `tag` bypasses the "follows no tag" refusal.** The target is then explicit (`<service's image_repo>:<tag>`), so a `repo@digest` service can be moved to a tag. Removed, self, built and container refusals still apply. Without a `tag`, the target is the followed tag (bare image = `:latest`); a service that follows no tag is left alone (`available_*` untouched) and an auto one is `refused` with the no-tag reason. Because of this, the UI allows `auto` for a service that follows no tag (`jobs.AutoAllowed`), unlike the Update button (`jobs.Updatable`).
- **The target ref is built from the service's own repo.** `imageref.Parse(req.Image).Repo` only selects services; it never reaches the job, so a request cannot point a service at another registry.
- **Token is optional on the master.** Unset `UPDATER_API_KEY` means `POST /update` answers `503` and startup logs `webhook_disabled`; the master still runs.
- **`tag` is validated** against Docker's tag grammar (`400 invalid tag`). The spec does not say; without it a tag such as `v1@sha256:x` or one with spaces would end up in the override file and fail on the agent.
- **An oversized body (over 64 KiB) answers `400 invalid JSON body`**, not `413`.
- **`UPDATE_TIMEOUT` exists on both sides** with the same name and parsing: on the agent it bounds one job, on the master it bounds one webhook request. The master never fails or cancels a job that outlives the request; the agent's own timeout does.
- **Marking and starting jobs runs on `context.WithoutCancel(r.Context())`.** A client that disconnects stops only the waiting; otherwise a queued job could be left that no agent knows about.
## Review Focus
| Failure case | Test that covers it |
|---|---|
| Wrong, missing or malformed token never reaches the lookup; unset key answers 503. | webhook `TestAuth` |
| An unknown or differently-cased image answers 404 and starts nothing. | webhook `TestNoMatch` |
| The image string is normalised (tag, digest, registry with port) before matching. | webhook `TestImageIsNormalisedBeforeMatching` |
| The auto job's target is built from the service's repo and the request tag, never the request's image string. | webhook `TestAutoJobTargetIsBuiltFromServiceRepo` |
| A job that fails at start (agent offline) is reported at once, not waited for. | webhook `TestJobFailedAtStartIsNotWaitedFor` |
| A dropped broker signal is recovered by the fallback poll. | webhook `TestDroppedSignalIsRecoveredByPolling` |
| A timed-out job is reported `timeout` and left untouched. | webhook `TestTimeout` |
| A client that disconnects does not abort marking or starting. | webhook `TestClientCancelStopsWaitingButNotStarting` |
| The webhook waits for the real agent's result and answers 200 / 500. | e2e `TestWebhookWaitsForAutoJob`, `TestWebhookFailedAutoJobIs500` |
| The available ref is cleared only by a successful job created at or after the suggestion; a newer suggestion and a failed job keep it. | store `TestFinishJob_ClearsAvailableOnlyWhenJobIsNewer` |
| Re-reporting inventory never resets a service's update mode; new services are manual. | store `TestUpsertKeepsUpdateMode` |
| `auto` is refused for removed, self, built and container services; `manual` is always allowed. | jobs `TestAutoAllowed`; web `TestSetMode_AutoRefusedShows409`, `TestSetMode_ManualAlwaysAllowed` |
## Frozen contracts
### C1. Migration `internal/master/store/migrations/0003_webhook.sql` (complete)
```sql
ALTER TABLE services ADD COLUMN available_ref text NOT NULL DEFAULT '';
ALTER TABLE services ADD COLUMN available_source text NOT NULL DEFAULT ''
CHECK (available_source IN ('', 'webhook', 'poll'));
ALTER TABLE services ADD COLUMN available_at timestamptz;
CREATE INDEX services_by_image_repo ON services (image_repo) WHERE removed_at IS NULL;
```
### C2. `internal/master/model` additions
`Service` gains `AvailableRef string`, `AvailableSource string`, `AvailableAt *time.Time`. Constants `ModeManual`, `ModeAuto`, `SourceWebhook`, `SourcePoll`. `type Available struct { Ref, Source string; At time.Time }`.
### C3. Store methods
```go
ServicesByRepo(ctx, repo string) ([]model.Service, error) // non-removed, same order as ListServices
SetAvailable(ctx, id int64, a model.Available) error
SetUpdateMode(ctx, id int64, mode string) error // model.ErrNotFound for unknown or removed
```
`FinishJob` additionally reads the job's `created_at` and, on success, clears `available_*` in the same transaction when `available_at <= created_at`.
### C4. `internal/master/jobs`
```go
type StartRequest struct {
Trigger model.JobTrigger
RequestedBy string
Ref string // empty: the service's followed tag; set: used as is
}
func (s *Service) Start(ctx, svc model.Service, req StartRequest) (model.Job, error)
func (s *Service) AutoAllowed(svc model.Service) error
func FollowedRef(actualRef string) string // "repo:tag", or "" when no tag is followed
```
`Start` has the return contract `Trigger` had: `*model.NotUpdatableError` with no job, `model.ErrJobActive` with the existing job, and an already-failed job for offline / too-old / start failures. `Trigger(ctx, id, by)` is `GetService` plus `Start` with `TriggerManual`.
### C5. `internal/master/webhook`
```go
type Config struct { Token string; Timeout time.Duration; PublicURL string }
func New(store Store, jobs Jobs, broker Broker, cfg Config, log *slog.Logger) *Handler
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request)
```
Request `{"image","tag"}`; response `{"request_id","image","tag","matched","results":[{"service_id","service","kind","mode","status","job_id","job_url","to_ref","error","duration_ms"}]}`. Statuses `succeeded | failed | timeout | marked | busy | refused`; HTTP `500` when any result is `failed` or `timeout`, else `200`. Other codes: `400`, `401`, `404`, `503`. See the README for the full table.
### C6. Master configuration
`UPDATER_API_KEY` (optional) and `UPDATE_TIMEOUT` (default `5m`, positive Go duration, same error text style as the agent's).
### C7. Web
`POST /services/{id}/mode` with `mode=manual|auto` and optional `return=overview`. `303` to the service page (or `/`), `400` for a bad mode, `404` for an unknown or removed service, `409` re-rendering the service page with "Cannot enable auto: <reason>." Publishes `events.Services`.
## Tasks
### Task 1: Store and model
Migration C1, model C2, store C3. Integration tests (`//go:build integration`): `ServicesByRepo` excludes removed rows and other repos; `SetAvailable` round trip; `SetUpdateMode` including `ErrNotFound`; new service is manual and upsert/snapshot keep the mode; `FinishJob` clears, keeps newer, keeps on failure.
### Task 2: Jobs service
Extract `Start` from `Trigger`; `updatable(svc, explicitRef)`; `AutoAllowed`; `FollowedRef`. Tests: webhook trigger with explicit ref is recorded; explicit ref bypasses only the no-tag refusal; `AutoAllowed` table; existing `TestTrigger*` stay green.
### Task 3: Config, webhook, wiring
Config C6 with tests. Handler C5 with fakes and the real `events.Broker`. `cmd/master/main.go` mounts `POST /update` directly on the mux and logs `webhook_disabled` when the key is empty.
### Task 4: Web
Store gains `SetUpdateMode`, Jobs gains `AutoAllowed`. Overview: mode column with a toggle form (hidden `csrf_token`, `mode`, `return=overview`; disabled with the reason as title when auto is not allowed) and an "update available" badge. Service page: Available row, Mode row, toggle form, 409 error. The forms are inside the `services` template that the SSE fragment re-renders, so they survive a refresh.
### Task 5: End-to-end
`internal/e2e/webhook_test.go`: `TestWebhookWaitsForAutoJob` (real in-process master and agent, gate executor, the handler on an `httptest` server) and `TestWebhookFailedAutoJobIs500`. The in-memory `memStore` gains `ServicesByRepo`, `SetAvailable` and the same clearing rule as the SQL.
### Task 6: Documentation
README (webhook section, master variables, warning about `auto`), `gitea-action/README.md` (do not change `action.yml`), `deploy/master.stack.yml` (`UPDATER_API_KEY` as a Docker secret), `CLAUDE.md`, this plan.
## Final verification
```bash
gofmt -l . ; go vet ./...
go build ./cmd/master ./cmd/agent && rm -f master agent
go test ./...
go test -race ./internal/agent/... ./internal/master/... ./internal/e2e/
DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/ # when a Postgres is available
```
Manual smoke test: set `UPDATER_API_KEY`, make a throwaway Compose service `auto`, then `curl -X POST -H "Authorization: Bearer $KEY" -d '{"image":"<repo>","tag":"<pushed tag>"}' https://<master>/update`. Expect `200` with `succeeded` after the job finishes, the job visible in the UI with trigger `webhook`, and the "update available" badge gone.
+15 -4
View File
@@ -1,6 +1,6 @@
# Deploy via package-updater (composite action)
Notifies `package-updater` to `docker compose pull` + `up -d` for the matching service(s) after a CI build.
Notifies the `package-updater` master after a CI build. The master records the pushed version on every matching service and, for services in auto mode, updates them and waits for the result.
## Usage
@@ -20,7 +20,7 @@ jobs:
token: ${{ secrets.UPDATER_TOKEN }}
```
`UPDATER_TOKEN` should be set as an organisation-level secret in Gitea so all repos share it.
`endpoint` is the master's `/update`, and `UPDATER_TOKEN` is the master's `UPDATER_API_KEY`. It should be set as an organisation-level secret in Gitea so all repos share it.
## Inputs
@@ -28,9 +28,20 @@ jobs:
|---|---|---|---|
| `endpoint` | yes | — | Full URL to `/update` |
| `image` | yes | — | Image reference without tag |
| `tag` | no | `""` | Tag that was just pushed (logged for audit) |
| `tag` | no | `""` | Tag that was just pushed. Auto services are updated to `<their image repo>:<tag>`, so it must exist in the registry. Omit it to re-pull the tag each service already follows. |
| `token` | yes | — | Bearer token configured in package-updater |
## Failure modes
The step exits non-zero if `package-updater` returns HTTP 4xx or 5xx. This is intentional — the workflow surfaces the deploy failure to whoever pushed.
The step exits non-zero on any HTTP 4xx or 5xx. This is intentional: the workflow surfaces the deploy failure to whoever pushed.
| Code | Meaning |
|---|---|
| `200` | Every match was marked, updated, busy or refused. Manual services are only marked, which is not an error. |
| `401` | Wrong or missing token. |
| `404` | No service uses the image. Check `image` for typos; matching is case-sensitive. |
| `500` | At least one auto job failed or timed out (`UPDATE_TIMEOUT` on the master). |
| `503` | The master has no `UPDATER_API_KEY`. |
A request that matches both a working and a failed service now fails as a whole (`500`); the old single-instance updater's `207` no longer exists.
The master waits for auto jobs, so the request can take minutes; raise any step timeout in your workflow accordingly.