shcizo
b1d6e5e705
Clear available_* only when the deployed ref equals it, drop the unused poll source, stop leaking internal errors to CI, use the request-scoped logger, validate X-Request-ID, re-read jobs once after the timeout, report refused when no target ref exists, and tidy UI, deploy and docs.
149 lines
12 KiB
Markdown
149 lines
12 KiB
Markdown
# Master/Agent Phase 3 — Webhook and Auto Mode Implementation Plan
|
|
|
|
> This plan documents what was built for issue #8 and where it deviates from the spec. Code in "Frozen contracts" is final; tasks give the contract and the test cases.
|
|
|
|
**Goal:** CI calls `POST /update` on the master after pushing an image. The master records the pushed version as *available* on every service that uses the image. Services in `auto` mode are updated at once and the request waits for the result, so CI sees a broken deploy. People switch a service between `manual` and `auto` in the UI.
|
|
|
|
**Architecture:** A new package `internal/master/webhook` is an `http.Handler` mounted on the master's mux outside the OIDC login; it does its own bearer-token check. It looks up services by image repo, writes `available_*` on each match, starts a job through `jobs.Service.Start` for the `auto` ones and waits for them. Waiting subscribes to the in-process broker before any job exists, reacts to `events.Jobs`, re-reads each pending job from Postgres on every signal and on a fallback ticker, and gives up at `UPDATE_TIMEOUT`. `FinishJob` clears `available_*` in the transaction that records a successful deploy.
|
|
|
|
**Tech Stack:** unchanged. No new dependencies.
|
|
|
|
**Spec:** `docs/superpowers/specs/2026-10-12-master-agent-design.md`, "Webhook" section. Phase 2 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md` (its conventions still apply).
|
|
|
|
## Global Constraints
|
|
|
|
- `go test ./...` passes with no Docker daemon and no Postgres. Postgres tests carry `//go:build integration` and need `DATABASE_URL`. Run `go test -race ./internal/agent/... ./internal/master/... ./internal/e2e/`.
|
|
- Interfaces are declared in the consuming package. Hand-written fakes, `stretchr/testify`, table-driven where there is more than one case.
|
|
- Every exported identifier gets a GoDoc comment. Conventional Commits; every commit message ends with ` (#8)`.
|
|
- The token (`UPDATER_API_KEY`) never appears in logs.
|
|
- Every state-changing UI request is a POST carrying the CSRF token (the mode toggle included).
|
|
- The HTTP server for the UI has no `WriteTimeout`, so a webhook request can wait up to `UPDATE_TIMEOUT`.
|
|
|
|
## Deliberate deviations from the spec in this phase
|
|
|
|
- **No `image_versions` table and no `available_digest` column.** Both belong to phase 4 (version history and the version picker). Phase 3 stores only `available_ref`, `available_source` (`webhook`) and `available_at` on `services`.
|
|
- **The old `207` is gone.** A request with both a working and a failed service answers `500` as a whole; there is no partial-success code. The Gitea action fails on any 4xx/5xx, so CI surfaces it.
|
|
- **`busy` and `refused` are not failures.** A service that already has an active job (`busy`, with that job's ID, not waited for) or may not be updated (`refused`, with the reason) does not make the request fail. Only `failed` and `timeout` do. A manual match is `marked`, also not a failure.
|
|
- **A webhook with a `tag` bypasses the "follows no tag" refusal.** The target is then explicit (`<service's image_repo>:<tag>`), so a `repo@digest` service can be moved to a tag. Removed, self, built and container refusals still apply. Without a `tag`, the target is the followed tag (bare image = `:latest`); a service that follows no tag is left alone (`available_*` untouched) and is reported `refused` ("no tag given and the service follows no tag"), manual or auto. Because of this, the UI allows `auto` for a service that follows no tag (`jobs.AutoAllowed`), unlike the Update button (`jobs.Updatable`).
|
|
- **The target ref is built from the service's own repo.** `imageref.Parse(req.Image).Repo` only selects services; it never reaches the job, so a request cannot point a service at another registry.
|
|
- **Token is optional on the master.** Unset `UPDATER_API_KEY` means `POST /update` answers `503` and startup logs `webhook_disabled`; the master still runs.
|
|
- **`tag` is validated** against Docker's tag grammar (`400 invalid tag`). The spec does not say; without it a tag such as `v1@sha256:x` or one with spaces would end up in the override file and fail on the agent.
|
|
- **An oversized body (over 64 KiB) answers `400 invalid JSON body`**, not `413`.
|
|
- **`UPDATE_TIMEOUT` exists on both sides** with the same name and parsing: on the agent it bounds one job, on the master it bounds one webhook request. The master never fails or cancels a job that outlives the request; the agent's own timeout does.
|
|
- **Marking and starting jobs runs on `context.WithoutCancel(r.Context())`.** A client that disconnects stops only the waiting; otherwise a queued job could be left that no agent knows about.
|
|
|
|
## Review Focus
|
|
|
|
| Failure case | Test that covers it |
|
|
|---|---|
|
|
| Wrong, missing or malformed token never reaches the lookup; unset key answers 503. | webhook `TestAuth` |
|
|
| An unknown or differently-cased image answers 404 and starts nothing. | webhook `TestNoMatch` |
|
|
| The image string is normalised (tag, digest, registry with port) before matching. | webhook `TestImageIsNormalisedBeforeMatching` |
|
|
| The auto job's target is built from the service's repo and the request tag, never the request's image string. | webhook `TestAutoJobTargetIsBuiltFromServiceRepo` |
|
|
| A job that fails at start (agent offline) is reported at once, not waited for. | webhook `TestJobFailedAtStartIsNotWaitedFor` |
|
|
| A dropped broker signal is recovered by the fallback poll. | webhook `TestDroppedSignalIsRecoveredByPolling` |
|
|
| A timed-out job is reported `timeout` and left untouched. | webhook `TestTimeout` |
|
|
| A client that disconnects does not abort marking or starting. | webhook `TestClientCancelStopsWaitingButNotStarting` |
|
|
| The webhook waits for the real agent's result and answers 200 / 500. | e2e `TestWebhookWaitsForAutoJob`, `TestWebhookFailedAutoJobIs500` |
|
|
| The available ref is cleared only by a successful job whose `to_ref` equals it and that was created at or after the suggestion; a different ref, a newer suggestion and a failed job keep it. | store `TestFinishJob_ClearsAvailableOnlyWhenThatVersionWasDeployed` |
|
|
| Re-reporting inventory never resets a service's update mode; new services are manual. | store `TestUpsertKeepsUpdateMode` |
|
|
| `auto` is refused for removed, self, built and container services; `manual` is always allowed. | jobs `TestAutoAllowed`; web `TestSetMode_AutoRefusedShows409`, `TestSetMode_ManualAlwaysAllowed` |
|
|
|
|
## Frozen contracts
|
|
|
|
### C1. Migration `internal/master/store/migrations/0003_webhook.sql` (complete)
|
|
|
|
```sql
|
|
ALTER TABLE services ADD COLUMN available_ref text NOT NULL DEFAULT '';
|
|
ALTER TABLE services ADD COLUMN available_source text NOT NULL DEFAULT ''
|
|
CHECK (available_source IN ('', 'webhook'));
|
|
ALTER TABLE services ADD COLUMN available_at timestamptz;
|
|
|
|
CREATE INDEX services_by_image_repo ON services (image_repo) WHERE removed_at IS NULL;
|
|
```
|
|
|
|
### C2. `internal/master/model` additions
|
|
|
|
`Service` gains `AvailableRef string`, `AvailableSource string`, `AvailableAt *time.Time`. Constants `ModeManual`, `ModeAuto`, `SourceWebhook`. `type Available struct { Ref, Source string; At time.Time }`.
|
|
|
|
### C3. Store methods
|
|
|
|
```go
|
|
ServicesByRepo(ctx, repo string) ([]model.Service, error) // non-removed, same order as ListServices
|
|
SetAvailable(ctx, id int64, a model.Available) error
|
|
SetUpdateMode(ctx, id int64, mode string) error // model.ErrNotFound for unknown or removed
|
|
```
|
|
|
|
`FinishJob` additionally reads the job's `created_at` and, on success, clears `available_*` in the same transaction when the job's `to_ref` (as created, without `@digest`) equals `available_ref` and `available_at <= created_at`.
|
|
|
|
### C4. `internal/master/jobs`
|
|
|
|
```go
|
|
type StartRequest struct {
|
|
Trigger model.JobTrigger
|
|
RequestedBy string
|
|
Ref string // empty: the service's followed tag; set: used as is
|
|
}
|
|
func (s *Service) Start(ctx, svc model.Service, req StartRequest) (model.Job, error)
|
|
func (s *Service) AutoAllowed(svc model.Service) error
|
|
func FollowedRef(actualRef string) string // "repo:tag", or "" when no tag is followed
|
|
```
|
|
|
|
`Start` has the return contract `Trigger` had: `*model.NotUpdatableError` with no job, `model.ErrJobActive` with the existing job, and an already-failed job for offline / too-old / start failures. `Trigger(ctx, id, by)` is `GetService` plus `Start` with `TriggerManual`.
|
|
|
|
### C5. `internal/master/webhook`
|
|
|
|
```go
|
|
type Config struct { Token string; Timeout time.Duration; PublicURL string }
|
|
func New(store Store, jobs Jobs, broker Broker, cfg Config, log *slog.Logger) *Handler
|
|
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request)
|
|
```
|
|
|
|
Request `{"image","tag"}`; response `{"request_id","image","tag","matched","results":[{"service_id","service","kind","mode","status","job_id","job_url","to_ref","error","duration_ms"}]}`. Statuses `succeeded | failed | timeout | marked | busy | refused`; HTTP `500` when any result is `failed` or `timeout`, else `200`. Other codes: `400`, `401`, `404`, `503`. See the README for the full table.
|
|
|
|
### C6. Master configuration
|
|
|
|
`UPDATER_API_KEY` (optional) and `UPDATE_TIMEOUT` (default `5m`, positive Go duration, same error text style as the agent's).
|
|
|
|
### C7. Web
|
|
|
|
`POST /services/{id}/mode` with `mode=manual|auto` and optional `return=overview`. `303` to the service page (or `/`), `400` for a bad mode, `404` for an unknown or removed service, `409` re-rendering the service page with "Cannot enable auto: <reason>." Publishes `events.Services`.
|
|
|
|
## Tasks
|
|
|
|
### Task 1: Store and model
|
|
|
|
Migration C1, model C2, store C3. Integration tests (`//go:build integration`): `ServicesByRepo` excludes removed rows and other repos; `SetAvailable` round trip; `SetUpdateMode` including `ErrNotFound`; new service is manual and upsert/snapshot keep the mode; `FinishJob` clears the same ref, keeps a different ref, keeps newer, keeps on failure.
|
|
|
|
### Task 2: Jobs service
|
|
|
|
Extract `Start` from `Trigger`; `updatable(svc, explicitRef)`; `AutoAllowed`; `FollowedRef`. Tests: webhook trigger with explicit ref is recorded; explicit ref bypasses only the no-tag refusal; `AutoAllowed` table; existing `TestTrigger*` stay green.
|
|
|
|
### Task 3: Config, webhook, wiring
|
|
|
|
Config C6 with tests. Handler C5 with fakes and the real `events.Broker`. `cmd/master/main.go` mounts `POST /update` directly on the mux and logs `webhook_disabled` when the key is empty.
|
|
|
|
### Task 4: Web
|
|
|
|
Store gains `SetUpdateMode`, Jobs gains `AutoAllowed`. Overview: mode column with a toggle form (hidden `csrf_token`, `mode`, `return=overview`; disabled with the reason as visible text when auto is not allowed) and an "update available" badge. Service page: Available row, Mode row, toggle form, 409 error. The forms are inside the `services` template that the SSE fragment re-renders, so they survive a refresh.
|
|
|
|
### Task 5: End-to-end
|
|
|
|
`internal/e2e/webhook_test.go`: `TestWebhookWaitsForAutoJob` (real in-process master and agent, gate executor, the handler on an `httptest` server) and `TestWebhookFailedAutoJobIs500`. The in-memory `memStore` gains `ServicesByRepo`, `SetAvailable` and the same clearing rule as the SQL.
|
|
|
|
### Task 6: Documentation
|
|
|
|
README (webhook section, master variables, warning about `auto`), `gitea-action/README.md` (do not change `action.yml`), `deploy/master.stack.yml` (`UPDATER_API_KEY` as a Docker secret), `CLAUDE.md`, this plan.
|
|
|
|
## Final verification
|
|
|
|
```bash
|
|
gofmt -l . ; go vet ./...
|
|
go build ./cmd/master ./cmd/agent && rm -f master agent
|
|
go test ./...
|
|
go test -race ./internal/agent/... ./internal/master/... ./internal/e2e/
|
|
DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/ # when a Postgres is available
|
|
```
|
|
|
|
Manual smoke test: set `UPDATER_API_KEY`, make a throwaway Compose service `auto`, then `curl -X POST -H "Authorization: Bearer $KEY" -d '{"image":"<repo>","tag":"<pushed tag>"}' https://<master>/update`. Expect `200` with `succeeded` after the job finishes, the job visible in the UI with trigger `webhook`, and the "update available" badge gone.
|