diff --git a/CLAUDE.md b/CLAUDE.md index bf4e34b..d00e89d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -36,6 +36,8 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/ - `internal/master/config` — env-var loading - `internal/master/store` — Postgres access and migrations - `internal/master/hub` — agent connections: auth, per-connection writer, routing, job reconcile on hello +- `internal/master/registry` — HEAD-only digest lookup against a registry (go-containerregistry, `DOCKER_CONFIG` credentials) +- `internal/master/poller` — every `POLL_INTERVAL`: one lookup per followed `repo:tag`, sets the available version, starts auto jobs - `internal/master/jobs` — decides whether a service may be updated and starts a job on the right agent - `internal/master/webhook` — `POST /update`: records the available version, starts jobs for auto services, waits for them - `internal/master/web`, `webauth` — HTMX UI behind OIDC login, SSE signals @@ -68,6 +70,16 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/ ## Gotchas +- **Registry access is HEAD only** (`internal/master/registry`). Docker Hub counts a manifest + GET as a pull; the client never falls back to GET, and a test asserts this. +- **`image_versions` uses `''` instead of NULL** for a missing tag or digest, so + `UNIQUE (repo, tag, digest)` deduplicates. The webhook records `(repo, tag, '')`; that + row is hidden from `RecentVersions` once a row with the same tag and a digest exists. +- **The UI picks a version by `image_versions.id`, never by a ref from the form**, and the row + must belong to the service's `image_repo`. A digest-pinned or tagless pick needs agent + protocol 3 (`protocol.MinPinnedRefVersion`); `Hub.Route` takes the minimum version. +- **Rolling back a service that follows no tag afterwards:** a pick of `repo@digest` makes the + service follow no tag, so it is no longer polled. - **Image matching is case-sensitive.** The tricky case is `localhost:5000/foo:v1` where the first colon is a port, not a tag. See `internal/imageref`. - **Path safety uses `filepath.Rel` + ".." prefix check**, NOT `strings.HasPrefix`. @@ -102,9 +114,10 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/ repo, never from the request's image string. With an explicit target the "follows no tag" refusal does not apply; removed, self, built and container still do. - **`available_*` is cleared by `FinishJob`** on success only when the job's `to_ref` - (as created, without `@digest`) equals `available_ref` and `available_at <= job.created_at` - (that version is now running). A manual Update of the followed tag does not clear a - badge for a different tag. + (as created, without `@digest`) equals `available_ref`, `available_at <= job.created_at` + (that version is now running) and, when the job's `to_ref` carries a digest, that digest + equals `available_digest`. A rollback to an older digest of the tag therefore keeps the + newer suggestion, and a digest-pinned pick does not clear a digest-less (webhook) one. - **Job output is capped at `protocol.MaxJobOutput` (128 KiB, tail).** NUL bytes are stripped by the hub because Postgres text rejects them. @@ -114,6 +127,7 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/ - Phase 1 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-1-inventory.md` - Phase 2 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md` - Phase 3 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-3-webhook.md` +- Phase 4 plan: `docs/superpowers/plans/2026-10-13-master-agent-phase-4-polling.md` - Original single-instance design (historical): `docs/superpowers/specs/2026-05-22-package-updater-design.md` - Consumer-side CI integration: `gitea-action/` (calls the master's `POST /update`) diff --git a/README.md b/README.md index dc79ae5..1fce50b 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,7 @@ the same image: `updater-master` and `updater-agent` (the default entrypoint). Design: [master/agent spec](docs/superpowers/specs/2026-10-12-master-agent-design.md). Phase 2 plan: [jobs and manual update](docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md). Phase 3 plan: [webhook and auto mode](docs/superpowers/plans/2026-10-12-master-agent-phase-3-webhook.md). +Phase 4 plan: [registry polling and version history](docs/superpowers/plans/2026-10-13-master-agent-phase-4-polling.md). ## What an update does @@ -33,6 +34,8 @@ pinned TLS, reached directly on the internal network). |---|---|---|---| | `UPDATER_API_KEY` | no | — | Bearer token for `POST /update`. Without it the webhook answers `503` and the master logs `webhook_disabled`. | | `UPDATE_TIMEOUT` | no | `5m` | How long one `POST /update` waits for its auto jobs (Go duration). | +| `POLL_INTERVAL` | no | `6h` | Time between registry polling cycles (Go duration, must be positive). The first cycle runs one minute after start. | +| `DOCKER_CONFIG` | no | `~/.docker` | Directory with a `config.json` holding registry credentials for polling. Only inline `auths` entries work; see [Registry polling](#registry-polling). | The master's other settings (`DATABASE_URL`, `PUBLIC_URL`, `AGENT_URL`, `OIDC_*`, `ALLOWED_GROUP`, `AGENT_IMAGE`, `PORT`, `AGENT_PORT`, `LOG_LEVEL`) are shown in @@ -112,6 +115,57 @@ The request is open for as long as the jobs run, so the reverse proxy in front o master must allow a read timeout longer than `UPDATE_TIMEOUT`. If the client disconnects, marking and starting jobs still complete; only the waiting stops. +## Registry polling + +Besides the webhook, the master asks each registry whether the tag a service follows has +a new digest. Every `POLL_INTERVAL` it sends one `HEAD /v2//manifests/` per +distinct followed `repo:tag` (a bare image counts as `:latest`; `repo:tag@digest` is polled +as `repo:tag`). It never downloads a manifest: Docker Hub counts a `GET` as a pull against +the rate limit and a `HEAD` as nothing +([source](https://docs.docker.com/docker-hub/usage/pulls/)), and Docker Hub also has a +separate abuse limit for many pulls from one IP address, so the poller stays HEAD-only. For a +multi-arch image the digest is the index digest, the same one `docker inspect` shows in +`RepoDigests`. + +- **What counts as new.** A digest different from the one the service runs becomes the + service's *available* version (the overview badge, source `poll`). A digest the master + already suggested is not suggested again. A service in **auto** mode also gets an update + job to `repo:tag` (the agent resolves the digest itself, so agents on protocol version 2 keep + working). A failed auto job is not retried every interval. +- **What is not polled.** Services that follow no tag (`repo@digest`), `build:` services, and + services without a registry digest (locally built images). The service page says which. + The updater's own services and plain containers are polled, so their badge works, but + jobs for them are still refused. +- **Errors.** A failed lookup is stored for the services that follow that tag and shown as a + "poll failed" badge on the overview and in the "Polling" row of the service page; other + tags continue. A later success clears it. +- **Credentials.** The master reads `$DOCKER_CONFIG/config.json` (a file written by + `docker login`), mounted read-only into the container; without it registries are accessed + anonymously. Only inline `auths` entries are supported. A `credsStore` or `credHelpers` + entry makes every lookup fail with `error getting credentials ... executable file not found + in $PATH`, because the container has no credential helper binaries. The master logs + `registry_credentials` at startup with the directory and whether `config.json` exists, + never its contents. + +### Version history and rollback + +The master records every version it sees (webhook, polling, and what agents report as +running) in `image_versions`. The service page lists the 10 most recently first seen +versions of the service's image repo, marks the one that is running, and has a *Deploy* +button on each row. A webhook announces a tag but not its digest, so such a row has no digest until polling or an +agent reports one; it is hidden once a row with the same tag and a digest exists. + +Picking a version starts a manual job to exactly that reference. Picking an older one is a +rollback, including to a digest-pinned reference. A version without a tag deploys `repo@digest` and the +service then follows no tag, so it is no longer polled or updatable with the plain *Update* button until +it is moved to a tagged version again. Picks of digest-pinned or tagless versions need an agent speaking protocol +version 3 (this release); older agents get a failed job "agent too old for updates". + +On a service in **auto** mode a rollback is only a warning in the UI: the service stays auto and +is moved forward again at the next new version, announced by the webhook or found by polling. +Rolling back to an older digest of the tag the service follows keeps the newer version marked as +available. + ## Adding agents Open the master's **Agents** page and add one. It shows a ready-to-run compose file with diff --git a/deploy/master.stack.yml b/deploy/master.stack.yml index 3c0e413..9f3cdc1 100644 --- a/deploy/master.stack.yml +++ b/deploy/master.stack.yml @@ -10,6 +10,9 @@ # To run without the webhook, remove the pu_updater_api_key lines (the UPDATER_API_KEY # part of the entrypoint, the service secret and the secrets entry); the endpoint then answers 503. # The reverse proxy must allow a read timeout longer than UPDATE_TIMEOUT (default 5m). +# 5. Optional, for registry polling of private images: put a config.json written by +# `docker login` (inline "auths" only; no credsStore) in /path/to/docker-config on the +# manager and keep the volumes entry below. Without it registries are accessed anonymously. services: master: image: gitea.shcizo.se/shcizo/package-updater:latest @@ -21,10 +24,14 @@ services: OIDC_CLIENT_ID: package-updater ALLOWED_GROUP: updater-admins # UPDATE_TIMEOUT: 5m + # POLL_INTERVAL: 6h + DOCKER_CONFIG: /docker-config secrets: - pu_database_url - pu_oidc_client_secret - pu_updater_api_key + volumes: + - /path/to/docker-config:/docker-config:ro ports: # Agents connect here directly over the internal network, not via the proxy. - target: 8090 diff --git a/docs/superpowers/plans/2026-10-13-master-agent-phase-4-polling.md b/docs/superpowers/plans/2026-10-13-master-agent-phase-4-polling.md new file mode 100644 index 0000000..e28dae2 --- /dev/null +++ b/docs/superpowers/plans/2026-10-13-master-agent-phase-4-polling.md @@ -0,0 +1,308 @@ +# Master/Agent Phase 4 — Registry Polling and Version History Implementation Plan + +> This plan documents what to build for issue #9 and where it deviates from the spec. Code in "Frozen contracts" is final; tasks give the contract and the test cases. + +**Goal:** The master records every version it sees (webhook, registry polling, running inventory) in `image_versions`. Every `POLL_INTERVAL` (default 6h) it checks each distinct followed `repo:tag` against its registry with one HEAD request. A new digest becomes the service's available suggestion, and `auto` services get a job. The Service page lists the 10 most recent versions of the service's repo. Picking one starts a manual job to exactly that reference; picking an older one is a rollback, including to a digest-pinned or tagless reference. + +**Architecture:** +- New package `internal/master/registry`: thin wrapper around `go-containerregistry`'s `remote.Head` with `authn.DefaultKeychain` (reads `DOCKER_CONFIG`/`config.json`). +- New package `internal/master/poller`: owns the cycle (list services, group by followed ref, run HEADs with bounded concurrency, then set available / start auto jobs / store poll error per service). `cmd/master` runs it as a background goroutine on the signal context. +- Store: `image_versions` written in `SetAvailable` (webhook and poll, same transaction) and in `upsertService` (running inventory, inside the existing snapshot/upsert transactions). +- Jobs gains `TriggerVersion`, which takes a version ID, never a ref from the form. `Hub.Route` gains a minimum protocol version so a pinned or tagless ref only goes to agents that understand it. +- Agent: resolver passes digest-pinned refs through unchanged. Protocol version goes to 3. + +**Tech Stack:** adds `github.com/google/go-containerregistry` (`pkg/name`, `pkg/v1/remote`, `pkg/authn`; `pkg/registry` and `pkg/v1/random` in tests only). + +**Spec:** `docs/superpowers/specs/2026-10-12-master-agent-design.md`, sections "Versions: what the system has seen", "Data model", "Update", "Failure handling", "Registry polling", "UI". Phase 3 plan conventions still apply. + +## Global Constraints + +- `go test ./...` passes with no Docker, no Postgres and no network. Registry tests use `httptest` plus `ggcr/pkg/registry`. Postgres tests carry `//go:build integration`. +- Interfaces declared at the consumer (`poller.Store`, `poller.Jobs`, `poller.Registry`). Hand-written fakes and testify, table-driven where there is more than one case. +- GoDoc on every exported identifier. Conventional Commits, each ending with ` (#9)`. +- Registry credentials never appear in logs or in `last_poll_error`. Error text stored truncated to 500 bytes. +- Registry traffic is HEAD only: never `GET /v2//manifests/...` (Docker Hub counts GETs as pulls). A test asserts this. +- A pick from the UI is a POST with the CSRF token and a version ID. The ref is built server-side from the `image_versions` row, and the row must belong to the service's `image_repo`. +- The poller never overlaps itself. Next cycle is scheduled after the previous one finishes. +- `go.mod` `go` directive must stay `1.26.3` after `go get`. + +## Deliberate deviations from the spec in this phase + +1. **Webhook records `(repo, tag, '')`, no digest.** The master does not resolve the digest at webhook time (no registry round-trip/credentials in the CI path). The digest arrives later from polling or running inventory. The picker hides a digest-less row once a row with the same `(repo, tag)` and a digest exists. +2. **Empty string instead of NULL** for missing tag or digest in `image_versions`, so `UNIQUE (repo, tag, digest)` works. A CHECK forbids both being empty. +3. **`first_seen_at` is never bumped.** Re-observation is `ON CONFLICT DO NOTHING`; `source` is the first source that saw it. "10 most recent" = "10 most recently first seen". A new digest of a tag is a new row at the top; old digest stays below (enables rollback of moving tags). +4. **Not polled:** services that follow no tag, `build:` services, services with empty `actual_digest`. UI shows the reason. Self and container services ARE polled (badge informative; jobs still refused by `jobs`). +5. **Poller does not re-suggest the same version.** If `available_ref == ref && available_digest == D`, nothing happens for that service (no `available_at` bump, no new auto job). A failed auto job is not retried every interval; a rollback of an auto service is not undone by polling until a new digest appears. +6. **FinishJob clearing rule gets a digest condition** (C3). Rollback to an older digest of the same tag no longer clears the newer-version badge. +7. **Poll auto jobs use `repo:tag`, not `repo:tag@D`.** v2 agents keep working for poll and webhook jobs. Only UI picks of digest-pinned or tagless versions need a v3 agent. +8. **First cycle runs 1 minute after start**, then every `POLL_INTERVAL` measured from the end of the previous cycle. +9. **No `last_polled_at` column** (YAGNI). +10. **Overview gets two new badges: "follows no tag" and "poll failed"** (error as `title`). Service page gets a "Polling" row. "Drifted" is not in this phase. +11. **Credentials from `DOCKER_CONFIG`** (a directory), same name and meaning as on the agent. Only inline `auths` entries supported in the container. + +## Implementation notes (differences found while building) + +- The plan's test names `TestResolve` and `TestComposeWritesPinnedRef` became `TestResolver` + (extended), a subtest of `TestComposeExecutor` and `TestSwarmUsesPinnedRef`. +- Tasks 2 and 3 are one commit: changing `Hub.Route`'s signature breaks `jobs` and the e2e hub + wrapper until both are updated. +- `Poller.Cycle` runs the registry lookups concurrently and then records the results + sequentially (on `context.WithoutCancel`). +- `web.Config` gained `PollInterval` for the "every " text. +- `authn.DefaultKeychain` with a `credsStore` whose helper binary is missing makes the lookup fail + with `error getting credentials - err: exec: "docker-credential-...": executable file not found in $PATH` + (covered by `TestCredsStoreWithoutHelperFailsTheLookup`); it does not fall back to anonymous access. + +## Review Focus + +| Failure case | Test that covers it | +|---|---| +| Registry access is HEAD only; no manifest GET ever happens. | registry `TestDigestUsesHeadOnly` | +| Multi-arch: digest returned is the index digest (same as `RepoDigests` / Swarm spec). | registry `TestDigestOfIndexIsIndexDigest` | +| Credentials from `$DOCKER_CONFIG/config.json` are sent; anonymous works without the file. | registry `TestCredentialsFromDockerConfig`, `TestAnonymousWithoutConfig` | +| One HEAD per distinct followed ref (`repo:tag`, `repo:tag@d`, bare → `:latest`). | poller `TestOneHeadPerDistinctRef` | +| No tag / built / no digest are skipped, no HEAD. | poller `TestSkipped` | +| Error on one ref goes into its services' `last_poll_error`; others continue; later success clears it. | poller `TestErrorIsPerRef`, `TestErrorClearedOnSuccess` | +| Already-suggested digest neither re-suggested nor re-started. | poller `TestSameSuggestionIsIdempotent` | +| Auto gets exactly one `TriggerPoll` job to `repo:tag`; manual only gets marked. | poller `TestNewDigest` table | +| `ErrJobActive` / `NotUpdatableError` from `Start` do not abort the cycle. | poller `TestStartErrorsDoNotAbortCycle` | +| Concurrency within limit; cancellation stops the cycle; marking/starting on `WithoutCancel`. | poller `TestConcurrencyLimit`, `TestRunStopsOnCancel` | +| Form cannot deploy an arbitrary ref: unknown version ID → 404; version of another repo → 404, no job. | jobs `TestTriggerVersion_*`, web `TestUpdateService_Version*` | +| Pinned/tagless refs routed only to protocol ≥ 3 agents; old agents → failed "agent too old" job. | jobs `TestTriggerVersion_RequiresPinnedProtocol`, hub `TestRoute` | +| Agent deploys a pinned ref verbatim, no registry resolve. | updater `TestResolve`, `TestComposeWritesPinnedRef`, `TestSwarmUsesPinnedRef` | +| Rollback to older digest of same tag keeps newer suggestion; pick of suggested digest clears it. | store `TestFinishJob_ClearsAvailableOnlyWhenThatVersionWasDeployed` (extended) | +| Running versions recorded only with a digest; re-snapshot does not duplicate or reorder. | store `TestSnapshotRecordsRunningVersions` | +| Digest-less webhook row hidden once the tag's digest is known; order, limit, repo filter. | store `TestRecentVersions` | +| `available_source = 'poll'` passes the CHECK. | store `TestSetAvailable_RecordsVersion` | +| E2E: pick of a pinned version reaches the agent as that exact ref. | e2e `TestPickPinnedVersion` | + +## Frozen contracts + +### C1. Migration `internal/master/store/migrations/0004_polling.sql` + +```sql +ALTER TABLE services DROP CONSTRAINT services_available_source_check; +ALTER TABLE services ADD CONSTRAINT services_available_source_check + CHECK (available_source IN ('', 'webhook', 'poll')); +ALTER TABLE services ADD COLUMN available_digest text NOT NULL DEFAULT ''; +ALTER TABLE services ADD COLUMN last_poll_error text NOT NULL DEFAULT ''; + +-- Every version the master has seen. '' means "no tag" / "digest unknown"; +-- never NULL, so the unique constraint deduplicates. +CREATE TABLE image_versions ( + id bigserial PRIMARY KEY, + repo text NOT NULL, + tag text NOT NULL DEFAULT '', + digest text NOT NULL DEFAULT '', + source text NOT NULL CHECK (source IN ('webhook', 'poll', 'running')), + first_seen_at timestamptz NOT NULL, + CONSTRAINT image_versions_tag_or_digest CHECK (tag <> '' OR digest <> ''), + CONSTRAINT image_versions_unique UNIQUE (repo, tag, digest) +); +CREATE INDEX image_versions_recent ON image_versions (repo, first_seen_at DESC, id DESC); +``` + +Confirm the existing constraint name in `0003_webhook.sql` before relying on it. + +### C2. `internal/master/model` + +```go +// Service gains: +AvailableDigest string // digest of AvailableRef when known (poll); "" from the webhook +LastPollError string // last registry error for this service's followed ref; "" when fine + +const SourcePoll = "poll" // alongside SourceWebhook +const SourceRunning = "running" // image_versions only + +type Available struct { Ref, Digest, Source string; At time.Time } // Digest added + +// ImageVersion is one row of image_versions. +type ImageVersion struct { + ID int64 + Repo, Tag, Digest, Source string + FirstSeenAt time.Time +} +// Ref returns "repo:tag@digest", "repo:tag" or "repo@digest". +func (v ImageVersion) Ref() string +``` + +### C3. Store + +```go +// SetAvailable sets available_ref/_digest/_source/_at and, in the same +// transaction, inserts image_versions(repo, tag parsed from a.Ref, a.Digest, +// a.Source, a.At) ON CONFLICT DO NOTHING. +SetAvailable(ctx, id int64, a model.Available) error + +// RecentVersions returns up to limit versions of repo, newest first_seen_at +// first (ties: id DESC). It leaves out a digest-less row when a row with the +// same repo and tag and a digest exists. +RecentVersions(ctx, repo string, limit int) ([]model.ImageVersion, error) + +// GetImageVersion returns one version or model.ErrNotFound. +GetImageVersion(ctx, id int64) (model.ImageVersion, error) + +// SetPollError sets last_poll_error on the given services ("" clears). +SetPollError(ctx, ids []int64, msg string) error +``` + +`RecentVersions` SQL: + +```sql +SELECT id, repo, tag, digest, source, first_seen_at FROM image_versions v +WHERE v.repo = $1 + AND NOT (v.digest = '' AND EXISTS ( + SELECT 1 FROM image_versions d + WHERE d.repo = v.repo AND d.tag = v.tag AND d.digest <> '')) +ORDER BY v.first_seen_at DESC, v.id DESC +LIMIT $2 +``` + +`upsertService` (running inventory): after the service upsert, on the same `execer`, only when `svc.Digest != ""`: + +```sql +INSERT INTO image_versions (repo, tag, digest, source, first_seen_at) +VALUES ($1, $2, $3, 'running', $4) ON CONFLICT DO NOTHING +``` + +`repo`/`tag` from `imageref.Parse(svc.Image)`, `digest` = `svc.Digest`. A bare image is recorded with `tag = ''`, not `latest`. Covers `ApplySnapshot` and `UpsertService` with no hub change. + +`serviceColumns` / `scanService` gain `s.available_digest, s.last_poll_error`. + +`FinishJob` clearing rule (replaces phase 3's). With `requestedRef, requestedDigest, _ := strings.Cut(to_ref_as_created, "@")`: + +```sql +available_ref = $4 AND available_at <= $3 + AND ($5 = '' OR available_digest = $5) +``` + +`$5` = `requestedDigest`. A digest-pinned job does NOT clear a digest-less (webhook) suggestion: a rollback to `main@old` must not hide that a newer `main` was announced. The badge then stays until a plain tag job succeeds; that is accepted. Clearing sets `available_ref=''`, `available_digest=''`, `available_source=''`, `available_at=NULL`. + +### C4. `internal/master/registry` + +```go +// Client looks up tag digests with HEAD requests only. +type Client struct{ /* keychain authn.Keychain, userAgent string */ } +// New uses authn.DefaultKeychain ($DOCKER_CONFIG/config.json, else ~/.docker/config.json). +func New(userAgent string) *Client +// Digest returns the digest ref ("repo:tag", any digest suffix ignored) points +// at: the index digest for multi-arch images. HEAD /v2//manifests/ only. +func (c *Client) Digest(ctx context.Context, ref string) (string, error) +``` + +Implementation: `name.ParseReference(repo+":"+tag)`; `remote.Head(r, remote.WithContext(ctx), remote.WithAuthFromKeychain(c.keychain), remote.WithUserAgent(c.userAgent))`; return `desc.Digest.String()`. Never fall back to GET if `Docker-Content-Digest` is missing. + +### C5. `internal/master/poller` + +```go +type Store interface { + ListServices(ctx context.Context) ([]model.Service, error) + SetAvailable(ctx context.Context, id int64, a model.Available) error + SetPollError(ctx context.Context, ids []int64, msg string) error +} +type Jobs interface { + Start(ctx context.Context, svc model.Service, req jobs.StartRequest) (model.Job, error) +} +type Registry interface { Digest(ctx context.Context, ref string) (string, error) } +type Publisher interface { Publish(t events.Topic) } + +type Config struct { + Interval time.Duration // POLL_INTERVAL + FirstDelay time.Duration // 1m in production; small in tests + Workers int // 4 +} +func New(st Store, j Jobs, reg Registry, pub Publisher, cfg Config, log *slog.Logger) *Poller +// Run polls until ctx is done. Blocking. +func (p *Poller) Run(ctx context.Context) +// Cycle runs one poll over all services (exported for tests and e2e). +func (p *Poller) Cycle(ctx context.Context) +// SkipReason says why svc is not polled: "follows no tag", "built from +// source", "no registry digest", or "" when it is polled. +func SkipReason(svc model.Service) string +``` + +Cycle semantics, per distinct `ref = jobs.FollowedRef(svc.ActualRef)` over non-removed, non-skipped services: +1. `D, err := reg.Digest(ctx, ref)` with a 30s timeout per call. +2. On error: `SetPollError(ids whose LastPollError != msg, msg)`, log `poll_failed` (ref, error). +3. On success: `SetPollError(ids with LastPollError != "", "")`. Then per service: + - `svc.ActualDigest == D` → nothing. + - `svc.AvailableRef == ref && svc.AvailableDigest == D` → nothing. + - Otherwise `SetAvailable({ref, D, poll, now})`, and if `UpdateMode == auto`, `Start(TriggerPoll, Ref: ref)`. `ErrJobActive` and `NotUpdatableError` logged at debug and ignored. + +Steps 2 and 3 use `context.WithoutCancel(ctx)`. At the end, publish `events.Services` if anything was written; log `poll_cycle` with refs, ok, failed, suggested, jobs, skipped, duration_ms. + +### C6. `internal/master/jobs` and hub + +```go +// Hub.Route gains the minimum protocol version for this job: +Route(svc model.Service, minProtocol int) (model.Route, error) + +// Store gains: +GetImageVersion(ctx context.Context, id int64) (model.ImageVersion, error) + +// TriggerVersion starts a manual job to exactly version versionID of the +// service's repo. model.ErrNotFound for an unknown service, an unknown +// version, or a version of another repo. Otherwise what Start returns. +func (s *Service) TriggerVersion(ctx context.Context, serviceID, versionID int64, requestedBy string) (model.Job, error) +``` + +- Ref = `svc.ImageRepo` + `":"+tag` if tag set + `"@"+digest` if digest set. +- Calls `Start` with `TriggerManual` and explicit ref ("follows no tag" refusal does not apply; removed, self, built, container still do). +- `Start` passes `protocol.MinPinnedRefVersion` to `Route` when the ref has a digest or no tag, otherwise `protocol.MinJobVersion`. + +### C7. Protocol and agent + +```go +const Version = 3 +const MinJobVersion = 2 // unchanged: repo:tag jobs +const MinPinnedRefVersion = 3 // jobs whose Ref has a digest or no tag +``` + +`JobStart` doc: "Ref is `repo:tag`, `repo:tag@digest` or `repo@digest`. For `repo:tag` the agent resolves the tag's current digest; a ref with a digest is deployed as is." + +`Resolver.Resolve`: digest present → return ref unchanged, no `DistributionInspect`; tag present → resolve as today; neither → error "has no tag to resolve" (unchanged). Compose writes the ref verbatim into the override, Swarm sets it as the image. Executors print `pinned ` instead of `resolved ` when nothing was resolved. + +### C8. Master configuration + +- `POLL_INTERVAL`: default `6h`, positive Go duration, error `"POLL_INTERVAL must be a positive duration such as 6h, got %q"`. +- `DOCKER_CONFIG`: not parsed by `config`; `go-containerregistry` reads it. `main` logs `registry_credentials` with the path and whether `config.json` exists, never its contents. + +### C9. Web + +- `POST /services/{id}/update` gains optional form field `version` (an `image_versions.id`): empty → `Trigger` as today; non-numeric → `400 invalid version`; otherwise `TriggerVersion`. Results: `303` to the job (also on `ErrJobActive`), `404` on `ErrNotFound`, `409` re-render "Cannot update: ." +- `web.Store` gains `RecentVersions`; `web.Jobs` gains `TriggerVersion`. +- Service page: "Versions" table, up to 10 rows: ref (shortRef, full ref as title), source, first seen, "running" marker when digest equals `ActualDigest`, a POST form with `csrf_token` and `version`. Forms shown only when `AutoBlocked == ""`. Rows with empty tag note "the service will follow no tag afterwards". When the service is `auto`, header notes auto mode will move it forward at the next new version. A "Polling" row shows `SkipReason`, or `last_poll_error`, or "every ". +- Overview: badge "follows no tag" when `FollowedRef(ActualRef) == ""`; badge "poll failed" (title = error) when `LastPollError != ""`. Template func `followsNoTag`. + +## Tasks + +### Task 1: Store and model (C1, C2, C3) +Migration, model fields, store methods, `upsertService` running versions, `FinishJob` rule. Integration tests: SetAvailable writes digest/source poll and records version, second call keeps first_seen_at; TestSnapshotRecordsRunningVersions (tag+digest, tagless digest, no digest skipped, re-snapshot idempotent); TestRecentVersions (order, limit 10, repo filter, digest-less row hidden only when a digest row for same tag exists); GetImageVersion ErrNotFound; SetPollError sets/clears, upsert does not touch last_poll_error/available_*; FinishJob table extended (tag job clears poll suggestion; pick tag@old keeps suggestion tag@new; pick tag@new clears; webhook suggestion (digest '') cleared by matching tag; tagless pick keeps it). Update the e2e `memStore` in the same commit. + +### Task 2: Protocol, agent and hub routing (C7, C6 Route) +Bump protocol, MinPinnedRefVersion, docs; Resolver pass-through; compose/swarm output line; `hub.Route(svc, minProtocol)`. Tests: TestResolve table (tag resolves; tag@d and @d pass through with zero inspect calls; bare errors); compose pinned ref verbatim in override; swarm spec image equals pinned ref; route rows v2 agent + min 3 → ErrAgentTooOld, v3 → ok, swarm fallback to v3 manager when reporting manager is v2. + +### Task 3: Jobs (C6) +`Start` picks min version from the ref; `TriggerVersion`. Tests: version of other repo → ErrNotFound, no CreateJob; ref construction table; min version passed to fake hub (plain Trigger → 2, pinned → 3); explicit ref bypasses no-tag but not removed/self/built/container; existing tests green. + +### Task 4: Registry client and poller (C4, C5) +Registry tests (local httptest + ggcr/pkg/registry, no network): random.Index digest equals index digest; recording handler shows only HEAD on /manifests/; basic-auth handler succeeds with temp DOCKER_CONFIG (t.Setenv) and fails without; 404 → error. Verify behaviour of DefaultKeychain when config.json has credsStore but no helper binary; document. Poller tests per Review Focus, plus Run test (first cycle after delay, no overlap, returns on cancel). + +### Task 5: Config, wiring, web (C8, C9) +Config POLL_INTERVAL with tests; add to setAll list. `cmd/master/main.go`: registry.New, poller, `go p.Run(ctx)` with done channel, wait for it after HTTP shutdown and before st.Close; log registry_credentials. Web handlers/templates per C9 with tests (picker ≤10 rows with CSRF and version IDs; hidden for built/self/container/removed; POST with version → TriggerVersion, without → Trigger; 400/404/409; overview badges; Polling row). + +### Task 6: End-to-end +`internal/e2e/versions_test.go`: TestPickPinnedVersion; TestPollStartsAutoJob (poller.Cycle with fake registry → one TriggerPoll job to repo:tag; second Cycle starts nothing). `currentHub.Route` new signature. + +### Task 7: Documentation +README (POLL_INTERVAL, DOCKER_CONFIG, "Registry polling" section: polled/skipped, credentials only inline auths, Docker Hub note, version picker & rollback, auto caveat, agent protocol 3 for pinned/tagless picks). Spec: replace "Believed not to count against Docker Hub pull limits for HEAD requests — verify during implementation" with verified statement + source link (docs.docker.com/docker-hub/usage/pulls: "Using GET emulates a real pull and counts towards the limit. Using HEAD won't."; note the separate unquantified abuse rate limit). `deploy/master.stack.yml`: mount config dir read-only, set DOCKER_CONFIG. CLAUDE.md: architecture entries for registry and poller; gotchas (HEAD only; digest-less webhook row hidden once digest known; image_versions uses '' not NULL; picks by version ID and pinned/tagless need protocol 3; FinishJob rule includes digest (replace existing bullet); rollback to repo@digest makes the service follow no tag); phase 4 plan reference. Save this plan as `docs/superpowers/plans/2026-10-13-master-agent-phase-4-polling.md`. + +## Final verification + +```bash +gofmt -l . ; go vet ./... +go build ./cmd/master ./cmd/agent && rm -f master agent +go test ./... +go test -race ./internal/agent/... ./internal/master/... ./internal/e2e/ +DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/ +``` diff --git a/docs/superpowers/specs/2026-10-12-master-agent-design.md b/docs/superpowers/specs/2026-10-12-master-agent-design.md index ce50afc..063b511 100644 --- a/docs/superpowers/specs/2026-10-12-master-agent-design.md +++ b/docs/superpowers/specs/2026-10-12-master-agent-design.md @@ -247,8 +247,12 @@ them. An agent recreating itself would kill the job it is running. - Mechanism: `HEAD /v2//manifests/`; the returned digest is compared with `actual_digest`. Use `go-containerregistry` — it handles Docker Hub's token auth and multi-arch indexes. -- Interval `POLL_INTERVAL`, default 6h. Believed not to count against Docker Hub pull - limits for HEAD requests — verify during implementation. +- Interval `POLL_INTERVAL`, default 6h. Docker Hub documents that HEAD requests do not + count against the pull limit ("Using GET emulates a real pull and counts towards the + limit. Using HEAD won't."; verified against + https://docs.docker.com/docker-hub/usage/pulls/). The same page mentions a separate, + unquantified abuse rate limit for many pulls from one IP address, so the poller stays + HEAD-only and never downloads a manifest. - Credentials: a Docker `config.json` (as written by `docker login`) mounted into the master. No credential UI, no encryption in Postgres.