docs: registry polling, version history and phase 4 plan (#9)

2026-10-13 08:16:47 +02:00
parent 687e411052
commit 2e87d98012
5 changed files with 392 additions and 5 deletions
@@ -36,6 +36,8 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
 - `internal/master/config`   — env-var loading
 - `internal/master/store`    — Postgres access and migrations
 - `internal/master/hub`      — agent connections: auth, per-connection writer, routing, job reconcile on hello
 - `internal/master/registry` — HEAD-only digest lookup against a registry (go-containerregistry, `DOCKER_CONFIG` credentials)
 - `internal/master/poller`   — every `POLL_INTERVAL`: one lookup per followed `repo:tag`, sets the available version, starts auto jobs
 - `internal/master/jobs`     — decides whether a service may be updated and starts a job on the right agent
 - `internal/master/webhook`  — `POST /update`: records the available version, starts jobs for auto services, waits for them
 - `internal/master/web`, `webauth` — HTMX UI behind OIDC login, SSE signals
@@ -68,6 +70,16 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
 ## Gotchas
 - **Registry access is HEAD only** (`internal/master/registry`). Docker Hub counts a manifest
  GET as a pull; the client never falls back to GET, and a test asserts this.
 - **`image_versions` uses `''` instead of NULL** for a missing tag or digest, so
  `UNIQUE (repo, tag, digest)` deduplicates. The webhook records `(repo, tag, '')`; that
  row is hidden from `RecentVersions` once a row with the same tag and a digest exists.
 - **The UI picks a version by `image_versions.id`, never by a ref from the form**, and the row
  must belong to the service's `image_repo`. A digest-pinned or tagless pick needs agent
  protocol 3 (`protocol.MinPinnedRefVersion`); `Hub.Route` takes the minimum version.
 - **Rolling back a service that follows no tag afterwards:** a pick of `repo@digest` makes the
  service follow no tag, so it is no longer polled.
 - **Image matching is case-sensitive.** The tricky case is `localhost:5000/foo:v1` where
  the first colon is a port, not a tag. See `internal/imageref`.
 - **Path safety uses `filepath.Rel` + ".." prefix check**, NOT `strings.HasPrefix`.
@@ -102,9 +114,10 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
  repo, never from the request's image string. With an explicit target the "follows no tag"
  refusal does not apply; removed, self, built and container still do.
 - **`available_*` is cleared by `FinishJob`** on success only when the job's `to_ref`
-  (as created, without `@digest`) equals `available_ref` and `available_at <= job.created_at`
+  (as created, without `@digest`) equals `available_ref`, `available_at <= job.created_at`
-  (that version is now running). A manual Update of the followed tag does not clear a
+  (that version is now running) and, when the job's `to_ref` carries a digest, that digest
-  badge for a different tag.
+  equals `available_digest`. A rollback to an older digest of the tag therefore keeps the
  newer suggestion, and a digest-pinned pick does not clear a digest-less (webhook) one.
 - **Job output is capped at `protocol.MaxJobOutput` (128 KiB, tail).** NUL bytes are
  stripped by the hub because Postgres text rejects them.
@@ -114,6 +127,7 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
 - Phase 1 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-1-inventory.md`
 - Phase 2 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md`
 - Phase 3 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-3-webhook.md`
 - Phase 4 plan: `docs/superpowers/plans/2026-10-13-master-agent-phase-4-polling.md`
 - Original single-instance design (historical):
  `docs/superpowers/specs/2026-05-22-package-updater-design.md`
 - Consumer-side CI integration: `gitea-action/` (calls the master's `POST /update`)
@@ -9,6 +9,7 @@ the same image: `updater-master` and `updater-agent` (the default entrypoint).
 Design: [master/agent spec](docs/superpowers/specs/2026-10-12-master-agent-design.md).
 Phase 2 plan: [jobs and manual update](docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md).
 Phase 3 plan: [webhook and auto mode](docs/superpowers/plans/2026-10-12-master-agent-phase-3-webhook.md).
 Phase 4 plan: [registry polling and version history](docs/superpowers/plans/2026-10-13-master-agent-phase-4-polling.md).
 ## What an update does
@@ -33,6 +34,8 @@ pinned TLS, reached directly on the internal network).
 |---|---|---|---|
 | `UPDATER_API_KEY` | no | — | Bearer token for `POST /update`. Without it the webhook answers `503` and the master logs `webhook_disabled`. |
 | `UPDATE_TIMEOUT` | no | `5m` | How long one `POST /update` waits for its auto jobs (Go duration). |
 | `POLL_INTERVAL` | no | `6h` | Time between registry polling cycles (Go duration, must be positive). The first cycle runs one minute after start. |
 | `DOCKER_CONFIG` | no | `~/.docker` | Directory with a `config.json` holding registry credentials for polling. Only inline `auths` entries work; see [Registry polling](#registry-polling). |
 The master's other settings (`DATABASE_URL`, `PUBLIC_URL`, `AGENT_URL`, `OIDC_*`,
 `ALLOWED_GROUP`, `AGENT_IMAGE`, `PORT`, `AGENT_PORT`, `LOG_LEVEL`) are shown in
@@ -112,6 +115,57 @@ The request is open for as long as the jobs run, so the reverse proxy in front o
 master must allow a read timeout longer than `UPDATE_TIMEOUT`. If the client disconnects,
 marking and starting jobs still complete; only the waiting stops.
 ## Registry polling
 Besides the webhook, the master asks each registry whether the tag a service follows has
 a new digest. Every `POLL_INTERVAL` it sends one `HEAD /v2/<repo>/manifests/<tag>` per
 distinct followed `repo:tag` (a bare image counts as `:latest`; `repo:tag@digest` is polled
 as `repo:tag`). It never downloads a manifest: Docker Hub counts a `GET` as a pull against
 the rate limit and a `HEAD` as nothing
 ([source](https://docs.docker.com/docker-hub/usage/pulls/)), and Docker Hub also has a
 separate abuse limit for many pulls from one IP address, so the poller stays HEAD-only. For a
 multi-arch image the digest is the index digest, the same one `docker inspect` shows in
 `RepoDigests`.
 - **What counts as new.** A digest different from the one the service runs becomes the
  service's *available* version (the overview badge, source `poll`). A digest the master
  already suggested is not suggested again. A service in **auto** mode also gets an update
  job to `repo:tag` (the agent resolves the digest itself, so agents on protocol version 2 keep
  working). A failed auto job is not retried every interval.
 - **What is not polled.** Services that follow no tag (`repo@digest`), `build:` services, and
  services without a registry digest (locally built images). The service page says which.
  The updater's own services and plain containers are polled, so their badge works, but
  jobs for them are still refused.
 - **Errors.** A failed lookup is stored for the services that follow that tag and shown as a
  "poll failed" badge on the overview and in the "Polling" row of the service page; other
  tags continue. A later success clears it.
 - **Credentials.** The master reads `$DOCKER_CONFIG/config.json` (a file written by
  `docker login`), mounted read-only into the container; without it registries are accessed
  anonymously. Only inline `auths` entries are supported. A `credsStore` or `credHelpers`
  entry makes every lookup fail with `error getting credentials ... executable file not found
  in $PATH`, because the container has no credential helper binaries. The master logs
  `registry_credentials` at startup with the directory and whether `config.json` exists,
  never its contents.
 ### Version history and rollback
 The master records every version it sees (webhook, polling, and what agents report as
 running) in `image_versions`. The service page lists the 10 most recently first seen
 versions of the service's image repo, marks the one that is running, and has a *Deploy*
 button on each row. A webhook announces a tag but not its digest, so such a row has no digest until polling or an
 agent reports one; it is hidden once a row with the same tag and a digest exists.
 Picking a version starts a manual job to exactly that reference. Picking an older one is a
 rollback, including to a digest-pinned reference. A version without a tag deploys `repo@digest` and the
 service then follows no tag, so it is no longer polled or updatable with the plain *Update* button until
 it is moved to a tagged version again. Picks of digest-pinned or tagless versions need an agent speaking protocol
 version 3 (this release); older agents get a failed job "agent too old for updates".
 On a service in **auto** mode a rollback is only a warning in the UI: the service stays auto and
 is moved forward again at the next new version, announced by the webhook or found by polling.
 Rolling back to an older digest of the tag the service follows keeps the newer version marked as
 available.
 ## Adding agents
 Open the master's **Agents** page and add one. It shows a ready-to-run compose file with
@@ -10,6 +10,9 @@
 #      To run without the webhook, remove the pu_updater_api_key lines (the UPDATER_API_KEY
 #      part of the entrypoint, the service secret and the secrets entry); the endpoint then answers 503.
 #      The reverse proxy must allow a read timeout longer than UPDATE_TIMEOUT (default 5m).
 #   5. Optional, for registry polling of private images: put a config.json written by
 #      `docker login` (inline "auths" only; no credsStore) in /path/to/docker-config on the
 #      manager and keep the volumes entry below. Without it registries are accessed anonymously.
 services:
  master:
    image: gitea.shcizo.se/shcizo/package-updater:latest
@@ -21,10 +24,14 @@ services:
      OIDC_CLIENT_ID: package-updater
      ALLOWED_GROUP: updater-admins
      # UPDATE_TIMEOUT: 5m
      # POLL_INTERVAL: 6h
      DOCKER_CONFIG: /docker-config
    secrets:
      - pu_database_url
      - pu_oidc_client_secret
      - pu_updater_api_key
    volumes:
      - /path/to/docker-config:/docker-config:ro
    ports:
      # Agents connect here directly over the internal network, not via the proxy.
      - target: 8090
@@ -0,0 +1,308 @@
 # Master/Agent Phase 4 — Registry Polling and Version History Implementation Plan
 > This plan documents what to build for issue #9 and where it deviates from the spec. Code in "Frozen contracts" is final; tasks give the contract and the test cases.
 **Goal:** The master records every version it sees (webhook, registry polling, running inventory) in `image_versions`. Every `POLL_INTERVAL` (default 6h) it checks each distinct followed `repo:tag` against its registry with one HEAD request. A new digest becomes the service's available suggestion, and `auto` services get a job. The Service page lists the 10 most recent versions of the service's repo. Picking one starts a manual job to exactly that reference; picking an older one is a rollback, including to a digest-pinned or tagless reference.
 **Architecture:**
 - New package `internal/master/registry`: thin wrapper around `go-containerregistry`'s `remote.Head` with `authn.DefaultKeychain` (reads `DOCKER_CONFIG`/`config.json`).
 - New package `internal/master/poller`: owns the cycle (list services, group by followed ref, run HEADs with bounded concurrency, then set available / start auto jobs / store poll error per service). `cmd/master` runs it as a background goroutine on the signal context.
 - Store: `image_versions` written in `SetAvailable` (webhook and poll, same transaction) and in `upsertService` (running inventory, inside the existing snapshot/upsert transactions).
 - Jobs gains `TriggerVersion`, which takes a version ID, never a ref from the form. `Hub.Route` gains a minimum protocol version so a pinned or tagless ref only goes to agents that understand it.
 - Agent: resolver passes digest-pinned refs through unchanged. Protocol version goes to 3.
 **Tech Stack:** adds `github.com/google/go-containerregistry` (`pkg/name`, `pkg/v1/remote`, `pkg/authn`; `pkg/registry` and `pkg/v1/random` in tests only).
 **Spec:** `docs/superpowers/specs/2026-10-12-master-agent-design.md`, sections "Versions: what the system has seen", "Data model", "Update", "Failure handling", "Registry polling", "UI". Phase 3 plan conventions still apply.
 ## Global Constraints
 - `go test ./...` passes with no Docker, no Postgres and no network. Registry tests use `httptest` plus `ggcr/pkg/registry`. Postgres tests carry `//go:build integration`.
 - Interfaces declared at the consumer (`poller.Store`, `poller.Jobs`, `poller.Registry`). Hand-written fakes and testify, table-driven where there is more than one case.
 - GoDoc on every exported identifier. Conventional Commits, each ending with ` (#9)`.
 - Registry credentials never appear in logs or in `last_poll_error`. Error text stored truncated to 500 bytes.
 - Registry traffic is HEAD only: never `GET /v2/<repo>/manifests/...` (Docker Hub counts GETs as pulls). A test asserts this.
 - A pick from the UI is a POST with the CSRF token and a version ID. The ref is built server-side from the `image_versions` row, and the row must belong to the service's `image_repo`.
 - The poller never overlaps itself. Next cycle is scheduled after the previous one finishes.
 - `go.mod` `go` directive must stay `1.26.3` after `go get`.
 ## Deliberate deviations from the spec in this phase
 1. **Webhook records `(repo, tag, '')`, no digest.** The master does not resolve the digest at webhook time (no registry round-trip/credentials in the CI path). The digest arrives later from polling or running inventory. The picker hides a digest-less row once a row with the same `(repo, tag)` and a digest exists.
 2. **Empty string instead of NULL** for missing tag or digest in `image_versions`, so `UNIQUE (repo, tag, digest)` works. A CHECK forbids both being empty.
 3. **`first_seen_at` is never bumped.** Re-observation is `ON CONFLICT DO NOTHING`; `source` is the first source that saw it. "10 most recent" = "10 most recently first seen". A new digest of a tag is a new row at the top; old digest stays below (enables rollback of moving tags).
 4. **Not polled:** services that follow no tag, `build:` services, services with empty `actual_digest`. UI shows the reason. Self and container services ARE polled (badge informative; jobs still refused by `jobs`).
 5. **Poller does not re-suggest the same version.** If `available_ref == ref && available_digest == D`, nothing happens for that service (no `available_at` bump, no new auto job). A failed auto job is not retried every interval; a rollback of an auto service is not undone by polling until a new digest appears.
 6. **FinishJob clearing rule gets a digest condition** (C3). Rollback to an older digest of the same tag no longer clears the newer-version badge.
 7. **Poll auto jobs use `repo:tag`, not `repo:tag@D`.** v2 agents keep working for poll and webhook jobs. Only UI picks of digest-pinned or tagless versions need a v3 agent.
 8. **First cycle runs 1 minute after start**, then every `POLL_INTERVAL` measured from the end of the previous cycle.
 9. **No `last_polled_at` column** (YAGNI).
 10. **Overview gets two new badges: "follows no tag" and "poll failed"** (error as `title`). Service page gets a "Polling" row. "Drifted" is not in this phase.
 11. **Credentials from `DOCKER_CONFIG`** (a directory), same name and meaning as on the agent. Only inline `auths` entries supported in the container.
 ## Implementation notes (differences found while building)
 - The plan's test names `TestResolve` and `TestComposeWritesPinnedRef` became `TestResolver`
  (extended), a subtest of `TestComposeExecutor` and `TestSwarmUsesPinnedRef`.
 - Tasks 2 and 3 are one commit: changing `Hub.Route`'s signature breaks `jobs` and the e2e hub
  wrapper until both are updated.
 - `Poller.Cycle` runs the registry lookups concurrently and then records the results
  sequentially (on `context.WithoutCancel`).
 - `web.Config` gained `PollInterval` for the "every <interval>" text.
 - `authn.DefaultKeychain` with a `credsStore` whose helper binary is missing makes the lookup fail
  with `error getting credentials - err: exec: "docker-credential-...": executable file not found in $PATH`
  (covered by `TestCredsStoreWithoutHelperFailsTheLookup`); it does not fall back to anonymous access.
 ## Review Focus
 | Failure case | Test that covers it |
 |---|---|
 | Registry access is HEAD only; no manifest GET ever happens. | registry `TestDigestUsesHeadOnly` |
 | Multi-arch: digest returned is the index digest (same as `RepoDigests` / Swarm spec). | registry `TestDigestOfIndexIsIndexDigest` |
 | Credentials from `$DOCKER_CONFIG/config.json` are sent; anonymous works without the file. | registry `TestCredentialsFromDockerConfig`, `TestAnonymousWithoutConfig` |
 | One HEAD per distinct followed ref (`repo:tag`, `repo:tag@d`, bare → `:latest`). | poller `TestOneHeadPerDistinctRef` |
 | No tag / built / no digest are skipped, no HEAD. | poller `TestSkipped` |
 | Error on one ref goes into its services' `last_poll_error`; others continue; later success clears it. | poller `TestErrorIsPerRef`, `TestErrorClearedOnSuccess` |
 | Already-suggested digest neither re-suggested nor re-started. | poller `TestSameSuggestionIsIdempotent` |
 | Auto gets exactly one `TriggerPoll` job to `repo:tag`; manual only gets marked. | poller `TestNewDigest` table |
 | `ErrJobActive` / `NotUpdatableError` from `Start` do not abort the cycle. | poller `TestStartErrorsDoNotAbortCycle` |
 | Concurrency within limit; cancellation stops the cycle; marking/starting on `WithoutCancel`. | poller `TestConcurrencyLimit`, `TestRunStopsOnCancel` |
 | Form cannot deploy an arbitrary ref: unknown version ID → 404; version of another repo → 404, no job. | jobs `TestTriggerVersion_*`, web `TestUpdateService_Version*` |
 | Pinned/tagless refs routed only to protocol ≥ 3 agents; old agents → failed "agent too old" job. | jobs `TestTriggerVersion_RequiresPinnedProtocol`, hub `TestRoute` |
 | Agent deploys a pinned ref verbatim, no registry resolve. | updater `TestResolve`, `TestComposeWritesPinnedRef`, `TestSwarmUsesPinnedRef` |
 | Rollback to older digest of same tag keeps newer suggestion; pick of suggested digest clears it. | store `TestFinishJob_ClearsAvailableOnlyWhenThatVersionWasDeployed` (extended) |
 | Running versions recorded only with a digest; re-snapshot does not duplicate or reorder. | store `TestSnapshotRecordsRunningVersions` |
 | Digest-less webhook row hidden once the tag's digest is known; order, limit, repo filter. | store `TestRecentVersions` |
 | `available_source = 'poll'` passes the CHECK. | store `TestSetAvailable_RecordsVersion` |
 | E2E: pick of a pinned version reaches the agent as that exact ref. | e2e `TestPickPinnedVersion` |
 ## Frozen contracts
 ### C1. Migration `internal/master/store/migrations/0004_polling.sql`
 ```sql
 ALTER TABLE services DROP CONSTRAINT services_available_source_check;
 ALTER TABLE services ADD CONSTRAINT services_available_source_check
    CHECK (available_source IN ('', 'webhook', 'poll'));
 ALTER TABLE services ADD COLUMN available_digest text NOT NULL DEFAULT '';
 ALTER TABLE services ADD COLUMN last_poll_error  text NOT NULL DEFAULT '';
 -- Every version the master has seen. '' means "no tag" / "digest unknown";
 -- never NULL, so the unique constraint deduplicates.
 CREATE TABLE image_versions (
    id            bigserial   PRIMARY KEY,
    repo          text        NOT NULL,
    tag           text        NOT NULL DEFAULT '',
    digest        text        NOT NULL DEFAULT '',
    source        text        NOT NULL CHECK (source IN ('webhook', 'poll', 'running')),
    first_seen_at timestamptz NOT NULL,
    CONSTRAINT image_versions_tag_or_digest CHECK (tag <> '' OR digest <> ''),
    CONSTRAINT image_versions_unique UNIQUE (repo, tag, digest)
 );
 CREATE INDEX image_versions_recent ON image_versions (repo, first_seen_at DESC, id DESC);
 ```
 Confirm the existing constraint name in `0003_webhook.sql` before relying on it.
 ### C2. `internal/master/model`
 ```go
 // Service gains:
 AvailableDigest string // digest of AvailableRef when known (poll); "" from the webhook
 LastPollError   string // last registry error for this service's followed ref; "" when fine
 const SourcePoll = "poll"       // alongside SourceWebhook
 const SourceRunning = "running" // image_versions only
 type Available struct { Ref, Digest, Source string; At time.Time } // Digest added
 // ImageVersion is one row of image_versions.
 type ImageVersion struct {
    ID          int64
    Repo, Tag, Digest, Source string
    FirstSeenAt time.Time
 }
 // Ref returns "repo:tag@digest", "repo:tag" or "repo@digest".
 func (v ImageVersion) Ref() string
 ```
 ### C3. Store
 ```go
 // SetAvailable sets available_ref/_digest/_source/_at and, in the same
 // transaction, inserts image_versions(repo, tag parsed from a.Ref, a.Digest,
 // a.Source, a.At) ON CONFLICT DO NOTHING.
 SetAvailable(ctx, id int64, a model.Available) error
 // RecentVersions returns up to limit versions of repo, newest first_seen_at
 // first (ties: id DESC). It leaves out a digest-less row when a row with the
 // same repo and tag and a digest exists.
 RecentVersions(ctx, repo string, limit int) ([]model.ImageVersion, error)
 // GetImageVersion returns one version or model.ErrNotFound.
 GetImageVersion(ctx, id int64) (model.ImageVersion, error)
 // SetPollError sets last_poll_error on the given services ("" clears).
 SetPollError(ctx, ids []int64, msg string) error
 ```
 `RecentVersions` SQL:
 ```sql
 SELECT id, repo, tag, digest, source, first_seen_at FROM image_versions v
 WHERE v.repo = $1
  AND NOT (v.digest = '' AND EXISTS (
        SELECT 1 FROM image_versions d
        WHERE d.repo = v.repo AND d.tag = v.tag AND d.digest <> ''))
 ORDER BY v.first_seen_at DESC, v.id DESC
 LIMIT $2
 ```
 `upsertService` (running inventory): after the service upsert, on the same `execer`, only when `svc.Digest != ""`:
 ```sql
 INSERT INTO image_versions (repo, tag, digest, source, first_seen_at)
 VALUES ($1, $2, $3, 'running', $4) ON CONFLICT DO NOTHING
 ```
 `repo`/`tag` from `imageref.Parse(svc.Image)`, `digest` = `svc.Digest`. A bare image is recorded with `tag = ''`, not `latest`. Covers `ApplySnapshot` and `UpsertService` with no hub change.
 `serviceColumns` / `scanService` gain `s.available_digest, s.last_poll_error`.
 `FinishJob` clearing rule (replaces phase 3's). With `requestedRef, requestedDigest, _ := strings.Cut(to_ref_as_created, "@")`:
 ```sql
 available_ref = $4 AND available_at <= $3
  AND ($5 = '' OR available_digest = $5)
 ```
 `$5` = `requestedDigest`. A digest-pinned job does NOT clear a digest-less (webhook) suggestion: a rollback to `main@old` must not hide that a newer `main` was announced. The badge then stays until a plain tag job succeeds; that is accepted. Clearing sets `available_ref=''`, `available_digest=''`, `available_source=''`, `available_at=NULL`.
 ### C4. `internal/master/registry`
 ```go
 // Client looks up tag digests with HEAD requests only.
 type Client struct{ /* keychain authn.Keychain, userAgent string */ }
 // New uses authn.DefaultKeychain ($DOCKER_CONFIG/config.json, else ~/.docker/config.json).
 func New(userAgent string) *Client
 // Digest returns the digest ref ("repo:tag", any digest suffix ignored) points
 // at: the index digest for multi-arch images. HEAD /v2/<repo>/manifests/<tag> only.
 func (c *Client) Digest(ctx context.Context, ref string) (string, error)
 ```
 Implementation: `name.ParseReference(repo+":"+tag)`; `remote.Head(r, remote.WithContext(ctx), remote.WithAuthFromKeychain(c.keychain), remote.WithUserAgent(c.userAgent))`; return `desc.Digest.String()`. Never fall back to GET if `Docker-Content-Digest` is missing.
 ### C5. `internal/master/poller`
 ```go
 type Store interface {
    ListServices(ctx context.Context) ([]model.Service, error)
    SetAvailable(ctx context.Context, id int64, a model.Available) error
    SetPollError(ctx context.Context, ids []int64, msg string) error
 }
 type Jobs interface {
    Start(ctx context.Context, svc model.Service, req jobs.StartRequest) (model.Job, error)
 }
 type Registry interface { Digest(ctx context.Context, ref string) (string, error) }
 type Publisher interface { Publish(t events.Topic) }
 type Config struct {
    Interval   time.Duration // POLL_INTERVAL
    FirstDelay time.Duration // 1m in production; small in tests
    Workers    int           // 4
 }
 func New(st Store, j Jobs, reg Registry, pub Publisher, cfg Config, log *slog.Logger) *Poller
 // Run polls until ctx is done. Blocking.
 func (p *Poller) Run(ctx context.Context)
 // Cycle runs one poll over all services (exported for tests and e2e).
 func (p *Poller) Cycle(ctx context.Context)
 // SkipReason says why svc is not polled: "follows no tag", "built from
 // source", "no registry digest", or "" when it is polled.
 func SkipReason(svc model.Service) string
 ```
 Cycle semantics, per distinct `ref = jobs.FollowedRef(svc.ActualRef)` over non-removed, non-skipped services:
 1. `D, err := reg.Digest(ctx, ref)` with a 30s timeout per call.
 2. On error: `SetPollError(ids whose LastPollError != msg, msg)`, log `poll_failed` (ref, error).
 3. On success: `SetPollError(ids with LastPollError != "", "")`. Then per service:
   - `svc.ActualDigest == D` → nothing.
   - `svc.AvailableRef == ref && svc.AvailableDigest == D` → nothing.
   - Otherwise `SetAvailable({ref, D, poll, now})`, and if `UpdateMode == auto`, `Start(TriggerPoll, Ref: ref)`. `ErrJobActive` and `NotUpdatableError` logged at debug and ignored.
 Steps 2 and 3 use `context.WithoutCancel(ctx)`. At the end, publish `events.Services` if anything was written; log `poll_cycle` with refs, ok, failed, suggested, jobs, skipped, duration_ms.
 ### C6. `internal/master/jobs` and hub
 ```go
 // Hub.Route gains the minimum protocol version for this job:
 Route(svc model.Service, minProtocol int) (model.Route, error)
 // Store gains:
 GetImageVersion(ctx context.Context, id int64) (model.ImageVersion, error)
 // TriggerVersion starts a manual job to exactly version versionID of the
 // service's repo. model.ErrNotFound for an unknown service, an unknown
 // version, or a version of another repo. Otherwise what Start returns.
 func (s *Service) TriggerVersion(ctx context.Context, serviceID, versionID int64, requestedBy string) (model.Job, error)
 ```
 - Ref = `svc.ImageRepo` + `":"+tag` if tag set + `"@"+digest` if digest set.
 - Calls `Start` with `TriggerManual` and explicit ref ("follows no tag" refusal does not apply; removed, self, built, container still do).
 - `Start` passes `protocol.MinPinnedRefVersion` to `Route` when the ref has a digest or no tag, otherwise `protocol.MinJobVersion`.
 ### C7. Protocol and agent
 ```go
 const Version = 3
 const MinJobVersion = 2       // unchanged: repo:tag jobs
 const MinPinnedRefVersion = 3 // jobs whose Ref has a digest or no tag
 ```
 `JobStart` doc: "Ref is `repo:tag`, `repo:tag@digest` or `repo@digest`. For `repo:tag` the agent resolves the tag's current digest; a ref with a digest is deployed as is."
 `Resolver.Resolve`: digest present → return ref unchanged, no `DistributionInspect`; tag present → resolve as today; neither → error "has no tag to resolve" (unchanged). Compose writes the ref verbatim into the override, Swarm sets it as the image. Executors print `pinned <ref>` instead of `resolved <ref>` when nothing was resolved.
 ### C8. Master configuration
 - `POLL_INTERVAL`: default `6h`, positive Go duration, error `"POLL_INTERVAL must be a positive duration such as 6h, got %q"`.
 - `DOCKER_CONFIG`: not parsed by `config`; `go-containerregistry` reads it. `main` logs `registry_credentials` with the path and whether `config.json` exists, never its contents.
 ### C9. Web
 - `POST /services/{id}/update` gains optional form field `version` (an `image_versions.id`): empty → `Trigger` as today; non-numeric → `400 invalid version`; otherwise `TriggerVersion`. Results: `303` to the job (also on `ErrJobActive`), `404` on `ErrNotFound`, `409` re-render "Cannot update: <reason>."
 - `web.Store` gains `RecentVersions`; `web.Jobs` gains `TriggerVersion`.
 - Service page: "Versions" table, up to 10 rows: ref (shortRef, full ref as title), source, first seen, "running" marker when digest equals `ActualDigest`, a POST form with `csrf_token` and `version`. Forms shown only when `AutoBlocked == ""`. Rows with empty tag note "the service will follow no tag afterwards". When the service is `auto`, header notes auto mode will move it forward at the next new version. A "Polling" row shows `SkipReason`, or `last_poll_error`, or "every <interval>".
 - Overview: badge "follows no tag" when `FollowedRef(ActualRef) == ""`; badge "poll failed" (title = error) when `LastPollError != ""`. Template func `followsNoTag`.
 ## Tasks
 ### Task 1: Store and model (C1, C2, C3)
 Migration, model fields, store methods, `upsertService` running versions, `FinishJob` rule. Integration tests: SetAvailable writes digest/source poll and records version, second call keeps first_seen_at; TestSnapshotRecordsRunningVersions (tag+digest, tagless digest, no digest skipped, re-snapshot idempotent); TestRecentVersions (order, limit 10, repo filter, digest-less row hidden only when a digest row for same tag exists); GetImageVersion ErrNotFound; SetPollError sets/clears, upsert does not touch last_poll_error/available_*; FinishJob table extended (tag job clears poll suggestion; pick tag@old keeps suggestion tag@new; pick tag@new clears; webhook suggestion (digest '') cleared by matching tag; tagless pick keeps it). Update the e2e `memStore` in the same commit.
 ### Task 2: Protocol, agent and hub routing (C7, C6 Route)
 Bump protocol, MinPinnedRefVersion, docs; Resolver pass-through; compose/swarm output line; `hub.Route(svc, minProtocol)`. Tests: TestResolve table (tag resolves; tag@d and @d pass through with zero inspect calls; bare errors); compose pinned ref verbatim in override; swarm spec image equals pinned ref; route rows v2 agent + min 3 → ErrAgentTooOld, v3 → ok, swarm fallback to v3 manager when reporting manager is v2.
 ### Task 3: Jobs (C6)
 `Start` picks min version from the ref; `TriggerVersion`. Tests: version of other repo → ErrNotFound, no CreateJob; ref construction table; min version passed to fake hub (plain Trigger → 2, pinned → 3); explicit ref bypasses no-tag but not removed/self/built/container; existing tests green.
 ### Task 4: Registry client and poller (C4, C5)
 Registry tests (local httptest + ggcr/pkg/registry, no network): random.Index digest equals index digest; recording handler shows only HEAD on /manifests/; basic-auth handler succeeds with temp DOCKER_CONFIG (t.Setenv) and fails without; 404 → error. Verify behaviour of DefaultKeychain when config.json has credsStore but no helper binary; document. Poller tests per Review Focus, plus Run test (first cycle after delay, no overlap, returns on cancel).
 ### Task 5: Config, wiring, web (C8, C9)
 Config POLL_INTERVAL with tests; add to setAll list. `cmd/master/main.go`: registry.New, poller, `go p.Run(ctx)` with done channel, wait for it after HTTP shutdown and before st.Close; log registry_credentials. Web handlers/templates per C9 with tests (picker ≤10 rows with CSRF and version IDs; hidden for built/self/container/removed; POST with version → TriggerVersion, without → Trigger; 400/404/409; overview badges; Polling row).
 ### Task 6: End-to-end
 `internal/e2e/versions_test.go`: TestPickPinnedVersion; TestPollStartsAutoJob (poller.Cycle with fake registry → one TriggerPoll job to repo:tag; second Cycle starts nothing). `currentHub.Route` new signature.
 ### Task 7: Documentation
 README (POLL_INTERVAL, DOCKER_CONFIG, "Registry polling" section: polled/skipped, credentials only inline auths, Docker Hub note, version picker & rollback, auto caveat, agent protocol 3 for pinned/tagless picks). Spec: replace "Believed not to count against Docker Hub pull limits for HEAD requests — verify during implementation" with verified statement + source link (docs.docker.com/docker-hub/usage/pulls: "Using GET emulates a real pull and counts towards the limit. Using HEAD won't."; note the separate unquantified abuse rate limit). `deploy/master.stack.yml`: mount config dir read-only, set DOCKER_CONFIG. CLAUDE.md: architecture entries for registry and poller; gotchas (HEAD only; digest-less webhook row hidden once digest known; image_versions uses '' not NULL; picks by version ID and pinned/tagless need protocol 3; FinishJob rule includes digest (replace existing bullet); rollback to repo@digest makes the service follow no tag); phase 4 plan reference. Save this plan as `docs/superpowers/plans/2026-10-13-master-agent-phase-4-polling.md`.
 ## Final verification
 ```bash
 gofmt -l . ; go vet ./...
 go build ./cmd/master ./cmd/agent && rm -f master agent
 go test ./...
 go test -race ./internal/agent/... ./internal/master/... ./internal/e2e/
 DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
 ```
@@ -247,8 +247,12 @@ them. An agent recreating itself would kill the job it is running.
 - Mechanism: `HEAD /v2/<repo>/manifests/<tag>`; the returned digest is compared with
  `actual_digest`. Use `go-containerregistry` — it handles Docker Hub's token auth and
  multi-arch indexes.
- Interval `POLL_INTERVAL`, default 6h. Believed not to count against Docker Hub pull
+- Interval `POLL_INTERVAL`, default 6h. Docker Hub documents that HEAD requests do not
-  limits for HEAD requests — verify during implementation.
+  count against the pull limit ("Using GET emulates a real pull and counts towards the
  limit. Using HEAD won't."; verified against
  https://docs.docker.com/docker-hub/usage/pulls/). The same page mentions a separate,
  unquantified abuse rate limit for many pulls from one IP address, so the poller stays
  HEAD-only and never downloads a manifest.
 - Credentials: a Docker `config.json` (as written by `docker login`) mounted into the
  master. No credential UI, no encryption in Postgres.