Detect newer version tags for third-party images #11

New Issue

2026-10-12T12:09:17Z

shcizo commented

2026-10-12 12:09:17 +00:00

Detect newer version tags for images we do not build (e.g. gitea/gitea:1.27.3 → 1.27.4, postgres:17.10 → 17.11, grafana/loki:3.4.2 → 3.4.3). Polling (phase 4) only sees a new digest under the same tag, and a pinned patch tag like 1.27.3 is never rebuilt, so today such a service looks current forever. The newer tag never reaches image_versions either, so it cannot be picked in the UI.

Pinning exact versions is good practice; this issue makes it cheap to follow. Floating tags (postgres:17, gitea/gitea:1.27) are already covered by phase 4 and are not the target here.

Scope (MVP)

Suggest only. The UI shows that a newer tag exists. No automatic job to a new tag, not even for a patch, and not for auto services.
Accepting a suggestion is an ordinary manual job to <image_repo>:<new tag>, the same path a webhook job with an explicit tag already takes.
Unparseable tags are ignored silently (latest, nightly, main, sha-…).

Comparison rules

A tag is split into prefix, numeric part (\d+(\.\d+)*) and suffix. A candidate is only suggested when, compared with the currently followed tag:

prefix and suffix are identical (17.10-alpine is only compared with *-alpine, never -bookworm),
the numeric part has the same number of components (17.10 → 17.11, never 17 or 17.11.1),
the first component (major) is equal,
the numeric parts compare greater, component by component as integers.

The highest candidate wins. These rules belong in their own small package with a table-driven test.

Prior art (checked)

Diun (crazy-max/diun, MIT) was read for reuse. It does not decide "newer": watch_repo only notifies about tags it has not stored before, and sort_tags: semver only decides which tags survive max_tags. It mixes variants (17.11-bookworm outranks 17.10-alpine) and has no major boundary. Worth taking:

the regex that drops cosign artifact tags (sha256-<64 hex>[.att|.sbom|.sig], pkg/registry/tags.go),
ignoring unparseable tags.

Its listing code (containers/image) is not worth pulling in. Renovate has the best Docker tag versioning rules but is AGPL-3.0: read the rules, do not copy code. Watchtower only checks the same tag, like phase 4.

Technical notes

Tag listing: remote.List in go-containerregistry (v0.22.1) does GET /v2/<repo>/tags/list and follows the Link header through all pages (pkg/v1/remote/puller.go:134-150). Same authn.DefaultKeychain as the existing HEAD lookup.
HEAD-only rule gets narrower, not dropped. Docker Hub counts a manifest GET as a pull; a tag-list GET is not. The comment in internal/master/registry and the test that asserts no GET must say "no manifest GET" instead of "HEAD only".
One tag list per followed repo per poll cycle, not per service.

Open questions (decide before implementing)

What happens to the followed tag and desired_ref when a suggestion is accepted? Does the service now follow 1.27.4? This interacts with drift (#10: actual_ref ≠ desired_ref) and with when FinishJob clears available_*.
Where is the suggestion stored? A separate column next to available_*, or available_* with a different meaning? The existing clearing rules assume to_ref has the same tag as available_ref.
Which tag formats actually occur? Check the stack files on the other two servers to confirm the four rules cover them.

Order

After #10 (drift and merge), because both change how desired_ref and available_* behave. UI badges after #18 (new service row template).

Spec, "Out of scope": https://gitea.shcizo.se/shcizo/package-updater/src/branch/main/docs/superpowers/specs/2026-10-12-master-agent-design.md

Detect newer version *tags* for images we do not build (e.g. `gitea/gitea:1.27.3` → `1.27.4`, `postgres:17.10` → `17.11`, `grafana/loki:3.4.2` → `3.4.3`). Polling (phase 4) only sees a new digest under the same tag, and a pinned patch tag like `1.27.3` is never rebuilt, so today such a service looks current forever. The newer tag never reaches `image_versions` either, so it cannot be picked in the UI. Pinning exact versions is good practice; this issue makes it cheap to follow. Floating tags (`postgres:17`, `gitea/gitea:1.27`) are already covered by phase 4 and are not the target here. ## Scope (MVP) - **Suggest only.** The UI shows that a newer tag exists. No automatic job to a new tag, not even for a patch, and not for `auto` services. - **Accepting a suggestion is an ordinary manual job** to `<image_repo>:<new tag>`, the same path a webhook job with an explicit tag already takes. - **Unparseable tags are ignored silently** (`latest`, `nightly`, `main`, `sha-…`). ## Comparison rules A tag is split into **prefix**, **numeric part** (`\d+(\.\d+)*`) and **suffix**. A candidate is only suggested when, compared with the currently followed tag: 1. prefix and suffix are identical (`17.10-alpine` is only compared with `*-alpine`, never `-bookworm`), 2. the numeric part has the same number of components (`17.10` → `17.11`, never `17` or `17.11.1`), 3. the first component (major) is equal, 4. the numeric parts compare greater, component by component as integers. The highest candidate wins. These rules belong in their own small package with a table-driven test. ## Prior art (checked) **Diun** (crazy-max/diun, MIT) was read for reuse. It does **not** decide "newer": `watch_repo` only notifies about tags it has not stored before, and `sort_tags: semver` only decides which tags survive `max_tags`. It mixes variants (`17.11-bookworm` outranks `17.10-alpine`) and has no major boundary. Worth taking: - the regex that drops cosign artifact tags (`sha256-<64 hex>[.att|.sbom|.sig]`, `pkg/registry/tags.go`), - ignoring unparseable tags. Its listing code (containers/image) is not worth pulling in. **Renovate** has the best Docker tag versioning rules but is AGPL-3.0: read the rules, do not copy code. Watchtower only checks the same tag, like phase 4. ## Technical notes - **Tag listing:** `remote.List` in go-containerregistry (v0.22.1) does `GET /v2/<repo>/tags/list` and follows the `Link` header through all pages (`pkg/v1/remote/puller.go:134-150`). Same `authn.DefaultKeychain` as the existing HEAD lookup. - **HEAD-only rule gets narrower, not dropped.** Docker Hub counts a *manifest* GET as a pull; a tag-list GET is not. The comment in `internal/master/registry` and the test that asserts no GET must say "no manifest GET" instead of "HEAD only". - One tag list per followed repo per poll cycle, not per service. ## Open questions (decide before implementing) - **What happens to the followed tag and `desired_ref` when a suggestion is accepted?** Does the service now follow `1.27.4`? This interacts with drift (#10: `actual_ref` ≠ `desired_ref`) and with when `FinishJob` clears `available_*`. - **Where is the suggestion stored?** A separate column next to `available_*`, or `available_*` with a different meaning? The existing clearing rules assume `to_ref` has the same tag as `available_ref`. - **Which tag formats actually occur?** Check the stack files on the other two servers to confirm the four rules cover them. ## Order After #10 (drift and merge), because both change how `desired_ref` and `available_*` behave. UI badges after #18 (new service row template). Spec, "Out of scope": https://gitea.shcizo.se/shcizo/package-updater/src/branch/main/docs/superpowers/specs/2026-10-12-master-agent-design.md

shcizo added the later label 2026-10-12 12:09:17 +00:00

shcizo referenced this issue from a commit

2026-10-13 09:58:54 +00:00

feat(master): suggest newer version tags for third-party images (#11)

shcizo referenced this issue from a commit

2026-10-13 09:58:54 +00:00

fix(master): review fixes for newer-tag suggestions (#11)

shcizo referenced a pull request that will close this issue

2026-10-13 10:14:32 +00:00

feat(master): suggest newer version tags for third-party images (#11) #31

shcizo closed this issue

2026-10-13 10:18:59 +00:00