Compare commits

..
5 Commits
Author SHA1 Message Date
shcizo b1d6e5e705 fix: review fixes for webhook and auto mode (#8)
Clear available_* only when the deployed ref equals it, drop the unused
poll source, stop leaking internal errors to CI, use the request-scoped
logger, validate X-Request-ID, re-read jobs once after the timeout,
report refused when no target ref exists, and tidy UI, deploy and docs.
2026-10-12 20:48:43 +02:00
shcizo f25a7a7779 docs: webhook contract, auto mode and phase 3 plan (#8) 2026-10-12 20:43:11 +02:00
shcizo 71d66846f2 feat(web): update-mode toggle and available-version badge (#8) 2026-10-12 20:41:41 +02:00
shcizo 5d95a35b3b feat(master): POST /update webhook with auto-mode updates (#8) 2026-10-12 20:40:38 +02:00
shcizo cfc6ea6713 feat(store): available ref, update mode and services-by-repo for the webhook (#8) 2026-10-12 20:40:04 +02:00
58 changed files with 177 additions and 5746 deletions
+3 -60
View File
@@ -36,8 +36,6 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
- `internal/master/config` — env-var loading - `internal/master/config` — env-var loading
- `internal/master/store` — Postgres access and migrations - `internal/master/store` — Postgres access and migrations
- `internal/master/hub` — agent connections: auth, per-connection writer, routing, job reconcile on hello - `internal/master/hub` — agent connections: auth, per-connection writer, routing, job reconcile on hello
- `internal/master/registry` — HEAD-only digest lookup against a registry (go-containerregistry, `DOCKER_CONFIG` credentials)
- `internal/master/poller` — every `POLL_INTERVAL`: one lookup per followed `repo:tag`, sets the available version, starts auto jobs
- `internal/master/jobs` — decides whether a service may be updated and starts a job on the right agent - `internal/master/jobs` — decides whether a service may be updated and starts a job on the right agent
- `internal/master/webhook` — `POST /update`: records the available version, starts jobs for auto services, waits for them - `internal/master/webhook` — `POST /update`: records the available version, starts jobs for auto services, waits for them
- `internal/master/web`, `webauth` — HTMX UI behind OIDC login, SSE signals - `internal/master/web`, `webauth` — HTMX UI behind OIDC login, SSE signals
@@ -64,37 +62,12 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
list in `known_jobs`, and only then accepts jobs for that connection. list in `known_jobs`, and only then accepts jobs for that connection.
- **The webhook never creates services; new services always start `manual`.** The upsert - **The webhook never creates services; new services always start `manual`.** The upsert
does not touch `update_mode`. does not touch `update_mode`.
- **Drift is never corrected automatically; poll/webhook jobs are refused for a drifted
service** (`jobs.Start`, `ReasonDrifted`). Only a manual job, `RestoreDesired` or an
accept gets it out.
- **Merging is never automatic; merge is the only path that deletes a `services` row**
(`store.MergeService`). The old row keeps its id and takes the duplicate's location.
- **Auth**: agents by token over pinned TLS; people by OIDC; CI by the `UPDATER_API_KEY` - **Auth**: agents by token over pinned TLS; people by OIDC; CI by the `UPDATER_API_KEY`
bearer token on `POST /update` only. Every state-changing UI request is a POST with a bearer token on `POST /update` only. Every state-changing UI request is a POST with a
CSRF token. CSRF token.
## Gotchas ## Gotchas
- **Registry access is HEAD only** (`internal/master/registry`). Docker Hub counts a manifest
GET as a pull; the client never falls back to GET, and a test asserts this.
- **`image_versions` uses `''` instead of NULL** for a missing tag or digest, so
`UNIQUE (repo, tag, digest)` deduplicates. The webhook records `(repo, tag, '')`; that
row is hidden from `RecentVersions` when a row with the same tag and a digest was first
seen at or after it.
- **The UI picks a version by `image_versions.id`, never by a ref from the form**, and the row
must belong to the service's `image_repo`. A digest-pinned or tagless pick needs agent
protocol 3 (`protocol.MinPinnedRefVersion`); `Hub.Route` takes the minimum version.
- **Picking a tagless version (`repo@digest`) leaves the service following no tag:** it is no
longer polled and the plain Update button is refused.
- **`services.last_tag_digest`** is set by the poller when it starts an auto job and when it
sees the service already running the tag's digest, and by `FinishJob` after any successful
plain tag job (not pinned/tagless picks; the digest is the one the agent pinned). The poller
starts no job for that digest, nor for one `desired_ref` (`ref@digest`) already names, and
re-reads the service before `Start`; so a rollback of an auto service holds until the
registry shows a different digest. `ErrJobActive` and `NotUpdatableError` leave it unset so
the next cycle retries.
- **A bare image's running version is recorded with tag `latest`** (like `jobs.FollowedRef`);
`repo@digest` is recorded with tag `''`. Only `sha256:<64 hex>` digests are recorded.
- **Image matching is case-sensitive.** The tricky case is `localhost:5000/foo:v1` where - **Image matching is case-sensitive.** The tricky case is `localhost:5000/foo:v1` where
the first colon is a port, not a tag. See `internal/imageref`. the first colon is a port, not a tag. See `internal/imageref`.
- **Path safety uses `filepath.Rel` + ".." prefix check**, NOT `strings.HasPrefix`. - **Path safety uses `filepath.Rel` + ".." prefix check**, NOT `strings.HasPrefix`.
@@ -110,12 +83,6 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
agent's own jobs always pass it explicitly. agent's own jobs always pass it explicitly.
- **Override entries are never pruned.** A service deleted from the stack file keeps its - **Override entries are never pruned.** A service deleted from the stack file keeps its
`image:` line in `compose.override.yaml`. `image:` line in `compose.override.yaml`.
- **The override file gets the owner of the existing override, else of the stack's first
compose file.** The exception is a root-owned existing override (written before this
fix): then the compose file's owner is used, so it stops being root-owned. The agent runs
as root, so `WriteOverride` chowns the open temp file before the rename (never the path
afterwards: a symlink could be swapped in). A failure is a `warning:` line in the job
output, never a failed job.
- **There is no `STACKS_ROOT`.** Stack directories must be mounted read-write at the host - **There is no `STACKS_ROOT`.** Stack directories must be mounted read-write at the host
path (`/path:/path`), and `SELF_ID` must resolve to the agent's own container. On Docker path (`/path:/path`), and `SELF_ID` must resolve to the agent's own container. On Docker
Desktop/OrbStack bind sources are translated, so Compose jobs fail there. Desktop/OrbStack bind sources are translated, so Compose jobs fail there.
@@ -127,8 +94,6 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
- **Swarm updates need manager-node API access** for the agent doing the update. - **Swarm updates need manager-node API access** for the agent doing the update.
- **The image's default entrypoint is `updater-agent`.** The master overrides it with - **The image's default entrypoint is `updater-agent`.** The master overrides it with
`updater-master`. `updater-master`.
- **The example stack runs the master as nobody (`user: "65534:65534"`).** Never add `USER`
to the Dockerfile: the agent uses the same image and needs root for the Docker socket.
- **The webhook waits via a broker signal plus a store re-read, with a fallback poll** - **The webhook waits via a broker signal plus a store re-read, with a fallback poll**
(the broker drops signals for slow subscribers). Marking and starting jobs runs on (the broker drops signals for slow subscribers). Marking and starting jobs runs on
`context.WithoutCancel`, so a client that disconnects cannot leave a queued job no agent `context.WithoutCancel`, so a client that disconnects cannot leave a queued job no agent
@@ -137,31 +102,11 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
repo, never from the request's image string. With an explicit target the "follows no tag" repo, never from the request's image string. With an explicit target the "follows no tag"
refusal does not apply; removed, self, built and container still do. refusal does not apply; removed, self, built and container still do.
- **`available_*` is cleared by `FinishJob`** on success only when the job's `to_ref` - **`available_*` is cleared by `FinishJob`** on success only when the job's `to_ref`
(as created, without `@digest`) equals `available_ref`, `available_at <= job.created_at` (as created, without `@digest`) equals `available_ref` and `available_at <= job.created_at`
(that version is now running) and, when the job's `to_ref` carries a digest, that digest (that version is now running). A manual Update of the followed tag does not clear a
equals `available_digest`. A rollback to an older digest of the tag therefore keeps the badge for a different tag.
newer suggestion, and a digest-pinned pick does not clear a digest-less (webhook) one.
- **Job output is capped at `protocol.MaxJobOutput` (128 KiB, tail).** NUL bytes are - **Job output is capped at `protocol.MaxJobOutput` (128 KiB, tail).** NUL bytes are
stripped by the hub because Postgres text rejects them. stripped by the hub because Postgres text rejects them.
- **Drift compares by digest, not by string** (`model.RunsRef`): `repo:tag@D` is run by
`repo:tag` reporting digest D. Same digest under another tag is not drift. Different
repo strings (`nginx` vs `docker.io/library/nginx`) are.
- **`services.awaiting_inventory`** is true from a successful `FinishJob` until the next
inventory write for the row; `Service.Drifted()` is false meanwhile, since `desired_ref`
is ahead of `actual_ref` for a debounce interval after every job.
- **Accept never touches `last_tag_digest`, `available_*` or `update_mode`**, and the
`actual` form field (`Service.RunningRef()`, ref plus digest) is only a guard against a
changed service; the value written comes from the row. `RestoreDesired` sets
`last_tag_digest` to the drifted digest when it is another digest of the desired tag and
`last_tag_digest` is empty or equals the desired digest (otherwise it holds back a newer
digest and stays).
- **`awaiting_inventory` can hide real drift** until the next inventory change when a job
result arrives after the snapshot that shows the drift (an agent reconnect resends its
outbox after the snapshot), or when Swarm coalesces an update and a rollback into one
debounce interval. This is accepted.
- **Merge lock order is agent, then service rows by id** (as in inventory writes), and the
duplicate row is deleted before the old row is updated, because `location_key` is unique
and not deferrable. A removal racing a merge can be lost; the next snapshot fixes it.
## References ## References
@@ -169,8 +114,6 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
- Phase 1 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-1-inventory.md` - Phase 1 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-1-inventory.md`
- Phase 2 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md` - Phase 2 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md`
- Phase 3 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-3-webhook.md` - Phase 3 plan: `docs/superpowers/plans/2026-10-12-master-agent-phase-3-webhook.md`
- Phase 4 plan: `docs/superpowers/plans/2026-10-13-master-agent-phase-4-polling.md`
- Phase 5 plan: `docs/superpowers/plans/2026-10-13-master-agent-phase-5-drift-merge.md`
- Original single-instance design (historical): - Original single-instance design (historical):
`docs/superpowers/specs/2026-05-22-package-updater-design.md` `docs/superpowers/specs/2026-05-22-package-updater-design.md`
- Consumer-side CI integration: `gitea-action/` (calls the master's `POST /update`) - Consumer-side CI integration: `gitea-action/` (calls the master's `POST /update`)
+1 -120
View File
@@ -9,8 +9,6 @@ the same image: `updater-master` and `updater-agent` (the default entrypoint).
Design: [master/agent spec](docs/superpowers/specs/2026-10-12-master-agent-design.md). Design: [master/agent spec](docs/superpowers/specs/2026-10-12-master-agent-design.md).
Phase 2 plan: [jobs and manual update](docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md). Phase 2 plan: [jobs and manual update](docs/superpowers/plans/2026-10-12-master-agent-phase-2-jobs.md).
Phase 3 plan: [webhook and auto mode](docs/superpowers/plans/2026-10-12-master-agent-phase-3-webhook.md). Phase 3 plan: [webhook and auto mode](docs/superpowers/plans/2026-10-12-master-agent-phase-3-webhook.md).
Phase 4 plan: [registry polling and version history](docs/superpowers/plans/2026-10-13-master-agent-phase-4-polling.md).
Phase 5 plan: [drift and merge](docs/superpowers/plans/2026-10-13-master-agent-phase-5-drift-merge.md).
## What an update does ## What an update does
@@ -29,16 +27,12 @@ a time per service, and one job at a time per agent.
Use [`deploy/master.stack.yml`](deploy/master.stack.yml) as a starting point (Swarm stack; Use [`deploy/master.stack.yml`](deploy/master.stack.yml) as a starting point (Swarm stack;
needs Postgres and an OIDC provider). The master listens for the UI on `PORT` (default needs Postgres and an OIDC provider). The master listens for the UI on `PORT` (default
`8080`, put it behind your reverse proxy) and for agents on `AGENT_PORT` (default `8090`, `8080`, put it behind your reverse proxy) and for agents on `AGENT_PORT` (default `8090`,
pinned TLS, reached directly on the internal network). The example runs the master as pinned TLS, reached directly on the internal network).
`nobody` (`65534`); it needs no root. The agent runs as root because access to the Docker
socket is root on the host anyway.
| Variable | Required | Default | Purpose | | Variable | Required | Default | Purpose |
|---|---|---|---| |---|---|---|---|
| `UPDATER_API_KEY` | no | — | Bearer token for `POST /update`. Without it the webhook answers `503` and the master logs `webhook_disabled`. | | `UPDATER_API_KEY` | no | — | Bearer token for `POST /update`. Without it the webhook answers `503` and the master logs `webhook_disabled`. |
| `UPDATE_TIMEOUT` | no | `5m` | How long one `POST /update` waits for its auto jobs (Go duration). | | `UPDATE_TIMEOUT` | no | `5m` | How long one `POST /update` waits for its auto jobs (Go duration). |
| `POLL_INTERVAL` | no | `6h` | Time between registry polling cycles (Go duration, must be positive). The first cycle runs one minute after start. |
| `DOCKER_CONFIG` | no | `~/.docker` | Directory with a `config.json` holding registry credentials for polling, readable by the master's user. Only inline `auths` entries work; see [Registry polling](#registry-polling). |
The master's other settings (`DATABASE_URL`, `PUBLIC_URL`, `AGENT_URL`, `OIDC_*`, The master's other settings (`DATABASE_URL`, `PUBLIC_URL`, `AGENT_URL`, `OIDC_*`,
`ALLOWED_GROUP`, `AGENT_IMAGE`, `PORT`, `AGENT_PORT`, `LOG_LEVEL`) are shown in `ALLOWED_GROUP`, `AGENT_IMAGE`, `PORT`, `AGENT_PORT`, `LOG_LEVEL`) are shown in
@@ -106,9 +100,6 @@ Response (`200` or `500`):
| `500` | At least one result is `failed` or `timeout`. | | `500` | At least one result is `failed` or `timeout`. |
| `503` | `UPDATER_API_KEY` is not set on the master. | | `503` | `UPDATER_API_KEY` is not set on the master. |
A service that has **drifted** (see [Drift and moved services](#drift-and-moved-services))
is refused with its reason, in auto mode too, until someone restores or accepts it.
The webhook never creates services: the first deployment of a stack is done by hand. A The webhook never creates services: the first deployment of a stack is done by hand. A
new service always starts in **manual** mode; switch it with the *Make auto* button on the new service always starts in **manual** mode; switch it with the *Make auto* button on the
overview or the service page. Auto is refused for the updater's own services, `build:` overview or the service page. Auto is refused for the updater's own services, `build:`
@@ -121,116 +112,6 @@ The request is open for as long as the jobs run, so the reverse proxy in front o
master must allow a read timeout longer than `UPDATE_TIMEOUT`. If the client disconnects, master must allow a read timeout longer than `UPDATE_TIMEOUT`. If the client disconnects,
marking and starting jobs still complete; only the waiting stops. marking and starting jobs still complete; only the waiting stops.
## Registry polling
Besides the webhook, the master asks each registry whether the tag a service follows has
a new digest. Every `POLL_INTERVAL` it sends one `HEAD /v2/<repo>/manifests/<tag>` per
distinct followed `repo:tag` (a bare image counts as `:latest`; `repo:tag@digest` is polled
as `repo:tag`). It never downloads a manifest: Docker Hub counts a `GET` as a pull against
the rate limit and a `HEAD` as nothing
([source](https://docs.docker.com/docker-hub/usage/pulls/)), and Docker Hub also has a
separate abuse limit for many pulls from one IP address, so the poller stays HEAD-only. For a
multi-arch image the digest is the index digest, the same one `docker inspect` shows in
`RepoDigests`.
- **What counts as new.** A digest different from the one the service runs becomes the
service's *available* version (the overview badge, source `poll`). A digest the master
already suggested is not suggested again. A service in **auto** mode also gets an update
job to `repo:tag` (the agent resolves the digest itself, so agents on protocol version 2
keep working). The master remembers the newest digest it started or deployed for the tag
in `services.last_tag_digest` (set when polling starts a job, when polling sees the
service already running that digest, and when any plain tag job succeeds, whoever started
it) and starts no job for that digest again, whether the job
failed or the user rolled back afterwards. If the service already had a job when the
digest was found, the next cycle tries again.
- **What is not polled.** Services that follow no tag (`repo@digest`), `build:` services, and
services without a registry digest (locally built images). The service page says which.
The updater's own services and plain containers are polled, so their badge works, but
jobs for them are still refused.
- **Errors.** A failed lookup is stored for the services that follow that tag and shown as a
"poll failed" badge on the overview and in the "Polling" row of the service page; other
tags continue. A later success clears it.
- **Credentials.** The master reads `$DOCKER_CONFIG/config.json` (a file written by
`docker login`), mounted read-only into the container; without it registries are accessed
anonymously. The directory and file must be readable by the master's user (`65534` in the
example stack) without being world-readable: `chown -R 65534:65534` the directory and keep
`0700`/`0600`. Only inline `auths` entries are supported. A `credsStore` or `credHelpers`
entry makes every lookup fail with `error getting credentials ... executable file not found
in $PATH`, because the container has no credential helper binaries. The master logs
`registry_credentials` at startup with the directory, `config_json_found` and
`config_json_readable`, never the contents. If the file exists but cannot be opened it also
logs a `registry_credentials_unreadable` warning with the error.
- **Requests go wherever the image name points.** The master sends the lookup to the
registry host in the image name, including a `GET /v2/` ping that follows redirects.
Anyone who can start containers on a managed host can therefore make the master send
blind requests to internal hosts. The registry's answer is not shown: an error keeps only
the HTTP status and the registry's error codes; any other failure shows only "timeout",
"cannot reach registry" or "lookup failed". The full error goes to the master's log
(`registry_lookup_failed`).
### Version history and rollback
The master records every version it sees (webhook, polling, and what agents report as
running) in `image_versions`. The service page lists the 10 most recently first seen
versions of the service's image repo, marks the one that is running, and has a *Deploy*
button on each row. A webhook announces a tag but not its digest, so such a row has no
digest until polling or an agent reports one; it is hidden when a row with the same tag
and a digest was first seen at or after it. A bare image (`postgres`) is recorded as tag
`latest`.
Picking a version starts a manual job to exactly that reference. Picking an older one is a
rollback, including to a digest-pinned reference. A version without a tag deploys
`repo@digest` and the service then follows no tag, so it is no longer polled or updatable
with the plain *Update* button until it is moved to a tagged version again. Picks of
digest-pinned or tagless versions need an agent speaking protocol version 3 (this
release); on an older agent the job fails with "agent too old to deploy a pinned version
(needs protocol 3)".
A rollback made before the first poll cycle after the upgrade has seen the service is not
protected, because the master has not yet recorded the digest it was running.
A service in **auto** mode stays auto after a rollback. Polling does not undo it: the
service stays on the older version until the registry shows a digest of the followed tag
that differs from the newest one the master deployed. The newer version it was rolled back
from stays marked as available. A webhook that names a tag still starts a job for an auto
service every time, so a CI push moves it forward again.
## Drift and moved services
A service has **drifted** when it runs something other than the version the last update
job deployed. A service the updater never touched has nothing to drift from. Drift is
decided by what runs, not by how the reference is written: the digest decides when both
sides know one, so `repo:tag@sha256:…` is run by a service reporting `repo:tag` with that
digest, and a different tag with the same digest is not drift. (The design spec compares
the two reference strings; this is deliberately looser.) Between a successful job and the
agent's next inventory report the service is not counted as drifted, because the report
has not arrived yet.
The master never corrects drift on its own, because a manual `docker stack deploy` of a
newer tag may be intentional. Polling and the webhook start no job for a drifted
service, in auto mode too; a person can still start one by hand. There are two ways out:
- **Restore desired** deploys the recorded version again (a manual job to the exact
`repo:tag@digest`; an agent speaking protocol 3 is needed). If the service had drifted
to another digest of the same tag, that digest is remembered, so polling does not
deploy it again until the registry shows a newer one (unless `last_tag_digest` already
holds back a newer digest, which then stays).
- **Accept running** records what runs now as the desired version. It starts no job and
leaves the available suggestion and the update mode alone.
When a Compose service disappears from one agent and exactly one service with the same
project and service name appears on another, and the removed one has history (a desired
version, auto mode or jobs) while the new one has none, the service page suggests that it
moved. **Merge** moves the old row, with its history, mode and desired version, to the new
location and deletes the duplicate. Merging is never automatic. It can also be done by
hand for any removed service of the same kind (Compose or Swarm, names may differ), but
not for plain containers, the updater itself, or a service with a queued or running job.
A merge keeps the old row's mode and desired version, so a manually merged target with its
own history may show drift right after the merge and can be accepted.
The restore, accept and merge buttons arrive with the UI follow-up (issue #28); the endpoints
(`POST /services/{id}/restore`, `/accept`, `/merge`) and the data for them are in place.
## Adding agents ## Adding agents
Open the master's **Agents** page and add one. It shows a ready-to-run compose file with Open the master's **Agents** page and add one. It shows a ready-to-run compose file with
+2 -60
View File
@@ -11,7 +11,6 @@ import (
"net/http" "net/http"
"os" "os"
"os/signal" "os/signal"
"path/filepath"
"strings" "strings"
"syscall" "syscall"
"time" "time"
@@ -21,8 +20,6 @@ import (
"github.com/shcizo/package-updater/internal/master/events" "github.com/shcizo/package-updater/internal/master/events"
"github.com/shcizo/package-updater/internal/master/hub" "github.com/shcizo/package-updater/internal/master/hub"
"github.com/shcizo/package-updater/internal/master/jobs" "github.com/shcizo/package-updater/internal/master/jobs"
"github.com/shcizo/package-updater/internal/master/poller"
"github.com/shcizo/package-updater/internal/master/registry"
"github.com/shcizo/package-updater/internal/master/store" "github.com/shcizo/package-updater/internal/master/store"
"github.com/shcizo/package-updater/internal/master/web" "github.com/shcizo/package-updater/internal/master/web"
"github.com/shcizo/package-updater/internal/master/webauth" "github.com/shcizo/package-updater/internal/master/webauth"
@@ -40,39 +37,6 @@ func main() {
} }
} }
// pollFirstDelay is how long the master runs before its first registry
// polling cycle, so agents have reported their inventory.
const pollFirstDelay = time.Minute
// pollWorkers is how many registry lookups run at once.
const pollWorkers = 4
// registryCredentialsDir returns the directory the registry client reads
// credentials from (DOCKER_CONFIG, else ~/.docker), whether a config.json is
// in it, and the error from opening it when it exists but cannot be read (for
// example because the master runs as another user than the file owner). The
// file is opened and closed, never read; only the path is logged.
func registryCredentialsDir() (dir string, found bool, readErr error) {
dir = os.Getenv("DOCKER_CONFIG")
if dir == "" {
home, err := os.UserHomeDir()
if err != nil {
return "", false, nil
}
dir = filepath.Join(home, ".docker")
}
path := filepath.Join(dir, "config.json")
if _, err := os.Stat(path); err != nil {
return dir, false, nil
}
f, err := os.Open(path)
if err != nil {
return dir, true, err
}
_ = f.Close()
return dir, true, nil
}
func run() error { func run() error {
cfg, err := config.Load() cfg, err := config.Load()
if err != nil { if err != nil {
@@ -117,7 +81,6 @@ func run() error {
ui, err := web.New(st, agentHub, jobService, broker, web.Config{ ui, err := web.New(st, agentHub, jobService, broker, web.Config{
AgentURL: cfg.AgentURL, MasterKey: fingerprint, AgentImage: cfg.AgentImage, AgentURL: cfg.AgentURL, MasterKey: fingerprint, AgentImage: cfg.AgentImage,
PollInterval: cfg.PollInterval,
}, logger) }, logger)
if err != nil { if err != nil {
return err return err
@@ -164,43 +127,22 @@ func run() error {
BaseContext: baseContext, BaseContext: baseContext,
} }
credDir, credFound, credReadErr := registryCredentialsDir()
logger.Info("registry_credentials", "docker_config", credDir, "config_json_found", credFound,
"config_json_readable", credFound && credReadErr == nil)
if credReadErr != nil {
logger.Warn("registry_credentials_unreadable", "docker_config", credDir, "error", credReadErr)
}
pollCtx, stopPolling := context.WithCancel(ctx)
defer stopPolling()
registryPoller := poller.New(st, jobService, registry.New("package-updater-master/"+version, logger), broker, poller.Config{
Interval: cfg.PollInterval, FirstDelay: pollFirstDelay, Workers: pollWorkers,
}, logger)
pollDone := make(chan struct{})
go func() {
defer close(pollDone)
registryPoller.Run(pollCtx)
}()
errCh := make(chan error, 2) errCh := make(chan error, 2)
go func() { errCh <- webSrv.ListenAndServe() }() go func() { errCh <- webSrv.ListenAndServe() }()
go func() { errCh <- agentSrv.ListenAndServeTLS("", "") }() go func() { errCh <- agentSrv.ListenAndServeTLS("", "") }()
logger.Info("listening", "web", webSrv.Addr, "agents", agentSrv.Addr) logger.Info("listening", "web", webSrv.Addr, "agents", agentSrv.Addr)
var serveErr error
select { select {
case <-ctx.Done(): case <-ctx.Done():
logger.Info("shutdown_signal") logger.Info("shutdown_signal")
case err := <-errCh: case err := <-errCh:
if !errors.Is(err, http.ErrServerClosed) { if !errors.Is(err, http.ErrServerClosed) {
serveErr = err return err
} }
} }
shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second) shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel() defer cancel()
_ = webSrv.Shutdown(shutdownCtx) _ = webSrv.Shutdown(shutdownCtx)
_ = agentSrv.Shutdown(shutdownCtx) _ = agentSrv.Shutdown(shutdownCtx)
// The poller writes to the store, which is closed when run returns. return nil
stopPolling()
<-pollDone
return serveErr
} }
-42
View File
@@ -1,42 +0,0 @@
package main
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestRegistryCredentialsDir(t *testing.T) {
t.Run("missing file", func(t *testing.T) {
dir := t.TempDir()
t.Setenv("DOCKER_CONFIG", dir)
got, found, readErr := registryCredentialsDir()
assert.Equal(t, dir, got)
assert.False(t, found)
assert.NoError(t, readErr)
})
t.Run("readable file", func(t *testing.T) {
dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, "config.json"), []byte("{}"), 0o600))
t.Setenv("DOCKER_CONFIG", dir)
_, found, readErr := registryCredentialsDir()
assert.True(t, found)
assert.NoError(t, readErr)
})
t.Run("unreadable file", func(t *testing.T) {
if os.Geteuid() == 0 {
t.Skip("root can read any file")
}
dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, "config.json"), []byte("{}"), 0o000))
t.Setenv("DOCKER_CONFIG", dir)
_, found, readErr := registryCredentialsDir()
assert.True(t, found)
assert.Error(t, readErr)
})
}
-14
View File
@@ -10,19 +10,9 @@
# To run without the webhook, remove the pu_updater_api_key lines (the UPDATER_API_KEY # To run without the webhook, remove the pu_updater_api_key lines (the UPDATER_API_KEY
# part of the entrypoint, the service secret and the secrets entry); the endpoint then answers 503. # part of the entrypoint, the service secret and the secrets entry); the endpoint then answers 503.
# The reverse proxy must allow a read timeout longer than UPDATE_TIMEOUT (default 5m). # The reverse proxy must allow a read timeout longer than UPDATE_TIMEOUT (default 5m).
# 5. Optional, for registry polling of private images: put a config.json written by
# `docker login` (inline "auths" only; no credsStore) in /path/to/docker-config and
# uncomment DOCKER_CONFIG and the volumes entry below. The master runs as 65534:65534, so
# the directory and config.json must be readable by that user while staying private:
# `chown -R 65534:65534 /path/to/docker-config` (keep 0700/0600), or set `user:` below
# to the owner's uid:gid. Do not make config.json world-readable; it holds registry
# passwords. The directory must exist on every manager node (the master is placed on a
# manager), or Swarm rejects the task. Without it registries are accessed anonymously.
services: services:
master: master:
image: gitea.shcizo.se/shcizo/package-updater:latest image: gitea.shcizo.se/shcizo/package-updater:latest
# Runs as nobody: the master needs no root (the agent does, for the Docker socket).
user: "65534:65534"
entrypoint: ["/bin/sh", "-c", "DATABASE_URL=$$(cat /run/secrets/pu_database_url) OIDC_CLIENT_SECRET=$$(cat /run/secrets/pu_oidc_client_secret) UPDATER_API_KEY=$$(cat /run/secrets/pu_updater_api_key) exec /usr/local/bin/updater-master"] entrypoint: ["/bin/sh", "-c", "DATABASE_URL=$$(cat /run/secrets/pu_database_url) OIDC_CLIENT_SECRET=$$(cat /run/secrets/pu_oidc_client_secret) UPDATER_API_KEY=$$(cat /run/secrets/pu_updater_api_key) exec /usr/local/bin/updater-master"]
environment: environment:
PUBLIC_URL: https://updater.example.com PUBLIC_URL: https://updater.example.com
@@ -31,14 +21,10 @@ services:
OIDC_CLIENT_ID: package-updater OIDC_CLIENT_ID: package-updater
ALLOWED_GROUP: updater-admins ALLOWED_GROUP: updater-admins
# UPDATE_TIMEOUT: 5m # UPDATE_TIMEOUT: 5m
# POLL_INTERVAL: 6h
# DOCKER_CONFIG: /docker-config # step 5
secrets: secrets:
- pu_database_url - pu_database_url
- pu_oidc_client_secret - pu_oidc_client_secret
- pu_updater_api_key - pu_updater_api_key
# volumes: # step 5
# - /path/to/docker-config:/docker-config:ro
ports: ports:
# Agents connect here directly over the internal network, not via the proxy. # Agents connect here directly over the internal network, not via the proxy.
- target: 8090 - target: 8090
@@ -1,311 +0,0 @@
# Master/Agent Phase 4 — Registry Polling and Version History Implementation Plan
> This plan documents what to build for issue #9 and where it deviates from the spec. Code in "Frozen contracts" is final; tasks give the contract and the test cases.
**Goal:** The master records every version it sees (webhook, registry polling, running inventory) in `image_versions`. Every `POLL_INTERVAL` (default 6h) it checks each distinct followed `repo:tag` against its registry with one HEAD request. A new digest becomes the service's available suggestion, and `auto` services get a job. The Service page lists the 10 most recent versions of the service's repo. Picking one starts a manual job to exactly that reference; picking an older one is a rollback, including to a digest-pinned or tagless reference.
**Architecture:**
- New package `internal/master/registry`: thin wrapper around `go-containerregistry`'s `remote.Head` with `authn.DefaultKeychain` (reads `DOCKER_CONFIG`/`config.json`).
- New package `internal/master/poller`: owns the cycle (list services, group by followed ref, run HEADs with bounded concurrency, then set available / start auto jobs / store poll error per service). `cmd/master` runs it as a background goroutine on the signal context.
- Store: `image_versions` written in `SetAvailable` (webhook and poll, same transaction) and in `upsertService` (running inventory, inside the existing snapshot/upsert transactions).
- Jobs gains `TriggerVersion`, which takes a version ID, never a ref from the form. `Hub.Route` gains a minimum protocol version so a pinned or tagless ref only goes to agents that understand it.
- Agent: resolver passes digest-pinned refs through unchanged. Protocol version goes to 3.
**Tech Stack:** adds `github.com/google/go-containerregistry` (`pkg/name`, `pkg/v1/remote`, `pkg/authn`; `pkg/registry` and `pkg/v1/random` in tests only).
**Spec:** `docs/superpowers/specs/2026-10-12-master-agent-design.md`, sections "Versions: what the system has seen", "Data model", "Update", "Failure handling", "Registry polling", "UI". Phase 3 plan conventions still apply.
## Global Constraints
- `go test ./...` passes with no Docker, no Postgres and no network. Registry tests use `httptest` plus `ggcr/pkg/registry`. Postgres tests carry `//go:build integration`.
- Interfaces declared at the consumer (`poller.Store`, `poller.Jobs`, `poller.Registry`). Hand-written fakes and testify, table-driven where there is more than one case.
- GoDoc on every exported identifier. Conventional Commits, each ending with ` (#9)`.
- Registry credentials never appear in logs or in `last_poll_error`. Error text stored truncated to 500 bytes.
- Registry traffic is HEAD only: never `GET /v2/<repo>/manifests/...` (Docker Hub counts GETs as pulls). A test asserts this.
- A pick from the UI is a POST with the CSRF token and a version ID. The ref is built server-side from the `image_versions` row, and the row must belong to the service's `image_repo`.
- The poller never overlaps itself. Next cycle is scheduled after the previous one finishes.
- `go.mod` `go` directive must stay `1.26.3` after `go get`.
## Deliberate deviations from the spec in this phase
1. **Webhook records `(repo, tag, '')`, no digest.** The master does not resolve the digest at webhook time (no registry round-trip/credentials in the CI path). The digest arrives later from polling or running inventory. The picker hides a digest-less row when a row with the same `(repo, tag)` and a digest was first seen at or after it.
2. **Empty string instead of NULL** for missing tag or digest in `image_versions`, so `UNIQUE (repo, tag, digest)` works. A CHECK forbids both being empty.
3. **`first_seen_at` is never bumped.** Re-observation is `ON CONFLICT DO NOTHING`; `source` is the first source that saw it. "10 most recent" = "10 most recently first seen". A new digest of a tag is a new row at the top; old digest stays below (enables rollback of moving tags).
4. **Not polled:** services that follow no tag, `build:` services, services with empty `actual_digest`. UI shows the reason. Self and container services ARE polled (badge informative; jobs still refused by `jobs`).
5. **Poller does not re-suggest the same version.** If `available_ref == ref && available_digest == D`, the badge is not written again (no `available_at` bump). The auto job is decided separately: it starts only if the service is auto, `actual_digest != D`, `D != last_tag_digest` and `desired_ref != ref@D`, on a fresh read of the row. `last_tag_digest` is set when a job row was created (not on `ErrJobActive`/`NotUpdatableError`, which retry next cycle). A failed auto job is not retried every interval; a rollback of an auto service is not undone by polling until a new digest appears. The poller also sets `last_tag_digest` to D when `actual_digest == D` (seeding services that predate the column). `FinishJob` also sets `last_tag_digest` to the pinned digest after any successful job whose `to_ref` as created has no digest (manual, webhook or poll); pinned and tagless picks leave it, so a rollback holds whoever started the update.
6. **FinishJob clearing rule gets a digest condition** (C3). Rollback to an older digest of the same tag no longer clears the newer-version badge.
7. **Poll auto jobs use `repo:tag`, not `repo:tag@D`.** v2 agents keep working for poll and webhook jobs. Only UI picks of digest-pinned or tagless versions need a v3 agent.
8. **First cycle runs 1 minute after start**, then every `POLL_INTERVAL` measured from the end of the previous cycle.
9. **No `last_polled_at` column** (YAGNI).
10. **Overview gets two new badges: "follows no tag" and "poll failed"** (error as `title`). Service page gets a "Polling" row. "Drifted" is not in this phase.
11. **Credentials from `DOCKER_CONFIG`** (a directory), same name and meaning as on the agent. Only inline `auths` entries supported in the container.
## Implementation notes (differences found while building)
- The plan's test names `TestResolve` and `TestComposeWritesPinnedRef` became `TestResolver`
(extended), a subtest of `TestComposeExecutor` and `TestSwarmUsesPinnedRef`.
- Tasks 2 and 3 are one commit: changing `Hub.Route`'s signature breaks `jobs` and the e2e hub
wrapper until both are updated.
- `Poller.Cycle` runs the registry lookups concurrently and then records the results
sequentially (on `context.WithoutCancel`).
- `web.Config` gained `PollInterval` for the "every <interval>" text.
- `authn.DefaultKeychain` with a `credsStore` whose helper binary is missing makes the lookup fail
with `error getting credentials - err: exec: "docker-credential-...": executable file not found in $PATH`
(covered by `TestCredsStoreWithoutHelperFailsTheLookup`); it does not fall back to anonymous access.
## Review Focus
| Failure case | Test that covers it |
|---|---|
| Registry access is HEAD only; no manifest GET ever happens. | registry `TestDigestUsesHeadOnly` |
| Multi-arch: digest returned is the index digest (same as `RepoDigests` / Swarm spec). | registry `TestDigestOfIndexIsIndexDigest` |
| Credentials from `$DOCKER_CONFIG/config.json` are sent; anonymous works without the file. | registry `TestCredentialsFromDockerConfig`, `TestAnonymousWithoutConfig` |
| One HEAD per distinct followed ref (`repo:tag`, `repo:tag@d`, bare → `:latest`). | poller `TestOneHeadPerDistinctRef` |
| No tag / built / no digest are skipped, no HEAD. | poller `TestSkipped` |
| Error on one ref goes into its services' `last_poll_error`; others continue; later success clears it. | poller `TestErrorIsPerRef`, `TestErrorClearedOnSuccess` |
| Already-suggested digest neither re-suggested nor re-started. | poller `TestSameSuggestionIsIdempotent` |
| Auto gets exactly one `TriggerPoll` job to `repo:tag`; manual only gets marked. | poller `TestNewDigest` table |
| `ErrJobActive` / `NotUpdatableError` from `Start` do not abort the cycle. | poller `TestStartErrorsDoNotAbortCycle` |
| Concurrency within limit; cancellation stops the cycle; marking/starting on `WithoutCancel`. | poller `TestConcurrencyLimit`, `TestRunStopsOnCancel` |
| Form cannot deploy an arbitrary ref: unknown version ID → 404; version of another repo → 404, no job. | jobs `TestTriggerVersion_*`, web `TestUpdateService_Version*` |
| Pinned/tagless refs routed only to protocol ≥ 3 agents; old agents → failed "agent too old to deploy a pinned version (needs protocol 3)" job (`ErrAgentTooOldForPinned`, which also satisfies `errors.Is(err, ErrAgentTooOld)`). | jobs `TestTriggerVersion_RequiresPinnedProtocol`, hub `TestRoute` |
| Agent deploys a pinned ref verbatim, no registry resolve. | updater `TestResolve`, `TestComposeWritesPinnedRef`, `TestSwarmUsesPinnedRef` |
| Rollback to older digest of same tag keeps newer suggestion; pick of suggested digest clears it. | store `TestFinishJob_ClearsAvailableOnlyWhenThatVersionWasDeployed` (extended) |
| Running versions recorded only with a digest; re-snapshot does not duplicate or reorder. | store `TestSnapshotRecordsRunningVersions` |
| Digest-less webhook row hidden once the tag's digest is known; order, limit, repo filter. | store `TestRecentVersions` |
| `available_source = 'poll'` passes the CHECK. | store `TestSetAvailable_RecordsVersion` |
| E2E: pick of a pinned version reaches the agent as that exact ref. | e2e `TestPickPinnedVersion` |
## Frozen contracts
### C1. Migration `internal/master/store/migrations/0004_polling.sql`
```sql
ALTER TABLE services DROP CONSTRAINT services_available_source_check;
ALTER TABLE services ADD CONSTRAINT services_available_source_check
CHECK (available_source IN ('', 'webhook', 'poll'));
ALTER TABLE services ADD COLUMN available_digest text NOT NULL DEFAULT '';
ALTER TABLE services ADD COLUMN last_poll_error text NOT NULL DEFAULT '';
ALTER TABLE services ADD COLUMN last_tag_digest text NOT NULL DEFAULT '';
-- Every version the master has seen. '' means "no tag" / "digest unknown";
-- never NULL, so the unique constraint deduplicates.
CREATE TABLE image_versions (
id bigserial PRIMARY KEY,
repo text NOT NULL,
tag text NOT NULL DEFAULT '',
digest text NOT NULL DEFAULT '',
source text NOT NULL CHECK (source IN ('webhook', 'poll', 'running')),
first_seen_at timestamptz NOT NULL,
CONSTRAINT image_versions_tag_or_digest CHECK (tag <> '' OR digest <> ''),
CONSTRAINT image_versions_unique UNIQUE (repo, tag, digest)
);
CREATE INDEX image_versions_recent ON image_versions (repo, first_seen_at DESC, id DESC);
```
Confirm the existing constraint name in `0003_webhook.sql` before relying on it.
### C2. `internal/master/model`
```go
// Service gains:
AvailableDigest string // digest of AvailableRef when known (poll); "" from the webhook
LastPollError string // last registry error for this service's followed ref; "" when fine
const SourcePoll = "poll" // alongside SourceWebhook
const SourceRunning = "running" // image_versions only
type Available struct { Ref, Digest, Source string; At time.Time } // Digest added
// ImageVersion is one row of image_versions.
type ImageVersion struct {
ID int64
Repo, Tag, Digest, Source string
FirstSeenAt time.Time
}
// Ref returns "repo:tag@digest", "repo:tag" or "repo@digest".
func (v ImageVersion) Ref() string
```
### C3. Store
```go
// SetAvailable sets available_ref/_digest/_source/_at and, in the same
// transaction, inserts image_versions(repo, tag parsed from a.Ref, a.Digest,
// a.Source, a.At) ON CONFLICT DO NOTHING.
SetAvailable(ctx, id int64, a model.Available) error
// RecentVersions returns up to limit versions of repo, newest first_seen_at
// first (ties: id DESC). It leaves out a digest-less row when a row with the
// same repo and tag and a digest was first seen at or after it.
RecentVersions(ctx, repo string, limit int) ([]model.ImageVersion, error)
// GetImageVersion returns one version or model.ErrNotFound.
GetImageVersion(ctx, id int64) (model.ImageVersion, error)
// SetPollError sets last_poll_error on the given services ("" clears).
SetPollError(ctx, ids []int64, msg string) error
```
`RecentVersions` SQL:
```sql
SELECT id, repo, tag, digest, source, first_seen_at FROM image_versions v
WHERE v.repo = $1
AND NOT (v.digest = '' AND EXISTS (
SELECT 1 FROM image_versions d
WHERE d.repo = v.repo AND d.tag = v.tag AND d.digest <> ''
AND d.first_seen_at >= v.first_seen_at))
ORDER BY v.first_seen_at DESC, v.id DESC
LIMIT $2
```
`upsertService` (running inventory): after the service upsert, on the same `execer`, only when `svc.Digest` matches `^sha256:[0-9a-f]{64}$`:
```sql
INSERT INTO image_versions (repo, tag, digest, source, first_seen_at)
VALUES ($1, $2, $3, 'running', $4) ON CONFLICT DO NOTHING
```
`repo`/`tag` from `imageref.Parse(svc.Image)`, `digest` = `svc.Digest`. A bare image (no tag, no digest in the name) is recorded with `tag = 'latest'`, like `jobs.FollowedRef`, so it dedupes with the poll row; `repo@digest` is recorded with `tag = ''`. Covers `ApplySnapshot` and `UpsertService` with no hub change.
`serviceColumns` / `scanService` gain `s.available_digest, s.last_poll_error, s.last_tag_digest`. `SetLastTagDigest(ctx, id, digest)` sets the last column; inventory writes never touch it.
`FinishJob` clearing rule (replaces phase 3's). With `requestedRef, requestedDigest, _ := strings.Cut(to_ref_as_created, "@")`:
```sql
available_ref = $4 AND available_at <= $3
AND ($5 = '' OR available_digest = $5)
```
`$5` = `requestedDigest`. A digest-pinned job does NOT clear a digest-less (webhook) suggestion: a rollback to `main@old` must not hide that a newer `main` was announced. The badge then stays until a plain tag job succeeds; that is accepted. Clearing sets `available_ref=''`, `available_digest=''`, `available_source=''`, `available_at=NULL`.
### C4. `internal/master/registry`
```go
// Client looks up tag digests with HEAD requests only.
type Client struct{ /* keychain authn.Keychain, userAgent string */ }
// New uses authn.DefaultKeychain ($DOCKER_CONFIG/config.json, else ~/.docker/config.json).
func New(userAgent string) *Client
// Digest returns the digest ref ("repo:tag", any digest suffix ignored) points
// at: the index digest for multi-arch images. HEAD /v2/<repo>/manifests/<tag> only.
func (c *Client) Digest(ctx context.Context, ref string) (string, error)
```
Implementation: `name.ParseReference(repo+":"+tag)`; `remote.Head(r, remote.WithContext(ctx), remote.WithAuthFromKeychain(c.keychain), remote.WithUserAgent(c.userAgent))`; return `desc.Digest.String()`. Never fall back to GET if `Docker-Content-Digest` is missing.
### C5. `internal/master/poller`
```go
type Store interface {
ListServices(ctx context.Context) ([]model.Service, error)
GetService(ctx context.Context, id int64) (model.Service, error)
SetAvailable(ctx context.Context, id int64, a model.Available) error
SetLastTagDigest(ctx context.Context, id int64, digest string) error
SetPollError(ctx context.Context, ids []int64, msg string) error
}
type Jobs interface {
Start(ctx context.Context, svc model.Service, req jobs.StartRequest) (model.Job, error)
}
type Registry interface { Digest(ctx context.Context, ref string) (string, error) }
type Publisher interface { Publish(t events.Topic) }
type Config struct {
Interval time.Duration // POLL_INTERVAL
FirstDelay time.Duration // 1m in production; small in tests
Workers int // 4
}
func New(st Store, j Jobs, reg Registry, pub Publisher, cfg Config, log *slog.Logger) *Poller
// Run polls until ctx is done. Blocking.
func (p *Poller) Run(ctx context.Context)
// Cycle runs one poll over all services (exported for tests and e2e).
func (p *Poller) Cycle(ctx context.Context)
// SkipReason says why svc is not polled: "follows no tag", "built from
// source", "no registry digest", or "" when it is polled.
func SkipReason(svc model.Service) string
```
Cycle semantics, per distinct `ref = jobs.FollowedRef(svc.ActualRef)` over non-removed, non-skipped services:
1. `D, err := reg.Digest(ctx, ref)` with a 30s timeout per call.
2. On error: `SetPollError(ids whose LastPollError != msg, msg)`, log `poll_failed` (ref, error).
3. On success: `SetPollError(ids with LastPollError != "", "")`. Then per service:
- Suggestion: `svc.ActualDigest == D` → nothing; `svc.AvailableRef == ref && svc.AvailableDigest == D` → no write; otherwise `SetAvailable({ref, D, poll, now})`.
- Auto job, independent of the suggestion: if `UpdateMode == auto && ActualDigest != D && D != LastTagDigest && DesiredRef != ref+"@"+D`, re-read the service (`GetService`), re-check the same condition and removed/mode on the fresh row, then `Start(TriggerPoll, Ref: ref)`. If a job row was created, `SetLastTagDigest(D)`. `ErrJobActive` and `NotUpdatableError` are logged at debug and ignored, leaving `last_tag_digest` unset so the next cycle retries.
Steps 2 and 3 use `context.WithoutCancel(ctx)`. At the end, publish `events.Services` if anything was written; log `poll_cycle` with refs, ok, failed, suggested, jobs, skipped, duration_ms.
### C6. `internal/master/jobs` and hub
```go
// Hub.Route gains the minimum protocol version for this job:
Route(svc model.Service, minProtocol int) (model.Route, error)
// Store gains:
GetImageVersion(ctx context.Context, id int64) (model.ImageVersion, error)
// TriggerVersion starts a manual job to exactly version versionID of the
// service's repo. model.ErrNotFound for an unknown service, an unknown
// version, or a version of another repo. Otherwise what Start returns.
func (s *Service) TriggerVersion(ctx context.Context, serviceID, versionID int64, requestedBy string) (model.Job, error)
```
- Ref = `svc.ImageRepo` + `":"+tag` if tag set + `"@"+digest` if digest set.
- Calls `Start` with `TriggerManual` and explicit ref ("follows no tag" refusal does not apply; removed, self, built, container still do).
- `Start` passes `protocol.MinPinnedRefVersion` to `Route` when the ref has a digest or no tag, otherwise `protocol.MinJobVersion`.
### C7. Protocol and agent
```go
const Version = 3
const MinJobVersion = 2 // unchanged: repo:tag jobs
const MinPinnedRefVersion = 3 // jobs whose Ref has a digest or no tag
```
`JobStart` doc: "Ref is `repo:tag`, `repo:tag@digest` or `repo@digest`. For `repo:tag` the agent resolves the tag's current digest; a ref with a digest is deployed as is."
`Resolver.Resolve`: digest present → return ref unchanged, no `DistributionInspect`; tag present → resolve as today; neither → error "has no tag to resolve" (unchanged). Compose writes the ref verbatim into the override, Swarm sets it as the image. Executors print `pinned <ref>` instead of `resolved <ref>` when nothing was resolved.
### C8. Master configuration
- `POLL_INTERVAL`: default `6h`, positive Go duration, error `"POLL_INTERVAL must be a positive duration such as 6h, got %q"`.
- `DOCKER_CONFIG`: not parsed by `config`; `go-containerregistry` reads it. `main` logs `registry_credentials` with the path and whether `config.json` exists, never its contents.
### C9. Web
- `POST /services/{id}/update` gains optional form field `version` (an `image_versions.id`): empty → `Trigger` as today; non-numeric → `400 invalid version`; otherwise `TriggerVersion`. Results: `303` to the job (also on `ErrJobActive`), `404` on `ErrNotFound`, `409` re-render "Cannot update: <reason>."
- `web.Store` gains `RecentVersions`; `web.Jobs` gains `TriggerVersion`.
- Service page: "Versions" table, up to 10 rows: ref (shortRef, full ref as title), source, first seen, "running" marker when digest equals `ActualDigest`, a POST form with `csrf_token` and `version`. Forms shown only when `AutoBlocked == ""`. Rows with empty tag note "the service will follow no tag afterwards". When the service is `auto`, header notes auto mode will move it forward at the next new version. A "Polling" row shows `SkipReason`, or `last_poll_error`, or "every <interval>".
- Overview: badge "follows no tag" when `FollowedRef(ActualRef) == ""`; badge "poll failed" (title = error) when `LastPollError != ""`. Template func `followsNoTag`.
## Tasks
### Task 1: Store and model (C1, C2, C3)
Migration, model fields, store methods, `upsertService` running versions, `FinishJob` rule. Integration tests: SetAvailable writes digest/source poll and records version, second call keeps first_seen_at; TestSnapshotRecordsRunningVersions (tag+digest, tagless digest, no digest skipped, re-snapshot idempotent); TestRecentVersions (order, limit 10, repo filter, digest-less row hidden only when a digest row for the same tag was first seen at or after it); GetImageVersion ErrNotFound; SetPollError sets/clears, upsert does not touch last_poll_error/available_*; FinishJob table extended (tag job clears poll suggestion; pick tag@old keeps suggestion tag@new; pick tag@new clears; webhook suggestion (digest '') cleared by matching tag; tagless pick keeps it). Update the e2e `memStore` in the same commit.
### Task 2: Protocol, agent and hub routing (C7, C6 Route)
Bump protocol, MinPinnedRefVersion, docs; Resolver pass-through; compose/swarm output line; `hub.Route(svc, minProtocol)`. Tests: TestResolve table (tag resolves; tag@d and @d pass through with zero inspect calls; bare errors); compose pinned ref verbatim in override; swarm spec image equals pinned ref; route rows v2 agent + min 3 → ErrAgentTooOld, v3 → ok, swarm fallback to v3 manager when reporting manager is v2.
### Task 3: Jobs (C6)
`Start` picks min version from the ref; `TriggerVersion`. Tests: version of other repo → ErrNotFound, no CreateJob; ref construction table; min version passed to fake hub (plain Trigger → 2, pinned → 3); explicit ref bypasses no-tag but not removed/self/built/container; existing tests green.
### Task 4: Registry client and poller (C4, C5)
Registry tests (local httptest + ggcr/pkg/registry, no network): random.Index digest equals index digest; recording handler shows only HEAD on /manifests/; basic-auth handler succeeds with temp DOCKER_CONFIG (t.Setenv) and fails without; 404 → error. Verify behaviour of DefaultKeychain when config.json has credsStore but no helper binary; document. Poller tests per Review Focus, plus Run test (first cycle after delay, no overlap, returns on cancel).
### Task 5: Config, wiring, web (C8, C9)
Config POLL_INTERVAL with tests; add to setAll list. `cmd/master/main.go`: registry.New, poller, `go p.Run(ctx)` with done channel, wait for it after HTTP shutdown and before st.Close; log registry_credentials. Web handlers/templates per C9 with tests (picker ≤10 rows with CSRF and version IDs; hidden for built/self/container/removed; POST with version → TriggerVersion, without → Trigger; 400/404/409; overview badges; Polling row).
### Task 6: End-to-end
`internal/e2e/versions_test.go`: TestPickPinnedVersion; TestPollStartsAutoJob (poller.Cycle with fake registry → one TriggerPoll job to repo:tag; second Cycle starts nothing). `currentHub.Route` new signature.
### Task 7: Documentation
README (POLL_INTERVAL, DOCKER_CONFIG, "Registry polling" section: polled/skipped, credentials only inline auths, Docker Hub note, version picker & rollback, auto caveat, agent protocol 3 for pinned/tagless picks). Spec: replace "Believed not to count against Docker Hub pull limits for HEAD requests — verify during implementation" with verified statement + source link (docs.docker.com/docker-hub/usage/pulls: "Using GET emulates a real pull and counts towards the limit. Using HEAD won't."; note the separate unquantified abuse rate limit). `deploy/master.stack.yml`: mount config dir read-only, set DOCKER_CONFIG. CLAUDE.md: architecture entries for registry and poller; gotchas (HEAD only; digest-less webhook row hidden once digest known; image_versions uses '' not NULL; picks by version ID and pinned/tagless need protocol 3; FinishJob rule includes digest (replace existing bullet); rollback to repo@digest makes the service follow no tag); phase 4 plan reference. Save this plan as `docs/superpowers/plans/2026-10-13-master-agent-phase-4-polling.md`.
## Final verification
```bash
gofmt -l . ; go vet ./...
go build ./cmd/master ./cmd/agent && rm -f master agent
go test ./...
go test -race ./internal/agent/... ./internal/master/... ./internal/e2e/
DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
```
@@ -1,104 +0,0 @@
# Master/Agent Phase 5 — Drift and Merge Implementation Plan
> This plan documents what was built for issue #10 (backend only) and where it deviates from the spec. Code in "Frozen contracts" is final; tasks give the contract and the test cases.
**Goal:** The master knows when a service runs something other than the version the last job deployed (drift), never corrects it by itself, and offers two ways out: restore the desired version or accept the running one. When a Compose service disappears from one agent and appears on another, the master suggests that it moved, and a merge moves the old row (history, mode, desired ref) to the new location. Merging can also be done by hand.
**Architecture:**
- Model: `Service.Drifted()` plus `RunsRef` and `RunningRef` decide drift and what accept writes. `Move` carries a suggestion.
- Store: migration 0005 adds `services.awaiting_inventory`; `FinishJob` sets it, every inventory upsert clears it. New `AcceptRunning`, `MoveCandidates`, `MergeSources`, `MergeService` (the only code that deletes a `services` row).
- Jobs: `Start` refuses poll and webhook jobs for a drifted service. `Restorable` and `RestoreDesired` deploy the recorded ref again as a manual job.
- Web: three POST routes (`/services/{id}/restore`, `/accept`, `/merge`) and new `pageData` fields. No template or CSS changes: issue #18 (the Ledger rewrite of the Services and Agents pages) is a separate UI change; the restore, accept and merge buttons arrive with the UI follow-up, issue #28.
**Tech Stack:** no new dependencies.
**Spec:** `docs/superpowers/specs/2026-10-12-master-agent-design.md`, sections "Drift" and "Service identity and moves". Phase 4 plan conventions still apply.
## Global Constraints
- `go test ./...` passes with no Docker and no Postgres. Postgres tests carry `//go:build integration`.
- No changes to `internal/master/web/templates/*`, `static/*`, `layout.html` or `groupServices`.
- Interfaces declared at the consumer (`jobs.Store`, `web.Store`, `web.Jobs`). Hand-written fakes and testify, table-driven where there is more than one case.
- GoDoc on every exported identifier. Conventional Commits, each ending with ` (#10)`.
- Every state-changing request is a POST with a CSRF token. Refusals render the service page with the reason (409), not a bare error.
- The invariants of earlier phases hold: one active job per service (`update_jobs_one_active`), the webhook and poller never create services, the updater's own services are never updated or merged.
## Deliberate deviations from the spec in this phase
1. **Drift compares what runs, not reference strings.** The spec says a service drifts when `actual_ref` differs from `desired_ref`. `desired_ref` is always `repo:tag@digest` while `actual_ref` is the raw image string, so a string comparison would flag every healthy service. `RunsRef` compares the repo exactly (no `docker.io` normalisation), then the digest when both sides know one, else the followed tag. The same digest under a different tag is therefore not drift.
2. **`awaiting_inventory`.** The agent's upsert arrives a debounce interval after `job.result`; until then `desired_ref` is newer than `actual_ref`. The flag, set by `FinishJob` and cleared by the next upsert of the row, keeps that window from counting as drift.
3. **Poll and webhook jobs are refused for a drifted service** (`ReasonDrifted`). Manual jobs are allowed, so a person can still update or roll back.
4. **Move suggestion is stricter than "exactly one appears".** Besides one removed and one live row on each side, the live row must be newer, have no desired ref and no jobs, and the removed row must have something worth moving (desired ref, auto mode or jobs). Otherwise every restarted stack would produce a suggestion.
5. **Manual merge is allowed for Compose and Swarm of the same kind**, names may differ, same agent allowed. Plain containers and the updater itself are refused.
6. **Accept does not clear `available_*`**, and the form's `actual` field (`Service.RunningRef()`) is only a guard; the value written comes from the row.
7. **The overview does not show moves or drift yet.** `MoveCandidates` is only read on the service page. `Service.Drifted()` is a model method, so the UI follow-up needs no further wiring.
8. **`CreateJob` maps the foreign key violation on `update_jobs_service_id_fkey` to `ErrNotFound`**, so a job for a service that a merge just deleted fails cleanly.
## Review Focus
| Failure case | Test that covers it |
|---|---|
| Drift by digest, not string: tag+digest vs tag and reported digest, bare image, tagless, port colon, other repo, case. | model `TestRunsRef`, `TestDrifted` |
| Accept writes `repo:tag@digest` from the running version. | model `TestRunningRef` |
| A fresh job result is not drift until the inventory arrives. | store `TestFinishJob_SetsAwaitingInventoryUntilTheNextUpsert`; e2e `TestNoDriftAfterASuccessfulJob` |
| Accept is guarded by the running version, refuses removed/never-updated, leaves `last_tag_digest`, `available_*`, mode alone. | store `TestAcceptRunning_*` |
| Poll and webhook start no job for a drifted service; manual does. | jobs `TestStart_RefusesPollAndWebhookWhileDrifted`; e2e `TestDriftedAutoServiceIsNotUpdatedByPollOrWebhook` |
| Restore deploys the recorded ref and remembers the drifted digest of the same tag. | jobs `TestRestoreDesired_*`; e2e `TestRestoreDeploysThePinnedRefOnTheAgent` |
| Move suggestion only for a unique, new, history-less target and a removed source with history. | store `TestMoveCandidates` |
| Merge keeps id, mode, desired ref, first seen; takes location and inventory; moves jobs; deletes the duplicate. | store `TestMergeService_MovesHistoryModeAndDesiredToTheNewLocation` |
| Merge refusals change nothing (active job, kinds, container, self, live old, removed new, revoked agent, unknown). | store `TestMergeService_Refusals` |
| Inventory after a merge updates the survivor; the old agent returning creates a new row. | store `TestMergeService_InventoryAfterTheMerge` |
| A job for a deleted service is `ErrNotFound`. | store `TestCreateJob_UnknownServiceIsNotFound` |
| Handlers: redirects, 400/404/409, signals published, survivor id in the redirect. | web `TestRestoreDesired`, `TestAcceptRunning_*`, `TestMergeService_*`, `TestServiceDetail_DriftAndMoveData` |
## Frozen contracts
### C1. Migration `0005_drift_merge.sql`
`services.awaiting_inventory boolean NOT NULL DEFAULT false`; index `services_by_compose_name` on `(compose_project, compose_service) WHERE kind = 'compose'`.
### C2. `internal/master/model`
`Service.AwaitingInventory`; `Service.Drifted()`; `RunsRef(desired, actualRef, actualDigest)`; `RunningRef(actualRef, actualDigest)`; `Move{FromID, ToID, FromAgent, ToAgent}` (FromID is the removed row). `ReasonDrifted`, `ReasonNotDrifted`; `MergeRefusedError{Reason}` with the `ReasonMerge*` constants (`ReasonSelf` is reused).
### C3. Store
- `upsertService` sets `awaiting_inventory = false`; `FinishJob` success sets it to true.
- `AcceptRunning(ctx, id, actualRef, actualDigest, desired)`: updates only a live, previously updated row that still reports that actual ref and digest. `ErrNotFound` for unknown or removed, `ErrConflict` otherwise.
- `MoveCandidates(ctx) ([]model.Move, error)`: Compose only, not self, other agent, unique on both sides before the extra filters (live row newer, no desired ref, no jobs; removed row has a desired ref, auto mode or jobs). Agent names by `LEFT JOIN`, so a revoked agent keeps its name.
- `MergeSources(ctx, toID)`: removed rows of the same kind (not container, not self), same name first, then `removed_at DESC`, at most 50.
- `MergeService(ctx, fromID, toID)`: one transaction. Lock order: the live row's agent (`FOR SHARE`, `ErrRevoked`), then both rows `ORDER BY id FOR UPDATE`. Validates (`MergeRefusedError`), refuses an active job on either row (`ErrJobActive`), moves the duplicate's jobs, reads its location and inventory, **deletes it**, then updates the old row (keeps id, kind, `first_seen_at`, `update_mode`, `desired_ref`; takes agent, location, inventory columns, available suggestion and poll error; `last_tag_digest` only if the old row has none; `removed_at = NULL`, `awaiting_inventory = false`). The delete comes first because `location_key` is unique and not deferrable.
- `CreateJob`: foreign key violation `update_jobs_service_id_fkey` becomes `ErrNotFound`.
### C4. Jobs
`Start` returns `NotUpdatableError{ReasonDrifted}` for a non-manual trigger on a drifted service. `Restorable(svc)`: drifted, then `updatable(svc, true)`, and the desired ref's repo is not the agent image repo. `RestoreDesired(ctx, id, by)`: `Start` with `Ref: svc.DesiredRef`, manual. After a created job, `SetLastTagDigest` to the drifted digest when the actual tag equals the desired tag, the digest is well formed, differs from the desired digest and from `last_tag_digest`, and `last_tag_digest` is empty or equals the desired digest (otherwise it holds back a newer digest and stays).
### C5. Web
Routes `POST /services/{id}/restore`, `/accept` (form `actual` required, the `Service.RunningRef()` the user saw, compared with the row's; optional `return=overview`), `/merge` (form `from` = removed row, `{id}` = live row; redirects to the surviving id). `pageData` gains `RestoreBlocked`, `MoveFrom`, `MoveTo`, `MergeSources`. Log events `drift_accepted` and `service_merged`.
### C6. Concurrency
An upsert for the live row's location waits on its lock, then updates the survivor. A snapshot after the merge lists the survivor's key, so it is not marked removed. A removal racing a merge can be lost; the next snapshot fixes it. `CreateJob` on the deleted row gets `ErrNotFound`; the poller and webhook re-read and log it.
## Tasks
### Task 1: Store and model (C1, C2, C3)
Migration, `imageref.IsDigest` (replaces the local pattern in the store), model types and functions, store methods. Tests: model tables; integration tests listed in Review Focus.
### Task 2: Jobs (C4)
`Store.SetLastTagDigest` on the consumer interface, `Start` refusal, `Restorable`, `RestoreDesired`. Tests: refusal table, manual job to the desired ref with protocol 3, active job, old agent, last-tag-digest table.
### Task 3: Web (C5)
Handlers, interfaces, `pageData`. Tests with fakes.
### Task 4: End-to-end
`memStore.FinishJob` sets `AwaitingInventory`. `internal/e2e/drift_test.go`.
### Task 5: Documentation
README section "Drift and moved services", webhook note, CLAUDE.md design intent and gotchas, this plan.
## Final verification
```bash
gofmt -l . ; go vet ./...
go build ./cmd/master ./cmd/agent && rm -f master agent
go test ./...
go test -race ./internal/agent/... ./internal/master/... ./internal/e2e/
DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
```
@@ -247,12 +247,8 @@ them. An agent recreating itself would kill the job it is running.
- Mechanism: `HEAD /v2/<repo>/manifests/<tag>`; the returned digest is compared with - Mechanism: `HEAD /v2/<repo>/manifests/<tag>`; the returned digest is compared with
`actual_digest`. Use `go-containerregistry` — it handles Docker Hub's token auth and `actual_digest`. Use `go-containerregistry` — it handles Docker Hub's token auth and
multi-arch indexes. multi-arch indexes.
- Interval `POLL_INTERVAL`, default 6h. Docker Hub documents that HEAD requests do not - Interval `POLL_INTERVAL`, default 6h. Believed not to count against Docker Hub pull
count against the pull limit ("Using GET emulates a real pull and counts towards the limits for HEAD requests — verify during implementation.
limit. Using HEAD won't."; verified against
https://docs.docker.com/docker-hub/usage/pulls/). The same page mentions a separate,
unquantified abuse rate limit for many pulls from one IP address, so the poller stays
HEAD-only and never downloads a manifest.
- Credentials: a Docker `config.json` (as written by `docker login`) mounted into the - Credentials: a Docker `config.json` (as written by `docker login`) mounted into the
master. No credential UI, no encryption in Postgres. master. No credential UI, no encryption in Postgres.
+2 -7
View File
@@ -7,7 +7,6 @@ require (
github.com/coreos/go-oidc/v3 v3.21.0 github.com/coreos/go-oidc/v3 v3.21.0
github.com/distribution/reference v0.6.0 github.com/distribution/reference v0.6.0
github.com/docker/docker v28.5.2+incompatible github.com/docker/docker v28.5.2+incompatible
github.com/google/go-containerregistry v0.22.1
github.com/jackc/pgx/v5 v5.10.0 github.com/jackc/pgx/v5 v5.10.0
github.com/opencontainers/go-digest v1.0.0 github.com/opencontainers/go-digest v1.0.0
github.com/stretchr/testify v1.11.1 github.com/stretchr/testify v1.11.1
@@ -22,8 +21,6 @@ require (
github.com/containerd/errdefs/pkg v0.3.0 // indirect github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/containerd/log v0.1.0 // indirect github.com/containerd/log v0.1.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect github.com/davecgh/go-spew v1.1.1 // indirect
github.com/docker/cli v29.7.2+incompatible // indirect
github.com/docker/docker-credential-helpers v0.9.3 // indirect
github.com/docker/go-connections v0.7.0 // indirect github.com/docker/go-connections v0.7.0 // indirect
github.com/docker/go-units v0.5.0 // indirect github.com/docker/go-units v0.5.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect github.com/felixge/httpsnoop v1.0.4 // indirect
@@ -33,7 +30,6 @@ require (
github.com/jackc/pgpassfile v1.0.0 // indirect github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/klauspost/compress v1.19.2 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/sys/atomicwriter v0.1.0 // indirect github.com/moby/sys/atomicwriter v0.1.0 // indirect
github.com/moby/term v0.5.2 // indirect github.com/moby/term v0.5.2 // indirect
@@ -41,15 +37,14 @@ require (
github.com/opencontainers/image-spec v1.1.1 // indirect github.com/opencontainers/image-spec v1.1.1 // indirect
github.com/pkg/errors v0.9.1 // indirect github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/sirupsen/logrus v1.9.4 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0 // indirect go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0 // indirect
go.opentelemetry.io/otel v1.43.0 // indirect go.opentelemetry.io/otel v1.43.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0 // indirect go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0 // indirect
go.opentelemetry.io/otel/metric v1.43.0 // indirect go.opentelemetry.io/otel/metric v1.43.0 // indirect
go.opentelemetry.io/otel/trace v1.43.0 // indirect go.opentelemetry.io/otel/trace v1.43.0 // indirect
golang.org/x/sync v0.22.0 // indirect golang.org/x/sync v0.20.0 // indirect
golang.org/x/sys v0.47.0 // indirect golang.org/x/sys v0.42.0 // indirect
golang.org/x/text v0.35.0 // indirect golang.org/x/text v0.35.0 // indirect
golang.org/x/time v0.15.0 // indirect golang.org/x/time v0.15.0 // indirect
gotest.tools/v3 v3.5.2 // indirect gotest.tools/v3 v3.5.2 // indirect
+6 -18
View File
@@ -21,12 +21,8 @@ github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk= github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E= github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/docker/cli v29.7.2+incompatible h1:dlkwallR8XqfeVnA2ELEhdwvb4lsSwuB4IgsG8Q9cLY=
github.com/docker/cli v29.7.2+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/docker v28.5.2+incompatible h1:DBX0Y0zAjZbSrm1uzOkdr1onVghKaftjlSWt4AFexzM= github.com/docker/docker v28.5.2+incompatible h1:DBX0Y0zAjZbSrm1uzOkdr1onVghKaftjlSWt4AFexzM=
github.com/docker/docker v28.5.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= github.com/docker/docker v28.5.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker-credential-helpers v0.9.3 h1:gAm/VtF9wgqJMoxzT3Gj5p4AqIjCBS4wrsOh9yRqcz8=
github.com/docker/docker-credential-helpers v0.9.3/go.mod h1:x+4Gbw9aGmChi3qTLZj8Dfn0TD20M/fuWy0E5+WDeCo=
github.com/docker/go-connections v0.7.0 h1:6SsRfJddP22WMrCkj19x9WKjEDTB+ahsdiGYf0mN39c= github.com/docker/go-connections v0.7.0 h1:6SsRfJddP22WMrCkj19x9WKjEDTB+ahsdiGYf0mN39c=
github.com/docker/go-connections v0.7.0/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q= github.com/docker/go-connections v0.7.0/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
@@ -42,8 +38,6 @@ github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/go-containerregistry v0.22.1 h1:RZuuSYhTvlDvtsK+NkutoCZ//C0X2ebLK8X8l3ULs84=
github.com/google/go-containerregistry v0.22.1/go.mod h1:bJR35SK8XgisYmhg/FMQ/5RK0S/XrOAqLBV5/LR2XE0=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 h1:HWRh5R2+9EifMyIHV7ZV+MIZqgz+PMpZ14Jynv3O2Zs= github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 h1:HWRh5R2+9EifMyIHV7ZV+MIZqgz+PMpZ14Jynv3O2Zs=
@@ -56,8 +50,6 @@ github.com/jackc/pgx/v5 v5.10.0 h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0=
github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4= github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo= github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4= github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/klauspost/compress v1.19.2 h1:hMRETovs/pu/dVWN7zIT1PGG8t509MwT6bO7XSi26R8=
github.com/klauspost/compress v1.19.2/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
@@ -82,8 +74,8 @@ github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZb
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ= github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc= github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/sirupsen/logrus v1.9.4 h1:TsZE7l11zFCLZnZ+teH4Umoq5BhEIfIzfRDZ1Uzql2w= github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.4/go.mod h1:ftWc9WdOfJ0a92nsE2jF5u5ZwH8Bv2zdeOC42RjbV2g= github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
@@ -109,22 +101,18 @@ go.opentelemetry.io/otel/trace v1.43.0 h1:BkNrHpup+4k4w+ZZ86CZoHHEkohws8AY+WTX09
go.opentelemetry.io/otel/trace v1.43.0/go.mod h1:/QJhyVBUUswCphDVxq+8mld+AvhXZLhe+8WVFxiFff0= go.opentelemetry.io/otel/trace v1.43.0/go.mod h1:/QJhyVBUUswCphDVxq+8mld+AvhXZLhe+8WVFxiFff0=
go.opentelemetry.io/proto/otlp v1.10.0 h1:IQRWgT5srOCYfiWnpqUYz9CVmbO8bFmKcwYxpuCSL2g= go.opentelemetry.io/proto/otlp v1.10.0 h1:IQRWgT5srOCYfiWnpqUYz9CVmbO8bFmKcwYxpuCSL2g=
go.opentelemetry.io/proto/otlp v1.10.0/go.mod h1:/CV4QoCR/S9yaPj8utp3lvQPoqMtxXdzn7ozvvozVqk= go.opentelemetry.io/proto/otlp v1.10.0/go.mod h1:/CV4QoCR/S9yaPj8utp3lvQPoqMtxXdzn7ozvvozVqk=
golang.org/x/mod v0.39.0 h1:UF5zwQdCRRUpHfyPwr7d4UrGiVeldIsogtzWVnczL74=
golang.org/x/mod v0.39.0/go.mod h1:bvIbwjQ0HUFFf5AKukeeYQG4ZBUG9yxQbR9aEweIwYY=
golang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0= golang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0=
golang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw= golang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs= golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q= golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek= golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/text v0.35.0 h1:JOVx6vVDFokkpaq1AEptVzLTpDe9KGpj5tR4/X+ybL8= golang.org/x/text v0.35.0 h1:JOVx6vVDFokkpaq1AEptVzLTpDe9KGpj5tR4/X+ybL8=
golang.org/x/text v0.35.0/go.mod h1:khi/HExzZJ2pGnjenulevKNX1W67CUy0AsXcNubPGCA= golang.org/x/text v0.35.0/go.mod h1:khi/HExzZJ2pGnjenulevKNX1W67CUy0AsXcNubPGCA=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U= golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno= golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/tools v0.49.0 h1:3NI7VXzL9+1WZD52Dx2ttoPwD5DWrFGpl9mFZDlmisI=
golang.org/x/tools v0.49.0/go.mod h1:SJNXV9DBKT0UbdttsQjbfJlAE/q+y36++zo3uL3N0Oo=
google.golang.org/genproto/googleapis/api v0.0.0-20260401024825-9d38bb4040a9 h1:VPWxll4HlMw1Vs/qXtN7BvhZqsS9cdAittCNvVENElA= google.golang.org/genproto/googleapis/api v0.0.0-20260401024825-9d38bb4040a9 h1:VPWxll4HlMw1Vs/qXtN7BvhZqsS9cdAittCNvVENElA=
google.golang.org/genproto/googleapis/api v0.0.0-20260401024825-9d38bb4040a9/go.mod h1:7QBABkRtR8z+TEnmXTqIqwJLlzrZKVfAUm7tY3yGv0M= google.golang.org/genproto/googleapis/api v0.0.0-20260401024825-9d38bb4040a9/go.mod h1:7QBABkRtR8z+TEnmXTqIqwJLlzrZKVfAUm7tY3yGv0M=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260401024825-9d38bb4040a9 h1:m8qni9SQFH0tJc1X0vmnpw/0t+AImlSvp30sEupozUg= google.golang.org/genproto/googleapis/rpc v0.0.0-20260401024825-9d38bb4040a9 h1:m8qni9SQFH0tJc1X0vmnpw/0t+AImlSvp30sEupozUg=
+6 -12
View File
@@ -91,10 +91,10 @@ func NewComposeExecutor(cli ComposeDocker, resolve *Resolver, selfID string, run
return &ComposeExecutor{cli: cli, resolve: resolve, selfID: selfID, run: run, build: NewBuildChecker(cli, selfID, runOut)} return &ComposeExecutor{cli: cli, resolve: resolve, selfID: selfID, run: run, build: NewBuildChecker(cli, selfID, runOut)}
} }
// Execute pins job.Ref to its current digest (a ref that has one is used as // Execute pins job.Ref to its current digest, writes it to the stack's
// is), writes it to the stack's override file and recreates the service. // override file and recreates the service. Working directory and config
// Working directory and config files come from the service's container // files come from the service's container labels on this machine, never from
// labels on this machine, never from the job. It returns the pinned ref. // the job. It returns the pinned ref.
func (e *ComposeExecutor) Execute(ctx context.Context, job protocol.JobStart, out io.Writer) (string, error) { func (e *ComposeExecutor) Execute(ctx context.Context, job protocol.JobStart, out io.Writer) (string, error) {
project, service := job.ComposeProject, job.ComposeService project, service := job.ComposeProject, job.ComposeService
cts, err := e.cli.ContainerList(ctx, container.ListOptions{ cts, err := e.cli.ContainerList(ctx, container.ListOptions{
@@ -150,17 +150,11 @@ func (e *ComposeExecutor) Execute(ctx context.Context, job protocol.JobStart, ou
if err != nil { if err != nil {
return "", err return "", err
} }
fmt.Fprintln(out, resolveNote(job.Ref, pinned)) fmt.Fprintf(out, "resolved %s\n", pinned)
// Giving the override the stack's owner is best effort: a failure is only if _, err := WriteOverride(dir, service, pinned); err != nil {
// a warning, the update itself is fine.
_, ownerErr, err := WriteOverride(dir, service, pinned, files[0])
if err != nil {
return "", err return "", err
} }
if ownerErr != nil {
fmt.Fprintf(out, "warning: could not give the override the stack's owner: %v\n", ownerErr)
}
base := composeBase(project, workingDir, files, envs) base := composeBase(project, workingDir, files, envs)
for _, step := range [][]string{{"pull", service}, {"up", "-d", service}} { for _, step := range [][]string{{"pull", service}, {"up", "-d", service}} {
-32
View File
@@ -69,7 +69,6 @@ func TestComposeExecutor(t *testing.T) {
root, _ := filepath.EvalSymlinks(t.TempDir()) root, _ := filepath.EvalSymlinks(t.TempDir())
stack := filepath.Join(root, "slusk") stack := filepath.Join(root, "slusk")
require.NoError(t, os.MkdirAll(stack, 0o755)) require.NoError(t, os.MkdirAll(stack, 0o755))
composeFile := stackFile(t, stack)
job := protocol.JobStart{Kind: protocol.KindCompose, ComposeProject: "slusk", ComposeService: "web", Ref: "gitea.shcizo.se/shcizo/slusk:1.4"} job := protocol.JobStart{Kind: protocol.KindCompose, ComposeProject: "slusk", ComposeService: "web", Ref: "gitea.shcizo.se/shcizo/slusk:1.4"}
pinned := job.Ref + "@" + d pinned := job.Ref + "@" + d
@@ -108,37 +107,6 @@ func TestComposeExecutor(t *testing.T) {
_, doc := readOverride(t, stack) _, doc := readOverride(t, stack)
assert.Equal(t, pinned, serviceImage(doc, "web")) assert.Equal(t, pinned, serviceImage(doc, "web"))
assert.Contains(t, out.String(), "resolved "+pinned) assert.Contains(t, out.String(), "resolved "+pinned)
assert.NotContains(t, out.String(), "warning")
})
t.Run("an unreadable owner is a warning and the job still succeeds", func(t *testing.T) {
t.Cleanup(cleanOverride)
require.NoError(t, os.Rename(composeFile, composeFile+".bak"))
t.Cleanup(func() { os.Rename(composeFile+".bak", composeFile) })
var out strings.Builder
got, err := setup(fakeCompose{rw, []container.Summary{ct}}, &fakeDist{digest: d}, &recorder{}).Execute(context.Background(), job, &out)
require.NoError(t, err)
assert.Equal(t, pinned, got)
assert.Contains(t, out.String(), "warning: could not give the override the stack's owner: read owner: ")
_, doc := readOverride(t, stack)
assert.Equal(t, pinned, serviceImage(doc, "web"))
})
t.Run("a pinned ref is written verbatim and not resolved", func(t *testing.T) {
t.Cleanup(cleanOverride)
for _, ref := range []string{"gitea.shcizo.se/shcizo/slusk:1.3@" + d, "gitea.shcizo.se/shcizo/slusk@" + d} {
rec := &recorder{}
dist := &fakeDist{digest: "sha256:other"}
var out strings.Builder
pinnedJob := job
pinnedJob.Ref = ref
got, err := setup(fakeCompose{rw, []container.Summary{ct}}, dist, rec).Execute(context.Background(), pinnedJob, &out)
require.NoError(t, err)
assert.Equal(t, ref, got)
assert.Empty(t, dist.refs, "no registry lookup")
_, doc := readOverride(t, stack)
assert.Equal(t, ref, serviceImage(doc, "web"))
assert.Contains(t, out.String(), "pinned "+ref)
assert.NotContains(t, out.String(), "resolved")
}
}) })
t.Run("built from source is refused before anything is written", func(t *testing.T) { t.Run("built from source is refused before anything is written", func(t *testing.T) {
t.Cleanup(cleanOverride) t.Cleanup(cleanOverride)
+13 -75
View File
@@ -8,7 +8,6 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
"syscall"
"github.com/docker/docker/api/types/container" "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/mount" "github.com/docker/docker/api/types/mount"
@@ -35,68 +34,15 @@ var overrideNames = []string{
// keeping every other entry in the file, and returns the file's path. It // keeping every other entry in the file, and returns the file's path. It
// refuses when the file exists without the header, or when another // refuses when the file exists without the header, or when another
// auto-loaded override file exists in dir. The write is atomic. // auto-loaded override file exists in dir. The write is atomic.
// func WriteOverride(dir, service, ref string) (string, error) {
// composeFile is the stack's first compose file. The new file gets the owner
// of the existing override, or of composeFile when there is none or the
// existing one is owned by root (see pickOwner). The agent runs as root and the
// rename creates a new file, so without this the stack's owner would lose
// control of the file. ownerErr reports that the owner could not be read or
// set; the override is still written. err means it was not written.
func WriteOverride(dir, service, ref, composeFile string) (path string, ownerErr error, err error) {
overridePath := filepath.Join(dir, OverrideName)
return writeOverride(dir, service, ref, func() (fileOwner, error) {
return resolveOwner(overridePath, composeFile)
})
}
// fileOwner is the uid and gid of a file.
type fileOwner struct{ uid, gid int }
// readOwner returns the owner of the file at path.
func readOwner(path string) (fileOwner, error) {
fi, err := os.Stat(path)
if err != nil {
return fileOwner{}, fmt.Errorf("read owner: %w", err)
}
st, ok := fi.Sys().(*syscall.Stat_t)
if !ok {
return fileOwner{}, fmt.Errorf("read owner of %s: not available on this platform", path)
}
return fileOwner{int(st.Uid), int(st.Gid)}, nil
}
// pickOwner chooses the owner of a new override: the existing override's
// (nil when there is none), unless that is root, then the compose file's.
// Overrides written before the agent set owners are root-owned and would
// otherwise stay so. compose is only called when needed.
func pickOwner(override *fileOwner, compose func() (fileOwner, error)) (fileOwner, error) {
if override != nil && override.uid != 0 {
return *override, nil
}
return compose()
}
// resolveOwner applies pickOwner to the files on disk.
func resolveOwner(overridePath, composeFile string) (fileOwner, error) {
var existing *fileOwner
if o, err := readOwner(overridePath); err == nil {
existing = &o
} else if !errors.Is(err, os.ErrNotExist) {
return fileOwner{}, err
}
return pickOwner(existing, func() (fileOwner, error) { return readOwner(composeFile) })
}
// writeOverride is WriteOverride with the owner lookup injected, for tests.
func writeOverride(dir, service, ref string, owner func() (fileOwner, error)) (string, error, error) {
for _, name := range overrideNames { for _, name := range overrideNames {
if name == OverrideName { if name == OverrideName {
continue continue
} }
if _, err := os.Lstat(filepath.Join(dir, name)); err == nil { if _, err := os.Lstat(filepath.Join(dir, name)); err == nil {
return "", nil, fmt.Errorf("refused: %s exists; the agent only manages %s and never edits operator files", name, OverrideName) return "", fmt.Errorf("refused: %s exists; the agent only manages %s and never edits operator files", name, OverrideName)
} else if !errors.Is(err, os.ErrNotExist) { } else if !errors.Is(err, os.ErrNotExist) {
return "", nil, fmt.Errorf("check %s: %w", name, err) return "", fmt.Errorf("check %s: %w", name, err)
} }
} }
path := filepath.Join(dir, OverrideName) path := filepath.Join(dir, OverrideName)
@@ -105,12 +51,12 @@ func writeOverride(dir, service, ref string, owner func() (fileOwner, error)) (s
switch { switch {
case errors.Is(err, os.ErrNotExist): case errors.Is(err, os.ErrNotExist):
case err != nil: case err != nil:
return "", nil, fmt.Errorf("read override: %w", err) return "", fmt.Errorf("read override: %w", err)
case !bytes.HasPrefix(raw, []byte(overrideHeader)): case !bytes.HasPrefix(raw, []byte(overrideHeader)):
return "", nil, fmt.Errorf("refused: %s exists and was not written by package-updater", path) return "", fmt.Errorf("refused: %s exists and was not written by package-updater", path)
default: default:
if err := yaml.Unmarshal(raw, &doc); err != nil { if err := yaml.Unmarshal(raw, &doc); err != nil {
return "", nil, fmt.Errorf("parse override: %w", err) return "", fmt.Errorf("parse override: %w", err)
} }
if doc == nil { if doc == nil {
doc = map[string]any{} doc = map[string]any{}
@@ -129,38 +75,30 @@ func writeOverride(dir, service, ref string, owner func() (fileOwner, error)) (s
doc["services"] = services doc["services"] = services
body, err := yaml.Marshal(doc) body, err := yaml.Marshal(doc)
if err != nil { if err != nil {
return "", nil, fmt.Errorf("encode override: %w", err) return "", fmt.Errorf("encode override: %w", err)
} }
// Temp file in the same directory + rename: a crash never leaves a // Temp file in the same directory + rename: a crash never leaves a
// half-written override that compose would then load. // half-written override that compose would then load.
tmp, err := os.CreateTemp(dir, "."+OverrideName+".*") tmp, err := os.CreateTemp(dir, "."+OverrideName+".*")
if err != nil { if err != nil {
return "", nil, fmt.Errorf("write override: %w", err) return "", fmt.Errorf("write override: %w", err)
} }
defer os.Remove(tmp.Name()) // no-op after a successful rename defer os.Remove(tmp.Name()) // no-op after a successful rename
if _, err := tmp.WriteString(overrideHeader + string(body)); err != nil { if _, err := tmp.WriteString(overrideHeader + string(body)); err != nil {
tmp.Close() tmp.Close()
return "", nil, fmt.Errorf("write override: %w", err) return "", fmt.Errorf("write override: %w", err)
} }
if err := tmp.Chmod(0o644); err != nil { if err := tmp.Chmod(0o644); err != nil {
tmp.Close() tmp.Close()
return "", nil, fmt.Errorf("write override: %w", err) return "", fmt.Errorf("write override: %w", err)
}
// Chown the open file, not the path after the rename: a path could have
// been swapped for a symlink by someone with write access to the stack.
var ownerErr error
if o, err := owner(); err != nil {
ownerErr = err
} else if err := tmp.Chown(o.uid, o.gid); err != nil {
ownerErr = fmt.Errorf("set owner %d:%d: %w", o.uid, o.gid, err)
} }
if err := tmp.Close(); err != nil { if err := tmp.Close(); err != nil {
return "", nil, fmt.Errorf("write override: %w", err) return "", fmt.Errorf("write override: %w", err)
} }
if err := os.Rename(tmp.Name(), path); err != nil { if err := os.Rename(tmp.Name(), path); err != nil {
return "", nil, fmt.Errorf("write override: %w", err) return "", fmt.Errorf("write override: %w", err)
} }
return path, ownerErr, nil return path, nil
} }
// SelfInspector is the subset of the Docker SDK MountRoots needs. // SelfInspector is the subset of the Docker SDK MountRoots needs.
+8 -94
View File
@@ -29,96 +29,11 @@ func serviceImage(doc map[string]any, name string) any {
return entry["image"] return entry["image"]
} }
// stackFile creates dir/compose.yaml and returns its path.
func stackFile(t *testing.T, dir string) string {
t.Helper()
p := filepath.Join(dir, "compose.yaml")
require.NoError(t, os.WriteFile(p, []byte("services: {}\n"), 0o644))
return p
}
func TestPickOwner(t *testing.T) {
compose := fileOwner{1000, 1000}
fromCompose := func() (fileOwner, error) { return compose, nil }
failing := func() (fileOwner, error) { return fileOwner{}, errors.New("boom") }
tests := []struct {
name string
override *fileOwner
compose func() (fileOwner, error)
want fileOwner
wantErr bool
}{
{"no override uses the compose file", nil, fromCompose, compose, false},
{"override owned by a user wins", &fileOwner{1001, 1002}, fromCompose, fileOwner{1001, 1002}, false},
{"root-owned override uses the compose file", &fileOwner{0, 0}, fromCompose, compose, false},
{"compose file is not read when the override decides", &fileOwner{1001, 1002}, failing, fileOwner{1001, 1002}, false},
{"unreadable compose file is an error", nil, failing, fileOwner{}, true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := pickOwner(tt.override, tt.compose)
assert.Equal(t, tt.wantErr, err != nil)
assert.Equal(t, tt.want, got)
})
}
}
func TestResolveOwner(t *testing.T) {
me := fileOwner{os.Getuid(), os.Getgid()}
t.Run("no override uses the compose file", func(t *testing.T) {
dir := t.TempDir()
got, err := resolveOwner(filepath.Join(dir, OverrideName), stackFile(t, dir))
require.NoError(t, err)
assert.Equal(t, me, got)
})
t.Run("existing override is used without the compose file", func(t *testing.T) {
dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, OverrideName), nil, 0o644))
got, err := resolveOwner(filepath.Join(dir, OverrideName), filepath.Join(dir, "missing.yaml"))
require.NoError(t, err)
assert.Equal(t, me, got)
})
t.Run("neither file is an error", func(t *testing.T) {
dir := t.TempDir()
_, err := resolveOwner(filepath.Join(dir, OverrideName), filepath.Join(dir, "missing.yaml"))
assert.Error(t, err)
})
}
func TestWriteOverride_Owner(t *testing.T) {
t.Run("success sets no ownerErr", func(t *testing.T) {
dir := t.TempDir()
_, ownerErr, err := WriteOverride(dir, "web", "a:1@sha256:aa", stackFile(t, dir))
require.NoError(t, err)
assert.NoError(t, ownerErr)
})
t.Run("unreadable owner is reported and the file is still written", func(t *testing.T) {
dir := t.TempDir()
_, ownerErr, err := WriteOverride(dir, "web", "a:1@sha256:aa", filepath.Join(dir, "missing.yaml"))
require.NoError(t, err)
assert.ErrorContains(t, ownerErr, "read owner")
_, doc := readOverride(t, dir)
assert.Equal(t, "a:1@sha256:aa", serviceImage(doc, "web"))
})
t.Run("chown failure is reported and the file is still written", func(t *testing.T) {
if os.Getuid() == 0 {
t.Skip("root can chown to anyone")
}
dir := t.TempDir()
_, ownerErr, err := writeOverride(dir, "web", "a:1@sha256:aa", func() (fileOwner, error) { return fileOwner{0, 0}, nil })
require.NoError(t, err)
assert.ErrorContains(t, ownerErr, "set owner 0:0")
_, doc := readOverride(t, dir)
assert.Equal(t, "a:1@sha256:aa", serviceImage(doc, "web"))
})
}
func TestWriteOverride(t *testing.T) { func TestWriteOverride(t *testing.T) {
t.Run("new file", func(t *testing.T) { t.Run("new file", func(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
path, ownerErr, err := WriteOverride(dir, "web", "a:1@sha256:aa", stackFile(t, dir)) path, err := WriteOverride(dir, "web", "a:1@sha256:aa")
require.NoError(t, err) require.NoError(t, err)
assert.NoError(t, ownerErr)
assert.Equal(t, filepath.Join(dir, OverrideName), path) assert.Equal(t, filepath.Join(dir, OverrideName), path)
raw, doc := readOverride(t, dir) raw, doc := readOverride(t, dir)
assert.True(t, len(raw) > len(overrideHeader) && raw[:len(overrideHeader)] == overrideHeader) assert.True(t, len(raw) > len(overrideHeader) && raw[:len(overrideHeader)] == overrideHeader)
@@ -127,7 +42,7 @@ func TestWriteOverride(t *testing.T) {
t.Run("keeps other services", func(t *testing.T) { t.Run("keeps other services", func(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, OverrideName), []byte(overrideHeader+"services:\n db:\n image: db:1\n"), 0o644)) require.NoError(t, os.WriteFile(filepath.Join(dir, OverrideName), []byte(overrideHeader+"services:\n db:\n image: db:1\n"), 0o644))
_, _, err := WriteOverride(dir, "web", "a:1@sha256:aa", stackFile(t, dir)) _, err := WriteOverride(dir, "web", "a:1@sha256:aa")
require.NoError(t, err) require.NoError(t, err)
_, doc := readOverride(t, dir) _, doc := readOverride(t, dir)
assert.Equal(t, "db:1", serviceImage(doc, "db")) assert.Equal(t, "db:1", serviceImage(doc, "db"))
@@ -136,7 +51,7 @@ func TestWriteOverride(t *testing.T) {
t.Run("replaces own entry", func(t *testing.T) { t.Run("replaces own entry", func(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, OverrideName), []byte(overrideHeader+"services:\n web:\n image: old\n"), 0o644)) require.NoError(t, os.WriteFile(filepath.Join(dir, OverrideName), []byte(overrideHeader+"services:\n web:\n image: old\n"), 0o644))
_, _, err := WriteOverride(dir, "web", "a:2@sha256:bb", stackFile(t, dir)) _, err := WriteOverride(dir, "web", "a:2@sha256:bb")
require.NoError(t, err) require.NoError(t, err)
_, doc := readOverride(t, dir) _, doc := readOverride(t, dir)
assert.Equal(t, "a:2@sha256:bb", serviceImage(doc, "web")) assert.Equal(t, "a:2@sha256:bb", serviceImage(doc, "web"))
@@ -145,7 +60,7 @@ func TestWriteOverride(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
const body = "services:\n web:\n environment: [A=1]\n" const body = "services:\n web:\n environment: [A=1]\n"
require.NoError(t, os.WriteFile(filepath.Join(dir, OverrideName), []byte(body), 0o644)) require.NoError(t, os.WriteFile(filepath.Join(dir, OverrideName), []byte(body), 0o644))
_, _, err := WriteOverride(dir, "web", "a:1@sha256:aa", stackFile(t, dir)) _, err := WriteOverride(dir, "web", "a:1@sha256:aa")
assert.ErrorContains(t, err, "refused") assert.ErrorContains(t, err, "refused")
got, _ := os.ReadFile(filepath.Join(dir, OverrideName)) got, _ := os.ReadFile(filepath.Join(dir, OverrideName))
assert.Equal(t, body, string(got)) assert.Equal(t, body, string(got))
@@ -154,7 +69,7 @@ func TestWriteOverride(t *testing.T) {
t.Run("refuses next to "+other, func(t *testing.T) { t.Run("refuses next to "+other, func(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, other), []byte("services: {}\n"), 0o644)) require.NoError(t, os.WriteFile(filepath.Join(dir, other), []byte("services: {}\n"), 0o644))
_, _, err := WriteOverride(dir, "web", "a:1@sha256:aa", stackFile(t, dir)) _, err := WriteOverride(dir, "web", "a:1@sha256:aa")
assert.ErrorContains(t, err, other) assert.ErrorContains(t, err, other)
_, statErr := os.Stat(filepath.Join(dir, OverrideName)) _, statErr := os.Stat(filepath.Join(dir, OverrideName))
assert.True(t, errors.Is(statErr, os.ErrNotExist)) assert.True(t, errors.Is(statErr, os.ErrNotExist))
@@ -162,13 +77,12 @@ func TestWriteOverride(t *testing.T) {
} }
t.Run("leaves no temp files", func(t *testing.T) { t.Run("leaves no temp files", func(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
_, _, err := WriteOverride(dir, "web", "a:1@sha256:aa", stackFile(t, dir)) _, err := WriteOverride(dir, "web", "a:1@sha256:aa")
require.NoError(t, err) require.NoError(t, err)
entries, err := os.ReadDir(dir) entries, err := os.ReadDir(dir)
require.NoError(t, err) require.NoError(t, err)
require.Len(t, entries, 2) require.Len(t, entries, 1)
assert.Equal(t, "compose.override.yaml", entries[0].Name()) assert.Equal(t, OverrideName, entries[0].Name())
assert.Equal(t, "compose.yaml", entries[1].Name())
}) })
} }
+4 -17
View File
@@ -16,8 +16,7 @@ type DistributionClient interface {
} }
// Resolver turns "repo:tag" into "repo:tag@digest" by asking the local // Resolver turns "repo:tag" into "repo:tag@digest" by asking the local
// daemon, which contacts the registry. A ref that already has a digest is // daemon, which contacts the registry.
// passed through.
type Resolver struct { type Resolver struct {
cli DistributionClient cli DistributionClient
auth *RegistryAuth auth *RegistryAuth
@@ -28,15 +27,13 @@ func NewResolver(cli DistributionClient, auth *RegistryAuth) *Resolver {
return &Resolver{cli: cli, auth: auth} return &Resolver{cli: cli, auth: auth}
} }
// Resolve returns ref pinned to the digest its tag currently points at. A ref // Resolve returns ref pinned to the digest its tag currently points at. It
// that already has a digest ("repo:tag@digest" or "repo@digest") is returned // fails for a ref without a tag or with a digest already.
// unchanged without asking the daemon. It fails for a ref with neither tag nor
// digest.
func (r *Resolver) Resolve(ctx context.Context, ref string) (string, error) { func (r *Resolver) Resolve(ctx context.Context, ref string) (string, error) {
p := imageref.Parse(ref) p := imageref.Parse(ref)
switch { switch {
case p.Digest != "": case p.Digest != "":
return ref, nil return "", fmt.Errorf("image %q already has a digest", ref)
case p.Tag == "": case p.Tag == "":
return "", fmt.Errorf("image %q has no tag to resolve", ref) return "", fmt.Errorf("image %q has no tag to resolve", ref)
} }
@@ -49,13 +46,3 @@ func (r *Resolver) Resolve(ctx context.Context, ref string) (string, error) {
} }
return ref + "@" + info.Descriptor.Digest.String(), nil return ref + "@" + info.Descriptor.Digest.String(), nil
} }
// resolveNote is the job-output line that says which ref is deployed:
// "resolved" when pinned came from resolving a tag, "pinned" when ref already
// carried its digest.
func resolveNote(ref, pinned string) string {
if pinned == ref {
return "pinned " + pinned
}
return "resolved " + pinned
}
+4 -11
View File
@@ -21,21 +21,14 @@ func TestResolver(t *testing.T) {
assert.Equal(t, []string{"gitea.shcizo.se/shcizo/slusk:1.4"}, dist.refs) assert.Equal(t, []string{"gitea.shcizo.se/shcizo/slusk:1.4"}, dist.refs)
assert.Equal(t, []string{"enc-gitea"}, dist.auths) assert.Equal(t, []string{"enc-gitea"}, dist.auths)
}) })
t.Run("a ref with a digest passes through without asking the daemon", func(t *testing.T) { t.Run("digest or missing tag is refused without asking the daemon", func(t *testing.T) {
for _, ref := range []string{"repo@sha256:x", "repo:1@sha256:x", "localhost:5000/repo:1@sha256:x"} { for _, ref := range []string{"repo@sha256:x", "repo:1@sha256:x", "repo"} {
dist := &fakeDist{digest: d} dist := &fakeDist{digest: d}
got, err := NewResolver(dist, nil).Resolve(context.Background(), ref) _, err := NewResolver(dist, nil).Resolve(context.Background(), ref)
require.NoError(t, err, ref) assert.Error(t, err, ref)
assert.Equal(t, ref, got)
assert.Empty(t, dist.refs) assert.Empty(t, dist.refs)
} }
}) })
t.Run("a bare ref is refused without asking the daemon", func(t *testing.T) {
dist := &fakeDist{digest: d}
_, err := NewResolver(dist, nil).Resolve(context.Background(), "repo")
assert.ErrorContains(t, err, "has no tag to resolve")
assert.Empty(t, dist.refs)
})
t.Run("daemon error is wrapped", func(t *testing.T) { t.Run("daemon error is wrapped", func(t *testing.T) {
cause := errors.New("unauthorized") cause := errors.New("unauthorized")
_, err := NewResolver(&fakeDist{err: cause}, nil).Resolve(context.Background(), "repo:1") _, err := NewResolver(&fakeDist{err: cause}, nil).Resolve(context.Background(), "repo:1")
+6 -6
View File
@@ -35,11 +35,11 @@ func NewSwarmExecutor(cli SwarmClient, resolve *Resolver, auth *RegistryAuth) *S
} }
// Execute checks that the local daemon is a manager of the job's cluster, // Execute checks that the local daemon is a manager of the job's cluster,
// pins job.Ref to its current digest (a ref that has one is used as is) and // pins job.Ref to its current digest and updates the service to it. The
// updates the service to it. The update is addressed by the ID and version of // update is addressed by the ID and version of the service that was
// the service that was inspected, so it changes exactly that object. Success // inspected, so it changes exactly that object. Success means the daemon
// means the daemon accepted the new spec, not that the tasks have converged. // accepted the new spec, not that the tasks have converged. It returns the
// It returns the pinned ref. // pinned ref.
func (e *SwarmExecutor) Execute(ctx context.Context, job protocol.JobStart, out io.Writer) (string, error) { func (e *SwarmExecutor) Execute(ctx context.Context, job protocol.JobStart, out io.Writer) (string, error) {
info, err := e.cli.Info(ctx) info, err := e.cli.Info(ctx)
if err != nil { if err != nil {
@@ -71,7 +71,7 @@ func (e *SwarmExecutor) Execute(ctx context.Context, job protocol.JobStart, out
if err != nil { if err != nil {
return "", err return "", err
} }
fmt.Fprintln(out, resolveNote(job.Ref, pinned)) fmt.Fprintf(out, "resolved %s\n", pinned)
spec := svc.Spec spec := svc.Spec
spec.TaskTemplate.ContainerSpec.Image = pinned spec.TaskTemplate.ContainerSpec.Image = pinned
-16
View File
@@ -77,22 +77,6 @@ func swarmSvc(name string) swarm.Service {
} }
} }
func TestSwarmUsesPinnedRef(t *testing.T) {
const ref = "gitea/gitea:1.26@sha256:abababababababababababababababababababababababababababababababab"
job := protocol.JobStart{Kind: protocol.KindSwarm, SwarmClusterID: "c1", SwarmService: "gitea_gitea", Ref: ref}
fake := &fakeSwarm{info: managerOf("c1"), svc: swarmSvc("gitea_gitea")}
dist := &fakeDist{digest: "sha256:other"}
var out bytes.Buffer
got, err := NewSwarmExecutor(fake, NewResolver(dist, nil), nil).Execute(context.Background(), job, &out)
require.NoError(t, err)
assert.Equal(t, ref, got)
assert.Equal(t, ref, fake.updSpec.TaskTemplate.ContainerSpec.Image)
assert.Empty(t, dist.refs, "no registry lookup")
assert.Contains(t, out.String(), "pinned "+ref)
}
func TestSwarmExecutor(t *testing.T) { func TestSwarmExecutor(t *testing.T) {
job := protocol.JobStart{Kind: protocol.KindSwarm, SwarmClusterID: "c1", SwarmService: "gitea_gitea", Ref: "gitea/gitea:1.27"} job := protocol.JobStart{Kind: protocol.KindSwarm, SwarmClusterID: "c1", SwarmService: "gitea_gitea", Ref: "gitea/gitea:1.27"}
workerInfo := managerOf("c1") workerInfo := managerOf("c1")
+5 -70
View File
@@ -24,7 +24,6 @@ import (
"github.com/shcizo/package-updater/internal/agent/conn" "github.com/shcizo/package-updater/internal/agent/conn"
"github.com/shcizo/package-updater/internal/agent/updater" "github.com/shcizo/package-updater/internal/agent/updater"
"github.com/shcizo/package-updater/internal/agenttoken" "github.com/shcizo/package-updater/internal/agenttoken"
"github.com/shcizo/package-updater/internal/imageref"
"github.com/shcizo/package-updater/internal/master/events" "github.com/shcizo/package-updater/internal/master/events"
"github.com/shcizo/package-updater/internal/master/hub" "github.com/shcizo/package-updater/internal/master/hub"
"github.com/shcizo/package-updater/internal/master/model" "github.com/shcizo/package-updater/internal/master/model"
@@ -45,63 +44,6 @@ type memStore struct {
jobs map[int64]*model.Job jobs map[int64]*model.Job
nextJob int64 nextJob int64
finishApplied int // FinishJob calls that returned true finishApplied int // FinishJob calls that returned true
versions map[int64]model.ImageVersion
}
// seedVersion adds an image_versions row that GetImageVersion returns.
func (m *memStore) seedVersion(v model.ImageVersion) {
m.mu.Lock()
defer m.mu.Unlock()
if m.versions == nil {
m.versions = map[int64]model.ImageVersion{}
}
m.versions[v.ID] = v
}
func (m *memStore) GetImageVersion(_ context.Context, id int64) (model.ImageVersion, error) {
m.mu.Lock()
defer m.mu.Unlock()
v, ok := m.versions[id]
if !ok {
return model.ImageVersion{}, model.ErrNotFound
}
return v, nil
}
func (m *memStore) ListServices(_ context.Context) ([]model.Service, error) {
m.mu.Lock()
defer m.mu.Unlock()
var out []model.Service
for _, svc := range m.svcRows {
if svc.RemovedAt == nil {
out = append(out, svc)
}
}
sort.Slice(out, func(i, j int) bool { return out[i].ID < out[j].ID })
return out, nil
}
func (m *memStore) SetPollError(_ context.Context, ids []int64, msg string) error {
m.mu.Lock()
defer m.mu.Unlock()
for _, id := range ids {
svc := m.svcRows[id]
svc.LastPollError = msg
m.svcRows[id] = svc
}
return nil
}
func (m *memStore) SetLastTagDigest(_ context.Context, id int64, digest string) error {
m.mu.Lock()
defer m.mu.Unlock()
svc, ok := m.svcRows[id]
if !ok {
return model.ErrNotFound
}
svc.LastTagDigest = digest
m.svcRows[id] = svc
return nil
} }
// seedService adds a service row that jobs can be triggered for. // seedService adds a service row that jobs can be triggered for.
@@ -146,7 +88,7 @@ func (m *memStore) SetAvailable(_ context.Context, id int64, a model.Available)
if !ok { if !ok {
return model.ErrNotFound return model.ErrNotFound
} }
svc.AvailableRef, svc.AvailableDigest, svc.AvailableSource, svc.AvailableAt = a.Ref, a.Digest, a.Source, &a.At svc.AvailableRef, svc.AvailableSource, svc.AvailableAt = a.Ref, a.Source, &a.At
m.svcRows[id] = svc m.svcRows[id] = svc
return nil return nil
} }
@@ -242,15 +184,10 @@ func (m *memStore) FinishJob(_ context.Context, agentID, jobID int64, f model.Jo
if f.Status == model.JobSucceeded { if f.Status == model.JobSucceeded {
svc := m.svcRows[j.ServiceID] svc := m.svcRows[j.ServiceID]
svc.DesiredRef = f.Ref svc.DesiredRef = f.Ref
svc.AwaitingInventory = true
if _, d, _ := strings.Cut(f.Ref, "@"); !strings.Contains(requestedRef, "@") && imageref.IsDigest(d) {
svc.LastTagDigest = d
}
// Same rule as the SQL: the job was created for the suggested ref after it was reported. // Same rule as the SQL: the job was created for the suggested ref after it was reported.
requested, digest, _ := strings.Cut(requestedRef, "@") requested, _, _ := strings.Cut(requestedRef, "@")
if svc.AvailableAt != nil && svc.AvailableRef == requested && !svc.AvailableAt.After(j.CreatedAt) && if svc.AvailableAt != nil && svc.AvailableRef == requested && !svc.AvailableAt.After(j.CreatedAt) {
(digest == "" || svc.AvailableDigest == digest) { svc.AvailableRef, svc.AvailableSource, svc.AvailableAt = "", "", nil
svc.AvailableRef, svc.AvailableDigest, svc.AvailableSource, svc.AvailableAt = "", "", "", nil
} }
m.svcRows[j.ServiceID] = svc m.svcRows[j.ServiceID] = svc
} }
@@ -345,9 +282,7 @@ type currentHub struct{ p atomic.Pointer[hub.Hub] }
func (c *currentHub) ServeHTTP(w http.ResponseWriter, r *http.Request) { c.p.Load().ServeHTTP(w, r) } func (c *currentHub) ServeHTTP(w http.ResponseWriter, r *http.Request) { c.p.Load().ServeHTTP(w, r) }
func (c *currentHub) Disconnect(id int64) { c.p.Load().Disconnect(id) } func (c *currentHub) Disconnect(id int64) { c.p.Load().Disconnect(id) }
func (c *currentHub) Route(svc model.Service, minProtocol int) (model.Route, error) { func (c *currentHub) Route(svc model.Service) (model.Route, error) { return c.p.Load().Route(svc) }
return c.p.Load().Route(svc, minProtocol)
}
func (c *currentHub) StartJob(r model.Route, id int64, s protocol.JobStart) error { func (c *currentHub) StartJob(r model.Route, id int64, s protocol.JobStart) error {
return c.p.Load().StartJob(r, id, s) return c.p.Load().StartJob(r, id, s)
} }
-113
View File
@@ -1,113 +0,0 @@
package e2e
import (
"context"
"io"
"log/slog"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/shcizo/package-updater/internal/master/model"
"github.com/shcizo/package-updater/internal/master/poller"
"github.com/shcizo/package-updater/internal/protocol"
)
const (
driftDesired = "sha256:" + "d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1"
driftOther = "sha256:" + "0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a"
)
// seedDrifted replaces the environment's service with an auto service that was
// updated to slusk:1.4.2 at driftDesired but runs driftOther now.
func (e *jobEnv) seedDrifted() {
e.store.seedService(model.Service{ID: 1, Kind: protocol.KindCompose, ComposeProject: "slusk", ComposeService: "web",
AgentID: ptr(agentID), ImageRepo: slusk, UpdateMode: model.ModeAuto,
DesiredRef: slusk + ":1.4.2@" + driftDesired,
ActualRef: slusk + ":1.4.2", ActualDigest: driftOther})
}
// inventoryWrite is what the store does for an agent's upsert of the service.
func (e *jobEnv) inventoryWrite(id int64, actualRef, actualDigest string) {
e.store.mu.Lock()
defer e.store.mu.Unlock()
svc := e.store.svcRows[id]
svc.ActualRef, svc.ActualDigest, svc.AwaitingInventory = actualRef, actualDigest, false
e.store.svcRows[id] = svc
}
func TestDriftedAutoServiceIsNotUpdatedByPollOrWebhook(t *testing.T) {
e := newJobEnv(t)
e.seedDrifted()
reg := &fixedRegistry{digests: map[string]string{slusk + ":1.4.2": "sha256:new"}}
p := poller.New(e.store, e.jobs, reg, e.m.broker, poller.Config{Workers: 1}, slog.New(slog.NewTextHandler(io.Discard, nil)))
p.Cycle(context.Background())
assert.Equal(t, 1, reg.calls)
assert.Zero(t, e.store.jobCount(), "polling starts no job for a drifted service")
svc, err := e.store.GetService(context.Background(), 1)
require.NoError(t, err)
assert.Equal(t, slusk+":1.4.2", svc.AvailableRef, "the suggestion is still recorded")
code, body := callWebhook(t, e.webhookServer(t), `{"image":"`+slusk+`:ignored","tag":"2.0.0"}`)
assert.Equal(t, 200, code)
require.Len(t, body.Results, 1)
assert.Equal(t, "refused", body.Results[0].Status)
assert.Contains(t, body.Results[0].Error, "drifted")
assert.Zero(t, e.store.jobCount(), "the webhook starts no job either")
}
func TestRestoreDeploysThePinnedRefOnTheAgent(t *testing.T) {
e := newJobEnv(t)
e.seedDrifted()
close(e.exec.release)
job, err := e.jobs.RestoreDesired(context.Background(), 1, "alice")
require.NoError(t, err)
assert.Equal(t, model.TriggerManual, job.Trigger)
select {
case ref := <-e.exec.started:
assert.Equal(t, slusk+":1.4.2@"+driftDesired, ref, "the agent receives exactly the desired ref")
case <-time.After(waitFor):
t.Fatal("the agent never received the job")
}
done := e.waitStatus(t, job.ID, model.JobSucceeded)
assert.Equal(t, slusk+":1.4.2@"+driftDesired, done.ToRef)
svc, err := e.store.GetService(context.Background(), 1)
require.NoError(t, err)
assert.Equal(t, driftOther, svc.LastTagDigest, "the drifted digest is remembered so polling does not redeploy it")
assert.False(t, svc.Drifted(), "the inventory has not caught up yet")
e.inventoryWrite(1, slusk+":1.4.2", driftDesired)
svc, err = e.store.GetService(context.Background(), 1)
require.NoError(t, err)
assert.False(t, svc.Drifted())
}
func TestNoDriftAfterASuccessfulJob(t *testing.T) {
e := newJobEnv(t)
close(e.exec.release)
job, err := e.jobs.Trigger(context.Background(), 1, "alice")
require.NoError(t, err)
e.waitStatus(t, job.ID, model.JobSucceeded)
svc, err := e.store.GetService(context.Background(), 1)
require.NoError(t, err)
assert.Equal(t, slusk+":1.4.2@"+e2eDigest, svc.DesiredRef)
assert.Equal(t, slusk+":1.4.2@sha256:old", svc.ActualRef, "the agent's inventory write has not arrived")
assert.False(t, svc.Drifted(), "between job.result and the inventory write the row is not drifted")
e.inventoryWrite(1, slusk+":1.4.2", e2eDigest)
svc, _ = e.store.GetService(context.Background(), 1)
assert.False(t, svc.Drifted())
e.inventoryWrite(1, slusk+":1.4.2", driftOther)
svc, _ = e.store.GetService(context.Background(), 1)
assert.True(t, svc.Drifted(), "a later inventory write of another digest is drift")
}
+5 -11
View File
@@ -32,9 +32,6 @@ const (
lostMessage = "lost: the agent did not know this job when it reconnected" lostMessage = "lost: the agent did not know this job when it reconnected"
) )
// e2eDigest is the digest the fake executor pins a tag to.
const e2eDigest = "sha256:" + "e2e0e2e0e2e0e2e0e2e0e2e0e2e0e2e0e2e0e2e0e2e0e2e0e2e0e2e0e2e0e2e0"
// gateExecutor writes a line of output, then blocks until release is closed // gateExecutor writes a line of output, then blocks until release is closed
// or the job is cancelled. // or the job is cancelled.
type gateExecutor struct { type gateExecutor struct {
@@ -55,10 +52,7 @@ func (g *gateExecutor) Execute(ctx context.Context, job protocol.JobStart, out i
return "", ctx.Err() return "", ctx.Err()
} }
_, _ = io.WriteString(out, "recreated\n") _, _ = io.WriteString(out, "recreated\n")
if strings.Contains(job.Ref, "@") { return job.Ref + "@sha256:e2e", nil
return job.Ref, nil
}
return job.Ref + "@" + e2eDigest, nil
} }
// jobAgent is a connected agent with a real runner and its two lifetimes: the // jobAgent is a connected agent with a real runner and its two lifetimes: the
@@ -125,7 +119,7 @@ func (e *jobEnv) waitRoutable(t *testing.T) {
t.Helper() t.Helper()
svc, err := e.store.GetService(context.Background(), 1) svc, err := e.store.GetService(context.Background(), 1)
require.NoError(t, err) require.NoError(t, err)
require.Eventually(t, func() bool { _, err := e.m.hub.Route(svc, protocol.MinJobVersion); return err == nil }, waitFor, pollEvery) require.Eventually(t, func() bool { _, err := e.m.hub.Route(svc); return err == nil }, waitFor, pollEvery)
} }
func (e *jobEnv) job(t *testing.T, id int64) model.Job { func (e *jobEnv) job(t *testing.T, id int64) model.Job {
@@ -165,12 +159,12 @@ func TestManualJobFlowsToAgentAndBack(t *testing.T) {
j := e.trigger(t) j := e.trigger(t)
done := e.waitStatus(t, j.ID, model.JobSucceeded) done := e.waitStatus(t, j.ID, model.JobSucceeded)
assert.Equal(t, slusk+":1.4.2@"+e2eDigest, done.ToRef) assert.Equal(t, slusk+":1.4.2@sha256:e2e", done.ToRef)
assert.Contains(t, done.Output, "pulling "+slusk+":1.4.2") assert.Contains(t, done.Output, "pulling "+slusk+":1.4.2")
assert.Contains(t, done.Output, "recreated") assert.Contains(t, done.Output, "recreated")
svc, err := e.store.GetService(context.Background(), 1) svc, err := e.store.GetService(context.Background(), 1)
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, slusk+":1.4.2@"+e2eDigest, svc.DesiredRef) assert.Equal(t, slusk+":1.4.2@sha256:e2e", svc.DesiredRef)
require.Eventually(t, func() bool { return len(e.agent.runner.Known()) == 0 }, waitFor, pollEvery, "the result must be acked") require.Eventually(t, func() bool { return len(e.agent.runner.Known()) == 0 }, waitFor, pollEvery, "the result must be acked")
} }
@@ -246,7 +240,7 @@ func TestPhaseOneAgentGetsNoJobs(t *testing.T) {
require.NoError(t, wsjson.Write(ctx, c, snap)) require.NoError(t, wsjson.Write(ctx, c, snap))
require.Eventually(t, func() bool { require.Eventually(t, func() bool {
_, err := m.hub.Route(svcRow, protocol.MinJobVersion) _, err := m.hub.Route(svcRow)
return errors.Is(err, model.ErrAgentTooOld) return errors.Is(err, model.ErrAgentTooOld)
}, waitFor, pollEvery) }, waitFor, pollEvery)
-133
View File
@@ -1,133 +0,0 @@
package e2e
import (
"context"
"io"
"log/slog"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/shcizo/package-updater/internal/master/jobs"
"github.com/shcizo/package-updater/internal/master/model"
"github.com/shcizo/package-updater/internal/master/poller"
"github.com/shcizo/package-updater/internal/protocol"
)
// jobCount is the number of jobs created so far.
func (m *memStore) jobCount() int {
m.mu.Lock()
defer m.mu.Unlock()
return len(m.jobs)
}
// fixedRegistry answers every lookup from a fixed table.
type fixedRegistry struct {
digests map[string]string
calls int
}
func (f *fixedRegistry) Digest(_ context.Context, ref string) (string, error) {
f.calls++
return f.digests[ref], nil
}
func TestPickPinnedVersion(t *testing.T) {
env := newJobEnv(t)
env.store.seedVersion(model.ImageVersion{ID: 5, Repo: slusk, Tag: "1.3.0", Digest: "sha256:old130", Source: model.SourceRunning})
job, err := env.jobs.TriggerVersion(context.Background(), 1, 5, "alice")
require.NoError(t, err)
assert.Equal(t, slusk+":1.3.0@sha256:old130", job.ToRef)
select {
case ref := <-env.exec.started:
assert.Equal(t, slusk+":1.3.0@sha256:old130", ref, "the agent receives exactly the picked version")
case <-time.After(waitFor):
t.Fatal("the agent never received the job")
}
close(env.exec.release)
done := env.waitStatus(t, job.ID, model.JobSucceeded)
assert.Equal(t, model.TriggerManual, done.Trigger)
assert.Equal(t, "alice", done.RequestedBy)
}
func TestPickVersionOfAnotherRepoIsRefused(t *testing.T) {
env := newJobEnv(t)
env.store.seedVersion(model.ImageVersion{ID: 6, Repo: "evil/other", Tag: "1", Digest: "sha256:x", Source: model.SourcePoll})
_, err := env.jobs.TriggerVersion(context.Background(), 1, 6, "alice")
assert.ErrorIs(t, err, model.ErrNotFound)
assert.Zero(t, env.store.jobCount(), "no job was created")
}
func TestPollStartsAutoJob(t *testing.T) {
env := newJobEnv(t)
env.store.seedService(model.Service{ID: 1, Kind: protocol.KindCompose, ComposeProject: "slusk", ComposeService: "web",
AgentID: ptr(agentID), ImageRepo: slusk, ActualRef: slusk + ":1.4.2@sha256:old", ActualDigest: "sha256:old",
UpdateMode: model.ModeAuto})
reg := &fixedRegistry{digests: map[string]string{slusk + ":1.4.2": "sha256:new"}}
p := poller.New(env.store, env.jobs, reg, env.m.broker, poller.Config{Workers: 1}, slog.New(slog.NewTextHandler(io.Discard, nil)))
p.Cycle(context.Background())
select {
case ref := <-env.exec.started:
assert.Equal(t, slusk+":1.4.2", ref, "the agent gets repo:tag and resolves the digest itself")
case <-time.After(waitFor):
t.Fatal("the agent never received the job")
}
require.Equal(t, 1, env.store.jobCount())
job := env.job(t, 1)
assert.Equal(t, model.TriggerPoll, job.Trigger)
assert.Equal(t, slusk+":1.4.2", job.ToRef)
svc, err := env.store.GetService(context.Background(), 1)
require.NoError(t, err)
assert.Equal(t, slusk+":1.4.2", svc.AvailableRef)
assert.Equal(t, "sha256:new", svc.AvailableDigest)
assert.Equal(t, model.SourcePoll, svc.AvailableSource)
assert.Equal(t, "sha256:new", svc.LastTagDigest)
p.Cycle(context.Background())
assert.Equal(t, 2, reg.calls, "the registry is asked again")
assert.Equal(t, 1, env.store.jobCount(), "the same digest does not start another job")
close(env.exec.release)
env.waitStatus(t, job.ID, model.JobSucceeded)
}
// An update started by the webhook, then a rollback by hand: polling must not
// redo the update while the registry still shows the digest that was deployed.
func TestPollKeepsARollbackAfterAWebhookUpdate(t *testing.T) {
env := newJobEnv(t)
ctx := context.Background()
close(env.exec.release)
env.store.seedService(model.Service{ID: 1, Kind: protocol.KindCompose, ComposeProject: "slusk", ComposeService: "web",
AgentID: ptr(agentID), ImageRepo: slusk, ActualRef: slusk + ":main@sha256:old", ActualDigest: "sha256:old",
UpdateMode: model.ModeAuto})
env.store.seedVersion(model.ImageVersion{ID: 5, Repo: slusk, Tag: "main", Digest: "sha256:old", Source: model.SourceRunning})
svc, err := env.store.GetService(ctx, 1)
require.NoError(t, err)
hook, err := env.jobs.Start(ctx, svc, jobs.StartRequest{Trigger: model.TriggerWebhook, Ref: slusk + ":main"})
require.NoError(t, err)
env.waitStatus(t, hook.ID, model.JobSucceeded)
svc, err = env.store.GetService(ctx, 1)
require.NoError(t, err)
assert.Equal(t, e2eDigest, svc.LastTagDigest, "the deployed digest of the tag is remembered")
pick, err := env.jobs.TriggerVersion(ctx, 1, 5, "alice")
require.NoError(t, err)
env.waitStatus(t, pick.ID, model.JobSucceeded)
jobsBefore := env.store.jobCount()
reg := &fixedRegistry{digests: map[string]string{slusk + ":main": e2eDigest}}
p := poller.New(env.store, env.jobs, reg, env.m.broker, poller.Config{Workers: 1}, slog.New(slog.NewTextHandler(io.Discard, nil)))
p.Cycle(ctx)
assert.Equal(t, 1, reg.calls)
assert.Equal(t, jobsBefore, env.store.jobCount(), "the rollback stays")
}
+2 -3
View File
@@ -16,7 +16,6 @@ import (
"github.com/shcizo/package-updater/internal/master/model" "github.com/shcizo/package-updater/internal/master/model"
"github.com/shcizo/package-updater/internal/master/webhook" "github.com/shcizo/package-updater/internal/master/webhook"
"github.com/shcizo/package-updater/internal/protocol"
) )
// webhookServer mounts the webhook handler on a test server, wired to the // webhookServer mounts the webhook handler on a test server, wired to the
@@ -99,7 +98,7 @@ func TestWebhookWaitsForAutoJob(t *testing.T) {
job := e.job(t, r.body.Results[0].JobID) job := e.job(t, r.body.Results[0].JobID)
assert.Equal(t, model.TriggerWebhook, job.Trigger) assert.Equal(t, model.TriggerWebhook, job.Trigger)
assert.Equal(t, model.JobSucceeded, job.Status) assert.Equal(t, model.JobSucceeded, job.Status)
assert.Equal(t, slusk+":2.0.0@"+e2eDigest, job.ToRef) assert.Equal(t, slusk+":2.0.0@sha256:e2e", job.ToRef)
svc, err := e.store.GetService(context.Background(), 1) svc, err := e.store.GetService(context.Background(), 1)
require.NoError(t, err) require.NoError(t, err)
assert.Empty(t, svc.AvailableRef, "the job deployed the suggestion, so it is cleared") assert.Empty(t, svc.AvailableRef, "the job deployed the suggestion, so it is cleared")
@@ -112,7 +111,7 @@ func TestWebhookFailedAutoJobIs500(t *testing.T) {
e.agent.stopConn() // no agent connection: the job fails at once with "agent offline" e.agent.stopConn() // no agent connection: the job fails at once with "agent offline"
require.Eventually(t, func() bool { require.Eventually(t, func() bool {
svc, _ := e.store.GetService(context.Background(), 1) svc, _ := e.store.GetService(context.Background(), 1)
_, err := e.m.hub.Route(svc, protocol.MinJobVersion) _, err := e.m.hub.Route(svc)
return err != nil return err != nil
}, waitFor, pollEvery) }, waitFor, pollEvery)
+1 -9
View File
@@ -2,10 +2,7 @@
// and digest. // and digest.
package imageref package imageref
import ( import "strings"
"regexp"
"strings"
)
// Ref is an image reference split into its parts. Tag and Digest are empty // Ref is an image reference split into its parts. Tag and Digest are empty
// when the reference has none. // when the reference has none.
@@ -44,8 +41,3 @@ func (r Ref) String() string {
} }
return s return s
} }
var digestPattern = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`)
// IsDigest reports whether d is a well-formed sha256 digest.
func IsDigest(d string) bool { return digestPattern.MatchString(d) }
-17
View File
@@ -1,7 +1,6 @@
package imageref package imageref
import ( import (
"strings"
"testing" "testing"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
@@ -28,19 +27,3 @@ func TestParse(t *testing.T) {
}) })
} }
} }
func TestIsDigest(t *testing.T) {
cases := []struct {
in string
want bool
}{
{"sha256:" + strings.Repeat("a", 64), true},
{"sha256:" + strings.Repeat("a", 63), false},
{"sha256:" + strings.Repeat("A", 64), false},
{"sha512:" + strings.Repeat("a", 64), false},
{"", false},
}
for _, c := range cases {
assert.Equal(t, c.want, IsDigest(c.in), c.in)
}
}
-14
View File
@@ -28,17 +28,11 @@ type Config struct {
// UpdateTimeout bounds how long one webhook request waits for its auto // UpdateTimeout bounds how long one webhook request waits for its auto
// jobs (UPDATE_TIMEOUT, default 5m). // jobs (UPDATE_TIMEOUT, default 5m).
UpdateTimeout time.Duration UpdateTimeout time.Duration
// PollInterval is the time between registry polling cycles
// (POLL_INTERVAL, default 6h).
PollInterval time.Duration
} }
// defaultUpdateTimeout is used when UPDATE_TIMEOUT is unset. // defaultUpdateTimeout is used when UPDATE_TIMEOUT is unset.
const defaultUpdateTimeout = 5 * time.Minute const defaultUpdateTimeout = 5 * time.Minute
// defaultPollInterval is used when POLL_INTERVAL is unset.
const defaultPollInterval = 6 * time.Hour
// Load reads and validates the master configuration, failing fast on // Load reads and validates the master configuration, failing fast on
// anything missing so a half-configured master never starts. // anything missing so a half-configured master never starts.
func Load() (*Config, error) { func Load() (*Config, error) {
@@ -56,7 +50,6 @@ func Load() (*Config, error) {
LogLevel: getenvDefault("LOG_LEVEL", "info"), LogLevel: getenvDefault("LOG_LEVEL", "info"),
UpdaterAPIKey: os.Getenv("UPDATER_API_KEY"), UpdaterAPIKey: os.Getenv("UPDATER_API_KEY"),
UpdateTimeout: defaultUpdateTimeout, UpdateTimeout: defaultUpdateTimeout,
PollInterval: defaultPollInterval,
} }
for _, r := range []struct{ name, value string }{ for _, r := range []struct{ name, value string }{
{"DATABASE_URL", cfg.DatabaseURL}, {"DATABASE_URL", cfg.DatabaseURL},
@@ -84,13 +77,6 @@ func Load() (*Config, error) {
} }
cfg.UpdateTimeout = d cfg.UpdateTimeout = d
} }
if v := os.Getenv("POLL_INTERVAL"); v != "" {
d, err := time.ParseDuration(v)
if err != nil || d <= 0 {
return nil, fmt.Errorf("POLL_INTERVAL must be a positive duration such as 6h, got %q", v)
}
cfg.PollInterval = d
}
return cfg, nil return cfg, nil
} }
+1 -21
View File
@@ -22,7 +22,7 @@ func setAll(t *testing.T) {
for k, v := range required { for k, v := range required {
t.Setenv(k, v) t.Setenv(k, v)
} }
for _, k := range []string{"PORT", "AGENT_PORT", "AGENT_IMAGE", "LOG_LEVEL", "UPDATER_API_KEY", "UPDATE_TIMEOUT", "POLL_INTERVAL"} { for _, k := range []string{"PORT", "AGENT_PORT", "AGENT_IMAGE", "LOG_LEVEL", "UPDATER_API_KEY", "UPDATE_TIMEOUT"} {
t.Setenv(k, "") t.Setenv(k, "")
} }
} }
@@ -38,26 +38,6 @@ func TestLoad_Defaults(t *testing.T) {
assert.Equal(t, "updater-admins", cfg.AllowedGroup) assert.Equal(t, "updater-admins", cfg.AllowedGroup)
assert.Empty(t, cfg.UpdaterAPIKey) assert.Empty(t, cfg.UpdaterAPIKey)
assert.Equal(t, 5*time.Minute, cfg.UpdateTimeout) assert.Equal(t, 5*time.Minute, cfg.UpdateTimeout)
assert.Equal(t, 6*time.Hour, cfg.PollInterval)
}
func TestLoad_PollInterval(t *testing.T) {
setAll(t)
t.Setenv("POLL_INTERVAL", "30m")
cfg, err := Load()
require.NoError(t, err)
assert.Equal(t, 30*time.Minute, cfg.PollInterval)
}
func TestLoad_InvalidPollInterval(t *testing.T) {
for _, v := range []string{"abc", "0s", "-1h", "6"} {
t.Run(v, func(t *testing.T) {
setAll(t)
t.Setenv("POLL_INTERVAL", v)
_, err := Load()
assert.ErrorContains(t, err, "POLL_INTERVAL must be a positive duration such as 6h, got")
})
}
} }
func TestLoad_WebhookSettings(t *testing.T) { func TestLoad_WebhookSettings(t *testing.T) {
+2 -2
View File
@@ -43,7 +43,7 @@ func waitRoute(t *testing.T, h *Hub) model.Route {
var r model.Route var r model.Route
require.Eventually(t, func() bool { require.Eventually(t, func() bool {
var err error var err error
r, err = h.Route(svc, protocol.MinJobVersion) r, err = h.Route(svc)
return err == nil return err == nil
}, 2*time.Second, 5*time.Millisecond) }, 2*time.Second, 5*time.Millisecond)
return r return r
@@ -61,7 +61,7 @@ func TestHub_ReconcilesBeforeRoutable(t *testing.T) {
send(t, c, protocol.TypeHello, protocol.Hello{Hostname: "node-2", ProtocolVersion: 2, KnownJobs: []string{"7", "9", "x"}}) send(t, c, protocol.TypeHello, protocol.Hello{Hostname: "node-2", ProtocolVersion: 2, KnownJobs: []string{"7", "9", "x"}})
svc := model.Service{Kind: protocol.KindCompose, AgentID: ptr(node2.ID)} svc := model.Service{Kind: protocol.KindCompose, AgentID: ptr(node2.ID)}
time.Sleep(50 * time.Millisecond) // hello is now blocked inside FailLostJobs time.Sleep(50 * time.Millisecond) // hello is now blocked inside FailLostJobs
_, err = h.Route(svc, protocol.MinJobVersion) _, err = h.Route(svc)
assert.ErrorIs(t, err, model.ErrAgentOffline) assert.ErrorIs(t, err, model.ErrAgentOffline)
close(store.lostGate) close(store.lostGate)
+6 -11
View File
@@ -14,11 +14,9 @@ type candidate struct {
hello protocol.Hello hello protocol.Hello
} }
// Route picks the connection a job for svc goes to, among agents that speak // Route picks the connection a job for svc goes to. It returns
// at least protocol version minProtocol. It returns model.ErrAgentOffline or // model.ErrAgentOffline or model.ErrAgentTooOld when there is none.
// model.ErrAgentTooOld (model.ErrAgentTooOldForPinned when minProtocol asks func (h *Hub) Route(svc model.Service) (model.Route, error) {
// for a pinned ref) when there is none.
func (h *Hub) Route(svc model.Service, minProtocol int) (model.Route, error) {
h.mu.Lock() h.mu.Lock()
conns := make([]candidate, 0, len(h.conns)) conns := make([]candidate, 0, len(h.conns))
for id, c := range h.conns { for id, c := range h.conns {
@@ -27,7 +25,7 @@ func (h *Hub) Route(svc model.Service, minProtocol int) (model.Route, error) {
} }
} }
h.mu.Unlock() h.mu.Unlock()
return route(svc, conns, minProtocol) return route(svc, conns)
} }
// route picks the connection a job for svc goes to. Compose services and // route picks the connection a job for svc goes to. Compose services and
@@ -35,7 +33,7 @@ func (h *Hub) Route(svc model.Service, minProtocol int) (model.Route, error) {
// run on the reporting agent if it qualifies, else on any other connected // run on the reporting agent if it qualifies, else on any other connected
// manager of the same cluster (lowest agent ID first). Too-old wins over // manager of the same cluster (lowest agent ID first). Too-old wins over
// offline only when a connection would otherwise have qualified. // offline only when a connection would otherwise have qualified.
func route(svc model.Service, conns []candidate, minProtocol int) (model.Route, error) { func route(svc model.Service, conns []candidate) (model.Route, error) {
sort.Slice(conns, func(a, b int) bool { return conns[a].agentID < conns[b].agentID }) sort.Slice(conns, func(a, b int) bool { return conns[a].agentID < conns[b].agentID })
tooOld := false tooOld := false
try := func(c candidate) bool { try := func(c candidate) bool {
@@ -44,7 +42,7 @@ func route(svc model.Service, conns []candidate, minProtocol int) (model.Route,
return false return false
} }
} }
if c.hello.ProtocolVersion < minProtocol { if c.hello.ProtocolVersion < protocol.MinJobVersion {
tooOld = true tooOld = true
return false return false
} }
@@ -65,9 +63,6 @@ func route(svc model.Service, conns []candidate, minProtocol int) (model.Route,
} }
} }
if tooOld { if tooOld {
if minProtocol >= protocol.MinPinnedRefVersion {
return model.Route{}, model.ErrAgentTooOldForPinned
}
return model.Route{}, model.ErrAgentTooOld return model.Route{}, model.ErrAgentTooOld
} }
return model.Route{}, model.ErrAgentOffline return model.Route{}, model.ErrAgentOffline
+15 -24
View File
@@ -12,46 +12,37 @@ import (
func TestRoute(t *testing.T) { func TestRoute(t *testing.T) {
composeSvc := model.Service{Kind: protocol.KindCompose, AgentID: ptr(int64(2))} composeSvc := model.Service{Kind: protocol.KindCompose, AgentID: ptr(int64(2))}
swarmSvc := model.Service{Kind: protocol.KindSwarm, AgentID: ptr(int64(1)), SwarmClusterID: "c1"} swarmSvc := model.Service{Kind: protocol.KindSwarm, AgentID: ptr(int64(1)), SwarmClusterID: "c1"}
v2 := protocol.Hello{ProtocolVersion: 2} v2 := protocol.Hello{ProtocolVersion: protocol.Version}
v3 := protocol.Hello{ProtocolVersion: 3} mgrV2 := protocol.Hello{ProtocolVersion: protocol.Version, SwarmManager: true, SwarmClusterID: "c1"}
mgrV2 := protocol.Hello{ProtocolVersion: 2, SwarmManager: true, SwarmClusterID: "c1"}
mgrV3 := protocol.Hello{ProtocolVersion: 3, SwarmManager: true, SwarmClusterID: "c1"}
jobs, pinned := protocol.MinJobVersion, protocol.MinPinnedRefVersion
tests := []struct { tests := []struct {
name string name string
svc model.Service svc model.Service
conns []candidate conns []candidate
min int
want model.Route want model.Route
err error err error
}{ }{
{"compose on own agent", composeSvc, []candidate{{2, 20, v2}}, jobs, model.Route{AgentID: 2, ConnID: 20}, nil}, {"compose on own agent", composeSvc, []candidate{{2, 20, v2}}, model.Route{AgentID: 2, ConnID: 20}, nil},
{"compose, own agent too old", composeSvc, []candidate{{2, 20, protocol.Hello{}}}, jobs, model.Route{}, model.ErrAgentTooOld}, {"compose, own agent too old", composeSvc, []candidate{{2, 20, protocol.Hello{}}}, model.Route{}, model.ErrAgentTooOld},
{"compose, own agent missing", composeSvc, nil, jobs, model.Route{}, model.ErrAgentOffline}, {"compose, own agent missing", composeSvc, nil, model.Route{}, model.ErrAgentOffline},
{"compose, only another agent online", composeSvc, []candidate{{5, 50, v2}}, jobs, model.Route{}, model.ErrAgentOffline}, {"compose, only another agent online", composeSvc, []candidate{{5, 50, v2}}, model.Route{}, model.ErrAgentOffline},
{"compose, v2 agent and a pinned ref", composeSvc, []candidate{{2, 20, v2}}, pinned, model.Route{}, model.ErrAgentTooOldForPinned}, {"swarm on reporting agent", swarmSvc, []candidate{{1, 10, mgrV2}, {3, 30, mgrV2}}, model.Route{AgentID: 1, ConnID: 10}, nil},
{"compose, v3 agent and a pinned ref", composeSvc, []candidate{{2, 20, v3}}, pinned, model.Route{AgentID: 2, ConnID: 20}, nil}, {"swarm, reporting agent offline, other manager", swarmSvc, []candidate{{3, 30, mgrV2}}, model.Route{AgentID: 3, ConnID: 30}, nil},
{"swarm on reporting agent", swarmSvc, []candidate{{1, 10, mgrV2}, {3, 30, mgrV2}}, jobs, model.Route{AgentID: 1, ConnID: 10}, nil}, {"swarm, lowest agent id first", swarmSvc, []candidate{{4, 40, mgrV2}, {3, 30, mgrV2}}, model.Route{AgentID: 3, ConnID: 30}, nil},
{"swarm, reporting agent offline, other manager", swarmSvc, []candidate{{3, 30, mgrV2}}, jobs, model.Route{AgentID: 3, ConnID: 30}, nil},
{"swarm, lowest agent id first", swarmSvc, []candidate{{4, 40, mgrV2}, {3, 30, mgrV2}}, jobs, model.Route{AgentID: 3, ConnID: 30}, nil},
{"swarm, other manager in another cluster", swarmSvc, {"swarm, other manager in another cluster", swarmSvc,
[]candidate{{3, 30, protocol.Hello{ProtocolVersion: 2, SwarmManager: true, SwarmClusterID: "c2"}}}, jobs, model.Route{}, model.ErrAgentOffline}, []candidate{{3, 30, protocol.Hello{ProtocolVersion: 2, SwarmManager: true, SwarmClusterID: "c2"}}}, model.Route{}, model.ErrAgentOffline},
{"swarm, only a v0 manager of the cluster", swarmSvc, {"swarm, only a v0 manager of the cluster", swarmSvc,
[]candidate{{1, 10, protocol.Hello{SwarmManager: true, SwarmClusterID: "c1"}}}, jobs, model.Route{}, model.ErrAgentTooOld}, []candidate{{1, 10, protocol.Hello{SwarmManager: true, SwarmClusterID: "c1"}}}, model.Route{}, model.ErrAgentTooOld},
{"swarm, v0 manager but a v2 manager exists", swarmSvc, {"swarm, v0 manager but a v2 manager exists", swarmSvc,
[]candidate{{1, 10, protocol.Hello{SwarmManager: true, SwarmClusterID: "c1"}}, {3, 30, mgrV2}}, jobs, model.Route{AgentID: 3, ConnID: 30}, nil}, []candidate{{1, 10, protocol.Hello{SwarmManager: true, SwarmClusterID: "c1"}}, {3, 30, mgrV2}}, model.Route{AgentID: 3, ConnID: 30}, nil},
{"swarm, worker of the cluster", swarmSvc, {"swarm, worker of the cluster", swarmSvc,
[]candidate{{1, 10, protocol.Hello{ProtocolVersion: 2, SwarmClusterID: "c1"}}}, jobs, model.Route{}, model.ErrAgentOffline}, []candidate{{1, 10, protocol.Hello{ProtocolVersion: 2, SwarmClusterID: "c1"}}}, model.Route{}, model.ErrAgentOffline},
{"swarm, manager with empty cluster id", model.Service{Kind: protocol.KindSwarm}, {"swarm, manager with empty cluster id", model.Service{Kind: protocol.KindSwarm},
[]candidate{{1, 10, protocol.Hello{ProtocolVersion: 2, SwarmManager: true}}}, jobs, model.Route{}, model.ErrAgentOffline}, []candidate{{1, 10, protocol.Hello{ProtocolVersion: 2, SwarmManager: true}}}, model.Route{}, model.ErrAgentOffline},
{"swarm, pinned ref falls back to a v3 manager when the reporting one is v2", swarmSvc,
[]candidate{{1, 10, mgrV2}, {3, 30, mgrV3}}, pinned, model.Route{AgentID: 3, ConnID: 30}, nil},
{"swarm, pinned ref and only v2 managers", swarmSvc, []candidate{{1, 10, mgrV2}}, pinned, model.Route{}, model.ErrAgentTooOldForPinned},
} }
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
got, err := route(tt.svc, tt.conns, tt.min) got, err := route(tt.svc, tt.conns)
assert.ErrorIs(t, err, tt.err) assert.ErrorIs(t, err, tt.err)
assert.Equal(t, tt.want, got) assert.Equal(t, tt.want, got)
}) })
-237
View File
@@ -1,237 +0,0 @@
package jobs
import (
"context"
"errors"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/shcizo/package-updater/internal/master/model"
"github.com/shcizo/package-updater/internal/protocol"
)
const (
driftRepo = "gitea.shcizo.se/shcizo/slusk"
driftD1 = "sha256:1111111111111111111111111111111111111111111111111111111111111111"
driftD2 = "sha256:2222222222222222222222222222222222222222222222222222222222222222"
driftD3 = "sha256:3333333333333333333333333333333333333333333333333333333333333333"
)
// driftedSvc was updated to 1.4.2 at D1 but now runs 1.4.2 at D2.
func driftedSvc() model.Service {
svc := composeSvc
svc.ImageRepo = driftRepo
svc.DesiredRef = driftRepo + ":1.4.2@" + driftD1
svc.ActualRef = driftRepo + ":1.4.2"
svc.ActualDigest = driftD2
return svc
}
func notUpdatableReason(t *testing.T, err error) string {
t.Helper()
var nu *model.NotUpdatableError
require.True(t, errors.As(err, &nu), "got %v", err)
return nu.Reason
}
func TestRestorable(t *testing.T) {
tests := []struct {
name string
mutate func(*model.Service)
reason string // "" means restorable
}{
{"drifted", func(*model.Service) {}, ""},
{"drifted service that follows no tag", func(s *model.Service) {
s.ActualRef = driftRepo + "@" + driftD2
}, ""},
{"not drifted", func(s *model.Service) { s.ActualDigest = driftD1 }, model.ReasonNotDrifted},
{"never updated", func(s *model.Service) { s.DesiredRef = "" }, model.ReasonNotDrifted},
{"awaiting inventory", func(s *model.Service) { s.AwaitingInventory = true }, model.ReasonNotDrifted},
{"removed", func(s *model.Service) { s.RemovedAt = ptr(time.Now()) }, model.ReasonRemoved},
{"flagged self", func(s *model.Service) { s.Self = true }, model.ReasonSelf},
{"built", func(s *model.Service) { s.Built = true }, model.ReasonBuilt},
{"container", func(s *model.Service) { s.Kind = protocol.KindContainer }, model.ReasonContainer},
{"desired ref is the agent image", func(s *model.Service) {
s.DesiredRef = selfImage + "@" + driftD1
}, model.ReasonSelf},
}
s, _ := newService(&fakeStore{}, &fakeHub{})
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
svc := driftedSvc()
tt.mutate(&svc)
err := s.Restorable(svc)
if tt.reason == "" {
assert.NoError(t, err)
return
}
assert.Equal(t, tt.reason, notUpdatableReason(t, err))
})
}
}
func TestRestoreDesired_StartsAManualJobToTheDesiredRef(t *testing.T) {
svc := driftedSvc()
st := &fakeStore{services: map[int64]model.Service{1: svc}, jobs: map[int64]model.Job{}}
h := &fakeHub{route: model.Route{AgentID: 7, ConnID: 70}}
s, _ := newService(st, h)
job, err := s.RestoreDesired(context.Background(), 1, "sub-1")
require.NoError(t, err)
assert.Equal(t, model.JobQueued, job.Status)
require.Len(t, st.created, 1)
assert.Equal(t, svc.DesiredRef, st.created[0].ToRef)
assert.Equal(t, svc.ActualRef, st.created[0].FromRef)
assert.Equal(t, "sub-1", st.created[0].RequestedBy)
assert.Equal(t, model.TriggerManual, st.created[0].Trigger)
assert.Equal(t, []int{protocol.MinPinnedRefVersion}, h.mins)
require.Len(t, h.starts, 1)
assert.Equal(t, svc.DesiredRef, h.starts[0].start.Ref)
}
func TestRestoreDesired_Refusals(t *testing.T) {
tests := []struct {
name string
mutate func(*model.Service)
reason string
}{
{"not drifted", func(s *model.Service) { s.ActualDigest = driftD1 }, model.ReasonNotDrifted},
{"self", func(s *model.Service) { s.Self = true }, model.ReasonSelf},
{"built", func(s *model.Service) { s.Built = true }, model.ReasonBuilt},
{"container", func(s *model.Service) { s.Kind = protocol.KindContainer }, model.ReasonContainer},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
svc := driftedSvc()
tt.mutate(&svc)
st := &fakeStore{services: map[int64]model.Service{1: svc}, jobs: map[int64]model.Job{}}
h := &fakeHub{route: model.Route{AgentID: 7, ConnID: 70}}
s, pub := newService(st, h)
_, err := s.RestoreDesired(context.Background(), 1, "sub-1")
assert.Equal(t, tt.reason, notUpdatableReason(t, err))
assert.Empty(t, st.created)
assert.Empty(t, h.starts)
assert.Empty(t, pub.topics)
assert.Empty(t, st.lastTag, "last_tag_digest untouched")
})
}
}
func TestRestoreDesired_UnknownService(t *testing.T) {
s, _ := newService(&fakeStore{services: map[int64]model.Service{}}, &fakeHub{})
_, err := s.RestoreDesired(context.Background(), 9, "sub-1")
assert.ErrorIs(t, err, model.ErrNotFound)
}
func TestRestoreDesired_ActiveJobReturnsItAndMarksNothing(t *testing.T) {
activeJob := model.Job{ID: 99, Status: model.JobRunning}
st := &fakeStore{services: map[int64]model.Service{1: driftedSvc()}, jobs: map[int64]model.Job{99: activeJob}, active: &activeJob}
s, _ := newService(st, &fakeHub{route: model.Route{AgentID: 7, ConnID: 70}})
job, err := s.RestoreDesired(context.Background(), 1, "sub-1")
assert.ErrorIs(t, err, model.ErrJobActive)
assert.Equal(t, int64(99), job.ID)
assert.Empty(t, st.lastTag)
}
func TestRestoreDesired_OldAgentGivesAFailedJob(t *testing.T) {
st := &fakeStore{services: map[int64]model.Service{1: driftedSvc()}, jobs: map[int64]model.Job{}}
h := &fakeHub{route: model.Route{AgentID: 7, ConnID: 70}, agentProtocol: 2}
s, _ := newService(st, h)
job, err := s.RestoreDesired(context.Background(), 1, "sub-1")
require.NoError(t, err)
assert.Equal(t, model.JobFailed, job.Status)
assert.Equal(t, model.ErrAgentTooOldForPinned.Error(), job.Error)
assert.Empty(t, h.starts)
}
func TestRestoreDesired_LastTagDigest(t *testing.T) {
tests := []struct {
name string
mutate func(*model.Service)
want string // "" means SetLastTagDigest is not called
}{
{"drifted to another digest of the same tag", func(*model.Service) {}, driftD2},
{"digest in the actual ref", func(s *model.Service) {
s.ActualRef, s.ActualDigest = driftRepo+":1.4.2@"+driftD3, ""
}, driftD3},
{"last tag digest holds back a newer one", func(s *model.Service) {
s.DesiredRef = driftRepo + ":1.4.2@" + driftD1
s.LastTagDigest = driftD3
}, ""},
{"last tag digest equals the desired digest", func(s *model.Service) { s.LastTagDigest = driftD1 }, driftD2},
{"already remembered", func(s *model.Service) { s.LastTagDigest = driftD2 }, ""},
{"different tag", func(s *model.Service) { s.ActualRef = driftRepo + ":1.5.0" }, ""},
{"actual digest malformed", func(s *model.Service) { s.ActualDigest = "sha256:abc" }, ""},
{"desired follows no tag", func(s *model.Service) { s.DesiredRef = driftRepo + "@" + driftD1 }, ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
svc := driftedSvc()
tt.mutate(&svc)
st := &fakeStore{services: map[int64]model.Service{1: svc}, jobs: map[int64]model.Job{}}
s, _ := newService(st, &fakeHub{route: model.Route{AgentID: 7, ConnID: 70}})
_, err := s.RestoreDesired(context.Background(), 1, "sub-1")
require.NoError(t, err)
if tt.want == "" {
assert.Empty(t, st.lastTag)
return
}
assert.Equal(t, map[int64]string{1: tt.want}, st.lastTag)
})
}
}
func TestStart_RefusesPollAndWebhookWhileDrifted(t *testing.T) {
tests := []struct {
trigger model.JobTrigger
refused bool
}{
{model.TriggerPoll, true},
{model.TriggerWebhook, true},
{model.TriggerManual, false},
}
for _, tt := range tests {
t.Run(string(tt.trigger), func(t *testing.T) {
svc := driftedSvc()
st := &fakeStore{services: map[int64]model.Service{1: svc}, jobs: map[int64]model.Job{}}
h := &fakeHub{route: model.Route{AgentID: 7, ConnID: 70}}
s, _ := newService(st, h)
_, err := s.Start(context.Background(), svc, StartRequest{Trigger: tt.trigger, Ref: driftRepo + ":2.0.0"})
if tt.refused {
assert.Equal(t, model.ReasonDrifted, notUpdatableReason(t, err))
assert.Empty(t, st.created)
return
}
require.NoError(t, err)
assert.Len(t, st.created, 1)
})
}
}
func TestStart_PollAllowedWhenNotDrifted(t *testing.T) {
svc := driftedSvc()
svc.AwaitingInventory = true
st := &fakeStore{services: map[int64]model.Service{1: svc}, jobs: map[int64]model.Job{}}
s, _ := newService(st, &fakeHub{route: model.Route{AgentID: 7, ConnID: 70}})
_, err := s.Start(context.Background(), svc, StartRequest{Trigger: model.TriggerPoll})
require.NoError(t, err)
assert.Len(t, st.created, 1)
}
+7 -117
View File
@@ -20,16 +20,13 @@ type Store interface {
CreateJob(ctx context.Context, j model.NewJob) (model.Job, error) CreateJob(ctx context.Context, j model.NewJob) (model.Job, error)
FailJob(ctx context.Context, id int64, msg string, at time.Time) (bool, error) FailJob(ctx context.Context, id int64, msg string, at time.Time) (bool, error)
GetJob(ctx context.Context, id int64) (model.Job, error) GetJob(ctx context.Context, id int64) (model.Job, error)
GetImageVersion(ctx context.Context, id int64) (model.ImageVersion, error)
SetLastTagDigest(ctx context.Context, id int64, digest string) error
} }
// Hub routes jobs to agent connections. *hub.Hub satisfies it. // Hub routes jobs to agent connections. *hub.Hub satisfies it.
type Hub interface { type Hub interface {
// Route picks the connection for svc among agents speaking at least // Route picks the connection for svc: model.ErrAgentOffline or
// protocol version minProtocol: model.ErrAgentOffline or
// model.ErrAgentTooOld when there is none. // model.ErrAgentTooOld when there is none.
Route(svc model.Service, minProtocol int) (model.Route, error) Route(svc model.Service) (model.Route, error)
// StartJob sends job.start over exactly the routed connection. // StartJob sends job.start over exactly the routed connection.
// model.ErrAgentOffline if that connection is gone or was replaced. // model.ErrAgentOffline if that connection is gone or was replaced.
StartJob(r model.Route, jobID int64, start protocol.JobStart) error StartJob(r model.Route, jobID int64, start protocol.JobStart) error
@@ -115,88 +112,12 @@ func (s *Service) updatable(svc model.Service, explicitRef bool) error {
return nil return nil
} }
// Restorable returns nil or a *model.NotUpdatableError saying why the service
// cannot be restored to its desired version: it must have drifted, and the
// desired version is deployed like an explicit ref, so a service that follows
// no tag is fine. The service page calls it for every render.
func (s *Service) Restorable(svc model.Service) error {
if svc.RemovedAt != nil {
return &model.NotUpdatableError{Reason: model.ReasonRemoved}
}
if !svc.Drifted() {
return &model.NotUpdatableError{Reason: model.ReasonNotDrifted}
}
if err := s.updatable(svc, true); err != nil {
return err
}
if imageref.Parse(svc.DesiredRef).Repo == s.selfRepo {
return &model.NotUpdatableError{Reason: model.ReasonSelf}
}
return nil
}
// RestoreDesired starts a manual job that deploys the drifted service's
// desired ref again. The ref is read from the row, never from the caller. It
// returns model.ErrNotFound for an unknown service and otherwise what Start
// returns, with a *model.NotUpdatableError from Restorable if it cannot be
// restored.
//
// When the service had drifted to another digest of the same tag, that digest
// becomes last_tag_digest, so the poller does not deploy it again before the
// registry shows a newer one (the same rule as for any rollback). It is not
// written when last_tag_digest holds back a newer digest, that is when it is
// set and differs from the digest the desired ref names.
func (s *Service) RestoreDesired(ctx context.Context, serviceID int64, requestedBy string) (model.Job, error) {
svc, err := s.store.GetService(ctx, serviceID)
if err != nil {
return model.Job{}, err
}
if err := s.Restorable(svc); err != nil {
return model.Job{}, err
}
job, err := s.Start(ctx, svc, StartRequest{Trigger: model.TriggerManual, RequestedBy: requestedBy, Ref: svc.DesiredRef})
if err != nil {
return job, err
}
if d := driftedTagDigest(svc); d != "" && d != svc.LastTagDigest && holdsNothingNewer(svc) {
if err := s.store.SetLastTagDigest(ctx, svc.ID, d); err != nil {
s.log.Warn("last_tag_digest_not_set", "service_id", svc.ID, "error", err)
}
}
return job, nil
}
// holdsNothingNewer reports whether last_tag_digest may be overwritten by a
// restore: it is empty or equals the digest the desired ref names. Anything
// else is a digest the poller must keep holding back.
func holdsNothingNewer(svc model.Service) bool {
return svc.LastTagDigest == "" || svc.LastTagDigest == imageref.Parse(svc.DesiredRef).Digest
}
// driftedTagDigest returns the digest a drifted service runs when it runs the
// same tag as desired at another digest, else "".
func driftedTagDigest(svc model.Service) string {
tag := FollowedRef(svc.DesiredRef)
if tag == "" || tag != FollowedRef(svc.ActualRef) {
return ""
}
d := imageref.Parse(svc.ActualRef).Digest
if d == "" {
d = svc.ActualDigest
}
if !imageref.IsDigest(d) || d == imageref.Parse(svc.DesiredRef).Digest {
return ""
}
return d
}
// StartRequest says who asks for an update and what to deploy. // StartRequest says who asks for an update and what to deploy.
type StartRequest struct { type StartRequest struct {
Trigger model.JobTrigger Trigger model.JobTrigger
RequestedBy string // OIDC subject for manual jobs; "" otherwise RequestedBy string // OIDC subject for manual jobs; "" otherwise
// Ref is the ref to deploy ("repo:tag", "repo:tag@digest" or // Ref is the ref to deploy ("repo:tag"). Empty means the service's
// "repo@digest"). Empty means the service's followed tag, derived from // followed tag, derived from its actual ref.
// its actual ref.
Ref string Ref string
} }
@@ -210,39 +131,12 @@ func (s *Service) Trigger(ctx context.Context, serviceID int64, requestedBy stri
return s.Start(ctx, svc, StartRequest{Trigger: model.TriggerManual, RequestedBy: requestedBy}) return s.Start(ctx, svc, StartRequest{Trigger: model.TriggerManual, RequestedBy: requestedBy})
} }
// TriggerVersion starts a manual update of the service to exactly the image
// version versionID, which must be a version of the service's own repo; the
// ref is built from the stored version, never taken from the caller. It
// returns model.ErrNotFound for an unknown service, an unknown version or a
// version of another repo, and otherwise what Start returns. A version
// without a tag leaves the service following no tag.
func (s *Service) TriggerVersion(ctx context.Context, serviceID, versionID int64, requestedBy string) (model.Job, error) {
svc, err := s.store.GetService(ctx, serviceID)
if err != nil {
return model.Job{}, err
}
v, err := s.store.GetImageVersion(ctx, versionID)
if err != nil {
return model.Job{}, err
}
if v.Repo != svc.ImageRepo {
return model.Job{}, model.ErrNotFound
}
return s.Start(ctx, svc, StartRequest{Trigger: model.TriggerManual, RequestedBy: requestedBy, Ref: v.Ref()})
}
// Start starts an update of svc. It returns the job, which may already be // Start starts an update of svc. It returns the job, which may already be
// failed ("agent offline", "agent too old for updates", or "agent too old to // failed ("agent offline", "agent too old for updates"); a
// deploy a pinned version" when the ref has a digest or no tag and the agent
// speaks protocol version 2); a
// *model.NotUpdatableError when the service cannot be updated (no job // *model.NotUpdatableError when the service cannot be updated (no job
// created); the active job and model.ErrJobActive when one is queued or // created); the active job and model.ErrJobActive when one is queued or
// running. Poll and webhook jobs are refused for a drifted service; only a // running.
// person starts one then.
func (s *Service) Start(ctx context.Context, svc model.Service, req StartRequest) (model.Job, error) { func (s *Service) Start(ctx context.Context, svc model.Service, req StartRequest) (model.Job, error) {
if req.Trigger != model.TriggerManual && svc.Drifted() {
return model.Job{}, &model.NotUpdatableError{Reason: model.ReasonDrifted}
}
if err := s.updatable(svc, req.Ref != ""); err != nil { if err := s.updatable(svc, req.Ref != ""); err != nil {
return model.Job{}, err return model.Job{}, err
} }
@@ -251,11 +145,7 @@ func (s *Service) Start(ctx context.Context, svc model.Service, req StartRequest
ref = FollowedRef(svc.ActualRef) // updatable checked the tag ref = FollowedRef(svc.ActualRef) // updatable checked the tag
} }
minProtocol := protocol.MinJobVersion r, routeErr := s.hub.Route(svc)
if p := imageref.Parse(ref); p.Digest != "" || p.Tag == "" {
minProtocol = protocol.MinPinnedRefVersion
}
r, routeErr := s.hub.Route(svc, minProtocol)
nj := model.NewJob{ nj := model.NewJob{
ServiceID: svc.ID, FromRef: svc.ActualRef, ToRef: ref, ServiceID: svc.ID, FromRef: svc.ActualRef, ToRef: ref,
Trigger: req.Trigger, RequestedBy: req.RequestedBy, CreatedAt: s.now(), Trigger: req.Trigger, RequestedBy: req.RequestedBy, CreatedAt: s.now(),
+6 -165
View File
@@ -21,10 +21,8 @@ type fakeStore struct {
jobs map[int64]model.Job jobs map[int64]model.Job
created []model.NewJob created []model.NewJob
active *model.Job // returned with ErrJobActive when set active *model.Job // returned with ErrJobActive when set
versions map[int64]model.ImageVersion
revokedAgents map[int64]bool // CreateJob returns ErrAgentOffline for these revokedAgents map[int64]bool // CreateJob returns ErrAgentOffline for these
lastTag map[int64]string // SetLastTagDigest calls: service ID to digest
} }
func (f *fakeStore) GetService(_ context.Context, id int64) (model.Service, error) { func (f *fakeStore) GetService(_ context.Context, id int64) (model.Service, error) {
@@ -56,22 +54,6 @@ func (f *fakeStore) FailJob(_ context.Context, id int64, msg string, at time.Tim
return true, nil return true, nil
} }
func (f *fakeStore) GetImageVersion(_ context.Context, id int64) (model.ImageVersion, error) {
v, ok := f.versions[id]
if !ok {
return model.ImageVersion{}, model.ErrNotFound
}
return v, nil
}
func (f *fakeStore) SetLastTagDigest(_ context.Context, id int64, digest string) error {
if f.lastTag == nil {
f.lastTag = map[int64]string{}
}
f.lastTag[id] = digest
return nil
}
func (f *fakeStore) GetJob(_ context.Context, id int64) (model.Job, error) { return f.jobs[id], nil } func (f *fakeStore) GetJob(_ context.Context, id int64) (model.Job, error) { return f.jobs[id], nil }
type startCall struct { type startCall struct {
@@ -81,24 +63,13 @@ type startCall struct {
} }
type fakeHub struct { type fakeHub struct {
route model.Route route model.Route
routeErr error routeErr error
startErr error startErr error
starts []startCall starts []startCall
agentProtocol int // when set, Route fails with ErrAgentTooOld for a higher minProtocol
mins []int // minProtocol of every Route call
} }
func (h *fakeHub) Route(_ model.Service, minProtocol int) (model.Route, error) { func (h *fakeHub) Route(model.Service) (model.Route, error) { return h.route, h.routeErr }
h.mins = append(h.mins, minProtocol)
if h.agentProtocol != 0 && minProtocol > h.agentProtocol {
if minProtocol >= protocol.MinPinnedRefVersion {
return model.Route{}, model.ErrAgentTooOldForPinned
}
return model.Route{}, model.ErrAgentTooOld
}
return h.route, h.routeErr
}
func (h *fakeHub) StartJob(r model.Route, id int64, s protocol.JobStart) error { func (h *fakeHub) StartJob(r model.Route, id int64, s protocol.JobStart) error {
h.starts = append(h.starts, startCall{r, id, s}) h.starts = append(h.starts, startCall{r, id, s})
@@ -370,133 +341,3 @@ func TestFollowedRef(t *testing.T) {
assert.Equal(t, "localhost:5000/foo:latest", FollowedRef("localhost:5000/foo")) assert.Equal(t, "localhost:5000/foo:latest", FollowedRef("localhost:5000/foo"))
assert.Empty(t, FollowedRef("redis@sha256:abc")) assert.Empty(t, FollowedRef("redis@sha256:abc"))
} }
func TestStart_RoutesWithTheMinimumProtocolOfTheRef(t *testing.T) {
tests := []struct {
name string
ref string // "" means the followed tag
want int
}{
{"followed tag", "", protocol.MinJobVersion},
{"repo:tag", "x/y:2", protocol.MinJobVersion},
{"repo:tag@digest", "x/y:2@sha256:a", protocol.MinPinnedRefVersion},
{"repo@digest", "x/y@sha256:a", protocol.MinPinnedRefVersion},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
st := &fakeStore{services: map[int64]model.Service{1: composeSvc}, jobs: map[int64]model.Job{}}
h := &fakeHub{route: model.Route{AgentID: 7, ConnID: 70}}
s, _ := newService(st, h)
_, err := s.Start(context.Background(), composeSvc, StartRequest{Trigger: model.TriggerManual, Ref: tt.ref})
require.NoError(t, err)
assert.Equal(t, []int{tt.want}, h.mins)
})
}
}
func TestTriggerVersion_Ref(t *testing.T) {
const repo = "gitea.shcizo.se/shcizo/slusk"
noTag := model.Service{ID: 4, Kind: protocol.KindCompose, ComposeProject: "n", ComposeService: "n",
AgentID: ptr(int64(7)), ImageRepo: repo, ActualRef: repo + "@sha256:d1"}
svc := composeSvc
svc.ImageRepo = repo
tests := []struct {
name string
svc model.Service
version model.ImageVersion
wantRef string
wantMin int
}{
{"tag and digest", svc, model.ImageVersion{ID: 10, Repo: repo, Tag: "1.3.0", Digest: "sha256:old"}, repo + ":1.3.0@sha256:old", protocol.MinPinnedRefVersion},
{"tag only", svc, model.ImageVersion{ID: 10, Repo: repo, Tag: "main"}, repo + ":main", protocol.MinJobVersion},
{"digest only", svc, model.ImageVersion{ID: 10, Repo: repo, Digest: "sha256:old"}, repo + "@sha256:old", protocol.MinPinnedRefVersion},
{"service that follows no tag", noTag, model.ImageVersion{ID: 10, Repo: repo, Tag: "1.3.0", Digest: "sha256:old"}, repo + ":1.3.0@sha256:old", protocol.MinPinnedRefVersion},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
st := &fakeStore{
services: map[int64]model.Service{tt.svc.ID: tt.svc},
jobs: map[int64]model.Job{},
versions: map[int64]model.ImageVersion{10: tt.version},
}
h := &fakeHub{route: model.Route{AgentID: 7, ConnID: 70}}
s, _ := newService(st, h)
job, err := s.TriggerVersion(context.Background(), tt.svc.ID, 10, "sub-1")
require.NoError(t, err)
assert.Equal(t, model.JobQueued, job.Status)
require.Len(t, st.created, 1)
assert.Equal(t, model.TriggerManual, st.created[0].Trigger)
assert.Equal(t, "sub-1", st.created[0].RequestedBy)
assert.Equal(t, tt.wantRef, st.created[0].ToRef)
require.Len(t, h.starts, 1)
assert.Equal(t, tt.wantRef, h.starts[0].start.Ref)
assert.Equal(t, []int{tt.wantMin}, h.mins)
})
}
}
func TestTriggerVersion_NotFound(t *testing.T) {
const repo = "gitea.shcizo.se/shcizo/slusk"
svc := composeSvc
svc.ImageRepo = repo
tests := []struct {
name string
serviceID int64
versionID int64
}{
{"unknown service", 9, 10},
{"unknown version", 1, 99},
{"version of another repo", 1, 11},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
st := &fakeStore{
services: map[int64]model.Service{1: svc},
jobs: map[int64]model.Job{},
versions: map[int64]model.ImageVersion{
10: {ID: 10, Repo: repo, Tag: "1"},
11: {ID: 11, Repo: "evil/other", Tag: "1"},
},
}
h := &fakeHub{route: model.Route{AgentID: 7, ConnID: 70}}
s, pub := newService(st, h)
_, err := s.TriggerVersion(context.Background(), tt.serviceID, tt.versionID, "sub-1")
assert.ErrorIs(t, err, model.ErrNotFound)
assert.Empty(t, st.created, "no job")
assert.Empty(t, h.starts)
assert.Empty(t, pub.topics)
})
}
}
func TestTriggerVersion_RequiresPinnedProtocol(t *testing.T) {
const repo = "gitea.shcizo.se/shcizo/slusk"
svc := composeSvc
svc.ImageRepo = repo
st := &fakeStore{
services: map[int64]model.Service{1: svc},
jobs: map[int64]model.Job{},
versions: map[int64]model.ImageVersion{
10: {ID: 10, Repo: repo, Tag: "1.3.0", Digest: "sha256:old"},
11: {ID: 11, Repo: repo, Tag: "main"},
},
}
h := &fakeHub{route: model.Route{AgentID: 7, ConnID: 70}, agentProtocol: 2}
s, _ := newService(st, h)
pinned, err := s.TriggerVersion(context.Background(), 1, 10, "sub-1")
require.NoError(t, err)
assert.Equal(t, model.JobFailed, pinned.Status)
assert.Equal(t, "agent too old to deploy a pinned version (needs protocol 3)", pinned.Error)
assert.Empty(t, h.starts, "nothing sent to the old agent")
plain, err := s.TriggerVersion(context.Background(), 1, 11, "sub-1")
require.NoError(t, err)
assert.Equal(t, model.JobQueued, plain.Status)
}
-33
View File
@@ -15,20 +15,8 @@ var (
// ErrAgentTooOld means the only agent that could run the job speaks a // ErrAgentTooOld means the only agent that could run the job speaks a
// protocol version without jobs. // protocol version without jobs.
ErrAgentTooOld = errors.New("agent too old for updates") ErrAgentTooOld = errors.New("agent too old for updates")
// ErrAgentTooOldForPinned means the only agent that could run the job
// speaks protocol version 2, which cannot deploy a digest-pinned or
// tagless ref. errors.Is(err, ErrAgentTooOld) is true for it as well.
ErrAgentTooOldForPinned error = agentTooOldForPinned{}
) )
type agentTooOldForPinned struct{}
func (agentTooOldForPinned) Error() string {
return "agent too old to deploy a pinned version (needs protocol 3)"
}
func (agentTooOldForPinned) Is(target error) bool { return target == ErrAgentTooOld }
// NotUpdatableError says why a service cannot be updated. Reason is shown // NotUpdatableError says why a service cannot be updated. Reason is shown
// in the UI as is. // in the UI as is.
type NotUpdatableError struct { type NotUpdatableError struct {
@@ -44,27 +32,6 @@ const (
ReasonNoTag = "follows no tag" ReasonNoTag = "follows no tag"
ReasonRemoved = "service no longer exists" ReasonRemoved = "service no longer exists"
ReasonBuilt = "built from source (build:)" ReasonBuilt = "built from source (build:)"
ReasonDrifted = "drifted: restore or accept the running version first"
ReasonNotDrifted = "runs its desired version"
)
// MergeRefusedError says why two services cannot be merged. Reason is shown
// in the UI as is.
type MergeRefusedError struct {
Reason string
}
func (e *MergeRefusedError) Error() string { return "cannot merge: " + e.Reason }
// Reasons two services cannot be merged. ReasonSelf is reused for the
// updater itself.
const (
ReasonMergeSame = "a service cannot be merged into itself"
ReasonMergeLiveOld = "the old service still exists"
ReasonMergeGoneNew = "the new service no longer exists"
ReasonMergeKind = "only services of the same kind can be merged"
ReasonMergeContainer = "plain containers cannot be merged"
) )
// JobStatus is the state of an update job. // JobStatus is the state of an update job.
+6 -106
View File
@@ -8,15 +8,12 @@ import (
"strings" "strings"
"time" "time"
"github.com/shcizo/package-updater/internal/imageref"
"github.com/shcizo/package-updater/internal/protocol" "github.com/shcizo/package-updater/internal/protocol"
) )
// Sentinel errors returned by storage. // Sentinel errors returned by storage.
var ( var (
ErrNotFound = errors.New("not found") ErrNotFound = errors.New("not found")
// ErrConflict means the row already exists, or that it changed meanwhile
// (AcceptRunning, MergeService).
ErrConflict = errors.New("already exists") ErrConflict = errors.New("already exists")
ErrRevoked = errors.New("agent revoked") ErrRevoked = errors.New("agent revoked")
) )
@@ -59,24 +56,11 @@ type Service struct {
// AvailableRef is the newest ref a webhook or poll reported for this // AvailableRef is the newest ref a webhook or poll reported for this
// service; "" if none, or if a later job already deployed it. // service; "" if none, or if a later job already deployed it.
AvailableRef string AvailableRef string
AvailableDigest string // digest of AvailableRef when known (poll); "" from the webhook AvailableSource string // SourceWebhook; "" when AvailableRef is empty
AvailableSource string // SourceWebhook or SourcePoll; "" when AvailableRef is empty
AvailableAt *time.Time AvailableAt *time.Time
LastPollError string // last registry error for the followed ref; "" when fine FirstSeenAt time.Time
// LastTagDigest is the newest digest of the followed tag that was deployed UpdatedAt time.Time
// or started: set when the poller starts an auto job, when it sees the service RemovedAt *time.Time
// already running that digest, and when any plain tag job succeeds (the
// digest the agent pinned); "" if never. The poller does
// not start another job for the same digest, so a rollback sticks until the
// registry shows a different one.
LastTagDigest string
// AwaitingInventory is true from a successful job until the next inventory
// write for the row. Until then DesiredRef is newer than ActualRef without
// the service having drifted.
AwaitingInventory bool
FirstSeenAt time.Time
UpdatedAt time.Time
RemovedAt *time.Time
} }
// Update modes of a service. // Update modes of a service.
@@ -85,102 +69,18 @@ const (
ModeAuto = "auto" ModeAuto = "auto"
) )
// Sources of an available ref and of an ImageVersion. // Sources of an available ref.
const ( const (
SourceWebhook = "webhook" SourceWebhook = "webhook"
SourcePoll = "poll"
// SourceRunning is used by image_versions only: an agent reported the
// version as running.
SourceRunning = "running"
) )
// Available is a newer ref reported for a service. // Available is a newer ref reported for a service.
type Available struct { type Available struct {
Ref string Ref string
Digest string // "" when unknown (webhook) Source string // SourceWebhook
Source string // SourceWebhook or SourcePoll
At time.Time At time.Time
} }
// ImageVersion is one version of an image repo the master has seen. Tag and
// Digest are "" when the version has no tag or its digest is unknown; they
// are never both empty.
type ImageVersion struct {
ID int64
Repo string
Tag string
Digest string
Source string // SourceWebhook, SourcePoll or SourceRunning: the first source that saw it
FirstSeenAt time.Time
}
// Ref returns "repo:tag@digest", "repo:tag" or "repo@digest".
func (v ImageVersion) Ref() string {
return imageref.Ref{Repo: v.Repo, Tag: v.Tag, Digest: v.Digest}.String()
}
// Drifted reports whether the service runs something other than the version
// the last job deployed. A service the updater never touched, a removed one
// and one whose inventory has not caught up with the last job do not drift.
func (s Service) Drifted() bool {
return s.DesiredRef != "" && s.RemovedAt == nil && !s.AwaitingInventory &&
!RunsRef(s.DesiredRef, s.ActualRef, s.ActualDigest)
}
// RunsRef reports whether a service reporting actualRef and actualDigest runs
// the desired ref. The digest decides when both sides know one, so
// "repo:tag@D" is run by "repo:tag" reporting digest D, and by "repo@D".
// Otherwise the repo (compared exactly, without registry normalisation) and
// the followed tag must be equal.
func RunsRef(desired, actualRef, actualDigest string) bool {
d, a := imageref.Parse(desired), imageref.Parse(actualRef)
if d.Repo != a.Repo {
return false
}
ad := a.Digest
if ad == "" {
ad = actualDigest
}
if d.Digest != "" && ad != "" {
return d.Digest == ad
}
return followedTag(d) == followedTag(a)
}
// followedTag is the tag a ref follows: a bare image follows "latest" and a
// ref with a digest but no tag follows nothing.
func followedTag(r imageref.Ref) string {
if r.Tag == "" && r.Digest == "" {
return "latest"
}
return r.Tag
}
// RunningRef is the ref to record as the desired one when the running version
// is accepted: actualRef itself if it carries a digest, "repo:tag@digest" if
// only actualDigest is known (a bare image is "latest"), else actualRef.
func RunningRef(actualRef, actualDigest string) string {
r := imageref.Parse(actualRef)
if r.Digest != "" || !imageref.IsDigest(actualDigest) {
return actualRef
}
r.Tag, r.Digest = followedTag(r), actualDigest
return r.String()
}
// RunningRef returns the ref accepting the running version records as the
// desired one; see the function RunningRef. Templates post it as the guard of
// the accept form.
func (s Service) RunningRef() string { return RunningRef(s.ActualRef, s.ActualDigest) }
// Move says that the removed service FromID has probably moved to the live
// service ToID on another agent. The names are the agents' own; a revoked
// agent keeps its name.
type Move struct {
FromID, ToID int64
FromAgent, ToAgent string
}
// DisplayName is the human-facing name of the service. // DisplayName is the human-facing name of the service.
func (s Service) DisplayName() string { func (s Service) DisplayName() string {
switch s.Kind { switch s.Kind {
-97
View File
@@ -2,7 +2,6 @@ package model
import ( import (
"testing" "testing"
"time"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
@@ -41,102 +40,6 @@ func TestDisplayName(t *testing.T) {
} }
} }
func TestImageVersionRef(t *testing.T) {
cases := []struct {
name string
v ImageVersion
want string
}{
{"tag and digest", ImageVersion{Repo: "r/x", Tag: "1", Digest: "sha256:a"}, "r/x:1@sha256:a"},
{"tag only", ImageVersion{Repo: "r/x", Tag: "1"}, "r/x:1"},
{"digest only", ImageVersion{Repo: "r/x", Digest: "sha256:a"}, "r/x@sha256:a"},
{"registry port", ImageVersion{Repo: "localhost:5000/x", Tag: "1"}, "localhost:5000/x:1"},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
assert.Equal(t, c.want, c.v.Ref())
})
}
}
const (
dA = "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
dB = "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
)
func TestRunsRef(t *testing.T) {
cases := []struct {
name string
desired, actualRef, actualDig string
want bool
}{
{"same string", "repo:v1@" + dA, "repo:v1@" + dA, "", true},
{"tag and digest, actual reports it separately", "repo:tag@" + dA, "repo:tag", dA, true},
{"bare image reports latest digest", "postgres:latest@" + dA, "postgres", dA, true},
{"tagless actual with the same digest", "repo:tag@" + dA, "repo@" + dA, "", true},
{"same digest under another tag", "repo:v1@" + dA, "repo:v2@" + dA, "", true},
{"same digest, actual tag differs, digest reported", "repo:v1@" + dA, "repo:v2", dA, true},
{"same tag, actual digest unknown", "repo:tag@" + dA, "repo:tag", "", true},
{"desired without digest, same tag", "repo:tag", "repo:tag@" + dA, "", true},
{"bare desired, latest actual", "repo", "repo:latest", "", true},
{"registry with port", "localhost:5000/foo:v1@" + dA, "localhost:5000/foo:v1", dA, true},
{"registry with port, other digest", "localhost:5000/foo:v1@" + dA, "localhost:5000/foo:v1", dB, false},
{"same tag, other digest", "repo:tag@" + dA, "repo:tag", dB, false},
{"other digest in the ref", "repo:tag@" + dA, "repo:tag@" + dB, "", false},
{"other tag, digest unknown", "repo:v1@" + dA, "repo:v2", "", false},
{"tagless desired, tagged actual, digest unknown", "repo@" + dA, "repo:v1", "", false},
{"other repo", "repo:tag@" + dA, "other:tag", dA, false},
{"no docker.io normalisation", "nginx:1@" + dA, "docker.io/library/nginx:1", dA, false},
{"case-sensitive repo", "Repo:1@" + dA, "repo:1", dA, false},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
assert.Equal(t, c.want, RunsRef(c.desired, c.actualRef, c.actualDig))
})
}
}
func TestDrifted(t *testing.T) {
now := time.Now()
drifted := Service{DesiredRef: "repo:tag@" + dA, ActualRef: "repo:tag", ActualDigest: dB}
cases := []struct {
name string
mutate func(*Service)
want bool
}{
{"runs another digest", func(*Service) {}, true},
{"never updated", func(s *Service) { s.DesiredRef = "" }, false},
{"removed", func(s *Service) { s.RemovedAt = &now }, false},
{"awaiting inventory", func(s *Service) { s.AwaitingInventory = true }, false},
{"runs the desired digest", func(s *Service) { s.ActualDigest = dA }, false},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
svc := drifted
c.mutate(&svc)
assert.Equal(t, c.want, svc.Drifted())
})
}
}
func TestRunningRef(t *testing.T) {
cases := []struct {
name, actualRef, actualDigest, want string
}{
{"ref with digest is kept", "repo:v1@" + dB, dA, "repo:v1@" + dB},
{"tag plus reported digest", "repo:v1", dA, "repo:v1@" + dA},
{"bare image becomes latest", "postgres", dA, "postgres:latest@" + dA},
{"registry with port", "localhost:5000/foo:v1", dA, "localhost:5000/foo:v1@" + dA},
{"no digest known", "repo:v1", "", "repo:v1"},
{"malformed digest", "repo:v1", "sha256:abc", "repo:v1"},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
assert.Equal(t, c.want, RunningRef(c.actualRef, c.actualDigest))
})
}
}
func TestJob_Active(t *testing.T) { func TestJob_Active(t *testing.T) {
cases := []struct { cases := []struct {
status JobStatus status JobStatus
-320
View File
@@ -1,320 +0,0 @@
// Package poller checks the registry for new digests of the tags the
// services follow. A new digest becomes the service's available version, and
// a service in auto mode gets an update job.
package poller
import (
"context"
"errors"
"log/slog"
"strings"
"sync"
"time"
"github.com/shcizo/package-updater/internal/master/events"
"github.com/shcizo/package-updater/internal/master/jobs"
"github.com/shcizo/package-updater/internal/master/model"
)
// lookupTimeout bounds one registry lookup.
const lookupTimeout = 30 * time.Second
// maxErrorBytes caps the registry error stored in services.last_poll_error.
const maxErrorBytes = 500
// Store is the storage the poller needs. *store.Store satisfies it.
type Store interface {
ListServices(ctx context.Context) ([]model.Service, error)
GetService(ctx context.Context, id int64) (model.Service, error)
SetAvailable(ctx context.Context, id int64, a model.Available) error
SetLastTagDigest(ctx context.Context, id int64, digest string) error
SetPollError(ctx context.Context, ids []int64, msg string) error
}
// Jobs starts updates. *jobs.Service satisfies it.
type Jobs interface {
Start(ctx context.Context, svc model.Service, req jobs.StartRequest) (model.Job, error)
}
// Registry looks up the digest a "repo:tag" ref points at. *registry.Client
// satisfies it.
type Registry interface {
Digest(ctx context.Context, ref string) (string, error)
}
// Publisher receives change signals. *events.Broker satisfies it.
type Publisher interface{ Publish(t events.Topic) }
// Config holds the poller settings.
type Config struct {
Interval time.Duration // time between the end of a cycle and the start of the next (POLL_INTERVAL)
FirstDelay time.Duration // wait before the first cycle; 1 minute in production
Workers int // registry lookups in flight at once; 4 in production
}
// Poller runs polling cycles.
type Poller struct {
store Store
jobs Jobs
reg Registry
pub Publisher
cfg Config
log *slog.Logger
now func() time.Time
}
// New returns a Poller.
func New(st Store, j Jobs, reg Registry, pub Publisher, cfg Config, log *slog.Logger) *Poller {
if cfg.Workers < 1 {
cfg.Workers = 1
}
return &Poller{store: st, jobs: j, reg: reg, pub: pub, cfg: cfg, log: log, now: time.Now}
}
// SkipReason says why svc is not polled: "follows no tag", "built from
// source", "no registry digest", or "" when it is polled.
func SkipReason(svc model.Service) string {
switch {
case jobs.FollowedRef(svc.ActualRef) == "":
return model.ReasonNoTag
case svc.Built:
return "built from source"
case svc.ActualDigest == "":
return "no registry digest"
}
return ""
}
// Run polls until ctx is done. The first cycle starts Config.FirstDelay after
// the call and each following one Config.Interval after the previous cycle
// ended, so cycles never overlap. It blocks.
func (p *Poller) Run(ctx context.Context) {
timer := time.NewTimer(p.cfg.FirstDelay)
defer timer.Stop()
for {
select {
case <-ctx.Done():
return
case <-timer.C:
}
p.Cycle(ctx)
timer.Reset(p.cfg.Interval)
}
}
// lookup is the registry's answer for one followed ref.
type lookup struct {
ref string
digest string
err error
}
// cycleStats is what one cycle did, for the poll_cycle log line.
type cycleStats struct {
refs, ok, failed, suggested, jobs, skipped int
wrote bool
}
// Cycle runs one poll over all services. It looks up every distinct followed
// ref once, with at most Config.Workers lookups at a time, then records the
// results. Writing results and starting jobs is not cut short by ctx ending
// during the lookups, so a job that was marked is always started.
func (p *Poller) Cycle(ctx context.Context) {
began := p.now()
svcs, err := p.store.ListServices(ctx)
if err != nil {
p.log.Error("poll_list_failed", "error", err)
return
}
var stats cycleStats
byRef := map[string][]model.Service{}
var refs []string
for _, svc := range svcs {
if svc.RemovedAt != nil {
continue
}
if SkipReason(svc) != "" {
stats.skipped++
continue
}
ref := jobs.FollowedRef(svc.ActualRef)
if _, seen := byRef[ref]; !seen {
refs = append(refs, ref)
}
byRef[ref] = append(byRef[ref], svc)
}
stats.refs = len(refs)
write := context.WithoutCancel(ctx)
for _, l := range p.lookupAll(ctx, refs) {
if l.err != nil && ctx.Err() != nil {
continue // cancelled mid-lookup: not the registry's fault
}
p.record(write, l, byRef[l.ref], &stats)
}
if stats.wrote {
p.pub.Publish(events.Services)
}
p.log.Info("poll_cycle", "refs", stats.refs, "ok", stats.ok, "failed", stats.failed,
"suggested", stats.suggested, "jobs", stats.jobs, "skipped", stats.skipped,
"duration_ms", p.now().Sub(began).Milliseconds())
}
// lookupAll asks the registry for every ref, Config.Workers at a time. It
// returns the answers in ref order and stops starting new lookups once ctx
// is done; refs never looked up are missing from the result.
func (p *Poller) lookupAll(ctx context.Context, refs []string) []lookup {
results := make([]*lookup, len(refs))
next := make(chan int)
var wg sync.WaitGroup
for w := 0; w < p.cfg.Workers; w++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := range next {
lctx, cancel := context.WithTimeout(ctx, lookupTimeout)
d, err := p.reg.Digest(lctx, refs[i])
cancel()
results[i] = &lookup{ref: refs[i], digest: d, err: err}
}
}()
}
feed:
for i := range refs {
select {
case next <- i:
case <-ctx.Done():
break feed
}
}
close(next)
wg.Wait()
var out []lookup
for _, r := range results {
if r != nil {
out = append(out, *r)
}
}
return out
}
// record stores one lookup result for the services that follow l.ref.
func (p *Poller) record(ctx context.Context, l lookup, svcs []model.Service, stats *cycleStats) {
if l.err != nil {
stats.failed++
msg := truncate(l.err.Error())
p.log.Warn("poll_failed", "ref", l.ref, "error", msg)
p.setPollError(ctx, svcs, msg, stats, func(s model.Service) bool { return s.LastPollError != msg })
return
}
stats.ok++
p.setPollError(ctx, svcs, "", stats, func(s model.Service) bool { return s.LastPollError != "" })
for _, svc := range svcs {
if !p.suggest(ctx, svc, l, stats) {
continue
}
if autoWanted(svc, l.ref, l.digest) {
p.startAuto(ctx, svc.ID, l.ref, l.digest, stats)
}
}
}
// suggest marks l.digest as the available version of svc unless the service
// already runs it or already has exactly that suggestion. A service that
// runs l.digest also gets it as last_tag_digest. It returns false only when
// writing the suggestion failed.
func (p *Poller) suggest(ctx context.Context, svc model.Service, l lookup, stats *cycleStats) bool {
if svc.ActualDigest == l.digest && svc.LastTagDigest != l.digest {
// Seed last_tag_digest for a service that was not updated by a tag job
// (it predates the column or was updated by hand), so that a later
// rollback is not undone.
if err := p.store.SetLastTagDigest(ctx, svc.ID, l.digest); err != nil {
p.log.Error("poll_set_last_tag_digest_failed", "service_id", svc.ID, "error", err)
}
}
if svc.ActualDigest == l.digest || (svc.AvailableRef == l.ref && svc.AvailableDigest == l.digest) {
return true
}
err := p.store.SetAvailable(ctx, svc.ID, model.Available{Ref: l.ref, Digest: l.digest, Source: model.SourcePoll, At: p.now()})
if err != nil {
p.log.Error("poll_set_available_failed", "service_id", svc.ID, "ref", l.ref, "error", err)
return false
}
stats.wrote = true
stats.suggested++
return true
}
// autoWanted reports whether an update of svc to ref at digest should be
// started automatically: the service is in auto mode, not removed, does not
// already run that digest, was not already updated automatically to it
// (otherwise a rollback to an older version would be undone at the next
// cycle), and its last job did not already deploy it (desired_ref holds the
// digest until the agent reports the new inventory).
func autoWanted(svc model.Service, ref, digest string) bool {
return svc.RemovedAt == nil &&
svc.UpdateMode == model.ModeAuto &&
svc.ActualDigest != digest &&
svc.LastTagDigest != digest &&
svc.DesiredRef != ref+"@"+digest
}
// startAuto starts the update job of an auto service to ref and remembers
// digest as services.last_tag_digest, so the same digest is not started
// again, for instance after the user rolled back. It re-reads the service
// first, because a cycle can run for a long time and the user may have
// switched the service to manual meanwhile. A service that already has a
// job or cannot be updated is not an error for the cycle, and last_tag_digest
// is left alone so the next cycle tries again.
func (p *Poller) startAuto(ctx context.Context, id int64, ref, digest string, stats *cycleStats) {
svc, err := p.store.GetService(ctx, id)
if err != nil {
p.log.Warn("poll_get_service_failed", "service_id", id, "error", err)
return
}
if !autoWanted(svc, ref, digest) {
return
}
_, err = p.jobs.Start(ctx, svc, jobs.StartRequest{Trigger: model.TriggerPoll, Ref: ref})
var notUpdatable *model.NotUpdatableError
switch {
case err == nil:
stats.jobs++
if err := p.store.SetLastTagDigest(ctx, id, digest); err != nil {
p.log.Error("poll_set_last_tag_digest_failed", "service_id", id, "error", err)
}
case errors.Is(err, model.ErrJobActive), errors.As(err, &notUpdatable):
p.log.Debug("poll_job_not_started", "service_id", id, "ref", ref, "reason", err.Error())
default:
p.log.Warn("poll_job_failed", "service_id", id, "ref", ref, "error", err)
}
}
// setPollError sets msg as last_poll_error on the services for which
// changed returns true; with none, it writes nothing.
func (p *Poller) setPollError(ctx context.Context, svcs []model.Service, msg string, stats *cycleStats, changed func(model.Service) bool) {
var ids []int64
for _, svc := range svcs {
if changed(svc) {
ids = append(ids, svc.ID)
}
}
if len(ids) == 0 {
return
}
if err := p.store.SetPollError(ctx, ids, msg); err != nil {
p.log.Error("poll_set_error_failed", "error", err)
return
}
stats.wrote = true
}
// truncate cuts msg to maxErrorBytes without leaving a broken UTF-8
// sequence, and drops NUL bytes, which Postgres text cannot hold.
func truncate(msg string) string {
msg = strings.ReplaceAll(msg, "\x00", "")
if len(msg) > maxErrorBytes {
msg = strings.ToValidUTF8(msg[:maxErrorBytes], "")
}
return msg
}
-651
View File
@@ -1,651 +0,0 @@
package poller
import (
"context"
"errors"
"fmt"
"io"
"log/slog"
"sort"
"strings"
"sync"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/shcizo/package-updater/internal/master/events"
"github.com/shcizo/package-updater/internal/master/jobs"
"github.com/shcizo/package-updater/internal/master/model"
)
type availableCall struct {
id int64
a model.Available
}
type pollErrorCall struct {
ids []int64
msg string
}
type lastTagCall struct {
id int64
digest string
}
type fakeStore struct {
mu sync.Mutex
services []model.Service
fresh map[int64]model.Service // what GetService returns instead of the listed row
lastTag []lastTagCall
listErr error
available []availableCall
pollErrors []pollErrorCall
ctxErrs []error // ctx.Err() seen by every write
}
func (f *fakeStore) ListServices(context.Context) ([]model.Service, error) {
return f.services, f.listErr
}
func (f *fakeStore) GetService(_ context.Context, id int64) (model.Service, error) {
f.mu.Lock()
defer f.mu.Unlock()
if s, ok := f.fresh[id]; ok {
return s, nil
}
for _, s := range f.services {
if s.ID == id {
return s, nil
}
}
return model.Service{}, model.ErrNotFound
}
// SetLastTagDigest records the call and, like the database, changes the
// row the next cycle lists.
func (f *fakeStore) SetLastTagDigest(_ context.Context, id int64, digest string) error {
f.mu.Lock()
defer f.mu.Unlock()
f.lastTag = append(f.lastTag, lastTagCall{id, digest})
for i := range f.services {
if f.services[i].ID == id {
f.services[i].LastTagDigest = digest
}
}
return nil
}
func (f *fakeStore) SetAvailable(ctx context.Context, id int64, a model.Available) error {
f.mu.Lock()
defer f.mu.Unlock()
f.ctxErrs = append(f.ctxErrs, ctx.Err())
f.available = append(f.available, availableCall{id, a})
return nil
}
func (f *fakeStore) SetPollError(ctx context.Context, ids []int64, msg string) error {
f.mu.Lock()
defer f.mu.Unlock()
f.ctxErrs = append(f.ctxErrs, ctx.Err())
f.pollErrors = append(f.pollErrors, pollErrorCall{ids, msg})
return nil
}
type startCall struct {
svcID int64
req jobs.StartRequest
}
type fakeJobs struct {
mu sync.Mutex
starts []startCall
errFor map[int64]error // by service ID
ctxErrs []error
}
func (f *fakeJobs) Start(ctx context.Context, svc model.Service, req jobs.StartRequest) (model.Job, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.ctxErrs = append(f.ctxErrs, ctx.Err())
f.starts = append(f.starts, startCall{svc.ID, req})
return model.Job{ID: int64(len(f.starts))}, f.errFor[svc.ID]
}
type fakeRegistry struct {
mu sync.Mutex
digests map[string]string // ref -> digest
errs map[string]error // ref -> error
calls []string
inFlight int
maxSeen int
delay time.Duration
started chan struct{} // receives a value whenever a lookup starts, if set
release chan struct{} // lookups wait for it to be closed, if set
onResult func() // called after a lookup computed its answer
}
func (f *fakeRegistry) Digest(ctx context.Context, ref string) (string, error) {
f.mu.Lock()
f.calls = append(f.calls, ref)
f.inFlight++
if f.inFlight > f.maxSeen {
f.maxSeen = f.inFlight
}
f.mu.Unlock()
defer func() {
f.mu.Lock()
f.inFlight--
f.mu.Unlock()
}()
if f.started != nil {
f.started <- struct{}{}
}
if f.release != nil {
select {
case <-f.release:
case <-ctx.Done():
return "", ctx.Err()
}
}
if f.delay > 0 {
time.Sleep(f.delay)
}
if f.onResult != nil {
f.onResult()
}
if err := f.errs[ref]; err != nil {
return "", err
}
return f.digests[ref], nil
}
func (f *fakeRegistry) sortedCalls() []string {
f.mu.Lock()
defer f.mu.Unlock()
out := append([]string(nil), f.calls...)
sort.Strings(out)
return out
}
type fakePub struct {
mu sync.Mutex
topics []events.Topic
}
func (p *fakePub) Publish(t events.Topic) {
p.mu.Lock()
defer p.mu.Unlock()
p.topics = append(p.topics, t)
}
const (
repoA = "gitea.shcizo.se/shcizo/slusk"
repoB = "gitea/gitea"
)
// svc builds a polled service whose running version has digest sha256:old.
func svc(id int64, actualRef string) model.Service {
return model.Service{ID: id, Kind: "compose", ActualRef: actualRef, ActualDigest: "sha256:old", UpdateMode: model.ModeManual}
}
func newPoller(st *fakeStore, j *fakeJobs, reg *fakeRegistry, cfg Config) (*Poller, *fakePub) {
pub := &fakePub{}
if cfg.Workers == 0 {
cfg.Workers = 4
}
p := New(st, j, reg, pub, cfg, slog.New(slog.NewTextHandler(io.Discard, nil)))
p.now = func() time.Time { return time.Date(2026, 10, 13, 12, 0, 0, 0, time.UTC) }
return p, pub
}
func TestSkipReason(t *testing.T) {
tests := []struct {
name string
svc model.Service
want string
}{
{"tag and digest", model.Service{ActualRef: "r:1", ActualDigest: "sha256:a"}, ""},
{"bare image follows latest", model.Service{ActualRef: "r", ActualDigest: "sha256:a"}, ""},
{"digest only follows no tag", model.Service{ActualRef: "r@sha256:a", ActualDigest: "sha256:a"}, model.ReasonNoTag},
{"built", model.Service{ActualRef: "r:1", ActualDigest: "sha256:a", Built: true}, "built from source"},
{"no digest", model.Service{ActualRef: "r:1"}, "no registry digest"},
{"self and container are polled", model.Service{ActualRef: "r:1", ActualDigest: "sha256:a", Self: true, Kind: "container"}, ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, SkipReason(tt.svc))
})
}
}
func TestOneHeadPerDistinctRef(t *testing.T) {
st := &fakeStore{services: []model.Service{
svc(1, repoA+":1.4"),
svc(2, repoA+":1.4@sha256:old"),
svc(3, repoB),
svc(4, repoB+":latest"),
svc(5, repoB+":1.27"),
}}
reg := &fakeRegistry{digests: map[string]string{}}
p, _ := newPoller(st, &fakeJobs{}, reg, Config{})
p.Cycle(context.Background())
assert.Equal(t, []string{repoA + ":1.4", repoB + ":1.27", repoB + ":latest"}, reg.sortedCalls())
}
func TestSkipped(t *testing.T) {
removed := svc(4, "r:1")
removed.RemovedAt = ptr(time.Now())
st := &fakeStore{services: []model.Service{
{ID: 1, ActualRef: "r@sha256:a", ActualDigest: "sha256:a"},
{ID: 2, ActualRef: "r:1", ActualDigest: "sha256:a", Built: true},
{ID: 3, ActualRef: "r:1"},
removed,
}}
reg := &fakeRegistry{}
j := &fakeJobs{}
p, pub := newPoller(st, j, reg, Config{})
p.Cycle(context.Background())
assert.Empty(t, reg.calls, "no lookups")
assert.Empty(t, st.available)
assert.Empty(t, st.pollErrors)
assert.Empty(t, j.starts)
assert.Empty(t, pub.topics)
}
func TestNewDigest(t *testing.T) {
ref := repoA + ":1.4"
tests := []struct {
name string
mode string
wantStart bool
}{
{"auto gets one poll job to repo:tag", model.ModeAuto, true},
{"manual is only marked", model.ModeManual, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
s := svc(1, ref+"@sha256:old")
s.UpdateMode = tt.mode
st := &fakeStore{services: []model.Service{s}}
reg := &fakeRegistry{digests: map[string]string{ref: "sha256:new"}}
j := &fakeJobs{}
p, pub := newPoller(st, j, reg, Config{})
p.Cycle(context.Background())
require.Len(t, st.available, 1)
assert.Equal(t, int64(1), st.available[0].id)
assert.Equal(t, model.Available{Ref: ref, Digest: "sha256:new", Source: model.SourcePoll,
At: time.Date(2026, 10, 13, 12, 0, 0, 0, time.UTC)}, st.available[0].a)
if tt.wantStart {
assert.Equal(t, []startCall{{1, jobs.StartRequest{Trigger: model.TriggerPoll, Ref: ref}}}, j.starts)
assert.Equal(t, []lastTagCall{{1, "sha256:new"}}, st.lastTag)
} else {
assert.Empty(t, j.starts)
assert.Empty(t, st.lastTag)
}
assert.Equal(t, []events.Topic{events.Services}, pub.topics)
})
}
}
func TestSameSuggestionIsIdempotent(t *testing.T) {
ref := repoA + ":1.4"
running := svc(1, ref+"@sha256:old")
running.UpdateMode = model.ModeAuto
running.ActualDigest = "sha256:new" // already on the registry's digest
suggested := svc(2, ref)
suggested.UpdateMode = model.ModeAuto
suggested.AvailableRef, suggested.AvailableDigest = ref, "sha256:new"
suggested.LastTagDigest = "sha256:new" // its job was started in an earlier cycle
staleSuggestion := svc(3, ref)
staleSuggestion.AvailableRef, staleSuggestion.AvailableDigest = ref, "sha256:older"
st := &fakeStore{services: []model.Service{running, suggested, staleSuggestion}}
reg := &fakeRegistry{digests: map[string]string{ref: "sha256:new"}}
j := &fakeJobs{}
p, _ := newPoller(st, j, reg, Config{})
p.Cycle(context.Background())
require.Len(t, st.available, 1, "only the service with an older suggestion is updated")
assert.Equal(t, int64(3), st.available[0].id)
assert.Empty(t, j.starts, "no new job for an already suggested digest")
}
func TestStartErrorsDoNotAbortCycle(t *testing.T) {
ref := repoA + ":1.4"
var svcs []model.Service
for id := int64(1); id <= 3; id++ {
s := svc(id, ref)
s.UpdateMode = model.ModeAuto
svcs = append(svcs, s)
}
st := &fakeStore{services: svcs}
reg := &fakeRegistry{digests: map[string]string{ref: "sha256:new"}}
j := &fakeJobs{errFor: map[int64]error{
1: model.ErrJobActive,
2: &model.NotUpdatableError{Reason: model.ReasonSelf},
3: errors.New("database down"),
}}
p, _ := newPoller(st, j, reg, Config{})
p.Cycle(context.Background())
assert.Len(t, st.available, 3)
assert.Len(t, j.starts, 3)
assert.Empty(t, st.lastTag, "a job that was not created is retried next cycle")
}
func TestRollbackOfAutoServiceIsNotUndone(t *testing.T) {
ref := repoA + ":main"
// Updated to sha256:new automatically, then rolled back to sha256:old by hand.
s := svc(1, ref+"@sha256:old")
s.UpdateMode = model.ModeAuto
s.LastTagDigest = "sha256:new"
s.DesiredRef = ref + "@sha256:old"
st := &fakeStore{services: []model.Service{s}}
reg := &fakeRegistry{digests: map[string]string{ref: "sha256:new"}}
j := &fakeJobs{}
p, _ := newPoller(st, j, reg, Config{})
p.Cycle(context.Background())
p.Cycle(context.Background())
assert.Empty(t, j.starts, "same registry digest: the rollback stays")
assert.Equal(t, "sha256:new", st.available[0].a.Digest, "the newer version is still offered")
reg.digests[ref] = "sha256:newer"
p.Cycle(context.Background())
assert.Equal(t, []startCall{{1, jobs.StartRequest{Trigger: model.TriggerPoll, Ref: ref}}}, j.starts, "a new digest moves it again")
assert.Equal(t, []lastTagCall{{1, "sha256:newer"}}, st.lastTag)
}
func TestActiveJobIsRetriedNextCycle(t *testing.T) {
ref := repoA + ":1.4"
s := svc(1, ref)
s.UpdateMode = model.ModeAuto
st := &fakeStore{services: []model.Service{s}}
reg := &fakeRegistry{digests: map[string]string{ref: "sha256:new"}}
j := &fakeJobs{errFor: map[int64]error{1: model.ErrJobActive}}
p, _ := newPoller(st, j, reg, Config{})
p.Cycle(context.Background())
require.Len(t, j.starts, 1)
assert.Empty(t, st.lastTag)
j.errFor = nil
p.Cycle(context.Background())
assert.Len(t, j.starts, 2)
assert.Equal(t, []lastTagCall{{1, "sha256:new"}}, st.lastTag)
p.Cycle(context.Background())
assert.Len(t, j.starts, 2, "started once the job exists")
}
func TestJobThatAlreadyDeployedTheDigestIsNotStartedAgain(t *testing.T) {
ref := repoA + ":1.4"
// FinishJob set desired_ref; the agent has not reported the new digest yet.
s := svc(1, ref+"@sha256:old")
s.UpdateMode = model.ModeAuto
s.DesiredRef = ref + "@sha256:new"
st := &fakeStore{services: []model.Service{s}}
reg := &fakeRegistry{digests: map[string]string{ref: "sha256:new"}}
j := &fakeJobs{}
p, _ := newPoller(st, j, reg, Config{})
p.Cycle(context.Background())
assert.Empty(t, j.starts)
}
func TestServiceSwitchedToManualDuringTheCycleIsNotStarted(t *testing.T) {
ref := repoA + ":1.4"
s := svc(1, ref)
s.UpdateMode = model.ModeAuto
manual := s
manual.UpdateMode = model.ModeManual
st := &fakeStore{services: []model.Service{s}, fresh: map[int64]model.Service{1: manual}}
reg := &fakeRegistry{digests: map[string]string{ref: "sha256:new"}}
j := &fakeJobs{}
p, _ := newPoller(st, j, reg, Config{})
p.Cycle(context.Background())
assert.Len(t, st.available, 1, "still suggested")
assert.Empty(t, j.starts)
assert.Empty(t, st.lastTag)
}
func TestErrorIsPerRef(t *testing.T) {
bad, good := repoA+":1.4", repoB+":1.27"
badOld := svc(1, bad)
badSame := svc(2, bad)
badSame.LastPollError = "head: 401 unauthorized"
goodSvc := svc(3, good)
st := &fakeStore{services: []model.Service{badOld, badSame, goodSvc}}
reg := &fakeRegistry{
errs: map[string]error{bad: errors.New("head: 401 unauthorized")},
digests: map[string]string{good: "sha256:new"},
}
p, pub := newPoller(st, &fakeJobs{}, reg, Config{})
p.Cycle(context.Background())
assert.Equal(t, []pollErrorCall{{[]int64{1}, "head: 401 unauthorized"}}, st.pollErrors,
"only the service whose stored error differs is written")
require.Len(t, st.available, 1, "the other ref is still processed")
assert.Equal(t, int64(3), st.available[0].id)
assert.Equal(t, []events.Topic{events.Services}, pub.topics)
}
func TestErrorTextIsTruncated(t *testing.T) {
ref := repoA + ":1.4"
st := &fakeStore{services: []model.Service{svc(1, ref)}}
reg := &fakeRegistry{errs: map[string]error{ref: errors.New(strings.Repeat("é", 400) + "\x00")}}
p, _ := newPoller(st, &fakeJobs{}, reg, Config{})
p.Cycle(context.Background())
require.Len(t, st.pollErrors, 1)
msg := st.pollErrors[0].msg
assert.LessOrEqual(t, len(msg), maxErrorBytes)
assert.Equal(t, strings.Repeat("é", 250), msg, "cut on a character boundary, NUL removed")
}
func TestErrorClearedOnSuccess(t *testing.T) {
ref := repoA + ":1.4"
failing := svc(1, ref)
failing.LastPollError = "head: boom"
fine := svc(2, ref)
failing.ActualDigest, fine.ActualDigest = "sha256:new", "sha256:new"
st := &fakeStore{services: []model.Service{failing, fine}}
reg := &fakeRegistry{digests: map[string]string{ref: "sha256:new"}}
p, pub := newPoller(st, &fakeJobs{}, reg, Config{})
p.Cycle(context.Background())
assert.Equal(t, []pollErrorCall{{[]int64{1}, ""}}, st.pollErrors)
assert.Empty(t, st.available)
assert.Equal(t, []events.Topic{events.Services}, pub.topics)
}
func TestNothingWrittenNothingPublished(t *testing.T) {
ref := repoA + ":1.4"
s := svc(1, ref)
s.ActualDigest = "sha256:new"
st := &fakeStore{services: []model.Service{s}}
reg := &fakeRegistry{digests: map[string]string{ref: "sha256:new"}}
p, pub := newPoller(st, &fakeJobs{}, reg, Config{})
p.Cycle(context.Background())
assert.Empty(t, st.pollErrors)
assert.Empty(t, st.available)
assert.Empty(t, pub.topics)
}
func TestListErrorEndsTheCycle(t *testing.T) {
st := &fakeStore{listErr: errors.New("db down")}
reg := &fakeRegistry{}
p, _ := newPoller(st, &fakeJobs{}, reg, Config{})
p.Cycle(context.Background())
assert.Empty(t, reg.calls)
}
func TestConcurrencyLimit(t *testing.T) {
const workers = 3
var svcs []model.Service
for i := 0; i < 8; i++ {
svcs = append(svcs, svc(int64(i+1), fmt.Sprintf("%s%d:1", repoA, i)))
}
st := &fakeStore{services: svcs}
reg := &fakeRegistry{release: make(chan struct{}), started: make(chan struct{}, 8)}
p, _ := newPoller(st, &fakeJobs{}, reg, Config{Workers: workers})
done := make(chan struct{})
go func() { p.Cycle(context.Background()); close(done) }()
for i := 0; i < workers; i++ {
select {
case <-reg.started:
case <-time.After(2 * time.Second):
t.Fatalf("only %d lookups started, want %d", i, workers)
}
}
select {
case <-reg.started:
t.Fatal("more lookups in flight than Workers")
case <-time.After(100 * time.Millisecond):
}
close(reg.release)
select {
case <-done:
case <-time.After(2 * time.Second):
t.Fatal("Cycle did not finish")
}
assert.Len(t, reg.calls, 8)
}
func TestCycleStopsOnCancel(t *testing.T) {
var svcs []model.Service
for i := 0; i < 6; i++ {
svcs = append(svcs, svc(int64(i+1), fmt.Sprintf("%s%d:1", repoA, i)))
}
st := &fakeStore{services: svcs}
reg := &fakeRegistry{release: make(chan struct{}), started: make(chan struct{}, 6)}
p, pub := newPoller(st, &fakeJobs{}, reg, Config{Workers: 2})
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
go func() { p.Cycle(ctx); close(done) }()
<-reg.started
<-reg.started
cancel()
select {
case <-done:
case <-time.After(2 * time.Second):
t.Fatal("Cycle did not return after cancel")
}
assert.Len(t, reg.calls, 2, "no new lookups after cancel")
assert.Empty(t, st.pollErrors, "cancelled lookups are not recorded as registry errors")
assert.Empty(t, st.available)
assert.Empty(t, pub.topics)
}
func TestMarkedJobsAreStartedEvenIfCancelledDuringTheCycle(t *testing.T) {
ref := repoA + ":1.4"
s := svc(1, ref)
s.UpdateMode = model.ModeAuto
st := &fakeStore{services: []model.Service{s}}
ctx, cancel := context.WithCancel(context.Background())
reg := &fakeRegistry{digests: map[string]string{ref: "sha256:new"}, onResult: cancel}
j := &fakeJobs{}
p, _ := newPoller(st, j, reg, Config{})
p.Cycle(ctx)
require.Len(t, st.available, 1)
require.Len(t, j.starts, 1)
assert.NoError(t, st.ctxErrs[0], "writes do not see the cancelled context")
assert.NoError(t, j.ctxErrs[0])
}
func TestRun(t *testing.T) {
ref := repoA + ":1.4"
st := &fakeStore{services: []model.Service{svc(1, ref)}}
reg := &fakeRegistry{digests: map[string]string{ref: "sha256:old"}, delay: 30 * time.Millisecond}
p, _ := newPoller(st, &fakeJobs{}, reg, Config{Interval: 5 * time.Millisecond, FirstDelay: 60 * time.Millisecond, Workers: 1})
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
began := time.Now()
go func() { p.Run(ctx); close(done) }()
require.Eventually(t, func() bool {
reg.mu.Lock()
defer reg.mu.Unlock()
return len(reg.calls) >= 3
}, 5*time.Second, 5*time.Millisecond)
cancel()
select {
case <-done:
case <-time.After(2 * time.Second):
t.Fatal("Run did not return after cancel")
}
assert.GreaterOrEqual(t, time.Since(began), 60*time.Millisecond+2*30*time.Millisecond, "first cycle waits FirstDelay")
assert.Equal(t, 1, reg.maxSeen, "cycles never overlap")
}
func TestRunReturnsDuringFirstDelay(t *testing.T) {
p, _ := newPoller(&fakeStore{}, &fakeJobs{}, &fakeRegistry{}, Config{Interval: time.Hour, FirstDelay: time.Hour})
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
go func() { p.Run(ctx); close(done) }()
cancel()
select {
case <-done:
case <-time.After(2 * time.Second):
t.Fatal("Run did not return after cancel")
}
}
func ptr[T any](v T) *T { return &v }
func TestSeedsTheTagDigestOfARunningService(t *testing.T) {
ref := repoA + ":main"
s := svc(1, ref+"@sha256:new")
s.UpdateMode = model.ModeAuto
s.ActualDigest = "sha256:new" // updated outside the updater: last_tag_digest is empty
st := &fakeStore{services: []model.Service{s}}
reg := &fakeRegistry{digests: map[string]string{ref: "sha256:new"}}
j := &fakeJobs{}
p, _ := newPoller(st, j, reg, Config{})
p.Cycle(context.Background())
assert.Equal(t, []lastTagCall{{1, "sha256:new"}}, st.lastTag)
assert.Empty(t, j.starts)
// Rolled back by hand to the older digest.
st.services[0].ActualRef, st.services[0].ActualDigest = ref+"@sha256:old", "sha256:old"
st.services[0].DesiredRef = ref + "@sha256:old"
p.Cycle(context.Background())
assert.Empty(t, j.starts, "the rollback stays")
assert.Len(t, st.lastTag, 1, "nothing more to seed")
}
-114
View File
@@ -1,114 +0,0 @@
// Package registry looks up the digest a tag points at in a container
// registry, using HEAD requests only.
package registry
import (
"context"
"errors"
"fmt"
"log/slog"
"net"
"net/http"
"net/url"
"regexp"
"strings"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
"github.com/shcizo/package-updater/internal/imageref"
)
// Client looks up tag digests with HEAD requests only. Docker Hub counts a
// GET of a manifest as a pull against the rate limit and a HEAD as nothing,
// so Client never issues a manifest GET.
type Client struct {
keychain authn.Keychain
userAgent string
log *slog.Logger
}
// New returns a Client that finds credentials the way the Docker CLI does:
// in $DOCKER_CONFIG/config.json, else ~/.docker/config.json. Registries
// without an entry are accessed anonymously. userAgent is added to the
// User-Agent header of every request. log receives the full error of a
// failed lookup, which Digest itself returns only in a sanitised form.
func New(userAgent string, log *slog.Logger) *Client {
return &Client{keychain: authn.DefaultKeychain, userAgent: userAgent, log: log}
}
// Digest returns the digest that "repo:tag" points at, as the registry
// reports it: for a multi-arch image that is the digest of the index, the
// same one `docker inspect` shows in RepoDigests. A digest suffix in ref is
// ignored. It sends HEAD /v2/<repo>/manifests/<tag> and nothing else to the
// manifest endpoint; it does not fall back to GET when the registry omits
// the digest header.
func (c *Client) Digest(ctx context.Context, ref string) (string, error) {
p := imageref.Parse(ref)
if p.Tag == "" {
return "", fmt.Errorf("image %q has no tag", ref)
}
r, err := name.ParseReference(p.Repo + ":" + p.Tag)
if err != nil {
return "", fmt.Errorf("parse %s: %w", ref, err)
}
// Credentials are resolved here, not by ggcr, so that a broken credential
// helper keeps its own message: that text comes from the local config.
auth, err := c.keychain.Resolve(r.Context())
if err != nil {
return "", fmt.Errorf("lookup %s: %w", r, err)
}
desc, err := remote.Head(r,
remote.WithContext(ctx),
remote.WithAuth(auth),
remote.WithUserAgent(c.userAgent))
if err != nil {
var terr *transport.Error
if errors.As(err, &terr) {
return "", fmt.Errorf("lookup %s: %s", r, describe(terr))
}
c.log.Warn("registry_lookup_failed", "ref", ref, "error", err)
return "", fmt.Errorf("lookup %s: %s", r, classify(err))
}
return desc.Digest.String(), nil
}
// classify returns a fixed text for an error that is not a registry answer.
// ggcr copies registry-controlled text (token responses, authentication
// challenges, redirect URLs, certificate names) into such errors, so none of
// it is passed on.
func classify(err error) string {
var netErr net.Error
var urlErr *url.Error
switch {
case errors.Is(err, context.DeadlineExceeded), errors.Is(err, context.Canceled):
return "timeout"
case errors.As(err, &netErr), errors.As(err, &urlErr):
return "cannot reach registry"
}
return "lookup failed"
}
// errorCodePattern matches the registry error codes of the distribution
// spec ("MANIFEST_UNKNOWN"). Anything else is dropped from error text.
var errorCodePattern = regexp.MustCompile(`^[A-Z_]{1,40}$`)
// describe returns the HTTP status and the structured error codes of a
// registry error, without the messages, details or raw body that ggcr copies
// into its error text. The registry host comes from an image name, so
// whatever it answers must not end up in the UI.
func describe(e *transport.Error) string {
msg := fmt.Sprintf("registry answered %d %s", e.StatusCode, http.StatusText(e.StatusCode))
var codes []string
for _, d := range e.Errors {
if errorCodePattern.MatchString(string(d.Code)) {
codes = append(codes, string(d.Code))
}
}
if len(codes) > 0 {
msg += " (" + strings.Join(codes, ", ") + ")"
}
return msg
}
-336
View File
@@ -1,336 +0,0 @@
package registry
import (
"context"
"encoding/base64"
"errors"
"fmt"
"io"
"log"
"log/slog"
"net"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
ggcrregistry "github.com/google/go-containerregistry/pkg/registry"
"github.com/google/go-containerregistry/pkg/v1/random"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
var discardLog = slog.New(slog.NewTextHandler(io.Discard, nil))
func newClient() *Client { return New("test", discardLog) }
// request is one request the test registry received.
type request struct{ method, path string }
// testRegistry is an in-memory registry that records every request.
type testRegistry struct {
srv *httptest.Server
host string // "127.0.0.1:port", the registry part of a ref
mu sync.Mutex
requests []request
}
// newTestRegistry starts a registry. When user is set every request needs
// those basic-auth credentials.
func newTestRegistry(t *testing.T, user, pass string) *testRegistry {
t.Helper()
backend := ggcrregistry.New(ggcrregistry.Logger(log.New(io.Discard, "", 0)))
tr := &testRegistry{}
tr.srv = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
tr.mu.Lock()
tr.requests = append(tr.requests, request{r.Method, r.URL.Path})
tr.mu.Unlock()
if user != "" {
if u, p, ok := r.BasicAuth(); !ok || u != user || p != pass {
w.Header().Set("WWW-Authenticate", `Basic realm="test"`)
w.WriteHeader(http.StatusUnauthorized)
return
}
}
backend.ServeHTTP(w, r)
}))
t.Cleanup(tr.srv.Close)
tr.host = strings.TrimPrefix(tr.srv.URL, "http://")
return tr
}
// pushIndex stores a random multi-arch index as repo:tag and returns its
// digest. user and pass are used when the registry requires them.
func (tr *testRegistry) pushIndex(t *testing.T, repo, tag string, opts ...remote.Option) string {
t.Helper()
idx, err := random.Index(64, 1, 2)
require.NoError(t, err)
ref, err := name.ParseReference(tr.host + "/" + repo + ":" + tag)
require.NoError(t, err)
require.NoError(t, remote.WriteIndex(ref, idx, opts...))
d, err := idx.Digest()
require.NoError(t, err)
return d.String()
}
// manifestRequests returns the requests that touched a manifest.
func (tr *testRegistry) manifestRequests() []request {
tr.mu.Lock()
defer tr.mu.Unlock()
var out []request
for _, r := range tr.requests {
if strings.Contains(r.path, "/manifests/") {
out = append(out, r)
}
}
return out
}
// reset forgets the requests received so far.
func (tr *testRegistry) reset() {
tr.mu.Lock()
defer tr.mu.Unlock()
tr.requests = nil
}
// dockerConfig points DOCKER_CONFIG at a fresh directory holding config.json
// with the given content; "" leaves the directory empty.
func dockerConfig(t *testing.T, configJSON string) {
t.Helper()
dir := t.TempDir()
if configJSON != "" {
require.NoError(t, os.WriteFile(filepath.Join(dir, "config.json"), []byte(configJSON), 0o600))
}
t.Setenv("DOCKER_CONFIG", dir)
}
func TestDigestOfIndexIsIndexDigest(t *testing.T) {
dockerConfig(t, "")
reg := newTestRegistry(t, "", "")
want := reg.pushIndex(t, "team/app", "1.2")
got, err := newClient().Digest(context.Background(), reg.host+"/team/app:1.2")
require.NoError(t, err)
assert.Equal(t, want, got)
}
func TestDigestIgnoresDigestSuffix(t *testing.T) {
dockerConfig(t, "")
reg := newTestRegistry(t, "", "")
want := reg.pushIndex(t, "team/app", "1.2")
got, err := newClient().Digest(context.Background(), reg.host+"/team/app:1.2@sha256:"+strings.Repeat("0", 64))
require.NoError(t, err)
assert.Equal(t, want, got)
}
func TestDigestUsesHeadOnly(t *testing.T) {
dockerConfig(t, "")
reg := newTestRegistry(t, "", "")
reg.pushIndex(t, "team/app", "1.2")
reg.reset()
_, err := newClient().Digest(context.Background(), reg.host+"/team/app:1.2")
require.NoError(t, err)
assert.Equal(t, []request{{http.MethodHead, "/v2/team/app/manifests/1.2"}}, reg.manifestRequests())
for _, r := range reg.requests {
if !strings.Contains(r.path, "/manifests/") {
assert.Equal(t, request{http.MethodGet, "/v2/"}, r, "the only other request is the API ping")
}
}
}
func TestDigestSendsUserAgent(t *testing.T) {
dockerConfig(t, "")
var ua string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ua = r.Header.Get("User-Agent")
w.WriteHeader(http.StatusNotFound)
}))
t.Cleanup(srv.Close)
_, _ = New("package-updater-test/1", discardLog).Digest(context.Background(), strings.TrimPrefix(srv.URL, "http://")+"/a/b:1")
assert.Contains(t, ua, "package-updater-test/1")
}
func TestCredentialsFromDockerConfig(t *testing.T) {
reg := newTestRegistry(t, "bot", "s3cret")
want := reg.pushIndex(t, "team/app", "1.2", remote.WithAuth(&authn.Basic{Username: "bot", Password: "s3cret"}))
auth := base64.StdEncoding.EncodeToString([]byte("bot:s3cret"))
dockerConfig(t, `{"auths":{"`+reg.host+`":{"auth":"`+auth+`"}}}`)
got, err := newClient().Digest(context.Background(), reg.host+"/team/app:1.2")
require.NoError(t, err)
assert.Equal(t, want, got)
}
func TestWrongCredentialsFail(t *testing.T) {
reg := newTestRegistry(t, "bot", "s3cret")
reg.pushIndex(t, "team/app", "1.2", remote.WithAuth(&authn.Basic{Username: "bot", Password: "s3cret"}))
auth := base64.StdEncoding.EncodeToString([]byte("bot:wrong"))
dockerConfig(t, `{"auths":{"`+reg.host+`":{"auth":"`+auth+`"}}}`)
_, err := newClient().Digest(context.Background(), reg.host+"/team/app:1.2")
require.Error(t, err)
assert.NotContains(t, err.Error(), "wrong", "credentials do not leak into errors")
assert.NotContains(t, err.Error(), auth)
}
func TestAnonymousWithoutConfig(t *testing.T) {
dockerConfig(t, "")
open := newTestRegistry(t, "", "")
want := open.pushIndex(t, "team/app", "1.2")
protected := newTestRegistry(t, "bot", "s3cret")
got, err := newClient().Digest(context.Background(), open.host+"/team/app:1.2")
require.NoError(t, err)
assert.Equal(t, want, got, "anonymous access works without a config file")
_, err = newClient().Digest(context.Background(), protected.host+"/team/app:1.2")
assert.Error(t, err, "a registry that needs credentials fails without them")
}
func TestUnknownTagIsAnError(t *testing.T) {
dockerConfig(t, "")
reg := newTestRegistry(t, "", "")
reg.pushIndex(t, "team/app", "1.2")
_, err := newClient().Digest(context.Background(), reg.host+"/team/app:nope")
assert.Error(t, err)
}
func TestRefWithoutTagIsAnError(t *testing.T) {
_, err := newClient().Digest(context.Background(), "team/app@sha256:"+strings.Repeat("a", 64))
assert.ErrorContains(t, err, "has no tag")
}
// A config.json whose credsStore names a helper that is not installed makes
// the lookup fail for every registry, including ones that would work
// anonymously. The master container has no credential helpers, so only
// inline "auths" entries are supported there.
func TestCredsStoreWithoutHelperFailsTheLookup(t *testing.T) {
reg := newTestRegistry(t, "", "")
reg.pushIndex(t, "team/app", "1.2")
dockerConfig(t, `{"credsStore":"no-such-helper-for-package-updater-tests"}`)
_, err := newClient().Digest(context.Background(), reg.host+"/team/app:1.2")
assert.ErrorContains(t, err, "error getting credentials")
assert.ErrorContains(t, err, "docker-credential-no-such-helper-for-package-updater-tests")
}
// A registry that leaves out Docker-Content-Digest must not make the client
// retry with a manifest download, which Docker Hub counts as a pull.
func TestMissingDigestHeaderDoesNotFallBackToDownload(t *testing.T) {
dockerConfig(t, "")
var mu sync.Mutex
var methods []string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if !strings.Contains(r.URL.Path, "/manifests/") {
return
}
mu.Lock()
methods = append(methods, r.Method)
mu.Unlock()
w.Header().Set("Content-Type", "application/vnd.oci.image.index.v1+json")
w.Header().Set("Content-Length", "10")
}))
t.Cleanup(srv.Close)
_, err := newClient().Digest(context.Background(), strings.TrimPrefix(srv.URL, "http://")+"/a/b:1")
assert.Error(t, err)
assert.Equal(t, []string{http.MethodHead}, methods)
}
// ggcr copies up to 64 KiB of the registry's response into its errors. The
// registry host comes from an image name, so that text must not reach the
// stored poll error.
func TestRegistryErrorBodyDoesNotLeak(t *testing.T) {
dockerConfig(t, "")
tests := []struct {
name string
body string
want string
}{
{"raw body", "internal-secret-body", "registry answered 400 Bad Request"},
{"structured errors", `{"errors":[{"code":"MANIFEST_UNKNOWN","message":"internal-secret-body","detail":"internal-secret-body"},{"code":"internal-secret-body"}]}`,
"registry answered 400 Bad Request (MANIFEST_UNKNOWN)"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// HEAD answers carry no body, so the leak is in the GET /v2/ ping.
w.WriteHeader(http.StatusBadRequest)
_, _ = w.Write([]byte(tt.body))
}))
t.Cleanup(srv.Close)
host := strings.TrimPrefix(srv.URL, "http://")
_, err := newClient().Digest(context.Background(), host+"/a/b:1")
require.Error(t, err)
assert.NotContains(t, err.Error(), "internal-secret-body")
assert.Equal(t, "lookup "+host+"/a/b:1: "+tt.want, strings.Replace(err.Error(), "index.docker.io/", "", 1))
})
}
}
// ggcr puts the token server's answer into its error when the answer has no
// token, and the token server is named by the registry's challenge.
func TestNonRegistryErrorsAreFixedText(t *testing.T) {
dockerConfig(t, "")
tokenSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(`{"detail":"internal-secret-body"}`))
}))
t.Cleanup(tokenSrv.Close)
reg := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("WWW-Authenticate", `Bearer realm="`+tokenSrv.URL+`/internal-secret-realm",service="internal-secret-service"`)
w.WriteHeader(http.StatusUnauthorized)
}))
t.Cleanup(reg.Close)
host := strings.TrimPrefix(reg.URL, "http://")
_, err := newClient().Digest(context.Background(), host+"/a/b:1")
require.Error(t, err)
assert.NotContains(t, err.Error(), "internal-secret")
assert.Equal(t, "lookup "+host+"/a/b:1: lookup failed", err.Error())
}
func TestClassify(t *testing.T) {
refused := &url.Error{Op: "Get", URL: "http://internal-secret-host/v2/", Err: errors.New("connection refused")}
tests := []struct {
name string
err error
want string
}{
{"deadline", fmt.Errorf("x: %w", context.DeadlineExceeded), "timeout"},
{"canceled", context.Canceled, "timeout"},
{"url error", refused, "cannot reach registry"},
{"net error", &net.DNSError{Name: "internal-secret-host"}, "cannot reach registry"},
{"deadline inside url error", &url.Error{Op: "Get", URL: "http://x", Err: context.DeadlineExceeded}, "timeout"},
{"anything else", errors.New("internal-secret"), "lookup failed"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, classify(tt.err))
})
}
}
@@ -1,550 +0,0 @@
//go:build integration
package store
import (
"context"
"errors"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/shcizo/package-updater/internal/master/model"
"github.com/shcizo/package-updater/internal/protocol"
)
var (
digNew = "sha256:" + strings.Repeat("e", 64)
digOld = "sha256:" + strings.Repeat("f", 64)
sluskDrift = "gitea.shcizo.se/shcizo/slusk"
)
// serviceID returns the ID of the service the agent reports under svc's key.
func serviceID(t *testing.T, s *Store, agentID int64, svc protocol.Service) int64 {
t.Helper()
var id int64
require.NoError(t, s.pool.QueryRow(context.Background(),
`SELECT id FROM services WHERE location_key = $1`, model.LocationKey(agentID, "", svc.Key())).Scan(&id))
return id
}
// finishJob runs a job for the service to success with the deployed ref.
func finishJob(t *testing.T, s *Store, agentID, svcID int64, toRef, deployed string) {
t.Helper()
ctx := context.Background()
nj := newJob(svcID, agentID)
nj.ToRef = toRef
j, err := s.CreateJob(ctx, nj)
require.NoError(t, err)
ok, err := s.FinishJob(ctx, agentID, j.ID, model.JobFinish{Status: model.JobSucceeded, Ref: deployed, StartedAt: t0, FinishedAt: t0})
require.NoError(t, err)
require.True(t, ok)
}
func mustExec(t *testing.T, s *Store, sql string, args ...any) {
t.Helper()
_, err := s.pool.Exec(context.Background(), sql, args...)
require.NoError(t, err)
}
func TestFinishJob_SetsAwaitingInventoryUntilTheNextUpsert(t *testing.T) {
s := openTestStore(t)
ctx := context.Background()
agent, svc := jobFixture(t, s, "a")
got, err := s.GetService(ctx, svc)
require.NoError(t, err)
assert.False(t, got.AwaitingInventory)
finishJob(t, s, agent, svc, "r:1", "r:1@"+digNew)
got, err = s.GetService(ctx, svc)
require.NoError(t, err)
assert.True(t, got.AwaitingInventory, "set by the successful job")
assert.False(t, got.Drifted(), "desired is ahead of actual only until the inventory arrives")
require.NoError(t, s.UpsertService(ctx, agent, "", slusk, t0.Add(time.Minute)))
got, err = s.GetService(ctx, svc)
require.NoError(t, err)
assert.False(t, got.AwaitingInventory, "cleared by the agent's upsert")
assert.True(t, got.Drifted(), "the agent still reports the old version")
}
func TestFinishJob_FailedJobDoesNotSetAwaitingInventory(t *testing.T) {
s := openTestStore(t)
ctx := context.Background()
agent, svc := jobFixture(t, s, "a")
j, err := s.CreateJob(ctx, newJob(svc, agent))
require.NoError(t, err)
ok, err := s.FinishJob(ctx, agent, j.ID, model.JobFinish{Status: model.JobFailed, Error: "boom", StartedAt: t0, FinishedAt: t0})
require.NoError(t, err)
require.True(t, ok)
got, err := s.GetService(ctx, svc)
require.NoError(t, err)
assert.False(t, got.AwaitingInventory)
}
func TestCreateJob_UnknownServiceIsNotFound(t *testing.T) {
s := openTestStore(t)
agent := newAgent(t, s, "a")
_, err := s.CreateJob(context.Background(), newJob(9999, agent))
assert.ErrorIs(t, err, model.ErrNotFound)
}
// driftedFixture is a service updated to digNew whose agent now reports digD1.
func driftedFixture(t *testing.T, s *Store) (agent, svc int64) {
t.Helper()
agent, svc = jobFixture(t, s, "a")
finishJob(t, s, agent, svc, "r:1", sluskDrift+":1.4.2@"+digNew)
require.NoError(t, s.UpsertService(context.Background(), agent, "", slusk, t0.Add(time.Minute)))
return agent, svc
}
func TestAcceptRunning_WritesTheDesiredRef(t *testing.T) {
s := openTestStore(t)
ctx := context.Background()
_, svc := driftedFixture(t, s)
require.NoError(t, s.SetLastTagDigest(ctx, svc, "sha256:ltd"))
require.NoError(t, s.SetAvailable(ctx, svc, model.Available{Ref: sluskDrift + ":1.5.0", Source: model.SourcePoll, At: t0}))
require.NoError(t, s.SetUpdateMode(ctx, svc, model.ModeAuto))
before, err := s.GetService(ctx, svc)
require.NoError(t, err)
require.True(t, before.Drifted())
running := model.RunningRef(before.ActualRef, before.ActualDigest)
require.NoError(t, s.AcceptRunning(ctx, svc, before.ActualRef, before.ActualDigest, running))
got, err := s.GetService(ctx, svc)
require.NoError(t, err)
assert.Equal(t, sluskDrift+":1.4.2@"+digD1, got.DesiredRef)
assert.False(t, got.Drifted())
assert.False(t, got.AwaitingInventory)
assert.Equal(t, "sha256:ltd", got.LastTagDigest, "last_tag_digest untouched")
assert.Equal(t, sluskDrift+":1.5.0", got.AvailableRef, "available suggestion untouched")
assert.Equal(t, model.ModeAuto, got.UpdateMode)
}
func TestAcceptRunning_Refusals(t *testing.T) {
tests := []struct {
name string
prepare func(t *testing.T, s *Store, agent, svc int64) (id int64, actualRef, actualDigest string)
wantError error
}{
{"running version changed", func(t *testing.T, s *Store, agent, svc int64) (int64, string, string) {
return svc, sluskDrift + ":1.4.2", digOld
}, model.ErrConflict},
{"actual ref changed", func(t *testing.T, s *Store, agent, svc int64) (int64, string, string) {
return svc, sluskDrift + ":1.3.0", digD1
}, model.ErrConflict},
{"never updated", func(t *testing.T, s *Store, agent, svc int64) (int64, string, string) {
mustExec(t, s, `UPDATE services SET desired_ref = '' WHERE id = $1`, svc)
return svc, sluskDrift + ":1.4.2", digD1
}, model.ErrConflict},
{"removed", func(t *testing.T, s *Store, agent, svc int64) (int64, string, string) {
require.NoError(t, s.RemoveService(context.Background(), agent, "", slusk.Key(), t0.Add(time.Hour)))
return svc, sluskDrift + ":1.4.2", digD1
}, model.ErrNotFound},
{"unknown", func(t *testing.T, s *Store, agent, svc int64) (int64, string, string) {
return 9999, sluskDrift + ":1.4.2", digD1
}, model.ErrNotFound},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
s := openTestStore(t)
agent, svc := driftedFixture(t, s)
id, ref, digest := tt.prepare(t, s, agent, svc)
err := s.AcceptRunning(context.Background(), id, ref, digest, "x:1@"+digOld)
assert.ErrorIs(t, err, tt.wantError)
got, gerr := s.GetService(context.Background(), svc)
require.NoError(t, gerr)
assert.NotEqual(t, "x:1@"+digOld, got.DesiredRef, "nothing written")
})
}
}
// moveFixture has agent "a" with slusk removed after it was updated (history,
// auto mode) and agent "b" reporting a new slusk row.
type moveFixture struct {
a, b int64
from, to int64
}
func newMoveFixture(t *testing.T, s *Store) moveFixture {
t.Helper()
ctx := context.Background()
f := moveFixture{a: newAgent(t, s, "a"), b: newAgent(t, s, "b")}
require.NoError(t, s.ApplySnapshot(ctx, f.a, worker, []protocol.Service{slusk}, t0))
f.from = serviceID(t, s, f.a, slusk)
finishJob(t, s, f.a, f.from, "r:1", sluskDrift+":1.4.2@"+digNew)
require.NoError(t, s.SetUpdateMode(ctx, f.from, model.ModeAuto))
require.NoError(t, s.ApplySnapshot(ctx, f.a, worker, nil, t0.Add(time.Hour)))
require.NoError(t, s.ApplySnapshot(ctx, f.b, worker, []protocol.Service{slusk}, t0.Add(2*time.Hour)))
f.to = serviceID(t, s, f.b, slusk)
return f
}
func TestMoveCandidates(t *testing.T) {
tests := []struct {
name string
mutate func(t *testing.T, s *Store, f moveFixture)
want bool
from string // agent name of the removed row
}{
{"one removed and one live", func(*testing.T, *Store, moveFixture) {}, true, "a"},
{"removed row's agent was revoked", func(t *testing.T, s *Store, f moveFixture) {
_, err := s.RevokeAgent(context.Background(), f.a, t0.Add(3*time.Hour))
require.NoError(t, err)
}, true, "a"},
{"two live targets", func(t *testing.T, s *Store, f moveFixture) {
c := newAgent(t, s, "c")
require.NoError(t, s.ApplySnapshot(context.Background(), c, worker, []protocol.Service{slusk}, t0.Add(3*time.Hour)))
}, false, ""},
{"two removed sources", func(t *testing.T, s *Store, f moveFixture) {
ctx := context.Background()
d := newAgent(t, s, "d")
require.NoError(t, s.ApplySnapshot(ctx, d, worker, []protocol.Service{slusk}, t0))
require.NoError(t, s.ApplySnapshot(ctx, d, worker, nil, t0.Add(time.Hour)))
}, false, ""},
{"target has history", func(t *testing.T, s *Store, f moveFixture) {
finishJob(t, s, f.b, f.to, "r:1", sluskDrift+":1.4.2@"+digNew)
}, false, ""},
{"target has a desired ref", func(t *testing.T, s *Store, f moveFixture) {
mustExec(t, s, `UPDATE services SET desired_ref = 'r:1@x' WHERE id = $1`, f.to)
}, false, ""},
{"source has no history", func(t *testing.T, s *Store, f moveFixture) {
mustExec(t, s, `DELETE FROM update_jobs WHERE service_id = $1`, f.from)
mustExec(t, s, `UPDATE services SET desired_ref = '', update_mode = 'manual' WHERE id = $1`, f.from)
}, false, ""},
{"source only has auto mode", func(t *testing.T, s *Store, f moveFixture) {
mustExec(t, s, `DELETE FROM update_jobs WHERE service_id = $1`, f.from)
mustExec(t, s, `UPDATE services SET desired_ref = '' WHERE id = $1`, f.from)
}, true, "a"},
{"source only has jobs", func(t *testing.T, s *Store, f moveFixture) {
mustExec(t, s, `UPDATE services SET desired_ref = '', update_mode = 'manual' WHERE id = $1`, f.from)
}, true, "a"},
{"target is older than the source", func(t *testing.T, s *Store, f moveFixture) {
mustExec(t, s, `UPDATE services SET first_seen_at = $2 WHERE id = $1`, f.to, t0.Add(-time.Hour))
}, false, ""},
{"plain containers", func(t *testing.T, s *Store, f moveFixture) {
mustExec(t, s, `UPDATE services SET kind = 'container' WHERE id IN ($1, $2)`, f.from, f.to)
}, false, ""},
{"swarm services", func(t *testing.T, s *Store, f moveFixture) {
mustExec(t, s, `UPDATE services SET kind = 'swarm' WHERE id IN ($1, $2)`, f.from, f.to)
}, false, ""},
{"the updater itself (target)", func(t *testing.T, s *Store, f moveFixture) {
mustExec(t, s, `UPDATE services SET self = true WHERE id = $1`, f.to)
}, false, ""},
{"the updater itself (source)", func(t *testing.T, s *Store, f moveFixture) {
mustExec(t, s, `UPDATE services SET self = true WHERE id = $1`, f.from)
}, false, ""},
{"live row also removed", func(t *testing.T, s *Store, f moveFixture) {
require.NoError(t, s.RemoveService(context.Background(), f.b, "", slusk.Key(), t0.Add(3*time.Hour)))
}, false, ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
s := openTestStore(t)
f := newMoveFixture(t, s)
tt.mutate(t, s, f)
got, err := s.MoveCandidates(context.Background())
require.NoError(t, err)
if !tt.want {
assert.Empty(t, got)
return
}
assert.Equal(t, []model.Move{{FromID: f.from, ToID: f.to, FromAgent: tt.from, ToAgent: "b"}}, got)
})
}
}
func TestMergeService_MovesHistoryModeAndDesiredToTheNewLocation(t *testing.T) {
s := openTestStore(t)
ctx := context.Background()
f := newMoveFixture(t, s)
finishJob(t, s, f.b, f.to, "r:2", sluskDrift+":2.0.0@"+digOld)
require.NoError(t, s.SetLastTagDigest(ctx, f.to, "sha256:fromto"))
require.NoError(t, s.SetAvailable(ctx, f.to, model.Available{Ref: sluskDrift + ":3", Digest: digNew, Source: model.SourcePoll, At: t0}))
require.NoError(t, s.SetPollError(ctx, []int64{f.to}, "registry down"))
// A job on the old row is finished, and awaiting_inventory is set on both.
before, err := s.GetService(ctx, f.from)
require.NoError(t, err)
require.NoError(t, s.MergeService(ctx, f.from, f.to))
got, err := s.GetService(ctx, f.from)
require.NoError(t, err)
assert.Equal(t, f.from, got.ID, "the old id survives")
assert.Nil(t, got.RemovedAt)
assert.False(t, got.AwaitingInventory)
assert.Equal(t, "b", got.AgentName)
assert.Equal(t, model.LocationKey(f.b, "", slusk.Key()), got.LocationKey)
assert.Equal(t, before.FirstSeenAt, got.FirstSeenAt, "first_seen_at kept")
assert.Equal(t, model.ModeAuto, got.UpdateMode, "mode kept")
assert.Equal(t, before.DesiredRef, got.DesiredRef, "desired ref kept")
assert.Equal(t, slusk.Image, got.ActualRef, "inventory from the duplicate")
assert.Equal(t, digD1, got.ActualDigest)
assert.Equal(t, "running", got.Status)
assert.Equal(t, sluskDrift+":3", got.AvailableRef)
assert.Equal(t, digNew, got.AvailableDigest)
assert.Equal(t, "registry down", got.LastPollError)
assert.Equal(t, before.LastTagDigest, got.LastTagDigest, "an existing last_tag_digest wins")
_, err = s.GetService(ctx, f.to)
assert.ErrorIs(t, err, model.ErrNotFound, "the duplicate is deleted")
jobs, err := s.ListJobs(ctx, model.JobFilter{ServiceID: f.from})
require.NoError(t, err)
assert.Len(t, jobs, 2, "the duplicate's job moved to the survivor")
live, err := s.ListServices(ctx)
require.NoError(t, err)
assert.Equal(t, []string{"slusk/web"}, names(live))
}
func TestMergeService_TakesLastTagDigestOfTheDuplicateWhenTheOldRowHasNone(t *testing.T) {
s := openTestStore(t)
ctx := context.Background()
f := newMoveFixture(t, s)
mustExec(t, s, `UPDATE services SET last_tag_digest = '' WHERE id = $1`, f.from)
require.NoError(t, s.SetLastTagDigest(ctx, f.to, "sha256:fromto"))
require.NoError(t, s.MergeService(ctx, f.from, f.to))
got, err := s.GetService(ctx, f.from)
require.NoError(t, err)
assert.Equal(t, "sha256:fromto", got.LastTagDigest)
}
func TestMergeService_ManualMergeOfAnyRemovedServiceOfTheSameKind(t *testing.T) {
s := openTestStore(t)
ctx := context.Background()
f := newMoveFixture(t, s)
other := protocol.Service{Kind: protocol.KindCompose, ComposeProject: "renamed", ComposeService: "app", Image: "x/y:1", Status: "running"}
require.NoError(t, s.UpsertService(ctx, f.b, "", other, t0.Add(3*time.Hour)))
target := serviceID(t, s, f.b, other)
require.NoError(t, s.MergeService(ctx, f.from, target))
got, err := s.GetService(ctx, f.from)
require.NoError(t, err)
assert.Equal(t, "renamed/app", got.DisplayName(), "names may differ")
}
func TestMergeService_SameAgentIsAllowed(t *testing.T) {
s := openTestStore(t)
ctx := context.Background()
f := newMoveFixture(t, s)
other := protocol.Service{Kind: protocol.KindCompose, ComposeProject: "renamed", ComposeService: "app", Image: "x/y:1", Status: "running"}
require.NoError(t, s.UpsertService(ctx, f.a, "", other, t0.Add(3*time.Hour)))
require.NoError(t, s.MergeService(ctx, f.from, serviceID(t, s, f.a, other)))
got, err := s.GetService(ctx, f.from)
require.NoError(t, err)
assert.Equal(t, "a", got.AgentName)
}
func TestMergeService_Swarm(t *testing.T) {
s := openTestStore(t)
ctx := context.Background()
m1 := newAgent(t, s, "m1")
require.NoError(t, s.ApplySnapshot(ctx, m1, manager, []protocol.Service{gitea}, t0))
oldID := serviceIDByName(t, s, "gitea_gitea")
require.NoError(t, s.ApplySnapshot(ctx, m1, manager, nil, t0.Add(time.Hour)))
renamed := gitea
renamed.SwarmService = "gitea2_gitea"
require.NoError(t, s.ApplySnapshot(ctx, m1, manager, []protocol.Service{renamed}, t0.Add(2*time.Hour)))
newID := serviceIDByName(t, s, "gitea2_gitea")
require.NoError(t, s.MergeService(ctx, oldID, newID))
got, err := s.GetService(ctx, oldID)
require.NoError(t, err)
assert.Equal(t, "gitea2_gitea", got.SwarmService)
assert.Nil(t, got.RemovedAt)
}
func serviceIDByName(t *testing.T, s *Store, swarmService string) int64 {
t.Helper()
var id int64
require.NoError(t, s.pool.QueryRow(context.Background(),
`SELECT id FROM services WHERE swarm_service = $1`, swarmService).Scan(&id))
return id
}
func TestMergeService_Refusals(t *testing.T) {
refused := func(reason string) func(*testing.T, error) {
return func(t *testing.T, err error) {
var mr *model.MergeRefusedError
require.True(t, errors.As(err, &mr), "got %v", err)
assert.Equal(t, reason, mr.Reason)
}
}
is := func(want error) func(*testing.T, error) {
return func(t *testing.T, err error) { assert.ErrorIs(t, err, want) }
}
tests := []struct {
name string
run func(t *testing.T, s *Store, f moveFixture) error
assert func(*testing.T, error)
}{
{"into itself", func(t *testing.T, s *Store, f moveFixture) error {
return s.MergeService(context.Background(), f.to, f.to)
}, refused(model.ReasonMergeSame)},
{"the old service still exists", func(t *testing.T, s *Store, f moveFixture) error {
mustExec(t, s, `UPDATE services SET removed_at = NULL WHERE id = $1`, f.from)
return s.MergeService(context.Background(), f.from, f.to)
}, refused(model.ReasonMergeLiveOld)},
{"the new service was removed", func(t *testing.T, s *Store, f moveFixture) error {
require.NoError(t, s.RemoveService(context.Background(), f.b, "", slusk.Key(), t0.Add(3*time.Hour)))
return s.MergeService(context.Background(), f.from, f.to)
}, refused(model.ReasonMergeGoneNew)},
{"different kinds", func(t *testing.T, s *Store, f moveFixture) error {
mustExec(t, s, `UPDATE services SET kind = 'swarm' WHERE id = $1`, f.to)
return s.MergeService(context.Background(), f.from, f.to)
}, refused(model.ReasonMergeKind)},
{"plain containers", func(t *testing.T, s *Store, f moveFixture) error {
mustExec(t, s, `UPDATE services SET kind = 'container' WHERE id IN ($1, $2)`, f.from, f.to)
return s.MergeService(context.Background(), f.from, f.to)
}, refused(model.ReasonMergeContainer)},
{"the updater itself (new)", func(t *testing.T, s *Store, f moveFixture) error {
mustExec(t, s, `UPDATE services SET self = true WHERE id = $1`, f.to)
return s.MergeService(context.Background(), f.from, f.to)
}, refused(model.ReasonSelf)},
{"the updater itself (old)", func(t *testing.T, s *Store, f moveFixture) error {
mustExec(t, s, `UPDATE services SET self = true WHERE id = $1`, f.from)
return s.MergeService(context.Background(), f.from, f.to)
}, refused(model.ReasonSelf)},
{"active job on the old service", func(t *testing.T, s *Store, f moveFixture) error {
_, err := s.CreateJob(context.Background(), newJob(f.from, f.a))
require.NoError(t, err)
return s.MergeService(context.Background(), f.from, f.to)
}, is(model.ErrJobActive)},
{"active job on the new service", func(t *testing.T, s *Store, f moveFixture) error {
_, err := s.CreateJob(context.Background(), newJob(f.to, f.b))
require.NoError(t, err)
return s.MergeService(context.Background(), f.from, f.to)
}, is(model.ErrJobActive)},
{"unknown old service", func(t *testing.T, s *Store, f moveFixture) error {
return s.MergeService(context.Background(), 9999, f.to)
}, is(model.ErrNotFound)},
{"unknown new service", func(t *testing.T, s *Store, f moveFixture) error {
return s.MergeService(context.Background(), f.from, 9999)
}, is(model.ErrNotFound)},
{"the new service's agent is revoked", func(t *testing.T, s *Store, f moveFixture) error {
_, err := s.RevokeAgent(context.Background(), f.b, t0.Add(3*time.Hour))
require.NoError(t, err)
return s.MergeService(context.Background(), f.from, f.to)
}, is(model.ErrRevoked)},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
s := openTestStore(t)
f := newMoveFixture(t, s)
var rowsBefore int
require.NoError(t, s.pool.QueryRow(context.Background(), `SELECT count(*) FROM services`).Scan(&rowsBefore))
err := tt.run(t, s, f)
tt.assert(t, err)
var rowsAfter int
require.NoError(t, s.pool.QueryRow(context.Background(), `SELECT count(*) FROM services`).Scan(&rowsAfter))
assert.Equal(t, rowsBefore, rowsAfter, "nothing is deleted when the merge is refused")
})
}
}
func TestMergeService_InventoryAfterTheMerge(t *testing.T) {
setup := func(t *testing.T) (*Store, moveFixture) {
s := openTestStore(t)
f := newMoveFixture(t, s)
require.NoError(t, s.MergeService(context.Background(), f.from, f.to))
return s, f
}
t.Run("an upsert for the new location updates the survivor", func(t *testing.T) {
s, f := setup(t)
ctx := context.Background()
moved := slusk
moved.Digest = digOld
require.NoError(t, s.UpsertService(ctx, f.b, "", moved, t0.Add(4*time.Hour)))
live, err := s.ListServices(ctx)
require.NoError(t, err)
require.Len(t, live, 1)
assert.Equal(t, f.from, live[0].ID)
assert.Equal(t, digOld, live[0].ActualDigest)
})
t.Run("a snapshot keeps the survivor", func(t *testing.T) {
s, f := setup(t)
ctx := context.Background()
require.NoError(t, s.ApplySnapshot(ctx, f.b, worker, []protocol.Service{slusk}, t0.Add(4*time.Hour)))
got, err := s.GetService(ctx, f.from)
require.NoError(t, err)
assert.Nil(t, got.RemovedAt)
live, err := s.ListServices(ctx)
require.NoError(t, err)
assert.Len(t, live, 1)
})
t.Run("the old agent returning with the stack creates a new row", func(t *testing.T) {
s, f := setup(t)
ctx := context.Background()
require.NoError(t, s.ApplySnapshot(ctx, f.a, worker, []protocol.Service{slusk}, t0.Add(4*time.Hour)))
returned := serviceID(t, s, f.a, slusk)
assert.NotEqual(t, f.from, returned)
got, err := s.GetService(ctx, returned)
require.NoError(t, err)
assert.Equal(t, model.ModeManual, got.UpdateMode)
assert.Empty(t, got.DesiredRef)
live, err := s.ListServices(ctx)
require.NoError(t, err)
assert.Len(t, live, 2)
})
}
func TestMergeSources(t *testing.T) {
s := openTestStore(t)
ctx := context.Background()
f := newMoveFixture(t, s)
// A removed service with another name, removed later than the matching one.
other := protocol.Service{Kind: protocol.KindCompose, ComposeProject: "renamed", ComposeService: "app", Image: "x/y:1", Status: "running"}
require.NoError(t, s.UpsertService(ctx, f.a, "", other, t0))
require.NoError(t, s.RemoveService(ctx, f.a, "", other.Key(), t0.Add(5*time.Hour)))
otherID := serviceID(t, s, f.a, other)
// Other kinds, the updater itself and a live service never show up.
require.NoError(t, s.ApplySnapshot(ctx, f.a, worker, []protocol.Service{adhoc}, t0.Add(6*time.Hour)))
require.NoError(t, s.ApplySnapshot(ctx, f.a, worker, nil, t0.Add(7*time.Hour)))
selfSvc := protocol.Service{Kind: protocol.KindCompose, ComposeProject: "pu", ComposeService: "agent", Image: "pu:1", Status: "running", Self: true}
require.NoError(t, s.UpsertService(ctx, f.a, "", selfSvc, t0))
require.NoError(t, s.RemoveService(ctx, f.a, "", selfSvc.Key(), t0.Add(8*time.Hour)))
got, err := s.MergeSources(ctx, f.to)
require.NoError(t, err)
var ids []int64
for _, m := range got {
ids = append(ids, m.ID)
}
assert.Equal(t, []int64{f.from, otherID}, ids, "same name first, then the most recently removed")
assert.Equal(t, "a", got[0].AgentName)
_, err = s.MergeSources(ctx, 9999)
assert.ErrorIs(t, err, model.ErrNotFound)
containerID := serviceID(t, s, f.a, adhoc)
got, err = s.MergeSources(ctx, containerID)
require.NoError(t, err)
assert.Empty(t, got, "plain containers cannot be merged")
}
+9 -23
View File
@@ -10,7 +10,6 @@ import (
"github.com/jackc/pgx/v5" "github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn" "github.com/jackc/pgx/v5/pgconn"
"github.com/shcizo/package-updater/internal/imageref"
"github.com/shcizo/package-updater/internal/master/model" "github.com/shcizo/package-updater/internal/master/model"
"github.com/shcizo/package-updater/internal/protocol" "github.com/shcizo/package-updater/internal/protocol"
) )
@@ -91,9 +90,6 @@ func (s *Store) CreateJob(ctx context.Context, j model.NewJob) (model.Job, error
} }
return active, model.ErrJobActive return active, model.ErrJobActive
} }
if errors.As(err, &pgErr) && pgErr.Code == "23503" && pgErr.ConstraintName == "update_jobs_service_id_fkey" {
return model.Job{}, model.ErrNotFound
}
if err != nil { if err != nil {
return model.Job{}, fmt.Errorf("create job: %w", err) return model.Job{}, fmt.Errorf("create job: %w", err)
} }
@@ -193,11 +189,9 @@ func (s *Store) JobProgress(ctx context.Context, agentID, jobID int64, chunk str
// started_at (the agent's, or the existing one if the agent sent none), // started_at (the agent's, or the existing one if the agent sent none),
// finished_at; to_ref = f.Ref when set. On success it also sets the // finished_at; to_ref = f.Ref when set. On success it also sets the
// service's desired_ref = f.Ref and clears its available ref when that is // service's desired_ref = f.Ref and clears its available ref when that is
// the ref the job was created for and was reported no later than the job was // the ref the job was created for (ignoring any @digest) and was reported no
// created (that version is now running), in the same transaction. A job // later than the job was created (that version is now running), in the same
// created for "repo:tag@digest" clears only a suggestion with that same // transaction. Returns false if no
// digest, so rolling back to an older digest keeps a newer suggestion (and a
// suggestion without a digest, such as a webhook's). Returns false if no
// such active job belongs to agentID. // such active job belongs to agentID.
func (s *Store) FinishJob(ctx context.Context, agentID, jobID int64, f model.JobFinish) (bool, error) { func (s *Store) FinishJob(ctx context.Context, agentID, jobID int64, f model.JobFinish) (bool, error) {
tx, err := s.pool.Begin(ctx) tx, err := s.pool.Begin(ctx)
@@ -232,21 +226,13 @@ func (s *Store) FinishJob(ctx context.Context, agentID, jobID int64, f model.Job
return false, fmt.Errorf("finish job: %w", err) return false, fmt.Errorf("finish job: %w", err)
} }
if f.Status == model.JobSucceeded { if f.Status == model.JobSucceeded {
requestedRef, requestedDigest, _ := strings.Cut(requestedRef, "@") requestedRef, _, _ = strings.Cut(requestedRef, "@")
// A plain tag job, whoever started it, deployed the tag at the digest
// the agent pinned; remember it. Pinned and tagless picks do not.
tagDigest := ""
if _, d, _ := strings.Cut(f.Ref, "@"); requestedDigest == "" && imageref.IsDigest(d) {
tagDigest = d
}
if _, err := tx.Exec(ctx, ` if _, err := tx.Exec(ctx, `
UPDATE services SET desired_ref = $2, awaiting_inventory = true, UPDATE services SET desired_ref = $2,
last_tag_digest = CASE WHEN $6 <> '' THEN $6 ELSE last_tag_digest END, available_ref = CASE WHEN available_ref = $4 AND available_at <= $3 THEN '' ELSE available_ref END,
available_ref = CASE WHEN available_ref = $4 AND available_at <= $3 AND ($5 = '' OR available_digest = $5) THEN '' ELSE available_ref END, available_source = CASE WHEN available_ref = $4 AND available_at <= $3 THEN '' ELSE available_source END,
available_digest = CASE WHEN available_ref = $4 AND available_at <= $3 AND ($5 = '' OR available_digest = $5) THEN '' ELSE available_digest END, available_at = CASE WHEN available_ref = $4 AND available_at <= $3 THEN NULL ELSE available_at END
available_source = CASE WHEN available_ref = $4 AND available_at <= $3 AND ($5 = '' OR available_digest = $5) THEN '' ELSE available_source END, WHERE id = $1`, serviceID, f.Ref, createdAt, requestedRef); err != nil {
available_at = CASE WHEN available_ref = $4 AND available_at <= $3 AND ($5 = '' OR available_digest = $5) THEN NULL ELSE available_at END
WHERE id = $1`, serviceID, f.Ref, createdAt, requestedRef, requestedDigest, tagDigest); err != nil {
return false, fmt.Errorf("finish job: desired ref: %w", err) return false, fmt.Errorf("finish job: desired ref: %w", err)
} }
} }
-249
View File
@@ -1,249 +0,0 @@
package store
import (
"context"
"errors"
"fmt"
"github.com/jackc/pgx/v5"
"github.com/shcizo/package-updater/internal/master/model"
"github.com/shcizo/package-updater/internal/protocol"
)
// moveCandidatesSQL pairs a removed Compose service with a live one of the
// same project and service name on another agent. A pair counts only when it
// is the single one for both rows, and the live row is new (newer than the
// removed one, never updated, no jobs) while the removed one has something
// worth moving (a desired ref, auto mode or jobs).
const moveCandidatesSQL = `
WITH pairs AS (
SELECT f.id AS from_id, t.id AS to_id
FROM services f
JOIN services t ON t.kind = 'compose' AND t.removed_at IS NULL AND NOT t.self
AND t.compose_project = f.compose_project AND t.compose_service = f.compose_service
AND t.agent_id IS DISTINCT FROM f.agent_id
WHERE f.kind = 'compose' AND f.removed_at IS NOT NULL AND NOT f.self
), uniq AS (
SELECT from_id, to_id FROM pairs p
WHERE (SELECT count(*) FROM pairs q WHERE q.from_id = p.from_id) = 1
AND (SELECT count(*) FROM pairs q WHERE q.to_id = p.to_id) = 1
)
SELECT u.from_id, u.to_id, COALESCE(fa.name, ''), COALESCE(ta.name, '')
FROM uniq u
JOIN services f ON f.id = u.from_id
JOIN services t ON t.id = u.to_id
LEFT JOIN agents fa ON fa.id = f.agent_id
LEFT JOIN agents ta ON ta.id = t.agent_id
WHERE t.first_seen_at > f.first_seen_at
AND t.desired_ref = ''
AND NOT EXISTS (SELECT 1 FROM update_jobs j WHERE j.service_id = t.id)
AND (f.desired_ref <> '' OR f.update_mode = 'auto'
OR EXISTS (SELECT 1 FROM update_jobs j WHERE j.service_id = f.id))
ORDER BY u.to_id`
// MoveCandidates returns the removed Compose services that look like they
// moved to another agent; see model.Move. It only suggests: nothing is merged.
func (s *Store) MoveCandidates(ctx context.Context) ([]model.Move, error) {
rows, err := s.pool.Query(ctx, moveCandidatesSQL)
if err != nil {
return nil, fmt.Errorf("move candidates: %w", err)
}
defer rows.Close()
var out []model.Move
for rows.Next() {
var m model.Move
if err := rows.Scan(&m.FromID, &m.ToID, &m.FromAgent, &m.ToAgent); err != nil {
return nil, fmt.Errorf("scan move: %w", err)
}
out = append(out, m)
}
return out, rows.Err()
}
// MergeSources returns up to 50 removed services that could be merged into
// the service toID: the same kind (Compose or Swarm), not the updater itself,
// those with the same name first, then the most recently removed. Returns
// model.ErrNotFound if toID does not exist.
func (s *Store) MergeSources(ctx context.Context, toID int64) ([]model.Service, error) {
to, err := s.GetService(ctx, toID)
if err != nil {
return nil, err
}
if to.Kind == protocol.KindContainer {
return nil, nil
}
rows, err := s.pool.Query(ctx, `
SELECT `+serviceColumns+`
FROM services s LEFT JOIN agents a ON a.id = s.agent_id
WHERE s.removed_at IS NOT NULL AND s.kind = $1 AND NOT s.self AND s.id <> $2
ORDER BY (s.compose_project, s.compose_service, s.swarm_service) = ($3, $4, $5) DESC,
s.removed_at DESC, s.id DESC
LIMIT 50`,
string(to.Kind), toID, to.ComposeProject, to.ComposeService, to.SwarmService)
if err != nil {
return nil, fmt.Errorf("merge sources: %w", err)
}
defer rows.Close()
var out []model.Service
for rows.Next() {
m, err := scanService(rows)
if err != nil {
return nil, fmt.Errorf("scan service: %w", err)
}
out = append(out, m)
}
return out, rows.Err()
}
// mergeRow is what MergeService needs to know about one of the two rows.
type mergeRow struct {
kind string
agentID *int64
removed bool
self bool
}
// MergeService moves the live service toID onto the removed service fromID:
// the old row keeps its id, first_seen_at, update mode and desired ref, takes
// the location and inventory of the live row (and its available suggestion),
// becomes live, and the live row is deleted. The jobs of the live row move to
// the old one. This is the only place a services row is deleted.
//
// It returns *model.MergeRefusedError when the pair cannot be merged,
// model.ErrJobActive when either row has a queued or running job,
// model.ErrRevoked when the live row's agent is revoked, model.ErrNotFound
// when a row does not exist, and model.ErrConflict when the live row moved to
// another agent while the merge started.
//
// Locks are taken in the order inventory writes use: the agent row first,
// then the service rows by id. The live row is deleted before the old one is
// updated because location_key is unique.
func (s *Store) MergeService(ctx context.Context, fromID, toID int64) error {
if fromID == toID {
return &model.MergeRefusedError{Reason: model.ReasonMergeSame}
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return fmt.Errorf("begin merge: %w", err)
}
defer func() { _ = tx.Rollback(ctx) }()
var toAgent *int64
err = tx.QueryRow(ctx, `SELECT agent_id FROM services WHERE id = $1`, toID).Scan(&toAgent)
if errors.Is(err, pgx.ErrNoRows) {
return model.ErrNotFound
}
if err != nil {
return fmt.Errorf("merge: find new service: %w", err)
}
if toAgent != nil {
if err := lockActiveAgent(ctx, tx, *toAgent); err != nil {
return err
}
}
rows, err := tx.Query(ctx, `
SELECT id, kind, agent_id, removed_at IS NOT NULL, self FROM services
WHERE id IN ($1, $2) ORDER BY id FOR UPDATE`, fromID, toID)
if err != nil {
return fmt.Errorf("merge: lock services: %w", err)
}
locked := map[int64]mergeRow{}
for rows.Next() {
var id int64
var r mergeRow
if err := rows.Scan(&id, &r.kind, &r.agentID, &r.removed, &r.self); err != nil {
rows.Close()
return fmt.Errorf("merge: scan service: %w", err)
}
locked[id] = r
}
rows.Close()
if err := rows.Err(); err != nil {
return fmt.Errorf("merge: lock services: %w", err)
}
from, okFrom := locked[fromID]
to, okTo := locked[toID]
if !okFrom || !okTo {
return model.ErrNotFound
}
if (to.agentID == nil) != (toAgent == nil) || (toAgent != nil && *to.agentID != *toAgent) {
return model.ErrConflict
}
if reason := mergeRefusal(from, to); reason != "" {
return &model.MergeRefusedError{Reason: reason}
}
var active bool
if err := tx.QueryRow(ctx, `
SELECT EXISTS (SELECT 1 FROM update_jobs
WHERE service_id IN ($1, $2) AND status IN ('queued', 'running'))`,
fromID, toID).Scan(&active); err != nil {
return fmt.Errorf("merge: check active jobs: %w", err)
}
if active {
return model.ErrJobActive
}
if _, err := tx.Exec(ctx, `UPDATE update_jobs SET service_id = $1 WHERE service_id = $2`, fromID, toID); err != nil {
return fmt.Errorf("merge: move jobs: %w", err)
}
var t model.Service
var lastTagDigest string
err = tx.QueryRow(ctx, `
SELECT agent_id, location_key, compose_project, compose_service, working_dir, config_files, built,
swarm_cluster_id, swarm_service, container_name, image_repo, actual_ref, actual_digest, status, self,
updated_at, available_ref, available_digest, available_source, available_at, last_poll_error,
last_tag_digest
FROM services WHERE id = $1`, toID).Scan(
&t.AgentID, &t.LocationKey, &t.ComposeProject, &t.ComposeService, &t.WorkingDir, &t.ConfigFiles, &t.Built,
&t.SwarmClusterID, &t.SwarmService, &t.ContainerName, &t.ImageRepo, &t.ActualRef, &t.ActualDigest, &t.Status, &t.Self,
&t.UpdatedAt, &t.AvailableRef, &t.AvailableDigest, &t.AvailableSource, &t.AvailableAt, &t.LastPollError,
&lastTagDigest)
if err != nil {
return fmt.Errorf("merge: read new service: %w", err)
}
if _, err := tx.Exec(ctx, `DELETE FROM services WHERE id = $1`, toID); err != nil {
return fmt.Errorf("merge: delete new service: %w", err)
}
if _, err := tx.Exec(ctx, `
UPDATE services SET
agent_id = $2, location_key = $3, compose_project = $4, compose_service = $5, working_dir = $6,
config_files = $7, built = $8, swarm_cluster_id = $9, swarm_service = $10, container_name = $11,
image_repo = $12, actual_ref = $13, actual_digest = $14, status = $15, self = $16, updated_at = $17,
available_ref = $18, available_digest = $19, available_source = $20, available_at = $21,
last_poll_error = $22,
last_tag_digest = CASE WHEN last_tag_digest <> '' THEN last_tag_digest ELSE $23 END,
removed_at = NULL, awaiting_inventory = false
WHERE id = $1`,
fromID, t.AgentID, t.LocationKey, t.ComposeProject, t.ComposeService, t.WorkingDir,
t.ConfigFiles, t.Built, t.SwarmClusterID, t.SwarmService, t.ContainerName,
t.ImageRepo, t.ActualRef, t.ActualDigest, t.Status, t.Self, t.UpdatedAt,
t.AvailableRef, t.AvailableDigest, t.AvailableSource, t.AvailableAt,
t.LastPollError, lastTagDigest); err != nil {
return fmt.Errorf("merge: update old service: %w", err)
}
if err := tx.Commit(ctx); err != nil {
return fmt.Errorf("commit merge: %w", err)
}
return nil
}
// mergeRefusal returns why from (the old row) cannot be merged with to (the
// live one), or "".
func mergeRefusal(from, to mergeRow) string {
switch {
case !from.removed:
return model.ReasonMergeLiveOld
case to.removed:
return model.ReasonMergeGoneNew
case from.kind != to.kind:
return model.ReasonMergeKind
case from.kind == string(protocol.KindContainer):
return model.ReasonMergeContainer
case from.self || to.self:
return model.ReasonSelf
}
return ""
}
@@ -1,20 +0,0 @@
ALTER TABLE services DROP CONSTRAINT services_available_source_check;
ALTER TABLE services ADD CONSTRAINT services_available_source_check
CHECK (available_source IN ('', 'webhook', 'poll'));
ALTER TABLE services ADD COLUMN available_digest text NOT NULL DEFAULT '';
ALTER TABLE services ADD COLUMN last_poll_error text NOT NULL DEFAULT '';
ALTER TABLE services ADD COLUMN last_tag_digest text NOT NULL DEFAULT '';
-- Every version the master has seen. '' means "no tag" / "digest unknown";
-- never NULL, so the unique constraint deduplicates.
CREATE TABLE image_versions (
id bigserial PRIMARY KEY,
repo text NOT NULL,
tag text NOT NULL DEFAULT '',
digest text NOT NULL DEFAULT '',
source text NOT NULL CHECK (source IN ('webhook', 'poll', 'running')),
first_seen_at timestamptz NOT NULL,
CONSTRAINT image_versions_tag_or_digest CHECK (tag <> '' OR digest <> ''),
CONSTRAINT image_versions_unique UNIQUE (repo, tag, digest)
);
CREATE INDEX image_versions_recent ON image_versions (repo, first_seen_at DESC, id DESC);
@@ -1,7 +0,0 @@
-- True from a successful job until the next inventory write for the row. The
-- agent's upsert arrives a debounce interval after job.result, and until then
-- desired_ref is newer than actual_ref without the service having drifted.
ALTER TABLE services ADD COLUMN awaiting_inventory boolean NOT NULL DEFAULT false;
-- Move suggestions look up Compose rows by project and service across agents.
CREATE INDEX services_by_compose_name ON services (compose_project, compose_service) WHERE kind = 'compose';
+10 -143
View File
@@ -54,11 +54,9 @@ ON CONFLICT (location_key) DO UPDATE SET
status = EXCLUDED.status, status = EXCLUDED.status,
self = EXCLUDED.self, self = EXCLUDED.self,
updated_at = EXCLUDED.updated_at, updated_at = EXCLUDED.updated_at,
removed_at = NULL, removed_at = NULL`
awaiting_inventory = false`
func upsertService(ctx context.Context, db execer, agentID int64, clusterID string, svc protocol.Service, at time.Time) error { func upsertService(ctx context.Context, db execer, agentID int64, clusterID string, svc protocol.Service, at time.Time) error {
image := imageref.Parse(svc.Image)
clusterCol := "" clusterCol := ""
if svc.Kind == protocol.KindSwarm { if svc.Kind == protocol.KindSwarm {
clusterCol = clusterID clusterCol = clusterID
@@ -67,34 +65,11 @@ func upsertService(ctx context.Context, db execer, agentID int64, clusterID stri
string(svc.Kind), model.LocationKey(agentID, clusterID, svc.Key()), agentID, string(svc.Kind), model.LocationKey(agentID, clusterID, svc.Key()), agentID,
svc.ComposeProject, svc.ComposeService, svc.WorkingDir, svc.ConfigFiles, svc.Built, svc.ComposeProject, svc.ComposeService, svc.WorkingDir, svc.ConfigFiles, svc.Built,
clusterCol, svc.SwarmService, svc.ContainerName, clusterCol, svc.SwarmService, svc.ContainerName,
image.Repo, svc.Image, svc.Digest, imageref.Parse(svc.Image).Repo, svc.Image, svc.Digest,
svc.Status, svc.Self, at) svc.Status, svc.Self, at)
if err != nil { if err != nil {
return fmt.Errorf("upsert service %s: %w", svc.Key(), err) return fmt.Errorf("upsert service %s: %w", svc.Key(), err)
} }
if imageref.IsDigest(svc.Digest) {
// A bare image runs "latest", as jobs.FollowedRef sees it, so it
// shares its row with the polled one. repo@digest has no tag.
if image.Tag == "" && image.Digest == "" {
image.Tag = "latest"
}
if err := insertVersion(ctx, db, image.Repo, image.Tag, svc.Digest, model.SourceRunning, at); err != nil {
return err
}
}
return nil
}
// insertVersion records a version in image_versions unless the same
// (repo, tag, digest) is already there; the first sighting keeps its source
// and first_seen_at.
func insertVersion(ctx context.Context, db execer, repo, tag, digest, source string, at time.Time) error {
if _, err := db.Exec(ctx, `
INSERT INTO image_versions (repo, tag, digest, source, first_seen_at)
VALUES ($1, $2, $3, $4, $5) ON CONFLICT DO NOTHING`,
repo, tag, digest, source, at); err != nil {
return fmt.Errorf("record version %s: %w", imageref.Ref{Repo: repo, Tag: tag, Digest: digest}, err)
}
return nil return nil
} }
@@ -174,8 +149,8 @@ func (s *Store) RemoveService(ctx context.Context, agentID int64, clusterID, age
const serviceColumns = `s.id, s.kind, s.location_key, s.agent_id, COALESCE(a.name, ''), const serviceColumns = `s.id, s.kind, s.location_key, s.agent_id, COALESCE(a.name, ''),
s.compose_project, s.compose_service, s.working_dir, s.config_files, s.built, s.swarm_cluster_id, s.compose_project, s.compose_service, s.working_dir, s.config_files, s.built, s.swarm_cluster_id,
s.swarm_service, s.container_name, s.image_repo, s.desired_ref, s.actual_ref, s.actual_digest, s.swarm_service, s.container_name, s.image_repo, s.desired_ref, s.actual_ref, s.actual_digest,
s.status, s.self, s.update_mode, s.available_ref, s.available_digest, s.available_source, s.available_at, s.status, s.self, s.update_mode, s.available_ref, s.available_source, s.available_at,
s.last_poll_error, s.last_tag_digest, s.awaiting_inventory, s.first_seen_at, s.updated_at, s.removed_at` s.first_seen_at, s.updated_at, s.removed_at`
func scanService(row pgx.Row) (model.Service, error) { func scanService(row pgx.Row) (model.Service, error) {
var m model.Service var m model.Service
@@ -183,8 +158,8 @@ func scanService(row pgx.Row) (model.Service, error) {
err := row.Scan(&m.ID, &kind, &m.LocationKey, &m.AgentID, &m.AgentName, err := row.Scan(&m.ID, &kind, &m.LocationKey, &m.AgentID, &m.AgentName,
&m.ComposeProject, &m.ComposeService, &m.WorkingDir, &m.ConfigFiles, &m.Built, &m.SwarmClusterID, &m.ComposeProject, &m.ComposeService, &m.WorkingDir, &m.ConfigFiles, &m.Built, &m.SwarmClusterID,
&m.SwarmService, &m.ContainerName, &m.ImageRepo, &m.DesiredRef, &m.ActualRef, &m.ActualDigest, &m.SwarmService, &m.ContainerName, &m.ImageRepo, &m.DesiredRef, &m.ActualRef, &m.ActualDigest,
&m.Status, &m.Self, &m.UpdateMode, &m.AvailableRef, &m.AvailableDigest, &m.AvailableSource, &m.AvailableAt, &m.Status, &m.Self, &m.UpdateMode, &m.AvailableRef, &m.AvailableSource, &m.AvailableAt,
&m.LastPollError, &m.LastTagDigest, &m.AwaitingInventory, &m.FirstSeenAt, &m.UpdatedAt, &m.RemovedAt) &m.FirstSeenAt, &m.UpdatedAt, &m.RemovedAt)
m.Kind = protocol.Kind(kind) m.Kind = protocol.Kind(kind)
return m, err return m, err
} }
@@ -223,96 +198,16 @@ func (s *Store) queryServices(ctx context.Context, what, extraWhere string, args
} }
// SetAvailable records a newer ref for the service, replacing any earlier // SetAvailable records a newer ref for the service, replacing any earlier
// suggestion, and records the version in image_versions in the same // suggestion.
// transaction (the tag is parsed from a.Ref).
func (s *Store) SetAvailable(ctx context.Context, id int64, a model.Available) error { func (s *Store) SetAvailable(ctx context.Context, id int64, a model.Available) error {
tx, err := s.pool.Begin(ctx) if _, err := s.pool.Exec(ctx, `
if err != nil { UPDATE services SET available_ref = $2, available_source = $3, available_at = $4
return fmt.Errorf("set available: begin: %w", err) WHERE id = $1`, id, a.Ref, a.Source, a.At); err != nil {
}
defer func() { _ = tx.Rollback(ctx) }()
if _, err := tx.Exec(ctx, `
UPDATE services SET available_ref = $2, available_digest = $3, available_source = $4, available_at = $5
WHERE id = $1`, id, a.Ref, a.Digest, a.Source, a.At); err != nil {
return fmt.Errorf("set available: %w", err) return fmt.Errorf("set available: %w", err)
} }
ref := imageref.Parse(a.Ref)
if err := insertVersion(ctx, tx, ref.Repo, ref.Tag, a.Digest, a.Source, a.At); err != nil {
return err
}
if err := tx.Commit(ctx); err != nil {
return fmt.Errorf("set available: commit: %w", err)
}
return nil return nil
} }
// SetPollError sets last_poll_error on the given services; msg "" clears it.
func (s *Store) SetPollError(ctx context.Context, ids []int64, msg string) error {
if len(ids) == 0 {
return nil
}
if _, err := s.pool.Exec(ctx,
`UPDATE services SET last_poll_error = $2 WHERE id = ANY($1)`, ids, msg); err != nil {
return fmt.Errorf("set poll error: %w", err)
}
return nil
}
// SetLastTagDigest sets last_tag_digest; see model.Service.LastTagDigest. The
// poller and jobs.RestoreDesired call it, and FinishJob sets the same column
// after a tag job. Inventory writes never change it.
func (s *Store) SetLastTagDigest(ctx context.Context, id int64, digest string) error {
if _, err := s.pool.Exec(ctx,
`UPDATE services SET last_tag_digest = $2 WHERE id = $1`, id, digest); err != nil {
return fmt.Errorf("set last tag digest: %w", err)
}
return nil
}
// RecentVersions returns up to limit versions of repo, most recently first
// seen first. A version without a digest is left out when a version with the
// same tag and a digest was first seen at or after it.
func (s *Store) RecentVersions(ctx context.Context, repo string, limit int) ([]model.ImageVersion, error) {
rows, err := s.pool.Query(ctx, `
SELECT id, repo, tag, digest, source, first_seen_at FROM image_versions v
WHERE v.repo = $1
AND NOT (v.digest = '' AND EXISTS (
SELECT 1 FROM image_versions d
WHERE d.repo = v.repo AND d.tag = v.tag AND d.digest <> ''
AND d.first_seen_at >= v.first_seen_at))
ORDER BY v.first_seen_at DESC, v.id DESC
LIMIT $2`, repo, limit)
if err != nil {
return nil, fmt.Errorf("recent versions: %w", err)
}
defer rows.Close()
var out []model.ImageVersion
for rows.Next() {
var v model.ImageVersion
if err := rows.Scan(&v.ID, &v.Repo, &v.Tag, &v.Digest, &v.Source, &v.FirstSeenAt); err != nil {
return nil, fmt.Errorf("scan version: %w", err)
}
out = append(out, v)
}
return out, rows.Err()
}
// GetImageVersion returns one version. Returns model.ErrNotFound if the ID
// does not exist.
func (s *Store) GetImageVersion(ctx context.Context, id int64) (model.ImageVersion, error) {
var v model.ImageVersion
err := s.pool.QueryRow(ctx, `
SELECT id, repo, tag, digest, source, first_seen_at FROM image_versions WHERE id = $1`, id,
).Scan(&v.ID, &v.Repo, &v.Tag, &v.Digest, &v.Source, &v.FirstSeenAt)
if errors.Is(err, pgx.ErrNoRows) {
return model.ImageVersion{}, model.ErrNotFound
}
if err != nil {
return model.ImageVersion{}, fmt.Errorf("get image version: %w", err)
}
return v, nil
}
// SetUpdateMode sets the service's update mode (model.ModeManual or // SetUpdateMode sets the service's update mode (model.ModeManual or
// model.ModeAuto). Returns model.ErrNotFound if the service does not exist // model.ModeAuto). Returns model.ErrNotFound if the service does not exist
// or is marked removed. // or is marked removed.
@@ -328,34 +223,6 @@ func (s *Store) SetUpdateMode(ctx context.Context, id int64, mode string) error
return nil return nil
} }
// AcceptRunning makes the running version the desired one: desired_ref =
// desired, provided the service is live, has been updated before and still
// reports actualRef and actualDigest (what the caller saw when it decided).
// It starts no job and leaves last_tag_digest, the available suggestion and
// the update mode alone. Returns model.ErrNotFound for an unknown or removed
// service and model.ErrConflict when the service was never updated or the
// running version changed meanwhile.
func (s *Store) AcceptRunning(ctx context.Context, id int64, actualRef, actualDigest, desired string) error {
tag, err := s.pool.Exec(ctx, `
UPDATE services SET desired_ref = $4, awaiting_inventory = false
WHERE id = $1 AND removed_at IS NULL AND desired_ref <> ''
AND actual_ref = $2 AND actual_digest = $3`, id, actualRef, actualDigest, desired)
if err != nil {
return fmt.Errorf("accept running: %w", err)
}
if tag.RowsAffected() == 1 {
return nil
}
svc, err := s.GetService(ctx, id)
if err != nil {
return err
}
if svc.RemovedAt != nil {
return model.ErrNotFound
}
return model.ErrConflict
}
// GetService returns one service, including a removed one (RemovedAt set). // GetService returns one service, including a removed one (RemovedAt set).
// Returns model.ErrNotFound if the ID does not exist. // Returns model.ErrNotFound if the ID does not exist.
func (s *Store) GetService(ctx context.Context, id int64) (model.Service, error) { func (s *Store) GetService(ctx context.Context, id int64) (model.Service, error) {
@@ -17,9 +17,9 @@ import (
var ( var (
t0 = time.Date(2026, 10, 12, 12, 0, 0, 0, time.UTC) t0 = time.Date(2026, 10, 12, 12, 0, 0, 0, time.UTC)
slusk = protocol.Service{Kind: protocol.KindCompose, ComposeProject: "slusk", ComposeService: "web", WorkingDir: "/srv/slusk", Image: "gitea.shcizo.se/shcizo/slusk:1.4.2", Digest: digD1, Status: "running"} slusk = protocol.Service{Kind: protocol.KindCompose, ComposeProject: "slusk", ComposeService: "web", WorkingDir: "/srv/slusk", Image: "gitea.shcizo.se/shcizo/slusk:1.4.2", Digest: "sha256:d1", Status: "running"}
adhoc = protocol.Service{Kind: protocol.KindContainer, ContainerName: "adhoc", Image: "redis:alpine", Status: "stopped"} adhoc = protocol.Service{Kind: protocol.KindContainer, ContainerName: "adhoc", Image: "redis:alpine", Status: "stopped"}
gitea = protocol.Service{Kind: protocol.KindSwarm, SwarmService: "gitea_gitea", Image: "gitea/gitea:1.27.3@sha256:87a6", Digest: digGitea, Status: "running"} gitea = protocol.Service{Kind: protocol.KindSwarm, SwarmService: "gitea_gitea", Image: "gitea/gitea:1.27.3@sha256:87a6", Digest: "sha256:87a6", Status: "running"}
worker = protocol.Hello{Hostname: "node-2"} worker = protocol.Hello{Hostname: "node-2"}
manager = protocol.Hello{Hostname: "swarm-mgr", SwarmManager: true, SwarmClusterID: "c1"} manager = protocol.Hello{Hostname: "swarm-mgr", SwarmManager: true, SwarmClusterID: "c1"}
) )
@@ -53,7 +53,7 @@ func TestApplySnapshot_InsertsAsManualWithAgentName(t *testing.T) {
assert.Equal(t, "node-2", got[0].AgentName) assert.Equal(t, "node-2", got[0].AgentName)
assert.Equal(t, "gitea.shcizo.se/shcizo/slusk", got[0].ImageRepo) assert.Equal(t, "gitea.shcizo.se/shcizo/slusk", got[0].ImageRepo)
assert.Equal(t, "gitea.shcizo.se/shcizo/slusk:1.4.2", got[0].ActualRef) assert.Equal(t, "gitea.shcizo.se/shcizo/slusk:1.4.2", got[0].ActualRef)
assert.Equal(t, digD1, got[0].ActualDigest) assert.Equal(t, "sha256:d1", got[0].ActualDigest)
assert.Equal(t, "compose|"+itoa(id)+"|slusk|web", got[0].LocationKey) assert.Equal(t, "compose|"+itoa(id)+"|slusk|web", got[0].LocationKey)
} }
@@ -1,216 +0,0 @@
//go:build integration
package store
import (
"context"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/shcizo/package-updater/internal/master/model"
"github.com/shcizo/package-updater/internal/protocol"
)
const sluskRepo = "gitea.shcizo.se/shcizo/slusk"
// Digests of the fixture services; only well-formed ones are recorded as versions.
var (
digD1 = "sha256:" + strings.Repeat("d", 63) + "1"
digGitea = "sha256:" + strings.Repeat("a", 63) + "1"
digPg = "sha256:" + strings.Repeat("b", 63) + "1"
)
func versionsOf(t *testing.T, s *Store, repo string) []model.ImageVersion {
t.Helper()
got, err := s.RecentVersions(context.Background(), repo, 100)
require.NoError(t, err)
return got
}
func TestSetAvailable_RecordsVersion(t *testing.T) {
s := openTestStore(t)
ctx := context.Background()
_, svc := jobFixture(t, s, "a")
require.NoError(t, s.SetAvailable(ctx, svc, model.Available{Ref: sluskRepo + ":1.5.0", Digest: "sha256:n1", Source: model.SourcePoll, At: t0}))
got, err := s.GetService(ctx, svc)
require.NoError(t, err)
assert.Equal(t, sluskRepo+":1.5.0", got.AvailableRef)
assert.Equal(t, "sha256:n1", got.AvailableDigest)
assert.Equal(t, model.SourcePoll, got.AvailableSource)
vs := versionsOf(t, s, sluskRepo)
var found model.ImageVersion
for _, v := range vs {
if v.Digest == "sha256:n1" {
found = v
}
}
require.NotZero(t, found.ID)
assert.Equal(t, "1.5.0", found.Tag)
assert.Equal(t, model.SourcePoll, found.Source)
// Seeing the same version again keeps the first sighting.
require.NoError(t, s.SetAvailable(ctx, svc, model.Available{Ref: sluskRepo + ":1.5.0", Digest: "sha256:n1", Source: model.SourceWebhook, At: t0.Add(time.Hour)}))
again, err := s.GetImageVersion(ctx, found.ID)
require.NoError(t, err)
assert.Equal(t, model.SourcePoll, again.Source)
assert.True(t, t0.Equal(again.FirstSeenAt))
assert.Len(t, versionsOf(t, s, sluskRepo), len(vs))
}
func TestSnapshotRecordsRunningVersions(t *testing.T) {
s := openTestStore(t)
ctx := context.Background()
id := newAgent(t, s, "node-2")
tagless := protocol.Service{Kind: protocol.KindSwarm, SwarmService: "g_g", Image: "gitea/gitea@" + digGitea, Digest: digGitea, Status: "running"}
bare := protocol.Service{Kind: protocol.KindContainer, ContainerName: "pg", Image: "postgres", Digest: digPg, Status: "running"}
nodigest := protocol.Service{Kind: protocol.KindContainer, ContainerName: "nd", Image: "nodigest:1", Status: "running"}
hello := protocol.Hello{Hostname: "node-2", SwarmManager: true, SwarmClusterID: "c1"}
svcs := []protocol.Service{slusk, tagless, bare, nodigest}
require.NoError(t, s.ApplySnapshot(ctx, id, hello, svcs, t0))
require.NoError(t, s.ApplySnapshot(ctx, id, hello, svcs, t0.Add(time.Hour)))
got := versionsOf(t, s, sluskRepo)
require.Len(t, got, 1)
assert.Equal(t, "1.4.2", got[0].Tag)
assert.Equal(t, digD1, got[0].Digest)
assert.Equal(t, model.SourceRunning, got[0].Source)
assert.True(t, t0.Equal(got[0].FirstSeenAt), "re-snapshot does not bump first_seen_at")
tl := versionsOf(t, s, "gitea/gitea")
require.Len(t, tl, 1)
assert.Equal(t, "", tl[0].Tag)
assert.Equal(t, digGitea, tl[0].Digest)
b := versionsOf(t, s, "postgres")
require.Len(t, b, 1)
assert.Equal(t, "latest", b[0].Tag, "a bare image runs latest, like the followed ref")
assert.Empty(t, versionsOf(t, s, "nodigest"))
}
func TestSnapshotRecordsOnlyWellFormedDigests(t *testing.T) {
s := openTestStore(t)
id := newAgent(t, s, "node-2")
bad := protocol.Service{Kind: protocol.KindContainer, ContainerName: "bad", Image: "bad:1", Digest: "sha256:short", Status: "running"}
upper := protocol.Service{Kind: protocol.KindContainer, ContainerName: "up", Image: "up:1", Digest: "sha256:" + strings.Repeat("A", 64), Status: "running"}
require.NoError(t, s.ApplySnapshot(context.Background(), id, protocol.Hello{Hostname: "node-2"}, []protocol.Service{bad, upper}, t0))
assert.Empty(t, versionsOf(t, s, "bad"))
assert.Empty(t, versionsOf(t, s, "up"))
}
func TestRecentVersions(t *testing.T) {
ctx := context.Background()
t.Run("order, limit and repo filter", func(t *testing.T) {
s := openTestStore(t)
_, svc := jobFixture(t, s, "a")
require.NoError(t, s.SetAvailable(ctx, svc, model.Available{Ref: "other/repo:x", Digest: "sha256:ox", Source: model.SourcePoll, At: t0}))
// 12 versions of the service's repo, one per minute.
for i := 0; i < 12; i++ {
tag := "t" + string(rune('a'+i))
require.NoError(t, s.SetAvailable(ctx, svc, model.Available{Ref: sluskRepo + ":" + tag, Digest: "sha256:" + tag, Source: model.SourcePoll, At: t0.Add(time.Duration(i+1) * time.Minute)}))
}
got, err := s.RecentVersions(ctx, sluskRepo, 10)
require.NoError(t, err)
require.Len(t, got, 10)
assert.Equal(t, "tl", got[0].Tag, "newest first")
assert.Equal(t, "tc", got[9].Tag)
for _, v := range got {
assert.Equal(t, sluskRepo, v.Repo)
}
})
t.Run("digest-less row is hidden once its tag has a digest", func(t *testing.T) {
s := openTestStore(t)
_, svc := jobFixture(t, s, "a")
add := func(tag, digest string, at time.Time) {
require.NoError(t, s.SetAvailable(ctx, svc, model.Available{Ref: sluskRepo + ":" + tag, Digest: digest, Source: model.SourceWebhook, At: at}))
}
add("main", "", t0.Add(time.Minute))
add("solo", "", t0.Add(2*time.Minute))
require.Len(t, versionsOf(t, s, sluskRepo), 3, "running 1.4.2 plus two digest-less rows")
add("main", "sha256:m1", t0.Add(3*time.Minute))
var refs []string
for _, v := range versionsOf(t, s, sluskRepo) {
refs = append(refs, v.Ref())
}
assert.Equal(t, []string{sluskRepo + ":main@sha256:m1", sluskRepo + ":solo", sluskRepo + ":1.4.2@" + digD1}, refs)
})
t.Run("digest-less row newer than the digest row of its tag is shown", func(t *testing.T) {
s := openTestStore(t)
_, svc := jobFixture(t, s, "a")
add := func(tag, digest string, at time.Time) {
require.NoError(t, s.SetAvailable(ctx, svc, model.Available{Ref: sluskRepo + ":" + tag, Digest: digest, Source: model.SourceWebhook, At: at}))
}
add("main", "sha256:m1", t0.Add(time.Minute))
add("main", "", t0.Add(2*time.Minute))
var refs []string
for _, v := range versionsOf(t, s, sluskRepo) {
refs = append(refs, v.Ref())
}
assert.Equal(t, []string{sluskRepo + ":main", sluskRepo + ":main@sha256:m1", sluskRepo + ":1.4.2@" + digD1}, refs)
})
}
func TestGetImageVersion_NotFound(t *testing.T) {
s := openTestStore(t)
_, err := s.GetImageVersion(context.Background(), 99999)
assert.ErrorIs(t, err, model.ErrNotFound)
}
func TestSetPollError(t *testing.T) {
s := openTestStore(t)
ctx := context.Background()
agent, svc := jobFixture(t, s, "a")
require.NoError(t, s.SetAvailable(ctx, svc, model.Available{Ref: sluskRepo + ":1.5.0", Digest: "sha256:n1", Source: model.SourcePoll, At: t0}))
require.NoError(t, s.SetPollError(ctx, []int64{svc}, "boom"))
require.NoError(t, s.SetPollError(ctx, nil, "ignored"))
got, err := s.GetService(ctx, svc)
require.NoError(t, err)
assert.Equal(t, "boom", got.LastPollError)
require.NoError(t, s.UpsertService(ctx, agent, "", slusk, t0.Add(time.Hour)))
got, err = s.GetService(ctx, svc)
require.NoError(t, err)
assert.Equal(t, "boom", got.LastPollError, "inventory writes do not touch the poll error")
assert.Equal(t, "sha256:n1", got.AvailableDigest, "nor the suggestion")
require.NoError(t, s.SetPollError(ctx, []int64{svc}, ""))
got, err = s.GetService(ctx, svc)
require.NoError(t, err)
assert.Empty(t, got.LastPollError)
}
func TestSetLastTagDigest(t *testing.T) {
s := openTestStore(t)
ctx := context.Background()
agent, svc := jobFixture(t, s, "a")
got, err := s.GetService(ctx, svc)
require.NoError(t, err)
assert.Empty(t, got.LastTagDigest)
require.NoError(t, s.SetLastTagDigest(ctx, svc, "sha256:n1"))
require.NoError(t, s.UpsertService(ctx, agent, "", slusk, t0.Add(time.Hour)))
got, err = s.GetService(ctx, svc)
require.NoError(t, err)
assert.Equal(t, "sha256:n1", got.LastTagDigest, "inventory writes do not touch it")
list, err := s.ListServices(ctx)
require.NoError(t, err)
require.Len(t, list, 1)
assert.Equal(t, "sha256:n1", list[0].LastTagDigest)
}
@@ -4,7 +4,6 @@ package store
import ( import (
"context" "context"
"strings"
"testing" "testing"
"time" "time"
@@ -92,34 +91,27 @@ func TestUpsertKeepsUpdateMode(t *testing.T) {
func TestFinishJob_ClearsAvailableOnlyWhenThatVersionWasDeployed(t *testing.T) { func TestFinishJob_ClearsAvailableOnlyWhenThatVersionWasDeployed(t *testing.T) {
succeed := model.JobFinish{Status: model.JobSucceeded, Ref: "r:2@sha256:d", StartedAt: t0, FinishedAt: t0} succeed := model.JobFinish{Status: model.JobSucceeded, Ref: "r:2@sha256:d", StartedAt: t0, FinishedAt: t0}
tests := []struct { tests := []struct {
name string name string
availableRef string availableRef string
availableDigest string availableAt time.Time // job is created at t0 for r:2
availableAt time.Time // the job is created at t0 finish model.JobFinish
toRef string // the job's to_ref as created wantCleared bool
finish model.JobFinish
wantCleared bool
}{ }{
{"same ref, suggestion before the job", "r:2", "", t0.Add(-time.Minute), "r:2", succeed, true}, {"same ref, suggestion before the job", "r:2", t0.Add(-time.Minute), succeed, true},
{"same ref, suggestion at the same instant", "r:2", "", t0, "r:2", succeed, true}, {"same ref, suggestion at the same instant", "r:2", t0, succeed, true},
{"same ref, suggestion after the job was created", "r:2", "", t0.Add(time.Minute), "r:2", succeed, false}, {"same ref, suggestion after the job was created", "r:2", t0.Add(time.Minute), succeed, false},
{"different ref keeps the suggestion", "r:3", "", t0.Add(-time.Minute), "r:2", succeed, false}, {"different ref keeps the suggestion", "r:3", t0.Add(-time.Minute), succeed, false},
{"failed job keeps the suggestion", "r:2", "", t0.Add(-time.Minute), "r:2", {"failed job keeps the suggestion", "r:2", t0.Add(-time.Minute),
model.JobFinish{Status: model.JobFailed, Error: "boom", StartedAt: t0, FinishedAt: t0}, false}, model.JobFinish{Status: model.JobFailed, Error: "boom", StartedAt: t0, FinishedAt: t0}, false},
{"tag job clears a poll suggestion with a digest", "r:2", "sha256:new", t0.Add(-time.Minute), "r:2", succeed, true},
{"pick of the suggested digest clears it", "r:2", "sha256:new", t0.Add(-time.Minute), "r:2@sha256:new", succeed, true},
{"rollback to an older digest keeps the newer suggestion", "r:2", "sha256:new", t0.Add(-time.Minute), "r:2@sha256:old", succeed, false},
{"digest pick keeps a suggestion without digest", "r:2", "", t0.Add(-time.Minute), "r:2@sha256:old", succeed, false},
{"tagless pick keeps the suggestion", "r:2", "sha256:new", t0.Add(-time.Minute), "r@sha256:old", succeed, false},
} }
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
s := openTestStore(t) s := openTestStore(t)
ctx := context.Background() ctx := context.Background()
agent, svc := jobFixture(t, s, "a") agent, svc := jobFixture(t, s, "a")
require.NoError(t, s.SetAvailable(ctx, svc, model.Available{Ref: tt.availableRef, Digest: tt.availableDigest, Source: model.SourcePoll, At: tt.availableAt})) require.NoError(t, s.SetAvailable(ctx, svc, model.Available{Ref: tt.availableRef, Source: model.SourceWebhook, At: tt.availableAt}))
nj := newJob(svc, agent) nj := newJob(svc, agent)
nj.ToRef = tt.toRef nj.ToRef = "r:2"
j, err := s.CreateJob(ctx, nj) j, err := s.CreateJob(ctx, nj)
require.NoError(t, err) require.NoError(t, err)
@@ -131,13 +123,11 @@ func TestFinishJob_ClearsAvailableOnlyWhenThatVersionWasDeployed(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
if tt.wantCleared { if tt.wantCleared {
assert.Empty(t, got.AvailableRef) assert.Empty(t, got.AvailableRef)
assert.Empty(t, got.AvailableDigest)
assert.Empty(t, got.AvailableSource) assert.Empty(t, got.AvailableSource)
assert.Nil(t, got.AvailableAt) assert.Nil(t, got.AvailableAt)
} else { } else {
assert.Equal(t, tt.availableRef, got.AvailableRef) assert.Equal(t, tt.availableRef, got.AvailableRef)
assert.Equal(t, tt.availableDigest, got.AvailableDigest) assert.Equal(t, model.SourceWebhook, got.AvailableSource)
assert.Equal(t, model.SourcePoll, got.AvailableSource)
assert.NotNil(t, got.AvailableAt) assert.NotNil(t, got.AvailableAt)
} }
}) })
@@ -161,48 +151,3 @@ func TestFinishJob_WithoutAvailableLeavesItEmpty(t *testing.T) {
assert.Nil(t, got.AvailableAt) assert.Nil(t, got.AvailableAt)
assert.Equal(t, "r:1@sha256:d", got.DesiredRef) assert.Equal(t, "r:1@sha256:d", got.DesiredRef)
} }
func TestFinishJob_RemembersTheDeployedDigestOfATagJob(t *testing.T) {
good := "sha256:" + strings.Repeat("a", 64)
old := "sha256:" + strings.Repeat("b", 64)
ok := func(ref string) model.JobFinish {
return model.JobFinish{Status: model.JobSucceeded, Ref: ref, StartedAt: t0, FinishedAt: t0}
}
tests := []struct {
name string
trigger model.JobTrigger
toRef string
finish model.JobFinish
want string
}{
{"webhook tag job", model.TriggerWebhook, "r:2", ok("r:2@" + good), good},
{"manual tag job", model.TriggerManual, "r:2", ok("r:2@" + good), good},
{"poll tag job", model.TriggerPoll, "r:2", ok("r:2@" + good), good},
{"pinned pick leaves it", model.TriggerManual, "r:2@" + old, ok("r:2@" + old), "sha256:before"},
{"tagless pick leaves it", model.TriggerManual, "r@" + old, ok("r@" + old), "sha256:before"},
{"failed job leaves it", model.TriggerWebhook, "r:2",
model.JobFinish{Status: model.JobFailed, Error: "boom", Ref: "r:2@" + good, StartedAt: t0, FinishedAt: t0}, "sha256:before"},
{"malformed digest leaves it", model.TriggerWebhook, "r:2", ok("r:2@sha256:d"), "sha256:before"},
{"no pinned ref leaves it", model.TriggerWebhook, "r:2", ok(""), "sha256:before"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
s := openTestStore(t)
ctx := context.Background()
agent, svc := jobFixture(t, s, "a")
require.NoError(t, s.SetLastTagDigest(ctx, svc, "sha256:before"))
nj := newJob(svc, agent)
nj.ToRef, nj.Trigger = tt.toRef, tt.trigger
j, err := s.CreateJob(ctx, nj)
require.NoError(t, err)
done, err := s.FinishJob(ctx, agent, j.ID, tt.finish)
require.NoError(t, err)
require.True(t, done)
got, err := s.GetService(ctx, svc)
require.NoError(t, err)
assert.Equal(t, tt.want, got.LastTagDigest)
})
}
}
-297
View File
@@ -1,297 +0,0 @@
package web
import (
"context"
"io"
"log/slog"
"net/http"
"net/http/httptest"
"net/url"
"strconv"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/shcizo/package-updater/internal/master/events"
"github.com/shcizo/package-updater/internal/master/model"
"github.com/shcizo/package-updater/internal/protocol"
)
const (
driftDesired = "sha256:1111111111111111111111111111111111111111111111111111111111111111"
driftActual = "sha256:2222222222222222222222222222222222222222222222222222222222222222"
)
type acceptCall struct {
id int64
actualRef, actualDigest, desiredRef string
}
func (f *fakeStore) AcceptRunning(_ context.Context, id int64, actualRef, actualDigest, desired string) error {
f.mu.Lock()
defer f.mu.Unlock()
f.acceptCalls = append(f.acceptCalls, acceptCall{id, actualRef, actualDigest, desired})
return f.acceptErr
}
func (f *fakeStore) MoveCandidates(context.Context) ([]model.Move, error) { return f.moves, nil }
func (f *fakeStore) MergeSources(context.Context, int64) ([]model.Service, error) {
return f.sources, nil
}
func (f *fakeStore) MergeService(_ context.Context, from, to int64) error {
f.mu.Lock()
defer f.mu.Unlock()
f.merged = append(f.merged, [2]int64{from, to})
return f.mergeErr
}
func (f *fakeJobs) Restorable(model.Service) error { return f.restorable }
func (f *fakeJobs) RestoreDesired(_ context.Context, serviceID int64, by string) (model.Job, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.requestedBy = by
f.restored = append(f.restored, serviceID)
return f.job, f.err
}
// driftService makes the first fixture service a drifted one: updated to
// 1.4.2 at driftDesired, now running 1.4.2 at driftActual.
func driftService(store *fakeStore) {
store.services[0].DesiredRef = "gitea.shcizo.se/shcizo/slusk:1.4.2@" + driftDesired
store.services[0].ActualDigest = driftActual
}
func postTo(t *testing.T, srv *httptest.Server, path string, form url.Values) (*http.Response, string) {
t.Helper()
form.Set("csrf_token", "csrf")
resp, err := noRedirectClient().PostForm(srv.URL+path, form)
require.NoError(t, err)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
return resp, string(body)
}
func TestRestoreDesired(t *testing.T) {
tests := []struct {
name string
job model.Job
err error
path string
wantCode int
wantLocation string
wantBody string
}{
{name: "started", job: model.Job{ID: 42}, path: "/services/1/restore",
wantCode: http.StatusSeeOther, wantLocation: "/jobs/42"},
{name: "already running", job: model.Job{ID: 41}, err: model.ErrJobActive, path: "/services/1/restore",
wantCode: http.StatusSeeOther, wantLocation: "/jobs/41"},
{name: "refused", err: &model.NotUpdatableError{Reason: model.ReasonNotDrifted}, path: "/services/1/restore",
wantCode: http.StatusConflict, wantBody: "Cannot restore: runs its desired version."},
{name: "unknown service", err: model.ErrNotFound, path: "/services/1/restore", wantCode: http.StatusNotFound},
{name: "invalid id", path: "/services/x/restore", wantCode: http.StatusBadRequest},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
srv, _, _, _, jobs := newTestEnv(t)
jobs.job, jobs.err = tt.job, tt.err
resp, body := postTo(t, srv, tt.path, url.Values{})
assert.Equal(t, tt.wantCode, resp.StatusCode)
assert.Equal(t, tt.wantLocation, resp.Header.Get("Location"))
assert.Contains(t, body, tt.wantBody)
if tt.wantCode != http.StatusBadRequest {
assert.Equal(t, "alice", jobs.triggeredBy())
}
})
}
}
func TestAcceptRunning_WritesRunningRefAndPublishes(t *testing.T) {
srv, store, _, broker, _ := newTestEnv(t)
driftService(store)
ch, cancel := broker.Subscribe()
defer cancel()
resp, _ := postTo(t, srv, "/services/1/accept", url.Values{"actual": {"gitea.shcizo.se/shcizo/slusk:1.4.2@" + driftActual}})
assert.Equal(t, http.StatusSeeOther, resp.StatusCode)
assert.Equal(t, "/services/1", resp.Header.Get("Location"))
assert.Equal(t, []acceptCall{{
id: 1, actualRef: "gitea.shcizo.se/shcizo/slusk:1.4.2", actualDigest: driftActual,
desiredRef: "gitea.shcizo.se/shcizo/slusk:1.4.2@" + driftActual,
}}, store.acceptCalls, "the value written comes from the row, not the form")
select {
case topic := <-ch:
assert.Equal(t, events.Services, topic)
case <-time.After(time.Second):
t.Fatal("no services signal")
}
}
func TestAcceptRunning_ReturnToOverview(t *testing.T) {
srv, store, _, _, _ := newTestEnv(t)
driftService(store)
resp, _ := postTo(t, srv, "/services/1/accept",
url.Values{"actual": {"gitea.shcizo.se/shcizo/slusk:1.4.2@" + driftActual}, "return": {"overview"}})
assert.Equal(t, http.StatusSeeOther, resp.StatusCode)
assert.Equal(t, "/", resp.Header.Get("Location"))
}
func TestAcceptRunning_Refusals(t *testing.T) {
tests := []struct {
name string
path string
form url.Values
notDrifted bool
removed bool
acceptErr error
wantCode int
wantBody string
}{
{name: "missing actual", path: "/services/1/accept", form: url.Values{}, wantCode: http.StatusBadRequest},
{name: "stale actual", path: "/services/1/accept", form: url.Values{"actual": {"gitea.shcizo.se/shcizo/slusk:1.3.0"}},
wantCode: http.StatusConflict, wantBody: "The running version changed; check it again."},
{name: "same ref, other digest", path: "/services/1/accept",
form: url.Values{"actual": {"gitea.shcizo.se/shcizo/slusk:1.4.2@" + driftDesired}},
wantCode: http.StatusConflict, wantBody: "The running version changed; check it again."},
{name: "not drifted", path: "/services/1/accept", form: url.Values{"actual": {"gitea.shcizo.se/shcizo/slusk:1.4.2@" + driftActual}},
notDrifted: true, wantCode: http.StatusConflict, wantBody: "Cannot accept: the service runs its desired version."},
{name: "changed under our feet", path: "/services/1/accept", form: url.Values{"actual": {"gitea.shcizo.se/shcizo/slusk:1.4.2@" + driftActual}},
acceptErr: model.ErrConflict, wantCode: http.StatusConflict, wantBody: "The running version changed; check it again."},
{name: "unknown service", path: "/services/9/accept", form: url.Values{"actual": {"x"}}, wantCode: http.StatusNotFound},
{name: "removed", path: "/services/1/accept", form: url.Values{"actual": {"gitea.shcizo.se/shcizo/slusk:1.4.2@" + driftActual}},
removed: true, wantCode: http.StatusNotFound},
{name: "removed meanwhile", path: "/services/1/accept", form: url.Values{"actual": {"gitea.shcizo.se/shcizo/slusk:1.4.2@" + driftActual}},
acceptErr: model.ErrNotFound, wantCode: http.StatusNotFound},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
srv, store, _, _, _ := newTestEnv(t)
if !tt.notDrifted {
driftService(store)
}
if tt.removed {
removedAt := time.Now()
store.services[0].RemovedAt = &removedAt
}
store.acceptErr = tt.acceptErr
resp, body := postTo(t, srv, tt.path, tt.form)
assert.Equal(t, tt.wantCode, resp.StatusCode)
assert.Contains(t, body, tt.wantBody)
if tt.acceptErr == nil {
assert.Empty(t, store.acceptCalls)
}
})
}
}
func TestMergeService_RedirectsToTheSurvivorAndPublishes(t *testing.T) {
srv, store, _, broker, _ := newTestEnv(t)
ch, cancel := broker.Subscribe()
defer cancel()
resp, _ := postTo(t, srv, "/services/1/merge", url.Values{"from": {"17"}})
assert.Equal(t, http.StatusSeeOther, resp.StatusCode)
assert.Equal(t, "/services/17", resp.Header.Get("Location"))
assert.Equal(t, [][2]int64{{17, 1}}, store.merged)
var got []events.Topic
for len(got) < 2 {
select {
case topic := <-ch:
got = append(got, topic)
case <-time.After(time.Second):
t.Fatalf("signals so far: %v", got)
}
}
assert.ElementsMatch(t, []events.Topic{events.Services, events.Jobs}, got)
}
func TestMergeService_Refusals(t *testing.T) {
tests := []struct {
name string
id, from string
err error
wantCode int
wantBody string
}{
{name: "invalid from", id: "1", from: "x", wantCode: http.StatusBadRequest},
{name: "empty from", id: "1", from: "", wantCode: http.StatusBadRequest},
{name: "refused", id: "1", from: "17", err: &model.MergeRefusedError{Reason: model.ReasonMergeLiveOld},
wantCode: http.StatusConflict, wantBody: "Cannot merge: the old service still exists."},
{name: "job active", id: "1", from: "17", err: model.ErrJobActive,
wantCode: http.StatusConflict, wantBody: "Cannot merge: an update is already in progress for this service."},
{name: "agent revoked", id: "1", from: "17", err: model.ErrRevoked, wantCode: http.StatusConflict, wantBody: "Cannot merge:"},
{name: "not found", id: "1", from: "17", err: model.ErrNotFound, wantCode: http.StatusNotFound},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
srv, store, _, _, _ := newTestEnv(t)
store.mergeErr = tt.err
resp, body := postTo(t, srv, "/services/"+tt.id+"/merge", url.Values{"from": {tt.from}})
assert.Equal(t, tt.wantCode, resp.StatusCode)
assert.Contains(t, body, tt.wantBody)
assert.Empty(t, resp.Header.Get("Location"))
})
}
}
// serviceDetail's data is what the service template reads.
func TestServiceDetail_DriftAndMoveData(t *testing.T) {
removed := time.Now()
live := model.Service{ID: 1, Kind: protocol.KindCompose, ComposeProject: "slusk", ComposeService: "web", ActualRef: "slusk:1"}
gone := model.Service{ID: 2, Kind: protocol.KindCompose, ComposeProject: "slusk", ComposeService: "web", ActualRef: "slusk:1", RemovedAt: &removed}
swarm := model.Service{ID: 3, Kind: protocol.KindSwarm, SwarmService: "gitea", ActualRef: "gitea:1"}
container := model.Service{ID: 4, Kind: protocol.KindContainer, ContainerName: "adhoc", ActualRef: "redis:7"}
self := model.Service{ID: 5, Kind: protocol.KindCompose, ComposeProject: "pu", ComposeService: "agent", ActualRef: "pu:1", Self: true}
move := model.Move{FromID: 2, ToID: 1, FromAgent: "old", ToAgent: "new"}
sources := []model.Service{gone}
tests := []struct {
name string
id int64
restorable error
wantBlocked string
wantFrom *model.Move
wantTo *model.Move
wantSources []model.Service
}{
{name: "live row gets the suggestion and merge choices", id: 1, wantFrom: &move, wantSources: sources},
{name: "removed row gets where it went", id: 2, wantTo: &move},
{name: "swarm service can merge, no suggestion", id: 3, wantSources: sources},
{name: "container cannot merge", id: 4},
{name: "self cannot merge", id: 5},
{name: "restore refusal is passed on", id: 1, restorable: &model.NotUpdatableError{Reason: model.ReasonSelf},
wantBlocked: model.ReasonSelf, wantFrom: &move, wantSources: sources},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
store := &fakeStore{services: []model.Service{live, gone, swarm, container, self},
moves: []model.Move{move}, sources: sources}
jobs := &fakeJobs{restorable: tt.restorable}
s, err := New(store, &fakeHub{}, jobs, events.NewBroker(), Config{}, slog.New(slog.NewTextHandler(io.Discard, nil)))
require.NoError(t, err)
req := httptest.NewRequest(http.MethodGet, "/services/"+strconv.FormatInt(tt.id, 10), nil)
data, err := s.serviceDetail(req, tt.id)
require.NoError(t, err)
assert.Equal(t, tt.wantBlocked, data.RestoreBlocked)
assert.Equal(t, tt.wantFrom, data.MoveFrom)
assert.Equal(t, tt.wantTo, data.MoveTo)
assert.Equal(t, tt.wantSources, data.MergeSources)
})
}
}
+25 -214
View File
@@ -18,7 +18,6 @@ import (
"github.com/shcizo/package-updater/internal/master/events" "github.com/shcizo/package-updater/internal/master/events"
"github.com/shcizo/package-updater/internal/master/model" "github.com/shcizo/package-updater/internal/master/model"
"github.com/shcizo/package-updater/internal/master/webauth" "github.com/shcizo/package-updater/internal/master/webauth"
"github.com/shcizo/package-updater/internal/protocol"
) )
//go:embed templates/*.html static/* //go:embed templates/*.html static/*
@@ -27,30 +26,14 @@ var assets embed.FS
// serviceJobsLimit is how many jobs the service page lists. // serviceJobsLimit is how many jobs the service page lists.
const serviceJobsLimit = 20 const serviceJobsLimit = 20
// versionsLimit is how many image versions the service page lists.
const versionsLimit = 10
var agentNamePattern = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,62}$`) var agentNamePattern = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,62}$`)
// Store is the storage the UI reads and writes. *store.Store satisfies it. // Store is the storage the UI reads and writes. *store.Store satisfies it.
type Store interface { type Store interface {
ListServices(ctx context.Context) ([]model.Service, error) ListServices(ctx context.Context) ([]model.Service, error)
GetService(ctx context.Context, id int64) (model.Service, error) GetService(ctx context.Context, id int64) (model.Service, error)
// RecentVersions returns up to limit versions of repo the master has
// seen, most recently first seen first.
RecentVersions(ctx context.Context, repo string, limit int) ([]model.ImageVersion, error)
// SetUpdateMode returns model.ErrNotFound for an unknown or removed service. // SetUpdateMode returns model.ErrNotFound for an unknown or removed service.
SetUpdateMode(ctx context.Context, id int64, mode string) error SetUpdateMode(ctx context.Context, id int64, mode string) error
// AcceptRunning sets the desired ref of a drifted service; model.ErrNotFound
// for an unknown or removed service, model.ErrConflict when the running
// version is no longer actualRef@actualDigest.
AcceptRunning(ctx context.Context, id int64, actualRef, actualDigest, desired string) error
// MoveCandidates returns the suggested moves of removed services.
MoveCandidates(ctx context.Context) ([]model.Move, error)
// MergeSources returns the removed services that could be merged into toID.
MergeSources(ctx context.Context, toID int64) ([]model.Service, error)
// MergeService moves the live service toID onto the removed fromID.
MergeService(ctx context.Context, fromID, toID int64) error
GetJob(ctx context.Context, id int64) (model.Job, error) GetJob(ctx context.Context, id int64) (model.Job, error)
ListJobs(ctx context.Context, f model.JobFilter) ([]model.Job, error) ListJobs(ctx context.Context, f model.JobFilter) ([]model.Job, error)
ListAgents(ctx context.Context) ([]model.Agent, error) ListAgents(ctx context.Context) ([]model.Agent, error)
@@ -71,13 +54,6 @@ type Jobs interface {
Updatable(svc model.Service) error Updatable(svc model.Service) error
AutoAllowed(svc model.Service) error AutoAllowed(svc model.Service) error
Trigger(ctx context.Context, serviceID int64, requestedBy string) (model.Job, error) Trigger(ctx context.Context, serviceID int64, requestedBy string) (model.Job, error)
// TriggerVersion updates to exactly the image version versionID of the
// service's repo; model.ErrNotFound for an unknown service or version.
TriggerVersion(ctx context.Context, serviceID, versionID int64, requestedBy string) (model.Job, error)
// Restorable explains why a drifted service cannot be restored.
Restorable(svc model.Service) error
// RestoreDesired deploys the service's desired ref again.
RestoreDesired(ctx context.Context, serviceID int64, requestedBy string) (model.Job, error)
} }
// Broker delivers change signals to open streams and accepts signals from // Broker delivers change signals to open streams and accepts signals from
@@ -87,12 +63,11 @@ type Broker interface {
Publish(t events.Topic) Publish(t events.Topic)
} }
// Config holds values shown in the "add agent" snippet and on the service page. // Config holds values shown in the "add agent" snippet.
type Config struct { type Config struct {
AgentURL string // what agents dial, e.g. wss://10.0.0.5:8090/agent AgentURL string // what agents dial, e.g. wss://10.0.0.5:8090/agent
MasterKey string // the master's fingerprint MasterKey string // the master's fingerprint
AgentImage string AgentImage string
PollInterval time.Duration // shown as "every <interval>" on the service page
} }
// Server renders the UI. // Server renders the UI.
@@ -110,7 +85,7 @@ type Server struct {
// New parses the templates. Each page is its own template set so every page // New parses the templates. Each page is its own template set so every page
// can define its own "content" block. // can define its own "content" block.
func New(store Store, hub Hub, jobs Jobs, broker Broker, cfg Config, log *slog.Logger) (*Server, error) { func New(store Store, hub Hub, jobs Jobs, broker Broker, cfg Config, log *slog.Logger) (*Server, error) {
funcs := template.FuncMap{"shortRef": shortRef, "duration": jobDuration, "followsNoTag": followsNoTag} funcs := template.FuncMap{"shortRef": shortRef, "duration": jobDuration}
pages := map[string]*template.Template{} pages := map[string]*template.Template{}
for _, page := range []string{"overview.html", "agents.html", "agent_created.html", "service.html", "jobs.html", "job.html"} { for _, page := range []string{"overview.html", "agents.html", "agent_created.html", "service.html", "jobs.html", "job.html"} {
t, err := template.New(page).Funcs(funcs).ParseFS(assets, "templates/layout.html", "templates/"+page) t, err := template.New(page).Funcs(funcs).ParseFS(assets, "templates/layout.html", "templates/"+page)
@@ -133,9 +108,6 @@ func (s *Server) Register(mux *http.ServeMux, protect func(http.Handler) http.Ha
route("GET /fragments/services/{id}/jobs", s.serviceJobsFragment) route("GET /fragments/services/{id}/jobs", s.serviceJobsFragment)
route("POST /services/{id}/update", s.updateService) route("POST /services/{id}/update", s.updateService)
route("POST /services/{id}/mode", s.setMode) route("POST /services/{id}/mode", s.setMode)
route("POST /services/{id}/restore", s.restoreDesired)
route("POST /services/{id}/accept", s.acceptRunning)
route("POST /services/{id}/merge", s.mergeService)
route("GET /jobs", s.jobsPage) route("GET /jobs", s.jobsPage)
route("GET /fragments/jobs", s.jobsFragment) route("GET /fragments/jobs", s.jobsFragment)
route("GET /jobs/{id}", s.jobPage) route("GET /jobs/{id}", s.jobPage)
@@ -160,17 +132,8 @@ type pageData struct {
ConfigFiles []string ConfigFiles []string
Reason string // why Service cannot be updated; "" when it can Reason string // why Service cannot be updated; "" when it can
AutoBlocked string // why Service cannot be set to auto; "" when it can AutoBlocked string // why Service cannot be set to auto; "" when it can
Versions []versionRow Jobs []model.Job
Polling string // what the service page says about registry polling Job model.Job
PollFailed bool // Polling is the last registry error
// RestoreBlocked is why a drifted Service cannot be restored; "" when it
// can. Only meaningful when Service.Drifted().
RestoreBlocked string
MoveFrom *model.Move // a live Service: the removed service it probably replaces
MoveTo *model.Move // a removed Service: the live service it probably moved to
MergeSources []model.Service // a live Compose or Swarm Service: removed ones to merge in
Jobs []model.Job
Job model.Job
} }
func (s *Server) base(r *http.Request, title string) pageData { func (s *Server) base(r *http.Request, title string) pageData {
@@ -259,45 +222,10 @@ func (s *Server) serviceDetail(r *http.Request, id int64) (pageData, error) {
return data, err return data, err
} }
data.AutoBlocked = s.autoBlocked(svc) data.AutoBlocked = s.autoBlocked(svc)
if err := s.jobs.Restorable(svc); errors.As(err, &nu) {
data.RestoreBlocked = nu.Reason
} else if err != nil {
return data, err
}
if err := s.moveData(r, &data); err != nil {
return data, err
}
versions, err := s.store.RecentVersions(r.Context(), svc.ImageRepo, versionsLimit)
if err != nil {
return data, err
}
data.Versions = versionRows(versions, svc.ActualDigest)
data.Polling, data.PollFailed = pollingStatus(svc, s.cfg.PollInterval)
data.Jobs, err = s.store.ListJobs(r.Context(), model.JobFilter{ServiceID: id, Limit: serviceJobsLimit}) data.Jobs, err = s.store.ListJobs(r.Context(), model.JobFilter{ServiceID: id, Limit: serviceJobsLimit})
return data, err return data, err
} }
// moveData fills the move suggestion and the merge choices of the service page.
func (s *Server) moveData(r *http.Request, data *pageData) error {
svc := data.Service
moves, err := s.store.MoveCandidates(r.Context())
if err != nil {
return err
}
for i, m := range moves {
switch svc.ID {
case m.ToID:
data.MoveFrom = &moves[i]
case m.FromID:
data.MoveTo = &moves[i]
}
}
if svc.RemovedAt == nil && !svc.Self && svc.Kind != protocol.KindContainer {
data.MergeSources, err = s.store.MergeSources(r.Context(), svc.ID)
}
return err
}
func (s *Server) servicePage(w http.ResponseWriter, r *http.Request) { func (s *Server) servicePage(w http.ResponseWriter, r *http.Request) {
id, ok := pathID(w, r) id, ok := pathID(w, r)
if !ok { if !ok {
@@ -332,147 +260,24 @@ func (s *Server) updateService(w http.ResponseWriter, r *http.Request) {
return return
} }
sess, _ := webauth.SessionFrom(r.Context()) sess, _ := webauth.SessionFrom(r.Context())
var job model.Job job, err := s.jobs.Trigger(r.Context(), id, sess.Subject)
var err error var nu *model.NotUpdatableError
if v := r.PostFormValue("version"); v == "" { switch {
job, err = s.jobs.Trigger(r.Context(), id, sess.Subject) case err == nil || errors.Is(err, model.ErrJobActive):
} else { http.Redirect(w, r, "/jobs/"+strconv.FormatInt(job.ID, 10), http.StatusSeeOther)
versionID, perr := strconv.ParseInt(v, 10, 64) case errors.As(err, &nu):
if perr != nil { data, derr := s.serviceDetail(r, id)
http.Error(w, "invalid version", http.StatusBadRequest) if derr != nil {
s.failLookup(w, derr)
return return
} }
job, err = s.jobs.TriggerVersion(r.Context(), id, versionID, sess.Subject) data.Error = "Cannot update: " + nu.Reason + "."
} s.render(w, http.StatusConflict, "service.html", "layout", data)
var nu *model.NotUpdatableError
switch {
case err == nil || errors.Is(err, model.ErrJobActive):
http.Redirect(w, r, "/jobs/"+strconv.FormatInt(job.ID, 10), http.StatusSeeOther)
case errors.As(err, &nu):
s.renderServiceError(w, r, id, http.StatusConflict, "Cannot update: "+nu.Reason+".")
default: default:
s.failLookup(w, err) s.failLookup(w, err)
} }
} }
// restoreDesired deploys the desired ref of a drifted service again.
func (s *Server) restoreDesired(w http.ResponseWriter, r *http.Request) {
id, ok := pathID(w, r)
if !ok {
return
}
sess, _ := webauth.SessionFrom(r.Context())
job, err := s.jobs.RestoreDesired(r.Context(), id, sess.Subject)
var nu *model.NotUpdatableError
switch {
case err == nil || errors.Is(err, model.ErrJobActive):
http.Redirect(w, r, "/jobs/"+strconv.FormatInt(job.ID, 10), http.StatusSeeOther)
case errors.As(err, &nu):
s.renderServiceError(w, r, id, http.StatusConflict, "Cannot restore: "+nu.Reason+".")
default:
s.failLookup(w, err)
}
}
// acceptRunning makes the running version of a drifted service the desired
// one. The form's actual field is the running ref the user saw (Service.RunningRef);
// it only guards against the service having changed since, the value written
// comes from the row.
func (s *Server) acceptRunning(w http.ResponseWriter, r *http.Request) {
id, ok := pathID(w, r)
if !ok {
return
}
actual := r.PostFormValue("actual")
if actual == "" {
http.Error(w, "missing actual", http.StatusBadRequest)
return
}
svc, err := s.store.GetService(r.Context(), id)
if err == nil && svc.RemovedAt != nil {
err = model.ErrNotFound
}
if err != nil {
s.failLookup(w, err)
return
}
if !svc.Drifted() {
s.renderServiceError(w, r, id, http.StatusConflict, "Cannot accept: the service runs its desired version.")
return
}
if actual != svc.RunningRef() {
s.renderServiceError(w, r, id, http.StatusConflict, "The running version changed; check it again.")
return
}
desired := svc.RunningRef()
err = s.store.AcceptRunning(r.Context(), id, svc.ActualRef, svc.ActualDigest, desired)
switch {
case errors.Is(err, model.ErrConflict):
s.renderServiceError(w, r, id, http.StatusConflict, "The running version changed; check it again.")
return
case err != nil:
s.failLookup(w, err)
return
}
sess, _ := webauth.SessionFrom(r.Context())
s.log.Info("drift_accepted", "service_id", id, "subject", sess.Subject, "desired", desired)
s.broker.Publish(events.Services)
if r.PostFormValue("return") == "overview" {
http.Redirect(w, r, "/", http.StatusSeeOther)
return
}
http.Redirect(w, r, "/services/"+strconv.FormatInt(id, 10), http.StatusSeeOther)
}
// mergeService moves the live service {id} onto the removed service in the
// form field "from", which keeps its history, mode and desired ref.
func (s *Server) mergeService(w http.ResponseWriter, r *http.Request) {
id, ok := pathID(w, r)
if !ok {
return
}
from, err := strconv.ParseInt(r.PostFormValue("from"), 10, 64)
if err != nil {
http.Error(w, "invalid service", http.StatusBadRequest)
return
}
err = s.store.MergeService(r.Context(), from, id)
var refused *model.MergeRefusedError
switch {
case errors.As(err, &refused):
s.renderServiceError(w, r, id, http.StatusConflict, "Cannot merge: "+refused.Reason+".")
return
case errors.Is(err, model.ErrJobActive):
s.renderServiceError(w, r, id, http.StatusConflict, "Cannot merge: "+model.ErrJobActive.Error()+".")
return
case errors.Is(err, model.ErrRevoked):
s.renderServiceError(w, r, id, http.StatusConflict, "Cannot merge: the agent is revoked.")
return
case errors.Is(err, model.ErrConflict):
s.renderServiceError(w, r, id, http.StatusConflict, "Cannot merge: the service changed; check it again.")
return
case err != nil:
s.failLookup(w, err)
return
}
sess, _ := webauth.SessionFrom(r.Context())
s.log.Info("service_merged", "from_id", from, "to_id", id, "subject", sess.Subject)
s.broker.Publish(events.Services)
s.broker.Publish(events.Jobs)
http.Redirect(w, r, "/services/"+strconv.FormatInt(from, 10), http.StatusSeeOther)
}
// renderServiceError renders the service page of id with msg as its error.
func (s *Server) renderServiceError(w http.ResponseWriter, r *http.Request, id int64, status int, msg string) {
data, err := s.serviceDetail(r, id)
if err != nil {
s.failLookup(w, err)
return
}
data.Error = msg
s.render(w, status, "service.html", "layout", data)
}
// setMode switches a service between manual and auto. Manual is always // setMode switches a service between manual and auto. Manual is always
// allowed; auto only when jobs.AutoAllowed says so. // allowed; auto only when jobs.AutoAllowed says so.
func (s *Server) setMode(w http.ResponseWriter, r *http.Request) { func (s *Server) setMode(w http.ResponseWriter, r *http.Request) {
@@ -496,7 +301,13 @@ func (s *Server) setMode(w http.ResponseWriter, r *http.Request) {
} }
var nu *model.NotUpdatableError var nu *model.NotUpdatableError
if err := s.jobs.AutoAllowed(svc); errors.As(err, &nu) { if err := s.jobs.AutoAllowed(svc); errors.As(err, &nu) {
s.renderServiceError(w, r, id, http.StatusConflict, "Cannot enable auto: "+nu.Reason+".") data, derr := s.serviceDetail(r, id)
if derr != nil {
s.failLookup(w, derr)
return
}
data.Error = "Cannot enable auto: " + nu.Reason + "."
s.render(w, http.StatusConflict, "service.html", "layout", data)
return return
} else if err != nil { } else if err != nil {
s.fail(w, err) s.fail(w, err)
-43
View File
@@ -34,24 +34,6 @@ type fakeStore struct {
created map[string]string // name → token hash created map[string]string // name → token hash
revokedID int64 revokedID int64
revokeFailed []int64 // job IDs RevokeAgent reports as failed revokeFailed []int64 // job IDs RevokeAgent reports as failed
versions []model.ImageVersion
acceptCalls []acceptCall
acceptErr error
moves []model.Move
sources []model.Service
merged [][2]int64 // from, to
mergeErr error
}
func (f *fakeStore) RecentVersions(_ context.Context, repo string, limit int) ([]model.ImageVersion, error) {
var out []model.ImageVersion
for _, v := range f.versions {
if v.Repo == repo && len(out) < limit {
out = append(out, v)
}
}
return out, nil
} }
func (f *fakeStore) ListServices(context.Context) ([]model.Service, error) { return f.services, nil } func (f *fakeStore) ListServices(context.Context) ([]model.Service, error) { return f.services, nil }
@@ -146,12 +128,6 @@ type fakeJobs struct {
mu sync.Mutex mu sync.Mutex
requestedBy string requestedBy string
triggered []int64 // version IDs passed to TriggerVersion
triggerSvc []int64 // service IDs passed to TriggerVersion
plainCalls int // Trigger calls
restorable error // returned by Restorable
restored []int64 // service IDs passed to RestoreDesired
} }
func (f *fakeJobs) Updatable(model.Service) error { return f.updatable } func (f *fakeJobs) Updatable(model.Service) error { return f.updatable }
@@ -162,16 +138,6 @@ func (f *fakeJobs) Trigger(_ context.Context, _ int64, by string) (model.Job, er
f.mu.Lock() f.mu.Lock()
defer f.mu.Unlock() defer f.mu.Unlock()
f.requestedBy = by f.requestedBy = by
f.plainCalls++
return f.job, f.err
}
func (f *fakeJobs) TriggerVersion(_ context.Context, serviceID, versionID int64, by string) (model.Job, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.requestedBy = by
f.triggerSvc = append(f.triggerSvc, serviceID)
f.triggered = append(f.triggered, versionID)
return f.job, f.err return f.job, f.err
} }
@@ -211,15 +177,7 @@ func newTestServer(t *testing.T) (*httptest.Server, *fakeStore, *fakeHub, *event
return srv, store, hub, broker return srv, store, hub, broker
} }
// newTestEnv starts the UI on fake storage, with polling off.
func newTestEnv(t *testing.T) (*httptest.Server, *fakeStore, *fakeHub, *events.Broker, *fakeJobs) { func newTestEnv(t *testing.T) (*httptest.Server, *fakeStore, *fakeHub, *events.Broker, *fakeJobs) {
t.Helper()
return newTestEnvPolling(t, 0)
}
// newTestEnvPolling is newTestEnv with pollInterval as the polling interval
// the service page shows.
func newTestEnvPolling(t *testing.T, pollInterval time.Duration) (*httptest.Server, *fakeStore, *fakeHub, *events.Broker, *fakeJobs) {
t.Helper() t.Helper()
agentID := int64(2) agentID := int64(2)
store := &fakeStore{ store := &fakeStore{
@@ -243,7 +201,6 @@ func newTestEnvPolling(t *testing.T, pollInterval time.Duration) (*httptest.Serv
jobs := &fakeJobs{} jobs := &fakeJobs{}
s, err := New(store, hub, jobs, broker, Config{ s, err := New(store, hub, jobs, broker, Config{
AgentURL: "wss://10.0.0.5:8090/agent", MasterKey: "sha256:abc", AgentImage: "gitea.shcizo.se/shcizo/package-updater:latest", AgentURL: "wss://10.0.0.5:8090/agent", MasterKey: "sha256:abc", AgentImage: "gitea.shcizo.se/shcizo/package-updater:latest",
PollInterval: pollInterval,
}, slog.New(slog.NewTextHandler(io.Discard, nil))) }, slog.New(slog.NewTextHandler(io.Discard, nil)))
require.NoError(t, err) require.NoError(t, err)
mux := http.NewServeMux() mux := http.NewServeMux()
-1
View File
@@ -29,6 +29,5 @@ table.details { width: auto; }
table.details th { padding-right: 24px; } table.details th { padding-right: 24px; }
pre.log { white-space: pre-wrap; } pre.log { white-space: pre-wrap; }
.badge-update { color: var(--warn); border-color: var(--warn); } .badge-update { color: var(--warn); border-color: var(--warn); }
.badge-error { color: var(--bad); border-color: var(--bad); }
form.mode { display: inline; margin-left: 8px; } form.mode { display: inline; margin-left: 8px; }
form.mode button { font-size: 12px; padding: 0 6px; } form.mode button { font-size: 12px; padding: 0 6px; }
+1 -1
View File
@@ -14,7 +14,7 @@
<tbody> <tbody>
{{range .Services}} {{range .Services}}
<tr> <tr>
<td><a href="/services/{{.ID}}">{{.DisplayName}}</a>{{if .Self}} <span class="badge">managed outside</span>{{end}}{{if .AvailableRef}} <span class="badge badge-update" title="{{.AvailableRef}}">update available</span>{{end}}{{if followsNoTag .Service}} <span class="badge">follows no tag</span>{{end}}{{if .LastPollError}} <span class="badge badge-error" title="{{.LastPollError}}">poll failed</span>{{end}}</td> <td><a href="/services/{{.ID}}">{{.DisplayName}}</a>{{if .Self}} <span class="badge">managed outside</span>{{end}}{{if .AvailableRef}} <span class="badge badge-update" title="{{.AvailableRef}}">update available</span>{{end}}</td>
<td><code>{{shortRef .ActualRef}}</code></td> <td><code>{{shortRef .ActualRef}}</code></td>
<td class="status status-{{.Status}}">{{.Status}}</td> <td class="status status-{{.Status}}">{{.Status}}</td>
<td>{{.UpdateMode}} <td>{{.UpdateMode}}
@@ -10,7 +10,6 @@
{{if .Service.DesiredRef}}<tr><th>Desired image</th><td><code>{{shortRef .Service.DesiredRef}}</code></td></tr>{{end}} {{if .Service.DesiredRef}}<tr><th>Desired image</th><td><code>{{shortRef .Service.DesiredRef}}</code></td></tr>{{end}}
{{if .Service.AvailableRef}}<tr><th>Available</th><td><code>{{.Service.AvailableRef}}</code> <span class="muted">via {{.Service.AvailableSource}}{{with .Service.AvailableAt}}, {{.Format "2006-01-02 15:04"}}{{end}}</span></td></tr>{{end}} {{if .Service.AvailableRef}}<tr><th>Available</th><td><code>{{.Service.AvailableRef}}</code> <span class="muted">via {{.Service.AvailableSource}}{{with .Service.AvailableAt}}, {{.Format "2006-01-02 15:04"}}{{end}}</span></td></tr>{{end}}
<tr><th>Mode</th><td>{{.Service.UpdateMode}}</td></tr> <tr><th>Mode</th><td>{{.Service.UpdateMode}}</td></tr>
<tr><th>Polling</th><td{{if .PollFailed}} class="error"{{end}}>{{.Polling}}</td></tr>
{{if .ConfigFiles}}<tr><th>Config files</th><td>{{range .ConfigFiles}}<code>{{.}}</code><br>{{end}}</td></tr>{{end}} {{if .ConfigFiles}}<tr><th>Config files</th><td>{{range .ConfigFiles}}<code>{{.}}</code><br>{{end}}</td></tr>{{end}}
</table> </table>
{{if .Reason}} {{if .Reason}}
@@ -32,29 +31,6 @@
<p class="muted">{{if .AutoBlocked}}Auto unavailable: {{.AutoBlocked}}.{{else}}The webhook will update this service as soon as CI pushes a new image.{{end}}</p> <p class="muted">{{if .AutoBlocked}}Auto unavailable: {{.AutoBlocked}}.{{else}}The webhook will update this service as soon as CI pushes a new image.{{end}}</p>
{{end}} {{end}}
</form> </form>
<h2>Versions</h2>
{{if eq .Service.UpdateMode "auto"}}<p class="warning">This service is in auto mode: a version picked here is replaced again when a newer version is announced or found by polling.</p>{{end}}
<table>
<thead><tr><th>Version</th><th>Seen via</th><th>First seen</th><th></th></tr></thead>
<tbody>
{{range .Versions}}
<tr>
<td><code title="{{.Ref}}">{{shortRef .Ref}}</code>{{if .Running}} <span class="badge">running</span>{{end}}{{if not .Tag}} <span class="muted">the service will follow no tag afterwards</span>{{end}}</td>
<td>{{.Source}}</td>
<td>{{.FirstSeenAt.Format "2006-01-02 15:04"}}</td>
{{/* AutoBlocked is the same check that decides whether the service can be updated with an explicit ref, which is what a pick is. */}}
<td>{{if not $.AutoBlocked}}
<form method="post" action="/services/{{$.Service.ID}}/update">
<input type="hidden" name="csrf_token" value="{{$.CSRF}}">
<input type="hidden" name="version" value="{{.ID}}">
<button type="submit">Deploy</button>
</form>{{end}}</td>
</tr>
{{else}}
<tr><td colspan="4">No versions seen yet.</td></tr>
{{end}}
</tbody>
</table>
<h2>Update history</h2> <h2>Update history</h2>
<div id="history" hx-get="/fragments/services/{{.Service.ID}}/jobs" hx-trigger="sse:jobs" hx-swap="innerHTML"> <div id="history" hx-get="/fragments/services/{{.Service.ID}}/jobs" hx-trigger="sse:jobs" hx-swap="innerHTML">
{{template "history" .}} {{template "history" .}}
-221
View File
@@ -1,221 +0,0 @@
package web
import (
"io"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/shcizo/package-updater/internal/master/model"
)
const sluskRepo = "gitea.shcizo.se/shcizo/slusk"
func TestServicePage_VersionPicker(t *testing.T) {
srv, store, _, _, _ := newTestEnv(t)
store.services[0].ImageRepo = sluskRepo
store.services[0].ActualDigest = "sha256:cur"
first := time.Date(2026, 10, 12, 9, 5, 0, 0, time.UTC)
store.versions = []model.ImageVersion{
{ID: 31, Repo: sluskRepo, Tag: "1.5.0", Digest: "sha256:" + strings.Repeat("a", 40), Source: "poll", FirstSeenAt: first},
{ID: 32, Repo: sluskRepo, Tag: "1.4.2", Digest: "sha256:cur", Source: "running", FirstSeenAt: first},
{ID: 33, Repo: sluskRepo, Digest: "sha256:" + strings.Repeat("b", 40), Source: "running", FirstSeenAt: first},
{ID: 34, Repo: sluskRepo, Tag: "main", Source: "webhook", FirstSeenAt: first},
{ID: 40, Repo: "other/repo", Tag: "x", Source: "poll", FirstSeenAt: first},
}
_, body := get(t, srv, "/services/1")
for _, id := range []string{"31", "32", "33", "34"} {
assert.Contains(t, body, `<input type="hidden" name="version" value="`+id+`">`, id)
}
assert.NotContains(t, body, `name="version" value="40"`, "versions of other repos are not listed")
assert.Equal(t, 4, strings.Count(body, `name="version"`))
assert.Contains(t, body, `name="csrf_token" value="csrf"`)
assert.Contains(t, body, `title="`+sluskRepo+`:1.5.0@sha256:`+strings.Repeat("a", 40)+`"`, "full ref as title")
assert.Contains(t, body, sluskRepo+":1.5.0@sha256:"+strings.Repeat("a", 12), "short ref as text")
assert.Equal(t, 1, strings.Count(body, `<span class="badge">running</span>`), "only the running digest is marked")
assert.Equal(t, 1, strings.Count(body, "the service will follow no tag afterwards"), "only the tagless row")
assert.Contains(t, body, "2026-10-12 09:05")
assert.NotContains(t, body, "auto mode", "no auto warning for a manual service")
}
func TestServicePage_VersionPickerShowsAtMostTenRows(t *testing.T) {
srv, store, _, _, _ := newTestEnv(t)
store.services[0].ImageRepo = sluskRepo
for i := int64(1); i <= 12; i++ {
store.versions = append(store.versions, model.ImageVersion{ID: i, Repo: sluskRepo, Tag: "t", Digest: "sha256:d", Source: "poll"})
}
_, body := get(t, srv, "/services/1")
assert.Equal(t, 10, strings.Count(body, `name="version"`))
}
func TestServicePage_VersionPickerWarnsForAutoServices(t *testing.T) {
srv, store, _, _, _ := newTestEnv(t)
store.services[0].UpdateMode = "auto"
_, body := get(t, srv, "/services/1")
assert.Contains(t, body, "This service is in auto mode")
}
func TestServicePage_VersionPickerHiddenWhenServiceCannotBeUpdated(t *testing.T) {
for _, reason := range []string{model.ReasonSelf, model.ReasonContainer, model.ReasonRemoved, model.ReasonBuilt} {
t.Run(reason, func(t *testing.T) {
srv, store, _, _, jobs := newTestEnv(t)
store.services[0].ImageRepo = sluskRepo
store.versions = []model.ImageVersion{{ID: 31, Repo: sluskRepo, Tag: "1.5.0", Source: "poll"}}
jobs.autoErr = &model.NotUpdatableError{Reason: reason}
_, body := get(t, srv, "/services/1")
assert.Contains(t, body, "<h2>Versions</h2>")
assert.Contains(t, body, "1.5.0", "the history is still shown")
assert.NotContains(t, body, `name="version"`)
})
}
}
func TestServicePage_VersionPickerOffersVersionsToAServiceThatFollowsNoTag(t *testing.T) {
srv, store, _, _, jobs := newTestEnv(t)
store.services[0].ImageRepo = sluskRepo
store.versions = []model.ImageVersion{{ID: 31, Repo: sluskRepo, Tag: "1.5.0", Digest: "sha256:a", Source: "poll"}}
jobs.updatable = &model.NotUpdatableError{Reason: model.ReasonNoTag} // autoErr stays nil: an explicit ref is fine
_, body := get(t, srv, "/services/1")
assert.Contains(t, body, `name="version" value="31"`)
assert.NotContains(t, body, `<button type="submit">Update</button>`, "the plain Update button stays hidden")
}
func TestServicePage_PollingRow(t *testing.T) {
tests := []struct {
name string
edit func(*model.Service)
want string
failed bool
}{
{"polled", func(s *model.Service) { s.ActualDigest = "sha256:cur" }, "every 6h", false},
{"poll error", func(s *model.Service) { s.ActualDigest = "sha256:cur"; s.LastPollError = "head x: 401 <b>" }, "last check failed: head x: 401 &lt;b&gt;", true},
{"built", func(s *model.Service) { s.ActualDigest = "sha256:cur"; s.Built = true }, "not polled: built from source", false},
{"no digest", func(s *model.Service) {}, "not polled: no registry digest", false},
{"no tag", func(s *model.Service) { s.ActualRef = "r@sha256:a"; s.ActualDigest = "sha256:a" }, "not polled: follows no tag", false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
srv, store, _, _, _ := newTestEnvPolling(t, 6*time.Hour)
tt.edit(&store.services[0])
_, body := get(t, srv, "/services/1")
assert.Contains(t, body, "<th>Polling</th><td")
assert.Contains(t, body, tt.want)
assert.Equal(t, tt.failed, strings.Contains(body, `<td class="error">`+tt.want))
})
}
}
func TestShortDuration(t *testing.T) {
tests := []struct {
d time.Duration
want string
}{
{6 * time.Hour, "6h"},
{90 * time.Minute, "1h30m"},
{30 * time.Minute, "30m"},
{90 * time.Second, "1m30s"},
{45 * time.Second, "45s"},
{6*time.Hour + 5*time.Second, "6h0m5s"},
}
for _, tt := range tests {
t.Run(tt.want, func(t *testing.T) {
assert.Equal(t, tt.want, shortDuration(tt.d))
})
}
}
func postVersion(t *testing.T, srv *httptest.Server, version string) (*http.Response, string) {
t.Helper()
resp, err := noRedirectClient().PostForm(srv.URL+"/services/1/update", url.Values{"csrf_token": {"csrf"}, "version": {version}})
require.NoError(t, err)
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
return resp, string(body)
}
func TestUpdateService_VersionStartsThatVersion(t *testing.T) {
srv, _, _, _, jobs := newTestEnv(t)
jobs.job = model.Job{ID: 42}
resp, _ := postVersion(t, srv, "31")
assert.Equal(t, http.StatusSeeOther, resp.StatusCode)
assert.Equal(t, "/jobs/42", resp.Header.Get("Location"))
assert.Equal(t, []int64{31}, jobs.triggered)
assert.Equal(t, []int64{1}, jobs.triggerSvc)
assert.Equal(t, "alice", jobs.triggeredBy())
assert.Zero(t, jobs.plainCalls)
}
func TestUpdateService_WithoutVersionTriggersTheFollowedTag(t *testing.T) {
srv, _, _, _, jobs := newTestEnv(t)
jobs.job = model.Job{ID: 42}
resp, _ := postVersion(t, srv, "")
assert.Equal(t, http.StatusSeeOther, resp.StatusCode)
assert.Equal(t, 1, jobs.plainCalls)
assert.Empty(t, jobs.triggered)
}
func TestUpdateService_VersionErrors(t *testing.T) {
tests := []struct {
name string
version string
err error
wantCode int
wantBody string
wantJob string // Location header
}{
{"not a number", "abc", nil, http.StatusBadRequest, "invalid version", ""},
{"a ref instead of an id", "repo:evil@sha256:x", nil, http.StatusBadRequest, "invalid version", ""},
{"unknown version or other repo", "77", model.ErrNotFound, http.StatusNotFound, "not found", ""},
{"not updatable", "31", &model.NotUpdatableError{Reason: model.ReasonBuilt}, http.StatusConflict, "Cannot update: " + model.ReasonBuilt + ".", ""},
{"job already active", "31", model.ErrJobActive, http.StatusSeeOther, "", "/jobs/41"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
srv, _, _, _, jobs := newTestEnv(t)
jobs.err, jobs.job = tt.err, model.Job{ID: 41}
resp, body := postVersion(t, srv, tt.version)
assert.Equal(t, tt.wantCode, resp.StatusCode)
assert.Contains(t, body, tt.wantBody)
assert.Equal(t, tt.wantJob, resp.Header.Get("Location"))
if tt.wantCode == http.StatusBadRequest {
assert.Empty(t, jobs.triggered, "no job for a malformed version")
}
})
}
}
func TestOverview_ShowsFollowsNoTagAndPollFailedBadges(t *testing.T) {
srv, store, _, _, _ := newTestEnv(t)
store.services[0].LastPollError = `head x: "boom" <script>`
for _, path := range []string{"/", "/fragments/services"} {
_, body := get(t, srv, path)
assert.Equal(t, 1, strings.Count(body, "follows no tag"), "only the digest-only Swarm service: "+path)
assert.Contains(t, body, `class="badge badge-error" title="head x: &#34;boom&#34; &lt;script&gt;">poll failed</span>`, path)
assert.Equal(t, 1, strings.Count(body, "poll failed"), path)
}
}
-47
View File
@@ -5,9 +5,7 @@ import (
"time" "time"
"github.com/shcizo/package-updater/internal/imageref" "github.com/shcizo/package-updater/internal/imageref"
"github.com/shcizo/package-updater/internal/master/jobs"
"github.com/shcizo/package-updater/internal/master/model" "github.com/shcizo/package-updater/internal/master/model"
"github.com/shcizo/package-updater/internal/master/poller"
"github.com/shcizo/package-updater/internal/protocol" "github.com/shcizo/package-updater/internal/protocol"
) )
@@ -56,51 +54,6 @@ func shortRef(ref string) string {
return r.String() return r.String()
} }
// followsNoTag says the service's image reference has a digest but no tag,
// so there is no tag to poll or update.
func followsNoTag(svc model.Service) bool {
return jobs.FollowedRef(svc.ActualRef) == ""
}
// versionRow is an image version as shown on the service page.
type versionRow struct {
model.ImageVersion
Running bool // the service runs exactly this digest
}
// versionRows marks the version the service runs now, by digest.
func versionRows(versions []model.ImageVersion, actualDigest string) []versionRow {
rows := make([]versionRow, len(versions))
for i, v := range versions {
rows[i] = versionRow{ImageVersion: v, Running: v.Digest != "" && v.Digest == actualDigest}
}
return rows
}
// pollingStatus is the text of the "Polling" row on the service page and
// whether it is a registry error.
func pollingStatus(svc model.Service, interval time.Duration) (text string, failed bool) {
switch {
case poller.SkipReason(svc) != "":
return "not polled: " + poller.SkipReason(svc), false
case svc.LastPollError != "":
return "last check failed: " + svc.LastPollError, true
}
return "every " + shortDuration(interval), false
}
// shortDuration formats d without trailing zero units: "6h", "1h30m", "90s".
func shortDuration(d time.Duration) string {
s := d.String()
if strings.HasSuffix(s, "m0s") {
s = s[:len(s)-2]
}
if strings.HasSuffix(s, "h0m") {
s = s[:len(s)-2]
}
return s
}
type agentRow struct { type agentRow struct {
model.Agent model.Agent
Online bool Online bool
+3 -10
View File
@@ -16,16 +16,11 @@ import (
// Version is the protocol version this build speaks. Agents send it in // Version is the protocol version this build speaks. Agents send it in
// Hello. Phase-1 agents do not send the field, which reads as 0. // Hello. Phase-1 agents do not send the field, which reads as 0.
const Version = 3 const Version = 2
// MinJobVersion is the lowest agent protocol version the master sends jobs to. // MinJobVersion is the lowest agent protocol version the master sends jobs to.
const MinJobVersion = 2 const MinJobVersion = 2
// MinPinnedRefVersion is the lowest agent protocol version the master sends
// jobs whose Ref has a digest or no tag to. Version-2 agents refuse a Ref
// with a digest.
const MinPinnedRefVersion = 3
// MaxJobOutput caps the job output an agent keeps and the master stores, in // MaxJobOutput caps the job output an agent keeps and the master stores, in
// bytes. Only the tail is kept. With JSON escaping (worst case 6 bytes per // bytes. Only the tail is kept. With JSON escaping (worst case 6 bytes per
// byte) a JobResult carrying this much output still fits the 1 MiB read limit. // byte) a JobResult carrying this much output still fits the 1 MiB read limit.
@@ -128,10 +123,8 @@ type Remove struct {
Key string `json:"key"` Key string `json:"key"`
} }
// JobStart asks the agent to deploy Ref to one service. Ref is "repo:tag", // JobStart asks the agent to deploy Ref to one service. Ref is "repo:tag";
// "repo:tag@digest" or "repo@digest". For "repo:tag" the agent resolves the // the agent resolves the tag's current digest and deploys "repo:tag@digest".
// tag's current digest and deploys "repo:tag@digest"; a Ref with a digest is
// deployed as is.
// The agent reads the Compose working directory and config files from its // The agent reads the Compose working directory and config files from its
// own Docker daemon, not from this message. // own Docker daemon, not from this message.
type JobStart struct { type JobStart struct {