Compare commits
18 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 2d668ddc63 | |||
| 07b95f5feb | |||
| bca2e252fb | |||
| 81bd01c5d6 | |||
| 2421c74df1 | |||
| fcd5908992 | |||
| 50ef1a9a07 | |||
| db84a2f6ba | |||
| 9881812eca | |||
| ac3912367f | |||
| 9a3a200873 | |||
| f92f8ac6b1 | |||
| 34865c9fd0 | |||
| a8524e5ff4 | |||
| 9cf484e164 | |||
| 1002bb7b96 | |||
| 8f999c69bd | |||
| df54bbaacf |
@@ -24,3 +24,4 @@ coverage.html
|
|||||||
|
|
||||||
# Serena MCP workspace
|
# Serena MCP workspace
|
||||||
.serena/
|
.serena/
|
||||||
|
.superpowers/
|
||||||
|
|||||||
@@ -0,0 +1,99 @@
|
|||||||
|
# package-updater — Claude context
|
||||||
|
|
||||||
|
Webhook-driven Docker Compose updater. Go service, single binary, deployed as
|
||||||
|
a container with `/var/run/docker.sock` mounted.
|
||||||
|
|
||||||
|
See `README.md` for user-facing docs. This file is for working ON the code.
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go test ./... # all tests, fast (no docker required)
|
||||||
|
go test -run TestFoo ./internal/api # single test
|
||||||
|
go build ./cmd/server # produces ./server
|
||||||
|
docker build -t package-updater:dev . # multi-stage, builds golang:1.26-alpine
|
||||||
|
|
||||||
|
# Local smoke test (builds from Dockerfile, runs against /var/run/docker.sock):
|
||||||
|
cp env.sample .env && $EDITOR .env # fill UPDATER_API_KEY
|
||||||
|
docker compose -f docker-compose.local.yml up --build
|
||||||
|
curl -sH "Authorization: Bearer $UPDATER_API_KEY" \
|
||||||
|
-d '{"image":"foo"}' http://localhost:8080/update | jq
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture (one-line per package)
|
||||||
|
|
||||||
|
- `cmd/server` — wiring only: config → docker client → discovery → queue → http
|
||||||
|
- `internal/api` — HTTP handlers, bearer-token auth, request-id + access-log middleware
|
||||||
|
- `internal/config` — env-var loading; fails fast if `UPDATER_API_KEY` missing
|
||||||
|
- `internal/discovery`— given an image, return Compose `Job`s to run (label parsing, path check, dedup)
|
||||||
|
- `internal/discovery/swarm.go` — SwarmDiscovery: same FindJobs signature, lists `docker service ls`, gates on service-level opt-in label instead of STACKS_ROOT path-check
|
||||||
|
- `internal/updater` — FIFO queue + single worker + `docker compose` subprocess executor
|
||||||
|
- `internal/updater/swarm_executor.go` — SwarmExecutor: `docker service update --image` via the Docker API instead of a `docker compose` subprocess
|
||||||
|
- `internal/selfupdate` — flush-then-exec wrapper for updating ourselves (NOT wired in live, see gotcha)
|
||||||
|
- `internal/metrics` — Prometheus collectors
|
||||||
|
- `internal/logging` — slog JSON, request-id context propagation
|
||||||
|
|
||||||
|
## Design intent (do not break without discussion)
|
||||||
|
|
||||||
|
- **Single FIFO worker by design** (`internal/updater/queue.go`). Spec §5.7. Never
|
||||||
|
parallelise the queue — two `docker compose` calls against the same stack race.
|
||||||
|
- **Defense in depth: token AND opt-in label AND STACKS_ROOT prefix** must all
|
||||||
|
hold before a container is touched. Weakening any of these breaks the security
|
||||||
|
model — discuss before changing.
|
||||||
|
- **Auth on `/update` only**. `/healthz`, `/metrics`, `/version` are intentionally
|
||||||
|
unauthenticated (internal network, scraper/healthcheck need them). See
|
||||||
|
`cmd/server/main.go` `routeAuth`.
|
||||||
|
- **`metrics *Metrics` parameters may be nil**; constructors and call sites check.
|
||||||
|
Tests rely on this — don't drop the nil-check.
|
||||||
|
- **Stateless**: no DB, no config file, no on-disk audit log. Docker daemon is
|
||||||
|
the source of truth.
|
||||||
|
- **`MODE` is exclusive because each instance owns exactly one Docker daemon.**
|
||||||
|
A composite "handle both at once" mode has been considered and rejected: it would
|
||||||
|
still only reach one daemon, so it buys nothing. Fleets run one instance per
|
||||||
|
server and CI fans out. Reopen this only if a Swarm manager node starts running
|
||||||
|
standalone Compose stacks locally. Rationale and rejected alternatives:
|
||||||
|
`docs/superpowers/specs/2026-08-04-multi-server-fanout-design.md`.
|
||||||
|
- **Swarm mode's security gate is opt-in label only** — there is no
|
||||||
|
STACKS_ROOT-equivalent path check, since Swarm services have no local compose
|
||||||
|
file. Don't add one; don't weaken Compose mode's three-factor gate to match.
|
||||||
|
|
||||||
|
## Gotchas
|
||||||
|
|
||||||
|
- **`internal/selfupdate.Wrapped` is implemented and unit-tested but NOT wired
|
||||||
|
into the live queue.** Spec §15 calls this out as a v1 gap. If you "fix" this,
|
||||||
|
read the package doc — flush-before-exec ordering is subtle.
|
||||||
|
- **Image matching is case-sensitive and tag/digest-agnostic.** The tricky case
|
||||||
|
is `localhost:5000/foo:v1` where the first colon is a port, not a tag. See
|
||||||
|
`internal/discovery/matching.go` `NormaliseImage`.
|
||||||
|
- **Path safety uses `filepath.Rel` + ".." prefix check**, NOT `strings.HasPrefix`.
|
||||||
|
Prevents the `/foo-evil` vs `/foo` confusion. See `internal/discovery/pathcheck.go`.
|
||||||
|
- **`STACKS_ROOT` defaults to `/home/shcizo/self-hosted`** in prod but `env.sample`
|
||||||
|
suggests `/tmp` for smoke tests. The default in `config.go` is the prod value
|
||||||
|
— set explicitly in test/dev envs.
|
||||||
|
- **Go 1.26.3** (`go.mod`). Dockerfile pins `golang:1.26-alpine`. Bumping one
|
||||||
|
without the other has caused a fix commit already.
|
||||||
|
- **`<summary>` C# convention from global CLAUDE.md does not apply here** — this
|
||||||
|
is Go. Use idiomatic GoDoc (`// FuncName does X.`).
|
||||||
|
- **Swarm mode requires manager-node API access.** `docker service update` fails
|
||||||
|
with a permission error against a worker-only node. This is an operator/deployment
|
||||||
|
concern (point `DOCKER_HOST` at a manager, or schedule the updater on a manager),
|
||||||
|
not something the code can detect or work around.
|
||||||
|
- **`tag` is cosmetic in compose mode but load-bearing in swarm mode.** `handlers.go` folds it
|
||||||
|
into the requested image; compose discovery strips it via `NormaliseImage`, but
|
||||||
|
`SwarmExecutor` assigns it directly to `ContainerSpec.Image`. A request without a tag updates
|
||||||
|
a Swarm service to `:latest`.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- Design spec: `docs/superpowers/specs/2026-05-22-package-updater-design.md` (489 lines, authoritative)
|
||||||
|
- Implementation plan: `docs/superpowers/plans/2026-05-22-package-updater-implementation.md`
|
||||||
|
- Consumer-side CI integration: `gitea-action/`
|
||||||
|
- Multi-server topology spec: `docs/superpowers/specs/2026-08-04-multi-server-fanout-design.md`
|
||||||
|
- Multi-server fan-out plan: `docs/superpowers/plans/2026-08-04-multi-server-fanout.md`
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
- Conventional Commits (`feat:`, `fix:`, `docs:`, `ci:`, `build:`, `chore:`).
|
||||||
|
- One package = one responsibility; interfaces defined at consumer site
|
||||||
|
(`api.Finder`, `api.Submitter`, `api.Pinger`) for testability.
|
||||||
|
- Table-driven tests with `stretchr/testify`. No mocks beyond hand-written fakes.
|
||||||
@@ -1,13 +1,13 @@
|
|||||||
# package-updater
|
# package-updater
|
||||||
|
|
||||||
Webhook-driven Docker Compose service updater. Fills the gap between Watchtower (polling, no CI integration) and full GitOps (Argo CD, Flux) for a self-hosted, single-host environment.
|
Webhook-driven Docker service updater — Compose stacks or Swarm services. Fills the gap between Watchtower (polling, no CI integration) and full GitOps (Argo CD, Flux) for self-hosted environments, one instance per server.
|
||||||
|
|
||||||
**Trigger flow:**
|
**Trigger flow:**
|
||||||
|
|
||||||
1. Gitea workflow builds and pushes a new image to your registry.
|
1. Gitea workflow builds and pushes a new image to your registry.
|
||||||
2. Workflow calls `POST /update` on this service with the image name.
|
2. Workflow calls `POST /update` on this service with the image name.
|
||||||
3. Service finds the matching Compose-managed container(s) on the host via Docker labels.
|
3. Service finds the matching container(s) or Swarm service(s) via Docker labels.
|
||||||
4. Runs `docker compose pull` + `up -d` for the relevant service(s).
|
4. Runs `docker compose pull` + `up -d`, or `docker service update`, depending on `MODE`.
|
||||||
|
|
||||||
See [design spec](docs/superpowers/specs/2026-05-22-package-updater-design.md) and [implementation plan](docs/superpowers/plans/2026-05-22-package-updater-implementation.md) for full design and rationale.
|
See [design spec](docs/superpowers/specs/2026-05-22-package-updater-design.md) and [implementation plan](docs/superpowers/plans/2026-05-22-package-updater-implementation.md) for full design and rationale.
|
||||||
|
|
||||||
@@ -25,7 +25,53 @@ A container is eligible for update only if it has **both**:
|
|||||||
- An image name matching the request (tag-agnostic), AND
|
- An image name matching the request (tag-agnostic), AND
|
||||||
- The opt-in label `se.shcizo.auto-update=true`.
|
- The opt-in label `se.shcizo.auto-update=true`.
|
||||||
|
|
||||||
Defense in depth: a valid bearer token AND the opt-in label must both be present before any container is touched.
|
Defense in depth, compose mode: a valid bearer token AND the opt-in label AND a working
|
||||||
|
directory inside `STACKS_ROOT` must all hold before a container is touched. A stack outside
|
||||||
|
`STACKS_ROOT` comes back as `refused` rather than being updated.
|
||||||
|
|
||||||
|
Swarm mode's gate is the token and the opt-in label only — a Swarm service has no local
|
||||||
|
compose file to anchor a path check against.
|
||||||
|
|
||||||
|
## Swarm mode
|
||||||
|
|
||||||
|
Set `MODE=swarm` to update Docker Swarm services instead of Compose stacks. The
|
||||||
|
service runs `docker service update --image` via the Docker API instead of
|
||||||
|
shelling out to `docker compose`.
|
||||||
|
|
||||||
|
Two things to get right when running in Swarm mode:
|
||||||
|
|
||||||
|
- **The opt-in label goes on the service, not the container/task.** Swarm mode
|
||||||
|
reads `Service.Spec.Labels`, so add it with
|
||||||
|
`docker service update --label-add se.shcizo.auto-update=true <service>` or
|
||||||
|
set it under `deploy.labels` in the stack file — a plain `labels:` entry on
|
||||||
|
the service (container-level) is not visible to Swarm mode's discovery.
|
||||||
|
- **The updater must talk to a Swarm manager.** `docker service update`
|
||||||
|
requires manager API access, so either point `DOCKER_HOST` at a manager node
|
||||||
|
or schedule the updater container itself on a manager with the socket
|
||||||
|
mounted. This is a deployment concern the service cannot detect or work
|
||||||
|
around.
|
||||||
|
|
||||||
|
## Deploying across multiple servers
|
||||||
|
|
||||||
|
An instance can only reach the Docker daemon it is configured against. It cannot
|
||||||
|
update stacks on other machines. The deployment model follows from that:
|
||||||
|
|
||||||
|
**One instance per server.** Each server runs its own package-updater against its
|
||||||
|
own local Docker socket, with `MODE` set to whatever that machine runs:
|
||||||
|
|
||||||
|
| Server | `MODE` | Updates |
|
||||||
|
|---|---|---|
|
||||||
|
| Swarm manager node | `swarm` | All Swarm services in the cluster |
|
||||||
|
| Standalone Compose host | `compose` | Compose stacks on that host |
|
||||||
|
|
||||||
|
**CI fans out.** The [Gitea composite action](gitea-action/README.md) takes a list of
|
||||||
|
endpoints and posts `/update` to every instance, so one workflow run reaches the
|
||||||
|
whole fleet. Each instance answers for its own machine; there is no aggregated
|
||||||
|
cross-server response and no instance coordinates any other.
|
||||||
|
|
||||||
|
This is why `MODE` is exclusive rather than a mode that handles both at once: an
|
||||||
|
instance that could do both would still only reach one daemon, so the extra
|
||||||
|
generality buys nothing.
|
||||||
|
|
||||||
## Quick start
|
## Quick start
|
||||||
|
|
||||||
@@ -35,7 +81,7 @@ Defense in depth: a valid bearer token AND the opt-in label must both be present
|
|||||||
4. Point your reverse proxy (NPM/Traefik/Caddy) at `package-updater:8080`. NPM should handle TLS.
|
4. Point your reverse proxy (NPM/Traefik/Caddy) at `package-updater:8080`. NPM should handle TLS.
|
||||||
5. `docker compose up -d`.
|
5. `docker compose up -d`.
|
||||||
6. Add the opt-in label `se.shcizo.auto-update: "true"` to each service you want auto-updated.
|
6. Add the opt-in label `se.shcizo.auto-update: "true"` to each service you want auto-updated.
|
||||||
7. Use the [Gitea composite action](https://gitea.shcizo.se/shcizo/package-updater-action) in your repos to call `/update` after a build.
|
7. Use the [Gitea composite action](gitea-action/README.md) in your repos to call `/update` after a build.
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
@@ -49,6 +95,7 @@ All via environment variables.
|
|||||||
| `LOG_LEVEL` | no | `info` | `debug` / `info` / `warn` / `error`. |
|
| `LOG_LEVEL` | no | `info` | `debug` / `info` / `warn` / `error`. |
|
||||||
| `UPDATE_TIMEOUT` | no | `5m` | Per-job timeout (Go duration). |
|
| `UPDATE_TIMEOUT` | no | `5m` | Per-job timeout (Go duration). |
|
||||||
| `OPT_IN_LABEL` | no | `se.shcizo.auto-update` | Label name to check; value must equal `"true"`. |
|
| `OPT_IN_LABEL` | no | `se.shcizo.auto-update` | Label name to check; value must equal `"true"`. |
|
||||||
|
| `MODE` | no | `compose` | `compose` or `swarm`. Which paradigm *this instance's* Docker daemon runs. See [Deploying across multiple servers](#deploying-across-multiple-servers). |
|
||||||
|
|
||||||
## Endpoints
|
## Endpoints
|
||||||
|
|
||||||
@@ -80,5 +127,8 @@ These are tracked in the spec's section 2 and section 15 as deliberate out-of-sc
|
|||||||
|
|
||||||
- **Self-update wiring**: `internal/selfupdate.Wrapped` exists and is unit-tested but is not wired into the live queue. The HTTP response flush ordering for self-replacement is a future enhancement; for now, expect to manually rerun `docker compose up -d` on the host if pushing a new image of `package-updater` itself causes a mid-response interruption.
|
- **Self-update wiring**: `internal/selfupdate.Wrapped` exists and is unit-tested but is not wired into the live queue. The HTTP response flush ordering for self-replacement is a future enhancement; for now, expect to manually rerun `docker compose up -d` on the host if pushing a new image of `package-updater` itself causes a mid-response interruption.
|
||||||
- **No rollback**: Compose's "keep old container if new fails to start" is the only safety net.
|
- **No rollback**: Compose's "keep old container if new fails to start" is the only safety net.
|
||||||
- **Single host only**.
|
- **One instance reaches one daemon.** An instance never updates another server;
|
||||||
|
fleets run one instance per server with CI fanning out to all of them (see
|
||||||
|
"Deploying across multiple servers"). Swarm mode is the exception in that a
|
||||||
|
single manager-node instance covers the whole cluster.
|
||||||
- **No per-repo API keys**: a single shared bearer token is used.
|
- **No per-repo API keys**: a single shared bearer token is used.
|
||||||
|
|||||||
+17
-4
@@ -42,7 +42,7 @@ func run() error {
|
|||||||
logger := logging.New(cfg.LogLevel)
|
logger := logging.New(cfg.LogLevel)
|
||||||
logger.Info("starting",
|
logger.Info("starting",
|
||||||
"version", version, "commit", commit, "port", cfg.Port,
|
"version", version, "commit", commit, "port", cfg.Port,
|
||||||
"stacks_root", cfg.StacksRoot, "opt_in_label", cfg.OptInLabel,
|
"mode", cfg.Mode, "stacks_root", cfg.StacksRoot, "opt_in_label", cfg.OptInLabel,
|
||||||
)
|
)
|
||||||
|
|
||||||
dockerCli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
|
dockerCli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
|
||||||
@@ -51,8 +51,21 @@ func run() error {
|
|||||||
}
|
}
|
||||||
defer dockerCli.Close()
|
defer dockerCli.Close()
|
||||||
|
|
||||||
disc := discovery.New(dockerCli, cfg.StacksRoot, cfg.OptInLabel)
|
var finder api.Finder
|
||||||
exec := updater.NewComposeExecutor()
|
var exec updater.Executor
|
||||||
|
switch cfg.Mode {
|
||||||
|
case "swarm":
|
||||||
|
finder = discovery.NewSwarm(dockerCli, cfg.OptInLabel)
|
||||||
|
exec = updater.NewSwarmExecutor(dockerCli)
|
||||||
|
case "compose":
|
||||||
|
finder = discovery.New(dockerCli, cfg.StacksRoot, cfg.OptInLabel)
|
||||||
|
exec = updater.NewComposeExecutor()
|
||||||
|
default:
|
||||||
|
// Unreachable: config.Load already validates Mode, but guard
|
||||||
|
// here too rather than silently falling through to a nil
|
||||||
|
// finder/exec pair.
|
||||||
|
return fmt.Errorf("unknown MODE %q", cfg.Mode)
|
||||||
|
}
|
||||||
|
|
||||||
reg := prometheus.NewRegistry()
|
reg := prometheus.NewRegistry()
|
||||||
m := metrics.New(reg)
|
m := metrics.New(reg)
|
||||||
@@ -62,7 +75,7 @@ func run() error {
|
|||||||
queue.Start(context.Background())
|
queue.Start(context.Background())
|
||||||
defer queue.Stop()
|
defer queue.Stop()
|
||||||
|
|
||||||
handlers := api.NewHandlers(disc, &submitterAdapter{queue: queue, timeout: cfg.UpdateTimeout}, dockerCli, version, commit, buildTime, m)
|
handlers := api.NewHandlers(finder, &submitterAdapter{queue: queue, timeout: cfg.UpdateTimeout}, dockerCli, version, commit, buildTime, m)
|
||||||
|
|
||||||
mux := http.NewServeMux()
|
mux := http.NewServeMux()
|
||||||
mux.HandleFunc("POST /update", handlers.Update)
|
mux.HandleFunc("POST /update", handlers.Update)
|
||||||
|
|||||||
@@ -0,0 +1,859 @@
|
|||||||
|
# Docker Swarm Support Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Add a `MODE=swarm` operating mode so package-updater can update Docker Swarm services (`docker service update --image`) as an alternative to the existing Compose-only flow (`docker compose pull && up -d`), selected once per deployment via config — not mixed within one running instance.
|
||||||
|
|
||||||
|
**Architecture:** Introduce parallel `SwarmDiscovery` (internal/discovery) and `SwarmExecutor` (internal/updater) implementations alongside the existing `Discovery`/`ComposeExecutor`. Both new types satisfy the exact same `api.Finder` / `updater.Executor` interfaces already in use, so `internal/api` and the FIFO `Queue` need no structural changes — only `cmd/server/main.go` branches on `cfg.Mode` to decide which pair to construct. The Compose-specific security gate (`STACKS_ROOT` path-check) does not apply to Swarm services (no local compose file exists); it is replaced by the same opt-in label mechanism, checked against the Swarm service's own labels instead of container labels.
|
||||||
|
|
||||||
|
**Tech Stack:** Go 1.26.3, `github.com/docker/docker` SDK v28.5.2+incompatible (`client`, `api/types`, `api/types/swarm`), `stretchr/testify`.
|
||||||
|
|
||||||
|
## Global Constraints
|
||||||
|
|
||||||
|
- Go 1.26.3 (go.mod) — do not bump without also bumping the Dockerfile `golang:1.26-alpine` base image.
|
||||||
|
- Single FIFO worker in `internal/updater/queue.go` must remain single — never parallelize job execution, including for Swarm jobs.
|
||||||
|
- Defense in depth for Compose mode (token + opt-in label + STACKS_ROOT prefix) must keep holding unchanged. Swarm mode's gate is opt-in label only (no filesystem path exists to check) — this is a deliberate, narrower model for Swarm and must not be backported to weaken Compose mode.
|
||||||
|
- Auth remains on `/update` only; `/healthz`, `/metrics`, `/version` stay unauthenticated.
|
||||||
|
- `metrics *Metrics` parameters may be nil; every new call site must follow the existing `if m.metrics != nil { ... }` pattern — never assume non-nil.
|
||||||
|
- Stateless: no DB, no config file, no on-disk audit log required for Swarm mode either.
|
||||||
|
- Conventional Commits (`feat:`, `fix:`, `docs:`, `test:`) for every commit in this plan.
|
||||||
|
- Table-driven / one-assertion-focus tests with `testify/require`, matching existing style in `internal/discovery/discovery_test.go`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: Add `Mode` to config
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `internal/config/config.go`
|
||||||
|
- Test: `internal/config/config_test.go`
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Produces: `Config.Mode string` — always one of `"compose"` or `"swarm"` after `Load()` returns successfully. Later tasks (`cmd/server/main.go`) branch on this exact string.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write the failing tests**
|
||||||
|
|
||||||
|
Add to `internal/config/config_test.go` (create the file if it doesn't already cover this — check first; if it exists, add these functions alongside the existing ones):
|
||||||
|
|
||||||
|
```go
|
||||||
|
func TestLoad_DefaultsToComposeMode(t *testing.T) {
|
||||||
|
t.Setenv("UPDATER_API_KEY", "secret")
|
||||||
|
cfg, err := config.Load()
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Equal(t, "compose", cfg.Mode)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoad_AcceptsSwarmMode(t *testing.T) {
|
||||||
|
t.Setenv("UPDATER_API_KEY", "secret")
|
||||||
|
t.Setenv("MODE", "swarm")
|
||||||
|
cfg, err := config.Load()
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Equal(t, "swarm", cfg.Mode)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoad_RejectsInvalidMode(t *testing.T) {
|
||||||
|
t.Setenv("UPDATER_API_KEY", "secret")
|
||||||
|
t.Setenv("MODE", "kubernetes")
|
||||||
|
_, err := config.Load()
|
||||||
|
require.Error(t, err)
|
||||||
|
require.Contains(t, err.Error(), "MODE")
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Make sure the file imports `"testing"`, `"github.com/shcizo/package-updater/internal/config"`, and `"github.com/stretchr/testify/require"` (match whatever the existing `config_test.go` already imports).
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run tests to verify they fail**
|
||||||
|
|
||||||
|
Run: `go test ./internal/config/... -run 'TestLoad_.*Mode' -v`
|
||||||
|
Expected: FAIL — `cfg.Mode` does not compile (`Config` has no field `Mode`).
|
||||||
|
|
||||||
|
- [ ] **Step 3: Implement**
|
||||||
|
|
||||||
|
In `internal/config/config.go`, add the field and validation:
|
||||||
|
|
||||||
|
```go
|
||||||
|
// Config holds all runtime configuration for the service.
|
||||||
|
type Config struct {
|
||||||
|
APIKey string
|
||||||
|
StacksRoot string
|
||||||
|
Port string
|
||||||
|
LogLevel string
|
||||||
|
UpdateTimeout time.Duration
|
||||||
|
OptInLabel string
|
||||||
|
Mode string
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
In `Load()`, after the existing field assignments (right after `OptInLabel`), add:
|
||||||
|
|
||||||
|
```go
|
||||||
|
Mode: getenvDefault("MODE", "compose"),
|
||||||
|
```
|
||||||
|
|
||||||
|
so the literal becomes:
|
||||||
|
|
||||||
|
```go
|
||||||
|
cfg := &Config{
|
||||||
|
APIKey: os.Getenv("UPDATER_API_KEY"),
|
||||||
|
StacksRoot: getenvDefault("STACKS_ROOT", "/home/shcizo/self-hosted"),
|
||||||
|
Port: getenvDefault("PORT", "8080"),
|
||||||
|
LogLevel: getenvDefault("LOG_LEVEL", "info"),
|
||||||
|
OptInLabel: getenvDefault("OPT_IN_LABEL", "se.shcizo.auto-update"),
|
||||||
|
Mode: getenvDefault("MODE", "compose"),
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Then, after the `APIKey` check and before the `UPDATE_TIMEOUT` parsing, add mode validation:
|
||||||
|
|
||||||
|
```go
|
||||||
|
if cfg.Mode != "compose" && cfg.Mode != "swarm" {
|
||||||
|
return nil, fmt.Errorf("MODE %q is invalid: must be %q or %q", cfg.Mode, "compose", "swarm")
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run tests to verify they pass**
|
||||||
|
|
||||||
|
Run: `go test ./internal/config/... -v`
|
||||||
|
Expected: PASS — all config tests including the three new ones, and no regressions in `TestLoad_RequiresAPIKey`, `_AppliesDefaults`, `_OverridesViaEnv`, `_InvalidTimeoutErrors`.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add internal/config/config.go internal/config/config_test.go
|
||||||
|
git commit -m "feat(config): add MODE env var to select compose or swarm operation"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 2: Generalize `discovery.Job` for non-Compose jobs
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `internal/discovery/discovery.go`
|
||||||
|
- Test: `internal/discovery/discovery_test.go`
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Produces: `discovery.Job` gains two fields, `Image string` (the full image reference the update request asked for — needed later by `SwarmExecutor` to know what to update *to*, since Swarm has no local compose file to `pull` against) and `ServiceID string` (empty for Compose jobs; the Swarm service ID for Swarm jobs, needed later by `SwarmExecutor` to target the right service). Both default to `""` for existing Compose jobs, so no existing caller breaks.
|
||||||
|
- Consumes: nothing new — this task only widens the struct and populates `Image` from the existing `FindJobs(ctx, image string)` parameter.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write the failing test**
|
||||||
|
|
||||||
|
Add to `internal/discovery/discovery_test.go`:
|
||||||
|
|
||||||
|
```go
|
||||||
|
func TestFindJobs_PopulatesImageField(t *testing.T) {
|
||||||
|
fake := &fakeDockerClient{containers: []types.Container{
|
||||||
|
mkContainer("registry.example.com/myapp:v1", mkComposeLabels(
|
||||||
|
"myapp-prod", "web",
|
||||||
|
"/home/shcizo/self-hosted/myapp-prod",
|
||||||
|
"/home/shcizo/self-hosted/myapp-prod/docker-compose.yml",
|
||||||
|
true,
|
||||||
|
)),
|
||||||
|
}}
|
||||||
|
d := discovery.New(fake, "/home/shcizo/self-hosted", "se.shcizo.auto-update")
|
||||||
|
jobs, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, jobs, 1)
|
||||||
|
require.Equal(t, "registry.example.com/myapp", jobs[0].Image)
|
||||||
|
require.Empty(t, jobs[0].ServiceID)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run test to verify it fails**
|
||||||
|
|
||||||
|
Run: `go test ./internal/discovery/... -run TestFindJobs_PopulatesImageField -v`
|
||||||
|
Expected: FAIL — `jobs[0].Image` does not compile (`Job` has no field `Image`).
|
||||||
|
|
||||||
|
- [ ] **Step 3: Implement**
|
||||||
|
|
||||||
|
In `internal/discovery/discovery.go`, widen the struct:
|
||||||
|
|
||||||
|
```go
|
||||||
|
// Job describes a single (project, service, config_files) update to execute.
|
||||||
|
type Job struct {
|
||||||
|
Project string
|
||||||
|
Service string
|
||||||
|
WorkingDir string
|
||||||
|
ConfigFiles []string
|
||||||
|
// Image is the full image reference the update request asked for.
|
||||||
|
// Compose jobs ignore it (compose.yml already pins the reference to
|
||||||
|
// pull); Swarm jobs need it to know what to set on the service spec.
|
||||||
|
Image string
|
||||||
|
// ServiceID is the Swarm service ID. Empty for Compose jobs.
|
||||||
|
ServiceID string
|
||||||
|
// Refused is true when the WorkingDir falls outside STACKS_ROOT.
|
||||||
|
// The job is returned so the caller can surface a per-job "refused"
|
||||||
|
// result, but it MUST NOT be executed.
|
||||||
|
Refused bool
|
||||||
|
RefusedReason string
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
In `FindJobs`, populate `Image` on the constructed job (leave `ServiceID` as its zero value):
|
||||||
|
|
||||||
|
```go
|
||||||
|
jobs = append(jobs, Job{
|
||||||
|
Project: cl.Project,
|
||||||
|
Service: cl.Service,
|
||||||
|
WorkingDir: cl.WorkingDir,
|
||||||
|
ConfigFiles: cl.ConfigFiles,
|
||||||
|
Image: image,
|
||||||
|
Refused: refused,
|
||||||
|
RefusedReason: reason,
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run tests to verify they pass**
|
||||||
|
|
||||||
|
Run: `go test ./internal/discovery/... -v`
|
||||||
|
Expected: PASS — all discovery tests, including the new one and all pre-existing `TestFindJobs_*` cases (they don't assert on `Image`/`ServiceID` so they're unaffected).
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add internal/discovery/discovery.go internal/discovery/discovery_test.go
|
||||||
|
git commit -m "feat(discovery): add Image and ServiceID fields to Job for swarm support"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 3: `SwarmDiscovery` — find matching Swarm services
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `internal/discovery/swarm.go`
|
||||||
|
- Test: `internal/discovery/swarm_test.go`
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: `discovery.Job{Project, Service, Image, ServiceID, Refused, RefusedReason}` (Task 2), `discovery.NormaliseImage(string) string` and `discovery.ImagesMatch(a, b string) bool` (existing, `internal/discovery/matching.go`), `discovery.HasOptIn(labels map[string]string, key string) bool` (existing, `internal/discovery/labels.go`).
|
||||||
|
- Produces: `discovery.SwarmDockerClient` interface, `discovery.NewSwarm(cli SwarmDockerClient, optInLabel string) *SwarmDiscovery`, method `(*SwarmDiscovery) FindJobs(ctx context.Context, image string) ([]Job, error)` — same signature as `*Discovery.FindJobs`, so both satisfy `api.Finder` (`internal/api/handlers.go`) without any change there.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Verify the Docker SDK method signature before writing code**
|
||||||
|
|
||||||
|
Run: `go doc github.com/docker/docker/client.Client.ServiceList`
|
||||||
|
|
||||||
|
Expected output shape (confirm it matches before proceeding — if the SDK signature differs, adjust the interface in Step 3 to match what `go doc` actually reports instead of the assumed signature below):
|
||||||
|
|
||||||
|
```
|
||||||
|
func (cli *Client) ServiceList(ctx context.Context, options types.ServiceListOptions) ([]swarm.Service, error)
|
||||||
|
```
|
||||||
|
|
||||||
|
Also run: `go doc github.com/docker/docker/api/types/swarm.Service` and `go doc github.com/docker/docker/api/types/swarm.ServiceSpec` to confirm `Service.ID`, `Service.Spec.Name` (via embedded `Annotations`), `Service.Spec.Labels` (via embedded `Annotations`), and `Service.Spec.TaskTemplate.ContainerSpec.Image` are reachable as described. Adjust field access in later steps if the embedding differs.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Write the failing tests**
|
||||||
|
|
||||||
|
Create `internal/discovery/swarm_test.go`:
|
||||||
|
|
||||||
|
```go
|
||||||
|
package discovery_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/docker/docker/api/types"
|
||||||
|
"github.com/docker/docker/api/types/swarm"
|
||||||
|
"github.com/shcizo/package-updater/internal/discovery"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
)
|
||||||
|
|
||||||
|
type fakeSwarmDockerClient struct {
|
||||||
|
services []swarm.Service
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSwarmDockerClient) ServiceList(_ context.Context, _ types.ServiceListOptions) ([]swarm.Service, error) {
|
||||||
|
return f.services, f.err
|
||||||
|
}
|
||||||
|
|
||||||
|
func mkService(id, name, image string, labels map[string]string) swarm.Service {
|
||||||
|
return swarm.Service{
|
||||||
|
ID: id,
|
||||||
|
Spec: swarm.ServiceSpec{
|
||||||
|
Annotations: swarm.Annotations{Name: name, Labels: labels},
|
||||||
|
TaskTemplate: swarm.TaskSpec{
|
||||||
|
ContainerSpec: &swarm.ContainerSpec{Image: image},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmFindJobs_MatchAndOptIn(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{services: []swarm.Service{
|
||||||
|
mkService("svc-myapp", "myapp_web", "registry.example.com/myapp:v1", map[string]string{
|
||||||
|
"se.shcizo.auto-update": "true",
|
||||||
|
}),
|
||||||
|
mkService("svc-other", "other_web", "registry.example.com/other:v1", map[string]string{
|
||||||
|
"se.shcizo.auto-update": "true",
|
||||||
|
}),
|
||||||
|
}}
|
||||||
|
d := discovery.NewSwarm(fake, "se.shcizo.auto-update")
|
||||||
|
jobs, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, jobs, 1)
|
||||||
|
require.Equal(t, "myapp_web", jobs[0].Service)
|
||||||
|
require.Equal(t, "svc-myapp", jobs[0].ServiceID)
|
||||||
|
require.Equal(t, "registry.example.com/myapp", jobs[0].Image)
|
||||||
|
require.False(t, jobs[0].Refused)
|
||||||
|
require.Empty(t, jobs[0].WorkingDir)
|
||||||
|
require.Empty(t, jobs[0].ConfigFiles)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmFindJobs_SkipsWithoutOptIn(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{services: []swarm.Service{
|
||||||
|
mkService("svc-myapp", "myapp_web", "registry.example.com/myapp:v1", nil),
|
||||||
|
}}
|
||||||
|
d := discovery.NewSwarm(fake, "se.shcizo.auto-update")
|
||||||
|
jobs, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Empty(t, jobs)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmFindJobs_DockerError(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{err: errors.New("connection refused")}
|
||||||
|
d := discovery.NewSwarm(fake, "se.shcizo.auto-update")
|
||||||
|
_, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
|
||||||
|
require.Error(t, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmFindJobs_NoContainerSpecIsSkipped(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{services: []swarm.Service{
|
||||||
|
{
|
||||||
|
ID: "svc-weird",
|
||||||
|
Spec: swarm.ServiceSpec{
|
||||||
|
Annotations: swarm.Annotations{Name: "weird", Labels: map[string]string{"se.shcizo.auto-update": "true"}},
|
||||||
|
TaskTemplate: swarm.TaskSpec{ContainerSpec: nil},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
d := discovery.NewSwarm(fake, "se.shcizo.auto-update")
|
||||||
|
jobs, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Empty(t, jobs)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 3: Run tests to verify they fail**
|
||||||
|
|
||||||
|
Run: `go test ./internal/discovery/... -run TestSwarmFindJobs -v`
|
||||||
|
Expected: FAIL to compile — `discovery.NewSwarm` and `discovery.SwarmDockerClient` don't exist yet.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Implement**
|
||||||
|
|
||||||
|
Create `internal/discovery/swarm.go`:
|
||||||
|
|
||||||
|
```go
|
||||||
|
package discovery
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
"github.com/docker/docker/api/types"
|
||||||
|
"github.com/docker/docker/api/types/swarm"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SwarmDockerClient is the subset of the Docker SDK SwarmDiscovery depends
|
||||||
|
// on. Defined as an interface so tests can supply a fake.
|
||||||
|
type SwarmDockerClient interface {
|
||||||
|
ServiceList(ctx context.Context, opts types.ServiceListOptions) ([]swarm.Service, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// SwarmDiscovery orchestrates "given an image, which Swarm services should
|
||||||
|
// we update?". Unlike Discovery, it has no filesystem path to check — the
|
||||||
|
// opt-in label on the service itself is the only gate, since Swarm services
|
||||||
|
// have no local compose file to anchor a STACKS_ROOT check against.
|
||||||
|
type SwarmDiscovery struct {
|
||||||
|
cli SwarmDockerClient
|
||||||
|
optInLabel string
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSwarm returns a SwarmDiscovery bound to the given Docker client and
|
||||||
|
// opt-in label.
|
||||||
|
func NewSwarm(cli SwarmDockerClient, optInLabel string) *SwarmDiscovery {
|
||||||
|
return &SwarmDiscovery{cli: cli, optInLabel: optInLabel}
|
||||||
|
}
|
||||||
|
|
||||||
|
// FindJobs lists Swarm services, filters by image match + opt-in label, and
|
||||||
|
// returns one Job per matching service. Signature matches Discovery.FindJobs
|
||||||
|
// so both satisfy api.Finder.
|
||||||
|
func (d *SwarmDiscovery) FindJobs(ctx context.Context, image string) ([]Job, error) {
|
||||||
|
services, err := d.cli.ServiceList(ctx, types.ServiceListOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("docker service list: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var jobs []Job
|
||||||
|
for _, svc := range services {
|
||||||
|
if svc.Spec.TaskTemplate.ContainerSpec == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !ImagesMatch(image, svc.Spec.TaskTemplate.ContainerSpec.Image) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !HasOptIn(svc.Spec.Labels, d.optInLabel) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
jobs = append(jobs, Job{
|
||||||
|
Service: svc.Spec.Name,
|
||||||
|
ServiceID: svc.ID,
|
||||||
|
Image: image,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
return jobs, nil
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 5: Run tests to verify they pass**
|
||||||
|
|
||||||
|
Run: `go test ./internal/discovery/... -v`
|
||||||
|
Expected: PASS — all discovery tests including the four new `TestSwarmFindJobs_*` cases, no regressions.
|
||||||
|
|
||||||
|
- [ ] **Step 6: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add internal/discovery/swarm.go internal/discovery/swarm_test.go
|
||||||
|
git commit -m "feat(discovery): add SwarmDiscovery to find opted-in Swarm services by image"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 4: `SwarmExecutor` — run `docker service update --image`
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `internal/updater/swarm_executor.go`
|
||||||
|
- Test: `internal/updater/swarm_executor_test.go`
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: `discovery.Job{Service, ServiceID, Image, Refused, RefusedReason}` (Task 2/3).
|
||||||
|
- Produces: `updater.SwarmDockerClient` interface, `updater.NewSwarmExecutor(cli SwarmDockerClient) *SwarmExecutor`, method `(*SwarmExecutor) Execute(ctx context.Context, job discovery.Job) error` — satisfies `updater.Executor` (`internal/updater/executor.go`), so `Queue` (`internal/updater/queue.go`) needs no changes.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Verify the Docker SDK method signatures before writing code**
|
||||||
|
|
||||||
|
Run: `go doc github.com/docker/docker/client.Client.ServiceInspectWithRaw` and `go doc github.com/docker/docker/client.Client.ServiceUpdate`
|
||||||
|
|
||||||
|
Expected output shape (confirm before proceeding — adjust the interface in Step 3 if the SDK reports something different):
|
||||||
|
|
||||||
|
```
|
||||||
|
func (cli *Client) ServiceInspectWithRaw(ctx context.Context, serviceID string, options types.ServiceInspectOptions) (swarm.Service, []byte, error)
|
||||||
|
func (cli *Client) ServiceUpdate(ctx context.Context, serviceID string, version swarm.Version, service swarm.ServiceSpec, options types.ServiceUpdateOptions) (types.ServiceUpdateResponse, error)
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Write the failing tests**
|
||||||
|
|
||||||
|
Create `internal/updater/swarm_executor_test.go`:
|
||||||
|
|
||||||
|
```go
|
||||||
|
package updater_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/docker/docker/api/types"
|
||||||
|
"github.com/docker/docker/api/types/swarm"
|
||||||
|
"github.com/shcizo/package-updater/internal/discovery"
|
||||||
|
"github.com/shcizo/package-updater/internal/updater"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
)
|
||||||
|
|
||||||
|
type fakeSwarmDockerClient struct {
|
||||||
|
inspectService swarm.Service
|
||||||
|
inspectErr error
|
||||||
|
updateErr error
|
||||||
|
|
||||||
|
updateCalledServiceID string
|
||||||
|
updateCalledVersion swarm.Version
|
||||||
|
updateCalledSpec swarm.ServiceSpec
|
||||||
|
updateCalledOpts types.ServiceUpdateOptions
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSwarmDockerClient) ServiceInspectWithRaw(_ context.Context, _ string, _ types.ServiceInspectOptions) (swarm.Service, []byte, error) {
|
||||||
|
return f.inspectService, nil, f.inspectErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSwarmDockerClient) ServiceUpdate(_ context.Context, serviceID string, version swarm.Version, spec swarm.ServiceSpec, opts types.ServiceUpdateOptions) (types.ServiceUpdateResponse, error) {
|
||||||
|
f.updateCalledServiceID = serviceID
|
||||||
|
f.updateCalledVersion = version
|
||||||
|
f.updateCalledSpec = spec
|
||||||
|
f.updateCalledOpts = opts
|
||||||
|
return types.ServiceUpdateResponse{}, f.updateErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func mkInspectService(version uint64, image string) swarm.Service {
|
||||||
|
return swarm.Service{
|
||||||
|
ID: "svc-myapp",
|
||||||
|
Version: swarm.Version{Index: version},
|
||||||
|
Spec: swarm.ServiceSpec{
|
||||||
|
Annotations: swarm.Annotations{Name: "myapp_web"},
|
||||||
|
TaskTemplate: swarm.TaskSpec{ContainerSpec: &swarm.ContainerSpec{Image: image}},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmExecutor_UpdatesImageAndVersion(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{inspectService: mkInspectService(7, "registry.example.com/myapp:v1")}
|
||||||
|
e := updater.NewSwarmExecutor(fake)
|
||||||
|
|
||||||
|
job := discovery.Job{Service: "myapp_web", ServiceID: "svc-myapp", Image: "registry.example.com/myapp:v2"}
|
||||||
|
err := e.Execute(context.Background(), job)
|
||||||
|
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Equal(t, "svc-myapp", fake.updateCalledServiceID)
|
||||||
|
require.Equal(t, uint64(7), fake.updateCalledVersion.Index)
|
||||||
|
require.Equal(t, "registry.example.com/myapp:v2", fake.updateCalledSpec.TaskTemplate.ContainerSpec.Image)
|
||||||
|
require.True(t, fake.updateCalledOpts.QueryRegistry)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmExecutor_InspectError(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{inspectErr: errors.New("service not found")}
|
||||||
|
e := updater.NewSwarmExecutor(fake)
|
||||||
|
err := e.Execute(context.Background(), discovery.Job{ServiceID: "svc-missing", Image: "x:v2"})
|
||||||
|
require.Error(t, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmExecutor_UpdateError(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{
|
||||||
|
inspectService: mkInspectService(1, "registry.example.com/myapp:v1"),
|
||||||
|
updateErr: errors.New("update rejected"),
|
||||||
|
}
|
||||||
|
e := updater.NewSwarmExecutor(fake)
|
||||||
|
err := e.Execute(context.Background(), discovery.Job{ServiceID: "svc-myapp", Image: "registry.example.com/myapp:v2"})
|
||||||
|
require.Error(t, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmExecutor_RefusedJobIsNotExecuted(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{}
|
||||||
|
e := updater.NewSwarmExecutor(fake)
|
||||||
|
job := discovery.Job{ServiceID: "svc-myapp", Image: "x:v2", Refused: true, RefusedReason: "not opted in"}
|
||||||
|
err := e.Execute(context.Background(), job)
|
||||||
|
require.Error(t, err)
|
||||||
|
require.Empty(t, fake.updateCalledServiceID)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 3: Run tests to verify they fail**
|
||||||
|
|
||||||
|
Run: `go test ./internal/updater/... -run TestSwarmExecutor -v`
|
||||||
|
Expected: FAIL to compile — `updater.NewSwarmExecutor` doesn't exist yet.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Implement**
|
||||||
|
|
||||||
|
Create `internal/updater/swarm_executor.go`:
|
||||||
|
|
||||||
|
```go
|
||||||
|
package updater
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
"github.com/docker/docker/api/types"
|
||||||
|
"github.com/docker/docker/api/types/swarm"
|
||||||
|
"github.com/shcizo/package-updater/internal/discovery"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SwarmDockerClient is the subset of the Docker SDK SwarmExecutor depends
|
||||||
|
// on. Defined as an interface so tests can supply a fake.
|
||||||
|
type SwarmDockerClient interface {
|
||||||
|
ServiceInspectWithRaw(ctx context.Context, serviceID string, opts types.ServiceInspectOptions) (swarm.Service, []byte, error)
|
||||||
|
ServiceUpdate(ctx context.Context, serviceID string, version swarm.Version, service swarm.ServiceSpec, opts types.ServiceUpdateOptions) (types.ServiceUpdateResponse, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// SwarmExecutor updates a Swarm service's image via the Docker API,
|
||||||
|
// equivalent to `docker service update --image`.
|
||||||
|
type SwarmExecutor struct {
|
||||||
|
cli SwarmDockerClient
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSwarmExecutor returns an executor that drives Swarm service updates
|
||||||
|
// through the Docker API.
|
||||||
|
func NewSwarmExecutor(cli SwarmDockerClient) *SwarmExecutor {
|
||||||
|
return &SwarmExecutor{cli: cli}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Execute inspects the service to get its current spec + version (required
|
||||||
|
// by the Docker API as an optimistic-concurrency token), sets the new image
|
||||||
|
// on the container spec, and calls ServiceUpdate. QueryRegistry is set so a
|
||||||
|
// floating tag (e.g. ":latest") resolves to a fresh digest and actually
|
||||||
|
// triggers a rolling update instead of being treated as unchanged.
|
||||||
|
func (e *SwarmExecutor) Execute(ctx context.Context, job discovery.Job) error {
|
||||||
|
if job.Refused {
|
||||||
|
return fmt.Errorf("refused: %s", job.RefusedReason)
|
||||||
|
}
|
||||||
|
|
||||||
|
svc, _, err := e.cli.ServiceInspectWithRaw(ctx, job.ServiceID, types.ServiceInspectOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("inspect service %s: %w", job.ServiceID, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
spec := svc.Spec
|
||||||
|
spec.TaskTemplate.ContainerSpec.Image = job.Image
|
||||||
|
|
||||||
|
if _, err := e.cli.ServiceUpdate(ctx, job.ServiceID, svc.Version, spec, types.ServiceUpdateOptions{QueryRegistry: true}); err != nil {
|
||||||
|
return fmt.Errorf("update service %s: %w", job.ServiceID, err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 5: Run tests to verify they pass**
|
||||||
|
|
||||||
|
Run: `go test ./internal/updater/... -v`
|
||||||
|
Expected: PASS — all updater tests including the four new `TestSwarmExecutor_*` cases, no regressions in `TestQueue_*` or `TestWorker_*`.
|
||||||
|
|
||||||
|
- [ ] **Step 6: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add internal/updater/swarm_executor.go internal/updater/swarm_executor_test.go
|
||||||
|
git commit -m "feat(updater): add SwarmExecutor to run docker service update via the Docker API"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 5: Surface `ServiceID` in the API response
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `internal/api/types.go`
|
||||||
|
- Modify: `internal/api/handlers.go`
|
||||||
|
- Test: `internal/api/swarm_result_test.go`
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: `discovery.Job.ServiceID` (Task 2), `updater.Result{Job discovery.Job, Status Status, Error string, DurationMs int64}` (existing, `internal/updater/queue.go`).
|
||||||
|
- Produces: `api.ResultDTO` gains `ServiceID string` (json `service_id`, omitempty) alongside the existing `ComposeFile` field, which stays empty for Swarm jobs (already guarded by the existing `len(res.Job.ConfigFiles) > 0` check).
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write the failing test**
|
||||||
|
|
||||||
|
Create `internal/api/swarm_result_test.go`:
|
||||||
|
|
||||||
|
```go
|
||||||
|
package api_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/docker/docker/api/types"
|
||||||
|
"github.com/shcizo/package-updater/internal/api"
|
||||||
|
"github.com/shcizo/package-updater/internal/discovery"
|
||||||
|
"github.com/shcizo/package-updater/internal/updater"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
)
|
||||||
|
|
||||||
|
type swarmResultFinder struct{ jobs []discovery.Job }
|
||||||
|
|
||||||
|
func (f *swarmResultFinder) FindJobs(_ context.Context, _ string) ([]discovery.Job, error) {
|
||||||
|
return f.jobs, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type swarmResultSubmitter struct{ results []updater.Result }
|
||||||
|
|
||||||
|
func (s *swarmResultSubmitter) Submit(_ context.Context, _ []discovery.Job) []updater.Result {
|
||||||
|
return s.results
|
||||||
|
}
|
||||||
|
|
||||||
|
type swarmResultPinger struct{}
|
||||||
|
|
||||||
|
func (swarmResultPinger) Ping(_ context.Context) (types.Ping, error) { return types.Ping{}, nil }
|
||||||
|
|
||||||
|
func TestUpdate_SwarmJobPopulatesServiceID(t *testing.T) {
|
||||||
|
job := discovery.Job{Service: "myapp_web", ServiceID: "svc123", Image: "registry.example.com/myapp:v2"}
|
||||||
|
finder := &swarmResultFinder{jobs: []discovery.Job{job}}
|
||||||
|
submitter := &swarmResultSubmitter{results: []updater.Result{
|
||||||
|
{Job: job, Status: updater.StatusUpdated, DurationMs: 42},
|
||||||
|
}}
|
||||||
|
h := api.NewHandlers(finder, submitter, swarmResultPinger{}, "v", "c", "b", nil)
|
||||||
|
|
||||||
|
req := httptest.NewRequest(http.MethodPost, "/update", strings.NewReader(`{"image":"registry.example.com/myapp"}`))
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
h.Update(rec, req)
|
||||||
|
|
||||||
|
require.Equal(t, http.StatusOK, rec.Code)
|
||||||
|
require.Contains(t, rec.Body.String(), `"service_id":"svc123"`)
|
||||||
|
require.Contains(t, rec.Body.String(), `"service":"myapp_web"`)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run test to verify it fails**
|
||||||
|
|
||||||
|
Run: `go test ./internal/api/... -run TestUpdate_SwarmJobPopulatesServiceID -v`
|
||||||
|
Expected: FAIL — response body has no `service_id` key (`ResultDTO` has no such field yet).
|
||||||
|
|
||||||
|
- [ ] **Step 3: Implement**
|
||||||
|
|
||||||
|
In `internal/api/types.go`, add the field to `ResultDTO`:
|
||||||
|
|
||||||
|
```go
|
||||||
|
// ResultDTO is one row in UpdateResponse.Results.
|
||||||
|
type ResultDTO struct {
|
||||||
|
Project string `json:"project"`
|
||||||
|
Service string `json:"service"`
|
||||||
|
ComposeFile string `json:"compose_file"`
|
||||||
|
ServiceID string `json:"service_id,omitempty"`
|
||||||
|
Status string `json:"status"`
|
||||||
|
Error string `json:"error,omitempty"`
|
||||||
|
DurationMs int64 `json:"duration_ms"`
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
In `internal/api/handlers.go`, inside the `Update` handler's results loop, add `ServiceID` to the constructed `ResultDTO`:
|
||||||
|
|
||||||
|
```go
|
||||||
|
resp.Results = append(resp.Results, ResultDTO{
|
||||||
|
Project: res.Job.Project,
|
||||||
|
Service: res.Job.Service,
|
||||||
|
ComposeFile: composeFile,
|
||||||
|
ServiceID: res.Job.ServiceID,
|
||||||
|
Status: string(res.Status),
|
||||||
|
Error: res.Error,
|
||||||
|
DurationMs: res.DurationMs,
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run tests to verify they pass**
|
||||||
|
|
||||||
|
Run: `go test ./internal/api/... -v`
|
||||||
|
Expected: PASS — the new test plus all existing `TestUpdate_*`, `TestHealthz_*`, `TestVersion` cases (they don't assert on absence of `service_id`, and `omitempty` keeps it out of Compose-mode responses since `ServiceID` is `""` for those).
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add internal/api/types.go internal/api/handlers.go internal/api/swarm_result_test.go
|
||||||
|
git commit -m "feat(api): surface ServiceID in update results for swarm jobs"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 6: Wire `MODE` in `cmd/server/main.go`
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `cmd/server/main.go`
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: `config.Config.Mode` (Task 1), `discovery.New` / `discovery.NewSwarm` (Task 3), `updater.NewComposeExecutor` / `updater.NewSwarmExecutor` (Task 4), `api.Finder` / `updater.Executor` (existing).
|
||||||
|
- Produces: nothing new for other tasks to consume — this is the final wiring step.
|
||||||
|
|
||||||
|
There is no unit test for `main.go` in this codebase (it's pure wiring); verification is a successful build plus the full test suite passing, run in Step 3.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Modify the wiring**
|
||||||
|
|
||||||
|
In `cmd/server/main.go`, replace:
|
||||||
|
|
||||||
|
```go
|
||||||
|
disc := discovery.New(dockerCli, cfg.StacksRoot, cfg.OptInLabel)
|
||||||
|
exec := updater.NewComposeExecutor()
|
||||||
|
```
|
||||||
|
|
||||||
|
with:
|
||||||
|
|
||||||
|
```go
|
||||||
|
var finder api.Finder
|
||||||
|
var exec updater.Executor
|
||||||
|
switch cfg.Mode {
|
||||||
|
case "swarm":
|
||||||
|
finder = discovery.NewSwarm(dockerCli, cfg.OptInLabel)
|
||||||
|
exec = updater.NewSwarmExecutor(dockerCli)
|
||||||
|
case "compose":
|
||||||
|
finder = discovery.New(dockerCli, cfg.StacksRoot, cfg.OptInLabel)
|
||||||
|
exec = updater.NewComposeExecutor()
|
||||||
|
default:
|
||||||
|
// Unreachable: config.Load already validates Mode, but guard
|
||||||
|
// here too rather than silently falling through to a nil
|
||||||
|
// finder/exec pair.
|
||||||
|
return fmt.Errorf("unknown MODE %q", cfg.Mode)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Then update the `NewHandlers` call, which currently reads:
|
||||||
|
|
||||||
|
```go
|
||||||
|
handlers := api.NewHandlers(disc, &submitterAdapter{queue: queue, timeout: cfg.UpdateTimeout}, dockerCli, version, commit, buildTime, m)
|
||||||
|
```
|
||||||
|
|
||||||
|
to use `finder` instead of `disc`:
|
||||||
|
|
||||||
|
```go
|
||||||
|
handlers := api.NewHandlers(finder, &submitterAdapter{queue: queue, timeout: cfg.UpdateTimeout}, dockerCli, version, commit, buildTime, m)
|
||||||
|
```
|
||||||
|
|
||||||
|
Note `queue := updater.NewQueue(exec, m)` (a few lines above) already references `exec` — no change needed there since `exec` is now declared by the switch instead of a direct `updater.NewComposeExecutor()` call, but it's the same variable name so the line is unchanged. Also update the startup log line to include the mode:
|
||||||
|
|
||||||
|
```go
|
||||||
|
logger.Info("starting",
|
||||||
|
"version", version, "commit", commit, "port", cfg.Port,
|
||||||
|
"mode", cfg.Mode, "stacks_root", cfg.StacksRoot, "opt_in_label", cfg.OptInLabel,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Build**
|
||||||
|
|
||||||
|
Run: `go build ./...`
|
||||||
|
Expected: builds cleanly with no errors.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Run the full test suite**
|
||||||
|
|
||||||
|
Run: `go test ./...`
|
||||||
|
Expected: PASS across all packages — this exercises every task in this plan together for the first time.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Manual smoke check (compose mode, default)**
|
||||||
|
|
||||||
|
Run: `go run ./cmd/server &` (with `UPDATER_API_KEY` set) and check the startup log line shows `"mode":"compose"` (or the equivalent field in whatever log format `internal/logging` renders). Stop the process afterward.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add cmd/server/main.go
|
||||||
|
git commit -m "feat: wire MODE config to select compose or swarm discovery/executor"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 7: Documentation
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `README.md`
|
||||||
|
- Modify: `CLAUDE.md`
|
||||||
|
|
||||||
|
**Interfaces:** None — documentation only, no code interfaces produced or consumed.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Update `README.md`**
|
||||||
|
|
||||||
|
Read the current README first to match its existing structure (env var table, "known limitations" section, etc. — check what's there before editing rather than guessing the exact heading names). Add:
|
||||||
|
- `MODE` to the environment variable table: `MODE` — `compose` (default) or `swarm`. Selects the update mechanism for the whole deployment; not mixed per-request.
|
||||||
|
- A short "Swarm mode" subsection explaining: services must carry the opt-in label (`OPT_IN_LABEL`, default `se.shcizo.auto-update`) directly on the *service* (`docker service update --label-add se.shcizo.auto-update=true <service>` or set in the stack file's `deploy.labels`), not on the container/task, since Swarm mode reads `Service.Spec.Labels`. Also note the updater must run on/against a Swarm manager node (`docker service update` requires manager API access) — pointing `DOCKER_HOST` at a manager, or scheduling the updater container itself on a manager node with the socket mounted, is the operator's responsibility.
|
||||||
|
- Update/remove the existing "single host only" limitation note (if present) to say Compose mode remains single-host; Swarm mode is the multi-node path but only from a manager node's point of view.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Update `CLAUDE.md`**
|
||||||
|
|
||||||
|
In the "Architecture" section, add a one-line entry for the new files:
|
||||||
|
```
|
||||||
|
- `internal/discovery/swarm.go` — SwarmDiscovery: same FindJobs signature, lists `docker service ls`, gates on service-level opt-in label instead of STACKS_ROOT path-check
|
||||||
|
- `internal/updater/swarm_executor.go` — SwarmExecutor: `docker service update --image` via the Docker API instead of a `docker compose` subprocess
|
||||||
|
```
|
||||||
|
|
||||||
|
In "Design intent (do not break without discussion)", add:
|
||||||
|
```
|
||||||
|
- **Compose mode and Swarm mode are selected once per deployment via `MODE`**, never
|
||||||
|
mixed at request time. Swarm mode's security gate is opt-in label only — there is
|
||||||
|
no STACKS_ROOT-equivalent path check, since Swarm services have no local compose
|
||||||
|
file. Don't add one; don't weaken Compose mode's three-factor gate to match.
|
||||||
|
```
|
||||||
|
|
||||||
|
In "Gotchas", add:
|
||||||
|
```
|
||||||
|
- **Swarm mode requires manager-node API access.** `docker service update` fails
|
||||||
|
with a permission error against a worker-only node. This is an operator/deployment
|
||||||
|
concern (point `DOCKER_HOST` at a manager, or schedule the updater on a manager),
|
||||||
|
not something the code can detect or work around.
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add README.md CLAUDE.md
|
||||||
|
git commit -m "docs: document MODE=swarm support and its opt-in label / manager-node requirements"
|
||||||
|
```
|
||||||
@@ -0,0 +1,511 @@
|
|||||||
|
# Multi-Server Deployment & CI Fan-Out Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Land the finished Swarm support on `main`, document the one-instance-per-server deployment model, and make the Gitea composite action post `/update` to every server in a fleet instead of a single endpoint.
|
||||||
|
|
||||||
|
**Architecture:** No Go code changes. Part 1 is a merge plus documentation reframing — `MODE=compose|swarm` stays exclusive per instance, because each instance owns exactly one Docker daemon. Part 2 changes `gitea-action/action.yml` so its `endpoint` input accepts a newline-separated list, looping over every URL and continuing past failures so one dead host cannot stop the rest of the fleet from updating.
|
||||||
|
|
||||||
|
**Tech Stack:** Go 1.26.3 (unchanged, no code edits), Bash inside a Gitea composite action, `curl`, `jq`.
|
||||||
|
|
||||||
|
## Global Constraints
|
||||||
|
|
||||||
|
- **No Go source changes in this plan.** If a task seems to need one, stop and re-read the spec — the design explicitly rejects composite Finder/Executor work.
|
||||||
|
- `MODE` stays `compose` or `swarm` only. Do not add a `both` value.
|
||||||
|
- Do not weaken Compose mode's three-factor gate (token AND opt-in label AND `STACKS_ROOT` prefix).
|
||||||
|
- Existing single-URL usages of the action must keep working unchanged — a single URL is a one-element list. Do not add a separate input for the list form.
|
||||||
|
- Separator for the endpoint list is **newline**, never comma. URLs may legally contain commas.
|
||||||
|
- Conventional Commits (`feat:`, `fix:`, `docs:`, `ci:`, `build:`, `chore:`).
|
||||||
|
- Full test suite (`go test ./...`) must be green at the end of every task that touches the repo.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: Merge Swarm support to `main`
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- No files edited. This task merges branch `worktree-docker-swarm-support` (10 commits) into `main`.
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: nothing.
|
||||||
|
- Produces: `main` gains `internal/discovery/swarm.go`, `internal/updater/swarm_executor.go`, `config.Config.Mode`, `discovery.Job.Image`, `discovery.Job.ServiceID`, and `api.ResultDTO.ServiceID`. Task 2 edits documentation files that this merge brings in (`README.md` "Swarm mode" section, `CLAUDE.md` swarm entries).
|
||||||
|
|
||||||
|
- [ ] **Step 1: Confirm the branch is where you think it is**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
cd /Users/samuelenocsson/dev/package-updater
|
||||||
|
git log --oneline main..worktree-docker-swarm-support
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: exactly 10 commits, oldest `8f999c6 docs: add docker swarm support implementation plan`, newest `db84a2f fix(api): plumb request tag into discovery so swarm mode deploys the requested version`.
|
||||||
|
|
||||||
|
If the list differs, stop and report — someone has moved the branch since this plan was written.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify the branch is green before merging**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
cd /Users/samuelenocsson/dev/package-updater/.claude/worktrees/docker-swarm-support
|
||||||
|
go build ./... && go test ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: build succeeds, all packages PASS. Do not proceed on a red branch — fix or report first.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Merge into `main`**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /Users/samuelenocsson/dev/package-updater
|
||||||
|
git checkout main
|
||||||
|
git merge --no-ff worktree-docker-swarm-support -m "$(cat <<'EOF'
|
||||||
|
Merge branch 'worktree-docker-swarm-support'
|
||||||
|
|
||||||
|
Adds MODE=compose|swarm. Each instance owns one Docker daemon and runs
|
||||||
|
one paradigm; see docs/superpowers/specs/2026-08-04-multi-server-fanout-design.md
|
||||||
|
for why this exclusivity is the intended design.
|
||||||
|
EOF
|
||||||
|
)"
|
||||||
|
```
|
||||||
|
|
||||||
|
`--no-ff` is deliberate: it keeps the ten swarm commits grouped under one merge commit, matching how PRs #1 and #2 already appear in this repo's history.
|
||||||
|
|
||||||
|
Note: `docs/superpowers/plans/2026-07-04-docker-swarm-support.md` currently exists as an *untracked* file on `main` and is *committed* on the swarm branch. The merge brings in the tracked copy. If git refuses the merge with "untracked working tree file would be overwritten", delete the untracked copy first (`rm docs/superpowers/plans/2026-07-04-docker-swarm-support.md`) — the branch's committed version is identical and authoritative.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Verify `main` is green after the merge**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
go build ./... && go test ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: build succeeds, all packages PASS. This is the first time the swarm code and the post-swarm `main` commits (`CLAUDE.md`, CI workflow) are compiled together on `main`.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Push**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git push origin main
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 2: Document the deployment topology
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `README.md` (the version merged in Task 1 — it already contains a "Swarm mode" section at lines ~30-47, a `MODE` row in the config table at line ~71, and a "Known v1 gaps" bullet about single-host at lines ~103-105)
|
||||||
|
- Modify: `CLAUDE.md` (the "Design intent" section, which after Task 1 contains a "Compose mode and Swarm mode are selected once per deployment via `MODE`" entry)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: the merged documentation from Task 1.
|
||||||
|
- Produces: nothing consumed by later tasks. Task 3 documents the action separately in `gitea-action/README.md`.
|
||||||
|
|
||||||
|
There is no test for documentation. Verification is reading it back and confirming the claims match the code in `cmd/server/main.go` and `internal/config/config.go`.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Rebase the working branch onto the updated `main`**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /Users/samuelenocsson/dev/package-updater
|
||||||
|
git checkout feat/multi-server-fanout
|
||||||
|
git rebase main
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: clean rebase. The branch currently holds one commit (`docs: design for multi-server deployment and CI fan-out`) touching only a new spec file, so there is nothing to conflict with.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Add a deployment-topology section to `README.md`**
|
||||||
|
|
||||||
|
Insert this section immediately **after** the "Swarm mode" section and **before** "## Quick start":
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
## Deploying across multiple servers
|
||||||
|
|
||||||
|
An instance can only reach the Docker daemon it is configured against. It cannot
|
||||||
|
update stacks on other machines. The deployment model follows from that:
|
||||||
|
|
||||||
|
**One instance per server.** Each server runs its own package-updater against its
|
||||||
|
own local Docker socket, with `MODE` set to whatever that machine runs:
|
||||||
|
|
||||||
|
| Server | `MODE` | Updates |
|
||||||
|
|---|---|---|
|
||||||
|
| Swarm manager node | `swarm` | All Swarm services in the cluster |
|
||||||
|
| Standalone Compose host | `compose` | Compose stacks on that host |
|
||||||
|
|
||||||
|
**CI fans out.** The [Gitea composite action](gitea-action/README.md) takes a list of
|
||||||
|
endpoints and posts `/update` to every instance, so one workflow run reaches the
|
||||||
|
whole fleet. Each instance answers for its own machine; there is no aggregated
|
||||||
|
cross-server response and no instance coordinates any other.
|
||||||
|
|
||||||
|
This is why `MODE` is exclusive rather than a mode that handles both at once: an
|
||||||
|
instance that could do both would still only reach one daemon, so the extra
|
||||||
|
generality buys nothing.
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 3: Reword the `MODE` row in the configuration table**
|
||||||
|
|
||||||
|
In `README.md`'s configuration table, replace this row:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
| `MODE` | no | `compose` | `compose` or `swarm`. Selects the update mechanism for the whole deployment; not mixed per-request. |
|
||||||
|
```
|
||||||
|
|
||||||
|
with:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
| `MODE` | no | `compose` | `compose` or `swarm`. Which paradigm *this instance's* Docker daemon runs. See [Deploying across multiple servers](#deploying-across-multiple-servers). |
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 4: Replace the "single host only" gap bullet**
|
||||||
|
|
||||||
|
In `README.md`'s "Known v1 gaps" section, replace this bullet:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
- **Single host only in Compose mode**. Swarm mode (`MODE=swarm`) is the
|
||||||
|
multi-node path, but only from a manager node's point of view — the updater
|
||||||
|
itself still needs manager API access (see "Swarm mode" above).
|
||||||
|
```
|
||||||
|
|
||||||
|
with:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
- **One instance reaches one daemon.** An instance never updates another server;
|
||||||
|
fleets run one instance per server with CI fanning out to all of them (see
|
||||||
|
"Deploying across multiple servers"). Swarm mode is the exception in that a
|
||||||
|
single manager-node instance covers the whole cluster.
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 5: Reword the design-intent entry in `CLAUDE.md`**
|
||||||
|
|
||||||
|
In `CLAUDE.md`'s "Design intent (do not break without discussion)" section, replace this entry:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
- **Compose mode and Swarm mode are selected once per deployment via `MODE`**, never
|
||||||
|
mixed at request time. Swarm mode's security gate is opt-in label only — there is
|
||||||
|
no STACKS_ROOT-equivalent path check, since Swarm services have no local compose
|
||||||
|
file. Don't add one; don't weaken Compose mode's three-factor gate to match.
|
||||||
|
```
|
||||||
|
|
||||||
|
with:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
- **`MODE` is exclusive because each instance owns exactly one Docker daemon.**
|
||||||
|
A composite "handle both at once" mode has been considered and rejected: it would
|
||||||
|
still only reach one daemon, so it buys nothing. Fleets run one instance per
|
||||||
|
server and CI fans out. Reopen this only if a Swarm manager node starts running
|
||||||
|
standalone Compose stacks locally. Rationale and rejected alternatives:
|
||||||
|
`docs/superpowers/specs/2026-08-04-multi-server-fanout-design.md`.
|
||||||
|
- **Swarm mode's security gate is opt-in label only** — there is no
|
||||||
|
STACKS_ROOT-equivalent path check, since Swarm services have no local compose
|
||||||
|
file. Don't add one; don't weaken Compose mode's three-factor gate to match.
|
||||||
|
```
|
||||||
|
|
||||||
|
Splitting the original entry in two is deliberate: the mode-exclusivity rationale and the Swarm security-gate rule are separate rules that were sharing a bullet, and only the first one is changing.
|
||||||
|
|
||||||
|
- [ ] **Step 6: Verify the anchor link resolves**
|
||||||
|
|
||||||
|
The `MODE` table row links to `#deploying-across-multiple-servers`. Confirm the heading added in Step 2 is exactly `## Deploying across multiple servers` — GitHub/Gitea derive the anchor by lowercasing and replacing spaces with hyphens, so any wording drift silently breaks the link.
|
||||||
|
|
||||||
|
Run: `grep -n "^## Deploying across multiple servers" README.md`
|
||||||
|
Expected: one match.
|
||||||
|
|
||||||
|
- [ ] **Step 7: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add README.md CLAUDE.md
|
||||||
|
git commit -m "docs: describe one-instance-per-server topology and why MODE is exclusive"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 3: Fan out to multiple endpoints in the Gitea action
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `gitea-action/action.yml` (whole `runs.steps` block, lines 17-37, and the `endpoint` input description at lines 5-6)
|
||||||
|
- Modify: `gitea-action/README.md` (usage example, inputs table, failure-modes section)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: nothing from earlier tasks.
|
||||||
|
- Produces: the action's `endpoint` input accepts one URL (unchanged behaviour) or several separated by newlines. No other input changes name, type, or meaning.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Rewrite `gitea-action/action.yml`**
|
||||||
|
|
||||||
|
Replace the entire file with:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
name: "Deploy via package-updater"
|
||||||
|
description: "Notifies one or more package-updater instances to pull & restart a service"
|
||||||
|
inputs:
|
||||||
|
endpoint:
|
||||||
|
description: "Full URL to /update. Give several, one per line, to update a fleet."
|
||||||
|
required: true
|
||||||
|
image:
|
||||||
|
description: "Image reference without tag (e.g. registry.example.com/myapp)"
|
||||||
|
required: true
|
||||||
|
tag:
|
||||||
|
description: "Tag that was just pushed (for logging)"
|
||||||
|
required: false
|
||||||
|
default: ""
|
||||||
|
token:
|
||||||
|
description: "Bearer token for package-updater"
|
||||||
|
required: true
|
||||||
|
runs:
|
||||||
|
using: "composite"
|
||||||
|
steps:
|
||||||
|
- name: Trigger update
|
||||||
|
shell: bash
|
||||||
|
env:
|
||||||
|
ENDPOINTS: ${{ inputs.endpoint }}
|
||||||
|
IMAGE: ${{ inputs.image }}
|
||||||
|
TAG: ${{ inputs.tag }}
|
||||||
|
TOKEN: ${{ inputs.token }}
|
||||||
|
run: |
|
||||||
|
# No `set -e`: a failing endpoint must not abort the loop, or one dead
|
||||||
|
# server leaves the rest of the fleet un-updated and hides which hosts
|
||||||
|
# actually succeeded.
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
payload=$(jq -nc --arg image "$IMAGE" --arg tag "$TAG" \
|
||||||
|
'{image: $image, tag: $tag}')
|
||||||
|
|
||||||
|
attempted=0
|
||||||
|
failed=0
|
||||||
|
|
||||||
|
while IFS= read -r endpoint; do
|
||||||
|
# URLs never contain whitespace, so stripping all of it safely
|
||||||
|
# handles indentation, blank lines and CRLF line endings.
|
||||||
|
endpoint=$(printf '%s' "$endpoint" | tr -d '[:space:]')
|
||||||
|
[ -z "$endpoint" ] && continue
|
||||||
|
|
||||||
|
attempted=$((attempted + 1))
|
||||||
|
echo "--- $endpoint"
|
||||||
|
|
||||||
|
if ! response=$(curl -sS -w "\n%{http_code}" \
|
||||||
|
-X POST "$endpoint" \
|
||||||
|
-H "Authorization: Bearer $TOKEN" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d "$payload"); then
|
||||||
|
echo "unreachable"
|
||||||
|
failed=$((failed + 1))
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Split on the last newline: curl's -w appended the status code
|
||||||
|
# there. Pure bash — `head -n -1` is GNU-only and fails on
|
||||||
|
# BSD/macOS, so the fan-out could not be tested locally.
|
||||||
|
code="${response##*$'\n'}"
|
||||||
|
body="${response%$'\n'*}"
|
||||||
|
echo "HTTP $code"
|
||||||
|
printf '%s' "$body" | jq . || printf '%s\n' "$body"
|
||||||
|
|
||||||
|
if [ "$code" -ge 400 ]; then
|
||||||
|
failed=$((failed + 1))
|
||||||
|
fi
|
||||||
|
done <<< "$ENDPOINTS"
|
||||||
|
|
||||||
|
if [ "$attempted" -eq 0 ]; then
|
||||||
|
echo "no endpoints given"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if [ "$failed" -gt 0 ]; then
|
||||||
|
echo "$failed of $attempted endpoint(s) failed"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "all $attempted endpoint(s) updated"
|
||||||
|
```
|
||||||
|
|
||||||
|
Four changes beyond the loop itself, each load-bearing:
|
||||||
|
|
||||||
|
1. **All inputs moved into `env:`.** A multi-line `${{ inputs.endpoint }}` interpolated directly into the script body would break it syntactically. Moving the others too keeps one consistent style in a short script, and stops a value containing shell metacharacters from being executed.
|
||||||
|
2. **`jq -nc` builds the payload** instead of hand-interpolating into a JSON string. A tag containing a quote previously produced malformed JSON and a confusing 400.
|
||||||
|
3. **`set -e` dropped**, `-uo pipefail` kept. This is the whole point of the task — see the comment in the script.
|
||||||
|
4. **`head -n -1` replaced by bash parameter expansion.** The existing action used `head -n -1`, which is GNU-only — it fails on BSD/macOS with `illegal line count`. That made the fan-out logic impossible to test on a developer machine, so the portability fix is what makes Steps 2-4 runnable at all. It also drops two subprocesses per endpoint.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify the loop continues past a failure**
|
||||||
|
|
||||||
|
This is the behaviour that justifies the task, so test it directly rather than trusting the code by inspection.
|
||||||
|
|
||||||
|
Start a server that answers POST with 200:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 -c "
|
||||||
|
import http.server
|
||||||
|
class H(http.server.BaseHTTPRequestHandler):
|
||||||
|
def do_POST(self):
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header('Content-Type','application/json')
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(b'{\"matched\":1,\"results\":[]}')
|
||||||
|
def log_message(self, *a): pass
|
||||||
|
http.server.HTTPServer(('127.0.0.1',8099), H).serve_forever()
|
||||||
|
" &
|
||||||
|
echo $! > /tmp/fanout-test-server.pid
|
||||||
|
```
|
||||||
|
|
||||||
|
Write the script body to a runnable file. This must be the **exact** text of the
|
||||||
|
`run:` block from Step 1, dedented — copy it, do not retype it, or you are testing
|
||||||
|
something other than what ships:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cat > /tmp/fanout-test.sh <<'SCRIPT'
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
payload=$(jq -nc --arg image "$IMAGE" --arg tag "$TAG" \
|
||||||
|
'{image: $image, tag: $tag}')
|
||||||
|
|
||||||
|
attempted=0
|
||||||
|
failed=0
|
||||||
|
|
||||||
|
while IFS= read -r endpoint; do
|
||||||
|
endpoint=$(printf '%s' "$endpoint" | tr -d '[:space:]')
|
||||||
|
[ -z "$endpoint" ] && continue
|
||||||
|
|
||||||
|
attempted=$((attempted + 1))
|
||||||
|
echo "--- $endpoint"
|
||||||
|
|
||||||
|
if ! response=$(curl -sS -w "\n%{http_code}" \
|
||||||
|
-X POST "$endpoint" \
|
||||||
|
-H "Authorization: Bearer $TOKEN" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d "$payload"); then
|
||||||
|
echo "unreachable"
|
||||||
|
failed=$((failed + 1))
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
code="${response##*$'\n'}"
|
||||||
|
body="${response%$'\n'*}"
|
||||||
|
echo "HTTP $code"
|
||||||
|
printf '%s' "$body" | jq . || printf '%s\n' "$body"
|
||||||
|
|
||||||
|
if [ "$code" -ge 400 ]; then
|
||||||
|
failed=$((failed + 1))
|
||||||
|
fi
|
||||||
|
done <<< "$ENDPOINTS"
|
||||||
|
|
||||||
|
if [ "$attempted" -eq 0 ]; then
|
||||||
|
echo "no endpoints given"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if [ "$failed" -gt 0 ]; then
|
||||||
|
echo "$failed of $attempted endpoint(s) failed"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "all $attempted endpoint(s) updated"
|
||||||
|
SCRIPT
|
||||||
|
```
|
||||||
|
|
||||||
|
Now execute it with a list where the middle endpoint is dead (port 1 always refuses connections):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ENDPOINTS="http://127.0.0.1:8099/update
|
||||||
|
http://127.0.0.1:1/update
|
||||||
|
http://127.0.0.1:8099/update" \
|
||||||
|
IMAGE="registry.example.com/myapp" \
|
||||||
|
TAG="abc123" \
|
||||||
|
TOKEN="dummy" \
|
||||||
|
bash /tmp/fanout-test.sh
|
||||||
|
echo "exit code: $?"
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected output shape:
|
||||||
|
```
|
||||||
|
--- http://127.0.0.1:8099/update
|
||||||
|
HTTP 200
|
||||||
|
{ "matched": 1, "results": [] }
|
||||||
|
--- http://127.0.0.1:1/update
|
||||||
|
unreachable
|
||||||
|
--- http://127.0.0.1:8099/update
|
||||||
|
HTTP 200
|
||||||
|
{ "matched": 1, "results": [] }
|
||||||
|
1 of 3 endpoint(s) failed
|
||||||
|
exit code: 1
|
||||||
|
```
|
||||||
|
|
||||||
|
Three things must all hold: the **third** endpoint was attempted (proves the loop continued), the exit code is **1**, and each endpoint's outcome is visible. If the third `---` line is missing, `set -e` crept back in.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Verify a single endpoint still works unchanged**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ENDPOINTS="http://127.0.0.1:8099/update" \
|
||||||
|
IMAGE="registry.example.com/myapp" TAG="abc123" TOKEN="dummy" \
|
||||||
|
bash /tmp/fanout-test.sh
|
||||||
|
echo "exit code: $?"
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: one `--- ` line, `HTTP 200`, `all 1 endpoint(s) updated`, exit code 0. This is the existing-usage regression check.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Verify blank lines and indentation are tolerated**
|
||||||
|
|
||||||
|
YAML block scalars routinely produce leading indentation and a trailing newline, so this is the realistic input shape, not an edge case.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ENDPOINTS=" http://127.0.0.1:8099/update
|
||||||
|
|
||||||
|
http://127.0.0.1:8099/update
|
||||||
|
" \
|
||||||
|
IMAGE="registry.example.com/myapp" TAG="abc123" TOKEN="dummy" \
|
||||||
|
bash /tmp/fanout-test.sh
|
||||||
|
echo "exit code: $?"
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: exactly two `--- http://127.0.0.1:8099/update` lines with no stray whitespace in the URL, `all 2 endpoint(s) updated`, exit code 0.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Clean up the test server**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kill "$(cat /tmp/fanout-test-server.pid)" && rm -f /tmp/fanout-test-server.pid /tmp/fanout-test.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 6: Update `gitea-action/README.md`**
|
||||||
|
|
||||||
|
Replace the usage example's `with:` block so it shows the fleet form:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
- uses: gitea.example.com/shcizo/package-updater/gitea-action@v1
|
||||||
|
with:
|
||||||
|
endpoint: |
|
||||||
|
https://updater-swarm.example.com/update
|
||||||
|
https://updater-web01.example.com/update
|
||||||
|
https://updater-web02.example.com/update
|
||||||
|
image: registry.example.com/${{ gitea.repository }}
|
||||||
|
tag: ${{ gitea.sha }}
|
||||||
|
token: ${{ secrets.UPDATER_TOKEN }}
|
||||||
|
```
|
||||||
|
|
||||||
|
Change the `endpoint` row in the inputs table to:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
| `endpoint` | yes | — | Full URL to `/update`. Several may be given, one per line, to update a fleet. |
|
||||||
|
```
|
||||||
|
|
||||||
|
Replace the "Failure modes" section with:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
## Failure modes
|
||||||
|
|
||||||
|
Every endpoint is attempted, even when an earlier one fails — otherwise one dead
|
||||||
|
server would leave the rest of the fleet un-updated, and the CI log would not show
|
||||||
|
which hosts actually succeeded.
|
||||||
|
|
||||||
|
The step exits non-zero if any endpoint returned 4xx/5xx or was unreachable. The
|
||||||
|
log lists each endpoint with its HTTP status and response body, so a partial
|
||||||
|
deploy is visible at a glance.
|
||||||
|
```
|
||||||
|
|
||||||
|
Add this note after the `UPDATER_TOKEN` sentence:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
All instances in the fleet must share the same bearer token, since one `token`
|
||||||
|
input is sent to every endpoint.
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 7: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add gitea-action/action.yml gitea-action/README.md
|
||||||
|
git commit -m "feat(action): post /update to every endpoint in a fleet, continuing past failures"
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 8: Push and open a PR**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git push -u origin feat/multi-server-fanout
|
||||||
|
```
|
||||||
|
|
||||||
|
Then open a PR against `main` (see `issue-tracker-cli` skill for whether this repo uses `gh` or `tea`). The PR covers the spec, the documentation reframing, and the action fan-out; the Swarm merge landed separately in Task 1.
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
# Multi-Server Deployment & CI Fan-Out — Design
|
||||||
|
|
||||||
|
**Date:** 2026-08-04
|
||||||
|
**Status:** Approved, ready for implementation planning
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
package-updater was built as a single instance talking to a single local Docker
|
||||||
|
socket. Two things surfaced at once:
|
||||||
|
|
||||||
|
1. Swarm support (branch `worktree-docker-swarm-support`) added `MODE=compose|swarm`,
|
||||||
|
selecting one paradigm per deployment. This looked like an artificial limitation:
|
||||||
|
"why can't one instance do both?"
|
||||||
|
2. The real constraint is narrower and harder: an instance can only reach the Docker
|
||||||
|
daemon it is configured against. Compose stacks live on other, separate servers.
|
||||||
|
No amount of generalising the application makes one instance reach them.
|
||||||
|
|
||||||
|
The second point dissolves the first. The limitation was never in the application —
|
||||||
|
it was in the deployment topology (one instance for a fleet of servers).
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
**One package-updater instance per server. CI fans out to all of them.**
|
||||||
|
|
||||||
|
- Each server runs its own instance against its own local Docker socket.
|
||||||
|
- The Swarm manager node runs an instance with `MODE=swarm`.
|
||||||
|
- Each standalone Compose host runs an instance with `MODE=compose`.
|
||||||
|
- The Gitea CI action posts `/update` to every instance.
|
||||||
|
|
||||||
|
`MODE` exclusivity is therefore correct, not a defect: each instance owns exactly one
|
||||||
|
machine running exactly one paradigm. No composite Finder or routing Executor is built.
|
||||||
|
|
||||||
|
### Rejected alternatives
|
||||||
|
|
||||||
|
| Alternative | Why rejected |
|
||||||
|
|---|---|
|
||||||
|
| Composite Finder + routing Executor (both modes in one instance) | Solves nothing — a single instance still cannot reach other servers' daemons. Only justified if a Swarm manager node *also* ran standalone Compose stacks locally, which it does not. |
|
||||||
|
| One instance holding N Docker clients (`ssh://` / TCP+TLS to each host) | Technically viable (`client.FromEnv` already supports it) but requires `Job` to carry host identity, discovery to iterate clients, and creates a single point of failure with broad network credentials. Buys nothing over per-server instances given CI can reach every server. |
|
||||||
|
| Controller + agents (pull model) | Only needed if servers cannot accept inbound traffic. They can. Would be a genuinely different application. |
|
||||||
|
|
||||||
|
### Assumptions
|
||||||
|
|
||||||
|
These held at design time; revisit the decision if any changes:
|
||||||
|
|
||||||
|
- Every server is reachable from Gitea CI over HTTP.
|
||||||
|
- Per-server outcomes are sufficient; no aggregated cross-server report is needed.
|
||||||
|
- The Swarm manager node runs Swarm services only, no local Compose stacks.
|
||||||
|
- All instances share one bearer token (org-level Gitea secret `UPDATER_TOKEN`).
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
### Part 1 — Merge Swarm support to main
|
||||||
|
|
||||||
|
Branch `worktree-docker-swarm-support` (10 commits) is complete, builds clean, and
|
||||||
|
its tests pass. No code changes required. Documentation is reframed so `MODE`
|
||||||
|
describes *what paradigm this machine runs*, not a product-level restriction, and the
|
||||||
|
one-instance-per-server deployment model is written down — it is the expensive insight
|
||||||
|
of this design and is not derivable from the code.
|
||||||
|
|
||||||
|
Documentation changes:
|
||||||
|
|
||||||
|
- `README.md` — describe `MODE` per-instance; add a deployment-topology section
|
||||||
|
covering one instance per server and CI fan-out.
|
||||||
|
- `CLAUDE.md` — reword the "Compose mode and Swarm mode are selected once per
|
||||||
|
deployment" design-intent entry to state the reason (each instance owns one daemon),
|
||||||
|
so a future reader does not re-litigate "make it handle both".
|
||||||
|
|
||||||
|
### Part 2 — CI fan-out in `gitea-action/`
|
||||||
|
|
||||||
|
`gitea-action/action.yml`'s `endpoint` input accepts a **newline-separated list** of
|
||||||
|
`/update` URLs instead of a single URL.
|
||||||
|
|
||||||
|
Behaviour:
|
||||||
|
|
||||||
|
- Post to each endpoint in list order.
|
||||||
|
- **Continue through the whole list even when one fails.** With `set -e` and a naive
|
||||||
|
loop, an unreachable server B leaves server C never updated — a half-deployed fleet
|
||||||
|
where the failure also hides which hosts succeeded.
|
||||||
|
- Print HTTP status and response body per endpoint, so the CI log shows the fleet
|
||||||
|
state at a glance.
|
||||||
|
- Exit non-zero at the end if any endpoint returned 4xx/5xx or was unreachable.
|
||||||
|
|
||||||
|
A single URL remains a one-element list and keeps working unchanged. This is a
|
||||||
|
property of the format, not backward-compatibility work.
|
||||||
|
|
||||||
|
Newline rather than comma as separator: URLs may legally contain commas but never
|
||||||
|
line breaks, so the separator cannot collide with the data. Choosing an impossible
|
||||||
|
delimiter is cheaper than building escaping. This also matches the Actions convention
|
||||||
|
for list inputs (`paths`, `files`).
|
||||||
|
|
||||||
|
Blank lines and surrounding whitespace are ignored, so YAML block scalars with
|
||||||
|
trailing newlines and indentation behave as expected.
|
||||||
|
|
||||||
|
`README.md` in `gitea-action/` documents the list form with a worked multi-server
|
||||||
|
example.
|
||||||
|
|
||||||
|
## Out of scope
|
||||||
|
|
||||||
|
- Composite/routing discovery within one instance — not needed under this topology.
|
||||||
|
- Aggregated cross-server reporting — per-endpoint CI output is sufficient.
|
||||||
|
- Per-endpoint distinct tokens — all instances share one token.
|
||||||
|
- Wiring `internal/selfupdate` into the live queue — a pre-existing v1 gap, unrelated.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
- Part 1: existing suite (`go test ./...`) must stay green after merge. No new tests —
|
||||||
|
no new code.
|
||||||
|
- Part 2: the action is shell in composite YAML with no test harness in this repo.
|
||||||
|
Verification is manual: run the loop logic against a mix of reachable and
|
||||||
|
unreachable endpoints and confirm (a) every endpoint is attempted, (b) the step
|
||||||
|
exits non-zero, (c) each endpoint's status appears in the output.
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
# Deploy via package-updater (composite action)
|
||||||
|
|
||||||
|
Notifies one or more `package-updater` instances to update the matching service(s) after a CI build. Each instance does whatever its own `MODE` dictates — `docker compose pull` + `up -d`, or `docker service update` for swarm.
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
In a consumer repo's `.gitea/workflows/deploy.yml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
jobs:
|
||||||
|
deploy:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
needs: [build-and-push]
|
||||||
|
steps:
|
||||||
|
- uses: gitea.example.com/shcizo/package-updater/gitea-action@v1
|
||||||
|
with:
|
||||||
|
endpoint: |
|
||||||
|
https://updater-swarm.example.com/update
|
||||||
|
https://updater-web01.example.com/update
|
||||||
|
https://updater-web02.example.com/update
|
||||||
|
image: registry.example.com/${{ gitea.repository }}
|
||||||
|
tag: ${{ gitea.sha }}
|
||||||
|
token: ${{ secrets.UPDATER_TOKEN }}
|
||||||
|
```
|
||||||
|
|
||||||
|
`UPDATER_TOKEN` should be set as an organisation-level secret in Gitea so all repos share it.
|
||||||
|
|
||||||
|
All instances in the fleet must share the same bearer token, since one `token`
|
||||||
|
input is sent to every endpoint.
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
| Name | Required | Default | Description |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `endpoint` | yes | — | Full URL to `/update`. Several may be given, one per line, to update a fleet. |
|
||||||
|
| `image` | yes | — | Image reference without tag |
|
||||||
|
| `tag` | no | `""` | Tag that was just pushed. Compose mode ignores it (the compose file pins the reference); **swarm mode sets the service image to it**, so omitting it deploys `:latest`. Always pass it. |
|
||||||
|
| `token` | yes | — | Bearer token configured in package-updater |
|
||||||
|
|
||||||
|
## Failure modes
|
||||||
|
|
||||||
|
Every endpoint is attempted, even when an earlier one fails — otherwise one dead
|
||||||
|
server would leave the rest of the fleet un-updated, and the CI log would not show
|
||||||
|
which hosts actually succeeded.
|
||||||
|
|
||||||
|
The step exits non-zero if any endpoint returned 4xx/5xx or was unreachable. The
|
||||||
|
log lists each endpoint with its HTTP status and response body, so a partial
|
||||||
|
deploy is visible at a glance.
|
||||||
|
|
||||||
|
Endpoints are contacted sequentially, so worst-case wall time is the number of endpoints times
|
||||||
|
how long one update takes. Each request allows 10s to connect and 15 minutes to complete —
|
||||||
|
`/update` is synchronous and waits for the deploy to finish.
|
||||||
@@ -0,0 +1,80 @@
|
|||||||
|
name: "Deploy via package-updater"
|
||||||
|
description: "Notifies one or more package-updater instances to pull & restart a service"
|
||||||
|
inputs:
|
||||||
|
endpoint:
|
||||||
|
description: "Full URL to /update. Give several, one per line, to update a fleet."
|
||||||
|
required: true
|
||||||
|
image:
|
||||||
|
description: "Image reference without tag (e.g. registry.example.com/myapp)"
|
||||||
|
required: true
|
||||||
|
tag:
|
||||||
|
description: "Tag that was just pushed. Required in practice for swarm instances — it becomes the image the service is set to. Omit it and swarm deploys :latest."
|
||||||
|
required: false
|
||||||
|
default: ""
|
||||||
|
token:
|
||||||
|
description: "Bearer token for package-updater"
|
||||||
|
required: true
|
||||||
|
runs:
|
||||||
|
using: "composite"
|
||||||
|
steps:
|
||||||
|
- name: Trigger update
|
||||||
|
shell: bash
|
||||||
|
env:
|
||||||
|
ENDPOINTS: ${{ inputs.endpoint }}
|
||||||
|
IMAGE: ${{ inputs.image }}
|
||||||
|
TAG: ${{ inputs.tag }}
|
||||||
|
TOKEN: ${{ inputs.token }}
|
||||||
|
run: |
|
||||||
|
# No `set -e`: a failing endpoint must not abort the loop, or one dead
|
||||||
|
# server leaves the rest of the fleet un-updated and hides which hosts
|
||||||
|
# actually succeeded.
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
payload=$(jq -nc --arg image "$IMAGE" --arg tag "$TAG" \
|
||||||
|
'{image: $image, tag: $tag}') || { echo "jq is required but failed"; exit 1; }
|
||||||
|
|
||||||
|
attempted=0
|
||||||
|
failed=0
|
||||||
|
|
||||||
|
while IFS= read -r endpoint; do
|
||||||
|
# URLs never contain whitespace, so stripping all of it safely
|
||||||
|
# handles indentation, blank lines and CRLF line endings.
|
||||||
|
endpoint=$(printf '%s' "$endpoint" | tr -d '[:space:]')
|
||||||
|
[ -z "$endpoint" ] && continue
|
||||||
|
|
||||||
|
attempted=$((attempted + 1))
|
||||||
|
echo "--- $endpoint"
|
||||||
|
|
||||||
|
if ! response=$(curl -sS -w "\n%{http_code}" \
|
||||||
|
--connect-timeout 10 --max-time 900 \
|
||||||
|
-X POST "$endpoint" \
|
||||||
|
-H "Authorization: Bearer $TOKEN" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d "$payload"); then
|
||||||
|
echo "unreachable"
|
||||||
|
failed=$((failed + 1))
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Split on the last newline: curl's -w appended the status code
|
||||||
|
# there. Pure bash — `head -n -1` is GNU-only and fails on
|
||||||
|
# BSD/macOS, so the fan-out could not be tested locally.
|
||||||
|
code="${response##*$'\n'}"
|
||||||
|
body="${response%$'\n'*}"
|
||||||
|
echo "HTTP $code"
|
||||||
|
printf '%s' "$body" | jq . || printf '%s\n' "$body"
|
||||||
|
|
||||||
|
if [ "$code" -ge 400 ]; then
|
||||||
|
failed=$((failed + 1))
|
||||||
|
fi
|
||||||
|
done <<< "$ENDPOINTS"
|
||||||
|
|
||||||
|
if [ "$attempted" -eq 0 ]; then
|
||||||
|
echo "no endpoints given"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if [ "$failed" -gt 0 ]; then
|
||||||
|
echo "$failed of $attempted endpoint(s) failed"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "all $attempted endpoint(s) updated"
|
||||||
@@ -56,7 +56,12 @@ func (h *Handlers) Update(w http.ResponseWriter, r *http.Request) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
jobs, err := h.finder.FindJobs(r.Context(), req.Image)
|
requestedImage := req.Image
|
||||||
|
if req.Tag != "" {
|
||||||
|
requestedImage = req.Image + ":" + req.Tag
|
||||||
|
}
|
||||||
|
|
||||||
|
jobs, err := h.finder.FindJobs(r.Context(), requestedImage)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeJSONError(w, http.StatusInternalServerError, "discovery failed: "+err.Error())
|
writeJSONError(w, http.StatusInternalServerError, "discovery failed: "+err.Error())
|
||||||
return
|
return
|
||||||
@@ -86,6 +91,7 @@ func (h *Handlers) Update(w http.ResponseWriter, r *http.Request) {
|
|||||||
Project: res.Job.Project,
|
Project: res.Job.Project,
|
||||||
Service: res.Job.Service,
|
Service: res.Job.Service,
|
||||||
ComposeFile: composeFile,
|
ComposeFile: composeFile,
|
||||||
|
ServiceID: res.Job.ServiceID,
|
||||||
Status: string(res.Status),
|
Status: string(res.Status),
|
||||||
Error: res.Error,
|
Error: res.Error,
|
||||||
DurationMs: res.DurationMs,
|
DurationMs: res.DurationMs,
|
||||||
|
|||||||
@@ -21,9 +21,21 @@ import (
|
|||||||
type fakeFinder struct {
|
type fakeFinder struct {
|
||||||
jobs []discovery.Job
|
jobs []discovery.Job
|
||||||
err error
|
err error
|
||||||
|
|
||||||
|
// gotImage records the image argument passed to FindJobs, for
|
||||||
|
// assertions on the exact reference the handler built.
|
||||||
|
gotImage string
|
||||||
|
// swarmShaped, when true, makes FindJobs return a Job whose Image
|
||||||
|
// field carries the received image argument, simulating
|
||||||
|
// discovery.SwarmDiscovery.FindJobs.
|
||||||
|
swarmShaped bool
|
||||||
}
|
}
|
||||||
|
|
||||||
func (f *fakeFinder) FindJobs(_ context.Context, _ string) ([]discovery.Job, error) {
|
func (f *fakeFinder) FindJobs(_ context.Context, image string) ([]discovery.Job, error) {
|
||||||
|
f.gotImage = image
|
||||||
|
if f.swarmShaped {
|
||||||
|
return []discovery.Job{{Service: "myapp_web", ServiceID: "svc1", Image: image}}, f.err
|
||||||
|
}
|
||||||
return f.jobs, f.err
|
return f.jobs, f.err
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -157,6 +169,42 @@ func TestHealthz_503WhenDockerDown(t *testing.T) {
|
|||||||
require.Equal(t, http.StatusServiceUnavailable, w.Code)
|
require.Equal(t, http.StatusServiceUnavailable, w.Code)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestUpdate_FindJobsCalledWithTaggedImage(t *testing.T) {
|
||||||
|
finder := &fakeFinder{}
|
||||||
|
h := api.NewHandlers(finder, &fakeSubmitter{}, &fakePinger{}, "v0.0.0", "abc", "now", nil)
|
||||||
|
body, _ := json.Marshal(api.UpdateRequest{Image: "registry.example.com/myapp", Tag: "v2"})
|
||||||
|
req := httptest.NewRequest(http.MethodPost, "/update", bytes.NewReader(body))
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
h.Update(w, req)
|
||||||
|
require.Equal(t, "registry.example.com/myapp:v2", finder.gotImage)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUpdate_FindJobsCalledWithBareImageWhenNoTag(t *testing.T) {
|
||||||
|
finder := &fakeFinder{}
|
||||||
|
h := api.NewHandlers(finder, &fakeSubmitter{}, &fakePinger{}, "v0.0.0", "abc", "now", nil)
|
||||||
|
body, _ := json.Marshal(api.UpdateRequest{Image: "registry.example.com/myapp"})
|
||||||
|
req := httptest.NewRequest(http.MethodPost, "/update", bytes.NewReader(body))
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
h.Update(w, req)
|
||||||
|
require.Equal(t, "registry.example.com/myapp", finder.gotImage)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUpdate_SwarmModeThreadsTagThroughDiscovery(t *testing.T) {
|
||||||
|
finder := &fakeFinder{swarmShaped: true}
|
||||||
|
h := api.NewHandlers(finder, &fakeSubmitter{}, &fakePinger{}, "v0.0.0", "abc", "now", nil)
|
||||||
|
body, _ := json.Marshal(api.UpdateRequest{Image: "registry.example.com/myapp", Tag: "v2"})
|
||||||
|
req := httptest.NewRequest(http.MethodPost, "/update", bytes.NewReader(body))
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
h.Update(w, req)
|
||||||
|
require.Equal(t, http.StatusOK, w.Code)
|
||||||
|
// The fake Finder's returned Job.Image (as SwarmDiscovery would build
|
||||||
|
// it) reflects the image argument it received from the handler.
|
||||||
|
// Asserting on the captured argument proves the full request ->
|
||||||
|
// discovery flow carries the tag through to what would reach the
|
||||||
|
// SwarmExecutor.
|
||||||
|
require.Equal(t, "registry.example.com/myapp:v2", finder.gotImage)
|
||||||
|
}
|
||||||
|
|
||||||
func TestVersion(t *testing.T) {
|
func TestVersion(t *testing.T) {
|
||||||
h := api.NewHandlers(&fakeFinder{}, &fakeSubmitter{}, &fakePinger{}, "v1.2.3", "abcdef", "2026-05-22T00:00:00Z", nil)
|
h := api.NewHandlers(&fakeFinder{}, &fakeSubmitter{}, &fakePinger{}, "v1.2.3", "abcdef", "2026-05-22T00:00:00Z", nil)
|
||||||
req := httptest.NewRequest(http.MethodGet, "/version", nil)
|
req := httptest.NewRequest(http.MethodGet, "/version", nil)
|
||||||
|
|||||||
@@ -0,0 +1,48 @@
|
|||||||
|
package api_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/docker/docker/api/types"
|
||||||
|
"github.com/shcizo/package-updater/internal/api"
|
||||||
|
"github.com/shcizo/package-updater/internal/discovery"
|
||||||
|
"github.com/shcizo/package-updater/internal/updater"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
)
|
||||||
|
|
||||||
|
type swarmResultFinder struct{ jobs []discovery.Job }
|
||||||
|
|
||||||
|
func (f *swarmResultFinder) FindJobs(_ context.Context, _ string) ([]discovery.Job, error) {
|
||||||
|
return f.jobs, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type swarmResultSubmitter struct{ results []updater.Result }
|
||||||
|
|
||||||
|
func (s *swarmResultSubmitter) Submit(_ context.Context, _ []discovery.Job) []updater.Result {
|
||||||
|
return s.results
|
||||||
|
}
|
||||||
|
|
||||||
|
type swarmResultPinger struct{}
|
||||||
|
|
||||||
|
func (swarmResultPinger) Ping(_ context.Context) (types.Ping, error) { return types.Ping{}, nil }
|
||||||
|
|
||||||
|
func TestUpdate_SwarmJobPopulatesServiceID(t *testing.T) {
|
||||||
|
job := discovery.Job{Service: "myapp_web", ServiceID: "svc123", Image: "registry.example.com/myapp:v2"}
|
||||||
|
finder := &swarmResultFinder{jobs: []discovery.Job{job}}
|
||||||
|
submitter := &swarmResultSubmitter{results: []updater.Result{
|
||||||
|
{Job: job, Status: updater.StatusUpdated, DurationMs: 42},
|
||||||
|
}}
|
||||||
|
h := api.NewHandlers(finder, submitter, swarmResultPinger{}, "v", "c", "b", nil)
|
||||||
|
|
||||||
|
req := httptest.NewRequest(http.MethodPost, "/update", strings.NewReader(`{"image":"registry.example.com/myapp"}`))
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
h.Update(rec, req)
|
||||||
|
|
||||||
|
require.Equal(t, http.StatusOK, rec.Code)
|
||||||
|
require.Contains(t, rec.Body.String(), `"service_id":"svc123"`)
|
||||||
|
require.Contains(t, rec.Body.String(), `"service":"myapp_web"`)
|
||||||
|
}
|
||||||
@@ -20,6 +20,7 @@ type ResultDTO struct {
|
|||||||
Project string `json:"project"`
|
Project string `json:"project"`
|
||||||
Service string `json:"service"`
|
Service string `json:"service"`
|
||||||
ComposeFile string `json:"compose_file"`
|
ComposeFile string `json:"compose_file"`
|
||||||
|
ServiceID string `json:"service_id,omitempty"`
|
||||||
Status string `json:"status"`
|
Status string `json:"status"`
|
||||||
Error string `json:"error,omitempty"`
|
Error string `json:"error,omitempty"`
|
||||||
DurationMs int64 `json:"duration_ms"`
|
DurationMs int64 `json:"duration_ms"`
|
||||||
|
|||||||
@@ -16,6 +16,7 @@ type Config struct {
|
|||||||
LogLevel string
|
LogLevel string
|
||||||
UpdateTimeout time.Duration
|
UpdateTimeout time.Duration
|
||||||
OptInLabel string
|
OptInLabel string
|
||||||
|
Mode string
|
||||||
}
|
}
|
||||||
|
|
||||||
// Load reads configuration from environment variables, applies defaults,
|
// Load reads configuration from environment variables, applies defaults,
|
||||||
@@ -28,12 +29,17 @@ func Load() (*Config, error) {
|
|||||||
Port: getenvDefault("PORT", "8080"),
|
Port: getenvDefault("PORT", "8080"),
|
||||||
LogLevel: getenvDefault("LOG_LEVEL", "info"),
|
LogLevel: getenvDefault("LOG_LEVEL", "info"),
|
||||||
OptInLabel: getenvDefault("OPT_IN_LABEL", "se.shcizo.auto-update"),
|
OptInLabel: getenvDefault("OPT_IN_LABEL", "se.shcizo.auto-update"),
|
||||||
|
Mode: getenvDefault("MODE", "compose"),
|
||||||
}
|
}
|
||||||
|
|
||||||
if cfg.APIKey == "" {
|
if cfg.APIKey == "" {
|
||||||
return nil, errors.New("UPDATER_API_KEY is required")
|
return nil, errors.New("UPDATER_API_KEY is required")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if cfg.Mode != "compose" && cfg.Mode != "swarm" {
|
||||||
|
return nil, fmt.Errorf("MODE %q is invalid: must be %q or %q", cfg.Mode, "compose", "swarm")
|
||||||
|
}
|
||||||
|
|
||||||
timeoutStr := getenvDefault("UPDATE_TIMEOUT", "5m")
|
timeoutStr := getenvDefault("UPDATE_TIMEOUT", "5m")
|
||||||
d, err := time.ParseDuration(timeoutStr)
|
d, err := time.ParseDuration(timeoutStr)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -57,3 +57,26 @@ func TestLoad_InvalidTimeoutErrors(t *testing.T) {
|
|||||||
require.Error(t, err)
|
require.Error(t, err)
|
||||||
require.Contains(t, err.Error(), "UPDATE_TIMEOUT")
|
require.Contains(t, err.Error(), "UPDATE_TIMEOUT")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestLoad_DefaultsToComposeMode(t *testing.T) {
|
||||||
|
t.Setenv("UPDATER_API_KEY", "secret")
|
||||||
|
cfg, err := config.Load()
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Equal(t, "compose", cfg.Mode)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoad_AcceptsSwarmMode(t *testing.T) {
|
||||||
|
t.Setenv("UPDATER_API_KEY", "secret")
|
||||||
|
t.Setenv("MODE", "swarm")
|
||||||
|
cfg, err := config.Load()
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Equal(t, "swarm", cfg.Mode)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoad_RejectsInvalidMode(t *testing.T) {
|
||||||
|
t.Setenv("UPDATER_API_KEY", "secret")
|
||||||
|
t.Setenv("MODE", "kubernetes")
|
||||||
|
_, err := config.Load()
|
||||||
|
require.Error(t, err)
|
||||||
|
require.Contains(t, err.Error(), "MODE")
|
||||||
|
}
|
||||||
|
|||||||
@@ -15,6 +15,12 @@ type Job struct {
|
|||||||
Service string
|
Service string
|
||||||
WorkingDir string
|
WorkingDir string
|
||||||
ConfigFiles []string
|
ConfigFiles []string
|
||||||
|
// Image is the full image reference the update request asked for.
|
||||||
|
// Compose jobs ignore it (compose.yml already pins the reference to
|
||||||
|
// pull); Swarm jobs need it to know what to set on the service spec.
|
||||||
|
Image string
|
||||||
|
// ServiceID is the Swarm service ID. Empty for Compose jobs.
|
||||||
|
ServiceID string
|
||||||
// Refused is true when the WorkingDir falls outside STACKS_ROOT.
|
// Refused is true when the WorkingDir falls outside STACKS_ROOT.
|
||||||
// The job is returned so the caller can surface a per-job "refused"
|
// The job is returned so the caller can surface a per-job "refused"
|
||||||
// result, but it MUST NOT be executed.
|
// result, but it MUST NOT be executed.
|
||||||
@@ -74,6 +80,7 @@ func (d *Discovery) FindJobs(ctx context.Context, image string) ([]Job, error) {
|
|||||||
Service: cl.Service,
|
Service: cl.Service,
|
||||||
WorkingDir: cl.WorkingDir,
|
WorkingDir: cl.WorkingDir,
|
||||||
ConfigFiles: cl.ConfigFiles,
|
ConfigFiles: cl.ConfigFiles,
|
||||||
|
Image: image,
|
||||||
Refused: refused,
|
Refused: refused,
|
||||||
RefusedReason: reason,
|
RefusedReason: reason,
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -153,3 +153,20 @@ func TestFindJobs_NonComposeContainerIsSkipped(t *testing.T) {
|
|||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
require.Empty(t, jobs)
|
require.Empty(t, jobs)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestFindJobs_PopulatesImageField(t *testing.T) {
|
||||||
|
fake := &fakeDockerClient{containers: []types.Container{
|
||||||
|
mkContainer("registry.example.com/myapp:v1", mkComposeLabels(
|
||||||
|
"myapp-prod", "web",
|
||||||
|
"/home/shcizo/self-hosted/myapp-prod",
|
||||||
|
"/home/shcizo/self-hosted/myapp-prod/docker-compose.yml",
|
||||||
|
true,
|
||||||
|
)),
|
||||||
|
}}
|
||||||
|
d := discovery.New(fake, "/home/shcizo/self-hosted", "se.shcizo.auto-update")
|
||||||
|
jobs, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, jobs, 1)
|
||||||
|
require.Equal(t, "registry.example.com/myapp", jobs[0].Image)
|
||||||
|
require.Empty(t, jobs[0].ServiceID)
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,60 @@
|
|||||||
|
package discovery
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
"github.com/docker/docker/api/types/swarm"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SwarmDockerClient is the subset of the Docker SDK SwarmDiscovery depends
|
||||||
|
// on. Defined as an interface so tests can supply a fake.
|
||||||
|
type SwarmDockerClient interface {
|
||||||
|
ServiceList(ctx context.Context, opts swarm.ServiceListOptions) ([]swarm.Service, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// SwarmDiscovery orchestrates "given an image, which Swarm services should
|
||||||
|
// we update?". Unlike Discovery, it has no filesystem path to check — the
|
||||||
|
// opt-in label on the service itself is the only gate, since Swarm services
|
||||||
|
// have no local compose file to anchor a STACKS_ROOT check against.
|
||||||
|
type SwarmDiscovery struct {
|
||||||
|
cli SwarmDockerClient
|
||||||
|
optInLabel string
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSwarm returns a SwarmDiscovery bound to the given Docker client and
|
||||||
|
// opt-in label.
|
||||||
|
func NewSwarm(cli SwarmDockerClient, optInLabel string) *SwarmDiscovery {
|
||||||
|
return &SwarmDiscovery{cli: cli, optInLabel: optInLabel}
|
||||||
|
}
|
||||||
|
|
||||||
|
// FindJobs lists Swarm services, filters by image match + opt-in label, and
|
||||||
|
// returns one Job per matching service. Signature matches Discovery.FindJobs
|
||||||
|
// so both satisfy api.Finder.
|
||||||
|
func (d *SwarmDiscovery) FindJobs(ctx context.Context, image string) ([]Job, error) {
|
||||||
|
services, err := d.cli.ServiceList(ctx, swarm.ServiceListOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("docker service list: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var jobs []Job
|
||||||
|
for _, svc := range services {
|
||||||
|
if svc.Spec.TaskTemplate.ContainerSpec == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !ImagesMatch(image, svc.Spec.TaskTemplate.ContainerSpec.Image) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !HasOptIn(svc.Spec.Labels, d.optInLabel) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
jobs = append(jobs, Job{
|
||||||
|
Service: svc.Spec.Name,
|
||||||
|
ServiceID: svc.ID,
|
||||||
|
Image: image,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
return jobs, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,86 @@
|
|||||||
|
package discovery_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/docker/docker/api/types/swarm"
|
||||||
|
"github.com/shcizo/package-updater/internal/discovery"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
)
|
||||||
|
|
||||||
|
type fakeSwarmDockerClient struct {
|
||||||
|
services []swarm.Service
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSwarmDockerClient) ServiceList(_ context.Context, _ swarm.ServiceListOptions) ([]swarm.Service, error) {
|
||||||
|
return f.services, f.err
|
||||||
|
}
|
||||||
|
|
||||||
|
func mkService(id, name, image string, labels map[string]string) swarm.Service {
|
||||||
|
return swarm.Service{
|
||||||
|
ID: id,
|
||||||
|
Spec: swarm.ServiceSpec{
|
||||||
|
Annotations: swarm.Annotations{Name: name, Labels: labels},
|
||||||
|
TaskTemplate: swarm.TaskSpec{
|
||||||
|
ContainerSpec: &swarm.ContainerSpec{Image: image},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmFindJobs_MatchAndOptIn(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{services: []swarm.Service{
|
||||||
|
mkService("svc-myapp", "myapp_web", "registry.example.com/myapp:v1", map[string]string{
|
||||||
|
"se.shcizo.auto-update": "true",
|
||||||
|
}),
|
||||||
|
mkService("svc-other", "other_web", "registry.example.com/other:v1", map[string]string{
|
||||||
|
"se.shcizo.auto-update": "true",
|
||||||
|
}),
|
||||||
|
}}
|
||||||
|
d := discovery.NewSwarm(fake, "se.shcizo.auto-update")
|
||||||
|
jobs, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, jobs, 1)
|
||||||
|
require.Equal(t, "myapp_web", jobs[0].Service)
|
||||||
|
require.Equal(t, "svc-myapp", jobs[0].ServiceID)
|
||||||
|
require.Equal(t, "registry.example.com/myapp", jobs[0].Image)
|
||||||
|
require.False(t, jobs[0].Refused)
|
||||||
|
require.Empty(t, jobs[0].WorkingDir)
|
||||||
|
require.Empty(t, jobs[0].ConfigFiles)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmFindJobs_SkipsWithoutOptIn(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{services: []swarm.Service{
|
||||||
|
mkService("svc-myapp", "myapp_web", "registry.example.com/myapp:v1", nil),
|
||||||
|
}}
|
||||||
|
d := discovery.NewSwarm(fake, "se.shcizo.auto-update")
|
||||||
|
jobs, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Empty(t, jobs)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmFindJobs_DockerError(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{err: errors.New("connection refused")}
|
||||||
|
d := discovery.NewSwarm(fake, "se.shcizo.auto-update")
|
||||||
|
_, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
|
||||||
|
require.Error(t, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmFindJobs_NoContainerSpecIsSkipped(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{services: []swarm.Service{
|
||||||
|
{
|
||||||
|
ID: "svc-weird",
|
||||||
|
Spec: swarm.ServiceSpec{
|
||||||
|
Annotations: swarm.Annotations{Name: "weird", Labels: map[string]string{"se.shcizo.auto-update": "true"}},
|
||||||
|
TaskTemplate: swarm.TaskSpec{ContainerSpec: nil},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
d := discovery.NewSwarm(fake, "se.shcizo.auto-update")
|
||||||
|
jobs, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Empty(t, jobs)
|
||||||
|
}
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
package updater
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
"github.com/docker/docker/api/types/swarm"
|
||||||
|
"github.com/shcizo/package-updater/internal/discovery"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SwarmDockerClient is the subset of the Docker SDK SwarmExecutor depends
|
||||||
|
// on. Defined as an interface so tests can supply a fake.
|
||||||
|
type SwarmDockerClient interface {
|
||||||
|
ServiceInspectWithRaw(ctx context.Context, serviceID string, opts swarm.ServiceInspectOptions) (swarm.Service, []byte, error)
|
||||||
|
ServiceUpdate(ctx context.Context, serviceID string, version swarm.Version, service swarm.ServiceSpec, opts swarm.ServiceUpdateOptions) (swarm.ServiceUpdateResponse, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// SwarmExecutor updates a Swarm service's image via the Docker API,
|
||||||
|
// equivalent to `docker service update --image`.
|
||||||
|
type SwarmExecutor struct {
|
||||||
|
cli SwarmDockerClient
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSwarmExecutor returns an executor that drives Swarm service updates
|
||||||
|
// through the Docker API.
|
||||||
|
func NewSwarmExecutor(cli SwarmDockerClient) *SwarmExecutor {
|
||||||
|
return &SwarmExecutor{cli: cli}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Execute inspects the service to get its current spec and version (required
|
||||||
|
// by the Docker API as an optimistic-concurrency token), sets the new image
|
||||||
|
// on the container spec, and calls ServiceUpdate. QueryRegistry is set so a
|
||||||
|
// floating tag (e.g. ":latest") resolves to a fresh digest and actually
|
||||||
|
// triggers a rolling update instead of being treated as unchanged.
|
||||||
|
func (e *SwarmExecutor) Execute(ctx context.Context, job discovery.Job) error {
|
||||||
|
if job.Refused {
|
||||||
|
return fmt.Errorf("refused: %s", job.RefusedReason)
|
||||||
|
}
|
||||||
|
|
||||||
|
svc, _, err := e.cli.ServiceInspectWithRaw(ctx, job.ServiceID, swarm.ServiceInspectOptions{})
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("inspect service %s: %w", job.ServiceID, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
spec := svc.Spec
|
||||||
|
spec.TaskTemplate.ContainerSpec.Image = job.Image
|
||||||
|
|
||||||
|
if _, err := e.cli.ServiceUpdate(ctx, job.ServiceID, svc.Version, spec, swarm.ServiceUpdateOptions{QueryRegistry: true}); err != nil {
|
||||||
|
return fmt.Errorf("update service %s: %w", job.ServiceID, err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
package updater_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/docker/docker/api/types/swarm"
|
||||||
|
"github.com/shcizo/package-updater/internal/discovery"
|
||||||
|
"github.com/shcizo/package-updater/internal/updater"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
)
|
||||||
|
|
||||||
|
type fakeSwarmDockerClient struct {
|
||||||
|
inspectService swarm.Service
|
||||||
|
inspectErr error
|
||||||
|
updateErr error
|
||||||
|
|
||||||
|
inspectCalled bool
|
||||||
|
updateCalled bool
|
||||||
|
updateCalledServiceID string
|
||||||
|
updateCalledVersion swarm.Version
|
||||||
|
updateCalledSpec swarm.ServiceSpec
|
||||||
|
updateCalledOpts swarm.ServiceUpdateOptions
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSwarmDockerClient) ServiceInspectWithRaw(_ context.Context, _ string, _ swarm.ServiceInspectOptions) (swarm.Service, []byte, error) {
|
||||||
|
f.inspectCalled = true
|
||||||
|
return f.inspectService, nil, f.inspectErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSwarmDockerClient) ServiceUpdate(_ context.Context, serviceID string, version swarm.Version, spec swarm.ServiceSpec, opts swarm.ServiceUpdateOptions) (swarm.ServiceUpdateResponse, error) {
|
||||||
|
f.updateCalled = true
|
||||||
|
f.updateCalledServiceID = serviceID
|
||||||
|
f.updateCalledVersion = version
|
||||||
|
f.updateCalledSpec = spec
|
||||||
|
f.updateCalledOpts = opts
|
||||||
|
return swarm.ServiceUpdateResponse{}, f.updateErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func mkInspectService(version uint64, image string) swarm.Service {
|
||||||
|
return swarm.Service{
|
||||||
|
ID: "svc-myapp",
|
||||||
|
Meta: swarm.Meta{Version: swarm.Version{Index: version}},
|
||||||
|
Spec: swarm.ServiceSpec{
|
||||||
|
Annotations: swarm.Annotations{Name: "myapp_web"},
|
||||||
|
TaskTemplate: swarm.TaskSpec{ContainerSpec: &swarm.ContainerSpec{Image: image}},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmExecutor_UpdatesImageAndVersion(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{inspectService: mkInspectService(7, "registry.example.com/myapp:v1")}
|
||||||
|
e := updater.NewSwarmExecutor(fake)
|
||||||
|
|
||||||
|
job := discovery.Job{Service: "myapp_web", ServiceID: "svc-myapp", Image: "registry.example.com/myapp:v2"}
|
||||||
|
err := e.Execute(context.Background(), job)
|
||||||
|
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.True(t, fake.updateCalled)
|
||||||
|
require.Equal(t, "svc-myapp", fake.updateCalledServiceID)
|
||||||
|
require.Equal(t, uint64(7), fake.updateCalledVersion.Index)
|
||||||
|
require.Equal(t, "registry.example.com/myapp:v2", fake.updateCalledSpec.TaskTemplate.ContainerSpec.Image)
|
||||||
|
require.True(t, fake.updateCalledOpts.QueryRegistry)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmExecutor_InspectError(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{inspectErr: errors.New("service not found")}
|
||||||
|
e := updater.NewSwarmExecutor(fake)
|
||||||
|
|
||||||
|
err := e.Execute(context.Background(), discovery.Job{ServiceID: "svc-missing", Image: "x:v2"})
|
||||||
|
|
||||||
|
require.Error(t, err)
|
||||||
|
require.False(t, fake.updateCalled)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmExecutor_UpdateError(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{
|
||||||
|
inspectService: mkInspectService(1, "registry.example.com/myapp:v1"),
|
||||||
|
updateErr: errors.New("update rejected"),
|
||||||
|
}
|
||||||
|
e := updater.NewSwarmExecutor(fake)
|
||||||
|
|
||||||
|
err := e.Execute(context.Background(), discovery.Job{ServiceID: "svc-myapp", Image: "registry.example.com/myapp:v2"})
|
||||||
|
|
||||||
|
require.Error(t, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSwarmExecutor_RefusedJobIsNotExecuted(t *testing.T) {
|
||||||
|
fake := &fakeSwarmDockerClient{}
|
||||||
|
e := updater.NewSwarmExecutor(fake)
|
||||||
|
job := discovery.Job{ServiceID: "svc-myapp", Image: "x:v2", Refused: true, RefusedReason: "not opted in"}
|
||||||
|
|
||||||
|
err := e.Execute(context.Background(), job)
|
||||||
|
|
||||||
|
require.Error(t, err)
|
||||||
|
require.False(t, fake.inspectCalled)
|
||||||
|
require.False(t, fake.updateCalled)
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user