# Multi-Server Deployment & CI Fan-Out Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Land the finished Swarm support on `main`, document the one-instance-per-server deployment model, and make the Gitea composite action post `/update` to every server in a fleet instead of a single endpoint. **Architecture:** No Go code changes. Part 1 is a merge plus documentation reframing — `MODE=compose|swarm` stays exclusive per instance, because each instance owns exactly one Docker daemon. Part 2 changes `gitea-action/action.yml` so its `endpoint` input accepts a newline-separated list, looping over every URL and continuing past failures so one dead host cannot stop the rest of the fleet from updating. **Tech Stack:** Go 1.26.3 (unchanged, no code edits), Bash inside a Gitea composite action, `curl`, `jq`. ## Global Constraints - **No Go source changes in this plan.** If a task seems to need one, stop and re-read the spec — the design explicitly rejects composite Finder/Executor work. - `MODE` stays `compose` or `swarm` only. Do not add a `both` value. - Do not weaken Compose mode's three-factor gate (token AND opt-in label AND `STACKS_ROOT` prefix). - Existing single-URL usages of the action must keep working unchanged — a single URL is a one-element list. Do not add a separate input for the list form. - Separator for the endpoint list is **newline**, never comma. URLs may legally contain commas. - Conventional Commits (`feat:`, `fix:`, `docs:`, `ci:`, `build:`, `chore:`). - Full test suite (`go test ./...`) must be green at the end of every task that touches the repo. --- ### Task 1: Merge Swarm support to `main` **Files:** - No files edited. This task merges branch `worktree-docker-swarm-support` (10 commits) into `main`. **Interfaces:** - Consumes: nothing. - Produces: `main` gains `internal/discovery/swarm.go`, `internal/updater/swarm_executor.go`, `config.Config.Mode`, `discovery.Job.Image`, `discovery.Job.ServiceID`, and `api.ResultDTO.ServiceID`. Task 2 edits documentation files that this merge brings in (`README.md` "Swarm mode" section, `CLAUDE.md` swarm entries). - [ ] **Step 1: Confirm the branch is where you think it is** Run: ```bash cd /Users/samuelenocsson/dev/package-updater git log --oneline main..worktree-docker-swarm-support ``` Expected: exactly 10 commits, oldest `8f999c6 docs: add docker swarm support implementation plan`, newest `db84a2f fix(api): plumb request tag into discovery so swarm mode deploys the requested version`. If the list differs, stop and report — someone has moved the branch since this plan was written. - [ ] **Step 2: Verify the branch is green before merging** Run: ```bash cd /Users/samuelenocsson/dev/package-updater/.claude/worktrees/docker-swarm-support go build ./... && go test ./... ``` Expected: build succeeds, all packages PASS. Do not proceed on a red branch — fix or report first. - [ ] **Step 3: Merge into `main`** ```bash cd /Users/samuelenocsson/dev/package-updater git checkout main git merge --no-ff worktree-docker-swarm-support -m "$(cat <<'EOF' Merge branch 'worktree-docker-swarm-support' Adds MODE=compose|swarm. Each instance owns one Docker daemon and runs one paradigm; see docs/superpowers/specs/2026-08-04-multi-server-fanout-design.md for why this exclusivity is the intended design. EOF )" ``` `--no-ff` is deliberate: it keeps the ten swarm commits grouped under one merge commit, matching how PRs #1 and #2 already appear in this repo's history. Note: `docs/superpowers/plans/2026-07-04-docker-swarm-support.md` currently exists as an *untracked* file on `main` and is *committed* on the swarm branch. The merge brings in the tracked copy. If git refuses the merge with "untracked working tree file would be overwritten", delete the untracked copy first (`rm docs/superpowers/plans/2026-07-04-docker-swarm-support.md`) — the branch's committed version is identical and authoritative. - [ ] **Step 4: Verify `main` is green after the merge** Run: ```bash go build ./... && go test ./... ``` Expected: build succeeds, all packages PASS. This is the first time the swarm code and the post-swarm `main` commits (`CLAUDE.md`, CI workflow) are compiled together on `main`. - [ ] **Step 5: Push** ```bash git push origin main ``` --- ### Task 2: Document the deployment topology **Files:** - Modify: `README.md` (the version merged in Task 1 — it already contains a "Swarm mode" section at lines ~30-47, a `MODE` row in the config table at line ~71, and a "Known v1 gaps" bullet about single-host at lines ~103-105) - Modify: `CLAUDE.md` (the "Design intent" section, which after Task 1 contains a "Compose mode and Swarm mode are selected once per deployment via `MODE`" entry) **Interfaces:** - Consumes: the merged documentation from Task 1. - Produces: nothing consumed by later tasks. Task 3 documents the action separately in `gitea-action/README.md`. There is no test for documentation. Verification is reading it back and confirming the claims match the code in `cmd/server/main.go` and `internal/config/config.go`. - [ ] **Step 1: Rebase the working branch onto the updated `main`** ```bash cd /Users/samuelenocsson/dev/package-updater git checkout feat/multi-server-fanout git rebase main ``` Expected: clean rebase. The branch currently holds one commit (`docs: design for multi-server deployment and CI fan-out`) touching only a new spec file, so there is nothing to conflict with. - [ ] **Step 2: Add a deployment-topology section to `README.md`** Insert this section immediately **after** the "Swarm mode" section and **before** "## Quick start": ```markdown ## Deploying across multiple servers An instance can only reach the Docker daemon it is configured against. It cannot update stacks on other machines. The deployment model follows from that: **One instance per server.** Each server runs its own package-updater against its own local Docker socket, with `MODE` set to whatever that machine runs: | Server | `MODE` | Updates | |---|---|---| | Swarm manager node | `swarm` | All Swarm services in the cluster | | Standalone Compose host | `compose` | Compose stacks on that host | **CI fans out.** The [Gitea composite action](gitea-action/README.md) takes a list of endpoints and posts `/update` to every instance, so one workflow run reaches the whole fleet. Each instance answers for its own machine; there is no aggregated cross-server response and no instance coordinates any other. This is why `MODE` is exclusive rather than a mode that handles both at once: an instance that could do both would still only reach one daemon, so the extra generality buys nothing. ``` - [ ] **Step 3: Reword the `MODE` row in the configuration table** In `README.md`'s configuration table, replace this row: ```markdown | `MODE` | no | `compose` | `compose` or `swarm`. Selects the update mechanism for the whole deployment; not mixed per-request. | ``` with: ```markdown | `MODE` | no | `compose` | `compose` or `swarm`. Which paradigm *this instance's* Docker daemon runs. See [Deploying across multiple servers](#deploying-across-multiple-servers). | ``` - [ ] **Step 4: Replace the "single host only" gap bullet** In `README.md`'s "Known v1 gaps" section, replace this bullet: ```markdown - **Single host only in Compose mode**. Swarm mode (`MODE=swarm`) is the multi-node path, but only from a manager node's point of view — the updater itself still needs manager API access (see "Swarm mode" above). ``` with: ```markdown - **One instance reaches one daemon.** An instance never updates another server; fleets run one instance per server with CI fanning out to all of them (see "Deploying across multiple servers"). Swarm mode is the exception in that a single manager-node instance covers the whole cluster. ``` - [ ] **Step 5: Reword the design-intent entry in `CLAUDE.md`** In `CLAUDE.md`'s "Design intent (do not break without discussion)" section, replace this entry: ```markdown - **Compose mode and Swarm mode are selected once per deployment via `MODE`**, never mixed at request time. Swarm mode's security gate is opt-in label only — there is no STACKS_ROOT-equivalent path check, since Swarm services have no local compose file. Don't add one; don't weaken Compose mode's three-factor gate to match. ``` with: ```markdown - **`MODE` is exclusive because each instance owns exactly one Docker daemon.** A composite "handle both at once" mode has been considered and rejected: it would still only reach one daemon, so it buys nothing. Fleets run one instance per server and CI fans out. Reopen this only if a Swarm manager node starts running standalone Compose stacks locally. Rationale and rejected alternatives: `docs/superpowers/specs/2026-08-04-multi-server-fanout-design.md`. - **Swarm mode's security gate is opt-in label only** — there is no STACKS_ROOT-equivalent path check, since Swarm services have no local compose file. Don't add one; don't weaken Compose mode's three-factor gate to match. ``` Splitting the original entry in two is deliberate: the mode-exclusivity rationale and the Swarm security-gate rule are separate rules that were sharing a bullet, and only the first one is changing. - [ ] **Step 6: Verify the anchor link resolves** The `MODE` table row links to `#deploying-across-multiple-servers`. Confirm the heading added in Step 2 is exactly `## Deploying across multiple servers` — GitHub/Gitea derive the anchor by lowercasing and replacing spaces with hyphens, so any wording drift silently breaks the link. Run: `grep -n "^## Deploying across multiple servers" README.md` Expected: one match. - [ ] **Step 7: Commit** ```bash git add README.md CLAUDE.md git commit -m "docs: describe one-instance-per-server topology and why MODE is exclusive" ``` --- ### Task 3: Fan out to multiple endpoints in the Gitea action **Files:** - Modify: `gitea-action/action.yml` (whole `runs.steps` block, lines 17-37, and the `endpoint` input description at lines 5-6) - Modify: `gitea-action/README.md` (usage example, inputs table, failure-modes section) **Interfaces:** - Consumes: nothing from earlier tasks. - Produces: the action's `endpoint` input accepts one URL (unchanged behaviour) or several separated by newlines. No other input changes name, type, or meaning. - [ ] **Step 1: Rewrite `gitea-action/action.yml`** Replace the entire file with: ```yaml name: "Deploy via package-updater" description: "Notifies one or more package-updater instances to pull & restart a service" inputs: endpoint: description: "Full URL to /update. Give several, one per line, to update a fleet." required: true image: description: "Image reference without tag (e.g. registry.example.com/myapp)" required: true tag: description: "Tag that was just pushed (for logging)" required: false default: "" token: description: "Bearer token for package-updater" required: true runs: using: "composite" steps: - name: Trigger update shell: bash env: ENDPOINTS: ${{ inputs.endpoint }} IMAGE: ${{ inputs.image }} TAG: ${{ inputs.tag }} TOKEN: ${{ inputs.token }} run: | # No `set -e`: a failing endpoint must not abort the loop, or one dead # server leaves the rest of the fleet un-updated and hides which hosts # actually succeeded. set -uo pipefail payload=$(jq -nc --arg image "$IMAGE" --arg tag "$TAG" \ '{image: $image, tag: $tag}') attempted=0 failed=0 while IFS= read -r endpoint; do # URLs never contain whitespace, so stripping all of it safely # handles indentation, blank lines and CRLF line endings. endpoint=$(printf '%s' "$endpoint" | tr -d '[:space:]') [ -z "$endpoint" ] && continue attempted=$((attempted + 1)) echo "--- $endpoint" if ! response=$(curl -sS -w "\n%{http_code}" \ -X POST "$endpoint" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d "$payload"); then echo "unreachable" failed=$((failed + 1)) continue fi # Split on the last newline: curl's -w appended the status code # there. Pure bash — `head -n -1` is GNU-only and fails on # BSD/macOS, so the fan-out could not be tested locally. code="${response##*$'\n'}" body="${response%$'\n'*}" echo "HTTP $code" printf '%s' "$body" | jq . || printf '%s\n' "$body" if [ "$code" -ge 400 ]; then failed=$((failed + 1)) fi done <<< "$ENDPOINTS" if [ "$attempted" -eq 0 ]; then echo "no endpoints given" exit 1 fi if [ "$failed" -gt 0 ]; then echo "$failed of $attempted endpoint(s) failed" exit 1 fi echo "all $attempted endpoint(s) updated" ``` Four changes beyond the loop itself, each load-bearing: 1. **All inputs moved into `env:`.** A multi-line `${{ inputs.endpoint }}` interpolated directly into the script body would break it syntactically. Moving the others too keeps one consistent style in a short script, and stops a value containing shell metacharacters from being executed. 2. **`jq -nc` builds the payload** instead of hand-interpolating into a JSON string. A tag containing a quote previously produced malformed JSON and a confusing 400. 3. **`set -e` dropped**, `-uo pipefail` kept. This is the whole point of the task — see the comment in the script. 4. **`head -n -1` replaced by bash parameter expansion.** The existing action used `head -n -1`, which is GNU-only — it fails on BSD/macOS with `illegal line count`. That made the fan-out logic impossible to test on a developer machine, so the portability fix is what makes Steps 2-4 runnable at all. It also drops two subprocesses per endpoint. - [ ] **Step 2: Verify the loop continues past a failure** This is the behaviour that justifies the task, so test it directly rather than trusting the code by inspection. Start a server that answers POST with 200: ```bash python3 -c " import http.server class H(http.server.BaseHTTPRequestHandler): def do_POST(self): self.send_response(200) self.send_header('Content-Type','application/json') self.end_headers() self.wfile.write(b'{\"matched\":1,\"results\":[]}') def log_message(self, *a): pass http.server.HTTPServer(('127.0.0.1',8099), H).serve_forever() " & echo $! > /tmp/fanout-test-server.pid ``` Write the script body to a runnable file. This must be the **exact** text of the `run:` block from Step 1, dedented — copy it, do not retype it, or you are testing something other than what ships: ```bash cat > /tmp/fanout-test.sh <<'SCRIPT' set -uo pipefail payload=$(jq -nc --arg image "$IMAGE" --arg tag "$TAG" \ '{image: $image, tag: $tag}') attempted=0 failed=0 while IFS= read -r endpoint; do endpoint=$(printf '%s' "$endpoint" | tr -d '[:space:]') [ -z "$endpoint" ] && continue attempted=$((attempted + 1)) echo "--- $endpoint" if ! response=$(curl -sS -w "\n%{http_code}" \ -X POST "$endpoint" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d "$payload"); then echo "unreachable" failed=$((failed + 1)) continue fi code="${response##*$'\n'}" body="${response%$'\n'*}" echo "HTTP $code" printf '%s' "$body" | jq . || printf '%s\n' "$body" if [ "$code" -ge 400 ]; then failed=$((failed + 1)) fi done <<< "$ENDPOINTS" if [ "$attempted" -eq 0 ]; then echo "no endpoints given" exit 1 fi if [ "$failed" -gt 0 ]; then echo "$failed of $attempted endpoint(s) failed" exit 1 fi echo "all $attempted endpoint(s) updated" SCRIPT ``` Now execute it with a list where the middle endpoint is dead (port 1 always refuses connections): ```bash ENDPOINTS="http://127.0.0.1:8099/update http://127.0.0.1:1/update http://127.0.0.1:8099/update" \ IMAGE="registry.example.com/myapp" \ TAG="abc123" \ TOKEN="dummy" \ bash /tmp/fanout-test.sh echo "exit code: $?" ``` Expected output shape: ``` --- http://127.0.0.1:8099/update HTTP 200 { "matched": 1, "results": [] } --- http://127.0.0.1:1/update unreachable --- http://127.0.0.1:8099/update HTTP 200 { "matched": 1, "results": [] } 1 of 3 endpoint(s) failed exit code: 1 ``` Three things must all hold: the **third** endpoint was attempted (proves the loop continued), the exit code is **1**, and each endpoint's outcome is visible. If the third `---` line is missing, `set -e` crept back in. - [ ] **Step 3: Verify a single endpoint still works unchanged** ```bash ENDPOINTS="http://127.0.0.1:8099/update" \ IMAGE="registry.example.com/myapp" TAG="abc123" TOKEN="dummy" \ bash /tmp/fanout-test.sh echo "exit code: $?" ``` Expected: one `--- ` line, `HTTP 200`, `all 1 endpoint(s) updated`, exit code 0. This is the existing-usage regression check. - [ ] **Step 4: Verify blank lines and indentation are tolerated** YAML block scalars routinely produce leading indentation and a trailing newline, so this is the realistic input shape, not an edge case. ```bash ENDPOINTS=" http://127.0.0.1:8099/update http://127.0.0.1:8099/update " \ IMAGE="registry.example.com/myapp" TAG="abc123" TOKEN="dummy" \ bash /tmp/fanout-test.sh echo "exit code: $?" ``` Expected: exactly two `--- http://127.0.0.1:8099/update` lines with no stray whitespace in the URL, `all 2 endpoint(s) updated`, exit code 0. - [ ] **Step 5: Clean up the test server** ```bash kill "$(cat /tmp/fanout-test-server.pid)" && rm -f /tmp/fanout-test-server.pid /tmp/fanout-test.sh ``` - [ ] **Step 6: Update `gitea-action/README.md`** Replace the usage example's `with:` block so it shows the fleet form: ```yaml - uses: gitea.example.com/shcizo/package-updater/gitea-action@v1 with: endpoint: | https://updater-swarm.example.com/update https://updater-web01.example.com/update https://updater-web02.example.com/update image: registry.example.com/${{ gitea.repository }} tag: ${{ gitea.sha }} token: ${{ secrets.UPDATER_TOKEN }} ``` Change the `endpoint` row in the inputs table to: ```markdown | `endpoint` | yes | — | Full URL to `/update`. Several may be given, one per line, to update a fleet. | ``` Replace the "Failure modes" section with: ```markdown ## Failure modes Every endpoint is attempted, even when an earlier one fails — otherwise one dead server would leave the rest of the fleet un-updated, and the CI log would not show which hosts actually succeeded. The step exits non-zero if any endpoint returned 4xx/5xx or was unreachable. The log lists each endpoint with its HTTP status and response body, so a partial deploy is visible at a glance. ``` Add this note after the `UPDATER_TOKEN` sentence: ```markdown All instances in the fleet must share the same bearer token, since one `token` input is sent to every endpoint. ``` - [ ] **Step 7: Commit** ```bash git add gitea-action/action.yml gitea-action/README.md git commit -m "feat(action): post /update to every endpoint in a fleet, continuing past failures" ``` - [ ] **Step 8: Push and open a PR** ```bash git push -u origin feat/multi-server-fanout ``` Then open a PR against `main` (see `issue-tracker-cli` skill for whether this repo uses `gh` or `tea`). The PR covers the spec, the documentation reframing, and the action fan-out; the Swarm merge landed separately in Task 1.