shcizo/package-updater
1
0
Fork 0
Code Issues Pull Requests 2 Actions Packages Projects Releases Wiki Activity

docs: implementation plan for multi-server fan-out

Browse Source 
Three tasks: merge swarm branch to main, reframe deployment docs, and
make the Gitea action post to every endpoint in a fleet.

Claude-Session: https://claude.ai/code/session_01S3aqJ4tvaPezQhsGNCybut
This commit is contained in:
Samuel Enocsson
2026-08-04 19:19:49 +02:00
parent fcd5908992
commit 2421c74df1
1 changed files with 507 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/superpowers/plans/2026-08-04-multi-server-fanout.md
+507
View File
@@ -0,0 +1,507 @@
# Multi-Server Deployment & CI Fan-Out Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Land the finished Swarm support on `main`, document the one-instance-per-server deployment model, and make the Gitea composite action post `/update` to every server in a fleet instead of a single endpoint.
**Architecture:** No Go code changes. Part 1 is a merge plus documentation reframing — `MODE=compose|swarm` stays exclusive per instance, because each instance owns exactly one Docker daemon. Part 2 changes `gitea-action/action.yml` so its `endpoint` input accepts a newline-separated list, looping over every URL and continuing past failures so one dead host cannot stop the rest of the fleet from updating.
**Tech Stack:** Go 1.26.3 (unchanged, no code edits), Bash inside a Gitea composite action, `curl`, `jq`.
## Global Constraints
- **No Go source changes in this plan.** If a task seems to need one, stop and re-read the spec — the design explicitly rejects composite Finder/Executor work.
- `MODE` stays `compose` or `swarm` only. Do not add a `both` value.
- Do not weaken Compose mode's three-factor gate (token AND opt-in label AND `STACKS_ROOT` prefix).
- Existing single-URL usages of the action must keep working unchanged — a single URL is a one-element list. Do not add a separate input for the list form.
- Separator for the endpoint list is **newline**, never comma. URLs may legally contain commas.
- Conventional Commits (`feat:`, `fix:`, `docs:`, `ci:`, `build:`, `chore:`).
- Full test suite (`go test ./...`) must be green at the end of every task that touches the repo.
---
### Task 1: Merge Swarm support to `main`
**Files:**
- No files edited. This task merges branch `worktree-docker-swarm-support` (10 commits) into `main`.
**Interfaces:**
- Consumes: nothing.
- Produces: `main` gains `internal/discovery/swarm.go`, `internal/updater/swarm_executor.go`, `config.Config.Mode`, `discovery.Job.Image`, `discovery.Job.ServiceID`, and `api.ResultDTO.ServiceID`. Task 2 edits documentation files that this merge brings in (`README.md` "Swarm mode" section, `CLAUDE.md` swarm entries).
- [ ] **Step 1: Confirm the branch is where you think it is**
Run:
```bash
cd /Users/samuelenocsson/dev/package-updater
git log --oneline main..worktree-docker-swarm-support
```
Expected: exactly 10 commits, oldest `8f999c6 docs: add docker swarm support implementation plan`, newest `db84a2f fix(api): plumb request tag into discovery so swarm mode deploys the requested version`.
If the list differs, stop and report — someone has moved the branch since this plan was written.
- [ ] **Step 2: Verify the branch is green before merging**
Run:
```bash
cd /Users/samuelenocsson/dev/package-updater/.claude/worktrees/docker-swarm-support
go build ./... && go test ./...
```
Expected: build succeeds, all packages PASS. Do not proceed on a red branch — fix or report first.
- [ ] **Step 3: Merge into `main`**
```bash
cd /Users/samuelenocsson/dev/package-updater
git checkout main
git merge --no-ff worktree-docker-swarm-support -m "$(cat <<'EOF'
Merge branch 'worktree-docker-swarm-support'
Adds MODE=compose|swarm. Each instance owns one Docker daemon and runs
one paradigm; see docs/superpowers/specs/2026-08-04-multi-server-fanout-design.md
for why this exclusivity is the intended design.
EOF
)"
```
`--no-ff` is deliberate: it keeps the ten swarm commits grouped under one merge commit, matching how PRs #1 and #2 already appear in this repo's history.
Note: `docs/superpowers/plans/2026-07-04-docker-swarm-support.md` currently exists as an *untracked* file on `main` and is *committed* on the swarm branch. The merge brings in the tracked copy. If git refuses the merge with "untracked working tree file would be overwritten", delete the untracked copy first (`rm docs/superpowers/plans/2026-07-04-docker-swarm-support.md`) — the branch's committed version is identical and authoritative.
- [ ] **Step 4: Verify `main` is green after the merge**
Run:
```bash
go build ./... && go test ./...
```
Expected: build succeeds, all packages PASS. This is the first time the swarm code and the post-swarm `main` commits (`CLAUDE.md`, CI workflow) are compiled together on `main`.
- [ ] **Step 5: Push**
```bash
git push origin main
```
---
### Task 2: Document the deployment topology
**Files:**
- Modify: `README.md` (the version merged in Task 1 — it already contains a "Swarm mode" section at lines ~30-47, a `MODE` row in the config table at line ~71, and a "Known v1 gaps" bullet about single-host at lines ~103-105)
- Modify: `CLAUDE.md` (the "Design intent" section, which after Task 1 contains a "Compose mode and Swarm mode are selected once per deployment via `MODE`" entry)
**Interfaces:**
- Consumes: the merged documentation from Task 1.
- Produces: nothing consumed by later tasks. Task 3 documents the action separately in `gitea-action/README.md`.
There is no test for documentation. Verification is reading it back and confirming the claims match the code in `cmd/server/main.go` and `internal/config/config.go`.
- [ ] **Step 1: Rebase the working branch onto the updated `main`**
```bash
cd /Users/samuelenocsson/dev/package-updater
git checkout feat/multi-server-fanout
git rebase main
```
Expected: clean rebase. The branch currently holds one commit (`docs: design for multi-server deployment and CI fan-out`) touching only a new spec file, so there is nothing to conflict with.
- [ ] **Step 2: Add a deployment-topology section to `README.md`**
Insert this section immediately **after** the "Swarm mode" section and **before** "## Quick start":
```markdown
## Deploying across multiple servers
An instance can only reach the Docker daemon it is configured against. It cannot
update stacks on other machines. The deployment model follows from that:
**One instance per server.** Each server runs its own package-updater against its
own local Docker socket, with `MODE` set to whatever that machine runs:
| Server | `MODE` | Updates |
|---|---|---|
| Swarm manager node | `swarm` | All Swarm services in the cluster |
| Standalone Compose host | `compose` | Compose stacks on that host |
**CI fans out.** The [Gitea composite action](gitea-action/README.md) takes a list of
endpoints and posts `/update` to every instance, so one workflow run reaches the
whole fleet. Each instance answers for its own machine; there is no aggregated
cross-server response and no instance coordinates any other.
This is why `MODE` is exclusive rather than a mode that handles both at once: an
instance that could do both would still only reach one daemon, so the extra
generality buys nothing.
```
- [ ] **Step 3: Reword the `MODE` row in the configuration table**
In `README.md`'s configuration table, replace this row:
```markdown
| `MODE` | no | `compose` | `compose` or `swarm`. Selects the update mechanism for the whole deployment; not mixed per-request. |
```
with:
```markdown
| `MODE` | no | `compose` | `compose` or `swarm`. Which paradigm *this instance's* Docker daemon runs. See [Deploying across multiple servers](#deploying-across-multiple-servers). |
```
- [ ] **Step 4: Replace the "single host only" gap bullet**
In `README.md`'s "Known v1 gaps" section, replace this bullet:
```markdown
- **Single host only in Compose mode**. Swarm mode (`MODE=swarm`) is the
multi-node path, but only from a manager node's point of view — the updater
itself still needs manager API access (see "Swarm mode" above).
```
with:
```markdown
- **One instance reaches one daemon.** An instance never updates another server;
fleets run one instance per server with CI fanning out to all of them (see
"Deploying across multiple servers"). Swarm mode is the exception in that a
single manager-node instance covers the whole cluster.
```
- [ ] **Step 5: Reword the design-intent entry in `CLAUDE.md`**
In `CLAUDE.md`'s "Design intent (do not break without discussion)" section, replace this entry:
```markdown
- **Compose mode and Swarm mode are selected once per deployment via `MODE`**, never
mixed at request time. Swarm mode's security gate is opt-in label only — there is
no STACKS_ROOT-equivalent path check, since Swarm services have no local compose
file. Don't add one; don't weaken Compose mode's three-factor gate to match.
```
with:
```markdown
- **`MODE` is exclusive because each instance owns exactly one Docker daemon.**
A composite "handle both at once" mode has been considered and rejected: it would
still only reach one daemon, so it buys nothing. Fleets run one instance per
server and CI fans out. Reopen this only if a Swarm manager node starts running
standalone Compose stacks locally. Rationale and rejected alternatives:
`docs/superpowers/specs/2026-08-04-multi-server-fanout-design.md`.
- **Swarm mode's security gate is opt-in label only** — there is no
STACKS_ROOT-equivalent path check, since Swarm services have no local compose
file. Don't add one; don't weaken Compose mode's three-factor gate to match.
```
Splitting the original entry in two is deliberate: the mode-exclusivity rationale and the Swarm security-gate rule are separate rules that were sharing a bullet, and only the first one is changing.
- [ ] **Step 6: Verify the anchor link resolves**
The `MODE` table row links to `#deploying-across-multiple-servers`. Confirm the heading added in Step 2 is exactly `## Deploying across multiple servers` — GitHub/Gitea derive the anchor by lowercasing and replacing spaces with hyphens, so any wording drift silently breaks the link.
Run: `grep -n "^## Deploying across multiple servers" README.md`
Expected: one match.
- [ ] **Step 7: Commit**
```bash
git add README.md CLAUDE.md
git commit -m "docs: describe one-instance-per-server topology and why MODE is exclusive"
```
---
### Task 3: Fan out to multiple endpoints in the Gitea action
**Files:**
- Modify: `gitea-action/action.yml` (whole `runs.steps` block, lines 17-37, and the `endpoint` input description at lines 5-6)
- Modify: `gitea-action/README.md` (usage example, inputs table, failure-modes section)
**Interfaces:**
- Consumes: nothing from earlier tasks.
- Produces: the action's `endpoint` input accepts one URL (unchanged behaviour) or several separated by newlines. No other input changes name, type, or meaning.
- [ ] **Step 1: Rewrite `gitea-action/action.yml`**
Replace the entire file with:
```yaml
name: "Deploy via package-updater"
description: "Notifies one or more package-updater instances to pull & restart a service"
inputs:
endpoint:
description: "Full URL to /update. Give several, one per line, to update a fleet."
required: true
image:
description: "Image reference without tag (e.g. registry.example.com/myapp)"
required: true
tag:
description: "Tag that was just pushed (for logging)"
required: false
default: ""
token:
description: "Bearer token for package-updater"
required: true
runs:
using: "composite"
steps:
- name: Trigger update
shell: bash
env:
ENDPOINTS: ${{ inputs.endpoint }}
IMAGE: ${{ inputs.image }}
TAG: ${{ inputs.tag }}
TOKEN: ${{ inputs.token }}
run: |
# No `set -e`: a failing endpoint must not abort the loop, or one dead
# server leaves the rest of the fleet un-updated and hides which hosts
# actually succeeded.
set -uo pipefail
payload=$(jq -nc --arg image "$IMAGE" --arg tag "$TAG" \
'{image: $image, tag: $tag}')
attempted=0
failed=0
while IFS= read -r endpoint; do
# URLs never contain whitespace, so stripping all of it safely
# handles indentation, blank lines and CRLF line endings.
endpoint=$(printf '%s' "$endpoint" | tr -d '[:space:]')
[ -z "$endpoint" ] && continue
attempted=$((attempted + 1))
echo "--- $endpoint"
if ! response=$(curl -sS -w "\n%{http_code}" \
-X POST "$endpoint" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d "$payload"); then
echo "unreachable"
failed=$((failed + 1))
continue
fi
body=$(printf '%s' "$response" | head -n -1)
code=$(printf '%s' "$response" | tail -n 1)
echo "HTTP $code"
printf '%s' "$body" | jq . || printf '%s\n' "$body"
if [ "$code" -ge 400 ]; then
failed=$((failed + 1))
fi
done <<< "$ENDPOINTS"
if [ "$attempted" -eq 0 ]; then
echo "no endpoints given"
exit 1
fi
if [ "$failed" -gt 0 ]; then
echo "$failed of $attempted endpoint(s) failed"
exit 1
fi
echo "all $attempted endpoint(s) updated"
```
Three changes beyond the loop itself, each load-bearing:
1. **All inputs moved into `env:`.** A multi-line `${{ inputs.endpoint }}` interpolated directly into the script body would break it syntactically. Moving the others too keeps one consistent style in a short script, and stops a value containing shell metacharacters from being executed.
2. **`jq -nc` builds the payload** instead of hand-interpolating into a JSON string. A tag containing a quote previously produced malformed JSON and a confusing 400.
3. **`set -e` dropped**, `-uo pipefail` kept. This is the whole point of the task — see the comment in the script.
- [ ] **Step 2: Verify the loop continues past a failure**
This is the behaviour that justifies the task, so test it directly rather than trusting the code by inspection.
Start a server that answers POST with 200:
```bash
python3 -c "
import http.server
class H(http.server.BaseHTTPRequestHandler):
def do_POST(self):
self.send_response(200)
self.send_header('Content-Type','application/json')
self.end_headers()
self.wfile.write(b'{\"matched\":1,\"results\":[]}')
def log_message(self, *a): pass
http.server.HTTPServer(('127.0.0.1',8099), H).serve_forever()
" &
echo $! > /tmp/fanout-test-server.pid
```
Write the script body to a runnable file. This must be the **exact** text of the
`run:` block from Step 1, dedented — copy it, do not retype it, or you are testing
something other than what ships:
```bash
cat > /tmp/fanout-test.sh <<'SCRIPT'
set -uo pipefail
payload=$(jq -nc --arg image "$IMAGE" --arg tag "$TAG" \
'{image: $image, tag: $tag}')
attempted=0
failed=0
while IFS= read -r endpoint; do
endpoint=$(printf '%s' "$endpoint" | tr -d '[:space:]')
[ -z "$endpoint" ] && continue
attempted=$((attempted + 1))
echo "--- $endpoint"
if ! response=$(curl -sS -w "\n%{http_code}" \
-X POST "$endpoint" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d "$payload"); then
echo "unreachable"
failed=$((failed + 1))
continue
fi
body=$(printf '%s' "$response" | head -n -1)
code=$(printf '%s' "$response" | tail -n 1)
echo "HTTP $code"
printf '%s' "$body" | jq . || printf '%s\n' "$body"
if [ "$code" -ge 400 ]; then
failed=$((failed + 1))
fi
done <<< "$ENDPOINTS"
if [ "$attempted" -eq 0 ]; then
echo "no endpoints given"
exit 1
fi
if [ "$failed" -gt 0 ]; then
echo "$failed of $attempted endpoint(s) failed"
exit 1
fi
echo "all $attempted endpoint(s) updated"
SCRIPT
```
Now execute it with a list where the middle endpoint is dead (port 1 always refuses connections):
```bash
ENDPOINTS="http://127.0.0.1:8099/update
http://127.0.0.1:1/update
http://127.0.0.1:8099/update" \
IMAGE="registry.example.com/myapp" \
TAG="abc123" \
TOKEN="dummy" \
bash /tmp/fanout-test.sh
echo "exit code: $?"
```
Expected output shape:
```
--- http://127.0.0.1:8099/update
HTTP 200
{ "matched": 1, "results": [] }
--- http://127.0.0.1:1/update
unreachable
--- http://127.0.0.1:8099/update
HTTP 200
{ "matched": 1, "results": [] }
1 of 3 endpoint(s) failed
exit code: 1
```
Three things must all hold: the **third** endpoint was attempted (proves the loop continued), the exit code is **1**, and each endpoint's outcome is visible. If the third `---` line is missing, `set -e` crept back in.
- [ ] **Step 3: Verify a single endpoint still works unchanged**
```bash
ENDPOINTS="http://127.0.0.1:8099/update" \
IMAGE="registry.example.com/myapp" TAG="abc123" TOKEN="dummy" \
bash /tmp/fanout-test.sh
echo "exit code: $?"
```
Expected: one `--- ` line, `HTTP 200`, `all 1 endpoint(s) updated`, exit code 0. This is the existing-usage regression check.
- [ ] **Step 4: Verify blank lines and indentation are tolerated**
YAML block scalars routinely produce leading indentation and a trailing newline, so this is the realistic input shape, not an edge case.
```bash
ENDPOINTS=" http://127.0.0.1:8099/update
http://127.0.0.1:8099/update
" \
IMAGE="registry.example.com/myapp" TAG="abc123" TOKEN="dummy" \
bash /tmp/fanout-test.sh
echo "exit code: $?"
```
Expected: exactly two `--- http://127.0.0.1:8099/update` lines with no stray whitespace in the URL, `all 2 endpoint(s) updated`, exit code 0.
- [ ] **Step 5: Clean up the test server**
```bash
kill "$(cat /tmp/fanout-test-server.pid)" && rm -f /tmp/fanout-test-server.pid /tmp/fanout-test.sh
```
- [ ] **Step 6: Update `gitea-action/README.md`**
Replace the usage example's `with:` block so it shows the fleet form:
```yaml
- uses: gitea.example.com/shcizo/package-updater/gitea-action@v1
with:
endpoint: |
https://updater-swarm.example.com/update
https://updater-web01.example.com/update
https://updater-web02.example.com/update
image: registry.example.com/${{ gitea.repository }}
tag: ${{ gitea.sha }}
token: ${{ secrets.UPDATER_TOKEN }}
```
Change the `endpoint` row in the inputs table to:
```markdown
| `endpoint` | yes | — | Full URL to `/update`. Several may be given, one per line, to update a fleet. |
```
Replace the "Failure modes" section with:
```markdown
## Failure modes
Every endpoint is attempted, even when an earlier one fails — otherwise one dead
server would leave the rest of the fleet un-updated, and the CI log would not show
which hosts actually succeeded.
The step exits non-zero if any endpoint returned 4xx/5xx or was unreachable. The
log lists each endpoint with its HTTP status and response body, so a partial
deploy is visible at a glance.
```
Add this note after the `UPDATER_TOKEN` sentence:
```markdown
All instances in the fleet must share the same bearer token, since one `token`
input is sent to every endpoint.
```
- [ ] **Step 7: Commit**
```bash
git add gitea-action/action.yml gitea-action/README.md
git commit -m "feat(action): post /update to every endpoint in a fleet, continuing past failures"
```
- [ ] **Step 8: Push and open a PR**
```bash
git push -u origin feat/multi-server-fanout
```
Then open a PR against `main` (see `issue-tracker-cli` skill for whether this repo uses `gh` or `tea`). The PR covers the spec, the documentation reframing, and the action fan-out; the Swarm merge landed separately in Task 1.