package-updater/docs/superpowers/specs/2026-08-04-multi-server-fanout-design.md

# Multi-Server Deployment & CI Fan-Out — Design

**Date:** 2026-08-04
**Status:** Approved, ready for implementation planning

## Problem

package-updater was built as a single instance talking to a single local Docker
socket. Two things surfaced at once:

1. Swarm support (branch `worktree-docker-swarm-support`) added `MODE=compose|swarm`,
   selecting one paradigm per deployment. This looked like an artificial limitation:
   "why can't one instance do both?"
2. The real constraint is narrower and harder: an instance can only reach the Docker
   daemon it is configured against. Compose stacks live on other, separate servers.
   No amount of generalising the application makes one instance reach them.

The second point dissolves the first. The limitation was never in the application —
it was in the deployment topology (one instance for a fleet of servers).

## Decision

**One package-updater instance per server. CI fans out to all of them.**

- Each server runs its own instance against its own local Docker socket.
- The Swarm manager node runs an instance with `MODE=swarm`.
- Each standalone Compose host runs an instance with `MODE=compose`.
- The Gitea CI action posts `/update` to every instance.

`MODE` exclusivity is therefore correct, not a defect: each instance owns exactly one
machine running exactly one paradigm. No composite Finder or routing Executor is built.

### Rejected alternatives

| Alternative | Why rejected |
|---|---|
| Composite Finder + routing Executor (both modes in one instance) | Solves nothing — a single instance still cannot reach other servers' daemons. Only justified if a Swarm manager node *also* ran standalone Compose stacks locally, which it does not. |
| One instance holding N Docker clients (`ssh://` / TCP+TLS to each host) | Technically viable (`client.FromEnv` already supports it) but requires `Job` to carry host identity, discovery to iterate clients, and creates a single point of failure with broad network credentials. Buys nothing over per-server instances given CI can reach every server. |
| Controller + agents (pull model) | Only needed if servers cannot accept inbound traffic. They can. Would be a genuinely different application. |

### Assumptions

These held at design time; revisit the decision if any changes:

- Every server is reachable from Gitea CI over HTTP.
- Per-server outcomes are sufficient; no aggregated cross-server report is needed.
- The Swarm manager node runs Swarm services only, no local Compose stacks.
- All instances share one bearer token (org-level Gitea secret `UPDATER_TOKEN`).

## Scope

### Part 1 — Merge Swarm support to main

Branch `worktree-docker-swarm-support` (10 commits) is complete, builds clean, and
its tests pass. No code changes required. Documentation is reframed so `MODE`
describes *what paradigm this machine runs*, not a product-level restriction, and the
one-instance-per-server deployment model is written down — it is the expensive insight
of this design and is not derivable from the code.

Documentation changes:

- `README.md` — describe `MODE` per-instance; add a deployment-topology section
  covering one instance per server and CI fan-out.
- `CLAUDE.md` — reword the "Compose mode and Swarm mode are selected once per
  deployment" design-intent entry to state the reason (each instance owns one daemon),
  so a future reader does not re-litigate "make it handle both".

### Part 2 — CI fan-out in `gitea-action/`

`gitea-action/action.yml`'s `endpoint` input accepts a **newline-separated list** of
`/update` URLs instead of a single URL.

Behaviour:

- Post to each endpoint in list order.
- **Continue through the whole list even when one fails.** With `set -e` and a naive
  loop, an unreachable server B leaves server C never updated — a half-deployed fleet
  where the failure also hides which hosts succeeded.
- Print HTTP status and response body per endpoint, so the CI log shows the fleet
  state at a glance.
- Exit non-zero at the end if any endpoint returned 4xx/5xx or was unreachable.

A single URL remains a one-element list and keeps working unchanged. This is a
property of the format, not backward-compatibility work.

Newline rather than comma as separator: URLs may legally contain commas but never
line breaks, so the separator cannot collide with the data. Choosing an impossible
delimiter is cheaper than building escaping. This also matches the Actions convention
for list inputs (`paths`, `files`).

Blank lines and surrounding whitespace are ignored, so YAML block scalars with
trailing newlines and indentation behave as expected.

`README.md` in `gitea-action/` documents the list form with a worked multi-server
example.

## Out of scope

- Composite/routing discovery within one instance — not needed under this topology.
- Aggregated cross-server reporting — per-endpoint CI output is sufficient.
- Per-endpoint distinct tokens — all instances share one token.
- Wiring `internal/selfupdate` into the live queue — a pre-existing v1 gap, unrelated.

## Testing

- Part 1: existing suite (`go test ./...`) must stay green after merge. No new tests —
  no new code.
- Part 2: the action is shell in composite YAML with no test harness in this repo.
  Verification is manual: run the loop logic against a mix of reachable and
  unreachable endpoints and confirm (a) every endpoint is attempted, (b) the step
  exits non-zero, (c) each endpoint's status appears in the output.