feat: master/agent phase 1 — live inventory (#6) #19

2026-10-12T13:25:06Z

shcizo commented

2026-10-12 13:25:06 +00:00

Summary

Master/agent phase 1 (inventory): per-machine agents (updater-agent) dial the master (updater-master) over WebSocket with pinned-key TLS (MASTER_KEY) and per-agent tokens (pua_…, only the hash is stored). They send hello, a full snapshot, then diffs after Docker events settle, plus a full re-inventory every 60 s.
The master stores inventory in Postgres (embedded migrations), tracks online agents in memory, and serves an htmx UI with live SSE updates (Overview, Agents with add/revoke). The UI sits behind Authentik OIDC (PKCE) with an ALLOWED_GROUP check, Postgres-backed sessions and CSRF.
Today's cmd/server and its packages are untouched, so the existing updater keeps running. Nothing can be updated from the new UI yet (phase 2+).

Closes #6

The UI follows the plan's minimal templates, not the Ledger design. The redesign is tracked in #18.

Notable decisions

Plan deviations are listed in docs/superpowers/plans/2026-10-12-master-agent-phase-1-inventory.md, section "Deliberate deviations". One is a known limitation: Swarm rows are filed under the cluster ID the agent reports. A leaked agent token can therefore rewrite another cluster's overview rows. This is consistent with the spec ("feed false inventory, not command machines"). Phase 2 must not route Swarm jobs on a claimed cluster ID alone.
Hardening added during review:
- Revocation is re-checked after the connection registers.
- A duplicate hello is rejected.
- Swarm messages are accepted only from managers that report a cluster ID.
- Inventory writes lock the agent row and refuse revoked agents, so revoke cleanup cannot be undone by writes already in flight.
- Empty OIDC state is rejected, and the CSRF token is read from the form body only.
- The agent dial has a 30 s timeout.
- The token page sends Cache-Control: no-store.
Migration 0001 has a partial unique index on agents(name) WHERE revoked_at IS NULL, so a revoked agent's name can be reused. Revoking also removes the agent's Compose and container rows.
The master does OIDC discovery at startup and exits if the issuer is unreachable; Swarm restarts it.

Test plan

go test ./... passes with no Docker and no Postgres
go test -tags integration ./internal/master/store/ passes against Postgres 17 (17 tests)
go test -race passes for internal/agent/..., internal/master/... and internal/e2e
go vet ./... is clean, and go.mod still says go 1.26.3
docker build produces package-updater, updater-master and updater-agent
Manual smoke test from the plan against the real Authentik, with agents on swarm-mgr and the machine running slusk:
- Login works with a user in ALLOWED_GROUP. A user outside the group gets 403.
- Add agent shows a snippet. The agent connects and its services show up live.
- Swarm services appear once, even with two manager agents.
- Stopping and starting a container updates the overview within a few seconds.
- Revoke cuts the agent off immediately, and its services leave the overview.
- An expired session redirects htmx and SSE requests to login instead of injecting HTML.

## Summary - Master/agent phase 1 (inventory): per-machine agents (`updater-agent`) dial the master (`updater-master`) over WebSocket with pinned-key TLS (`MASTER_KEY`) and per-agent tokens (`pua_…`, only the hash is stored). They send hello, a full snapshot, then diffs after Docker events settle, plus a full re-inventory every 60 s. - The master stores inventory in Postgres (embedded migrations), tracks online agents in memory, and serves an htmx UI with live SSE updates (Overview, Agents with add/revoke). The UI sits behind Authentik OIDC (PKCE) with an `ALLOWED_GROUP` check, Postgres-backed sessions and CSRF. - Today's `cmd/server` and its packages are untouched, so the existing updater keeps running. Nothing can be updated from the new UI yet (phase 2+). Closes #6 The UI follows the plan's minimal templates, not the Ledger design. The redesign is tracked in #18. ## Notable decisions - Plan deviations are listed in `docs/superpowers/plans/2026-10-12-master-agent-phase-1-inventory.md`, section "Deliberate deviations". One is a known limitation: Swarm rows are filed under the cluster ID the agent reports. A leaked agent token can therefore rewrite another cluster's overview rows. This is consistent with the spec ("feed false inventory, not command machines"). Phase 2 must not route Swarm jobs on a claimed cluster ID alone. - Hardening added during review: - Revocation is re-checked after the connection registers. - A duplicate hello is rejected. - Swarm messages are accepted only from managers that report a cluster ID. - Inventory writes lock the agent row and refuse revoked agents, so revoke cleanup cannot be undone by writes already in flight. - Empty OIDC state is rejected, and the CSRF token is read from the form body only. - The agent dial has a 30 s timeout. - The token page sends `Cache-Control: no-store`. - Migration 0001 has a partial unique index on `agents(name) WHERE revoked_at IS NULL`, so a revoked agent's name can be reused. Revoking also removes the agent's Compose and container rows. - The master does OIDC discovery at startup and exits if the issuer is unreachable; Swarm restarts it. ## Test plan - [x] `go test ./...` passes with no Docker and no Postgres - [x] `go test -tags integration ./internal/master/store/` passes against Postgres 17 (17 tests) - [x] `go test -race` passes for `internal/agent/...`, `internal/master/...` and `internal/e2e` - [x] `go vet ./...` is clean, and `go.mod` still says `go 1.26.3` - [x] `docker build` produces `package-updater`, `updater-master` and `updater-agent` - [ ] Manual smoke test from the plan against the real Authentik, with agents on `swarm-mgr` and the machine running slusk: - [ ] Login works with a user in `ALLOWED_GROUP`. A user outside the group gets 403. - [ ] Add agent shows a snippet. The agent connects and its services show up live. - [ ] Swarm services appear once, even with two manager agents. - [ ] Stopping and starting a container updates the overview within a few seconds. - [ ] Revoke cuts the agent off immediately, and its services leave the overview. - [ ] An expired session redirects htmx and SSE requests to login instead of injecting HTML.

shcizo added 19 commits 2026-10-12 13:25:07 +00:00

feat(imageref): parse image references into repo, tag and digest adeddf0254

feat(protocol): define agent-to-master wire messages b482c2e29c

feat: pinned master key TLS and per-agent tokens 61e738e39d

feat(agent): collect compose, swarm and container inventory from docker abee9abb1b

feat(agent): diff inventories and debounce docker events 3edaf4e602

feat(agent): connect to master over pinned TLS and stream inventory ad6ff1e066

feat(agent): add updater-agent binary and ship it in the image 9b74286c9b

feat(master): postgres store with migrations, master identity and agents 99c553b7c6

feat(master): store service inventory snapshots and web sessions 8a1fb45971

feat(master): in-process broker for live UI change signals 5d75f9a7da

feat(master): agent hub that authenticates agents and stores their inventory 75e28262f8

fix(master): recheck revocation after register, reject duplicate hello 049324b2f4

feat(master): OIDC login with group check, sessions and CSRF protection b4894db6d7

feat(master): htmx UI with live service overview and agent management a596d67728

Vendored static assets (sha256):
e209dda5c8235479f3166defc7750e1dbcd5a5c1808b7792fc2e6733768fb447  htmx.min.js (htmx.org@2.0.4)
83eca6fa0611fe2b0bf1700b424b88b5eced38ef448ef9760a2ea08fbc875611  htmx-ext-sse.js (htmx-ext-sse@2.2.2)

feat(master): add updater-master binary wiring store, hub, auth and UI e8f7cb3378

test: agent to master inventory over pinned TLS end to end 05610391e3

docs: document master/agent phase 1 and add master stack example 4f092eaf79

fix: address final review findings for master/agent phase 1 (#6 ) 0e9e34db06

- inventory: resolve the image reference via ContainerInspect when a moved
  tag makes ContainerList report the image ID
- store: allow reusing a revoked agent's name (partial unique index) and
  mark the revoked agent's compose/container services removed
- web: publish a services event on revoke; no-store on the one-time token page
- webauth: reject empty OIDC state; accept the CSRF form token from the body only
- agent conn: bound the WebSocket handshake to 30s
- hub: log agent_replaced when a connection replaces an older one

fix(master): block inventory writes from revoked agents (#6 ) 2fe9b5ba0d

ApplySnapshot and UpsertService now take a FOR SHARE lock on the agent's
row and refuse with model.ErrRevoked when it is revoked. RevokeAgent's
UPDATE waits on that lock, so an in-flight snapshot or upsert can no
longer bring a revoked agent's services back after the revoke cleanup.

shcizo merged commit 3090b6d4f0 into main

2026-10-12 13:50:13 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: shcizo/package-updater#19