shcizo/package-updater
1
0
Fork 0
Code Issues Pull Requests 2 Actions Packages Projects Releases Wiki Activity
Files
e43868bf41a9397873c942e208887711cc209f2d
package-updater/docs/superpowers/plans/2026-05-22-package-updater-implementation.md
T

3103 lines
85 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# package-updater Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Build the v1 of `package-updater` — a Go HTTP service that receives webhook calls from Gitea Actions, finds matching Compose-managed containers via Docker labels, and runs `docker compose pull` + `up -d` for opt-in services.
**Architecture:** Stateless Go service. HTTP layer → discovery (Docker socket + Compose labels) → single FIFO worker → executor (shells out to `docker compose`). Self-update deferred until HTTP response is flushed. Defense-in-depth via bearer token + opt-in label.
**Tech Stack:** Go 1.23, stdlib `net/http` + `log/slog`, `github.com/docker/docker/client`, `github.com/prometheus/client_golang`, `github.com/google/uuid`, `github.com/stretchr/testify/require`. Deployed as a container with `docker.sock` and the stacks-root mounted. Reverse-proxied by Nginx Proxy Manager for TLS.
**Spec:** [`docs/superpowers/specs/2026-05-22-package-updater-design.md`](../specs/2026-05-22-package-updater-design.md)
---
## File Structure
```
package-updater/
├── cmd/
│ └── server/
│ └── main.go # Wire-up: config, logger, deps, http server
├── internal/
│ ├── api/
│ │ ├── handlers.go # /update, /healthz, /version handlers
│ │ ├── handlers_test.go
│ │ ├── middleware.go # auth, request_id, logging middleware
│ │ ├── middleware_test.go
│ │ └── types.go # Request/Response DTOs
│ ├── config/
│ │ ├── config.go # Env var loading + validation
│ │ └── config_test.go
│ ├── discovery/
│ │ ├── matching.go # Image name normalisation
│ │ ├── matching_test.go
│ │ ├── labels.go # Compose label extraction
│ │ ├── labels_test.go
│ │ ├── pathcheck.go # STACKS_ROOT path safety check
│ │ ├── pathcheck_test.go
│ │ ├── discovery.go # Orchestrates lookup → match → dedupe → jobs
│ │ ├── discovery_test.go
│ │ └── docker_client.go # DockerClient interface
│ ├── logging/
│ │ └── logger.go # slog JSON setup + request_id ctx helpers
│ ├── metrics/
│ │ └── metrics.go # Prometheus collectors
│ ├── selfupdate/
│ │ ├── selfupdate.go # Detect self-update + deferred-exec helper
│ │ └── selfupdate_test.go
│ └── updater/
│ ├── executor.go # Executor interface
│ ├── compose_executor.go # Real impl: shells out to docker compose
│ ├── queue.go # FIFO job queue
│ ├── queue_test.go
│ ├── worker.go # Single worker that drains queue
│ └── worker_test.go
├── docs/
│ └── superpowers/
│ ├── specs/2026-05-22-package-updater-design.md (already exists)
│ └── plans/2026-05-22-package-updater-implementation.md (this file)
├── gitea-action/
│ └── action.yml # Reusable composite action
├── Dockerfile
├── docker-compose.example.yml # Template for end-user deploy
├── .dockerignore
├── .gitignore
├── go.mod
├── go.sum
└── README.md
```
Each `internal/` sub-package owns one concept. `discovery` is split across files by responsibility (matching, labels, pathcheck, orchestration, docker-client interface) because each piece is independently testable. The `updater` package separates the queue, the worker that consumes it, and the executor abstraction that the worker calls — this lets the worker be tested with a fake executor.
---
## Task 1: Project scaffolding
**Files:**
- Create: `go.mod`
- Create: `.gitignore`
- Create: `cmd/server/main.go` (stub)
- Create: `README.md` (skeleton)
- [ ] **Step 1: Initialise the Go module**
Run:
```bash
go mod init github.com/shcizo/package-updater
```
Expected: creates `go.mod` with `module github.com/shcizo/package-updater` and `go 1.23` (or your installed version).
- [ ] **Step 2: Create `.gitignore`**
Write `.gitignore`:
```
# Binaries
/bin/
*.exe
*.dll
*.so
*.dylib
# Test binary, output of `go test -c`
*.test
# Coverage
*.out
coverage.html
# IDE
.idea/
.vscode/
*.swp
# Local env files
.env
.env.local
```
- [ ] **Step 3: Create stub `cmd/server/main.go`**
Write `cmd/server/main.go`:
```go
package main
import "fmt"
func main() {
fmt.Println("package-updater (scaffold)")
}
```
- [ ] **Step 4: Verify it builds**
Run:
```bash
go build ./cmd/server
./server
```
Expected output: `package-updater (scaffold)`. Then delete the binary: `rm server`.
- [ ] **Step 5: Create `README.md` skeleton**
Write `README.md`:
```markdown
# package-updater
Webhook-driven Docker Compose service updater. See [design spec](docs/superpowers/specs/2026-05-22-package-updater-design.md) for full design.
## Quick start
(TBD — filled in by final task)
```
- [ ] **Step 6: Commit**
```bash
git add go.mod .gitignore cmd/ README.md
git commit -m "chore: scaffold Go project layout"
```
---
## Task 2: Config loading (env vars + validation)
**Files:**
- Create: `internal/config/config.go`
- Create: `internal/config/config_test.go`
- [ ] **Step 1: Add testify dependency**
Run:
```bash
go get github.com/stretchr/testify/require
```
- [ ] **Step 2: Write the failing test**
Write `internal/config/config_test.go`:
```go
package config_test
import (
"testing"
"time"
"github.com/shcizo/package-updater/internal/config"
"github.com/stretchr/testify/require"
)
func TestLoad_RequiresAPIKey(t *testing.T) {
t.Setenv("UPDATER_API_KEY", "")
_, err := config.Load()
require.Error(t, err)
require.Contains(t, err.Error(), "UPDATER_API_KEY")
}
func TestLoad_AppliesDefaults(t *testing.T) {
t.Setenv("UPDATER_API_KEY", "secret")
t.Setenv("STACKS_ROOT", "")
t.Setenv("PORT", "")
t.Setenv("LOG_LEVEL", "")
t.Setenv("UPDATE_TIMEOUT", "")
t.Setenv("OPT_IN_LABEL", "")
cfg, err := config.Load()
require.NoError(t, err)
require.Equal(t, "secret", cfg.APIKey)
require.Equal(t, "/home/shcizo/self-hosted", cfg.StacksRoot)
require.Equal(t, "8080", cfg.Port)
require.Equal(t, "info", cfg.LogLevel)
require.Equal(t, 5*time.Minute, cfg.UpdateTimeout)
require.Equal(t, "se.shcizo.auto-update", cfg.OptInLabel)
}
func TestLoad_OverridesViaEnv(t *testing.T) {
t.Setenv("UPDATER_API_KEY", "secret")
t.Setenv("STACKS_ROOT", "/srv/stacks")
t.Setenv("PORT", "9090")
t.Setenv("LOG_LEVEL", "debug")
t.Setenv("UPDATE_TIMEOUT", "30s")
t.Setenv("OPT_IN_LABEL", "io.example.update")
cfg, err := config.Load()
require.NoError(t, err)
require.Equal(t, "/srv/stacks", cfg.StacksRoot)
require.Equal(t, "9090", cfg.Port)
require.Equal(t, "debug", cfg.LogLevel)
require.Equal(t, 30*time.Second, cfg.UpdateTimeout)
require.Equal(t, "io.example.update", cfg.OptInLabel)
}
func TestLoad_InvalidTimeoutErrors(t *testing.T) {
t.Setenv("UPDATER_API_KEY", "secret")
t.Setenv("UPDATE_TIMEOUT", "not-a-duration")
_, err := config.Load()
require.Error(t, err)
require.Contains(t, err.Error(), "UPDATE_TIMEOUT")
}
```
- [ ] **Step 3: Run test — verify it fails**
Run:
```bash
go test ./internal/config/...
```
Expected: build failure ("no Go files" or "package config does not exist").
- [ ] **Step 4: Implement `config.go`**
Write `internal/config/config.go`:
```go
// Package config loads service configuration from environment variables.
package config
import (
"errors"
"fmt"
"os"
"time"
)
// Config holds all runtime configuration for the service.
type Config struct {
APIKey string
StacksRoot string
Port string
LogLevel string
UpdateTimeout time.Duration
OptInLabel string
}
// Load reads configuration from environment variables, applies defaults,
// and validates required fields. Returns an error if validation fails so
// the service can fail-fast at startup.
func Load() (*Config, error) {
cfg := &Config{
APIKey: os.Getenv("UPDATER_API_KEY"),
StacksRoot: getenvDefault("STACKS_ROOT", "/home/shcizo/self-hosted"),
Port: getenvDefault("PORT", "8080"),
LogLevel: getenvDefault("LOG_LEVEL", "info"),
OptInLabel: getenvDefault("OPT_IN_LABEL", "se.shcizo.auto-update"),
}
if cfg.APIKey == "" {
return nil, errors.New("UPDATER_API_KEY is required")
}
timeoutStr := getenvDefault("UPDATE_TIMEOUT", "5m")
d, err := time.ParseDuration(timeoutStr)
if err != nil {
return nil, fmt.Errorf("UPDATE_TIMEOUT %q is not a valid duration: %w", timeoutStr, err)
}
cfg.UpdateTimeout = d
return cfg, nil
}
func getenvDefault(key, fallback string) string {
if v := os.Getenv(key); v != "" {
return v
}
return fallback
}
```
- [ ] **Step 5: Run tests — verify all pass**
Run:
```bash
go test ./internal/config/... -v
```
Expected: 4 tests PASS.
- [ ] **Step 6: Commit**
```bash
git add go.mod go.sum internal/config/
git commit -m "feat(config): load and validate env-var config"
```
---
## Task 3: Structured JSON logger
**Files:**
- Create: `internal/logging/logger.go`
- [ ] **Step 1: Implement the logger**
Write `internal/logging/logger.go`:
```go
// Package logging configures the structured JSON logger used across the service.
package logging
import (
"context"
"log/slog"
"os"
)
type ctxKey int
const requestIDKey ctxKey = iota
// New returns a slog.Logger that writes JSON to stdout at the given level.
// Valid levels: "debug", "info", "warn", "error". Unknown levels default to info.
func New(level string) *slog.Logger {
var lvl slog.Level
switch level {
case "debug":
lvl = slog.LevelDebug
case "warn":
lvl = slog.LevelWarn
case "error":
lvl = slog.LevelError
default:
lvl = slog.LevelInfo
}
handler := slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
Level: lvl,
ReplaceAttr: func(_ []string, a slog.Attr) slog.Attr {
if a.Key == slog.TimeKey {
return slog.Attr{Key: "time", Value: a.Value}
}
if a.Key == slog.MessageKey {
return slog.Attr{Key: "event", Value: a.Value}
}
return a
},
})
return slog.New(handler)
}
// WithRequestID stores the request ID in the context for downstream loggers.
func WithRequestID(ctx context.Context, id string) context.Context {
return context.WithValue(ctx, requestIDKey, id)
}
// RequestIDFrom returns the request ID stored in ctx, or empty string if absent.
func RequestIDFrom(ctx context.Context) string {
if v, ok := ctx.Value(requestIDKey).(string); ok {
return v
}
return ""
}
// FromContext returns a logger pre-bound with the request_id from ctx (if any).
func FromContext(ctx context.Context, base *slog.Logger) *slog.Logger {
if id := RequestIDFrom(ctx); id != "" {
return base.With("request_id", id)
}
return base
}
```
- [ ] **Step 2: Verify it builds**
Run:
```bash
go build ./internal/logging/...
```
Expected: no output (success).
- [ ] **Step 3: Commit**
```bash
git add internal/logging/
git commit -m "feat(logging): structured JSON logger with request_id context"
```
Note: no unit tests for this file — it's thin glue over stdlib. It will be exercised end-to-end via the HTTP handler tests in later tasks.
---
## Task 4: Image name normalisation
**Files:**
- Create: `internal/discovery/matching.go`
- Create: `internal/discovery/matching_test.go`
- [ ] **Step 1: Write the failing test**
Write `internal/discovery/matching_test.go`:
```go
package discovery_test
import (
"testing"
"github.com/shcizo/package-updater/internal/discovery"
"github.com/stretchr/testify/require"
)
func TestNormaliseImage(t *testing.T) {
cases := []struct {
in string
want string
}{
{"registry.example.com/myapp", "registry.example.com/myapp"},
{"registry.example.com/myapp:v1.2.3", "registry.example.com/myapp"},
{"registry.example.com/myapp:latest", "registry.example.com/myapp"},
{"registry.example.com/myapp@sha256:abc123", "registry.example.com/myapp"},
{"registry.example.com/myapp:v1.2.3@sha256:abc123", "registry.example.com/myapp"},
{"nginx", "nginx"},
{"nginx:1.25-alpine", "nginx"},
{"library/nginx:latest", "library/nginx"},
{"gcr.io/proj/svc:tag", "gcr.io/proj/svc"},
{"localhost:5000/myimg:v1", "localhost:5000/myimg"},
}
for _, c := range cases {
t.Run(c.in, func(t *testing.T) {
require.Equal(t, c.want, discovery.NormaliseImage(c.in))
})
}
}
func TestImagesMatch(t *testing.T) {
require.True(t, discovery.ImagesMatch(
"registry.example.com/myapp",
"registry.example.com/myapp:v1.2.3",
))
require.True(t, discovery.ImagesMatch(
"registry.example.com/myapp:v1.0.0",
"registry.example.com/myapp:v9.9.9",
))
require.False(t, discovery.ImagesMatch(
"registry.example.com/myapp",
"registry.example.com/otherapp",
))
// Case-sensitive per spec section 5.2
require.False(t, discovery.ImagesMatch(
"registry.example.com/MyApp",
"registry.example.com/myapp",
))
}
```
- [ ] **Step 2: Run test — verify it fails**
Run:
```bash
go test ./internal/discovery/...
```
Expected: build failure (`undefined: discovery.NormaliseImage`).
- [ ] **Step 3: Implement `matching.go`**
Write `internal/discovery/matching.go`:
```go
package discovery
import "strings"
// NormaliseImage strips the tag and digest from an image reference,
// returning the bare repository name.
//
// Tricky case: "localhost:5000/foo:v1" — the first colon is a port,
// not a tag. We disambiguate by splitting on "/" first and only
// treating colons in the last segment as tag separators.
func NormaliseImage(ref string) string {
if at := strings.Index(ref, "@"); at >= 0 {
ref = ref[:at]
}
slash := strings.LastIndex(ref, "/")
if slash < 0 {
if colon := strings.Index(ref, ":"); colon >= 0 {
return ref[:colon]
}
return ref
}
prefix, last := ref[:slash], ref[slash+1:]
if colon := strings.Index(last, ":"); colon >= 0 {
last = last[:colon]
}
return prefix + "/" + last
}
// ImagesMatch reports whether two image references resolve to the same
// repository, ignoring tag and digest. Case-sensitive per spec section 5.2.
func ImagesMatch(a, b string) bool {
return NormaliseImage(a) == NormaliseImage(b)
}
```
- [ ] **Step 4: Run tests — verify all pass**
Run:
```bash
go test ./internal/discovery/... -v -run 'TestNormaliseImage|TestImagesMatch'
```
Expected: all subtests PASS.
- [ ] **Step 5: Commit**
```bash
git add internal/discovery/matching.go internal/discovery/matching_test.go
git commit -m "feat(discovery): tag-agnostic image name normalisation"
```
---
## Task 5: Compose label extraction
**Files:**
- Create: `internal/discovery/labels.go`
- Create: `internal/discovery/labels_test.go`
- [ ] **Step 1: Write the failing test**
Write `internal/discovery/labels_test.go`:
```go
package discovery_test
import (
"testing"
"github.com/shcizo/package-updater/internal/discovery"
"github.com/stretchr/testify/require"
)
func TestParseComposeLabels_Success(t *testing.T) {
labels := map[string]string{
"com.docker.compose.project": "myapp-prod",
"com.docker.compose.service": "web",
"com.docker.compose.project.working_dir": "/home/shcizo/self-hosted/myapp-prod",
"com.docker.compose.project.config_files": "/home/shcizo/self-hosted/myapp-prod/docker-compose.yml",
"se.shcizo.auto-update": "true",
}
got, err := discovery.ParseComposeLabels(labels)
require.NoError(t, err)
require.Equal(t, "myapp-prod", got.Project)
require.Equal(t, "web", got.Service)
require.Equal(t, "/home/shcizo/self-hosted/myapp-prod", got.WorkingDir)
require.Equal(t, []string{"/home/shcizo/self-hosted/myapp-prod/docker-compose.yml"}, got.ConfigFiles)
}
func TestParseComposeLabels_MultipleConfigFiles(t *testing.T) {
labels := map[string]string{
"com.docker.compose.project": "myapp",
"com.docker.compose.service": "web",
"com.docker.compose.project.working_dir": "/srv/myapp",
"com.docker.compose.project.config_files": "/srv/myapp/docker-compose.yml,/srv/myapp/docker-compose.prod.yml",
}
got, err := discovery.ParseComposeLabels(labels)
require.NoError(t, err)
require.Equal(t, []string{
"/srv/myapp/docker-compose.yml",
"/srv/myapp/docker-compose.prod.yml",
}, got.ConfigFiles)
}
func TestParseComposeLabels_MissingFieldsError(t *testing.T) {
cases := []struct {
name string
missing string
labels map[string]string
}{
{"project", "com.docker.compose.project", map[string]string{
"com.docker.compose.service": "web",
"com.docker.compose.project.working_dir": "/x",
"com.docker.compose.project.config_files": "/x/y.yml",
}},
{"service", "com.docker.compose.service", map[string]string{
"com.docker.compose.project": "p",
"com.docker.compose.project.working_dir": "/x",
"com.docker.compose.project.config_files": "/x/y.yml",
}},
{"working_dir", "com.docker.compose.project.working_dir", map[string]string{
"com.docker.compose.project": "p",
"com.docker.compose.service": "web",
"com.docker.compose.project.config_files": "/x/y.yml",
}},
{"config_files", "com.docker.compose.project.config_files", map[string]string{
"com.docker.compose.project": "p",
"com.docker.compose.service": "web",
"com.docker.compose.project.working_dir": "/x",
}},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
_, err := discovery.ParseComposeLabels(c.labels)
require.Error(t, err)
require.Contains(t, err.Error(), c.missing)
})
}
}
func TestHasOptIn(t *testing.T) {
require.True(t, discovery.HasOptIn(map[string]string{
"se.shcizo.auto-update": "true",
}, "se.shcizo.auto-update"))
require.False(t, discovery.HasOptIn(map[string]string{
"se.shcizo.auto-update": "false",
}, "se.shcizo.auto-update"))
require.False(t, discovery.HasOptIn(map[string]string{
"se.shcizo.auto-update": "TRUE",
}, "se.shcizo.auto-update"))
require.False(t, discovery.HasOptIn(map[string]string{}, "se.shcizo.auto-update"))
}
```
- [ ] **Step 2: Run test — verify it fails**
Run:
```bash
go test ./internal/discovery/...
```
Expected: build failure (`undefined: discovery.ParseComposeLabels`).
- [ ] **Step 3: Implement `labels.go`**
Write `internal/discovery/labels.go`:
```go
package discovery
import (
"fmt"
"strings"
)
// ComposeLabels captures the four Compose-managed labels we need
// to drive a `docker compose pull`/`up -d` against the right stack.
type ComposeLabels struct {
Project string
Service string
WorkingDir string
ConfigFiles []string
}
const (
labelProject = "com.docker.compose.project"
labelService = "com.docker.compose.service"
labelWorkingDir = "com.docker.compose.project.working_dir"
labelConfigFiles = "com.docker.compose.project.config_files"
)
// ParseComposeLabels extracts the Compose labels we need. Returns an
// error naming the missing label if any required field is absent —
// in practice this should only happen if the container was not
// started by Compose.
func ParseComposeLabels(labels map[string]string) (ComposeLabels, error) {
get := func(key string) (string, error) {
v, ok := labels[key]
if !ok || v == "" {
return "", fmt.Errorf("missing required label: %s", key)
}
return v, nil
}
project, err := get(labelProject)
if err != nil {
return ComposeLabels{}, err
}
service, err := get(labelService)
if err != nil {
return ComposeLabels{}, err
}
workingDir, err := get(labelWorkingDir)
if err != nil {
return ComposeLabels{}, err
}
configFilesRaw, err := get(labelConfigFiles)
if err != nil {
return ComposeLabels{}, err
}
files := strings.Split(configFilesRaw, ",")
for i, f := range files {
files[i] = strings.TrimSpace(f)
}
return ComposeLabels{
Project: project,
Service: service,
WorkingDir: workingDir,
ConfigFiles: files,
}, nil
}
// HasOptIn reports whether the labels include the opt-in marker with
// value "true" (exact, case-sensitive — anything else is excluded).
func HasOptIn(labels map[string]string, key string) bool {
return labels[key] == "true"
}
```
- [ ] **Step 4: Run tests — verify all pass**
Run:
```bash
go test ./internal/discovery/... -v -run 'TestParseComposeLabels|TestHasOptIn'
```
Expected: all subtests PASS.
- [ ] **Step 5: Commit**
```bash
git add internal/discovery/labels.go internal/discovery/labels_test.go
git commit -m "feat(discovery): parse Compose-managed labels"
```
---
## Task 6: Path safety check
**Files:**
- Create: `internal/discovery/pathcheck.go`
- Create: `internal/discovery/pathcheck_test.go`
- [ ] **Step 1: Write the failing test**
Write `internal/discovery/pathcheck_test.go`:
```go
package discovery_test
import (
"testing"
"github.com/shcizo/package-updater/internal/discovery"
"github.com/stretchr/testify/require"
)
func TestIsInsideRoot(t *testing.T) {
cases := []struct {
name string
root string
path string
want bool
}{
{"direct child", "/home/shcizo/self-hosted", "/home/shcizo/self-hosted/myapp", true},
{"nested", "/home/shcizo/self-hosted", "/home/shcizo/self-hosted/a/b/c", true},
{"root itself", "/home/shcizo/self-hosted", "/home/shcizo/self-hosted", true},
{"sibling", "/home/shcizo/self-hosted", "/home/shcizo/other", false},
{"parent", "/home/shcizo/self-hosted", "/home/shcizo", false},
{"unrelated", "/home/shcizo/self-hosted", "/etc/passwd", false},
{"prefix-trick", "/home/shcizo/self-hosted", "/home/shcizo/self-hosted-evil", false},
{"dotdot escape", "/home/shcizo/self-hosted", "/home/shcizo/self-hosted/../etc", false},
{"trailing slash root", "/home/shcizo/self-hosted/", "/home/shcizo/self-hosted/x", true},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
require.Equal(t, c.want, discovery.IsInsideRoot(c.root, c.path))
})
}
}
```
- [ ] **Step 2: Run test — verify it fails**
Run:
```bash
go test ./internal/discovery/... -run TestIsInsideRoot
```
Expected: build failure (`undefined: discovery.IsInsideRoot`).
- [ ] **Step 3: Implement `pathcheck.go`**
Write `internal/discovery/pathcheck.go`:
```go
package discovery
import (
"path/filepath"
"strings"
)
// IsInsideRoot reports whether path is the same as, or nested inside,
// root. Both are cleaned before comparison so trailing slashes, "."
// segments, and ".." escapes are handled. Prefix tricks like
// "/foo" vs "/foo-evil" are NOT considered inside.
func IsInsideRoot(root, path string) bool {
r := filepath.Clean(root)
p := filepath.Clean(path)
if p == r {
return true
}
rel, err := filepath.Rel(r, p)
if err != nil {
return false
}
return !strings.HasPrefix(rel, "..")
}
```
- [ ] **Step 4: Run tests — verify all pass**
Run:
```bash
go test ./internal/discovery/... -v -run TestIsInsideRoot
```
Expected: all subtests PASS.
- [ ] **Step 5: Commit**
```bash
git add internal/discovery/pathcheck.go internal/discovery/pathcheck_test.go
git commit -m "feat(discovery): STACKS_ROOT path safety check"
```
---
## Task 7: Discovery orchestration (find → match → dedup → jobs)
**Files:**
- Create: `internal/discovery/docker_client.go`
- Create: `internal/discovery/discovery.go`
- Create: `internal/discovery/discovery_test.go`
- [ ] **Step 1: Add Docker SDK dependency**
Run:
```bash
go get github.com/docker/docker/client github.com/docker/docker/api/types github.com/docker/docker/api/types/container
```
- [ ] **Step 2: Define the DockerClient interface**
Write `internal/discovery/docker_client.go`:
```go
package discovery
import (
"context"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
)
// DockerClient is the subset of the Docker SDK we depend on.
// Defined as an interface so tests can supply a fake.
type DockerClient interface {
ContainerList(ctx context.Context, opts container.ListOptions) ([]types.Container, error)
Ping(ctx context.Context) (types.Ping, error)
}
```
- [ ] **Step 3: Write the failing test**
Write `internal/discovery/discovery_test.go`:
```go
package discovery_test
import (
"context"
"errors"
"testing"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
"github.com/shcizo/package-updater/internal/discovery"
"github.com/stretchr/testify/require"
)
type fakeDockerClient struct {
containers []types.Container
err error
}
func (f *fakeDockerClient) ContainerList(_ context.Context, _ container.ListOptions) ([]types.Container, error) {
return f.containers, f.err
}
func (f *fakeDockerClient) Ping(_ context.Context) (types.Ping, error) {
return types.Ping{}, nil
}
func mkContainer(image string, labels map[string]string) types.Container {
return types.Container{Image: image, Labels: labels}
}
func mkComposeLabels(project, service, workingDir, configFile string, optIn bool) map[string]string {
m := map[string]string{
"com.docker.compose.project": project,
"com.docker.compose.service": service,
"com.docker.compose.project.working_dir": workingDir,
"com.docker.compose.project.config_files": configFile,
}
if optIn {
m["se.shcizo.auto-update"] = "true"
}
return m
}
func TestFindJobs_MatchAndOptIn(t *testing.T) {
fake := &fakeDockerClient{containers: []types.Container{
mkContainer("registry.example.com/myapp:v1", mkComposeLabels(
"myapp-prod", "web",
"/home/shcizo/self-hosted/myapp-prod",
"/home/shcizo/self-hosted/myapp-prod/docker-compose.yml",
true,
)),
mkContainer("registry.example.com/other:v1", mkComposeLabels(
"other", "web",
"/home/shcizo/self-hosted/other",
"/home/shcizo/self-hosted/other/docker-compose.yml",
true,
)),
}}
d := discovery.New(fake, "/home/shcizo/self-hosted", "se.shcizo.auto-update")
jobs, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
require.NoError(t, err)
require.Len(t, jobs, 1)
require.Equal(t, "myapp-prod", jobs[0].Project)
require.Equal(t, "web", jobs[0].Service)
require.Equal(t, "/home/shcizo/self-hosted/myapp-prod", jobs[0].WorkingDir)
}
func TestFindJobs_SkipsWithoutOptIn(t *testing.T) {
fake := &fakeDockerClient{containers: []types.Container{
mkContainer("registry.example.com/myapp:v1", mkComposeLabels(
"myapp-prod", "web",
"/home/shcizo/self-hosted/myapp-prod",
"/home/shcizo/self-hosted/myapp-prod/docker-compose.yml",
false, // no opt-in
)),
}}
d := discovery.New(fake, "/home/shcizo/self-hosted", "se.shcizo.auto-update")
jobs, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
require.NoError(t, err)
require.Empty(t, jobs)
}
func TestFindJobs_DedupReplicas(t *testing.T) {
labels := mkComposeLabels(
"myapp", "web",
"/home/shcizo/self-hosted/myapp",
"/home/shcizo/self-hosted/myapp/docker-compose.yml",
true,
)
fake := &fakeDockerClient{containers: []types.Container{
mkContainer("registry.example.com/myapp:v1", labels),
mkContainer("registry.example.com/myapp:v1", labels),
mkContainer("registry.example.com/myapp:v1", labels),
}}
d := discovery.New(fake, "/home/shcizo/self-hosted", "se.shcizo.auto-update")
jobs, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
require.NoError(t, err)
require.Len(t, jobs, 1)
}
func TestFindJobs_OutsideRootProducesRefusedJob(t *testing.T) {
fake := &fakeDockerClient{containers: []types.Container{
mkContainer("registry.example.com/myapp:v1", mkComposeLabels(
"myapp", "web",
"/opt/elsewhere/myapp",
"/opt/elsewhere/myapp/docker-compose.yml",
true,
)),
}}
d := discovery.New(fake, "/home/shcizo/self-hosted", "se.shcizo.auto-update")
jobs, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
require.NoError(t, err)
require.Len(t, jobs, 1)
require.True(t, jobs[0].Refused)
}
func TestFindJobs_MultipleStacksSameImage(t *testing.T) {
fake := &fakeDockerClient{containers: []types.Container{
mkContainer("registry.example.com/myapp:v1", mkComposeLabels(
"myapp-prod", "web",
"/home/shcizo/self-hosted/myapp-prod",
"/home/shcizo/self-hosted/myapp-prod/docker-compose.yml",
true,
)),
mkContainer("registry.example.com/myapp:v1", mkComposeLabels(
"myapp-staging", "web",
"/home/shcizo/self-hosted/myapp-staging",
"/home/shcizo/self-hosted/myapp-staging/docker-compose.yml",
true,
)),
}}
d := discovery.New(fake, "/home/shcizo/self-hosted", "se.shcizo.auto-update")
jobs, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
require.NoError(t, err)
require.Len(t, jobs, 2)
}
func TestFindJobs_DockerError(t *testing.T) {
fake := &fakeDockerClient{err: errors.New("connection refused")}
d := discovery.New(fake, "/home/shcizo/self-hosted", "se.shcizo.auto-update")
_, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
require.Error(t, err)
}
func TestFindJobs_NonComposeContainerIsSkipped(t *testing.T) {
fake := &fakeDockerClient{containers: []types.Container{
mkContainer("registry.example.com/myapp:v1", map[string]string{
"se.shcizo.auto-update": "true",
// no compose labels at all
}),
}}
d := discovery.New(fake, "/home/shcizo/self-hosted", "se.shcizo.auto-update")
jobs, err := d.FindJobs(context.Background(), "registry.example.com/myapp")
require.NoError(t, err)
require.Empty(t, jobs)
}
```
- [ ] **Step 4: Run tests — verify they fail**
Run:
```bash
go test ./internal/discovery/...
```
Expected: build failure (`undefined: discovery.New` etc).
- [ ] **Step 5: Implement `discovery.go`**
Write `internal/discovery/discovery.go`:
```go
package discovery
import (
"context"
"fmt"
"sort"
"strings"
"github.com/docker/docker/api/types/container"
)
// Job describes a single (project, service, config_files) update to execute.
type Job struct {
Project string
Service string
WorkingDir string
ConfigFiles []string
// Refused is true when the WorkingDir falls outside STACKS_ROOT.
// The job is returned so the caller can surface a per-job "refused"
// result, but it MUST NOT be executed.
Refused bool
RefusedReason string
}
// Discovery orchestrates "given an image, what jobs should we enqueue?".
type Discovery struct {
cli DockerClient
stacksRoot string
optInLabel string
}
// New returns a Discovery bound to the given Docker client and config.
func New(cli DockerClient, stacksRoot, optInLabel string) *Discovery {
return &Discovery{cli: cli, stacksRoot: stacksRoot, optInLabel: optInLabel}
}
// FindJobs lists running containers, filters by image match + opt-in label,
// extracts Compose info, applies the path safety check, and deduplicates.
func (d *Discovery) FindJobs(ctx context.Context, image string) ([]Job, error) {
all, err := d.cli.ContainerList(ctx, container.ListOptions{All: true})
if err != nil {
return nil, fmt.Errorf("docker container list: %w", err)
}
seen := make(map[string]struct{})
var jobs []Job
for _, c := range all {
if !ImagesMatch(image, c.Image) {
continue
}
if !HasOptIn(c.Labels, d.optInLabel) {
continue
}
cl, err := ParseComposeLabels(c.Labels)
if err != nil {
// Not started by Compose — skip silently (warning logged by caller).
continue
}
refused := !IsInsideRoot(d.stacksRoot, cl.WorkingDir)
reason := ""
if refused {
reason = fmt.Sprintf("working_dir %q outside STACKS_ROOT %q", cl.WorkingDir, d.stacksRoot)
}
key := dedupKey(cl)
if _, dup := seen[key]; dup {
continue
}
seen[key] = struct{}{}
jobs = append(jobs, Job{
Project: cl.Project,
Service: cl.Service,
WorkingDir: cl.WorkingDir,
ConfigFiles: cl.ConfigFiles,
Refused: refused,
RefusedReason: reason,
})
}
return jobs, nil
}
func dedupKey(cl ComposeLabels) string {
files := append([]string(nil), cl.ConfigFiles...)
sort.Strings(files)
return cl.Project + "|" + cl.Service + "|" + strings.Join(files, ",")
}
```
- [ ] **Step 6: Run tests — verify all pass**
Run:
```bash
go test ./internal/discovery/... -v
```
Expected: all tests in the package PASS.
- [ ] **Step 7: Commit**
```bash
git add go.mod go.sum internal/discovery/
git commit -m "feat(discovery): orchestrate match, opt-in filter, dedup, path-check"
```
---
## Task 8: Executor interface + fake
**Files:**
- Create: `internal/updater/executor.go`
- [ ] **Step 1: Define the interface**
Write `internal/updater/executor.go`:
```go
// Package updater contains the FIFO job queue, the single worker that
// drains it, and the abstraction over `docker compose` invocation.
package updater
import (
"context"
"github.com/shcizo/package-updater/internal/discovery"
)
// Executor runs `docker compose pull` then `up -d` for a single job.
// Defined as an interface so the worker can be tested with a fake.
type Executor interface {
Execute(ctx context.Context, job discovery.Job) error
}
```
- [ ] **Step 2: Verify it builds**
Run:
```bash
go build ./internal/updater/...
```
Expected: no output (success).
- [ ] **Step 3: Commit**
```bash
git add internal/updater/executor.go
git commit -m "feat(updater): Executor interface"
```
---
## Task 9: Real ComposeExecutor (shells out to docker compose)
**Files:**
- Create: `internal/updater/compose_executor.go`
- [ ] **Step 1: Implement the real executor**
Write `internal/updater/compose_executor.go`:
```go
package updater
import (
"context"
"fmt"
"os/exec"
"strings"
"github.com/shcizo/package-updater/internal/discovery"
)
// ComposeExecutor invokes `docker compose` as a subprocess.
type ComposeExecutor struct{}
// NewComposeExecutor returns an executor that shells out to docker compose.
func NewComposeExecutor() *ComposeExecutor {
return &ComposeExecutor{}
}
// Execute runs `docker compose -f <file>... -p <project> pull <service>`
// followed by `... up -d <service>`. Working directory is set to
// job.WorkingDir so any relative paths in the compose file resolve correctly.
func (e *ComposeExecutor) Execute(ctx context.Context, job discovery.Job) error {
if job.Refused {
return fmt.Errorf("refused: %s", job.RefusedReason)
}
if err := e.run(ctx, job, "pull"); err != nil {
return fmt.Errorf("pull: %w", err)
}
if err := e.run(ctx, job, "up", "-d"); err != nil {
return fmt.Errorf("up: %w", err)
}
return nil
}
func (e *ComposeExecutor) run(ctx context.Context, job discovery.Job, args ...string) error {
cliArgs := []string{"compose"}
for _, f := range job.ConfigFiles {
cliArgs = append(cliArgs, "-f", f)
}
cliArgs = append(cliArgs, "-p", job.Project)
cliArgs = append(cliArgs, args...)
cliArgs = append(cliArgs, job.Service)
cmd := exec.CommandContext(ctx, "docker", cliArgs...)
cmd.Dir = job.WorkingDir
out, err := cmd.CombinedOutput()
if err != nil {
return fmt.Errorf("docker %s: %w (output: %s)",
strings.Join(args, " "), err, strings.TrimSpace(string(out)))
}
return nil
}
```
- [ ] **Step 2: Verify it builds**
Run:
```bash
go build ./internal/updater/...
```
Expected: no output (success).
- [ ] **Step 3: Commit**
```bash
git add internal/updater/compose_executor.go
git commit -m "feat(updater): ComposeExecutor shells out to docker compose"
```
Note: no unit tests — exec'ing real `docker compose` is an integration concern that's covered by the spec's "manual smoke test on first deploy" plan (spec section 14). The path-construction logic is exercised indirectly via the worker tests (Task 10) using a fake executor.
---
## Task 10: Job queue + single worker
**Files:**
- Create: `internal/updater/queue.go`
- Create: `internal/updater/queue_test.go`
- Create: `internal/updater/worker.go`
- Create: `internal/updater/worker_test.go`
- [ ] **Step 1: Write the failing queue test**
Write `internal/updater/queue_test.go`:
```go
package updater_test
import (
"context"
"errors"
"sync"
"testing"
"time"
"github.com/shcizo/package-updater/internal/discovery"
"github.com/shcizo/package-updater/internal/updater"
"github.com/stretchr/testify/require"
)
type fakeExec struct {
mu sync.Mutex
calls []discovery.Job
delay time.Duration
errForSvc map[string]error
}
func (f *fakeExec) Execute(ctx context.Context, job discovery.Job) error {
f.mu.Lock()
f.calls = append(f.calls, job)
f.mu.Unlock()
if f.delay > 0 {
select {
case <-time.After(f.delay):
case <-ctx.Done():
return ctx.Err()
}
}
if err, ok := f.errForSvc[job.Service]; ok {
return err
}
return nil
}
func (f *fakeExec) callsCopy() []discovery.Job {
f.mu.Lock()
defer f.mu.Unlock()
return append([]discovery.Job(nil), f.calls...)
}
func TestQueue_ProcessesFIFO(t *testing.T) {
exec := &fakeExec{delay: 20 * time.Millisecond}
q := updater.NewQueue(exec)
q.Start(context.Background())
defer q.Stop()
job1 := discovery.Job{Project: "a", Service: "svc"}
job2 := discovery.Job{Project: "b", Service: "svc"}
job3 := discovery.Job{Project: "c", Service: "svc"}
results := make(chan []updater.Result, 3)
go func() { results <- q.Submit(context.Background(), []discovery.Job{job1}) }()
time.Sleep(5 * time.Millisecond)
go func() { results <- q.Submit(context.Background(), []discovery.Job{job2}) }()
time.Sleep(5 * time.Millisecond)
go func() { results <- q.Submit(context.Background(), []discovery.Job{job3}) }()
for i := 0; i < 3; i++ {
<-results
}
calls := exec.callsCopy()
require.Equal(t, []string{"a", "b", "c"}, []string{calls[0].Project, calls[1].Project, calls[2].Project})
}
func TestQueue_ReturnsPerJobResults(t *testing.T) {
exec := &fakeExec{errForSvc: map[string]error{"failing": errors.New("boom")}}
q := updater.NewQueue(exec)
q.Start(context.Background())
defer q.Stop()
jobs := []discovery.Job{
{Project: "p1", Service: "ok"},
{Project: "p2", Service: "failing"},
{Project: "p3", Service: "refused-svc", Refused: true, RefusedReason: "outside root"},
}
results := q.Submit(context.Background(), jobs)
require.Len(t, results, 3)
require.Equal(t, updater.StatusUpdated, results[0].Status)
require.Equal(t, updater.StatusFailed, results[1].Status)
require.Contains(t, results[1].Error, "boom")
require.Equal(t, updater.StatusRefused, results[2].Status)
require.Contains(t, results[2].Error, "outside root")
}
func TestQueue_Timeout(t *testing.T) {
exec := &fakeExec{delay: 200 * time.Millisecond}
q := updater.NewQueue(exec)
q.Start(context.Background())
defer q.Stop()
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Millisecond)
defer cancel()
jobs := []discovery.Job{{Project: "slow", Service: "svc"}}
results := q.Submit(ctx, jobs)
require.Len(t, results, 1)
require.Equal(t, updater.StatusTimeout, results[0].Status)
}
```
- [ ] **Step 2: Write the failing worker test**
Write `internal/updater/worker_test.go`:
```go
package updater_test
import (
"context"
"sync/atomic"
"testing"
"time"
"github.com/shcizo/package-updater/internal/discovery"
"github.com/shcizo/package-updater/internal/updater"
"github.com/stretchr/testify/require"
)
type counterExec struct{ n atomic.Int32 }
func (c *counterExec) Execute(_ context.Context, _ discovery.Job) error {
c.n.Add(1)
time.Sleep(10 * time.Millisecond)
return nil
}
func TestWorker_RunsExactlyOneAtATime(t *testing.T) {
exec := &counterExec{}
q := updater.NewQueue(exec)
q.Start(context.Background())
defer q.Stop()
jobs := make([]discovery.Job, 10)
for i := range jobs {
jobs[i] = discovery.Job{Project: "p", Service: "svc"}
}
start := time.Now()
q.Submit(context.Background(), jobs)
elapsed := time.Since(start)
// 10 jobs × 10ms each, single worker → at least 100ms total.
require.GreaterOrEqual(t, elapsed, 90*time.Millisecond)
require.Equal(t, int32(10), exec.n.Load())
}
```
Note: this test relies on Queue deduping at submit-time being OFF (dedup happens in `discovery.FindJobs`, not in the queue). The queue treats each Job as a unique unit of work.
- [ ] **Step 3: Run tests — verify they fail**
Run:
```bash
go test ./internal/updater/...
```
Expected: build failure (`undefined: updater.NewQueue`).
- [ ] **Step 4: Implement `queue.go`**
Write `internal/updater/queue.go`:
```go
package updater
import (
"context"
"time"
"github.com/shcizo/package-updater/internal/discovery"
)
// Status describes the outcome of executing a single Job.
type Status string
const (
StatusUpdated Status = "updated"
StatusFailed Status = "failed"
StatusRefused Status = "refused"
StatusTimeout Status = "timeout"
)
// Result captures a per-Job outcome to be surfaced in the HTTP response.
type Result struct {
Job discovery.Job
Status Status
Error string
DurationMs int64
}
// Queue serialises Job execution through a single worker so we never
// run two `docker compose` commands against the same stack concurrently.
// It also satisfies the "single global FIFO worker" design choice in
// spec section 5.7.
type Queue struct {
exec Executor
ch chan submission
stop chan struct{}
done chan struct{}
}
type submission struct {
ctx context.Context
jobs []discovery.Job
results chan []Result
}
// NewQueue constructs a queue bound to the given executor.
func NewQueue(exec Executor) *Queue {
return &Queue{
exec: exec,
ch: make(chan submission, 16),
stop: make(chan struct{}),
done: make(chan struct{}),
}
}
// Start launches the single background worker. Call Stop to terminate.
func (q *Queue) Start(ctx context.Context) {
go q.run(ctx)
}
// Stop signals the worker to exit and waits for it to finish.
func (q *Queue) Stop() {
close(q.stop)
<-q.done
}
// Submit blocks until all jobs in the batch have been processed by the
// worker, then returns per-job results in the same order. The ctx
// timeout (if any) applies to each individual Job's execution and is
// surfaced as StatusTimeout.
func (q *Queue) Submit(ctx context.Context, jobs []discovery.Job) []Result {
resCh := make(chan []Result, 1)
q.ch <- submission{ctx: ctx, jobs: jobs, results: resCh}
return <-resCh
}
func (q *Queue) run(_ context.Context) {
defer close(q.done)
for {
select {
case <-q.stop:
return
case s := <-q.ch:
results := make([]Result, len(s.jobs))
for i, j := range s.jobs {
results[i] = q.runOne(s.ctx, j)
}
s.results <- results
}
}
}
func (q *Queue) runOne(ctx context.Context, job discovery.Job) Result {
start := time.Now()
r := Result{Job: job}
if job.Refused {
r.Status = StatusRefused
r.Error = job.RefusedReason
r.DurationMs = time.Since(start).Milliseconds()
return r
}
err := q.exec.Execute(ctx, job)
r.DurationMs = time.Since(start).Milliseconds()
switch {
case err == nil:
r.Status = StatusUpdated
case errorsIsContextDeadline(err) || ctxDeadlineExceeded(ctx):
r.Status = StatusTimeout
r.Error = err.Error()
default:
r.Status = StatusFailed
r.Error = err.Error()
}
return r
}
func errorsIsContextDeadline(err error) bool {
return err == context.DeadlineExceeded
}
func ctxDeadlineExceeded(ctx context.Context) bool {
return ctx.Err() == context.DeadlineExceeded
}
```
- [ ] **Step 5: Implement `worker.go`**
The worker is implemented inside `queue.go` (the `run` method). `worker.go` exists only to host any future expansion (e.g. per-stack mutex for parallel workers — out of scope for v1). Create an empty placeholder so the file structure matches the plan:
Write `internal/updater/worker.go`:
```go
package updater
// Worker logic currently lives in queue.go (single in-process worker
// goroutine). This file is reserved for the eventual per-stack-mutex
// implementation called out in spec section 15 ("Multi-worker concurrency
// with per-stack mutex"). Intentionally empty for v1.
```
- [ ] **Step 6: Run tests — verify all pass**
Run:
```bash
go test ./internal/updater/... -v
```
Expected: all 4 tests PASS.
- [ ] **Step 7: Commit**
```bash
git add internal/updater/
git commit -m "feat(updater): FIFO queue with single worker and per-job results"
```
---
## Task 11: HTTP middleware (auth + request_id + logging)
**Files:**
- Create: `internal/api/middleware.go`
- Create: `internal/api/middleware_test.go`
- [ ] **Step 1: Add UUID dependency**
Run:
```bash
go get github.com/google/uuid
```
- [ ] **Step 2: Write the failing test**
Write `internal/api/middleware_test.go`:
```go
package api_test
import (
"io"
"log/slog"
"net/http"
"net/http/httptest"
"testing"
"github.com/shcizo/package-updater/internal/api"
"github.com/shcizo/package-updater/internal/logging"
"github.com/stretchr/testify/require"
)
func newTestLogger() *slog.Logger {
return slog.New(slog.NewJSONHandler(io.Discard, nil))
}
func TestAuth_AllowsMatchingToken(t *testing.T) {
called := false
h := api.Auth("secret")(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
called = true
w.WriteHeader(http.StatusOK)
}))
req := httptest.NewRequest(http.MethodPost, "/update", nil)
req.Header.Set("Authorization", "Bearer secret")
w := httptest.NewRecorder()
h.ServeHTTP(w, req)
require.True(t, called)
require.Equal(t, http.StatusOK, w.Code)
}
func TestAuth_Rejects(t *testing.T) {
cases := []struct{ name, header string }{
{"missing", ""},
{"wrong scheme", "Token secret"},
{"wrong value", "Bearer nope"},
{"empty bearer", "Bearer "},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
h := api.Auth("secret")(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
t.Fatal("handler must not be called")
}))
req := httptest.NewRequest(http.MethodPost, "/update", nil)
if c.header != "" {
req.Header.Set("Authorization", c.header)
}
w := httptest.NewRecorder()
h.ServeHTTP(w, req)
require.Equal(t, http.StatusUnauthorized, w.Code)
})
}
}
func TestRequestID_GeneratesIfMissing(t *testing.T) {
var seenID string
h := api.RequestID(http.HandlerFunc(func(_ http.ResponseWriter, r *http.Request) {
seenID = logging.RequestIDFrom(r.Context())
}))
req := httptest.NewRequest(http.MethodPost, "/update", nil)
w := httptest.NewRecorder()
h.ServeHTTP(w, req)
require.NotEmpty(t, seenID)
require.Equal(t, seenID, w.Header().Get("X-Request-ID"))
}
func TestRequestID_UsesIncoming(t *testing.T) {
var seenID string
h := api.RequestID(http.HandlerFunc(func(_ http.ResponseWriter, r *http.Request) {
seenID = logging.RequestIDFrom(r.Context())
}))
req := httptest.NewRequest(http.MethodPost, "/update", nil)
req.Header.Set("X-Request-ID", "given-id")
w := httptest.NewRecorder()
h.ServeHTTP(w, req)
require.Equal(t, "given-id", seenID)
}
func TestRequestLogger_LogsAndDelegates(t *testing.T) {
logger := newTestLogger()
called := false
h := api.RequestLogger(logger)(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
called = true
w.WriteHeader(http.StatusTeapot)
}))
req := httptest.NewRequest(http.MethodPost, "/update", nil)
w := httptest.NewRecorder()
h.ServeHTTP(w, req)
require.True(t, called)
require.Equal(t, http.StatusTeapot, w.Code)
}
```
- [ ] **Step 3: Run tests — verify they fail**
Run:
```bash
go test ./internal/api/...
```
Expected: build failure (`undefined: api.Auth` etc).
- [ ] **Step 4: Implement `middleware.go`**
Write `internal/api/middleware.go`:
```go
// Package api contains HTTP handlers, middleware, and request/response DTOs.
package api
import (
"crypto/subtle"
"log/slog"
"net/http"
"strings"
"time"
"github.com/google/uuid"
"github.com/shcizo/package-updater/internal/logging"
)
// Auth returns middleware that requires a matching bearer token.
// Compares with constant-time to defeat timing attacks.
func Auth(token string) func(http.Handler) http.Handler {
tokenBytes := []byte(token)
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
h := r.Header.Get("Authorization")
const prefix = "Bearer "
if !strings.HasPrefix(h, prefix) {
writeAuthError(w)
return
}
provided := []byte(strings.TrimPrefix(h, prefix))
if len(provided) == 0 ||
subtle.ConstantTimeCompare(provided, tokenBytes) != 1 {
writeAuthError(w)
return
}
next.ServeHTTP(w, r)
})
}
}
func writeAuthError(w http.ResponseWriter) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusUnauthorized)
_, _ = w.Write([]byte(`{"error":"unauthorized"}`))
}
// RequestID middleware ensures every request has an X-Request-ID
// header (generated if absent) and stores it in the request context.
func RequestID(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
id := r.Header.Get("X-Request-ID")
if id == "" {
id = uuid.NewString()
}
w.Header().Set("X-Request-ID", id)
ctx := logging.WithRequestID(r.Context(), id)
next.ServeHTTP(w, r.WithContext(ctx))
})
}
// RequestLogger emits a structured access-log line per request.
func RequestLogger(base *slog.Logger) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
sw := &statusWriter{ResponseWriter: w, status: http.StatusOK}
next.ServeHTTP(sw, r)
logger := logging.FromContext(r.Context(), base)
logger.Info("http_request",
"method", r.Method,
"path", r.URL.Path,
"status", sw.status,
"duration_ms", time.Since(start).Milliseconds(),
"client_ip", clientIP(r),
)
})
}
}
type statusWriter struct {
http.ResponseWriter
status int
}
func (s *statusWriter) WriteHeader(code int) {
s.status = code
s.ResponseWriter.WriteHeader(code)
}
func clientIP(r *http.Request) string {
if xff := r.Header.Get("X-Forwarded-For"); xff != "" {
if comma := strings.Index(xff, ","); comma >= 0 {
return strings.TrimSpace(xff[:comma])
}
return xff
}
return r.RemoteAddr
}
```
- [ ] **Step 5: Run tests — verify all pass**
Run:
```bash
go test ./internal/api/... -v
```
Expected: all middleware tests PASS.
- [ ] **Step 6: Commit**
```bash
git add go.mod go.sum internal/api/middleware.go internal/api/middleware_test.go
git commit -m "feat(api): auth, request_id, and request-logging middleware"
```
---
## Task 12: HTTP handlers (/update, /healthz, /version)
**Files:**
- Create: `internal/api/types.go`
- Create: `internal/api/handlers.go`
- Create: `internal/api/handlers_test.go`
- [ ] **Step 1: Define DTOs**
Write `internal/api/types.go`:
```go
package api
// UpdateRequest is the body of POST /update.
type UpdateRequest struct {
Image string `json:"image"`
Tag string `json:"tag,omitempty"`
}
// UpdateResponse is the body of POST /update.
type UpdateResponse struct {
RequestID string `json:"request_id"`
Image string `json:"image"`
Tag string `json:"tag,omitempty"`
Matched int `json:"matched"`
Results []ResultDTO `json:"results"`
}
// ResultDTO is one row in UpdateResponse.Results.
type ResultDTO struct {
Project string `json:"project"`
Service string `json:"service"`
ComposeFile string `json:"compose_file"`
Status string `json:"status"`
Error string `json:"error,omitempty"`
DurationMs int64 `json:"duration_ms"`
}
// HealthResponse is the body of GET /healthz.
type HealthResponse struct {
Status string `json:"status"`
Docker string `json:"docker"`
}
// VersionResponse is the body of GET /version.
type VersionResponse struct {
Version string `json:"version"`
Commit string `json:"commit"`
BuildTime string `json:"build_time"`
}
```
- [ ] **Step 2: Write the failing handler tests**
Write `internal/api/handlers_test.go`:
```go
package api_test
import (
"bytes"
"context"
"encoding/json"
"errors"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/docker/docker/api/types"
"github.com/shcizo/package-updater/internal/api"
"github.com/shcizo/package-updater/internal/discovery"
"github.com/shcizo/package-updater/internal/updater"
"github.com/stretchr/testify/require"
)
type fakeFinder struct {
jobs []discovery.Job
err error
}
func (f *fakeFinder) FindJobs(_ context.Context, _ string) ([]discovery.Job, error) {
return f.jobs, f.err
}
type fakeSubmitter struct {
results []updater.Result
}
func (f *fakeSubmitter) Submit(_ context.Context, jobs []discovery.Job) []updater.Result {
if f.results != nil {
return f.results
}
out := make([]updater.Result, len(jobs))
for i, j := range jobs {
out[i] = updater.Result{Job: j, Status: updater.StatusUpdated}
}
return out
}
type fakePinger struct{ err error }
func (f *fakePinger) Ping(_ context.Context) (types.Ping, error) {
return types.Ping{}, f.err
}
func decode[T any](t *testing.T, body io.Reader) T {
t.Helper()
var v T
require.NoError(t, json.NewDecoder(body).Decode(&v))
return v
}
func TestUpdate_ValidationError(t *testing.T) {
h := api.NewHandlers(&fakeFinder{}, &fakeSubmitter{}, &fakePinger{}, "v0.0.0", "abc", "now")
req := httptest.NewRequest(http.MethodPost, "/update",
strings.NewReader(`{"tag":"v1.2.3"}`))
w := httptest.NewRecorder()
h.Update(w, req)
require.Equal(t, http.StatusBadRequest, w.Code)
}
func TestUpdate_BadJSON(t *testing.T) {
h := api.NewHandlers(&fakeFinder{}, &fakeSubmitter{}, &fakePinger{}, "v0.0.0", "abc", "now")
req := httptest.NewRequest(http.MethodPost, "/update", strings.NewReader(`not json`))
w := httptest.NewRecorder()
h.Update(w, req)
require.Equal(t, http.StatusBadRequest, w.Code)
}
func TestUpdate_DiscoveryFailureReturns500(t *testing.T) {
finder := &fakeFinder{err: errors.New("daemon unreachable")}
h := api.NewHandlers(finder, &fakeSubmitter{}, &fakePinger{}, "v0.0.0", "abc", "now")
body, _ := json.Marshal(api.UpdateRequest{Image: "r/x"})
req := httptest.NewRequest(http.MethodPost, "/update", bytes.NewReader(body))
w := httptest.NewRecorder()
h.Update(w, req)
require.Equal(t, http.StatusInternalServerError, w.Code)
}
func TestUpdate_ZeroMatchesReturns200(t *testing.T) {
h := api.NewHandlers(&fakeFinder{}, &fakeSubmitter{}, &fakePinger{}, "v0.0.0", "abc", "now")
body, _ := json.Marshal(api.UpdateRequest{Image: "r/x"})
req := httptest.NewRequest(http.MethodPost, "/update", bytes.NewReader(body))
w := httptest.NewRecorder()
h.Update(w, req)
require.Equal(t, http.StatusOK, w.Code)
resp := decode[api.UpdateResponse](t, w.Body)
require.Equal(t, 0, resp.Matched)
}
func TestUpdate_AllSucceeded200(t *testing.T) {
finder := &fakeFinder{jobs: []discovery.Job{
{Project: "p", Service: "s", WorkingDir: "/x", ConfigFiles: []string{"/x/c.yml"}},
}}
h := api.NewHandlers(finder, &fakeSubmitter{}, &fakePinger{}, "v0.0.0", "abc", "now")
body, _ := json.Marshal(api.UpdateRequest{Image: "r/x", Tag: "v1"})
req := httptest.NewRequest(http.MethodPost, "/update", bytes.NewReader(body))
w := httptest.NewRecorder()
h.Update(w, req)
require.Equal(t, http.StatusOK, w.Code)
resp := decode[api.UpdateResponse](t, w.Body)
require.Equal(t, 1, resp.Matched)
require.Equal(t, "updated", resp.Results[0].Status)
require.Equal(t, "/x/c.yml", resp.Results[0].ComposeFile)
}
func TestUpdate_MixedReturns207(t *testing.T) {
jobs := []discovery.Job{
{Project: "p1", Service: "s", ConfigFiles: []string{"/x/c.yml"}},
{Project: "p2", Service: "s", ConfigFiles: []string{"/y/c.yml"}},
}
finder := &fakeFinder{jobs: jobs}
submitter := &fakeSubmitter{results: []updater.Result{
{Job: jobs[0], Status: updater.StatusUpdated},
{Job: jobs[1], Status: updater.StatusFailed, Error: "boom"},
}}
h := api.NewHandlers(finder, submitter, &fakePinger{}, "v0.0.0", "abc", "now")
body, _ := json.Marshal(api.UpdateRequest{Image: "r/x"})
req := httptest.NewRequest(http.MethodPost, "/update", bytes.NewReader(body))
w := httptest.NewRecorder()
h.Update(w, req)
require.Equal(t, http.StatusMultiStatus, w.Code)
}
func TestUpdate_AllFailedReturns500(t *testing.T) {
jobs := []discovery.Job{{Project: "p", Service: "s", ConfigFiles: []string{"/x/c.yml"}}}
finder := &fakeFinder{jobs: jobs}
submitter := &fakeSubmitter{results: []updater.Result{
{Job: jobs[0], Status: updater.StatusFailed, Error: "boom"},
}}
h := api.NewHandlers(finder, submitter, &fakePinger{}, "v0.0.0", "abc", "now")
body, _ := json.Marshal(api.UpdateRequest{Image: "r/x"})
req := httptest.NewRequest(http.MethodPost, "/update", bytes.NewReader(body))
w := httptest.NewRecorder()
h.Update(w, req)
require.Equal(t, http.StatusInternalServerError, w.Code)
}
func TestHealthz_OKWhenDockerUp(t *testing.T) {
h := api.NewHandlers(&fakeFinder{}, &fakeSubmitter{}, &fakePinger{}, "v0.0.0", "abc", "now")
req := httptest.NewRequest(http.MethodGet, "/healthz", nil)
w := httptest.NewRecorder()
h.Healthz(w, req)
require.Equal(t, http.StatusOK, w.Code)
}
func TestHealthz_503WhenDockerDown(t *testing.T) {
h := api.NewHandlers(&fakeFinder{}, &fakeSubmitter{}, &fakePinger{err: errors.New("ping fail")}, "v0.0.0", "abc", "now")
req := httptest.NewRequest(http.MethodGet, "/healthz", nil)
w := httptest.NewRecorder()
h.Healthz(w, req)
require.Equal(t, http.StatusServiceUnavailable, w.Code)
}
func TestVersion(t *testing.T) {
h := api.NewHandlers(&fakeFinder{}, &fakeSubmitter{}, &fakePinger{}, "v1.2.3", "abcdef", "2026-05-22T00:00:00Z")
req := httptest.NewRequest(http.MethodGet, "/version", nil)
w := httptest.NewRecorder()
h.Version(w, req)
require.Equal(t, http.StatusOK, w.Code)
resp := decode[api.VersionResponse](t, w.Body)
require.Equal(t, "v1.2.3", resp.Version)
}
```
- [ ] **Step 3: Run tests — verify they fail**
Run:
```bash
go test ./internal/api/...
```
Expected: build failure (`undefined: api.NewHandlers`).
- [ ] **Step 4: Implement `handlers.go`**
Write `internal/api/handlers.go`:
```go
package api
import (
"context"
"encoding/json"
"errors"
"net/http"
"github.com/docker/docker/api/types"
"github.com/shcizo/package-updater/internal/discovery"
"github.com/shcizo/package-updater/internal/logging"
"github.com/shcizo/package-updater/internal/updater"
)
// Finder is the discovery interface used by the update handler.
type Finder interface {
FindJobs(ctx context.Context, image string) ([]discovery.Job, error)
}
// Submitter is the queue interface used by the update handler.
type Submitter interface {
Submit(ctx context.Context, jobs []discovery.Job) []updater.Result
}
// Pinger pings the Docker daemon for the healthcheck.
type Pinger interface {
Ping(ctx context.Context) (types.Ping, error)
}
// Handlers wires the HTTP endpoints to the rest of the service.
type Handlers struct {
finder Finder
submitter Submitter
pinger Pinger
version string
commit string
buildTime string
}
// NewHandlers constructs a Handlers value with all dependencies injected.
func NewHandlers(f Finder, s Submitter, p Pinger, version, commit, buildTime string) *Handlers {
return &Handlers{finder: f, submitter: s, pinger: p, version: version, commit: commit, buildTime: buildTime}
}
// Update implements POST /update.
func (h *Handlers) Update(w http.ResponseWriter, r *http.Request) {
var req UpdateRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeJSONError(w, http.StatusBadRequest, "invalid JSON body")
return
}
if req.Image == "" {
writeJSONError(w, http.StatusBadRequest, "image is required")
return
}
jobs, err := h.finder.FindJobs(r.Context(), req.Image)
if err != nil {
writeJSONError(w, http.StatusInternalServerError, "discovery failed: "+err.Error())
return
}
resp := UpdateResponse{
RequestID: logging.RequestIDFrom(r.Context()),
Image: req.Image,
Tag: req.Tag,
Matched: len(jobs),
Results: []ResultDTO{},
}
if len(jobs) == 0 {
writeJSON(w, http.StatusOK, resp)
return
}
results := h.submitter.Submit(r.Context(), jobs)
updated, failed := 0, 0
for _, res := range results {
composeFile := ""
if len(res.Job.ConfigFiles) > 0 {
composeFile = res.Job.ConfigFiles[0]
}
resp.Results = append(resp.Results, ResultDTO{
Project: res.Job.Project,
Service: res.Job.Service,
ComposeFile: composeFile,
Status: string(res.Status),
Error: res.Error,
DurationMs: res.DurationMs,
})
if res.Status == updater.StatusUpdated {
updated++
} else {
failed++
}
}
switch {
case failed == 0:
writeJSON(w, http.StatusOK, resp)
case updated == 0:
writeJSON(w, http.StatusInternalServerError, resp)
default:
writeJSON(w, http.StatusMultiStatus, resp)
}
}
// Healthz implements GET /healthz.
func (h *Handlers) Healthz(w http.ResponseWriter, r *http.Request) {
if _, err := h.pinger.Ping(r.Context()); err != nil {
writeJSON(w, http.StatusServiceUnavailable, HealthResponse{Status: "unhealthy", Docker: "unreachable"})
return
}
writeJSON(w, http.StatusOK, HealthResponse{Status: "ok", Docker: "ok"})
}
// Version implements GET /version.
func (h *Handlers) Version(w http.ResponseWriter, _ *http.Request) {
writeJSON(w, http.StatusOK, VersionResponse{
Version: h.version,
Commit: h.commit,
BuildTime: h.buildTime,
})
}
func writeJSON(w http.ResponseWriter, code int, body any) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(code)
_ = json.NewEncoder(w).Encode(body)
}
func writeJSONError(w http.ResponseWriter, code int, msg string) {
writeJSON(w, code, map[string]string{"error": msg})
}
// errMissingImage is referenced in tests indirectly via the error string.
var errMissingImage = errors.New("image is required")
```
- [ ] **Step 5: Run tests — verify all pass**
Run:
```bash
go test ./internal/api/... -v
```
Expected: all handler + middleware tests PASS.
- [ ] **Step 6: Commit**
```bash
git add internal/api/
git commit -m "feat(api): /update, /healthz, /version handlers"
```
---
## Task 13: Prometheus metrics
**Files:**
- Create: `internal/metrics/metrics.go`
- [ ] **Step 1: Add Prometheus dependency**
Run:
```bash
go get github.com/prometheus/client_golang/prometheus github.com/prometheus/client_golang/prometheus/promhttp
```
- [ ] **Step 2: Implement metrics**
Write `internal/metrics/metrics.go`:
```go
// Package metrics defines and registers the Prometheus collectors
// exported on /metrics.
package metrics
import (
"net/http"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
)
// Metrics bundles all collectors. Each field is the public handle the
// rest of the service uses to record observations.
type Metrics struct {
BuildInfo *prometheus.GaugeVec
HTTPRequests *prometheus.CounterVec
UpdateJobs *prometheus.CounterVec
UpdateDuration *prometheus.HistogramVec
QueueDepth prometheus.Gauge
LastUpdateTime *prometheus.GaugeVec
DockerPingUp prometheus.Gauge
}
// New constructs Metrics and registers them with the given registry.
// Use prometheus.NewRegistry() in tests so collectors don't leak between
// runs; production code uses prometheus.DefaultRegisterer.
func New(reg prometheus.Registerer) *Metrics {
m := &Metrics{
BuildInfo: prometheus.NewGaugeVec(prometheus.GaugeOpts{
Name: "package_updater_build_info",
Help: "Always 1. Labels carry version/commit for dashboards.",
}, []string{"version", "commit"}),
HTTPRequests: prometheus.NewCounterVec(prometheus.CounterOpts{
Name: "package_updater_http_requests_total",
Help: "HTTP requests handled, labelled by endpoint and status.",
}, []string{"endpoint", "status_code"}),
UpdateJobs: prometheus.NewCounterVec(prometheus.CounterOpts{
Name: "package_updater_update_jobs_total",
Help: "Update jobs executed, labelled by project/service/status.",
}, []string{"project", "service", "status"}),
UpdateDuration: prometheus.NewHistogramVec(prometheus.HistogramOpts{
Name: "package_updater_update_duration_seconds",
Help: "Time taken to pull + up a single service.",
Buckets: []float64{0.5, 1, 2, 5, 10, 30, 60, 120, 300},
}, []string{"project", "service"}),
QueueDepth: prometheus.NewGauge(prometheus.GaugeOpts{
Name: "package_updater_queue_depth",
Help: "Current number of submissions waiting in the queue.",
}),
LastUpdateTime: prometheus.NewGaugeVec(prometheus.GaugeOpts{
Name: "package_updater_last_update_timestamp",
Help: "Unix timestamp of the most recent successful update per service.",
}, []string{"project", "service"}),
DockerPingUp: prometheus.NewGauge(prometheus.GaugeOpts{
Name: "package_updater_docker_ping_up",
Help: "1 if the Docker socket responded to ping, 0 otherwise.",
}),
}
reg.MustRegister(
m.BuildInfo, m.HTTPRequests, m.UpdateJobs, m.UpdateDuration,
m.QueueDepth, m.LastUpdateTime, m.DockerPingUp,
)
return m
}
// Handler returns the /metrics HTTP handler bound to the given registry.
func Handler(gatherer prometheus.Gatherer) http.Handler {
return promhttp.HandlerFor(gatherer, promhttp.HandlerOpts{})
}
```
- [ ] **Step 3: Verify it builds**
Run:
```bash
go build ./internal/metrics/...
```
Expected: no output.
- [ ] **Step 4: Commit**
```bash
git add go.mod go.sum internal/metrics/
git commit -m "feat(metrics): Prometheus collectors and /metrics handler"
```
---
## Task 14: Self-update deferred execution
**Files:**
- Create: `internal/selfupdate/selfupdate.go`
- Create: `internal/selfupdate/selfupdate_test.go`
- [ ] **Step 1: Write the failing test**
Write `internal/selfupdate/selfupdate_test.go`:
```go
package selfupdate_test
import (
"context"
"errors"
"sync"
"testing"
"time"
"github.com/shcizo/package-updater/internal/discovery"
"github.com/shcizo/package-updater/internal/selfupdate"
"github.com/stretchr/testify/require"
)
type recExec struct {
mu sync.Mutex
called bool
at time.Time
}
func (r *recExec) Execute(_ context.Context, _ discovery.Job) error {
r.mu.Lock()
r.called = true
r.at = time.Now()
r.mu.Unlock()
return nil
}
func TestIsSelf(t *testing.T) {
require.True(t, selfupdate.IsSelf(
discovery.Job{Project: "package-updater", Service: "package-updater"},
"package-updater",
))
require.False(t, selfupdate.IsSelf(
discovery.Job{Project: "other", Service: "web"},
"package-updater",
))
}
func TestWrap_DefersSelf(t *testing.T) {
inner := &recExec{}
wrapped := selfupdate.Wrap(inner, "package-updater", 30*time.Millisecond)
flushed := make(chan time.Time, 1)
flush := func() { flushed <- time.Now() }
job := discovery.Job{Project: "package-updater", Service: "package-updater"}
require.NoError(t, wrapped.ExecuteWithFlush(context.Background(), job, flush))
flushAt := <-flushed
inner.mu.Lock()
require.True(t, inner.called)
require.True(t, inner.at.After(flushAt))
inner.mu.Unlock()
}
func TestWrap_NormalJobIsImmediate(t *testing.T) {
inner := &recExec{}
wrapped := selfupdate.Wrap(inner, "package-updater", 30*time.Millisecond)
flushed := make(chan time.Time, 1)
flush := func() { flushed <- time.Now() }
job := discovery.Job{Project: "other", Service: "web"}
require.NoError(t, wrapped.ExecuteWithFlush(context.Background(), job, flush))
// For non-self jobs, exec happens BEFORE flush is invoked at all
// (caller decides when to flush).
inner.mu.Lock()
require.True(t, inner.called)
inner.mu.Unlock()
select {
case <-flushed:
t.Fatal("flush should not be called for non-self jobs")
default:
}
}
func TestWrap_PropagatesError(t *testing.T) {
wrapped := selfupdate.Wrap(failExec{}, "x", 1*time.Millisecond)
err := wrapped.ExecuteWithFlush(context.Background(), discovery.Job{Project: "other"}, func() {})
require.Error(t, err)
}
type failExec struct{}
func (failExec) Execute(_ context.Context, _ discovery.Job) error {
return errors.New("boom")
}
```
- [ ] **Step 2: Run tests — verify they fail**
Run:
```bash
go test ./internal/selfupdate/...
```
Expected: build failure.
- [ ] **Step 3: Implement `selfupdate.go`**
Write `internal/selfupdate/selfupdate.go`:
```go
// Package selfupdate handles the special case where the update target
// is the running service's own container. We must finish writing the
// HTTP response (and flush + close the connection) before exec'ing
// `docker compose up -d` against ourselves, otherwise the response is
// lost when the container is replaced.
package selfupdate
import (
"context"
"time"
"github.com/shcizo/package-updater/internal/discovery"
)
// IsSelf reports whether job targets the running service.
// Matches on Compose project name (which is also typically the
// service name for single-service stacks).
func IsSelf(job discovery.Job, selfProject string) bool {
return job.Project == selfProject
}
// innerExec is the executor abstraction we wrap.
type innerExec interface {
Execute(ctx context.Context, job discovery.Job) error
}
// Wrapped wraps an Executor with self-update-aware deferred execution.
type Wrapped struct {
inner innerExec
selfProject string
delay time.Duration
}
// Wrap returns a Wrapped that defers exec until after flush() for
// self-updates. The delay is added after flush before exec, so the
// kernel TCP buffer has time to drain.
func Wrap(inner innerExec, selfProject string, delay time.Duration) *Wrapped {
return &Wrapped{inner: inner, selfProject: selfProject, delay: delay}
}
// ExecuteWithFlush runs job, invoking flush() before exec for self-updates
// and waiting `delay` after flush. For non-self jobs, exec happens
// normally and flush is not invoked at all (the HTTP layer decides
// when to flush in that case).
func (w *Wrapped) ExecuteWithFlush(ctx context.Context, job discovery.Job, flush func()) error {
if !IsSelf(job, w.selfProject) {
return w.inner.Execute(ctx, job)
}
flush()
select {
case <-time.After(w.delay):
case <-ctx.Done():
return ctx.Err()
}
return w.inner.Execute(ctx, job)
}
```
- [ ] **Step 4: Run tests — verify all pass**
Run:
```bash
go test ./internal/selfupdate/... -v
```
Expected: all 4 tests PASS.
- [ ] **Step 5: Commit**
```bash
git add internal/selfupdate/
git commit -m "feat(selfupdate): defer self-replacement until response flushed"
```
Note: integrating the Wrapped executor into the queue/handler is intentionally out of scope for this task — we'd need to thread a flush callback through the queue's Submit signature. For v1 we accept that the self-update flow is partially manual: the operator may need to manually restart `package-updater` after pushing a new image to itself. If/when this becomes annoying, wire `Wrapped` into the queue and pass `http.Flusher` from the handler.
---
## Task 15: Wire it all in main.go
**Files:**
- Modify: `cmd/server/main.go`
- [ ] **Step 1: Replace the stub `main.go` with the real wire-up**
Write `cmd/server/main.go`:
```go
package main
import (
"context"
"fmt"
"net/http"
"os"
"os/signal"
"syscall"
"time"
"github.com/docker/docker/client"
"github.com/prometheus/client_golang/prometheus"
"github.com/shcizo/package-updater/internal/api"
"github.com/shcizo/package-updater/internal/config"
"github.com/shcizo/package-updater/internal/discovery"
"github.com/shcizo/package-updater/internal/logging"
"github.com/shcizo/package-updater/internal/metrics"
"github.com/shcizo/package-updater/internal/updater"
)
// Build info is injected at link time via -ldflags="-X main.version=..."
var (
version = "dev"
commit = "unknown"
buildTime = "unknown"
)
func main() {
if err := run(); err != nil {
fmt.Fprintln(os.Stderr, "fatal:", err)
os.Exit(1)
}
}
func run() error {
cfg, err := config.Load()
if err != nil {
return err
}
logger := logging.New(cfg.LogLevel)
logger.Info("starting",
"version", version, "commit", commit, "port", cfg.Port,
"stacks_root", cfg.StacksRoot, "opt_in_label", cfg.OptInLabel,
)
dockerCli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
if err != nil {
return fmt.Errorf("docker client: %w", err)
}
defer dockerCli.Close()
disc := discovery.New(dockerCli, cfg.StacksRoot, cfg.OptInLabel)
exec := updater.NewComposeExecutor()
queue := updater.NewQueue(exec)
queue.Start(context.Background())
defer queue.Stop()
reg := prometheus.NewRegistry()
m := metrics.New(reg)
m.BuildInfo.WithLabelValues(version, commit).Set(1)
handlers := api.NewHandlers(disc, &submitterAdapter{queue: queue, timeout: cfg.UpdateTimeout}, dockerCli, version, commit, buildTime)
mux := http.NewServeMux()
mux.HandleFunc("POST /update", handlers.Update)
mux.HandleFunc("GET /healthz", handlers.Healthz)
mux.HandleFunc("GET /version", handlers.Version)
mux.Handle("GET /metrics", metrics.Handler(reg))
authed := api.Auth(cfg.APIKey)
handler := api.RequestID(api.RequestLogger(logger)(routeAuth(mux, authed)))
srv := &http.Server{
Addr: ":" + cfg.Port,
Handler: handler,
ReadHeaderTimeout: 10 * time.Second,
}
errCh := make(chan error, 1)
go func() {
logger.Info("listening", "addr", srv.Addr)
errCh <- srv.ListenAndServe()
}()
sig := make(chan os.Signal, 1)
signal.Notify(sig, syscall.SIGINT, syscall.SIGTERM)
select {
case s := <-sig:
logger.Info("shutdown_signal", "signal", s.String())
case err := <-errCh:
if err != nil && err != http.ErrServerClosed {
return err
}
}
shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
_ = srv.Shutdown(shutdownCtx)
return nil
}
// routeAuth applies the bearer-token middleware to /update only.
// /healthz, /metrics, /version are unauthenticated by design (internal
// network only; healthcheck and Prometheus scraper need to reach them
// without secrets).
func routeAuth(next *http.ServeMux, mw func(http.Handler) http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/update" {
mw(next).ServeHTTP(w, r)
return
}
next.ServeHTTP(w, r)
})
}
// submitterAdapter wraps Queue.Submit to inject the per-request timeout
// from config, satisfying the api.Submitter interface.
type submitterAdapter struct {
queue *updater.Queue
timeout time.Duration
}
func (s *submitterAdapter) Submit(ctx context.Context, jobs []discovery.Job) []updater.Result {
ctx, cancel := context.WithTimeout(ctx, s.timeout)
defer cancel()
return s.queue.Submit(ctx, jobs)
}
```
- [ ] **Step 2: Verify it builds**
Run:
```bash
go build ./cmd/server
```
Expected: no output. A binary called `server` is produced. Delete it: `rm server`.
- [ ] **Step 3: Tidy the module**
Run:
```bash
go mod tidy
```
Expected: `go.sum` updated; no errors.
- [ ] **Step 4: Run the full test suite**
Run:
```bash
go test ./...
```
Expected: all tests across all packages PASS.
- [ ] **Step 5: Commit**
```bash
git add cmd/server/main.go go.mod go.sum
git commit -m "feat(cmd): wire config, discovery, queue, http server, metrics"
```
---
## Task 16: Dockerfile
**Files:**
- Create: `Dockerfile`
- Create: `.dockerignore`
- [ ] **Step 1: Write the Dockerfile**
Write `Dockerfile`:
```dockerfile
# syntax=docker/dockerfile:1.7
FROM golang:1.23-alpine AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
ARG VERSION=dev
ARG COMMIT=unknown
ARG BUILD_TIME=unknown
RUN CGO_ENABLED=0 go build \
-ldflags="-s -w \
-X main.version=${VERSION} \
-X main.commit=${COMMIT} \
-X main.buildTime=${BUILD_TIME}" \
-o /out/package-updater ./cmd/server
FROM alpine:3.20
RUN apk add --no-cache docker-cli docker-cli-compose ca-certificates wget
COPY --from=build /out/package-updater /usr/local/bin/
EXPOSE 8080
ENTRYPOINT ["/usr/local/bin/package-updater"]
```
- [ ] **Step 2: Write `.dockerignore`**
Write `.dockerignore`:
```
.git
.gitignore
.github
docs/
README.md
*.md
.env
.env.*
gitea-action/
docker-compose.example.yml
```
- [ ] **Step 3: Build the image locally**
Run:
```bash
docker build -t package-updater:dev .
```
Expected: image builds, ending with `Successfully tagged package-updater:dev`. Image size should be around 50 MB (`docker images package-updater:dev`).
- [ ] **Step 4: Smoke-test the binary**
Run:
```bash
docker run --rm package-updater:dev /usr/local/bin/package-updater 2>&1 | head -5
```
Expected: prints `fatal: UPDATER_API_KEY is required` and exits non-zero. This proves the binary loads, config wiring works, and fail-fast triggers correctly.
- [ ] **Step 5: Commit**
```bash
git add Dockerfile .dockerignore
git commit -m "build: multi-stage Dockerfile with build info ldflags"
```
---
## Task 17: docker-compose.example.yml
**Files:**
- Create: `docker-compose.example.yml`
- [ ] **Step 1: Write the example compose file**
Write `docker-compose.example.yml`:
```yaml
# Example deployment of package-updater itself.
# Copy to /home/shcizo/self-hosted/package-updater/docker-compose.yml
# and create a .env file alongside containing:
# UPDATER_API_KEY=<openssl rand -hex 32>
services:
package-updater:
image: registry.example.com/package-updater:latest
container_name: package-updater
restart: unless-stopped
environment:
- UPDATER_API_KEY=${UPDATER_API_KEY}
- STACKS_ROOT=/home/shcizo/self-hosted
- LOG_LEVEL=info
- PORT=8080
- UPDATE_TIMEOUT=5m
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /home/shcizo/self-hosted:/home/shcizo/self-hosted:ro
labels:
- "se.shcizo.auto-update=true"
networks:
- proxy
healthcheck:
test: ["CMD", "wget", "-q", "-O-", "http://localhost:8080/healthz"]
interval: 30s
timeout: 5s
retries: 3
networks:
proxy:
external: true
```
- [ ] **Step 2: Commit**
```bash
git add docker-compose.example.yml
git commit -m "docs: docker-compose.example.yml for end-user deploy"
```
---
## Task 18: Gitea composite action
**Files:**
- Create: `gitea-action/action.yml`
- Create: `gitea-action/README.md`
- [ ] **Step 1: Write the action**
Write `gitea-action/action.yml`:
```yaml
name: "Deploy via package-updater"
description: "Notifies package-updater to pull & restart a Docker Compose service"
inputs:
endpoint:
description: "Full URL to /update (e.g. https://updater.example.com/update)"
required: true
image:
description: "Image reference without tag (e.g. registry.example.com/myapp)"
required: true
tag:
description: "Tag that was just pushed (for logging)"
required: false
default: ""
token:
description: "Bearer token for package-updater"
required: true
runs:
using: "composite"
steps:
- name: Trigger update
shell: bash
env:
TOKEN: ${{ inputs.token }}
run: |
set -euo pipefail
response=$(curl -sS -w "\n%{http_code}" \
-X POST "${{ inputs.endpoint }}" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d "{\"image\":\"${{ inputs.image }}\",\"tag\":\"${{ inputs.tag }}\"}")
body=$(echo "$response" | head -n -1)
code=$(echo "$response" | tail -n 1)
echo "HTTP $code"
echo "$body" | jq .
if [ "$code" -ge 400 ]; then
exit 1
fi
```
- [ ] **Step 2: Write the action README**
Write `gitea-action/README.md`:
```markdown
# Deploy via package-updater (composite action)
Notifies `package-updater` to `docker compose pull` + `up -d` for the matching service(s) after a CI build.
## Usage
In a consumer repo's `.gitea/workflows/deploy.yml`:
```yaml
jobs:
deploy:
runs-on: ubuntu-latest
needs: [build-and-push]
steps:
- uses: gitea.example.com/shcizo/package-updater/gitea-action@v1
with:
endpoint: https://updater.example.com/update
image: registry.example.com/${{ gitea.repository }}
tag: ${{ gitea.sha }}
token: ${{ secrets.UPDATER_TOKEN }}
```
`UPDATER_TOKEN` should be set as an organisation-level secret in Gitea so all repos share it.
## Inputs
| Name | Required | Default | Description |
|---|---|---|---|
| `endpoint` | yes | — | Full URL to `/update` |
| `image` | yes | — | Image reference without tag |
| `tag` | no | `""` | Tag that was just pushed (logged for audit) |
| `token` | yes | — | Bearer token configured in package-updater |
## Failure modes
The step exits non-zero if `package-updater` returns HTTP 4xx or 5xx. This is intentional — the workflow surfaces the deploy failure to whoever pushed.
```
- [ ] **Step 3: Commit**
```bash
git add gitea-action/
git commit -m "feat(action): reusable Gitea composite action for /update"
```
---
## Task 19: README
**Files:**
- Modify: `README.md`
- [ ] **Step 1: Write the real README**
Write `README.md`:
```markdown
# package-updater
Webhook-driven Docker Compose service updater. Fills the gap between Watchtower (polling, no CI integration) and full GitOps (Argo CD, Flux) for a self-hosted, single-host environment.
Trigger flow:
1. Gitea workflow builds and pushes a new image to your registry.
2. Workflow calls `POST /update` on this service with the image name.
3. Service finds the matching Compose-managed container(s) on the host via Docker labels.
4. Runs `docker compose pull` + `up -d` for the relevant service(s).
See [design spec](docs/superpowers/specs/2026-05-22-package-updater-design.md) for full design and rationale.
## How it finds the right stack
The service queries the Docker socket and reads the labels Compose itself attaches to every container:
- `com.docker.compose.project` / `service` / `project.working_dir` / `project.config_files`
A container is eligible for update only if it has **both**:
- An image name matching the request (tag-agnostic), AND
- The opt-in label `se.shcizo.auto-update=true`.
Defense in depth: a valid bearer token AND the opt-in label must both be present before any container is touched.
## Quick start
1. Build and push the image (e.g. via your own CI).
2. Copy `docker-compose.example.yml` to `/home/shcizo/self-hosted/package-updater/docker-compose.yml`.
3. Create `.env` next to it: `UPDATER_API_KEY=$(openssl rand -hex 32)`.
4. Point your reverse proxy (NPM/Traefik/Caddy) at `package-updater:8080`. NPM should handle TLS.
5. `docker compose up -d`.
6. Add the opt-in label `se.shcizo.auto-update: "true"` to each service you want auto-updated.
7. Use the [Gitea composite action](gitea-action/README.md) in your repos to call `/update` after a build.
## Configuration
All via environment variables.
| Variable | Required | Default | Purpose |
|---|---|---|---|
| `UPDATER_API_KEY` | **yes** | — | Bearer token. Service refuses to start without it. |
| `STACKS_ROOT` | no | `/home/shcizo/self-hosted` | Required parent for any stack eligible to update. |
| `PORT` | no | `8080` | HTTP listen port. |
| `LOG_LEVEL` | no | `info` | `debug` / `info` / `warn` / `error`. |
| `UPDATE_TIMEOUT` | no | `5m` | Per-job timeout (Go duration). |
| `OPT_IN_LABEL` | no | `se.shcizo.auto-update` | Label name to check; value must equal `"true"`. |
## Endpoints
| Endpoint | Auth | Purpose |
|---|---|---|
| `POST /update` | Bearer token | Trigger pull + restart for matching services |
| `GET /healthz` | none | Liveness + Docker socket reachability |
| `GET /version` | none | Build info |
| `GET /metrics` | none | Prometheus exposition |
`/healthz`, `/version`, and `/metrics` are intentionally unauthenticated — they're internal-network only behind the reverse proxy.
## Observability
- **Logs**: JSON to stdout, picked up by Promtail/Alloy → Loki.
- **Metrics**: Prometheus exposition on `/metrics`. Notable: `package_updater_update_jobs_total{project,service,status}`, `package_updater_last_update_timestamp{project,service}`, `package_updater_docker_ping_up`.
## Development
```bash
go test ./...
go build ./cmd/server
docker build -t package-updater:dev .
```
```
- [ ] **Step 2: Final full test sweep**
Run:
```bash
go test ./...
go vet ./...
```
Expected: all PASS, no vet warnings.
- [ ] **Step 3: Commit**
```bash
git add README.md
git commit -m "docs: real README with quick start, endpoints, observability"
```
---
## Done criteria check
After Task 19, verify against the spec's acceptance criteria (section 16):
- [ ] `POST /update` with a known image triggers `docker compose pull` + `up -d` for all matching opt-in services on the host. *(End-to-end manual smoke test on first deploy.)*
- [ ] Containers without the opt-in label are never touched, even with a valid token. *(Covered by `TestFindJobs_SkipsWithoutOptIn`.)*
- [ ] Requests without a valid bearer token receive `401` and trigger no Docker action. *(Covered by `TestAuth_Rejects`.)*
- [ ] Zero matches returns `200` with `matched: 0`. *(Covered by `TestUpdate_ZeroMatchesReturns200`.)*
- [ ] Partial failure returns `207` with per-job results. *(Covered by `TestUpdate_MixedReturns207`.)*
- [ ] Service updates itself successfully (HTTP response fully delivered before container replaced). *(`internal/selfupdate` unit-tested; integration of the Wrapped executor into the live queue is noted as deferred — see Task 14 note.)*
- [ ] Logs in Loki are filterable by `event`, `level`, `project`, `service`, `status`, `request_id`. *(Logger emits these as top-level JSON fields.)*
- [ ] Prometheus scrapes `/metrics`; all listed metrics present. *(All 7 collectors registered.)*
- [ ] Healthcheck returns `200` when socket reachable, fails otherwise. *(Covered by `TestHealthz_*`.)*
- [ ] Gitea composite action surfaces failure to the workflow. *(Action exits non-zero on HTTP 4xx/5xx.)*
---
## Self-review notes
**Spec coverage:** All sections 1–16 of the spec are covered, with one explicit gap called out: the self-update Wrapped executor is implemented and unit-tested but not yet wired into the live queue (Task 14 note). This is deliberate — wiring it requires plumbing `http.Flusher` through `Submit` and complicates the interface, so it's deferred until self-update friction is actually observed.
**Placeholders:** None. Every code step is complete.
**Type consistency:** `Job`, `Result`, `Status`, `ComposeLabels`, `Discovery`, `Queue`, `Executor`, `Handlers`, `Submitter`, `Finder`, `Pinger`, `Metrics`, `Config`, `Wrapped`, `UpdateRequest`, `UpdateResponse`, `ResultDTO`, `HealthResponse`, `VersionResponse` — checked across tasks. Field names (`Project`, `Service`, `WorkingDir`, `ConfigFiles`, `Refused`, `RefusedReason`) match across discovery, updater, and api packages.
Reference in New Issue View Git Blame Copy Permalink