chore(master): run the master as nobody in the example stack (#25)
This commit is contained in:
@@ -122,6 +122,8 @@ DATABASE_URL=postgres://... go test -tags integration ./internal/master/store/
|
|||||||
- **Swarm updates need manager-node API access** for the agent doing the update.
|
- **Swarm updates need manager-node API access** for the agent doing the update.
|
||||||
- **The image's default entrypoint is `updater-agent`.** The master overrides it with
|
- **The image's default entrypoint is `updater-agent`.** The master overrides it with
|
||||||
`updater-master`.
|
`updater-master`.
|
||||||
|
- **The example stack runs the master as nobody (`user: "65534:65534"`).** Never add `USER`
|
||||||
|
to the Dockerfile: the agent uses the same image and needs root for the Docker socket.
|
||||||
- **The webhook waits via a broker signal plus a store re-read, with a fallback poll**
|
- **The webhook waits via a broker signal plus a store re-read, with a fallback poll**
|
||||||
(the broker drops signals for slow subscribers). Marking and starting jobs runs on
|
(the broker drops signals for slow subscribers). Marking and starting jobs runs on
|
||||||
`context.WithoutCancel`, so a client that disconnects cannot leave a queued job no agent
|
`context.WithoutCancel`, so a client that disconnects cannot leave a queued job no agent
|
||||||
|
|||||||
@@ -28,7 +28,9 @@ a time per service, and one job at a time per agent.
|
|||||||
Use [`deploy/master.stack.yml`](deploy/master.stack.yml) as a starting point (Swarm stack;
|
Use [`deploy/master.stack.yml`](deploy/master.stack.yml) as a starting point (Swarm stack;
|
||||||
needs Postgres and an OIDC provider). The master listens for the UI on `PORT` (default
|
needs Postgres and an OIDC provider). The master listens for the UI on `PORT` (default
|
||||||
`8080`, put it behind your reverse proxy) and for agents on `AGENT_PORT` (default `8090`,
|
`8080`, put it behind your reverse proxy) and for agents on `AGENT_PORT` (default `8090`,
|
||||||
pinned TLS, reached directly on the internal network).
|
pinned TLS, reached directly on the internal network). The example runs the master as
|
||||||
|
`nobody` (`65534`); it needs no root. The agent runs as root because access to the Docker
|
||||||
|
socket is root on the host anyway.
|
||||||
|
|
||||||
| Variable | Required | Default | Purpose |
|
| Variable | Required | Default | Purpose |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
@@ -145,12 +147,14 @@ multi-arch image the digest is the index digest, the same one `docker inspect` s
|
|||||||
"poll failed" badge on the overview and in the "Polling" row of the service page; other
|
"poll failed" badge on the overview and in the "Polling" row of the service page; other
|
||||||
tags continue. A later success clears it.
|
tags continue. A later success clears it.
|
||||||
- **Credentials.** The master reads `$DOCKER_CONFIG/config.json` (a file written by
|
- **Credentials.** The master reads `$DOCKER_CONFIG/config.json` (a file written by
|
||||||
`docker login`), mounted read-only into the container; without it registries are accessed
|
`docker login`), mounted read-only into the container and readable by the master's user (`65534` in the
|
||||||
|
example stack); without it registries are accessed
|
||||||
anonymously. Only inline `auths` entries are supported. A `credsStore` or `credHelpers`
|
anonymously. Only inline `auths` entries are supported. A `credsStore` or `credHelpers`
|
||||||
entry makes every lookup fail with `error getting credentials ... executable file not found
|
entry makes every lookup fail with `error getting credentials ... executable file not found
|
||||||
in $PATH`, because the container has no credential helper binaries. The master logs
|
in $PATH`, because the container has no credential helper binaries. The master logs
|
||||||
`registry_credentials` at startup with the directory and whether `config.json` exists,
|
`registry_credentials` at startup with the directory, `config_json_found` and
|
||||||
never its contents.
|
`config_json_readable`, never the contents. If the file exists but cannot be opened it also
|
||||||
|
logs a `registry_credentials_unreadable` warning with the error.
|
||||||
- **Requests go wherever the image name points.** The master sends the lookup to the
|
- **Requests go wherever the image name points.** The master sends the lookup to the
|
||||||
registry host in the image name, including a `GET /v2/` ping that follows redirects.
|
registry host in the image name, including a `GET /v2/` ping that follows redirects.
|
||||||
Anyone who can start containers on a managed host can therefore make the master send
|
Anyone who can start containers on a managed host can therefore make the master send
|
||||||
@@ -199,7 +203,7 @@ the agent's token (shown once) and the master's key fingerprint; the same layout
|
|||||||
| `AGENT_TOKEN` | yes | — | Token shown once when the agent was added. |
|
| `AGENT_TOKEN` | yes | — | Token shown once when the agent was added. |
|
||||||
| `SELF_ID` | no | hostname | Container ID or name of the agent itself; used to read its mounts. |
|
| `SELF_ID` | no | hostname | Container ID or name of the agent itself; used to read its mounts. |
|
||||||
| `UPDATE_TIMEOUT` | no | `5m` | Per-job timeout (Go duration). |
|
| `UPDATE_TIMEOUT` | no | `5m` | Per-job timeout (Go duration). |
|
||||||
| `DOCKER_CONFIG` | no | — | Directory with a `config.json` holding registry credentials. Only inline `auths` entries work; `credsStore`/`credHelpers` are not available in the container. |
|
| `DOCKER_CONFIG` | no | — | Directory with a `config.json` holding registry credentials, readable by the master's user (see the `config_json_readable` and `registry_credentials_unreadable` log fields). Only inline `auths` entries work; `credsStore`/`credHelpers` are not available in the container. |
|
||||||
| `LOG_LEVEL` | no | `info` | `debug` / `info` / `warn` / `error`. |
|
| `LOG_LEVEL` | no | `info` | `debug` / `info` / `warn` / `error`. |
|
||||||
|
|
||||||
`docker compose` runs inside the agent with only `PATH`, `HOME` and the `DOCKER_*`
|
`docker compose` runs inside the agent with only `PATH`, `HOME` and the `DOCKER_*`
|
||||||
|
|||||||
+22
-8
@@ -48,19 +48,29 @@ const pollFirstDelay = time.Minute
|
|||||||
const pollWorkers = 4
|
const pollWorkers = 4
|
||||||
|
|
||||||
// registryCredentialsDir returns the directory the registry client reads
|
// registryCredentialsDir returns the directory the registry client reads
|
||||||
// credentials from (DOCKER_CONFIG, else ~/.docker) and whether a config.json
|
// credentials from (DOCKER_CONFIG, else ~/.docker), whether a config.json is
|
||||||
// is in it. Only the path is logged, never the file.
|
// in it, and the error from opening it when it exists but cannot be read (for
|
||||||
func registryCredentialsDir() (dir string, found bool) {
|
// example because the master runs as another user than the file owner). The
|
||||||
|
// file is opened and closed, never read; only the path is logged.
|
||||||
|
func registryCredentialsDir() (dir string, found bool, readErr error) {
|
||||||
dir = os.Getenv("DOCKER_CONFIG")
|
dir = os.Getenv("DOCKER_CONFIG")
|
||||||
if dir == "" {
|
if dir == "" {
|
||||||
home, err := os.UserHomeDir()
|
home, err := os.UserHomeDir()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", false
|
return "", false, nil
|
||||||
}
|
}
|
||||||
dir = filepath.Join(home, ".docker")
|
dir = filepath.Join(home, ".docker")
|
||||||
}
|
}
|
||||||
_, err := os.Stat(filepath.Join(dir, "config.json"))
|
path := filepath.Join(dir, "config.json")
|
||||||
return dir, err == nil
|
if _, err := os.Stat(path); err != nil {
|
||||||
|
return dir, false, nil
|
||||||
|
}
|
||||||
|
f, err := os.Open(path)
|
||||||
|
if err != nil {
|
||||||
|
return dir, true, err
|
||||||
|
}
|
||||||
|
_ = f.Close()
|
||||||
|
return dir, true, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func run() error {
|
func run() error {
|
||||||
@@ -154,8 +164,12 @@ func run() error {
|
|||||||
BaseContext: baseContext,
|
BaseContext: baseContext,
|
||||||
}
|
}
|
||||||
|
|
||||||
credDir, credFound := registryCredentialsDir()
|
credDir, credFound, credReadErr := registryCredentialsDir()
|
||||||
logger.Info("registry_credentials", "docker_config", credDir, "config_json_found", credFound)
|
logger.Info("registry_credentials", "docker_config", credDir, "config_json_found", credFound,
|
||||||
|
"config_json_readable", credFound && credReadErr == nil)
|
||||||
|
if credReadErr != nil {
|
||||||
|
logger.Warn("registry_credentials_unreadable", "docker_config", credDir, "error", credReadErr)
|
||||||
|
}
|
||||||
pollCtx, stopPolling := context.WithCancel(ctx)
|
pollCtx, stopPolling := context.WithCancel(ctx)
|
||||||
defer stopPolling()
|
defer stopPolling()
|
||||||
registryPoller := poller.New(st, jobService, registry.New("package-updater-master/"+version, logger), broker, poller.Config{
|
registryPoller := poller.New(st, jobService, registry.New("package-updater-master/"+version, logger), broker, poller.Config{
|
||||||
|
|||||||
@@ -0,0 +1,42 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRegistryCredentialsDir(t *testing.T) {
|
||||||
|
t.Run("missing file", func(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
t.Setenv("DOCKER_CONFIG", dir)
|
||||||
|
got, found, readErr := registryCredentialsDir()
|
||||||
|
assert.Equal(t, dir, got)
|
||||||
|
assert.False(t, found)
|
||||||
|
assert.NoError(t, readErr)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("readable file", func(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
require.NoError(t, os.WriteFile(filepath.Join(dir, "config.json"), []byte("{}"), 0o600))
|
||||||
|
t.Setenv("DOCKER_CONFIG", dir)
|
||||||
|
_, found, readErr := registryCredentialsDir()
|
||||||
|
assert.True(t, found)
|
||||||
|
assert.NoError(t, readErr)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("unreadable file", func(t *testing.T) {
|
||||||
|
if os.Geteuid() == 0 {
|
||||||
|
t.Skip("root can read any file")
|
||||||
|
}
|
||||||
|
dir := t.TempDir()
|
||||||
|
require.NoError(t, os.WriteFile(filepath.Join(dir, "config.json"), []byte("{}"), 0o000))
|
||||||
|
t.Setenv("DOCKER_CONFIG", dir)
|
||||||
|
_, found, readErr := registryCredentialsDir()
|
||||||
|
assert.True(t, found)
|
||||||
|
assert.Error(t, readErr)
|
||||||
|
})
|
||||||
|
}
|
||||||
@@ -12,12 +12,16 @@
|
|||||||
# The reverse proxy must allow a read timeout longer than UPDATE_TIMEOUT (default 5m).
|
# The reverse proxy must allow a read timeout longer than UPDATE_TIMEOUT (default 5m).
|
||||||
# 5. Optional, for registry polling of private images: put a config.json written by
|
# 5. Optional, for registry polling of private images: put a config.json written by
|
||||||
# `docker login` (inline "auths" only; no credsStore) in /path/to/docker-config and
|
# `docker login` (inline "auths" only; no credsStore) in /path/to/docker-config and
|
||||||
# uncomment DOCKER_CONFIG and the volumes entry below. The directory must exist on every
|
# uncomment DOCKER_CONFIG and the volumes entry below. The master runs as 65534:65534, so
|
||||||
|
# config.json must be readable by that user: `chmod 0644` it, or set `user:` below to the
|
||||||
|
# file owner's uid:gid. The directory must exist on every
|
||||||
# manager node (the master is placed on a manager), or Swarm rejects the task. Without
|
# manager node (the master is placed on a manager), or Swarm rejects the task. Without
|
||||||
# it registries are accessed anonymously.
|
# it registries are accessed anonymously.
|
||||||
services:
|
services:
|
||||||
master:
|
master:
|
||||||
image: gitea.shcizo.se/shcizo/package-updater:latest
|
image: gitea.shcizo.se/shcizo/package-updater:latest
|
||||||
|
# Runs as nobody: the master needs no root (the agent does, for the Docker socket).
|
||||||
|
user: "65534:65534"
|
||||||
entrypoint: ["/bin/sh", "-c", "DATABASE_URL=$$(cat /run/secrets/pu_database_url) OIDC_CLIENT_SECRET=$$(cat /run/secrets/pu_oidc_client_secret) UPDATER_API_KEY=$$(cat /run/secrets/pu_updater_api_key) exec /usr/local/bin/updater-master"]
|
entrypoint: ["/bin/sh", "-c", "DATABASE_URL=$$(cat /run/secrets/pu_database_url) OIDC_CLIENT_SECRET=$$(cat /run/secrets/pu_oidc_client_secret) UPDATER_API_KEY=$$(cat /run/secrets/pu_updater_api_key) exec /usr/local/bin/updater-master"]
|
||||||
environment:
|
environment:
|
||||||
PUBLIC_URL: https://updater.example.com
|
PUBLIC_URL: https://updater.example.com
|
||||||
|
|||||||
Reference in New Issue
Block a user