Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions cmd/pyrycode-relay/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import (
"log/slog"
"net/http"
"os"
"syscall"
"time"

"github.com/pyrycode/pyrycode-relay/internal/relay"
Expand Down Expand Up @@ -69,6 +70,22 @@ func main() {
os.Exit(2)
}

// CheckRunningAsRoot is the in-process backstop for the CI non-root-build
// contract: docker run --user 0 or a missing/overridden USER directive at
// deploy time escapes CI and would otherwise silently run the
// internet-facing process as root. Runs before CheckCapabilities because
// production-mode misconfiguration is a deploy-shape concern that should
// be reported before Linux-specific runtime concerns. See
// docs/specs/architecture/78-refuse-boot-as-root-in-production.md § Wiring.
if err := relay.CheckRunningAsRoot(syscall.Geteuid, os.Getenv); err != nil {
logger.Error("refusing to start: production-mode misconfiguration",
"err", err,
"env_var", "PYRYCODE_RELAY_PRODUCTION",
"effective_uid", syscall.Geteuid(),
"fix", "drop privileges before exec (e.g. Dockerfile USER directive or --user <non-zero>, kubernetes securityContext.runAsUser), or unset PYRYCODE_RELAY_PRODUCTION if the deploy is truly dev")
os.Exit(2)
}

if err := relay.CheckCapabilities(); err != nil {
logger.Error("refusing to start: unexpected Linux capabilities",
"err", err,
Expand Down
2 changes: 1 addition & 1 deletion docs/knowledge/INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ One-line pointers into the evergreen knowledge base. Newest entries at the top o

- [Env-var config validator (boot-time refusal)](features/env-config-validator.md) — table-driven validation of every env var the relay reads at boot. Single source of truth is the unexported `envContracts []envContract` registry in `internal/relay/env_config.go`; each row carries `name`, `required` bool, and an inline `validate func(string) error`. `CheckEnvConfig(lookup func(string) (string, bool)) error` walks the registry and returns the structured `*ErrInvalidConfig{Key, Reason}` on the first failure (`Reason` is `"missing"` or `"malformed-value: <err>"`); the package-level sentinel `ErrInvalidConfigSentinel` is matched via a custom `Is` method (not `Unwrap`, which would double-print the message prefix) so `errors.Is(err, ErrInvalidConfigSentinel)` and `errors.As(err, &cfgErr)` form a dual contract. The `func(string) (string, bool)` (= `os.LookupEnv` shape) getter coexists with #77's `func(string) string` getter — the presence bit is necessary here to distinguish "missing-but-required" from "present-but-empty", semantically inert for `IsProductionMode`'s exact-`"1"` match. **Ordering is load-bearing**: wired in `main.go` BEFORE `CheckInsecureListenInProduction` so a typo like `PYRYCODE_RELAY_PRODUCTION=true` cannot slip through `IsProductionMode`'s silent-non-production fallback and reach the insecure-listen guard with an unvalidated value. Today's registry has one row (`PYRYCODE_RELAY_PRODUCTION`, optional-but-format-validated); future env-var reads register here at code-review time. `checkEnvConfigWith(lookup, contracts)` is the parameterised inner used by the `required: true` test case (today's production table has no required entries). Exit 2 = config-rejected-at-boot, matching the sibling refusals (#9, #77, #79) (#80).
- [Linux capability allowlist (boot-time refusal)](features/capability-allowlist.md) — relay parses `/proc/self/status`'s `CapEff:` hex mask at boot and refuses to start (exit 2) if any bit is set outside `AllowedCapabilities` (currently `{CAP_NET_BIND_SERVICE}` only, motivated by autocert binding `:80`/`:443` from uid 65532 in the distroless image). Exported sentinel `ErrUnexpectedCapability` is branchable via `errors.Is`; the wrapped error names every offending bit symbolically (`CAP_SYS_ADMIN (bit 21)` or `bit 63` for unknown), lists the allowlist contents, and embeds the operator fix string. `CapEff` only — `CapPrm/CapBnd/CapInh` would broaden false-positives (legitimate K8s default policy grants wide CapBnd) without adding load-bearing protection (relay never `capset(2)`s). Linux/non-Linux split at compile time via the new `_<goos>.go` / `_other.go` build-tag convention (see ADR-0009); non-Linux GOOS logs one skip line and returns nil. Unconditional — no production-mode gating, no env-var bypass, because stray capabilities are never legitimate. Reader-boundary test seam (`func() (string, error)`) exercises the parse + mask check end-to-end without touching real `/proc`. Joins the boot-time-refusal sentinel family (#9, #77, #79; future #78) (#79).
- [Production-mode contract & `--insecure-listen` startup refusal](features/production-mode.md) — `PYRYCODE_RELAY_PRODUCTION=1` env-var contract (exact-string match, lazy read via injected getter, mirrors `PYRYCODE_RELAY_SINGLE_INSTANCE` shape from #64/#65) plus the first boot-time check that consumes it: `relay.CheckInsecureListenInProduction` returns the exported `ErrInsecureListenInProduction` sentinel (branchable via `errors.Is`) when production mode is on AND `--insecure-listen` is set, wired into `cmd/pyrycode-relay/main.go` after flag-parse with `os.Exit(2)` (config-rejected-at-boot, distinct from runtime-failure exit 1) and structured log fields naming the env var (name only, never value) and a one-line `fix` listing both valid resolutions. `IsProductionMode` exported so sibling startup checks (#78 = uid-0) compose on the same predicate without re-reading the env var. Test seam is a `func(string) string` getter (smallest possible — no interface, no struct, no package-level var) so the 2×2 AC matrix and the value-space matrix run under `t.Parallel()` + `-race` without mutating process env (#77).
- [Production-mode contract & startup refusals](features/production-mode.md) — `PYRYCODE_RELAY_PRODUCTION=1` env-var contract (exact-string match, lazy read via injected getter, mirrors `PYRYCODE_RELAY_SINGLE_INSTANCE` shape from #64/#65) plus the boot-time checks that consume it. **#77** introduced `relay.CheckInsecureListenInProduction` + exported `ErrInsecureListenInProduction` sentinel (branchable via `errors.Is`) firing when production mode is on AND `--insecure-listen` is set. **#78** added the second consumer: `relay.CheckRunningAsRoot(geteuid, getenv)` + exported `ErrRunningAsRoot` sentinel firing when production mode is on AND `syscall.Geteuid() == 0`, closing the deploy-time gap (`docker run --user 0`, `securityContext.runAsUser: 0`, hand-edited Dockerfile dropping `USER`) that escapes the CI non-root-build contract (#32 Dockerfile, #68 Trivy). Both wired in `cmd/pyrycode-relay/main.go` after flag-parse with `os.Exit(2)` (config-rejected-at-boot, distinct from runtime-failure exit 1) and structured log fields: `env_var` carries the name only (never the value, even though `effective_uid` carries the kernel-supplied int — log-injection structurally impossible), one-line `fix` listing valid resolutions. `IsProductionMode` exported so siblings compose on the same predicate without re-reading the env var. Test seams: `func(string) string` for env, `func() int` for euid — both the smallest possible (no interface, no struct, no package-level var) and the only way to exercise the uid-0 branch in a unit test without re-execing the test binary as root. Two instances of the shape (#77, #78) now codify the "sibling boot-check" pattern; `Config.Validate()` consolidation deferred until ~5 checks exist (#77, #78).
- [Fly.io deploy](features/fly-deploy.md) — production host wiring: `fly.toml` declares TCP-passthrough on `:80`/`:443` (no Fly HTTP proxy, no Fly-managed certs) so TLS keeps terminating in the relay via autocert (#9), persistent Fly volume `relay_autocert` mounted at `/var/lib/relay/autocert`, and a single-machine hard cap encoded via `min_machines_running=1` + `auto_start_machines=false` + `auto_stop_machines="off"` + `[deploy] strategy="immediate"` (Fly Apps v2 has no `max_machines` key; the in-binary `PYRYCODE_RELAY_SINGLE_INSTANCE` self-check from #65 is the backstop). CI `deploy` job in `.github/workflows/ci.yml` runs `flyctl deploy --remote-only` on push to `main`, gated by branch-condition + `needs: [test, security, image-scan]` + `permissions: contents: read` so `FLY_API_TOKEN` is structurally unreachable from PR code; `superfly/flyctl-actions/setup-flyctl` pinned by commit SHA with `# Tracks:` comment (same convention as #68 / #41). Dedicated IPv4 is required (not optional) for autocert's HTTP-01 challenge; TCP passthrough preserves the real socket peer IP that #34's rate limiter reads. `__REGION__` / `__DOMAIN__` ship as placeholders that fail loud on first deploy (#38).
- [Connection-count gauges](features/connection-count-gauges.md) — `pyrycode_relay_connected_binaries` and `pyrycode_relay_connected_phones` exposed via a pull-based `prometheus.Collector` reading `Registry.Counts()` on each scrape; zero edits to `registry.go`; scalar (no labels) by design — `{server="..."}` would carry the attacker-influenced `x-pyrycode-server` header onto the metrics surface, which threat-model § Log hygiene forbids; stale grace-expiry fires can't move the gauge because the pointer-identity guard (ADR-0006) keeps the maps unchanged and the gauge IS the map size; race-tested against 16 mutator goroutines + a tight-loop scraper under `-race`. First collector wired into the #59 seam (#61).
- [Metrics registry (scaffolding)](features/metrics-registry.md) — private `*prometheus.Registry` + `NewMetricsHandler` factory wrapping `promhttp.HandlerFor` (text format only; OpenMetrics off; `HandlerOpts.Registry: reg` keeps `promhttp_metric_handler_*` off `DefaultRegisterer`). Seam shape for siblings: per-concern collector struct in its own file, constructed by a helper taking `prometheus.Registerer` (no mega-struct, no package-level vars) — first instantiated by #61's `connectionsCollector`. Listener still pending (#60). Structural defence against default-registry leaks via `TestMetricsRegistry_NoGlobalRegistrarLeak` (#59).
Expand Down
46 changes: 46 additions & 0 deletions docs/knowledge/codebase/78.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Ticket #78 — refuse to boot as effective uid 0 in production mode

Adds the second consumer of the `PYRYCODE_RELAY_PRODUCTION=1` contract introduced by #77: when production mode is on AND `syscall.Geteuid() == 0`, the relay refuses to start with exit 2 before any listener is opened. Closes the deploy-time half of the non-root contract that CI verifies for the build image (Dockerfile USER directive, #32; Trivy scan, #68) — `docker run --user 0`, `securityContext.runAsUser: 0`, or a hand-edited Dockerfile that drops the USER directive all escape CI today and would otherwise silently run the internet-facing process as root. Split from #42; sibling of #77.

## Implementation

- **`internal/relay/production.go` (+31 lines)** — appended to the file #77 created. Two new exported names; the unexported `envProductionMode` const and the `IsProductionMode` predicate are reused unchanged.
- `ErrRunningAsRoot` — sentinel; message names both prongs (`"relay: effective uid is 0 with PYRYCODE_RELAY_PRODUCTION=1; refusing to start"`) so a log line is self-documenting. Branchable via `errors.Is`.
- `CheckRunningAsRoot(geteuid func() int, getenv func(string) string) error` — single-conjunction body: `if IsProductionMode(getenv) && geteuid() == 0 { return ErrRunningAsRoot }`. No wrapping, no formatting; structured-log fields in `main.go` carry the operator-facing context.
- **`internal/relay/production_test.go` (+79 lines)** — appended alongside #77's tests:
- `fakeGeteuid(n int) func() int` — companion to the existing `fakeGetenv` closure helper.
- `TestCheckRunningAsRoot_Matrix` — four rows (the AC's three plus a `uid=65534` "nobody" row that locks in "uid 0 is the *only* refused uid"). All `t.Parallel()`-safe; process env and process uid are never mutated.
- `TestErrRunningAsRoot_IsBranchable` — locks the `errors.Is` contract so a future `fmt.Errorf(...: %w, ...)` refactor breaks the test, not downstream callers.
- **`cmd/pyrycode-relay/main.go` (+17 lines)** — `syscall` import added; check inserted between `CheckInsecureListenInProduction` (#77, lines 62-68) and `CheckCapabilities` (#79, lines 72+). `syscall.Geteuid` is passed directly as a function value (no closure). Structured fields: `err`, `env_var="PYRYCODE_RELAY_PRODUCTION"` (name only, never value), `effective_uid=syscall.Geteuid()` (kernel-supplied int — log-injection structurally impossible), and a `fix` string naming the two valid resolutions (drop privileges before exec; unset `PYRYCODE_RELAY_PRODUCTION` if the deploy is truly dev). Exit code 2.

The whole check is one boolean conjunction over two stateless inputs — no allocator, no parser, no value-dependent path. The `syscall.Geteuid` call at the log site is a second syscall, deliberate: the relay never calls `setuid`, so the value cannot change between the check and the log line.

## Acceptance criteria — verification map

- AC-1 (`ErrRunningAsRoot` exported sentinel, branchable via `errors.Is`): `production.go:48-57`, locked by `TestErrRunningAsRoot_IsBranchable`.
- AC-2 (check function returns sentinel iff production + uid 0, uid source injected): `production.go:59-77`.
- AC-3 (wired in `main.go` after flag parse, before any listener, log includes effective uid + `env_var=PYRYCODE_RELAY_PRODUCTION`, exits 2): `main.go:72-86`.
- AC-4 (unit tests cover the three primary rows via injected uid seam, not re-exec): `production_test.go:122-194`.
- AC-5 (no duplicate read of `PYRYCODE_RELAY_PRODUCTION`): `CheckRunningAsRoot` calls `IsProductionMode(getenv)` — single read path, no `os.Getenv` call inside `internal/relay/production.go` outside the predicate.

## Patterns established

- **Sibling boot-check shape: `ErrXxx` + `CheckXxxInProduction(injected_seams...)`.** Each new production-only startup check follows the two-name shape and wires into `main.go` as `if err := relay.CheckXxx(...); err != nil { logger.Error(...); os.Exit(2) }`. Twice is precedent, twice is the pattern — #77 and #78 together codify the shape for any future production-only check. When the wiring boilerplate becomes a real cost (~5 checks; we are at 4 today including the unconditional `CheckCapabilities`), a follow-up consolidates into `relay.Config` + `Config.Validate()`.
- **Injected `func() int` for syscall-backed inputs, paralleling injected `func(string) string` for env.** When a check depends on process-global state that has no stdlib `t.Set*` (uid, gid, hostname, pid), the smallest test seam is a typed function value — no interface, no struct, no package-level var. Call site passes the stdlib function directly (`syscall.Geteuid`); tests pass a closure returning a fixed value. The matrix test exercises every branch under `t.Parallel()` + `-race` without re-execing the test binary as root.
- **Log kernel-supplied integers as structured fields without sanitisation.** `effective_uid` is `int`; log-injection requires a string-formatting path. The `env_var` companion field still carries the *name*, never the env var's value, for the same reason #77 calls out (a confused operator might write a secret into the env var and we don't want it in centralised logs).

## Lessons learned

- **Once is precedent, twice is the pattern — don't file an ADR for the second instance.** #77 established the "production-mode sibling check" shape; #78 is the second instance of that shape. Filing an ADR for "we did the same thing again" would be process noise (same reasoning as #77's own "skip the ADR when the design follows precedent" lesson). The shape is now documented as a pattern in `features/production-mode.md` and `codebase/77.md`; subsequent instances are codebase notes only.
- **Inject the syscall, don't `t.Setenv` it.** The AC explicitly forbids exercising the uid-0 branch by re-execing as root, and there is no stdlib equivalent of `t.Setenv` for uid in the first place — a process cannot change its own euid mid-test. The `func() int` seam is the only shape that makes the branch testable without privileged re-exec. Reach for the smallest such seam whenever a check depends on a process-global the test cannot mutate; do *not* reach for an interface or a package-level var.
- **Resist the "two helpers per concern" symmetry.** The natural shape mirrors `IsProductionMode` + `CheckInsecureListenInProduction` → `IsRunningAsRoot` + `CheckRunningAsRoot`. Rejected during architect review. A bool helper with no second consumer is dead surface area today and a temptation tomorrow to call it without the production-mode guard. Inline the predicate inside the check until a second consumer materialises; export the helper at that point if it ever does.
- **Call the syscall a second time at the log site rather than threading the value back through the error.** The check returns the bare sentinel — no `fmt.Errorf("uid %d: %w", ...)` — so `errors.Is` works without unwrapping rules. `main.go` does the second `syscall.Geteuid()` for the structured-log field. The two syscalls observe the same value because the relay never calls `setuid`; the alternative (threading the int through the error) would have forced every downstream `errors.Is` caller to also unwrap.

## Cross-links

- [Production-mode contract & startup refusals](../features/production-mode.md) — feature doc; now documents both consumers (`CheckInsecureListenInProduction`, `CheckRunningAsRoot`) of the contract.
- [Codebase note #77](77.md) — sibling check (`--insecure-listen`); same wiring slot, same exit-code split, same getter-seam pattern. #78 is the second instance of the shape #77 established.
- [Docker image](../features/docker-image.md) — #32; build-time non-root contract this ticket complements at deploy time.
- [Linux capability allowlist](../features/capability-allowlist.md) — #79; the next boot-time refusal in the wiring sequence (unconditional, not production-mode-gated).
- [`internal/relay/tls.go`](../../../internal/relay/tls.go) — `ErrCacheDirInsecure`, the canonical boot-time-refusal sentinel that started this family.
- [#42 — parent ticket](https://github.com/pyrycode/pyrycode-relay/issues/42) — split into #77 / #78.
Loading