relay: startup security posture self-check — refuse to boot in misconfigured environments

[ilmoniemi]

Why

Dockerfile + CI scans + threat model are upstream guards (stochastic — depend on every contributor remembering them). A runtime startup self-check is the deterministic backstop that fires every single boot, regardless of how the deploy pipeline was assembled. Belt-and-suspenders per [[PROJECT-MEMORY#Behavioral / Instruction-Design]].

The pattern already exists: ErrCacheDirInsecure (#9) refuses to start if autocert cache is world/group-readable. This ticket extends that pattern into a coherent boot-time posture check.

What

Add internal/relay/startup_check.go exposing one function:

// CheckSecurityPosture runs all startup environment checks and returns
// a multi-error describing every failed posture invariant. Fail-fast
// on first error in production mode; in dev mode (--insecure-listen)
// log warnings and continue.
func CheckSecurityPosture(cfg Config) error

Invariants checked:

Check	Failure mode
Effective uid != 0 (production mode)	ErrRunningAsRoot
Effective capabilities ⊆ allowlist (Linux only via /proc/self/status)	ErrUnexpectedCapability
Autocert cache dir perms == 0700	already as ErrCacheDirInsecure (#9) — call from this layer
--insecure-listen flag is unset in production mode (env-var contract: PYRYCODE_RELAY_PRODUCTION=1)	ErrInsecureListenInProduction
Required env vars present + valid format	ErrInvalidConfig{key, reason}
Listener will bind only to expected ports (no stray :6060 pprof, no leaked debug ports)	ErrUnexpectedListener
Multi-instance check	covered by #39 — CheckSecurityPosture calls into that

Wired in cmd/pyrycode-relay/main.go after flag parse, before http.ListenAndServe — fail-fast with structured log line per invariant.

Implementation notes

• Sentinel errors per the established pattern (Err... naming, errors.Is branchable)
• Test the matrix: each invariant gets a unit test for both pass + fail; the public function gets an integration-shape test that asserts ALL invariants run (so a future invariant added to one but not the other surfaces immediately)
• Cross-platform: capability + uid checks are Linux-specific; gate with build tags or runtime OS checks. macOS dev runs skip them with a single "running on darwin, skipping linux-only checks" warning.
• Production-mode signal: explicit env var (PYRYCODE_RELAY_PRODUCTION=1) is cleaner than autodetecting. Dev / test runs default to non-production. Deploy manifest sets the flag explicitly.
• Loud failure log: each failed invariant logs structured error + the env-var or fix to set. Don't make the operator grep — the log line IS the runbook.

Why this exists alongside CI scans

CI scans verify the build is clean. The startup self-check verifies the runtime environment is clean. A correct image deployed wrong (root user via docker run --user 0, leaked debug port via env override, etc.) escapes CI but gets caught at boot.

Out of scope

• Periodic re-scans during runtime (most posture invariants are boot-time decisions; runtime drift is rare). Single boot-time check is enough for v1.
• Auto-remediation (don't try to fix; loud-fail per established "loud failure over silent correction" pattern)
• Telemetry of failed boot attempts (production posture doesn't change between boots; if it fails once, ops sees it; no need for time-series)

[ilmoniemi]

Demoting back to Inbox — this ticket bundles 7 invariants into a "posture check framework" and needs human triage before refinement is useful.

Sizing red lines tripped (must split or reduce scope)

• >5 acceptance criteria — one per invariant + framework wiring.
• >5 new exported sentinels — ErrRunningAsRoot, ErrUnexpectedCapability, ErrInsecureListenInProduction, ErrInvalidConfig{key,reason}, ErrUnexpectedListener.
• Body needs "and" repeatedly to describe scope ("uid check and capability check and env-var format check and listener-port check and production-mode contract and multi-instance call-through").
• >150 lines likely, with Linux-vs-darwin build-tag branching.

Overlap with shipped / in-flight work

• Autocert cache dir perms — ErrCacheDirInsecure (relay: TLS via Let's Encrypt autocert for production --domain mode #9) is already wired at relay.NewAutocertManager; this ticket's wording "call from this layer" implies a refactor that isn't justified.
• Multi-instance check — owned by relay: startup self-check refuses to run as multi-instance deploy #65 (architect-stage, rework-count:1). Env-var contract PYRYCODE_RELAY_SINGLE_INSTANCE is fixed by the merged relay: document v1 single-instance constraint in architecture.md #64 doc. Bundling here forks the contract.

Evidence-Based Fix Selection — which invariants are observed-needed?

Per docs/PROJECT-MEMORY.md and the pipeline principle: don't ship a defense for a failure mode that hasn't been observed.

Invariant	Observed?	Notes
uid != 0 in autocert mode	Plausible	Dockerfile (#32) runs nonroot, but docker run --user 0 overrides. Real foot-gun.
Capabilities ⊆ allowlist (Linux /proc/self/status parse)	No	Speculative. Adds a Linux-only parser + cross-platform branching for a hypothetical drift.
Required env vars present + valid format	No	Relay has no required env vars today (flag-driven). Defending a non-existent contract.
Listener binds only expected ports (no stray :6060)	No	cmd/pyrycode-relay/main.go has no net/http/pprof import and no debug listener. No observed leak.
--insecure-listen unset in production	Maybe	But there's no production-mode signal today; --domain vs --insecure-listen (mutually exclusive, loud-fail) already serves that role.

New env-var surface — PYRYCODE_RELAY_PRODUCTION=1

The ticket proposes a new mode signal distinct from the existing --domain / --insecure-listen pair. Question for the human: is there a reason --domain (autocert TLS) isn't a sufficient "production" signal? Adding a second mode signal is real cost (two contracts to keep in sync, two ways to misconfigure).

What I'd like from a human

Please identify which specific invariants have an observed failure mode (or near-miss) you want a deterministic backstop for, and file them as individual Inbox tickets. The "framework" can emerge organically once 2-3 checks coexist — it doesn't need to be designed up front. Likely candidates worth filing individually:

1. Refuse-to-start when running as uid 0 in autocert/--domain mode — small, deterministic, paired with relay: Dockerfile + base hardening — portable deploy artifact #32.
2. Anything else where you've actually seen the failure mode (or where a deploy-time misconfiguration nearly happened).

The other invariants in this body (capabilities, env-var format, listener ports, separate production-mode env var) should not ship until evidence justifies them.

[ilmoniemi]

🤖 Product Owner — creates structured issues

po agent has completed work on this ticket.

Agent output (click to expand)

Demoted #42 to Inbox with a structured comment.

**Reasoning:** The ticket bundles 7 invariants into a "posture check framework" — trips multiple sizing red lines (>5 ACs, >5 new sentinels, "and"-in-description, >150 lines likely with cross-platform branching). Two invariants overlap with shipped/in-flight work (cache perms via #9, multi-instance via #65), and most of the remaining checks (capabilities allowlist, env-var format, stray listener ports) defend against failure modes that haven't been observed — fails Evidence-Based Fix Selection. The new `PYRYCODE_RELAY_PRODUCTION=1` env var also adds a second mode signal where `--domain`/`--insecure-listen` already exists.

Comment asks the human to identify which specific invariants have observed-failure evidence and re-file them as individual Inbox tickets (the uid-0 check is the strongest candidate; the framework can emerge organically once a couple of checks coexist).

Ready for human review. Move to the next column when approved.

[ilmoniemi]

Second demote to Inbox — no substantive changes since my prior comment. The ticket still bundles 7 invariants, conflicts with shipped/in-flight work (#9, #65), and most checks defend against unobserved failure modes per Evidence-Based Fix Selection.

To unblock, please do one of:

1. Close this ticket and open a fresh one for the single observed failure mode you actually want backstopped — most likely "refuse to start as uid 0 when --domain is set". That's an XS ticket that can move through the pipeline today.
2. Edit the body of this ticket to scope it down to one invariant, then re-promote to Backlog.
3. Reply here naming which 2-3 invariants have observed evidence and I'll split into child tickets on the next pass.

What I can't do is refine this as-is — every iteration of the pipeline that runs on it will trip the same red lines and demote again. (And re-promoting without a body change just repeats this loop.)

[ilmoniemi]

🤖 Product Owner — creates structured issues

po agent has completed work on this ticket.

Agent output (click to expand)

Status confirmed at Inbox. Summary:

- Demoted #42 to Inbox for the second time.
- Posted a tighter comment giving the human three concrete unblock paths (close+refile, edit-down, or name the 2-3 invariants worth splitting).
- Verified status moved to Inbox this time (the prior demote either failed silently or was reverted; the comment flags the loop explicitly so the next promote-without-body-change won't just bounce again).
- Did NOT add `ready:po`. Did NOT modify the body. Did NOT split — the previous comment's analysis already established that splitting blind isn't appropriate without human direction on which invariants have observed evidence.

Ready for human review. Move to the next column when approved.

[ilmoniemi]

Going with PO's option 3 — naming the invariants with observed-evidence intent, please split into child tickets on the next pass.

Invariants to split out (5 children, one per concern):

1. Effective uid != 0 in production (ErrRunningAsRoot) — observed risk: docker run --user 0 or missing non-root USER directive in Dockerfile escapes CI build scans.
2. Effective capabilities ⊆ allowlist (ErrUnexpectedCapability) — Linux-only check via /proc/self/status. Concrete Linux container-runtime hardening.
3. Autocert cache dir perms == 0700 — already shipped as ErrCacheDirInsecure (relay: TLS via Let's Encrypt autocert for production --domain mode #9). Skip.
4. --insecure-listen unset in production (ErrInsecureListenInProduction) — observed risk: dev-config-promoted-to-prod operational mistake. Production-mode signal via PYRYCODE_RELAY_PRODUCTION=1 env var as proposed in the body.
5. Required env vars present + valid format (ErrInvalidConfig{key, reason}) — boot-time validation for the required env vars set by the deploy manifest.
6. Listener binds only to expected ports (ErrUnexpectedListener) — defends against leaked debug ports (e.g. stray :6060 pprof, env-overridden debug port).
7. Multi-instance check — covered by relay: single-instance constraint — doc + startup self-check (registry is in-memory) #39. Skip.

Suggested approach for the split:

• Five new XS / S tickets, one per invariant (relay: routing-envelope wrapper type — marshal, unmarshal, tests #1, feat(relay): routing-envelope wrapper type (#1) #2, relay: WS upgrade for /v1/server — accept binary connection, validate headers, claim server-id #4, relay: WS upgrade for /v1/client — accept phone connection, look up server-id, register #5, relay: frame forwarding loop — wrap/unwrap routing envelope, route by server-id and conn_id #6 from the table above).
• Each child stands alone (sentinel error + unit test + integration test slot). No "posture-check framework" wrapper — each invariant's Check* function lives in the same package and gets called explicitly from cmd/pyrycode-relay/main.go in order. The "framework" emerges from N small functions called sequentially, not from a coordinating type.
• internal/relay/startup_check.go can be the shared file all five contribute to, or each can ship in its own file (startup_check_uid.go, startup_check_capabilities.go, ...). Architect's call.
• The two already-covered concerns (autocert relay: TLS via Let's Encrypt autocert for production --domain mode #9, multi-instance relay: single-instance constraint — doc + startup self-check (registry is in-memory) #39) get linked from each new child's body as related context, but no work duplicated.

Per the sizing red lines, no single child should have >5 ACs. Each invariant is a single Check* function + a sentinel + tests — XS or S.

Close #42 as superseded once all five children are filed.