v0.3.0 — three verbs on Redis Streams

.github/workflows/ci.yml

@@ -28,11 +28,41 @@ jobs:
       fail-fast: false
       matrix:
         python-version: ["3.12", "3.13", "3.14"]
+    # Both backends are exercised on every leg. They are not interchangeable:
+    # a real XREADGROUP BLOCK is woken by an arriving message and fakeredis's is
+    # not, and a real Redis is shared across the whole run while fakeredis is
+    # fresh per test. Bugs hide on each side of that difference.
+    services:
+      redis:
+        image: redis:7-alpine
+        ports:
+          - 6379:6379
+        options: >-
+          --health-cmd "redis-cli ping"
+          --health-interval 5s
+          --health-timeout 3s
+          --health-retries 10
     steps:
       - uses: actions/checkout@v6
       - uses: astral-sh/setup-uv@v7
         with:
           enable-cache: true
           python-version: ${{ matrix.python-version }}
       - run: uv sync --extra dev
-      - run: uv run python -m pytest
+
+      - name: tests (fakeredis)
+        env:
+          RUNTIME_TEST_BACKEND: fake
+        run: uv run python -m pytest
+
+      # Coverage is measured on this leg rather than the fakeredis one: a real
+      # Redis reaches strictly more paths (blocking reads that are genuinely
+      # woken, the timeout-to-cancellation conversion). The floor is a ratchet
+      # against a large untested addition, not a target — read the missing lines,
+      # which are mostly failure and configuration paths.
+      - name: tests (real Redis, with coverage)
+        env:
+          RUNTIME_TEST_BACKEND: real
+        run: >-
+          uv run python -m pytest
+          --cov=runtime --cov-report=term-missing --cov-fail-under=90

.gitignore

@@ -29,3 +29,6 @@ htmlcov/
 context.md
 plan.md
 runtime_api.py
+# Proposal for a metrics seam that does not exist yet; shipping it as
+# documentation would describe an API nobody can call.
+doc/observability.md

CHANGELOG.md

@@ -0,0 +1,95 @@
+# Changelog
+
+## 0.3.0
+
+A clean break. The public API is replaced, and the Redis keyspace changes, so
+0.2 and 0.3 processes **cannot** talk to each other and messages queued by 0.2
+are not read by 0.3. Drain before upgrading, or run the two on separate
+`namespace`s.
+
+### Added
+
+**Three verbs instead of one.** `emit` used to be the only way to send anything.
+
+| | Cardinality | Reply |
+|---|---|---|
+| `await ctx.call(action, **params)` | exactly one executant | the return value, awaited |
+| `await ctx.dispatch(action, **params)` | exactly one executant | a task id |
+| `await ctx.emit(event, **params)` | every subscriber | none |
+
+- **Request/reply.** An action's return value crosses back to the caller; errors
+  are typed, serialized, and re-raised in the calling process.
+- **Per-message `ttl` and `delay`.** A `ttl` becomes an absolute deadline that
+  travels with the message; a worker claiming an expired one drops it without
+  running the handler. A child `call`/`dispatch` inherits the remaining budget
+  and can only narrow it.
+- **Correlation.** Every message carries a request id, call depth, provenance and
+  user `meta`, propagated across all three verbs, reachable as `ctx.request_id`,
+  `ctx.level`, `ctx.caller`, `ctx.meta`.
+- **Params validation.** `@service.action(name, params=SomeModel)` validates
+  before the handler runs; the handler receives the model instance.
+- **Task records.** `dispatch` returns an id; `ctx.task_status(id)` reports
+  `pending | done | failed | expired`.
+- **Runtime switches.** Pause and resume a service, action, subscription or
+  producer without a deploy, from `runtime.switches` or `ctx.set_enabled`.
+  Pausing stops consumption; queued work drains on resume.
+- **`App` knobs:** `default_ttl`, `max_depth`, `stream_maxlen`, `task_ttl`,
+  `claim_block`, `sweeper`, `switch_interval`.
+
+### Changed (breaking)
+
+| 0.2 | 0.3 |
+|---|---|
+| `Pool(...)` and `Scheduler(...)` | one `Service(...)` holding both |
+| `@pool.flow(consumes=X)` | `@service.action(X)` — or `@service.event(X)` for fan-out |
+| `ctx.emit(X, **p)` to reach one consumer | `ctx.dispatch(X, **p)` |
+| `Event` type alias | `Params` |
+| `Handler` | `ActionHandler` / `EventHandler` |
+| `broker.stream_name(t)` | `broker.action_stream(n)` / `event_stream(n)` |
+| stream `flow:{type}` | `act:{name}` (group `workers`) / `evt:{name}` (group per subscriber) |
+
+- **`ctx.emit` changed meaning** — from "deliver to the one consumer" to "fan out
+  to every subscriber". A site that should now be `dispatch` and is left as
+  `emit` writes to a stream with no subscribers, and the message is silently
+  lost. There is no registry, so this cannot be detected for you. Check every
+  `emit` call site when migrating.
+- **`max_slots` is required** for any service that consumes, and refused at boot
+  otherwise. Producer-only services need none.
+- **Every message is acked**, including one whose handler raised. With no
+  reclaim, leaving it un-acked meant lost anyway plus a pending entry nothing
+  would ever claim.
+- **A new event subscriber starts at the tail**, not the head — deploying one no
+  longer replays the retained stream into it.
+- **The `Broker` protocol grew** from 5 methods to cover replies, task records,
+  the delay set and the switch registry. Custom backends must implement them.
+- Handlers keep their `(ctx, params)` shape; only the decorator name changes.
+- **pydantic is now a dependency** (`>=2.12`).
+
+### Fixed
+
+- **Idle workers reconnected to Redis every few seconds, forever.** redis-py's
+  default `socket_timeout` is exactly the runtime's default `XREADGROUP BLOCK`
+  window, so every idle read raced its own client timeout — and the resulting
+  error was handled as an ordinary empty poll, hiding it. `connect()` now sets
+  the socket timeout above the longest blocking command.
+- **The connection pool was not sized to the topology.** redis-py caps a pool at
+  100 while every blocked worker holds one for its whole claim window; a large
+  enough deployment hit the ceiling as a crash. It is now computed from the
+  resolved slot count.
+- A failure anywhere outside a handler's own guard — a reply that would not
+  serialize, a blip on an ack — killed the worker task and with it the whole
+  process. One bad message no longer stops the worker.
+- Lifespans could be torn down while handlers were still using their resources,
+  because the shutdown path caught only `CancelledError`.
+
+### Known limits
+
+No reclaim, no retries, no dead-letter, no heartbeat, no graceful drain, no
+service registry. Each omission and what it costs is documented in
+[`doc/limits.md`](https://github.com/archipellabs/runtime/blob/main/doc/limits.md)
+— read it before deploying.
+
+## 0.2.0
+
+First public release. `Pool` + `Scheduler` over Redis Streams, one verb (`emit`),
+a flow per event type.