Skip to content

request-context.md

Latest commit

 

History

History
152 lines (111 loc) · 6.4 KB

request-context.md

File metadata and controls

152 lines (111 loc) · 6.4 KB
Raw

RequestContext — the per-request envelope

Status: Implemented in Step 1.1a (PR landing the RequestContext type, ctx-first SPI signatures, and the tests/contract/spi_signature.py linter).

Overview

RequestContext is a frozen Pydantic v2 model that travels with every SPI call. It carries everything downstream layers need to make correct, governed, budgeted decisions without reaching back to global state or guessing.

It is constructed exactly once — at the API gateway boundary — and passed through every subsequent call as the first argument. Downstream code treats it as a trusted, immutable object.

Why it exists

Before RequestContext, downstream layers had to retrieve tenant / principal / budget / trace state from a mix of contextvars, thread-locals, and parameter lists. Three failure modes followed:

  1. Coverage gaps — a new SPI method forgets to look up the principal; ACL checks silently no-op.
  2. Test friction — every fixture mocks contextvars in different ways.
  3. Concurrency hazards — contextvar propagation across asyncio.gather is subtle.

A single typed envelope on every call signature eliminates all three.

Shape

class RequestContext(BaseModel):
    model_config = {"frozen": True}

    request_id: RequestId                 # ULID-like ID; round-trips with the trace
    tenant_id: TenantId                   # required; matched against principal.tenant_id
    principal: Principal                  # user/service identity + ACL labels
    namespace: str                        # logical-isolation key (Step 6.1); defaults to tenant_id
    physical_index: str | None            # dedicated-index key (Step 6.2); None = shared base index
    pii_policy: PiiPolicy                 # per-tenant: redact | mask | encrypt | tag_only | block | allow
    trace: TraceContext                   # OTel span + correlation IDs
    budget: Budget                        # tokens, dollars, wall_ms, max_iter
    feature_flags: frozenset[str]         # for shadow mode, A/B, killswitches
    corpus_routing_hint: str | None       # optional sticky routing across agent-loop turns
    created_at: datetime                  # autopopulated at construction

Principal, PiiPolicy, TraceContext, Budget are all frozen Pydantic models in rag_core.types.

Usage

At the boundary (gateway)

async def query_endpoint(req: QueryRequest, auth: AuthInfo) -> QueryResponse:
    ctx = RequestContext(
        request_id=ulid.new(),
        tenant_id=auth.tenant_id,
        principal=auth.principal,
        pii_policy=tenant_config.pii_policy,
        trace=TraceContext.from_otel(current_span()),
        budget=req.budget or tenant_config.default_budget,
        feature_flags=tenant_config.feature_flags,
        corpus_routing_hint=req.session.last_corpus,
    )
    return await pipeline.execute(ctx, req.query)

The gateway resolves namespace, pii_policy, and the principal's acl_labels from the tenant's rag.yaml config via the TenantResolver (Step 6.1) — see multi-tenancy.md. namespace is the logical-isolation key threaded to every SPI; it defaults to tenant_id when unset, so isolation is never weaker than tenant scoping.

Inside an SPI

class VectorRetrievalBackend(ABC):
    async def retrieve_ids(
        self,
        ctx: RequestContext,            # always first
        query: Query,
        filter_expr: FilterExpr | None = None,
        top_k: int = 100,
    ) -> list[ChunkRef]: ...

Across an asyncio.gather

Pass ctx explicitly to every awaitable. Do not rely on contextvars.

vec, bm25, graph = await asyncio.gather(
    vector.retrieve_ids(ctx, query),
    keyword.retrieve_ids(ctx, query),
    graph.retrieve_ids(ctx, query),
)

Internals

Immutability

frozen=True. To produce a derived context (e.g., for an agent-loop sub-turn with a reduced budget), use ctx.model_copy(update={"budget": ctx.budget.spend(used_tokens=...)}).

The Step 2.11 AgentLoopV0 uses exactly this pattern between iterations:

new_budget = current_ctx.budget.spend(
    tokens=per_iter_token_estimate,
    wall_ms=measured_iter_wall_ms,
    iter_=1,
)
current_ctx = current_ctx.model_copy(update={"budget": new_budget})

Budget zero-vs-None convention (G-03 ✅).

  • None means uncapped for that dimension.
  • A positive integer / float means capped — the planner / agent loop stops when cumulative spend reaches the cap.
  • Budget(...) rejects 0 and negative values at construction time (mode="before" model validator) so the historical ambiguity ("uncapped" vs "no budget") can never reach the runtime.
  • Budget.spend() continues to produce zero-valued fields legitimately — that's the documented exhaustion floor. The validator skips non-dict inputs so spend-produced instances pass through nested-model field validation unchanged.
Budget(max_tokens=1000)        # ok — capped at 1000 tokens
Budget()                       # ok — fully uncapped (all None)
Budget(max_dollars=0.001)      # ok — capped at a tenth of a cent
Budget(max_iter=0)             # ValueError — use None or a positive int
Budget(max_tokens=-5)          # ValueError — negative was always nonsense

Validation

Validated exactly once at construction. Downstream SPIs accept it as trusted — they MUST NOT re-validate (cost is non-trivial in hot paths). See performance.md.

Type-level enforcement

A tests/contract/spi_signature.py linter asserts every public SPI method's first argument is ctx: RequestContext. CI fails on additions that omit it.

Lifetime caches

PolicyEngine and Storage (for BlobRef resolution) cache results keyed on ctx.request_id for the lifetime of a single request. The cache is held in a ContextLocal attached to the context, not a process-wide dict.

Extension points

Adding a field is a minor-version change. Process:

  1. Add the field to RequestContext with a default (so existing call sites don't break).
  2. Document it here (Shape table).
  3. Wire it through the gateway's RequestContext construction.
  4. Update downstream consumers that need it.

Removing or renaming a field is a major change and requires an ADR.

Related