Skip to content

Commit ec373a7

Browse files
authored
Merge pull request #26 from modern-python/feat/v0.5-streaming
feat: AsyncClient.stream() + Retry refuses streamed-body requests (0.5.0, Epic 4)
2 parents 2a9aac1 + 820addc commit ec373a7

11 files changed

Lines changed: 2188 additions & 27 deletions

File tree

README.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,25 @@ async def main() -> None:
6161
user = await client.get("/users/1", response_model=User)
6262
```
6363

64+
### Streaming responses
65+
66+
For large responses or server-sent events, stream the body chunk-by-chunk. `stream()` is an async context manager:
67+
68+
```python
69+
from httpware import AsyncClient
70+
71+
72+
async def main() -> None:
73+
async with AsyncClient(base_url="https://api.example.com") as client:
74+
async with client.stream("GET", "/big-file") as response:
75+
async for chunk in response.aiter_bytes():
76+
process(chunk)
77+
```
78+
79+
`stream()` auto-raises `StatusError` subclasses on 4xx/5xx with the response body pre-read, so `exc.response.content` is accessible from the caught exception.
80+
81+
It does NOT pass through the middleware chain: `Retry`, `Bulkhead`, and any custom middleware are bypassed. (Retry separately refuses to retry any request — stream or non-stream — whose body was an async-iterable, since streams can't replay across attempts.)
82+
6483
## Errors
6584

6685
All 4xx/5xx responses raise typed exceptions automatically: `NotFoundError`, `ServiceUnavailableError`, `RateLimitedError`, etc. — all subclasses of `httpware.StatusError`. Transport-layer transient failures raise `NetworkError`; the resilience middleware raise `RetryBudgetExhaustedError` and `BulkheadFullError`. Everything inherits `httpware.ClientError`.

docs/index.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,25 @@ async def main() -> None:
5959
user = await client.get("/users/1", response_model=User)
6060
```
6161

62+
### Streaming responses
63+
64+
For large responses or server-sent events, stream the body chunk-by-chunk. `stream()` is an async context manager:
65+
66+
```python
67+
from httpware import AsyncClient
68+
69+
70+
async def main() -> None:
71+
async with AsyncClient(base_url="https://api.example.com") as client:
72+
async with client.stream("GET", "/big-file") as response:
73+
async for chunk in response.aiter_bytes():
74+
process(chunk)
75+
```
76+
77+
`stream()` auto-raises `StatusError` subclasses on 4xx/5xx with the response body pre-read, so `exc.response.content` is accessible from the caught exception.
78+
79+
It does NOT pass through the middleware chain: `Retry`, `Bulkhead`, and any custom middleware are bypassed. (Retry separately refuses to retry any request — stream or non-stream — whose body was an async-iterable, since streams can't replay across attempts.)
80+
6281
## Errors
6382

6483
All 4xx/5xx responses raise typed exceptions automatically: `NotFoundError`, `ServiceUnavailableError`, `RateLimitedError`, etc. — all subclasses of `httpware.StatusError`. Transport-layer transient failures raise `NetworkError`; the resilience middleware raise `RetryBudgetExhaustedError` and `BulkheadFullError`. Everything inherits `httpware.ClientError`.

planning/deferred-work.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,6 @@ Items raised in reviews that are real but not actionable now.
44

55
## Open
66

7-
### Retry + streaming bodies (Epic 4 interaction)
8-
9-
- **`Retry` re-invokes `next(request)` with the same `httpx2.Request` on each attempt.** Safe for in-memory bytes/JSON bodies; unsafe for streaming/async-iterable bodies (consumed iterator can't replay). When Epic 4 ships `AsyncClient.stream` (`4-3`), Retry needs to refuse to retry streamed-body requests (or document that callers supply a body factory). Spec: `planning/specs/2026-06-05-retry-and-retry-budget-design.md` §"Open questions".
10-
117
### Decoder-side
128

139
- **`_get_adapter` `lru_cache` is module-global, not per-decoder instance** — keyed by `model` only; two `PydanticDecoder()` instances with different configurations (none today) would share adapters, and the cache survives across tests unless explicitly cleared. Revisit if/when a configurable `PydanticDecoder(mode=..., strict=...)` lands. (`src/httpware/decoders/pydantic.py:12-14`)
@@ -19,13 +15,18 @@ PR #21 (`feat/v0.3-pydantic-optional`) shipped 0.3.0 with pydantic moved to `[pr
1915
- **`pydantic` import not guarded the way `msgspec` is** — closed. `decoders/pydantic.py` now guards via `import_checker.is_pydantic_installed`; `PydanticDecoder.__init__` raises `ImportError` with the install hint; `AsyncClient(decoder=None)` fail-fast in `_default_pydantic_decoder()`.
2016
- **Empty/malformed payload tests** — closed. `tests/test_decoders_pydantic.py::test_malformed_payload_raises_validation_error` is a 7-case parametrized test pinning current pydantic-core behavior for `b""`, `b"null"`, `b"{}"`, malformed JSON, and invalid UTF-8.
2117

18+
## Closed by the 0.5.0 streaming release (2026-06-05)
19+
20+
- **`Retry` refuses streamed-body requests.** When `_request_with_body` is called with an async-iterable `content`/`data`/`files`, the request gets `extensions["httpware.streaming_body"] = True`. `Retry.__call__` reads the marker and re-raises with a PEP-678 note on retryable failures instead of retrying with a consumed iterator. Closes the prior Open entry.
21+
- **`httpx2.StreamError` family escape closed.** The new shared `_httpx2_exception_mapper` catches `httpx2.NetworkError` (which is the parent of `ReadError` / `WriteError` / `CloseError`), so stream-specific exceptions raised during body consumption now map to `httpware.NetworkError` consistently.
22+
2223
## Closed by the v0.2 thin-wrapper pivot (2026-06-03)
2324

2425
The pivot retired Request/Response/Httpx2Transport/RecordedTransport. The following deferred items are no longer applicable because their host code has been removed or because the responsibility shifted to `httpx2`:
2526

2627
- `extensions=dict(request.extensions)` opaque forwarding (host module removed).
2728
- Unbounded error body size on `StatusError.body` (the `body` field no longer exists; callers reach into `exc.response.content` themselves).
28-
- `httpx2.StreamError` family escape from the transport's `except httpx2.HTTPError` (mapping logic relocated to AsyncClient's terminal; revisit with Epic 4 streaming work).
29+
- `httpx2.StreamError` family escape from the transport's `except httpx2.HTTPError` (mapping logic relocated to AsyncClient's terminal; closed by 0.5.0 streaming work — exception mapping in _httpx2_exception_mapper covers the StreamError family via httpx2.NetworkError).
2930
- Header CRLF / log-injection at the transport seam (host module removed; httpx2 validates).
3031
- Userinfo on `StatusError.request_url` raw field (the field no longer exists; `__repr__` and summary still sanitize).
3132
- Concurrent `aclose()``__call__` races on `Httpx2Transport` (host class removed; lifecycle is `httpx2`'s concern).

planning/engineering.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ This doc is the single distilled reference for `httpware` design rationale, prot
44

55
## 1. Project intent
66

7-
`httpware` is a thin opinionated wrapper around `httpx2`. It re-exports `httpx2.Request` and `httpx2.Response` as the public request/response surface and adds three things on top: typed response decoding (via a `ResponseDecoder` protocol; pydantic and msgspec are both opt-in extras as of 0.3.0), a middleware chain composed at client construction, and a status-keyed exception tree raised automatically on 4xx and 5xx. `AsyncClient(decoder=None)` defaults to constructing a `PydanticDecoder` and so requires the `pydantic` extra; callers can supply an explicit `decoder=` argument to escape the default. As of 0.4.0, the package ships a small resilience suite under `httpware.middleware.resilience` — a `Retry` middleware with a Finagle-style `RetryBudget`, plus a `Bulkhead` concurrency limiter — composed via the standard middleware chain.
7+
`httpware` is a thin opinionated wrapper around `httpx2`. It re-exports `httpx2.Request` and `httpx2.Response` as the public request/response surface and adds three things on top: typed response decoding (via a `ResponseDecoder` protocol; pydantic and msgspec are both opt-in extras as of 0.3.0), a middleware chain composed at client construction, and a status-keyed exception tree raised automatically on 4xx and 5xx. `AsyncClient(decoder=None)` defaults to constructing a `PydanticDecoder` and so requires the `pydantic` extra; callers can supply an explicit `decoder=` argument to escape the default. As of 0.4.0, the package ships a small resilience suite under `httpware.middleware.resilience` — a `Retry` middleware with a Finagle-style `RetryBudget`, plus a `Bulkhead` concurrency limiter — composed via the standard middleware chain. As of 0.5.0, `AsyncClient.stream()` provides a context-manager API for chunked response bodies; it bypasses the middleware chain by design (see planning/specs/2026-06-05-streaming-design.md).
88

99
The 0.1.0 release attempted to own a full abstraction over the underlying HTTP client. v0.2 walks that back: `httpx2` is part of the public surface.
1010

@@ -132,7 +132,7 @@ Post-pivot, the roadmap has three categories. Topic slugs in `planning/specs/` a
132132
- **Shipped in v0.4 slice 1:** `Retry` middleware + Finagle-style `RetryBudget` token bucket + `attempt_timeout=` parameter (folded-in 3-1). See [`planning/specs/2026-06-05-retry-and-retry-budget-design.md`](specs/2026-06-05-retry-and-retry-budget-design.md) and [`planning/plans/2026-06-05-retry-and-retry-budget-plan.md`](plans/2026-06-05-retry-and-retry-budget-plan.md).
133133
- **Shipped in v0.4 slice 2:** `Bulkhead` middleware (concurrency limiter via `asyncio.Semaphore` with bounded acquire wait). See [`planning/specs/2026-06-05-bulkhead-design.md`](specs/2026-06-05-bulkhead-design.md) and [`planning/plans/2026-06-05-bulkhead-plan.md`](plans/2026-06-05-bulkhead-plan.md).
134134
- **Remaining:** `3-6` extension-slot docs.
135-
- **Epic 4 — Streaming:** `4-3` `AsyncClient.stream` context manager (forwards to `httpx2.AsyncClient.stream`; no `StreamResponse` type).
135+
- **Epic 4 — Streaming:** SHIPPED in v0.5 (PR #…): `AsyncClient.stream()` context manager + Retry refuses streamed-body requests. See [`planning/specs/2026-06-05-streaming-design.md`](specs/2026-06-05-streaming-design.md) and [`planning/plans/2026-06-05-streaming-plan.md`](plans/2026-06-05-streaming-plan.md).
136136
- **Epic 5 — Observability:** `5-1` Layer 1 middleware hooks, `5-2` wire into resilience middlewares, `5-4` OpenTelemetry middleware (will declare the `otel` extra at the same time the code lands), `5-5` logging policy CI grep.
137137
- **Epic 6 — Ship v1.0:** `6-2` docs site (`mkdocs`), `6-3` benchmarks, `6-5` release flow (Trusted Publishers + Sigstore).
138138
- **Carry-forward decoder:** `1-6` msgspec decoder via extras — second `ResponseDecoder` adapter, already implemented; verified surviving in the pivot.

0 commit comments

Comments
 (0)