Stop reporting dropped connections as application errors

[Makisuo]

maple-web sat at a 15% error rate that fired a critical High error rate alert. Every one of its five open issues traces back to the browser briefly losing connectivity — not to anything failing.

The evidence

The top issue by volume was Fetch failed: GET /api/sync/shape — TypeError: Failed to fetch (34 events). In trace f5378231 the maple-web span dies at 2.09s while the electric-sync server span for that same request completes Ok at 40s. The errors arrive in tight bursts inside a single session: at 16:55:39 four shape long-polls and every in-flight API call failed within two seconds, and that session then ran on until 17:09. A dashboard holding several Electric long-polls open around the clock meets this on every wifi blip, VPN reconnect and laptop wake.

The existing guard only recognised aborts we issue (pause-stream, AbortError). A connection that dies on its own throws a TypeError, which nothing caught.

Telling a blip from an outage

A blip and an outage differ in how long they last, not in what they throw, so that is what peer-reachability.ts measures. The first failure starts a clock per origin, any response at all stops it (a 500 included — the peer answered), and only failures still arriving after 15s are treated as real. tracedFetch and mapleFetch both feed it, so the ShapeStream long-polls and the API calls that die in the same instant share one view of each host.

• Inside the window a fetch span stays Ok carrying maple.http.unreachable and maple.http.unreachable_ms — the loss is still charted and alertable, just not fingerprinted — and the caller is rejected exactly as before and retries on its own.
• A real outage (unreachable API, CORS misconfiguration, bad base URL) still reports, continuously, from 15s in.
• Elapsed time rather than a failure count: a blip fails every concurrent request at once, so counting would escalate on the first one.

One blip also produced three nested error spans on the API path, so a transport failure inside the window now fails with WarehouseUnreachableError, which otel-layer.ts anticipates — QueryEngine.execute → timeseriesQuery → getQueryBuilderTimeseries record Ok together.

A user-facing bug found on the way

runQuerySetWindow flattens each executor failure into a string before re-raising the batch as QuerySetNoDataError, so a dropped connection reached the adapters as text and was re-raised as WarehouseInvalidInputError — "Invalid query", recovery: fix_request. The user was told to fix a request that never left the browser. Both adapters route that case through querySetFailure, which yields the "Cannot reach Maple API" body displayError already resolved a bare transport failure to.

anticipatedErrorIdentifiers never matched a single v2 API error

The tracer's matcher reads a top-level _tag, but an error that crossed an HTTP boundary is a decoded body: every v2 failure arrives as { error: { _tag } }, and the dashboard's whole anticipated set is derived from exactly those nested tags. Expected 4xx answers — a warehouse quota rejection, a too-wide replay range — recorded as Error spans whose entire message was the stringified envelope.

The matcher now unwraps one level. { error: … } is a common envelope convention rather than a Maple shape, so the rule is stated generically and documented in the SDK README and the public docs — effect-sdk is installed by customers, and a behaviour change there is a released one.

Breakdown's empty window

The breakdown adapter still failed on an empty window, which the timeseries adapter beside it stopped doing. An empty window is a normal answer; failing it marked the span Error and billed an exception event for a panel the user simply has no data for.

Not in this PR

packages/browser's FetchInstrumentation marks a dropped connection as an Error span, so customers on that SDK have the same problem. Fixing it means a span processor rewriting status — a behaviour change to a shipped SDK that deserves its own decision.

Verification

Each behaviour is pinned by a test verified to fail without its change. Full repo typecheck (40/40) and lint pass; web (1752), effect-sdk (107), browser (17) and query-engine (1224) suites are green.

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[github-actions]

🍁 Maple PR preview

Note

Preview resources were removed when this pull request closed.

Final commit 4599fe9 · View workflow run