From 4bb56f9f88e1a481430bdb08a7826664b9445840 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 20 Aug 2026 21:29:58 -0700 Subject: [PATCH 01/12] Add Prebid ad-latency optimizations design spec Three Trusted-Server-side levers for client-side auction properties, grounded in an instrumented local baseline against the autoblog origin: an opt-in requestBids coalescing window, a securepubads preconnect hint, and an opt-in first-refresh-auction prefetch at DOMContentLoaded that keeps the React-hydration-safe DOM application point unchanged. --- ...-prebid-ad-latency-optimizations-design.md | 98 +++++++++++++++++++ 1 file changed, 98 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md new file mode 100644 index 000000000..f969584c0 --- /dev/null +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -0,0 +1,98 @@ +# Prebid Ad-Latency Optimizations — Design + +**Date:** 2026-08-20 +**Status:** Approved design, pending implementation plan +**Scope:** Client-side auction properties (server-side ad templates inactive) + +## Problem + +On properties running the client-side auction path (`creative_opportunities.enabled = false`), ads render slowly relative to what the pipeline allows. A three-run instrumented baseline against a local Trusted Server proxying the production autoblog origin (article page, consent resolved, reader-style scrolling) measured: + +| Milestone | Time | +| --------------------------------------- | ------ | +| Prebid bundle + shim fully installed | ~2.1 s | +| `DOMContentLoaded` | ~2.1 s | +| `window.load` | ~3.0 s | +| First Trusted Server `/auction` request | ~3.3 s | +| Publisher's first `requestBids` | ~4.8 s | +| First non-empty ad render | ~5.0 s | + +Three structural costs stand out, all in Trusted-Server-owned surfaces: + +1. **The first refresh auction waits for `window.load`.** The post-`load` + double-`requestAnimationFrame` defer exists to avoid React hydration mismatches (React #418), but only the DOM application needs that defer — the auction network fetch is hydration-neutral. `DOMContentLoaded` fires ~1.2 s before `load` on this page, and the gap grows on resource-heavy pages. +2. **Publisher `requestBids` bursts pay one `/auction` round trip each.** The baseline captured two `requestBids` calls 1 ms apart producing two `/auction` POSTs 2 ms apart. Each call costs a full round trip plus ~865 ms median (p90 ~1075 ms) of server-side auction time. +3. **The first GAM ad request pays fresh connection setup.** `securepubads.g.doubleclick.net` serves both the `pubads_impl` script and every ad request; no connection is warmed before first use. + +Not addressed here (out of scope): re-enabling server-side ad templates, GPT lazy-load fetch margins (publisher-coordinated), the publisher's own ad-framework init latency (~1.8 s after `load` before their first `requestBids`), and server-side auction duration tuning (PBS `tmax`). + +## Design + +Three independent levers. Two are config-gated and default off, so shipping the binary changes nothing until an operator flips the flag; one is an unconditional resource hint. + +### Lever A — `requestBids` coalescing window (opt-in) + +**Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=500`. Serialized into the injected client config (`window.__tsjs_prebid`) as `requestBidsCoalesceMs`, omitted when `0`. + +**Behavior:** The tsjs Prebid shim already wraps `pbjs.requestBids` (bidder injection, snapshot capture, `bidsBackHandler` chaining). With a non-zero window, the wrapper holds a transformed call for up to the window duration and merges every mergeable call that arrives within it into one underlying `requestBids`: + +- **Merged request:** union of the pending calls' ad units (in arrival order), maximum of their `timeout`s (absent if none supplied), and a combined `bidsBackHandler` that invokes each pending call's (already-wrapped) handler in arrival order with the merged auction's results. A throwing handler must not prevent later handlers from running. +- **Merge-safety rule:** only calls whose request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler` (with a non-empty explicit `adUnits` array) are held. Any other call — extra keys such as `ortb2` or `labels`, or an implicit global-ad-units call — first flushes the pending queue synchronously, then dispatches solo. This preserves relative call order and never reinterprets options the merge logic does not understand. +- **Default `0`:** the wrapper dispatches immediately, byte-for-byte the current behavior. + +**Return value:** held calls return `undefined` from `pbjs.requestBids`. This matches the wrapper's existing contract in practice, and the flag's opt-in nature means an operator enabling it accepts this for their property. + +### Lever B — GAM preconnect hint + +The GPT integration's `head_inserts` emits, before its bootstrap scripts: + +```html + +``` + +Unconditional: it is a pure hint with no behavioral effect, and every GPT-enabled property talks to this host. Removes DNS + TCP + TLS setup from the first ad request (and benefits the `pubads_impl` fetch). + +### Lever C — first-auction prefetch at `DOMContentLoaded` (opt-in) + +**Config:** `[integrations.prebid] prefetch_first_refresh_auction` — `bool`, default `false`. Injected as `prefetchFirstRefreshAuction`, omitted when `false`. + +**Behavior:** When enabled, the first refresh auction's `/auction` request is dispatched as soon as all of the following hold, without waiting for `window.load`: + +- consent has resolved (the existing consent gate is unchanged), +- the GPT slots that the auction would target have been observed, +- `DOMContentLoaded` has fired. + +The response is held in memory and applied at the **unchanged** post-`load` + double-`rAF` application point. DOM work therefore keeps the exact hydration-safety timing the React #418 fix established; only the network round trip moves earlier. + +**Fallback:** if the prefetched response has not arrived by the time the application point runs, the path degrades to today's behavior (issue the request then). A prefetch failure is discarded and the normal path retries; no new error surface. + +## Config-blob compatibility + +Both new fields serialize only at non-default values, matching the repository's rollback discipline: a blob that never sets them is accepted by older binaries, and before rolling a binary back past this feature, an operator must clear the flags and re-push (the same procedure documented for prior `[integrations.prebid]` additions). + +## Testing + +**Vitest (shim):** + +- coalescing merges two mergeable calls in the window into one underlying `requestBids` with the union of ad units, and invokes both handlers in order; +- a call with extra option keys flushes the queue first and dispatches solo, preserving order; +- a throwing first handler does not prevent the second handler from running; +- window `0` / absent config leaves per-call dispatch unchanged (existing suite must pass untouched); +- prefetch: response held and applied at the application point; unresolved prefetch falls back to the current request-then-apply path. + +**Rust:** + +- config defaults (`request_bids_coalesce_ms = 0`, `prefetch_first_refresh_auction = false`) and the `0..=500` validation bound; +- injected-config serialization omits both fields at their defaults and includes them otherwise; +- GPT `head_inserts` contains the preconnect link. + +**End-to-end verification:** re-run the local baseline harness (Viceroy proxying the production origin, pre-seeded consent, identical scroll script) with each flag enabled, comparing against the recorded baseline: time to first non-empty render, `/auction` request count, and burst dedup on the publisher's paired calls. + +## Rollout + +1. Land binary with defaults off; the preconnect hint is the only immediate change. +2. Enable `request_bids_coalesce_ms` (initially 50 ms) and `prefetch_first_refresh_auction` on the autoblog tester property via config push. +3. Compare live tester metrics with the harness prediction; each flag reverts independently by config push, no binary rollback. From 796da051d7aeb054313bc556dc4cafaa31acf425 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 20 Aug 2026 21:36:34 -0700 Subject: [PATCH 02/12] Correct spec: slim-prebid loader gate and requestBids promise contract Lever C: the window.load gate is installSlimPrebidLoader (the auction module is not loaded before load), so the prefetch lives in the unified GPT bundle and slim-Prebid consumes the stashed response; the double-rAF defer belongs to the SSAT adInit path, not this flow. Lever A: Prebid 10 requestBids returns a promise, so held calls return a promise settling with the merged auction instead of undefined. --- ...20-prebid-ad-latency-optimizations-design.md | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index f969584c0..73910933f 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -39,7 +39,7 @@ Three independent levers. Two are config-gated and default off, so shipping the - **Merge-safety rule:** only calls whose request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler` (with a non-empty explicit `adUnits` array) are held. Any other call — extra keys such as `ortb2` or `labels`, or an implicit global-ad-units call — first flushes the pending queue synchronously, then dispatches solo. This preserves relative call order and never reinterprets options the merge logic does not understand. - **Default `0`:** the wrapper dispatches immediately, byte-for-byte the current behavior. -**Return value:** held calls return `undefined` from `pbjs.requestBids`. This matches the wrapper's existing contract in practice, and the flag's opt-in nature means an operator enabling it accepts this for their property. +**Return value:** Prebid 10's `requestBids` returns a promise, and the wrapper currently passes it through. Every held call therefore returns a promise that settles when the merged underlying `requestBids` promise settles, so callers awaiting the promise keep working; they observe the merged auction's completion rather than a per-call auction's. ### Lever B — GAM preconnect hint @@ -59,15 +59,19 @@ Unconditional: it is a pure hint with no behavioral effect, and every GPT-enable **Config:** `[integrations.prebid] prefetch_first_refresh_auction` — `bool`, default `false`. Injected as `prefetchFirstRefreshAuction`, omitted when `false`. -**Behavior:** When enabled, the first refresh auction's `/auction` request is dispatched as soon as all of the following hold, without waiting for `window.load`: +**Mechanism today:** the `window.load` gate is `installSlimPrebidLoader` in the unified GPT bundle — the slim-Prebid module that runs refresh auctions is not even _loaded_ until `load` fires, and its first auction follows its post-load initialization. (The post-`load` + double-`rAF` defer from the React #418 fix belongs to the server-side-template `adInit` path and is not part of this flow.) + +**Behavior when enabled:** the unified GPT bundle — which is loaded from head-start — dispatches the first refresh auction's `/auction` request as soon as all of the following hold, without waiting for `window.load`: - consent has resolved (the existing consent gate is unchanged), -- the GPT slots that the auction would target have been observed, +- the GPT slots the auction would target have been observed, - `DOMContentLoaded` has fired. -The response is held in memory and applied at the **unchanged** post-`load` + double-`rAF` application point. DOM work therefore keeps the exact hydration-safety timing the React #418 fix established; only the network round trip moves earlier. +The response is stashed on `window.tsjs`. The slim-Prebid module, still loaded at the unchanged `window.load` point, consumes the stash during its existing initialization instead of issuing a fresh request. Only the network round trip moves earlier; script loading, targeting application, and the GPT refresh that triggers rendering keep their current post-load timing, so no render work moves into the hydration window. + +**Alternative considered and rejected:** loading the slim-Prebid script itself at `DOMContentLoaded`. Simpler, but it would also pull GPT refresh — and therefore ad rendering into publisher containers — earlier into the hydration window, the same territory as React #418 and the ad-container hydration gating tracked in #969. -**Fallback:** if the prefetched response has not arrived by the time the application point runs, the path degrades to today's behavior (issue the request then). A prefetch failure is discarded and the normal path retries; no new error surface. +**Fallback:** if the stash is absent or its request has not resolved when slim-Prebid initializes, the module issues its own request exactly as today. A failed prefetch is discarded; no new error surface. ## Config-blob compatibility @@ -81,7 +85,8 @@ Both new fields serialize only at non-default values, matching the repository's - a call with extra option keys flushes the queue first and dispatches solo, preserving order; - a throwing first handler does not prevent the second handler from running; - window `0` / absent config leaves per-call dispatch unchanged (existing suite must pass untouched); -- prefetch: response held and applied at the application point; unresolved prefetch falls back to the current request-then-apply path. +- prefetch: a stashed response is consumed by slim-Prebid initialization without a second `/auction` request; an absent or unresolved stash falls back to the current request path; +- held `requestBids` calls return a promise that settles with the merged auction. **Rust:** From 63fb2211e3439ba8b1dfcf9a948d129dd14b6a86 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 20 Aug 2026 21:53:15 -0700 Subject: [PATCH 03/12] Return spec to draft and address review findings Reframe Lever A as a load/QPS lever (measured burst POSTs were already concurrent) with absolute deadlines, per-caller bid-map partitioning, bookkeeping-before-callbacks ordering, synthetic-refresh and client-side-bidder exclusions, disjoint-code and payload-size admission bounds, and full promise semantics. Gate Lever B behind [integrations.gpt] gam_preconnect, correct the pubads_impl and crossorigin claims, and note the pre-consent connection decision. Demote Lever C to discovery-first: the described lifecycle seam does not exist, responses cannot be stashed without the requesting auction's bid-request IDs, consent readiness needs a concrete API contract that distinguishes consent-denied no-bids, and fallbacks must share the in-flight promise rather than duplicate auctions. Add a billing and impression integrity section, correct config-blob compatibility to the raw-JSON retention layer, and expand testing, measurement methodology, and one-lever-at-a-time rollout. --- ...-prebid-ad-latency-optimizations-design.md | 151 +++++++++++------- 1 file changed, 92 insertions(+), 59 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index 73910933f..83d69a77f 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -1,103 +1,136 @@ -# Prebid Ad-Latency Optimizations — Design +# Prebid Ad-Latency and Auction-Load Optimizations — Design **Date:** 2026-08-20 -**Status:** Approved design, pending implementation plan +**Status:** Draft (returned from review; Lever C requires a discovery phase before implementation) **Scope:** Client-side auction properties (server-side ad templates inactive) ## Problem -On properties running the client-side auction path (`creative_opportunities.enabled = false`), ads render slowly relative to what the pipeline allows. A three-run instrumented baseline against a local Trusted Server proxying the production autoblog origin (article page, consent resolved, reader-style scrolling) measured: +On properties running the client-side auction path (`creative_opportunities.enabled = false`), ad delivery leaves measurable headroom. A three-run instrumented baseline against a local Trusted Server proxying the production autoblog origin (article page, consent resolved, reader-style scrolling) measured: -| Milestone | Time | -| --------------------------------------- | ------ | -| Prebid bundle + shim fully installed | ~2.1 s | -| `DOMContentLoaded` | ~2.1 s | -| `window.load` | ~3.0 s | -| First Trusted Server `/auction` request | ~3.3 s | -| Publisher's first `requestBids` | ~4.8 s | -| First non-empty ad render | ~5.0 s | +| Milestone | Time | +| -------------------------------------------------------------------- | ------ | +| Prebid bundle + shim installed (deferred head scripts, execute ~DCL) | ~2.1 s | +| `DOMContentLoaded` | ~2.1 s | +| `window.load` | ~3.0 s | +| First Trusted Server `/auction` request | ~3.3 s | +| Publisher's first `requestBids` | ~4.8 s | +| First non-empty ad render | ~5.0 s | -Three structural costs stand out, all in Trusted-Server-owned surfaces: +Observed costs, each owned by a different lever below: -1. **The first refresh auction waits for `window.load`.** The post-`load` + double-`requestAnimationFrame` defer exists to avoid React hydration mismatches (React #418), but only the DOM application needs that defer — the auction network fetch is hydration-neutral. `DOMContentLoaded` fires ~1.2 s before `load` on this page, and the gap grows on resource-heavy pages. -2. **Publisher `requestBids` bursts pay one `/auction` round trip each.** The baseline captured two `requestBids` calls 1 ms apart producing two `/auction` POSTs 2 ms apart. Each call costs a full round trip plus ~865 ms median (p90 ~1075 ms) of server-side auction time. -3. **The first GAM ad request pays fresh connection setup.** `securepubads.g.doubleclick.net` serves both the `pubads_impl` script and every ad request; no connection is warmed before first use. +1. **The first Trusted Server auction fires ~1.2 s after `DOMContentLoaded`.** The exact trigger of the observed 3.3 s `/auction` request is not yet attributed (it did not pass through `pbjs.requestBids`, and `gpt.slim_prebid_url` is not configured on the measured property). Lever C starts with a discovery task to attribute it precisely. +2. **Publisher `requestBids` bursts issue one `/auction` POST each.** The baseline captured two calls 1 ms apart producing two POSTs 2 ms apart. The POSTs run **concurrently**, so this is a server-load and bidder-QPS cost, not a first-render latency cost. Lever A is therefore a load/cost lever, not a latency lever. +3. **The first direct GAM ad request pays fresh connection setup** to `securepubads.g.doubleclick.net`. GPT scripts themselves are first-party proxied (the script guard rewrites the cascade), so only the direct ad request path can benefit from a warmed connection. -Not addressed here (out of scope): re-enabling server-side ad templates, GPT lazy-load fetch margins (publisher-coordinated), the publisher's own ad-framework init latency (~1.8 s after `load` before their first `requestBids`), and server-side auction duration tuning (PBS `tmax`). +Out of scope: re-enabling server-side ad templates, GPT lazy-load fetch margins (publisher-coordinated), the publisher's own ad-framework init latency, and server-side auction duration tuning (PBS `tmax`). + +## Billing and impression integrity (applies to every lever) + +Nothing in this design may create impression, win, or billing signals for ads that never render in a slot a user could see: + +- Prefetched or early-dispatched auctions are **targeting-only**: they produce bids, never renders. `nurl`/`burl` and any render-bridge activity remain tied to an actual GAM render of the slot, exactly as today. +- A prefetched bid that is never consumed expires without firing any beacon. +- Coalescing must not cause a caller's handler to render or fire beacons for another caller's ad units (see the partitioning rule in Lever A). +- Rollout guardrails (below) monitor duplicate-auction rate and beacon counts per rendered impression so any integrity regression is visible immediately. ## Design -Three independent levers. Two are config-gated and default off, so shipping the binary changes nothing until an operator flips the flag; one is an unconditional resource hint. +### Lever A — `requestBids` coalescing window (opt-in; load/cost reduction) + +**Objective:** reduce `/auction` request count and upstream bidder QPS for bursty publisher call patterns. Explicitly **not** claimed to reduce first-render latency; the hold can only add up to the window duration for the earliest caller, and rollout must verify render latency does not regress. + +**Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected into `window.__tsjs_prebid` as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves current behavior byte-for-byte. + +**Admission (merge-safety) rules.** A call is held only when all of: -### Lever A — `requestBids` coalescing window (opt-in) +- its request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler`, with a non-empty explicit `adUnits` array (any other key — `ortb2`, `labels`, `adUnitCodes`, `ttlBuffer`, `auctionId`, … — flushes the pending queue synchronously, then dispatches solo, preserving call order); +- it is **not** one of the shim's own synthetic refresh auctions (their GPT watchdog deadline starts when the wrapper returns; holding them races the watchdog and can discard valid bids); +- none of its ad units contains a bid entry for a configured client-side bidder (merging would merge those bidders' native auctions too, changing their request shape and analytics; if operators later want that, it is a separate, explicit decision); +- every held ad-unit `code` is non-empty and **disjoint** from the codes already pending (the `/auction` payload builder collapses duplicate codes, keeping the first unit's media types — a merged duplicate would produce a hybrid auction neither caller requested); +- the projected serialized payload of the merged request stays under a bound with safety margin (the endpoint rejects bodies over 256 KiB; bound pending calls, total ad units, and projected bytes — flush before admitting a call that would exceed any bound). -**Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=500`. Serialized into the injected client config (`window.__tsjs_prebid`) as `requestBidsCoalesceMs`, omitted when `0`. +**Deadlines.** Each call's effective deadline is absolute: queue residence counts against it. A held call with timeout `T` arriving at `t0` must reach Prebid with an adjusted timeout of `T − (dispatch − t0)`. Calls merge only when their absolute deadlines are within a compatibility tolerance; an incompatible deadline flushes the queue. A call without a timeout uses the configured default for this computation. -**Behavior:** The tsjs Prebid shim already wraps `pbjs.requestBids` (bidder injection, snapshot capture, `bidsBackHandler` chaining). With a non-zero window, the wrapper holds a transformed call for up to the window duration and merges every mergeable call that arrives within it into one underlying `requestBids`: +**Dispatch.** One underlying `requestBids` with: the union of ad units in arrival order; the minimum adjusted deadline; and a combined `bidsBackHandler` that: -- **Merged request:** union of the pending calls' ad units (in arrival order), maximum of their `timeout`s (absent if none supplied), and a combined `bidsBackHandler` that invokes each pending call's (already-wrapped) handler in arrival order with the merged auction's results. A throwing handler must not prevent later handlers from running. -- **Merge-safety rule:** only calls whose request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler` (with a non-empty explicit `adUnits` array) are held. Any other call — extra keys such as `ortb2` or `labels`, or an implicit global-ad-units call — first flushes the pending queue synchronously, then dispatches solo. This preserves relative call order and never reinterprets options the merge logic does not understand. -- **Default `0`:** the wrapper dispatches immediately, byte-for-byte the current behavior. +1. first runs **all** Trusted Server bookkeeping for every constituent call (pending-publisher-bid registration, EID cookie sync) — before any publisher callback runs, so a publisher callback that immediately calls `pubads.refresh()` cannot observe a constituent call whose bookkeeping has not happened; +2. then invokes each caller's original handler in arrival order, passing a bid map **partitioned to that caller's ad-unit codes**, with throws isolated per handler. -**Return value:** Prebid 10's `requestBids` returns a promise, and the wrapper currently passes it through. Every held call therefore returns a promise that settles when the merged underlying `requestBids` promise settles, so callers awaiting the promise keep working; they observe the merged auction's completion rather than a per-call auction's. +**Promise and result semantics.** Prebid 10's `requestBids` returns a promise resolving `{bids, timedOut, auctionId}`. Every held call returns a promise that settles when the merged auction settles, with `bids` partitioned to the caller's codes and the shared `auctionId`. The shared auction id and merged event stream (one `auctionInit`/`auctionEnd` for N calls) are documented operator-visible changes; analytics consumers on the property see merged auctions. If the underlying dispatch throws synchronously, every pending promise rejects with that error and the queue resets. -### Lever B — GAM preconnect hint +### Lever B — GAM preconnect hint (opt-in) -The GPT integration's `head_inserts` emits, before its bootstrap scripts: +**Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false`. -```html - -``` +When enabled, GPT `head_inserts` emits `` at head-start. -Unconditional: it is a pure hint with no behavioral effect, and every GPT-enabled property talks to this host. Removes DNS + TCP + TLS setup from the first ad request (and benefits the `pubads_impl` fetch). +**Corrections from review, reflected in scope and claims:** -### Lever C — first-auction prefetch at `DOMContentLoaded` (opt-in) +- This opens a connection to Google at head-start, **before consent resolution and on pages that may never issue an ad request**. That is a deliberate per-property privacy decision — hence config-gated, default off, for operators whose consent posture permits it. +- GPT scripts (including `pubads_impl`) are first-party proxied by the script guard; the hint can only help the **first direct ad request**. The claim is "may reduce" its connection setup; verification requires a HAR demonstrating connection reuse under the chosen credential mode. +- Credential mode matters: ad requests are cookie-credentialed, so the hint is emitted **without** `crossorigin` (credentialed preconnect). Browsers may partially perform or skip hints; this lever is best-effort by nature. -**Config:** `[integrations.prebid] prefetch_first_refresh_auction` — `bool`, default `false`. Injected as `prefetchFirstRefreshAuction`, omitted when `false`. +### Lever C — earlier first auction (discovery first; design contingent) -**Mechanism today:** the `window.load` gate is `installSlimPrebidLoader` in the unified GPT bundle — the slim-Prebid module that runs refresh auctions is not even _loaded_ until `load` fires, and its first auction follows its post-load initialization. (The post-`load` + double-`rAF` defer from the React #418 fix belongs to the server-side-template `adInit` path and is not part of this flow.) +**Status: not implementation-ready.** Two review findings block a concrete design: the previously described lifecycle seam does not exist, and a raw `/auction` response cannot be "stashed and consumed" — the adapter maps responses back to Prebid bids via the requesting auction's Prebid-generated bid-request IDs. -**Behavior when enabled:** the unified GPT bundle — which is loaded from head-start — dispatches the first refresh auction's `/auction` request as soon as all of the following hold, without waiting for `window.load`: +**Phase 1 — discovery (required before any design commitment):** -- consent has resolved (the existing consent gate is unchanged), -- the GPT slots the auction would target have been observed, -- `DOMContentLoaded` has fired. +- Attribute the observed 3.3 s first `/auction` request precisely (it bypassed `pbjs.requestBids`; `slim_prebid_url` is unset on the measured property). Identify the triggering module, its gate (`load` listener, GPT event, publisher call), and what state it needs. +- Determine at what point the inputs a valid first auction needs are actually available: consent readiness (see below), GPT slot set with live sizes/targeting, Prebid EIDs (collected at adapter request time), and publisher bidder params. -The response is stashed on `window.tsjs`. The slim-Prebid module, still loaded at the unchanged `window.load` point, consumes the stash during its existing initialization instead of issuing a fresh request. Only the network round trip moves earlier; script loading, targeting application, and the GPT refresh that triggers rendering keep their current post-load timing, so no render work moves into the hydration window. +**Phase 2 — design options to evaluate against discovery output:** -**Alternative considered and rejected:** loading the slim-Prebid script itself at `DOMContentLoaded`. Simpler, but it would also pull GPT refresh — and therefore ad rendering into publisher containers — earlier into the hydration window, the same territory as React #418 and the ad-container hydration gating tracked in #969. +1. **Transport-level cache behind the real `requestBids` lifecycle:** the adapter's transport layer may reuse an in-flight or completed `/auction` HTTP exchange when — and only when — the newly transformed request is byte-identical in its auction-relevant signature. Cache entries carry: exact transformed-request signature, consent fingerprint, navigation generation, creation time, expiry, and one-shot consumed state. The response must expose a signal distinguishing consent-denied no-bid from legitimate no-bid (`/auction` currently returns HTTP 200 for both), and consent-denied responses are never cacheable. +2. **Classified full early auction:** run the real Prebid auction earlier and accept that scripts, events, consent modules, identity work, and native client-side bidders all move earlier. Honest but larger; interacts with hydration-window rendering (React #418, ad-container gating in #969) because earlier auctions pull GPT refresh earlier. -**Fallback:** if the stash is absent or its request has not resolved when slim-Prebid initializes, the module issues its own request exactly as today. A failed prefetch is discarded; no new error surface. +**Hard requirements for either option:** + +- **Consent readiness is a concrete API contract, not an assumption:** the trigger must consume the CMP signal (GPP `signalStatus: ready` / TCF `tcloaded`/`tcstring`, USP response) with a defined timeout and default action, and record the consent fingerprint used. The GPT bundle currently has no consent gate (it is documented as a future hook), so this gate must be built, not referenced. +- **No duplicate auctions:** an unresolved early request is **awaited** by the normal path (share the promise), never replaced by a second request. Cover pending→fallback→late-success, SPA navigation, and slot-destruction cases; a navigation or slot change invalidates the entry. +- **Billing integrity:** per the integrity section — early responses are targeting data only. ## Config-blob compatibility -Both new fields serialize only at non-default values, matching the repository's rollback discipline: a blob that never sets them is accepted by older binaries, and before rolling a binary back past this feature, an operator must clear the flags and re-push (the same procedure documented for prior `[integrations.prebid]` additions). +Integration settings are retained as raw JSON in the pushed blob (`IntegrationSettings` flattens into a `HashMap`), so an explicitly configured `0`/`false` **is** present in the blob; only omitted keys are absent. `PrebidIntegrationConfig` and `GptConfig` do not `deny_unknown_fields`, so older binaries tolerate blobs carrying the new keys — no clear-before-rollback step is required. Testing includes a new-schema blob parsed by the legacy struct shape to lock this in. ## Testing -**Vitest (shim):** +**Vitest (shim), Lever A:** + +- two mergeable calls in the window → one underlying `requestBids`, union ad units, per-caller partitioned bid maps, handlers in arrival order; +- bookkeeping-before-callbacks: constituent-call registration observable before the first publisher callback runs; +- non-mergeable option keys flush then dispatch solo, preserving order; +- synthetic refresh auctions are never held (watchdog interplay covered with fake timers); +- calls containing client-side bidder entries are never held; +- duplicate/conflicting ad-unit codes flush before admission; payload-bound boundary flushes; +- unequal and absent timeouts: absolute-deadline adjustment, queue time counted, incompatible deadlines flush; +- throwing first handler does not block later handlers; synchronous dispatch failure rejects all pending promises and resets the queue; +- held-call promise resolves `{bids (partitioned), timedOut, auctionId (shared)}`; +- window `0` / absent config: existing suite passes untouched; +- callback reentrancy: a handler calling `requestBids` during the combined callback dispatches correctly. -- coalescing merges two mergeable calls in the window into one underlying `requestBids` with the union of ad units, and invokes both handlers in order; -- a call with extra option keys flushes the queue first and dispatches solo, preserving order; -- a throwing first handler does not prevent the second handler from running; -- window `0` / absent config leaves per-call dispatch unchanged (existing suite must pass untouched); -- prefetch: a stashed response is consumed by slim-Prebid initialization without a second `/auction` request; an absent or unresolved stash falls back to the current request path; -- held `requestBids` calls return a promise that settles with the merged auction. +**Real-artifact coverage:** the external-bundle integration test asserts the wrapper's promise return against real Prebid (the current test discards the return value; the unit mock returns `undefined` and must not be the only coverage). **Rust:** -- config defaults (`request_bids_coalesce_ms = 0`, `prefetch_first_refresh_auction = false`) and the `0..=500` validation bound; -- injected-config serialization omits both fields at their defaults and includes them otherwise; -- GPT `head_inserts` contains the preconnect link. +- config defaults (`request_bids_coalesce_ms = 0`, `gam_preconnect = false`) and the `0..=250` bound; +- injected-config serialization omits `requestBidsCoalesceMs` at `0`; +- GPT `head_inserts` includes the preconnect link only when `gam_preconnect = true`, without `crossorigin`; +- new-schema → legacy-schema blob compatibility test. + +**Lever C tests are defined with its design after discovery** (consent denial/change, EID readiness, navigation/slot destruction, late completion, cache one-shot semantics). -**End-to-end verification:** re-run the local baseline harness (Viceroy proxying the production origin, pre-seeded consent, identical scroll script) with each flag enabled, comparing against the recorded baseline: time to first non-empty render, `/auction` request count, and burst dedup on the publisher's paired calls. +## Measurement methodology + +The three-run baseline motivates the work but does not gate it. Acceptance runs use: ≥10 runs per arm on the same machine and network, local Viceroy against the production origin with pre-seeded consent and an identical scripted scroll; medians compared, with a regression limit on first non-empty render (no worse than baseline median + 5%) and the target metric per lever (Lever A: `/auction` request count and burst dedup; Lever B: HAR-verified connection reuse on the first direct GAM request; Lever C: first-auction dispatch time). ## Rollout -1. Land binary with defaults off; the preconnect hint is the only immediate change. -2. Enable `request_bids_coalesce_ms` (initially 50 ms) and `prefetch_first_refresh_auction` on the autoblog tester property via config push. -3. Compare live tester metrics with the harness prediction; each flag reverts independently by config push, no binary rollback. +One lever at a time, each independently config-reversible: + +1. Land binary; all flags default off — zero behavior change. +2. Enable `request_bids_coalesce_ms` (50 ms) alone on the autoblog tester property. Guardrails: fill/revenue, bid rate, timeout rate, client-side bidder traffic (must be unchanged), duplicate-auction rate, beacons per rendered impression, per-slot render latency. +3. After A stabilizes, enable `gam_preconnect` alone; verify via HAR and consent-denied network activity monitoring (no pre-consent regressions beyond the documented connection). +4. Lever C follows its own spec revision after discovery. From 7bc12352002ced0569eb922bf62b37f0f0b0b38e Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 21 Aug 2026 00:24:14 -0700 Subject: [PATCH 04/12] Address second review round on ad-latency spec Gate Lever A behind a burst-trace prerequisite (the baseline never recorded the burst calls' option keys or codes; same-code duplicate auctions would make the disjoint-code rule merge nothing), reframe bidder-QPS reduction as a hypothesis, document the shared-auction global-targeting caveat, make the deadline algorithm executable (bidderTimeout captured at enqueue, 50ms tolerance, monotonic flush at min(windowEnd, earliestDeadline - 100ms), 50ms dispatch floor so Prebid's timeout||bidderTimeout fallback never sees 0), add dispatch-time revalidation against in-place mutation with exact batch bounds, keep per-caller registration IDs so throw-rollback semantics survive merging, and generalize the flush-first rule to every ineligible arrival. Rewrite billing integrity per delivery path (the PUC bridge is not this scope's boundary; requestAds couples fetch to render; PBS server-side notices may bill off-render). Name tsjs.requestAds as Lever C's leading discovery hypothesis and add cookie-parity consent, server-owned equivalence inputs, and server-owned cache tokens. Add Lever B governance (net-log proof, GPC and denied-CMP cases, named approver, insertion-order test). Make measurement paired and thresholded, note the telemetry arm-label gap and read-once config rollback semantics, and sanitize property naming. --- ...-prebid-ad-latency-optimizations-design.md | 153 ++++++++++-------- 1 file changed, 89 insertions(+), 64 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index 83d69a77f..3d8bf2af1 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -1,12 +1,12 @@ # Prebid Ad-Latency and Auction-Load Optimizations — Design -**Date:** 2026-08-20 -**Status:** Draft (returned from review; Lever C requires a discovery phase before implementation) -**Scope:** Client-side auction properties (server-side ad templates inactive) +**Date:** 2026-08-20 (revised 2026-08-21) +**Status:** Draft (Lever A gated on a burst-trace prerequisite; Lever C gated on discovery) +**Scope:** Client-side auction properties (server-side ad templates inactive). Measurements in this spec come from a pilot news property; identifying details are kept out of this document per repository policy, and sanitized measurement artifacts live outside the spec. ## Problem -On properties running the client-side auction path (`creative_opportunities.enabled = false`), ad delivery leaves measurable headroom. A three-run instrumented baseline against a local Trusted Server proxying the production autoblog origin (article page, consent resolved, reader-style scrolling) measured: +On properties running the client-side auction path (`creative_opportunities.enabled = false`), ad delivery leaves measurable headroom. A three-run instrumented baseline against a local Trusted Server proxying the pilot property's origin (article page, consent resolved, reader-style scrolling) measured: | Milestone | Time | | -------------------------------------------------------------------- | ------ | @@ -17,120 +17,145 @@ On properties running the client-side auction path (`creative_opportunities.enab | Publisher's first `requestBids` | ~4.8 s | | First non-empty ad render | ~5.0 s | -Observed costs, each owned by a different lever below: +Observed costs, each owned by a lever below: -1. **The first Trusted Server auction fires ~1.2 s after `DOMContentLoaded`.** The exact trigger of the observed 3.3 s `/auction` request is not yet attributed (it did not pass through `pbjs.requestBids`, and `gpt.slim_prebid_url` is not configured on the measured property). Lever C starts with a discovery task to attribute it precisely. -2. **Publisher `requestBids` bursts issue one `/auction` POST each.** The baseline captured two calls 1 ms apart producing two POSTs 2 ms apart. The POSTs run **concurrently**, so this is a server-load and bidder-QPS cost, not a first-render latency cost. Lever A is therefore a load/cost lever, not a latency lever. +1. **The first Trusted Server auction fires ~1.2 s after `DOMContentLoaded`** and did not pass through `pbjs.requestBids`. The leading in-repo hypothesis is the `window.tsjs.requestAds` path, which builds its payload from the TSJS registry, POSTs `/auction` directly, and **immediately renders returned creatives** — fetch and render are coupled there. Lever C's discovery must confirm or refute this before any design commitment. +2. **Publisher `requestBids` bursts issue one `/auction` POST each.** The baseline captured two calls 1 ms apart producing two POSTs 2 ms apart. The POSTs run **concurrently**, so this is a load-reduction hypothesis, not a first-render latency lever. The baseline did **not** record the burst calls' option keys, ad-unit codes, bidder entries, timeouts, or payload sizes — the exact properties that decide merge eligibility — so Lever A carries a trace prerequisite (below). 3. **The first direct GAM ad request pays fresh connection setup** to `securepubads.g.doubleclick.net`. GPT scripts themselves are first-party proxied (the script guard rewrites the cascade), so only the direct ad request path can benefit from a warmed connection. Out of scope: re-enabling server-side ad templates, GPT lazy-load fetch margins (publisher-coordinated), the publisher's own ad-framework init latency, and server-side auction duration tuning (PBS `tmax`). ## Billing and impression integrity (applies to every lever) -Nothing in this design may create impression, win, or billing signals for ads that never render in a slot a user could see: +Nothing in this design may create impression, win, or billing signals for ads that never render in a slot a user could see. Because billing signals differ per delivery path, the requirement is stated per path: -- Prefetched or early-dispatched auctions are **targeting-only**: they produce bids, never renders. `nurl`/`burl` and any render-bridge activity remain tied to an actual GAM render of the slot, exactly as today. -- A prefetched bid that is never consumed expires without firing any beacon. -- Coalescing must not cause a caller's handler to render or fire beacons for another caller's ad units (see the partitioning rule in Lever A). -- Rollout guardrails (below) monitor duplicate-auction rate and beacon counts per rendered impression so any integrity regression is visible immediately. +- **Client `/auction` → Prebid adapter path:** the `/auction` response serializer does not propagate explicit `nurl`/`burl` to this consumer, and win notification is owned by Prebid/GAM rendering. Early or coalesced auctions on this path are targeting-only by construction; the invariant to preserve is that no lever triggers `pbjs` render or GPT refresh for units the publisher did not ask to render. +- **`tsjs.requestAds` path:** fetch and render are currently coupled. If discovery selects this path for Lever C, fetch must be **split from render** first; an early fetch must never trigger its render half. +- **Server-side notices:** some PBS deployments fire win/billing notices server-side, outside browser control. Any early-auction design must state whether the upstream configuration can bill on auction rather than render; properties where that is true are **excluded** from early auctions until the upstream policy is confirmed render-tied (OpenRTB leaves billing timing exchange-specific). +- **The PUC render bridge** (which fires beacons after posting a creative response, without proof of pixel render) consumes the server-template `tsjs.bids` path — inactive in this scope. It is listed here only to record that its beacon semantics are not the integrity boundary this spec relies on. +- A prefetched bid that is never consumed expires without firing any beacon; coalescing must not cause one caller's handler to render another caller's units (partitioning rule in Lever A, with the global-state caveat below). +- Guardrails use **per-path computable signals** (Lever A: `/auction` count vs. rendered-slot count from the harness; no cross-path "beacons per impression" universal metric is claimed). ## Design -### Lever A — `requestBids` coalescing window (opt-in; load/cost reduction) +### Lever A — `requestBids` coalescing window (opt-in; load-reduction hypothesis) -**Objective:** reduce `/auction` request count and upstream bidder QPS for bursty publisher call patterns. Explicitly **not** claimed to reduce first-render latency; the hold can only add up to the window duration for the earliest caller, and rollout must verify render latency does not regress. +**Phase 0 — trace prerequisite (blocks implementation).** Capture a sanitized trace of the production burst calls: full option-key set, per-call ad-unit codes and bid entries, effective timeouts, projected payload sizes, and callback behavior. Implementation proceeds only if the observed calls satisfy every admission predicate below. If the burst turns out to be **same-code duplicate auctions** (which the shim supports today and the disjoint-code rule deliberately refuses to merge), Lever A as specified reduces nothing; the follow-up decision is then identical-request deduplication as a separate design, or dropping the lever. -**Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected into `window.__tsjs_prebid` as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves current behavior byte-for-byte. +**Objective:** reduce `/auction` request count for bursty, disjoint-unit publisher call patterns. Downstream bidder-call reduction is a **hypothesis to measure**, not a claim: one PBS request with multiple impressions does not guarantee every PBS bidder adapter issues fewer HTTP calls. Explicitly not a first-render latency lever; rollout must verify render latency does not regress. -**Admission (merge-safety) rules.** A call is held only when all of: +**Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves current behavior byte-for-byte. -- its request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler`, with a non-empty explicit `adUnits` array (any other key — `ortb2`, `labels`, `adUnitCodes`, `ttlBuffer`, `auctionId`, … — flushes the pending queue synchronously, then dispatches solo, preserving call order); -- it is **not** one of the shim's own synthetic refresh auctions (their GPT watchdog deadline starts when the wrapper returns; holding them races the watchdog and can discard valid bids); -- none of its ad units contains a bid entry for a configured client-side bidder (merging would merge those bidders' native auctions too, changing their request shape and analytics; if operators later want that, it is a separate, explicit decision); -- every held ad-unit `code` is non-empty and **disjoint** from the codes already pending (the `/auction` payload builder collapses duplicate codes, keeping the first unit's media types — a merged duplicate would produce a hybrid auction neither caller requested); -- the projected serialized payload of the merged request stays under a bound with safety margin (the endpoint rejects bodies over 256 KiB; bound pending calls, total ad units, and projected bytes — flush before admitting a call that would exceed any bound). +**Admission rules.** A call is held only when all of: -**Deadlines.** Each call's effective deadline is absolute: queue residence counts against it. A held call with timeout `T` arriving at `t0` must reach Prebid with an adjusted timeout of `T − (dispatch − t0)`. Calls merge only when their absolute deadlines are within a compatibility tolerance; an incompatible deadline flushes the queue. A call without a timeout uses the configured default for this computation. +- its request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler`, with a non-empty explicit `adUnits` array; +- `timeout` is absent or a finite positive integer (zero, negative, `NaN`, `Infinity`, or non-number values dispatch solo unchanged); +- it is not one of the shim's own synthetic refresh auctions (their GPT watchdog starts when the wrapper returns); +- no ad unit contains a bid entry for a configured client-side bidder; +- ad-unit codes are non-empty strings, **unique within the call**, and **disjoint from every code already pending** (the `/auction` payload builder collapses duplicate codes, keeping the first unit's media types); +- batch bounds hold after admission: at most 4 pending calls, at most 32 total ad units, and a projected serialized payload of at most 192 KiB UTF-8 (64 KiB safety margin under the endpoint's 256 KiB limit). -**Dispatch.** One underlying `requestBids` with: the union of ad units in arrival order; the minimum adjusted deadline; and a combined `bidsBackHandler` that: +**Ineligible arrivals — one rule for all of them:** any call that fails any admission predicate (extra option keys, synthetic refresh, client-side bidders, invalid timeout, code collision, size overflow, serialization failure) **first synchronously flushes the pending batch, then dispatches solo**. Nothing ever overtakes an earlier caller; event order is preserved. -1. first runs **all** Trusted Server bookkeeping for every constituent call (pending-publisher-bid registration, EID cookie sync) — before any publisher callback runs, so a publisher callback that immediately calls `pubads.refresh()` cannot observe a constituent call whose bookkeeping has not happened; -2. then invokes each caller's original handler in arrival order, passing a bid map **partitioned to that caller's ad-unit codes**, with throws isolated per handler. +**Deadlines.** -**Promise and result semantics.** Prebid 10's `requestBids` returns a promise resolving `{bids, timedOut, auctionId}`. Every held call returns a promise that settles when the merged auction settles, with `bids` partitioned to the caller's codes and the shared `auctionId`. The shared auction id and merged event stream (one `auctionInit`/`auctionEnd` for N calls) are documented operator-visible changes; analytics consumers on the property see merged auctions. If the underlying dispatch throws synchronously, every pending promise rejects with that error and the queue resets. +- At enqueue, capture the live `pbjs.getConfig('bidderTimeout')`; a call without a timeout uses that captured value for deadline math. +- Each call's deadline is absolute: `arrival + effectiveTimeout`. Queue residence counts against it. +- Calls merge only when absolute deadlines agree within a **50 ms tolerance**; an incompatible deadline flushes the queue first. Later-arriving compatible callers accept the batch's earlier shared deadline and the shared `timedOut` result — documented behavior. +- A monotonic scheduler flushes at `min(windowEnd, earliestDeadline − 100 ms safety margin)` and re-arms if a new caller tightens the earliest deadline. +- The dispatched timeout is `earliestDeadline − now`, floored at **50 ms**; it is never `0` or negative (Prebid evaluates `timeout || bidderTimeout`, so `0` would silently restore the full global timeout). If the event loop wakes past a deadline (timer throttling, long tasks), dispatch immediately with the floor. + +**Dispatch-time revalidation.** The shim mutates publisher ad-unit objects in place, and publishers can mutate them further while a call is held; the adapter also builds the final payload (including then-current EIDs) only at dispatch. Every admission predicate — option keys, codes, bidders, bounds, projected serialized size — is therefore **re-checked at dispatch** against the live objects. A call that no longer qualifies is evicted from the batch and dispatched solo (current behavior); values that fail serialization (cycles, throwing getters/`toJSON`) are treated the same way. + +**Dispatch.** One underlying `requestBids` with: the union of ad units in arrival order; the floored shared deadline; and a combined `bidsBackHandler` that: + +1. runs Trusted Server bookkeeping for every constituent call first — each call keeps **its own registration ID**, so the existing throw-rollback semantics are preserved per caller; +2. invokes each caller's original handler in arrival order with callback `this` and the exact three arguments `(bids, timedOut, auctionId)`, with `bids` partitioned to that caller's codes and `timedOut`/`auctionId` shared. A throwing handler rolls back **only its own** registration (matching today's single-call behavior) and does not block later handlers. + +**Global-state caveat (documented behavior change).** Partitioned callbacks do not partition Prebid's global auction state: all merged bids belong to one auction, so a publisher callback that calls `pbjs.setTargetingForGPTAsync()` without codes applies targeting for every returned unit, and a bare `pubads.refresh()` can deliver another caller's units — all constituent calls' bookkeeping is registered before callbacks precisely so such a refresh is attributed as publisher delivery for every affected unit. Operators enabling the flag accept this shared-auction visibility; the test plan exercises unscoped targeting plus bare and mixed-slot refreshes from the first callback. + +**Promise and result semantics.** Every held call returns a promise settling with `{bids (partitioned), timedOut (shared), auctionId (shared)}`. A synchronous dispatch failure rejects all pending promises with that error; asynchronous rejection of the underlying promise fans out to all held promises; the queue resets in a `finally`. One `auctionInit`/`auctionEnd` event stream replaces N — a documented, operator-visible analytics change. ### Lever B — GAM preconnect hint (opt-in) **Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false`. -When enabled, GPT `head_inserts` emits `` at head-start. +When enabled, GPT `head_inserts` emits `` **before the GPT bootstrap inserts** (asserted by a transformed-HTML ordering test), without `crossorigin` — ad requests are cookie-credentialed, and the HTML preconnect algorithm keeps credentialed and anonymous connections distinct. Browsers may partially perform or skip hints; best-effort by nature. -**Corrections from review, reflected in scope and claims:** +**Scope of claim:** GPT scripts (including `pubads_impl`) are first-party proxied by the script guard; the hint can only affect the **first direct ad request**, and the claim is "may reduce" its connection setup. -- This opens a connection to Google at head-start, **before consent resolution and on pages that may never issue an ad request**. That is a deliberate per-property privacy decision — hence config-gated, default off, for operators whose consent posture permits it. -- GPT scripts (including `pubads_impl`) are first-party proxied by the script guard; the hint can only help the **first direct ad request**. The claim is "may reduce" its connection setup; verification requires a HAR demonstrating connection reuse under the chosen credential mode. -- Credential mode matters: ad requests are cookie-credentialed, so the hint is emitted **without** `crossorigin` (credentialed preconnect). Browsers may partially perform or skip hints; this lever is best-effort by nature. +**Governance (required before any property enables it):** -### Lever C — earlier first auction (discovery first; design contingent) +- A per-property, per-jurisdiction approval that pre-consent DNS/TCP/TLS/SNI contact with Google is permitted, with a named approver — this hint fires at head-start on pages that may never issue an ad request. +- Verification uses browser **net logging**, not HAR alone: prove connection reuse by the first ad request under the credentialed mode, and prove **zero HTTP request bytes** are transmitted during speculation, including with GPC set and with the CMP unresolved or denied. +- Rollback trigger: any observed HTTP request on the speculative connection before the normal ad request disables the flag. -**Status: not implementation-ready.** Two review findings block a concrete design: the previously described lifecycle seam does not exist, and a raw `/auction` response cannot be "stashed and consumed" — the adapter maps responses back to Prebid bids via the requesting auction's Prebid-generated bid-request IDs. +**Config-surface note:** environment overrides cannot create absent GPT config leaves today (documented in the GPT guide); enabling per-property therefore goes through the pushed app-config blob, and the implementation updates `trusted-server.example.toml`, the configuration tables, and the GPT/Prebid guides. -**Phase 1 — discovery (required before any design commitment):** +### Lever C — earlier first auction (discovery first; design contingent) -- Attribute the observed 3.3 s first `/auction` request precisely (it bypassed `pbjs.requestBids`; `slim_prebid_url` is unset on the measured property). Identify the triggering module, its gate (`load` listener, GPT event, publisher call), and what state it needs. -- Determine at what point the inputs a valid first auction needs are actually available: consent readiness (see below), GPT slot set with live sizes/targeting, Prebid EIDs (collected at adapter request time), and publisher bidder params. +**Status: not implementation-ready.** -**Phase 2 — design options to evaluate against discovery output:** +**Phase 1 — discovery (required):** -1. **Transport-level cache behind the real `requestBids` lifecycle:** the adapter's transport layer may reuse an in-flight or completed `/auction` HTTP exchange when — and only when — the newly transformed request is byte-identical in its auction-relevant signature. Cache entries carry: exact transformed-request signature, consent fingerprint, navigation generation, creation time, expiry, and one-shot consumed state. The response must expose a signal distinguishing consent-denied no-bid from legitimate no-bid (`/auction` currently returns HTTP 200 for both), and consent-denied responses are never cacheable. -2. **Classified full early auction:** run the real Prebid auction earlier and accept that scripts, events, consent modules, identity work, and native client-side bidders all move earlier. Honest but larger; interacts with hydration-window rendering (React #418, ad-container gating in #969) because earlier auctions pull GPT refresh earlier. +- Attribute the 3.3 s `/auction` request precisely. **Named hypothesis:** the `tsjs.requestAds` path (posts `/auction` directly from the TSJS registry and immediately renders returned creatives). If confirmed, both Phase 2 options below are mis-scoped as written: moving `requestAds` earlier moves **rendering** earlier too, and its registry-built payload is not interchangeable with a later Prebid-adapter auction. The design must then first split fetch from render, and decide whether an independent non-Prebid auction on the page should be deduplicated or removed rather than accelerated. +- Identify the actual transport owner for any "adapter transport cache": today the Prebid adapter returns a request descriptor and **Prebid core owns the HTTP operation**; the repository-owned `sendAuction` belongs to the separate core API. A cache needs a named interception point in one of those owners. +- Determine when a valid first auction's inputs exist: consent (see below), GPT slot set with live sizes/targeting, Prebid EIDs (collected at adapter request time), publisher bidder params. -**Hard requirements for either option:** +**Consent and equivalence model (server-owned inputs included):** -- **Consent readiness is a concrete API contract, not an assumption:** the trigger must consume the CMP signal (GPP `signalStatus: ready` / TCF `tcloaded`/`tcstring`, USP response) with a defined timeout and default action, and record the consent fingerprint used. The GPT bundle currently has no consent gate (it is documented as a future hook), so this gate must be built, not referenced. -- **No duplicate auctions:** an unresolved early request is **awaited** by the normal path (share the promise), never replaced by a second request. Cover pending→fallback→late-success, SPA navigation, and slot-destruction cases; a navigation or slot change invalidates the entry. -- **Billing integrity:** per the integrity section — early responses are targeting data only. +- CMP readiness in the browser is not the state the server consumes: the `/auction` body carries no consent envelope; the server reconstructs consent from cookies and `Sec-GPC`. A CMP can report granted while its cookie is absent or stale, so an early request could receive a consent-denied no-bid. Discovery must define how CMP readiness becomes the exact server-visible state — **cookie parity or an explicit validated consent envelope** — before any early dispatch. +- `/auction` returns HTTP 200 for both consent-denied and legitimate no-bid; a distinguishing response signal is required, and consent-denied responses are never reusable. +- Request equivalence is **not** body-byte equality: the server consumes EC identity, `ts-eids` fallback, KV-resolved EIDs, geo, IP, user agent, page identity, consent policy, and server configuration outside the body. Completed-response reuse is rejected unless the server provides a bounded, server-owned cache token covering those inputs. In-flight sharing ("no duplicate auctions": the normal path awaits the early request's promise) applies only while the equivalence snapshot — including consent and identity — is unchanged; a consent or identity change while pending invalidates/aborts the early request and runs a fresh normal one. +- Cache entries (if any) carry: transformed-request signature, consent fingerprint, navigation generation, creation time, expiry, one-shot consumed state. +- Billing integrity per the section above; on the `requestAds` path, split fetch from render before any reuse. ## Config-blob compatibility -Integration settings are retained as raw JSON in the pushed blob (`IntegrationSettings` flattens into a `HashMap`), so an explicitly configured `0`/`false` **is** present in the blob; only omitted keys are absent. `PrebidIntegrationConfig` and `GptConfig` do not `deny_unknown_fields`, so older binaries tolerate blobs carrying the new keys — no clear-before-rollback step is required. Testing includes a new-schema blob parsed by the legacy struct shape to lock this in. +Integration settings are retained as raw JSON in the pushed blob (`IntegrationSettings` flattens into a `HashMap`), so an explicitly configured `0`/`false` is present in the blob; only omitted keys are absent. `PrebidIntegrationConfig` and `GptConfig` do not `deny_unknown_fields`, so older binaries tolerate blobs carrying the new keys — no clear-before-rollback step. Testing includes a new-schema blob parsed by the legacy struct shape. + +**Runtime note:** pages read the injected Prebid config once at load; a config rollback affects new navigations, not already-open pages. ## Testing **Vitest (shim), Lever A:** -- two mergeable calls in the window → one underlying `requestBids`, union ad units, per-caller partitioned bid maps, handlers in arrival order; -- bookkeeping-before-callbacks: constituent-call registration observable before the first publisher callback runs; -- non-mergeable option keys flush then dispatch solo, preserving order; -- synthetic refresh auctions are never held (watchdog interplay covered with fake timers); -- calls containing client-side bidder entries are never held; -- duplicate/conflicting ad-unit codes flush before admission; payload-bound boundary flushes; -- unequal and absent timeouts: absolute-deadline adjustment, queue time counted, incompatible deadlines flush; -- throwing first handler does not block later handlers; synchronous dispatch failure rejects all pending promises and resets the queue; -- held-call promise resolves `{bids (partitioned), timedOut, auctionId (shared)}`; -- window `0` / absent config: existing suite passes untouched; -- callback reentrancy: a handler calling `requestBids` during the combined callback dispatches correctly. +- two mergeable calls → one underlying `requestBids`, union units, per-caller partitioned maps, handlers in arrival order with preserved `this` and exact `(bids, timedOut, auctionId)` arguments; +- bookkeeping-before-callbacks with **per-caller registration IDs**: a throwing first handler rolls back only its own registration, later handlers still run, and the existing throw-rollback regression test still passes; +- every ineligible-arrival class (extra keys, synthetic refresh, client-side bidders, invalid timeout, duplicate/colliding codes, oversized payload, serialization failure) flushes the pending batch first, then dispatches solo — ordering asserted; +- global-state scenario: first callback performs unscoped `setTargetingForGPTAsync()` plus bare and mixed-slot `refresh()`; asserted against the documented shared-auction behavior; +- deadline math: timeout shorter than the window, zero/negative/`NaN` timeouts dispatch solo, `bidderTimeout` captured at enqueue survives a mid-hold config change, later caller with earlier deadline re-arms the scheduler, timer overshoot dispatches with the 50 ms floor (never `0`); +- dispatch-time revalidation: in-place mutation during the hold (code change, added client bidder, size growth) evicts to solo dispatch; cyclic/getter/`toJSON` failures evict to solo; +- promise semantics: async rejection fan-out, queue reset in `finally`, window `0`/absent config leaves the existing suite untouched; +- callback reentrancy: a handler calling `requestBids` during the combined callback. -**Real-artifact coverage:** the external-bundle integration test asserts the wrapper's promise return against real Prebid (the current test discards the return value; the unit mock returns `undefined` and must not be the only coverage). +**Real-artifact coverage (external bundle):** drive two real calls and prove one fetch, two thenables, callback-before-promise ordering, partitioned results, a single event sequence, shared timeout/auction-id semantics, and rejection fan-out. (The unit mock returns `undefined` and the current artifact test discards the return value; neither may be the only promise coverage.) **Rust:** -- config defaults (`request_bids_coalesce_ms = 0`, `gam_preconnect = false`) and the `0..=250` bound; -- injected-config serialization omits `requestBidsCoalesceMs` at `0`; -- GPT `head_inserts` includes the preconnect link only when `gam_preconnect = true`, without `crossorigin`; +- config defaults and bounds; injected-config serialization omits `requestBidsCoalesceMs` at `0`; +- `gam_preconnect = true` emits the link without `crossorigin` and **before** the GPT bootstrap inserts (transformed-HTML ordering test); `false` emits nothing; - new-schema → legacy-schema blob compatibility test. -**Lever C tests are defined with its design after discovery** (consent denial/change, EID readiness, navigation/slot destruction, late completion, cache one-shot semantics). +**Documentation checklist:** `trusted-server.example.toml`, configuration tables, Prebid and GPT integration guides, environment-overlay behavior note for GPT leaves. + +**Lever C tests are defined with its design after discovery.** ## Measurement methodology -The three-run baseline motivates the work but does not gate it. Acceptance runs use: ≥10 runs per arm on the same machine and network, local Viceroy against the production origin with pre-seeded consent and an identical scripted scroll; medians compared, with a regression limit on first non-empty render (no worse than baseline median + 5%) and the target metric per lever (Lever A: `/auction` request count and burst dedup; Lever B: HAR-verified connection reuse on the first direct GAM request; Lever C: first-auction dispatch time). +The three-run baseline motivates the work but does not gate it. Acceptance uses a reproducible harness (local Viceroy against the pilot origin, pre-seeded consent, identical scripted scroll) with **alternating paired control/treatment arms** on the same machine and network, explicit warm/cold connection conditions, ≥10 pairs per comparison, reporting p50/p95 and dispersion. + +Numeric gates: + +- **Lever A retains if:** ≥30% of observed eligible bursts merge (per the Phase 0 trace definition), total `/auction` requests per page drop ≥20%, and first-non-empty-render p95 regresses <5% vs. paired control. Otherwise the flag returns to `0`. +- **Lever B retains if:** net logs show connection reuse on the first direct GAM ad request in ≥70% of cold-start runs with zero pre-ad-request HTTP bytes; rollback on any integrity violation regardless of performance. +- Server telemetry currently labels all POSTs `auction_api` with no experiment-arm or navigation join key, so acceptance is **harness-based**; live guardrails during rollout are property-level trends (fill/revenue, bid rate, timeout rate, client-bidder traffic, duplicate-auction rate, per-slot render latency, consent-denied network activity) with a minimum 7-day hold per lever. ## Rollout -One lever at a time, each independently config-reversible: +One lever at a time, each independently config-reversible (new navigations only, per the runtime note): 1. Land binary; all flags default off — zero behavior change. -2. Enable `request_bids_coalesce_ms` (50 ms) alone on the autoblog tester property. Guardrails: fill/revenue, bid rate, timeout rate, client-side bidder traffic (must be unchanged), duplicate-auction rate, beacons per rendered impression, per-slot render latency. -3. After A stabilizes, enable `gam_preconnect` alone; verify via HAR and consent-denied network activity monitoring (no pre-consent regressions beyond the documented connection). +2. Lever A Phase 0 trace on the pilot property. If predicates hold: enable `request_bids_coalesce_ms = 50` alone; hold ≥7 days against the guardrails; on failure return to `0` and **drain** (confirm zero merged auctions) before any further lever. +3. With A either retained-and-stable or fully drained to `0`: enable `gam_preconnect` alone under its governance contract. 4. Lever C follows its own spec revision after discovery. From e2c8de5e5bddb264c1bae73b752060d964fec2f2 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 21 Aug 2026 00:50:24 -0700 Subject: [PATCH 05/12] Address third review round on ad-latency spec Resolve the integrity contradiction by consistently accepting cross-caller delivery within a merged auction (forbidden only for units no caller requested) and adding callback-discipline capture plus the pending-state/synthetic-auction net-effect estimate to Phase 0. Make the queue reentrancy-safe via atomic batch detachment before dispatch, reframe deadlines as a reduced auction-time budget with a solo-dispatch threshold instead of an absolute deadline, define order-preserving contiguous-segment eviction, snapshot request shape and ad-unit array order at enqueue, define the payload projection with a reserved adapter allowance and a 413 re-dispatch-solo-once fallback, and pass through undefined cancelled-auction values with handler throws reported by async rethrow rather than facade rejection. Scope Lever B approval to every jurisdiction the deployed config serves, pin the NetLog protocol, and narrow the invariant to no request HEADERS/DATA frames. Strengthen Lever C discovery with server-side single-flight preference, no completed-response reuse (the adapter re-stamps ttl 300), a typed direct-path result, and input-ownership tables, and remove the dangling Phase 2 reference. Add the adapter activation matrix (Fastly-only pilot), the guardrail source table with a new coalesced-count signal, drain over a 48h document lifetime, randomized paired methodology with power-based sample sizes, and a real latency-benefit gate for B. --- ...-prebid-ad-latency-optimizations-design.md | 143 ++++++++++-------- 1 file changed, 82 insertions(+), 61 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index 3d8bf2af1..9bfc20f80 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -1,8 +1,8 @@ # Prebid Ad-Latency and Auction-Load Optimizations — Design -**Date:** 2026-08-20 (revised 2026-08-21) +**Date:** 2026-08-20 (revised 2026-08-21, round 3) **Status:** Draft (Lever A gated on a burst-trace prerequisite; Lever C gated on discovery) -**Scope:** Client-side auction properties (server-side ad templates inactive). Measurements in this spec come from a pilot news property; identifying details are kept out of this document per repository policy, and sanitized measurement artifacts live outside the spec. +**Scope:** Client-side auction properties (server-side ad templates inactive). Measurements come from a pilot news property; identifying details stay out of this document per repository policy, and sanitized measurement artifacts live outside the spec. The pilot rollout is scoped to the Fastly adapter (see the activation matrix). ## Problem @@ -20,61 +20,70 @@ On properties running the client-side auction path (`creative_opportunities.enab Observed costs, each owned by a lever below: 1. **The first Trusted Server auction fires ~1.2 s after `DOMContentLoaded`** and did not pass through `pbjs.requestBids`. The leading in-repo hypothesis is the `window.tsjs.requestAds` path, which builds its payload from the TSJS registry, POSTs `/auction` directly, and **immediately renders returned creatives** — fetch and render are coupled there. Lever C's discovery must confirm or refute this before any design commitment. -2. **Publisher `requestBids` bursts issue one `/auction` POST each.** The baseline captured two calls 1 ms apart producing two POSTs 2 ms apart. The POSTs run **concurrently**, so this is a load-reduction hypothesis, not a first-render latency lever. The baseline did **not** record the burst calls' option keys, ad-unit codes, bidder entries, timeouts, or payload sizes — the exact properties that decide merge eligibility — so Lever A carries a trace prerequisite (below). +2. **Publisher `requestBids` bursts issue one `/auction` POST each.** The baseline captured two calls 1 ms apart producing two POSTs 2 ms apart. The POSTs run **concurrently**, so this is a load-reduction hypothesis, not a first-render latency lever. The baseline did **not** record the burst calls' option keys, ad-unit codes, bidder entries, timeouts, payload sizes, or callback behavior — the properties that decide merge eligibility — so Lever A carries a trace prerequisite (below). 3. **The first direct GAM ad request pays fresh connection setup** to `securepubads.g.doubleclick.net`. GPT scripts themselves are first-party proxied (the script guard rewrites the cascade), so only the direct ad request path can benefit from a warmed connection. Out of scope: re-enabling server-side ad templates, GPT lazy-load fetch margins (publisher-coordinated), the publisher's own ad-framework init latency, and server-side auction duration tuning (PBS `tmax`). ## Billing and impression integrity (applies to every lever) -Nothing in this design may create impression, win, or billing signals for ads that never render in a slot a user could see. Because billing signals differ per delivery path, the requirement is stated per path: +Nothing in this design may create impression, win, or billing signals for ad units that **no caller requested to auction**. Signals differ per delivery path: -- **Client `/auction` → Prebid adapter path:** the `/auction` response serializer does not propagate explicit `nurl`/`burl` to this consumer, and win notification is owned by Prebid/GAM rendering. Early or coalesced auctions on this path are targeting-only by construction; the invariant to preserve is that no lever triggers `pbjs` render or GPT refresh for units the publisher did not ask to render. +- **Client `/auction` → Prebid adapter path:** the `/auction` response serializer does not propagate explicit `nurl`/`burl` to this consumer; win notification is owned by Prebid/GAM rendering. Early or coalesced auctions on this path are targeting-only by construction. Within a merged auction, **cross-caller delivery is accepted behavior** (see Lever A's shared-auction section): every merged unit was requested by _some_ constituent caller, and a publisher callback using unscoped targeting or a bare refresh may deliver units from a co-merged caller. What remains forbidden is delivery of units absent from every constituent call. - **`tsjs.requestAds` path:** fetch and render are currently coupled. If discovery selects this path for Lever C, fetch must be **split from render** first; an early fetch must never trigger its render half. - **Server-side notices:** some PBS deployments fire win/billing notices server-side, outside browser control. Any early-auction design must state whether the upstream configuration can bill on auction rather than render; properties where that is true are **excluded** from early auctions until the upstream policy is confirmed render-tied (OpenRTB leaves billing timing exchange-specific). -- **The PUC render bridge** (which fires beacons after posting a creative response, without proof of pixel render) consumes the server-template `tsjs.bids` path — inactive in this scope. It is listed here only to record that its beacon semantics are not the integrity boundary this spec relies on. -- A prefetched bid that is never consumed expires without firing any beacon; coalescing must not cause one caller's handler to render another caller's units (partitioning rule in Lever A, with the global-state caveat below). -- Guardrails use **per-path computable signals** (Lever A: `/auction` count vs. rendered-slot count from the harness; no cross-path "beacons per impression" universal metric is claimed). +- **The PUC render bridge** (which fires beacons after posting a creative response, without proof of pixel render) consumes the server-template `tsjs.bids` path — inactive in this scope; its beacon semantics are not this spec's integrity boundary. +- A prefetched bid that is never consumed expires without firing any beacon. +- Guardrails use **per-path computable signals** (Lever A: `/auction` count vs. rendered-slot count from the harness); no cross-path "beacons per impression" universal metric is claimed. ## Design ### Lever A — `requestBids` coalescing window (opt-in; load-reduction hypothesis) -**Phase 0 — trace prerequisite (blocks implementation).** Capture a sanitized trace of the production burst calls: full option-key set, per-call ad-unit codes and bid entries, effective timeouts, projected payload sizes, and callback behavior. Implementation proceeds only if the observed calls satisfy every admission predicate below. If the burst turns out to be **same-code duplicate auctions** (which the shim supports today and the disjoint-code rule deliberately refuses to merge), Lever A as specified reduces nothing; the follow-up decision is then identical-request deduplication as a separate design, or dropping the lever. +**Phase 0 — trace prerequisite (blocks implementation).** Capture a sanitized trace of the production burst calls recording: full option-key set, per-call ad-unit codes and bid entries, effective timeouts, projected payload sizes, and **callback behavior** — specifically whether callbacks use code-scoped targeting and slot-scoped refresh, or unscoped targeting / bare `pubads.refresh()`. Implementation proceeds only if the observed calls satisfy every admission predicate below **and** the callback discipline is compatible with the shared-auction behavior: -**Objective:** reduce `/auction` request count for bursty, disjoint-unit publisher call patterns. Downstream bidder-call reduction is a **hypothesis to measure**, not a claim: one PBS request with multiple impressions does not guarantee every PBS bidder adapter issues fewer HTTP calls. Explicitly not a first-render latency lever; rollout must verify render latency does not regress. +- If the burst is **same-code duplicate auctions** (supported today; the disjoint-code rule refuses to merge them), Lever A as specified reduces nothing — the follow-up is identical-request deduplication as a separate design, or dropping the lever. +- If callbacks perform **bare refreshes**, note the interaction: the first caller's bare refresh consumes the one-shot pending-delivery state for _all_ merged units, and a second caller's subsequent common refresh pattern can then be classified as an independent refresh and start a **synthetic auction** — potentially cancelling the load reduction. Phase 0 must estimate the net `/auction` effect under the observed callback pattern; if the net is not clearly positive, the lever stops here. + +**Objective:** reduce `/auction` request count for bursty, disjoint-unit publisher call patterns. Downstream bidder-call reduction is a **hypothesis to measure**: one PBS request with multiple impressions does not guarantee every PBS bidder adapter issues fewer HTTP calls. Explicitly not a first-render latency lever; rollout must verify render latency does not regress. **Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves current behavior byte-for-byte. +**Snapshot at enqueue.** The coalescer snapshots each admitted request the way Prebid itself does on entry: shallow-copy the request object and snapshot the ad-unit **array membership and order** (retaining unit references). Later additions of request keys or array push/splice do not retroactively change the issued call; unit-object mutations are caught by dispatch-time revalidation. + **Admission rules.** A call is held only when all of: - its request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler`, with a non-empty explicit `adUnits` array; -- `timeout` is absent or a finite positive integer (zero, negative, `NaN`, `Infinity`, or non-number values dispatch solo unchanged); +- `timeout` is absent or a finite positive integer, **and** the resulting auction-time budget stays above the solo-dispatch threshold: a call whose effective timeout is less than `window + 150 ms` dispatches solo (holding it would consume its budget; see Deadlines); - it is not one of the shim's own synthetic refresh auctions (their GPT watchdog starts when the wrapper returns); - no ad unit contains a bid entry for a configured client-side bidder; - ad-unit codes are non-empty strings, **unique within the call**, and **disjoint from every code already pending** (the `/auction` payload builder collapses duplicate codes, keeping the first unit's media types); -- batch bounds hold after admission: at most 4 pending calls, at most 32 total ad units, and a projected serialized payload of at most 192 KiB UTF-8 (64 KiB safety margin under the endpoint's 256 KiB limit). +- batch bounds hold after admission: at most 4 pending calls, at most 32 total ad units, and a projected payload within the size budget below. + +**Payload size budget.** "Projected serialized payload" is defined as the UTF-8 byte length of `JSON.stringify` applied to the `/auction`-shaped body built from the snapshotted units at admission, re-computed at dispatch. Budget: projected units ≤ **160 KiB**, plus a **32 KiB reserved allowance** for adapter-added inputs (EIDs, context config — which the shim does not bound), leaving ≥ 64 KiB margin under the endpoint's 256 KiB limit. This is a best-effort bound, not a guarantee: if the server still rejects the merged body with 413, the coalescer **re-dispatches each constituent call solo, once** — no caller silently fails because of merging. Serialization failures (cycles, throwing getters/`toJSON`) at admission or dispatch evict that call to solo dispatch. -**Ineligible arrivals — one rule for all of them:** any call that fails any admission predicate (extra option keys, synthetic refresh, client-side bidders, invalid timeout, code collision, size overflow, serialization failure) **first synchronously flushes the pending batch, then dispatches solo**. Nothing ever overtakes an earlier caller; event order is preserved. +**Ineligible arrivals — one rule for all classes:** any call failing any admission predicate (extra option keys, synthetic refresh, client-side bidders, invalid or sub-threshold timeout, code collision, size overflow, serialization failure) **first synchronously flushes the pending batch, then dispatches solo**. Nothing overtakes an earlier caller; event order is preserved. -**Deadlines.** +**Deadlines — a reduced auction-time budget, not an absolute deadline.** Prebid starts its auction timer only after request hooks and FPD enrichment, so no wrapper can guarantee completion by `arrival + timeout`; this design shapes the _budget_ instead: -- At enqueue, capture the live `pbjs.getConfig('bidderTimeout')`; a call without a timeout uses that captured value for deadline math. -- Each call's deadline is absolute: `arrival + effectiveTimeout`. Queue residence counts against it. -- Calls merge only when absolute deadlines agree within a **50 ms tolerance**; an incompatible deadline flushes the queue first. Later-arriving compatible callers accept the batch's earlier shared deadline and the shared `timedOut` result — documented behavior. -- A monotonic scheduler flushes at `min(windowEnd, earliestDeadline − 100 ms safety margin)` and re-arms if a new caller tightens the earliest deadline. -- The dispatched timeout is `earliestDeadline − now`, floored at **50 ms**; it is never `0` or negative (Prebid evaluates `timeout || bidderTimeout`, so `0` would silently restore the full global timeout). If the event loop wakes past a deadline (timer throttling, long tasks), dispatch immediately with the floor. +- At enqueue, capture the live `pbjs.getConfig('bidderTimeout')`. If it is missing or not a finite positive integer, calls without an explicit timeout dispatch solo (no deadline math is possible for them). +- Each call's nominal deadline is `arrival + effectiveTimeout`; queue residence counts against the budget. +- Compatibility: calls merge only while `maxDeadline − minDeadline ≤ 50 ms`; an incompatible arrival flushes first. Later-arriving compatible callers accept the batch's earlier shared budget and shared `timedOut` result — documented behavior. +- A monotonic scheduler flushes at `min(windowEnd, earliestDeadline − 100 ms)` and re-arms if a new caller tightens the earliest deadline. +- The dispatched timeout is `earliestDeadline − now`, floored at **50 ms** — never `0` (Prebid's `timeout || bidderTimeout` would silently restore the global default). The floor is reachable only through timer overshoot (throttling, long tasks), because sub-threshold budgets were never admitted; overshoot means the auction runs up to ~50 ms past the nominal deadline, which is accepted and documented. -**Dispatch-time revalidation.** The shim mutates publisher ad-unit objects in place, and publishers can mutate them further while a call is held; the adapter also builds the final payload (including then-current EIDs) only at dispatch. Every admission predicate — option keys, codes, bidders, bounds, projected serialized size — is therefore **re-checked at dispatch** against the live objects. A call that no longer qualifies is evicted from the batch and dispatched solo (current behavior); values that fail serialization (cycles, throwing getters/`toJSON`) are treated the same way. +**Dispatch-time revalidation and order-preserving eviction.** All predicates are re-checked at dispatch against the live unit objects (the shim mutates units in place and publishers can too). Revalidation walks the queue **in arrival order** and dispatches **contiguous eligible segments**, with each invalid call dispatched solo in its queue position: for A(valid), B(now-invalid), C(valid), the dispatch order is merged-[A], solo-B, merged-[C] — never a reordering. (A single-segment queue with one invalid member yields exactly today's per-call behavior for that member.) -**Dispatch.** One underlying `requestBids` with: the union of ad units in arrival order; the floored shared deadline; and a combined `bidsBackHandler` that: +**Queue lifecycle (reentrancy-safe).** At flush, the pending batch is **atomically detached** from the queue _before_ the underlying `requestBids` is invoked; settlement handlers own only the detached batch and never touch newer queue state. Prebid invokes `bidsBackHandler` before resolving its public promise, so a constituent callback may re-enter `requestBids` and start a new batch while the first is settling — the detached-batch rule makes that safe, and the reentrancy test must prove the second batch dispatches and settles. -1. runs Trusted Server bookkeeping for every constituent call first — each call keeps **its own registration ID**, so the existing throw-rollback semantics are preserved per caller; -2. invokes each caller's original handler in arrival order with callback `this` and the exact three arguments `(bids, timedOut, auctionId)`, with `bids` partitioned to that caller's codes and `timedOut`/`auctionId` shared. A throwing handler rolls back **only its own** registration (matching today's single-call behavior) and does not block later handlers. +**Dispatch.** One underlying `requestBids` per eligible segment with: the segment's units in arrival order; the floored shared budget; and a combined `bidsBackHandler` that: -**Global-state caveat (documented behavior change).** Partitioned callbacks do not partition Prebid's global auction state: all merged bids belong to one auction, so a publisher callback that calls `pbjs.setTargetingForGPTAsync()` without codes applies targeting for every returned unit, and a bare `pubads.refresh()` can deliver another caller's units — all constituent calls' bookkeeping is registered before callbacks precisely so such a refresh is attributed as publisher delivery for every affected unit. Operators enabling the flag accept this shared-auction visibility; the test plan exercises unscoped targeting plus bare and mixed-slot refreshes from the first callback. +1. runs Trusted Server bookkeeping for every constituent call first — each call keeps **its own registration ID**, preserving today's per-caller throw-rollback; +2. invokes each caller's original handler in arrival order with callback `this` and the exact three arguments `(bids, timedOut, auctionId)`. When `bids` is an object it is partitioned to the caller's codes; when Prebid supplies `undefined` (cancelled auction), `undefined` is passed through unaltered — as are `timedOut`/`auctionId`. A throwing handler rolls back only its own registration and does not block later handlers; the exception is reported by asynchronous rethrow (matching a lone call's observable behavior) and **does not reject any caller's facade promise**. -**Promise and result semantics.** Every held call returns a promise settling with `{bids (partitioned), timedOut (shared), auctionId (shared)}`. A synchronous dispatch failure rejects all pending promises with that error; asynchronous rejection of the underlying promise fans out to all held promises; the queue resets in a `finally`. One `auctionInit`/`auctionEnd` event stream replaces N — a documented, operator-visible analytics change. +**Shared-auction behavior (documented, accepted).** Partitioned callbacks do not partition Prebid's global auction state: merged bids belong to one auction, so unscoped `setTargetingForGPTAsync()` applies targeting for every returned unit and a bare `pubads.refresh()` can deliver a co-merged caller's units — attributed as publisher delivery because all constituent bookkeeping registers first. Operators enabling the flag accept this; the integrity section and acceptance tests treat cross-caller delivery within a merged auction as permitted. + +**Promise semantics.** Every held call returns a facade promise settling with the values described above. A synchronous dispatch failure rejects that segment's facade promises and resets only the detached segment. Prebid's public promise is resolve-only, so rejection fan-out is defensive unit/mock coverage — it cannot be proven against the real artifact without replacing the API under test. One `auctionInit`/`auctionEnd` event stream replaces N per merged segment — a documented, operator-visible analytics change. ### Lever B — GAM preconnect hint (opt-in) @@ -86,50 +95,48 @@ When enabled, GPT `head_inserts` emits `5% adverse move; owner: property operator) | existing, coarse | +| First-render / per-slot latency | sampled synthetic checks (scheduled harness runs against production), not RUM | synthetic | +| Consent-denied network activity (Lever B) | scheduled NetLog synthetic checks | synthetic | + +**Drain definition (Lever A):** after setting the flag to `0`, drain is complete when the `coalesced` signal reports zero merged auctions for a period covering the maximum expected open-document lifetime (48 h), since already-open documents keep their read-once config. ## Rollout -One lever at a time, each independently config-reversible (new navigations only, per the runtime note): +One lever at a time, Fastly pilot only, each independently config-reversible for new navigations: 1. Land binary; all flags default off — zero behavior change. -2. Lever A Phase 0 trace on the pilot property. If predicates hold: enable `request_bids_coalesce_ms = 50` alone; hold ≥7 days against the guardrails; on failure return to `0` and **drain** (confirm zero merged auctions) before any further lever. -3. With A either retained-and-stable or fully drained to `0`: enable `gam_preconnect` alone under its governance contract. +2. Lever A Phase 0 trace on the pilot property (including callback-discipline capture). If predicates and net-benefit hold: enable `request_bids_coalesce_ms = 50` alone; hold ≥7 days against the guardrail table; on failure return to `0` and confirm drain per the definition above. +3. With A either retained-and-stable or fully drained: enable `gam_preconnect` alone under its governance contract, measured against the then-current A state. 4. Lever C follows its own spec revision after discovery. From 84bb7c10c9d4d393eb462fc33943e66ba5dfa85d Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 21 Aug 2026 11:14:01 -0700 Subject: [PATCH 06/12] Address fourth review round on ad-latency spec Replace the unimplementable post-response 413 replay with an authoritative size bound at the adapter buildRequests seam (constituent boundaries carried in; over-limit merged bodies split into multiple transport descriptors within one auction, preserving the single event stream). Replace the global async rethrow with a synchronous first-error throw after all handlers so Prebid's existing catch path handles it. Define the telemetry wire path as a top-level constituentCallCount body field through endpoint, observation context, event row, and a schema-first Tinybird column. Bound drain with a 24h injected-config lifetime and shim self-disable. Add the Fastly single-flight discovery exit criterion, extend the navigation-scoped reservation to both architectures, and specify the pending-only state machine with settled-result discard and cancellation honesty. Require a traffic-weighted browser matrix and durable approval artifact for Lever B with HTTP/1 write predicates and a versioned parser. Replace the circular eligible-burst denominator with candidate-burst coverage plus a 95 percent merge-success health gate, add a 30ms minimum worthwhile effect with named paired estimators, correct the Spin activation row, use a side-effect-free size estimator, add the segment-failure continuation test, and soften byte-for-byte wording. --- ...-prebid-ad-latency-optimizations-design.md | 155 ++++++++++-------- 1 file changed, 85 insertions(+), 70 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index 9bfc20f80..a7c93d0b8 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -1,6 +1,6 @@ # Prebid Ad-Latency and Auction-Load Optimizations — Design -**Date:** 2026-08-20 (revised 2026-08-21, round 3) +**Date:** 2026-08-20 (revised 2026-08-21, round 4) **Status:** Draft (Lever A gated on a burst-trace prerequisite; Lever C gated on discovery) **Scope:** Client-side auction properties (server-side ad templates inactive). Measurements come from a pilot news property; identifying details stay out of this document per repository policy, and sanitized measurement artifacts live outside the spec. The pilot rollout is scoped to the Fastly adapter (see the activation matrix). @@ -29,154 +29,169 @@ Out of scope: re-enabling server-side ad templates, GPT lazy-load fetch margins Nothing in this design may create impression, win, or billing signals for ad units that **no caller requested to auction**. Signals differ per delivery path: -- **Client `/auction` → Prebid adapter path:** the `/auction` response serializer does not propagate explicit `nurl`/`burl` to this consumer; win notification is owned by Prebid/GAM rendering. Early or coalesced auctions on this path are targeting-only by construction. Within a merged auction, **cross-caller delivery is accepted behavior** (see Lever A's shared-auction section): every merged unit was requested by _some_ constituent caller, and a publisher callback using unscoped targeting or a bare refresh may deliver units from a co-merged caller. What remains forbidden is delivery of units absent from every constituent call. +- **Client `/auction` → Prebid adapter path:** the `/auction` response serializer does not propagate explicit `nurl`/`burl` to this consumer; win notification is owned by Prebid/GAM rendering. Early or coalesced auctions on this path are targeting-only by construction. Within a merged auction, **cross-caller delivery is accepted behavior** (see Lever A's shared-auction section): every merged unit was requested by _some_ constituent caller. What remains forbidden is delivery of units absent from every constituent call. - **`tsjs.requestAds` path:** fetch and render are currently coupled. If discovery selects this path for Lever C, fetch must be **split from render** first; an early fetch must never trigger its render half. -- **Server-side notices:** some PBS deployments fire win/billing notices server-side, outside browser control. Any early-auction design must state whether the upstream configuration can bill on auction rather than render; properties where that is true are **excluded** from early auctions until the upstream policy is confirmed render-tied (OpenRTB leaves billing timing exchange-specific). -- **The PUC render bridge** (which fires beacons after posting a creative response, without proof of pixel render) consumes the server-template `tsjs.bids` path — inactive in this scope; its beacon semantics are not this spec's integrity boundary. +- **Server-side notices:** some PBS deployments fire win/billing notices server-side, outside browser control. Any early-auction design must state whether the upstream configuration can bill on auction rather than render; properties where that is true are **excluded** from early auctions until the upstream policy is confirmed render-tied. +- **The PUC render bridge** consumes the server-template `tsjs.bids` path — inactive in this scope; its beacon semantics are not this spec's integrity boundary. - A prefetched bid that is never consumed expires without firing any beacon. -- Guardrails use **per-path computable signals** (Lever A: `/auction` count vs. rendered-slot count from the harness); no cross-path "beacons per impression" universal metric is claimed. +- Guardrails use **per-path computable signals**; no cross-path "beacons per impression" universal metric is claimed. ## Design ### Lever A — `requestBids` coalescing window (opt-in; load-reduction hypothesis) -**Phase 0 — trace prerequisite (blocks implementation).** Capture a sanitized trace of the production burst calls recording: full option-key set, per-call ad-unit codes and bid entries, effective timeouts, projected payload sizes, and **callback behavior** — specifically whether callbacks use code-scoped targeting and slot-scoped refresh, or unscoped targeting / bare `pubads.refresh()`. Implementation proceeds only if the observed calls satisfy every admission predicate below **and** the callback discipline is compatible with the shared-auction behavior: +**Phase 0 — trace prerequisite (blocks implementation).** Capture a sanitized trace of the production burst calls recording: full option-key set, per-call ad-unit codes and bid entries, effective timeouts, payload sizes, and **callback behavior** — specifically whether callbacks use code-scoped targeting and slot-scoped refresh, or unscoped targeting / bare `pubads.refresh()`. Implementation proceeds only if the observed calls satisfy every admission predicate below **and** the callback discipline is compatible with the shared-auction behavior: - If the burst is **same-code duplicate auctions** (supported today; the disjoint-code rule refuses to merge them), Lever A as specified reduces nothing — the follow-up is identical-request deduplication as a separate design, or dropping the lever. -- If callbacks perform **bare refreshes**, note the interaction: the first caller's bare refresh consumes the one-shot pending-delivery state for _all_ merged units, and a second caller's subsequent common refresh pattern can then be classified as an independent refresh and start a **synthetic auction** — potentially cancelling the load reduction. Phase 0 must estimate the net `/auction` effect under the observed callback pattern; if the net is not clearly positive, the lever stops here. +- If callbacks perform **bare refreshes**, the first caller's bare refresh consumes the one-shot pending-delivery state for _all_ merged units, and a second caller's subsequent refresh can be classified as an independent refresh and start a **synthetic auction** — potentially cancelling the load reduction. Phase 0 must estimate the net `/auction` effect under the observed callback pattern; if the net is not clearly positive, the lever stops here. -**Objective:** reduce `/auction` request count for bursty, disjoint-unit publisher call patterns. Downstream bidder-call reduction is a **hypothesis to measure**: one PBS request with multiple impressions does not guarantee every PBS bidder adapter issues fewer HTTP calls. Explicitly not a first-render latency lever; rollout must verify render latency does not regress. +**Objective:** reduce `/auction` request count for bursty, disjoint-unit publisher call patterns. Downstream bidder-call reduction is a **hypothesis to measure**. Explicitly not a first-render latency lever; rollout must verify render latency does not regress. -**Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves current behavior byte-for-byte. +**Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves the existing synchronous pass-through path and public observable behavior (the bundle bytes necessarily change when the coalescer ships). + +**Coalescing-config lifetime.** The injected coalescing config carries an issue timestamp and a maximum document lifetime (24 h). The shim self-disables coalescing when `now > issuedAt + lifetime`, so long-lived documents quiesce on their own (see Drain). **Snapshot at enqueue.** The coalescer snapshots each admitted request the way Prebid itself does on entry: shallow-copy the request object and snapshot the ad-unit **array membership and order** (retaining unit references). Later additions of request keys or array push/splice do not retroactively change the issued call; unit-object mutations are caught by dispatch-time revalidation. **Admission rules.** A call is held only when all of: - its request object consists solely of `adUnits`, `timeout`, and `bidsBackHandler`, with a non-empty explicit `adUnits` array; -- `timeout` is absent or a finite positive integer, **and** the resulting auction-time budget stays above the solo-dispatch threshold: a call whose effective timeout is less than `window + 150 ms` dispatches solo (holding it would consume its budget; see Deadlines); +- `timeout` is absent or a finite positive integer, **and** the resulting auction-time budget stays above the solo-dispatch threshold: a call whose effective timeout is less than `window + 150 ms` dispatches solo; - it is not one of the shim's own synthetic refresh auctions (their GPT watchdog starts when the wrapper returns); - no ad unit contains a bid entry for a configured client-side bidder; -- ad-unit codes are non-empty strings, **unique within the call**, and **disjoint from every code already pending** (the `/auction` payload builder collapses duplicate codes, keeping the first unit's media types); -- batch bounds hold after admission: at most 4 pending calls, at most 32 total ad units, and a projected payload within the size budget below. +- ad-unit codes are non-empty strings, **unique within the call**, and **disjoint from every code already pending**; +- structural safety holds: every ad unit is measurable by a **side-effect-free size estimator** that walks own enumerable _data_ properties only. Units carrying accessors, custom `toJSON`, or otherwise unmeasurable values dispatch solo — publisher getter/`toJSON` code must never execute during admission (regression-tested with a stateful `toJSON`); +- batch bounds hold after admission: at most 4 pending calls, at most 32 total ad units, and an estimated unit payload ≤ 160 KiB UTF-8. -**Payload size budget.** "Projected serialized payload" is defined as the UTF-8 byte length of `JSON.stringify` applied to the `/auction`-shaped body built from the snapshotted units at admission, re-computed at dispatch. Budget: projected units ≤ **160 KiB**, plus a **32 KiB reserved allowance** for adapter-added inputs (EIDs, context config — which the shim does not bound), leaving ≥ 64 KiB margin under the endpoint's 256 KiB limit. This is a best-effort bound, not a guarantee: if the server still rejects the merged body with 413, the coalescer **re-dispatches each constituent call solo, once** — no caller silently fails because of merging. Serialization failures (cycles, throwing getters/`toJSON`) at admission or dispatch evict that call to solo dispatch. +**Authoritative size bound (adapter seam).** The estimator above is a cheap pre-filter; the **authoritative** bound lives in the adapter's `buildRequests`, where the final body — including then-current EIDs and context — actually exists. The coalescer carries **constituent-call boundaries** into the merged auction, and `buildRequests` serializes the final body and, if it would exceed 192 KiB (64 KiB margin under the endpoint's 256 KiB limit), **splits along constituent boundaries and returns multiple transport descriptors** — Prebid dispatches each as a separate HTTP request within the _same_ auction. No oversized request is ever sent, no post-response replay exists, and the one-auction/one-event-stream contract holds. A server 413 is therefore not an expected outcome; if one occurs anyway it resolves as an ordinary no-bid auction exactly as an oversized solo call does today. -**Ineligible arrivals — one rule for all classes:** any call failing any admission predicate (extra option keys, synthetic refresh, client-side bidders, invalid or sub-threshold timeout, code collision, size overflow, serialization failure) **first synchronously flushes the pending batch, then dispatches solo**. Nothing overtakes an earlier caller; event order is preserved. +**Ineligible arrivals — one rule for all classes:** any call failing any admission predicate **first synchronously flushes the pending batch, then dispatches solo**. Nothing overtakes an earlier caller; event order is preserved. -**Deadlines — a reduced auction-time budget, not an absolute deadline.** Prebid starts its auction timer only after request hooks and FPD enrichment, so no wrapper can guarantee completion by `arrival + timeout`; this design shapes the _budget_ instead: +**Deadlines — a reduced auction-time budget, not an absolute deadline.** Prebid starts its auction timer only after request hooks and FPD enrichment, so no wrapper can guarantee completion by `arrival + timeout`; this design shapes the _budget_: -- At enqueue, capture the live `pbjs.getConfig('bidderTimeout')`. If it is missing or not a finite positive integer, calls without an explicit timeout dispatch solo (no deadline math is possible for them). +- At enqueue, capture the live `pbjs.getConfig('bidderTimeout')`. If it is missing or not a finite positive integer, calls without an explicit timeout dispatch solo. - Each call's nominal deadline is `arrival + effectiveTimeout`; queue residence counts against the budget. - Compatibility: calls merge only while `maxDeadline − minDeadline ≤ 50 ms`; an incompatible arrival flushes first. Later-arriving compatible callers accept the batch's earlier shared budget and shared `timedOut` result — documented behavior. - A monotonic scheduler flushes at `min(windowEnd, earliestDeadline − 100 ms)` and re-arms if a new caller tightens the earliest deadline. -- The dispatched timeout is `earliestDeadline − now`, floored at **50 ms** — never `0` (Prebid's `timeout || bidderTimeout` would silently restore the global default). The floor is reachable only through timer overshoot (throttling, long tasks), because sub-threshold budgets were never admitted; overshoot means the auction runs up to ~50 ms past the nominal deadline, which is accepted and documented. +- The dispatched timeout is `earliestDeadline − now`, floored at **50 ms** — never `0`. The floor is reachable only through timer overshoot, which means the auction runs up to ~50 ms past the nominal deadline — accepted and documented. -**Dispatch-time revalidation and order-preserving eviction.** All predicates are re-checked at dispatch against the live unit objects (the shim mutates units in place and publishers can too). Revalidation walks the queue **in arrival order** and dispatches **contiguous eligible segments**, with each invalid call dispatched solo in its queue position: for A(valid), B(now-invalid), C(valid), the dispatch order is merged-[A], solo-B, merged-[C] — never a reordering. (A single-segment queue with one invalid member yields exactly today's per-call behavior for that member.) +**Dispatch-time revalidation and order-preserving eviction.** All predicates are re-checked at dispatch against the live unit objects. Revalidation walks the queue **in arrival order** and dispatches **contiguous eligible segments**, with each invalid call dispatched solo in its queue position: A(valid), B(now-invalid), C(valid) → merged-[A], solo-B, merged-[C]. A synchronous dispatch failure of one segment rejects only that segment's facade promises; **later segments still dispatch in order and settle** (tested). -**Queue lifecycle (reentrancy-safe).** At flush, the pending batch is **atomically detached** from the queue _before_ the underlying `requestBids` is invoked; settlement handlers own only the detached batch and never touch newer queue state. Prebid invokes `bidsBackHandler` before resolving its public promise, so a constituent callback may re-enter `requestBids` and start a new batch while the first is settling — the detached-batch rule makes that safe, and the reentrancy test must prove the second batch dispatches and settles. +**Queue lifecycle (reentrancy-safe).** At flush, the pending batch is **atomically detached** from the queue _before_ the underlying `requestBids` is invoked; settlement handlers own only the detached batch and never touch newer queue state. A constituent callback may re-enter `requestBids` and start a new batch while the first settles — the detached-batch rule makes that safe, and the reentrancy test proves the second batch dispatches and settles. -**Dispatch.** One underlying `requestBids` per eligible segment with: the segment's units in arrival order; the floored shared budget; and a combined `bidsBackHandler` that: +**Dispatch.** One underlying `requestBids` per eligible segment with: the segment's units in arrival order (constituent boundaries attached for the adapter seam); the floored shared budget; and a combined `bidsBackHandler` that: 1. runs Trusted Server bookkeeping for every constituent call first — each call keeps **its own registration ID**, preserving today's per-caller throw-rollback; -2. invokes each caller's original handler in arrival order with callback `this` and the exact three arguments `(bids, timedOut, auctionId)`. When `bids` is an object it is partitioned to the caller's codes; when Prebid supplies `undefined` (cancelled auction), `undefined` is passed through unaltered — as are `timedOut`/`auctionId`. A throwing handler rolls back only its own registration and does not block later handlers; the exception is reported by asynchronous rethrow (matching a lone call's observable behavior) and **does not reject any caller's facade promise**. +2. invokes each caller's original handler in arrival order with callback `this` and the exact three arguments `(bids, timedOut, auctionId)`. When `bids` is an object it is partitioned to the caller's codes; `undefined` cancelled-auction values pass through unaltered. A throwing handler rolls back only its own registration and does not block later handlers; after **all** handlers have run, the **first captured error is thrown synchronously** so Prebid's existing catch-and-log path handles it — observably matching today's lone-call behavior. No global asynchronous rethrow. Facade promises still resolve. + +**Shared-auction behavior (documented, accepted).** Partitioned callbacks do not partition Prebid's global auction state: unscoped `setTargetingForGPTAsync()` applies targeting for every returned unit and a bare `pubads.refresh()` can deliver a co-merged caller's units — attributed as publisher delivery because all constituent bookkeeping registers first. Operators enabling the flag accept this; acceptance tests treat cross-caller delivery within a merged auction as permitted. -**Shared-auction behavior (documented, accepted).** Partitioned callbacks do not partition Prebid's global auction state: merged bids belong to one auction, so unscoped `setTargetingForGPTAsync()` applies targeting for every returned unit and a bare `pubads.refresh()` can deliver a co-merged caller's units — attributed as publisher delivery because all constituent bookkeeping registers first. Operators enabling the flag accept this; the integrity section and acceptance tests treat cross-caller delivery within a merged auction as permitted. +**Promise semantics.** Every held call returns a facade promise settling with the values described above. Rejection fan-out (synchronous dispatch failure) is per detached segment and is defensive unit coverage — Prebid's public promise is resolve-only, so it cannot be proven against the real artifact. One `auctionInit`/`auctionEnd` event stream per merged segment replaces N — a documented, operator-visible analytics change. -**Promise semantics.** Every held call returns a facade promise settling with the values described above. A synchronous dispatch failure rejects that segment's facade promises and resets only the detached segment. Prebid's public promise is resolve-only, so rejection fan-out is defensive unit/mock coverage — it cannot be proven against the real artifact without replacing the API under test. One `auctionInit`/`auctionEnd` event stream replaces N per merged segment — a documented, operator-visible analytics change. +**Coalescing telemetry (required for rollout).** A dedicated, bounded, untrusted metadata field — `constituentCallCount` (integer, `2..=4`) — travels in the `/auction` request body as a **top-level sibling of** `config`, _not_ inside it (the `config` object is publisher auction context filtered by `allowed_context_keys`; unknown keys there are dropped). Semantics: absent = solo/legacy call; present only on merged dispatches; the count reaches `buildRequests` via the constituent boundaries the coalescer attaches, and split descriptors each carry their own segment's count. Server side: the endpoint parses and clamps the field, `AuctionObservationContext` and the auction event summary row carry it, and the Tinybird datasource gains the column — **schema deployed before the emitting binary**. Endpoint, telemetry, sink-serialization, and schema tests are part of the implementation, not optional. ### Lever B — GAM preconnect hint (opt-in) **Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false`. -When enabled, GPT `head_inserts` emits `` **before the GPT bootstrap inserts** (asserted by a transformed-HTML ordering test), without `crossorigin` — ad requests are cookie-credentialed, and the HTML preconnect algorithm keeps credentialed and anonymous connections distinct. Browsers may partially perform or skip hints; best-effort by nature. +When enabled, GPT `head_inserts` emits `` **before the GPT bootstrap inserts** (asserted by a transformed-HTML ordering test), without `crossorigin` — ad requests are cookie-credentialed. Browsers may partially perform or skip hints; best-effort by nature. + +**Scope of claim:** GPT scripts (including `pubads_impl`) are first-party proxied; the hint can only affect the **first direct ad request**, and the claim is "may reduce" its connection setup. -**Scope of claim:** GPT scripts (including `pubads_impl`) are first-party proxied by the script guard; the hint can only affect the **first direct ad request**, and the claim is "may reduce" its connection setup. +**Browser coverage:** the flag emits to every browser, so verification cannot be Chromium-only. Governance requires a **traffic-weighted supported-browser matrix** (Chromium, WebKit/Safari, Firefox, material WebViews) with engine-appropriate low-level verification for each entry; engines that cannot be verified are documented as unverified in the approval. (Restricting emission per-engine would require request-scoped UA gating — out of scope, so the approval covers all engines the property serves.) **Governance (required before any property enables it):** -- The flag is a property-level boolean and GPT head insertion has no request-scoped jurisdiction or consent input; therefore the approval must cover **every jurisdiction served by the deployed configuration**, with a named approver. (Per-jurisdiction selectivity would require request-scoped consent/geo input to head insertion — out of scope here and stated as such.) -- **Verification protocol (NetLog):** pinned browser version, fresh profile per run, cache and socket pools cold, `--log-net-log` capture with sensitive data excluded, H2/H3/QUIC classified, and the speculative socket joined to the first GAM request via NetLog source IDs. Raw logs are redacted (no cookies/URLs beyond the GAM host) and retained only for the acceptance window. -- **Enforceable invariant:** no HTTP request HEADERS/DATA frames on the speculative connection before the normal GAM ad request. (Connection-level protocol frames — settings, pings — are inherent to preconnect and permitted.) Verified including GPC-set, CMP-unresolved, and CMP-denied cases. -- Rollback trigger: any observed request frames before the normal ad request disables the flag. +- The flag is a property-level boolean and GPT head insertion has no request-scoped jurisdiction or consent input; the approval must therefore cover **every jurisdiction served by the deployed configuration**. +- **The approval is a durable artifact** recording: configuration version, jurisdiction inventory and unknown-jurisdiction handling, verified browser matrix, approver, date, expiry, and a re-approval trigger when the served scope changes. +- **Verification protocol:** pinned browser build per engine, fresh profile per run, cold cache and socket pools, capture mode stated explicitly, raw logs treated as sensitive (redacted to the GAM host, retained only for the acceptance window). For Chromium: NetLog with the speculative socket joined to the first ad request via source IDs, using a **versioned parser algorithm with fixtures and one controlled end-to-end test**; equivalent engine-appropriate tooling for others. Current audit tooling has no NetLog surface — building this capture path is part of the lever's implementation cost. +- **Enforceable invariant:** no HTTP request writes on the speculative connection before the normal first GAM ad request — defined as no HTTP/2 or HTTP/3 HEADERS/DATA frames **and** no HTTP/1 request writes. "Normal first GAM request" = the first `securepubads.g.doubleclick.net` ad request initiated by GPT for the document. Verified including GPC-set, CMP-unresolved, and CMP-denied cases; connection-level protocol frames (settings, pings) are inherent to preconnect and permitted. +- Rollback trigger: any observed request write before the normal ad request disables the flag — an immediate-stop conformance failure regardless of performance. ### Lever C — earlier first auction (discovery first; design contingent) -**Status: not implementation-ready.** No design option is selected in this revision; discovery below produces the inputs for a follow-up spec revision that will select one. +**Status: not implementation-ready.** No design option is selected; discovery produces the inputs for a follow-up spec revision. **Discovery contract:** -- **Attribute the 3.3 s `/auction` request.** Named hypothesis: `tsjs.requestAds` (posts `/auction` directly from the TSJS registry, invokes its callback, renders asynchronously). If confirmed, acceleration is mis-scoped until fetch is split from render, and the prior question becomes whether an independent non-Prebid auction on the page should be **deduplicated or removed** rather than accelerated. -- **Typed direct-path result.** `requestAds` returns `void` and `sendAuction` collapses network failure, parse failure, and legitimate emptiness into `[]`. A fetch/render split requires a private result type — e.g. `{outcome, bids, completedAt}` — distinguishing those cases while preserving the public callback lifecycle. -- **Transport ownership.** The Prebid adapter returns a request descriptor and **Prebid core owns that HTTP operation**; the repository-owned `sendAuction` belongs to the separate core API. Any cache/single-flight needs a named owner at one of those seams — and **server-side single-flight at the common auction boundary is the preferred shape**, because the browser cannot observe the HttpOnly EC identity, server-side KV/geo/identity-graph state, or configuration changes that determine auction equivalence. If a client-side mechanism is chosen anyway, it requires a short-lived, opaque, authenticated, navigation-bound server-issued token that reveals no stable identity and carries capability/configuration versioning. -- **No completed-response reuse initially.** The `/auction` response carries no bid lifetime and the adapter stamps a fresh `ttl: 300` at interpretation time, so reusing an old response silently renews its apparent lifetime. Reuse (if ever supported) requires completion time and per-bid expiry in the response, with TTL set to remaining lifetime. Until then, only **in-flight sharing** is on the table: the normal path awaits the early request's promise, valid only while the equivalence snapshot (consent, identity, navigation) is unchanged; any change invalidates/aborts and a fresh normal request runs. -- **Consent parity.** CMP readiness in the browser is not the state the server consumes: the `/auction` body carries no consent envelope; the server reconstructs consent from cookies and `Sec-GPC`. Discovery must define cookie parity or a validated consent envelope, and `/auction` needs a response signal distinguishing consent-denied from legitimate no-bid (both are HTTP 200 today); consent-denied results are never shareable. -- **Input ownership tables.** Producer/consumer/invalidation tables per path: TSJS registry generations, Prebid ad-unit generations, navigation generation, render targets. GPT slot targeting is listed as an auction input only if discovery proves it affects request bytes. -- **Billing integrity** per the section above; on the `requestAds` path, split fetch from render before any reuse. +- **Attribute the 3.3 s `/auction` request.** Named hypothesis: `tsjs.requestAds`. If confirmed, acceleration is mis-scoped until fetch is split from render, and the prior question becomes whether an independent non-Prebid auction should be **deduplicated or removed** rather than accelerated. +- **Typed direct-path result.** `requestAds` returns `void` and `sendAuction` collapses network failure, parse failure, and legitimate emptiness into `[]`. A fetch/render split requires a private result type — `{outcome, bids, completedAt}` — preserving the public callback lifecycle. +- **Transport ownership and the Fastly constraint.** The Prebid adapter returns a request descriptor (Prebid core owns that HTTP operation); the repository-owned `sendAuction` is the separate core API. Server-side single-flight at the common auction boundary is the preferred _shape_, **but it currently has no viable Fastly owner**: Fastly application state is rebuilt per request, and the platform abstraction exposes KV/cache/HTTP but no atomic pending-join primitive. **Discovery exit criterion:** prove a cross-instance, atomic, _pending-only_ join with zero retention after settlement on Fastly. If Fastly cannot supply that contract, server-side single-flight is off the table for the pilot and the candidate becomes authenticated client-side coordination or a different architecture. An ordinary persistent-cache lookup is not an acceptable substitute — it _is_ the completed-response reuse this spec forbids. +- **Navigation-scoped reservation (both architectures).** Client- or server-side, sharing requires: an opaque, authenticated, **single-navigation reservation**; exactly one intended early/normal pair per reservation; server revalidation of hidden inputs (HttpOnly EC identity, geo, server-resolved EIDs, headers, provider mode, configuration version) on the joining request; a canonical key derived from the fully normalized provider input plus relevant headers/settings — never a content-only or stable-identity key, which could join different documents or users with identical units and expose a one-shot bid across contexts. +- **Pending-only state machine.** Joining is allowed **only while Pending**. If no normal waiter attaches before settlement, the result is **discarded immediately**; a later normal call starts a fresh auction; the pending→settled attachment race is atomic and tested. Note the measured gap (early ~3.3 s, publisher ~4.8 s) makes settled-before-join the _likely_ case — the expected benefit of in-flight sharing is correspondingly modest and must be measured before further investment. +- **Cancellation honesty.** `sendAuction` has no `AbortSignal` and uses `keepalive`; client detach discards the local result but does not prove the server or upstream provider stopped. The design distinguishes **detach/result-discard** from **proven upstream cancellation**, adds generation guards against late targeting/render, and states whether a fresh auction may overlap an invalidated one. +- **Server-seam event semantics** (if a server join is ever built): share only provider/orchestrator execution after each waiter's inputs are normalized; serialize request-specific responses separately; emit one leader auction event plus a joined-waiter event; never replay the leader's correlation data. +- **No completed-response reuse.** The `/auction` response carries no bid lifetime and the adapter stamps a fresh `ttl: 300` at interpretation; reuse would silently renew lifetimes. If ever supported, the response must carry completion time and per-bid expiry, with TTL set to remaining lifetime. +- **Consent parity.** The `/auction` body carries no consent envelope; the server reconstructs consent from cookies and `Sec-GPC`. Discovery must define cookie parity or a validated consent envelope, and `/auction` needs a response signal distinguishing consent-denied from legitimate no-bid; consent-denied results are never shareable. +- **Input ownership tables** per path (TSJS registry generations, Prebid ad-unit generations, navigation generation, render targets); GPT slot targeting is an auction input only if discovery proves it affects request bytes. +- **Billing integrity** per the section above; split fetch from render before any reuse on the `requestAds` path. ## Config-blob compatibility -Integration settings are retained as raw JSON in the pushed blob (`IntegrationSettings` flattens into a `HashMap`), so an explicitly configured `0`/`false` is present in the blob; only omitted keys are absent. `PrebidIntegrationConfig` and `GptConfig` do not `deny_unknown_fields`, so older binaries tolerate blobs carrying the new keys — no clear-before-rollback step. Testing includes a new-schema blob parsed by the legacy struct shape. +Integration settings are retained as raw JSON in the pushed blob, so an explicitly configured `0`/`false` is present in the blob; only omitted keys are absent. `PrebidIntegrationConfig` and `GptConfig` do not `deny_unknown_fields`, so older binaries tolerate blobs carrying the new keys. Compatibility tests carry the **non-default values** (`request_bids_coalesce_ms = 50`, `gam_preconnect = true`) through a full blob into the legacy struct shapes, plus present-leaf and absent-leaf environment-overlay tests for **both** fields. The public configuration guide's "unknown TOML keys fail" statement gains an explicit exception note for forward-compatible integration leaves. -**Adapter activation/rollback matrix.** Injected client config is read once per document, so _within_ an adapter a config change affects new navigations only. Across adapters, activation differs: Fastly instances are effectively per-request (config push suffices); Axum builds shared state at startup, and Cloudflare and Spin similarly hold startup state (config change requires restart/redeploy). **The pilot rollout is scoped to Fastly**; the other adapters inherit the flags but their activation path is restart-based and out of the pilot's scope. +**Adapter activation/rollback matrix.** Injected client config is read once per document, so within an adapter a config change affects new documents only — and "new navigation" is not "new document": client-path HTML can be browser-cached for 60 s. Across adapters: Fastly instances are effectively per-request (config push suffices). Axum builds shared state at startup (restart required). Cloudflare holds startup state (redeploy/restart). **Spin parses an embedded example config at build time — a restart cannot activate a changed flag; it requires a source-config change plus rebuild/redeploy (or future runtime config loading).** The pilot rollout is scoped to Fastly. ## Testing **Vitest (shim), Lever A:** -- two mergeable calls → one underlying `requestBids`, segment units in arrival order, per-caller partitioned maps, handlers in arrival order with preserved `this` and exact `(bids, timedOut, auctionId)` arguments; cancelled-auction `undefined` values passed through unpartitioned; -- bookkeeping-before-callbacks with per-caller registration IDs; a throwing first handler rolls back only its own registration, later handlers run, no facade promise rejects, the exception surfaces via async rethrow, and the existing throw-rollback regression test still passes; +- two mergeable calls → one underlying `requestBids`, segment units in arrival order, per-caller partitioned maps, handlers in arrival order with preserved `this` and exact `(bids, timedOut, auctionId)`; cancelled-auction `undefined` values pass through unpartitioned; +- bookkeeping-before-callbacks with per-caller registration IDs; a throwing first handler rolls back only its own registration, later handlers run, no facade promise rejects, and the first captured error is **synchronously** thrown after all handlers (asserted to land in Prebid's catch path, matching the existing lone-call regression); - every ineligible-arrival class flushes the pending batch first, then dispatches solo — ordering asserted; -- **order-preserving eviction:** middle-call mutation during the hold produces merged-[A], solo-B, merged-[C] dispatch and event order; -- shared-auction scenario: first callback performs unscoped `setTargetingForGPTAsync()` plus bare and mixed-slot `refresh()`; asserts the documented cross-caller delivery, pending-state consumption, and any resulting synthetic-auction classification (net `/auction` count asserted); -- **reentrancy:** a constituent callback enqueues a new batch while the first settles; the detached-batch rule is proven — the second batch dispatches and settles, and the first batch's cleanup does not clear it or its timer; -- deadline math: sub-threshold timeout dispatches solo; zero/negative/`NaN`/`Infinity` dispatch solo; missing/invalid captured `bidderTimeout` sends timeout-less calls solo; `bidderTimeout` captured at enqueue survives a mid-hold config change; a later caller with an earlier deadline re-arms the scheduler; timer overshoot dispatches with the 50 ms floor (never `0`), documented as budget overrun; -- snapshot semantics: post-enqueue request-key additions and ad-unit array push/splice do not alter the issued call; unit-object mutations are caught at revalidation; -- payload budget: boundary−1 / boundary / boundary+1 projections, multibyte (UTF-8) content, oversized EID allowance behavior, and the 413 → re-dispatch-solo-once fallback; -- async rejection fan-out and `finally`-scoped cleanup of the detached segment (defensive unit coverage; not provable against the real artifact); +- order-preserving eviction: merged-[A], solo-B, merged-[C]; **segment-failure isolation:** segment A's underlying dispatch throws synchronously, segments B and C still dispatch in order and settle; +- shared-auction scenario: unscoped `setTargetingForGPTAsync()` plus bare and mixed-slot `refresh()` from the first callback, asserting documented cross-caller delivery, pending-state consumption, synthetic-auction classification, and net `/auction` count; +- reentrancy: a constituent callback enqueues a new batch while the first settles; the detached-batch rule proven — second batch dispatches and settles un-clobbered; +- deadline math: sub-threshold, zero/negative/`NaN`/`Infinity`, missing/invalid captured `bidderTimeout`, mid-hold config change, earlier-deadline re-arm, overshoot floor (never `0`); +- snapshot semantics: post-enqueue request-key additions and array push/splice do not alter the issued call; unit-object mutations caught at revalidation; +- **side-effect-free estimator:** a stateful `toJSON`/getter is never invoked at admission (regression test); accessor-bearing units dispatch solo; +- **adapter split:** `buildRequests` with constituent boundaries splits an over-limit merged body into multiple descriptors along call boundaries (boundary−1/boundary/boundary+1, multibyte content, large EIDs), each carrying its segment's `constituentCallCount`; +- coalescing-config lifetime: coalescing self-disables past `issuedAt + lifetime`; - window `0` / absent config leaves the existing suite untouched. -**Real-artifact coverage (external bundle):** drive two real calls and prove one fetch, two thenables, callback-before-promise ordering, partitioned results, a single event sequence, and shared timeout/auction-id semantics. (Rejection fan-out stays in unit coverage; the real public promise is resolve-only.) +**Real-artifact coverage (external bundle):** two real calls → one fetch, two thenables, callback-before-promise ordering, partitioned results, a single event sequence, shared timeout/auction-id semantics; plus one real-artifact case where the merged body exceeds the limit and `buildRequests` returns multiple descriptors within one auction. **Rust:** -- config defaults and bounds; injected-config serialization omits `requestBidsCoalesceMs` at `0`; -- `gam_preconnect = true` emits the link without `crossorigin` and **before** the GPT bootstrap inserts (transformed-HTML ordering test); `false` emits nothing; -- new-schema → legacy-schema blob compatibility test. +- config defaults and bounds; injected-config serialization omits `requestBidsCoalesceMs` at `0`; injected coalescing config carries `issuedAt` + lifetime; +- `/auction` endpoint parses and clamps `constituentCallCount` (absent, 2..=4, out-of-range); `AuctionObservationContext` and the event summary row carry it; sink serialization includes it; a schema-ordering note ties the Tinybird column migration ahead of the emitting binary; +- `gam_preconnect = true` emits the link without `crossorigin` **before** the GPT bootstrap inserts; `false` emits nothing; +- new-schema → legacy-schema blob test with non-default values; present/absent env-leaf tests for both fields. -**Documentation checklist:** `trusted-server.example.toml`, configuration tables, Prebid and GPT integration guides, and environment-overlay leaf-creation behavior for **both** new fields (Prebid and GPT). +**Documentation checklist:** `trusted-server.example.toml`, configuration tables (including the unknown-keys exception note), Prebid and GPT integration guides, environment-overlay leaf behavior for both fields. **Lever C tests are defined with its design after discovery.** ## Measurement methodology -The three-run baseline motivates the work but does not gate it. Acceptance uses a reproducible harness (local Viceroy against the pilot origin, pre-seeded consent, identical scripted scroll) with **randomized, balanced AB/BA pair ordering** on the same machine and network, explicit warm/cold connection conditions, and a sample size derived from pilot variance (a pilot batch of ≥10 pairs estimates variance; the acceptance batch is sized for 80% power on the stated effect, and is never smaller than 20 pairs). Decisions use **paired confidence bounds**, reporting p50/p95 with dispersion. +Acceptance uses a reproducible harness (local Viceroy against the pilot origin, pre-seeded consent, identical scripted scroll) with **randomized, balanced AB/BA pair ordering**, explicit warm/cold connection conditions, and sample sizes derived from pilot variance with **quantile-specific power calculations** (a pilot batch of ≥10 pairs estimates variance; acceptance batches are powered at 80% for the stated effect at the stated quantile, never fewer than 20 pairs). Decisions use **named paired estimators**: median differences via the Hodges–Lehmann estimator with bootstrap 95% confidence intervals; non-inferiority stated per metric and quantile. -**Denominators, defined:** a _run_ is one full harness execution (fresh context); a _navigation_ is one document load within a run; an _eligible burst_ is a set of ≥2 `requestBids` calls within the configured window on one navigation that satisfy every admission predicate per the Phase 0 trace definition. +**Denominators:** a _run_ is one full harness execution (fresh context); a _navigation_ is one document load; a **candidate burst** is ≥2 `requestBids` calls within the configured window on one navigation (pre-admission); an **eligible burst** is a candidate burst whose calls pass every admission predicate. Numeric gates: -- **Lever A retains if:** ≥30% of eligible bursts merge, total `/auction` requests per navigation drop ≥20% (paired 95% CI excluding zero), **and** first-non-empty-render p95 shows non-inferiority within a 5% margin vs. paired control. Otherwise the flag returns to `0`. -- **Lever B retains if:** NetLog shows the speculative socket serving the first direct GAM ad request in ≥70% of cold-start pairs, the request-frame invariant holds in 100% of runs (any violation is an immediate rollback regardless of performance), **and** a real benefit gate passes: paired median first-GAM-request setup-time improvement with a 95% CI excluding zero, plus first-render and page-load non-inferiority (5% margin). Socket reuse alone does not retain the flag. -- **B is measured with identical Lever A state in both arms** (whatever A's disposition is at that point — the comparison is X vs. X+B, stated explicitly in the results). +- **Lever A retains if:** candidate-burst coverage is reported (share of candidate bursts that are eligible — this is the publisher-pattern fact); **merge success among eligible bursts is ≥95%** (implementation health — eligible bursts should merge absent runtime failure, and each failure is diagnosed); total `/auction` requests per navigation drop ≥20% (paired 95% CI excluding zero); and first-non-empty-render non-inferiority holds at p50 and p95 within a 5% margin. Otherwise the flag returns to `0`. +- **Lever B retains if:** the one-sided 95% **lower confidence bound** on cold-start speculative-socket reuse for the first GAM ad request exceeds 70%; the request-write invariant holds in 100% of runs (any violation = immediate rollback); and the benefit gate passes: paired median first-GAM-request setup-time improvement of at least the **minimum worthwhile effect of 30 ms** (Hodges–Lehmann, 95% CI excluding zero), with first-render (p50, p95) and `load`-event (p50) non-inferiority within 5%. Socket reuse alone does not retain the flag. +- **B is measured with identical Lever A state in both arms** — the comparison is X vs. X+B, stated in the results. -**Live guardrails and their sources.** Server telemetry today has no experiment arm, navigation ID, or coalesced-count signal, so live monitoring is explicitly split: +**Live guardrails and their sources:** -| Signal | Source | Type | -| ----------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- | -| Merged-auction count / drain state | new `coalesced` count field the shim adds to the `/auction` request `config` object; surfaced in server auction telemetry | new, required for Lever A rollout | -| `/auction` volume per property | existing server telemetry | existing | -| Fill/revenue, bid rate, timeout rate | ad-server / PBS reporting (property-level trend vs. 7-day pre-enable baseline, alert on >5% adverse move; owner: property operator) | existing, coarse | -| First-render / per-slot latency | sampled synthetic checks (scheduled harness runs against production), not RUM | synthetic | -| Consent-denied network activity (Lever B) | scheduled NetLog synthetic checks | synthetic | +| Signal | Source | Type / cadence | +| ----------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------- | +| Merged-auction count / drain state | `constituentCallCount` in server auction telemetry + Tinybird column | new; continuous | +| `/auction` volume per property | existing server telemetry | existing; continuous | +| Fill/revenue, bid rate, timeout rate | ad-server / PBS reporting vs. 7-day pre-enable baseline; alert on >5% adverse move; owner: property operator; reporting maturation delay 48 h | existing; daily | +| First-render / per-slot latency | scheduled synthetic harness runs against production | synthetic; daily during holds | +| Lever B socket reuse + setup time | scheduled engine-appropriate capture runs | synthetic; weekly during the B hold | +| Consent-denied network activity (Lever B) | scheduled capture runs (GPC / unresolved / denied) | synthetic; weekly during the B hold | -**Drain definition (Lever A):** after setting the flag to `0`, drain is complete when the `coalesced` signal reports zero merged auctions for a period covering the maximum expected open-document lifetime (48 h), since already-open documents keep their read-once config. +**Drain (Lever A):** the injected coalescing config expires after its 24 h lifetime and the shim self-disables, so drain is complete when the `constituentCallCount` signal shows zero merged auctions after `max(config lifetime, propagation bound, 60 s HTML freshness)` — a bounded guarantee, not an open-ended observation window. ## Rollout -One lever at a time, Fastly pilot only, each independently config-reversible for new navigations: +One lever at a time, Fastly pilot only, each independently config-reversible for newly generated documents: -1. Land binary; all flags default off — zero behavior change. +1. Land binary (including the telemetry field and Tinybird schema, deployed schema-first); all flags default off. 2. Lever A Phase 0 trace on the pilot property (including callback-discipline capture). If predicates and net-benefit hold: enable `request_bids_coalesce_ms = 50` alone; hold ≥7 days against the guardrail table; on failure return to `0` and confirm drain per the definition above. -3. With A either retained-and-stable or fully drained: enable `gam_preconnect` alone under its governance contract, measured against the then-current A state. +3. With A retained-and-stable or fully drained: enable `gam_preconnect` alone under its governance artifact; **hold ≥7 days** with the weekly synthetic reuse/setup and consent-network checks; retain only if the live checks continue to meet the acceptance gates, roll back on any conformance failure immediately. 4. Lever C follows its own spec revision after discovery. From 401c6418cbf379f47af340800de7790b26e02a0e Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sat, 22 Aug 2026 00:40:19 -0700 Subject: [PATCH 07/12] Address fifth review round on ad-latency spec Narrow the no-oversized-request guarantee to multi-constituent descriptors with an explicit singleton rule, key constituent boundaries by an internal auction ID read from the auction-scoped bidderRequest with a no-marker-leak test, and sanitize EIDs into plain data once per auction with unsanitizable EIDs making the body unsplittable. Replace constituentCallCount with a coalesced group/size/part object counted by distinct group, drop malformed values as absent, and name the auction_coalescing_daily rollup as the guardrail. Subordinate Lever C's pending-discard rule to one-consumer arbitration so the public requestAds render cannot be silently dropped, add the client-side real-artifact feasibility gate and the server-side side-effect-free prepare+fingerprint exit criterion, and require an outcome transition table. Block Lever B until every engine above 5 percent traffic is verified. Require the one-sided lower bound to exceed the 30ms effect, use a paired bootstrap quantile estimator for p95, define reuse denominators, split the causal harness from non-causal production guardrails with a fixed day-7 rule and daily batches, and make drain additive with a monotonic document lifetime and pipeline-liveness check. --- ...-prebid-ad-latency-optimizations-design.md | 46 +++++++++++-------- 1 file changed, 28 insertions(+), 18 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index a7c93d0b8..04d1858ec 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -1,6 +1,6 @@ # Prebid Ad-Latency and Auction-Load Optimizations — Design -**Date:** 2026-08-20 (revised 2026-08-21, round 4) +**Date:** 2026-08-20 (revised 2026-08-22, round 5) **Status:** Draft (Lever A gated on a burst-trace prerequisite; Lever C gated on discovery) **Scope:** Client-side auction properties (server-side ad templates inactive). Measurements come from a pilot news property; identifying details stay out of this document per repository policy, and sanitized measurement artifacts live outside the spec. The pilot rollout is scoped to the Fastly adapter (see the activation matrix). @@ -63,7 +63,12 @@ Nothing in this design may create impression, win, or billing signals for ad uni - structural safety holds: every ad unit is measurable by a **side-effect-free size estimator** that walks own enumerable _data_ properties only. Units carrying accessors, custom `toJSON`, or otherwise unmeasurable values dispatch solo — publisher getter/`toJSON` code must never execute during admission (regression-tested with a stateful `toJSON`); - batch bounds hold after admission: at most 4 pending calls, at most 32 total ad units, and an estimated unit payload ≤ 160 KiB UTF-8. -**Authoritative size bound (adapter seam).** The estimator above is a cheap pre-filter; the **authoritative** bound lives in the adapter's `buildRequests`, where the final body — including then-current EIDs and context — actually exists. The coalescer carries **constituent-call boundaries** into the merged auction, and `buildRequests` serializes the final body and, if it would exceed 192 KiB (64 KiB margin under the endpoint's 256 KiB limit), **splits along constituent boundaries and returns multiple transport descriptors** — Prebid dispatches each as a separate HTTP request within the _same_ auction. No oversized request is ever sent, no post-response replay exists, and the one-auction/one-event-stream contract holds. A server 413 is therefore not an expected outcome; if one occurs anyway it resolves as an ordinary no-bid auction exactly as an oversized solo call does today. +**Authoritative size bound (adapter seam).** The estimator above is a cheap pre-filter; the **authoritative** bound lives in the adapter's `buildRequests`, where the final body — including then-current EIDs and context — exists. Mechanics and guarantees: + +- **Boundary ownership:** the coalescer assigns each merged segment an **internal auction ID**, passes it through the merged `requestBids` call, and keys constituent boundaries by it in a module-scoped map. `buildRequests` reads the ID from the **auction-scoped `bidderRequest`** it receives (Prebid deep-clones ad units after enrichment, so unit-attached markers are unreliable and are not used); the map entry is deleted at auction end. A test proves no boundary marker leaks into bidder params or the wire payload. +- **EID sanitization:** the adapter deep-copies collected EIDs into plain JSON data (own enumerable data properties only) **once per auction** before any serialization, so stateful getters/`toJSON` in publisher-supplied EID objects are never invoked repeatedly. EIDs that cannot be sanitized this way make the body **unsplittable** (single descriptor). +- **Splitting:** if the final body exceeds 192 KiB (64 KiB margin under the endpoint's 256 KiB limit) and contains multiple constituents, `buildRequests` splits along constituent boundaries into multiple transport descriptors — Prebid dispatches each as a separate HTTP request within the _same_ auction, preserving the one-auction/one-event-stream contract. +- **Narrowed guarantee + singleton rule:** the "no oversized request" guarantee applies to **multi-constituent** descriptors only. A single constituent whose body exceeds the limit (e.g., through large common EIDs/context) cannot be split further and **dispatches as-is — exactly the behavior an oversized solo call has today** (sent, possibly answered 413, resolving as a no-bid auction). Oversized-common-EIDs is an explicit test case. **Ineligible arrivals — one rule for all classes:** any call failing any admission predicate **first synchronously flushes the pending batch, then dispatches solo**. Nothing overtakes an earlier caller; event order is preserved. @@ -88,7 +93,7 @@ Nothing in this design may create impression, win, or billing signals for ad uni **Promise semantics.** Every held call returns a facade promise settling with the values described above. Rejection fan-out (synchronous dispatch failure) is per detached segment and is defensive unit coverage — Prebid's public promise is resolve-only, so it cannot be proven against the real artifact. One `auctionInit`/`auctionEnd` event stream per merged segment replaces N — a documented, operator-visible analytics change. -**Coalescing telemetry (required for rollout).** A dedicated, bounded, untrusted metadata field — `constituentCallCount` (integer, `2..=4`) — travels in the `/auction` request body as a **top-level sibling of** `config`, _not_ inside it (the `config` object is publisher auction context filtered by `allowed_context_keys`; unknown keys there are dropped). Semantics: absent = solo/legacy call; present only on merged dispatches; the count reaches `buildRequests` via the constituent boundaries the coalescer attaches, and split descriptors each carry their own segment's count. Server side: the endpoint parses and clamps the field, `AuctionObservationContext` and the auction event summary row carry it, and the Tinybird datasource gains the column — **schema deployed before the emitting binary**. Endpoint, telemetry, sink-serialization, and schema tests are part of the implementation, not optional. +**Coalescing telemetry (required for rollout).** Merged dispatches carry a dedicated, bounded, untrusted metadata object as a **top-level sibling of** `config` in the `/auction` body (never inside the `allowed_context_keys`-filtered context): `coalesced: { group, size, part, parts }` where `group` is an opaque per-logical-merge ID, `size` is the constituent count (`2..=4`), and `part`/`parts` describe split descriptors (`1/1` when unsplit). Semantics: absent = solo/legacy; **logical merges are counted by distinct `group`**, so split descriptors never overcount; malformed or out-of-range values are **dropped as absent, never clamped** into valid observations. Server side: the endpoint parses and validates the object, `AuctionObservationContext` and the auction event summary row carry the fields, the Tinybird datasource gains the columns — **schema deployed before the emitting binary** — and a named rollup (`auction_coalescing_daily`: logical merges, constituent totals, split counts per property/day) plus a dashboard panel constitute the continuous guardrail; a raw column alone does not. Endpoint, telemetry, sink-serialization, schema, and rollup tests are part of the implementation. ### Lever B — GAM preconnect hint (opt-in) @@ -98,7 +103,7 @@ When enabled, GPT `head_inserts` emits `` **before the GPT bootstrap inserts** (asserted by a transformed-HTML ordering test), without `crossorigin` — ad requests are cookie-credentialed. Browsers may partially perform or skip hints; best-effort by nature. **Scope of claim:** GPT scripts (including `pubads_impl`) are first-party proxied; the hint can only affect the **first direct ad request**, and the claim is "may reduce" its connection setup. -**Browser coverage (blocking gate):** the flag emits to every browser, and per-engine emission restriction would require request-scoped UA gating (out of scope). Therefore **enablement is blocked until every engine above a 5% traffic share on the property is verified** with engine-appropriate low-level tooling (Chromium: NetLog; WebKit/Firefox: their native network logging), each with defined event predicates, a per-engine sample floor (at least 20 cold-start runs), and per-engine acceptance rules. Current repository browser coverage is Chromium-only, so building this matrix is part of the lever's cost — an unverified material engine blocks the flag; immaterial engines (below 5%) are documented as unverified in the approval. +**Browser coverage (blocking gates, two distinct scopes):** the flag emits to every browser, and per-engine emission restriction would require request-scoped UA gating (out of scope). Two gates with different scopes follow: + +- **Privacy/conformance gate — every emitted engine.** The no-request-write invariant is universal, so it must be verified on **every browser engine family that implements the preconnect hint** (Chromium/Blink, WebKit, Gecko — long-tail browsers embed these engines, which closes the enumeration), regardless of traffic share. An engine family that implements the hint and cannot be verified blocks enablement outright — a traffic threshold is never a privacy waiver. +- **Performance gate — material engines.** Reuse/benefit acceptance is required per engine **above a 5% traffic share** on the property; engines below 5% are documented as performance-unverified in the approval (their conformance is still required above). + +Each engine uses engine-appropriate low-level tooling (Chromium: NetLog; WebKit/Firefox: their native network logging) with defined event predicates, a per-engine sample floor (at least 20 cold-start runs), and per-engine acceptance rules. Current repository browser coverage is Chromium-only, so building this matrix is part of the lever's cost. **Governance (required before any property enables it):** - The flag is a property-level boolean and GPT head insertion has no request-scoped jurisdiction or consent input; the approval must therefore cover **every jurisdiction served by the deployed configuration**. -- **The approval is a durable artifact** recording: configuration version, jurisdiction inventory and unknown-jurisdiction handling, verified browser matrix, approver, date, expiry, and a re-approval trigger when the served scope changes. +- **The approval is a durable artifact** recording: configuration version, jurisdiction inventory and unknown-jurisdiction handling, verified browser matrix, approver, date, expiry, and a re-approval trigger when the served scope changes. Its identifier and expiry are what the config's `gam_preconnect_approval_id`/`gam_preconnect_valid_until` carry, so runtime enforcement and the artifact cannot drift apart. - **Verification protocol:** pinned browser build per engine, fresh profile per run, cold cache and socket pools, capture mode stated explicitly, raw logs treated as sensitive (redacted to the GAM host, retained only for the acceptance window). For Chromium: NetLog with the speculative socket joined to the first ad request via source IDs, using a **versioned parser algorithm with fixtures and one controlled end-to-end test**; equivalent engine-appropriate tooling for others. Current audit tooling has no NetLog surface — building this capture path is part of the lever's implementation cost. -- **Enforceable invariant:** no HTTP request writes on the speculative connection before the normal first GAM ad request — defined as no HTTP/2 or HTTP/3 HEADERS/DATA frames **and** no HTTP/1 request writes. "Normal first GAM request" = the first `securepubads.g.doubleclick.net` ad request initiated by GPT for the document. Verified including GPC-set, CMP-unresolved, and CMP-denied cases; connection-level protocol frames (settings, pings) are inherent to preconnect and permitted. +- **Enforceable invariant:** no HTTP request writes on **any connection attributable to the hint** — not only the socket the first ad request later reuses — before the normal first GAM ad request; defined as no HTTP/2 or HTTP/3 HEADERS/DATA frames **and** no HTTP/1 request writes, on every `securepubads.g.doubleclick.net` connection the browser opened speculatively. "Normal first GAM request" = the first `securepubads.g.doubleclick.net` ad request initiated by GPT for the document. **Terminal observation horizon:** a run in which no GAM ad request ever occurs is not indeterminate — it observes all hint-attributed connections for the full scripted session plus 60 s of idle, and passes only if zero request writes occurred in that window. Verified including GPC-set, CMP-unresolved, and CMP-denied cases; connection-level protocol frames (settings, pings) are inherent to preconnect and permitted. - Rollback trigger: any observed request write before the normal ad request disables the flag — an immediate-stop conformance failure regardless of performance. ### Lever C — earlier first auction (discovery first; design contingent) @@ -122,10 +128,10 @@ When enabled, GPT `head_inserts` emits ` Date: Sun, 23 Aug 2026 15:40:35 -0700 Subject: [PATCH 11/12] Address seventh review round on ad-latency spec MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Never evict live boundary entries — a full map dispatches new calls solo. Replace adapter-time EID sanitization with one raw-to-plain snapshot at collection reused by transport, split descriptors, bookkeeping, and cookie sync, with an explicit drop-and-count policy for non-serializable entries, retiring the per-constituent fallback. Make the drain bound hold end to end: config injection implies the 60s cache cap (cache-retaining response classes carry no config), document lifetime uses dual monotonic/wall-clock expiry with pageshow/visibilitychange resume guards, and propagation is observed across the served POP scope. Require a provider-free signed preflight token (input digest, navigation nonce, expiry; replay/mismatch/ navigation/expiry tests) for Lever C's client option, passive attach-rate measurement before any producer, admitted/rendered terminal outcomes, hard /auction and provider-execution non-regression gates, a keepalive 64 KiB transport gate, and blocking provider-log redaction. Turn Lever B's conformance gate into a finite product/OS/WebView matrix with an explicitly risk-accepted tail, bind emission to a signed approval manifest validated for hash, scope, and expiry, and record pre-consent DNS/TCP/TLS disclosure in the manifest. Use Wilson bounds for proportions with BCa reserved for paired continuous metrics, define no-render scoring, fix live control limits with minimum volumes and stale-data actions, constrain coalesced metadata to part <= parts <= size <= 4 with group consistency and a counted invalid-metadata signal, and surface the merged requestBids call visibility in Phase 0 and the real-artifact suite. --- ...-prebid-ad-latency-optimizations-design.md | 56 ++++++++++++------- 1 file changed, 36 insertions(+), 20 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index e8872c763..15bbb62d6 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -1,6 +1,6 @@ # Prebid Ad-Latency and Auction-Load Optimizations — Design -**Date:** 2026-08-20 (revised 2026-08-23, round 6) +**Date:** 2026-08-20 (revised 2026-08-23, round 7) **Status:** Draft (Lever A gated on a burst-trace prerequisite; Lever C gated on discovery) **Scope:** Client-side auction properties (server-side ad templates inactive). Measurements come from a pilot news property; identifying details stay out of this document per repository policy, and sanitized measurement artifacts live outside the spec. The pilot rollout is scoped to the Fastly adapter (see the activation matrix). @@ -49,7 +49,7 @@ Nothing in this design may create impression, win, or billing signals for ad uni **Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves the existing synchronous pass-through path and public observable behavior (the bundle bytes necessarily change when the coalescer ships). -**Coalescing-config lifetime.** The injected coalescing config carries an issue timestamp (diagnostic) and a maximum document lifetime (24 h). There is **one lifetime contract, and it is monotonic**: the shim latches config receipt at document start and tracks age with a `performance.now()`-based monotonic clock; it self-disables coalescing once document age exceeds the lifetime, so long-lived documents quiesce on their own regardless of wall-clock skew (see Drain, which uses the same contract). +**Coalescing-config lifetime.** The injected coalescing config carries an issue timestamp and a maximum document lifetime (24 h). There is **one lifetime contract — dual expiry with resume guards**: the shim latches config receipt and tracks a `performance.now()`-based monotonic document age **and** a wall-clock ceiling (`issuedAt + lifetime + 1 h skew allowance`), self-disabling coalescing when either trips, re-checked on `pageshow`/`visibilitychange` so system suspension cannot extend the lifetime (see Drain, which relies on this exact contract). **Snapshot at enqueue.** The coalescer snapshots each admitted request the way Prebid itself does on entry: shallow-copy the request object and snapshot the ad-unit **array membership and order** (retaining unit references). Later additions of request keys or array push/splice do not retroactively change the issued call; unit-object mutations are caught by dispatch-time revalidation. @@ -65,11 +65,11 @@ Nothing in this design may create impression, win, or billing signals for ad uni **Authoritative size bound (adapter seam).** The estimator above is a cheap pre-filter; the **authoritative** bound lives in the adapter's `buildRequests`, where the final body — including then-current EIDs and context — exists. Mechanics and guarantees: -- **Boundary ownership:** the coalescer assigns each merged segment an **internal auction ID**, passes it through the merged `requestBids` call, and keys constituent boundaries by it in a module-scoped map. `buildRequests` reads the ID from the **auction-scoped `bidderRequest`** it receives (Prebid deep-clones ad units after enrichment, so unit-attached markers are unreliable and are not used). **Cleanup covers every path:** the entry is deleted when `buildRequests` consumes it, on synchronous dispatch failure of the segment, and at auction end/settlement as a backstop; the map is additionally bounded (oldest-entry eviction at 16 entries) so a cancellation path that reaches none of those hooks cannot grow it unboundedly. A test proves no boundary marker leaks into bidder params or the wire payload, and a test covers each cleanup path. -- **EID sanitization:** the adapter deep-copies collected EIDs into plain JSON data (own enumerable data properties only) **once per auction** before any serialization; every descriptor serialization uses that sanitized copy, so stateful getters/`toJSON` in publisher-supplied EID objects run at most once per auction. This is not a new side effect: the current collector already retains `uid.ext` by reference and re-reads it at each serialization, so once-per-auction sanitization strictly reduces accessor invocations. -- **Sanitization-failure fallback:** if the sanitizing copy itself throws, the merged body is **never forced into one descriptor**. `buildRequests` falls back to **one descriptor per constituent** — byte-for-byte the bodies today's solo dispatches would have produced — so a failed sanitization can only reproduce solo behavior, not create a merged >256 KiB request out of individually valid calls. +- **Boundary ownership:** the coalescer assigns each merged segment an **internal auction ID**, passes it through the merged `requestBids` call, and keys constituent boundaries by it in a module-scoped map. `buildRequests` reads the ID from the **auction-scoped `bidderRequest`** it receives (Prebid deep-clones ad units after enrichment, so unit-attached markers are unreliable and are not used). **Cleanup covers every path:** the entry is deleted when `buildRequests` consumes it, on synchronous dispatch failure of the segment, and at auction end/settlement as a backstop. The map is bounded at 16 entries, but **live entries are never evicted** — evicting an active auction's entry before its deferred `buildRequests` runs would silently lose split boundaries and telemetry. When the map is full, **new calls dispatch solo** (no new merges are created) until entries release through the cleanup paths above. A test proves no boundary marker leaks into bidder params or the wire payload, and a test covers each cleanup path. +- **EID snapshot (one per auction, all consumers):** EID collection materializes the collected EIDs into plain JSON data (own enumerable data properties, one accessor read) **at collection time**, and that snapshot is the only EID representation any consumer sees — transport serialization, split descriptors, callback bookkeeping, and the `ts-eids` cookie sync all reuse it. Today the collector runs at three sites and retains `uid.ext` by reference, re-reading accessors at every serialization; the snapshot collapses those to one materialization per auction, strictly reducing accessor invocations. +- **Non-serializable EID policy (explicit):** an EID entry whose materialization throws is **dropped from that auction's EID set**, incrementing a diagnostic counter; the auction proceeds without it. Nothing else changes shape: no forced merged descriptor, no thrown auction, no per-body divergence. Because the snapshot is plain data, splitting is always serialization-safe. - **Splitting:** if the final body exceeds 192 KiB (64 KiB margin under the endpoint's 256 KiB limit) and contains multiple constituents, `buildRequests` splits along constituent boundaries into multiple transport descriptors — Prebid dispatches each as a separate HTTP request within the _same_ auction, preserving the one-auction/one-event-stream contract. -- **Narrowed guarantee + singleton rule:** the "no oversized request" guarantee applies to **multi-constituent** descriptors only. A single-constituent descriptor whose body exceeds the limit (e.g., through large common EIDs/context) cannot be split further and **dispatches as-is — exactly the behavior an oversized solo call has today** (sent, possibly answered 413, resolving as a no-bid auction). Oversized-common-EIDs is an explicit test case, in both the sanitized-split and per-constituent-fallback paths. +- **Narrowed guarantee + singleton rule:** the "no oversized request" guarantee applies to **multi-constituent** descriptors only. A single-constituent descriptor whose body exceeds the limit (e.g., through large common EIDs/context) cannot be split further and **dispatches as-is — exactly the behavior an oversized solo call has today** (sent, possibly answered 413, resolving as a no-bid auction). Oversized-common-EIDs is an explicit test case. **Ineligible arrivals — one rule for all classes:** any call failing any admission predicate **first synchronously flushes the pending batch, then dispatches solo**. Nothing overtakes an earlier caller; event order is preserved. @@ -92,13 +92,13 @@ Nothing in this design may create impression, win, or billing signals for ad uni **Shared-auction behavior (documented, accepted).** Partitioned callbacks do not partition Prebid's global auction state: unscoped `setTargetingForGPTAsync()` applies targeting for every returned unit and a bare `pubads.refresh()` can deliver a co-merged caller's units — attributed as publisher delivery because all constituent bookkeeping registers first. Operators enabling the flag accept this; acceptance tests treat cross-caller delivery within a merged auction as permitted. -**Promise semantics.** Every held call returns a facade promise settling with the values described above. Rejection fan-out (synchronous dispatch failure) is per detached segment and is defensive unit coverage — Prebid's public promise is resolve-only, so it cannot be proven against the real artifact. One `auctionInit`/`auctionEnd` event stream per merged segment replaces N — a documented, operator-visible analytics change. +**Promise semantics.** Every held call returns a facade promise settling with the values described above. Rejection fan-out (synchronous dispatch failure) is per detached segment and is defensive unit coverage — Prebid's public promise is resolve-only, so it cannot be proven against the real artifact. One `auctionInit`/`auctionEnd` event stream per merged segment replaces N — a documented, operator-visible analytics change. **The visible `requestBids` calls themselves also change:** anything hooking or wrapping `pbjs.requestBids` (analytics adapters, publisher wrappers) observes fewer, later, merged invocations. Phase 0 must capture whether any such hook exists on the property, and the real-artifact suite documents the observable difference. -**Coalescing telemetry (required for rollout).** Merged dispatches carry a dedicated, bounded, untrusted metadata object as a **top-level sibling of** `config` in the `/auction` body (never inside the `allowed_context_keys`-filtered context): `coalesced: { group, size, part, parts }` with a **typed, bounded schema**: `group` is a client-generated UUIDv4 string (exactly 36 characters, format-validated); `size` is the constituent count (`2..=4`); `parts` is `1..=8` and `part` satisfies `1 <= part <= parts`. Semantics: absent = solo/legacy; **logical merges are counted by distinct `group`**, so split descriptors never overcount; any field failing its type, bound, or cross-field constraint drops the **whole object as absent, never clamped** into valid observations. Server side: the endpoint parses and validates the object, `AuctionObservationContext` and the auction event summary row carry the fields, the Tinybird datasource gains the columns — **schema deployed before the emitting binary** — and a named rollup (`auction_coalescing_daily`: logical merges, constituent totals, split counts per property/day) plus a dashboard panel constitute the continuous guardrail; a raw column alone does not. Endpoint, telemetry, sink-serialization, schema, and rollup tests are part of the implementation. +**Coalescing telemetry (required for rollout).** Merged dispatches carry a dedicated, bounded, untrusted metadata object as a **top-level sibling of** `config` in the `/auction` body (never inside the `allowed_context_keys`-filtered context): `coalesced: { group, size, part, parts }` with a **typed, bounded schema**: `group` is a client-generated UUIDv4 string (exactly 36 characters, format-validated); `size` is the constituent count; splits run along constituent boundaries, so the full cross-field constraint is `1 <= part <= parts <= size <= 4` (with `size >= 2`). Semantics: absent = solo/legacy; **logical merges are counted by distinct `group`**, so split descriptors never overcount; rows sharing a `group` must agree on `size`/`parts` (**group consistency**, validated in the rollup — a disagreeing group is counted invalid, not merged); any field failing its type, bound, or cross-field constraint drops the **whole object as absent, never clamped** — and every such drop increments a **`coalesced_invalid` counter** carried through the observation context, event row, and rollup, so malformed merged traffic cannot masquerade as solo and silently satisfy drain. Server side: the endpoint parses and validates the object, `AuctionObservationContext` and the auction event summary row carry the fields, the Tinybird datasource gains the columns — **schema deployed before the emitting binary** — and a named rollup (`auction_coalescing_daily`: logical merges, constituent totals, split counts per property/day) plus a dashboard panel constitute the continuous guardrail; a raw column alone does not. Endpoint, telemetry, sink-serialization, schema, and rollup tests are part of the implementation. ### Lever B — GAM preconnect hint (opt-in) -**Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false` — plus two required-when-enabled siblings that make the governance artifact machine-enforced rather than merely recorded: `gam_preconnect_approval_id` (opaque string identifying the durable approval artifact) and `gam_preconnect_valid_until` (RFC 3339). Emission requires `gam_preconnect = true` **and** server time before `valid_until`; an expired or missing approval leaves the hint un-emitted and logs a warning at config load (automatic disable, no redeploy needed to stop emitting past expiry). +**Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false` — plus required-when-enabled siblings that bind emission to a **validated, signed approval manifest** rather than an arbitrary id and date: `gam_preconnect_approval_id` (identifier of the manifest), `gam_preconnect_approval_hash` (digest of the signed manifest content), and `gam_preconnect_valid_until` (RFC 3339, mirrored from the manifest). At config load the server validates the manifest reference: signature valid, hash matches, the manifest's recorded **configuration version and jurisdiction scope match the effective configuration**, and server time is before `valid_until`. Any failure leaves the hint un-emitted with a warning (automatic disable, no redeploy needed) — an approval that does not describe the running configuration proves nothing and must not enable emission. When enabled, GPT `head_inserts` emits `` **before the GPT bootstrap inserts** (asserted by a transformed-HTML ordering test), without `crossorigin` — ad requests are cookie-credentialed. Browsers may partially perform or skip hints; best-effort by nature. @@ -106,7 +106,7 @@ When enabled, GPT `head_inserts` emits `48 h late) reporting days extend the hold rather than pass it, with two consecutive inconclusive days escalating to manual review. + **Live guardrails and their sources:** | Signal | Source | Type / cadence | @@ -204,7 +214,13 @@ Numeric gates: | Lever B socket reuse + setup time | scheduled engine-appropriate capture runs | synthetic; daily during the B hold | | Consent-denied network activity (Lever B) | scheduled capture runs (GPC / unresolved / denied) | synthetic; daily during the B hold | -**Drain (Lever A):** the shim tracks document age with a **latched monotonic clock** (`performance.now()`-based age, immune to wall-clock skew) and self-disables coalescing once the age exceeds the injected 24 h lifetime. Drain completion is an **additive** bound: **observed** config propagation + 60 s HTML freshness + 24 h document lifetime after the flag returns to `0`. Propagation is not asserted from a platform figure: after the disabling push, a **multi-POP probe** (requests routed through at least 3 distinct Fastly POPs, or the platform's config-version observability where available) must observe the new configuration before the propagation clock stops; ~5 minutes is the expectation, the observation is the bound. Because auction telemetry is best-effort and can drop rows, zero merged-auction rows alone cannot prove drain: the check requires the `auction_coalescing_daily` rollup to show **zero logical merges while overall `/auction` row volume for the property confirms pipeline liveness** (an ingestion-freshness check). Lever B rollback completion is the same observed propagation + 60 s HTML freshness. +**Drain (Lever A):** three requirements make the bound hold end to end. + +- **Injection implies the cache cap.** The 60 s freshness term is only valid for responses that actually carry it. Response classes that retain origin/CDN cache policy (bot, prefetch, consent-denied/unresolved, non-GET — today's request-scoped cache-cap skips) **must not inject the coalescing client config**; any HTML response that injects either lever's config carries a browser-cache policy of at most 60 s. A test asserts the implication in both directions. +- **Document lifetime uses dual expiry with resume guards.** The shim tracks a latched monotonic (`performance.now()`-based) document age **and** a wall-clock ceiling (`issuedAt + lifetime + 1 h skew allowance`); coalescing self-disables when **either** trips, re-checked on `pageshow`/`visibilitychange` so suspension gaps (monotonic clocks may exclude suspended time in Firefox/WebKit) are caught at resume. Over-disabling is safe; only under-disabling breaks the bound. +- **Propagation is observed across the served scope.** After the disabling push, the effective **rendered** configuration must be observed across the POP scope actually serving the property (enumerated from recent traffic logs, or via platform config-version observability) — a fixed small probe count does not establish convergence for an eventually consistent config store; ~5 minutes is the expectation, the observation is the bound. + +Drain completion is then the **additive** bound: observed propagation + 60 s HTML freshness + the dual-expiry document lifetime after the flag returns to `0`. Because auction telemetry is best-effort and can drop rows, zero merged-auction rows alone cannot prove drain: the check requires the `auction_coalescing_daily` rollup to show **zero logical merges and no anomalous invalid-metadata counts while overall `/auction` row volume for the property confirms pipeline liveness** (an ingestion-freshness check). Lever B rollback completion is the same observed propagation + 60 s HTML freshness. ## Rollout From 1aab56e32142f52d176c18aa1b4598863c38774d Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sun, 23 Aug 2026 16:52:53 -0700 Subject: [PATCH 12/12] Address eighth review round on ad-latency spec Store constituent boundaries as code-to-membership sets partitioned in buildRequests, with unexpected or duplicate codes dispatching one unsplit descriptor and a post-handoff mutation test. Scope the EID snapshot to merged auctions with its own auction-keyed lifetime that outlives the boundary record, independent snapshots for overlapping reverse-settling auctions, an unchanged window-0 path, and a data-property walk that never invokes custom toJSON. Enforce the 24h lifetime fail-closed on every wrapped call and flush via monotonic age, elapsed wall time from receipt, and backward-jump disable, dropping the skew allowance. Make the Lever B approval an embedded JCS-canonical Ed25519-signed envelope with pinned rotating keys, a configuration digest computed at load, and per-emission expiry rechecks; stop emission at valid_until minus propagation and freshness so cached HTML cannot outlive approval. Extend the injection cache bound to CDN directives, Expires, and validators so an origin 304 cannot revive flag-carrying HTML. Bind Lever C's producer and joiner both to the preflight token, name the pending-join owner for single use, require pure fingerprints in both architectures, drop keepalive outright given the aggregate quota, widen the privacy audit to every provider and mediator including APS trace logs and client-visible debug bodies, and ban linkable stable hashes. Give live rollback rules degradation polarity with Bonferroni-split looks, restrict daily batches to feasible metrics with p95 on the accumulated pool, define the numeric no-render censoring value and the paired count-ratio estimator for Lever A's drop gate. Make the matrix cell product-OS-version-consent, preserve the full hint-transport graph before sanitization, add the navigation-keyed document span so the passive gate is computable, include coordination overhead in the load gate, and add map-capacity-at-flush, wrapper-order, invalid-metadata propagation, and group-consistency tests with a baseline-compared invalid signal that cannot hold drain open. --- ...-prebid-ad-latency-optimizations-design.md | 54 ++++++++++--------- 1 file changed, 29 insertions(+), 25 deletions(-) diff --git a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md index 15bbb62d6..9f670b502 100644 --- a/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md +++ b/docs/superpowers/specs/2026-08-20-prebid-ad-latency-optimizations-design.md @@ -1,6 +1,6 @@ # Prebid Ad-Latency and Auction-Load Optimizations — Design -**Date:** 2026-08-20 (revised 2026-08-23, round 7) +**Date:** 2026-08-20 (revised 2026-08-23, round 8) **Status:** Draft (Lever A gated on a burst-trace prerequisite; Lever C gated on discovery) **Scope:** Client-side auction properties (server-side ad templates inactive). Measurements come from a pilot news property; identifying details stay out of this document per repository policy, and sanitized measurement artifacts live outside the spec. The pilot rollout is scoped to the Fastly adapter (see the activation matrix). @@ -49,7 +49,7 @@ Nothing in this design may create impression, win, or billing signals for ad uni **Config:** `[integrations.prebid] request_bids_coalesce_ms` — `u32`, default `0`, validated `0..=250`. Injected as `requestBidsCoalesceMs`, omitted when `0`. Default `0` preserves the existing synchronous pass-through path and public observable behavior (the bundle bytes necessarily change when the coalescer ships). -**Coalescing-config lifetime.** The injected coalescing config carries an issue timestamp and a maximum document lifetime (24 h). There is **one lifetime contract — dual expiry with resume guards**: the shim latches config receipt and tracks a `performance.now()`-based monotonic document age **and** a wall-clock ceiling (`issuedAt + lifetime + 1 h skew allowance`), self-disabling coalescing when either trips, re-checked on `pageshow`/`visibilitychange` so system suspension cannot extend the lifetime (see Drain, which relies on this exact contract). +**Coalescing-config lifetime.** The injected coalescing config carries an issue timestamp (diagnostic) and a maximum document lifetime (24 h). The contract enforces the 24 h bound, fail-closed, with no skew allowance: at config receipt the shim latches both the wall-clock time and the monotonic (`performance.now()`) reading, and **evaluates the expiry check on every wrapped `requestBids` call and every flush** (not only on lifecycle events). Coalescing self-disables when **any** of these holds: monotonic age ≥ 24 h; elapsed wall time since receipt ≥ 24 h; or the wall clock reads **earlier than the latched receipt time or the previous check** (a backward jump — fail closed, since elapsed time is no longer provable). A suspended monotonic clock is covered by the wall-time comparison at the next call/flush; a negatively skewed wall clock is covered by the backward-jump rule. The enforced lifetime is therefore **at most 24 h of provable elapsed time**, which is what Drain's additive bound uses. **Snapshot at enqueue.** The coalescer snapshots each admitted request the way Prebid itself does on entry: shallow-copy the request object and snapshot the ad-unit **array membership and order** (retaining unit references). Later additions of request keys or array push/splice do not retroactively change the issued call; unit-object mutations are caught by dispatch-time revalidation. @@ -65,9 +65,9 @@ Nothing in this design may create impression, win, or billing signals for ad uni **Authoritative size bound (adapter seam).** The estimator above is a cheap pre-filter; the **authoritative** bound lives in the adapter's `buildRequests`, where the final body — including then-current EIDs and context — exists. Mechanics and guarantees: -- **Boundary ownership:** the coalescer assigns each merged segment an **internal auction ID**, passes it through the merged `requestBids` call, and keys constituent boundaries by it in a module-scoped map. `buildRequests` reads the ID from the **auction-scoped `bidderRequest`** it receives (Prebid deep-clones ad units after enrichment, so unit-attached markers are unreliable and are not used). **Cleanup covers every path:** the entry is deleted when `buildRequests` consumes it, on synchronous dispatch failure of the segment, and at auction end/settlement as a backstop. The map is bounded at 16 entries, but **live entries are never evicted** — evicting an active auction's entry before its deferred `buildRequests` runs would silently lose split boundaries and telemetry. When the map is full, **new calls dispatch solo** (no new merges are created) until entries release through the cleanup paths above. A test proves no boundary marker leaks into bidder params or the wire payload, and a test covers each cleanup path. -- **EID snapshot (one per auction, all consumers):** EID collection materializes the collected EIDs into plain JSON data (own enumerable data properties, one accessor read) **at collection time**, and that snapshot is the only EID representation any consumer sees — transport serialization, split descriptors, callback bookkeeping, and the `ts-eids` cookie sync all reuse it. Today the collector runs at three sites and retains `uid.ext` by reference, re-reading accessors at every serialization; the snapshot collapses those to one materialization per auction, strictly reducing accessor invocations. -- **Non-serializable EID policy (explicit):** an EID entry whose materialization throws is **dropped from that auction's EID set**, incrementing a diagnostic counter; the auction proceeds without it. Nothing else changes shape: no forced merged descriptor, no thrown auction, no per-body divergence. Because the snapshot is plain data, splitting is always serialization-safe. +- **Boundary ownership:** the coalescer assigns each merged segment an **internal auction ID**, passes it through the merged `requestBids` call, and keys constituent boundaries by it in a module-scoped map. `buildRequests` reads the ID from the **auction-scoped `bidderRequest`** it receives (Prebid deep-clones ad units after enrichment, so unit-attached markers are unreliable and are not used). Boundaries are stored as **code → constituent membership**, never numeric index ranges: between dispatch-time revalidation and `buildRequests`, Prebid hooks, ad-unit validation, and `beforeRequestBids` listeners can still remove or reorder units (or add a client bidder), so `buildRequests` partitions the units it actually receives by code membership. A received code that belongs to no constituent, or a duplicated code, marks the partition unreliable: the auction dispatches as a **single unsplit descriptor** with a diagnostic counter — never a mis-split. Post-handoff mutation (unit removal, reorder, and injected client bidder via a `beforeRequestBids` listener) is an explicit test. **Cleanup covers every path:** the entry is deleted when `buildRequests` consumes it, on synchronous dispatch failure of the segment, and at auction end/settlement as a backstop. The map is bounded at 16 entries, but **live entries are never evicted** — evicting an active auction's entry before its deferred `buildRequests` runs would silently lose split boundaries and telemetry. When the map is full, **new calls dispatch solo** (no new merges are created) until entries release through the cleanup paths above. A test proves no boundary marker leaks into bidder params or the wire payload, and a test covers each cleanup path. +- **EID snapshot (merged auctions only; defined lifetime):** for a **merged** auction, EID collection materializes the collected EIDs into plain JSON data **at collection time** — walking own enumerable **data** properties with one accessor read and **never invoking custom `toJSON`** — and that snapshot is the only EID representation any of that auction's consumers see: transport serialization, split descriptors, callback bookkeeping, and the `ts-eids` cookie sync. The snapshot is keyed by the internal auction ID in its own map with its own lifetime — it **outlives the boundary record** (which is consumed at `buildRequests`) and is deleted only when the auction settles and its callback bookkeeping/cookie sync completes. Overlapping auctions settling in reverse order each hold an independent snapshot (tested). **Scope:** solo dispatches and the `window 0` path keep today's collection behavior byte-for-byte — snapshot and drop semantics apply only where merging is active, so the flag-off path is unchanged. +- **Non-serializable EID policy (explicit):** an EID entry whose materialization throws is **dropped from that merged auction's EID set**, incrementing a diagnostic counter; the auction proceeds without it. Nothing else changes shape: no forced merged descriptor, no thrown auction, no per-body divergence. Because the snapshot is plain data, splitting is always serialization-safe. - **Splitting:** if the final body exceeds 192 KiB (64 KiB margin under the endpoint's 256 KiB limit) and contains multiple constituents, `buildRequests` splits along constituent boundaries into multiple transport descriptors — Prebid dispatches each as a separate HTTP request within the _same_ auction, preserving the one-auction/one-event-stream contract. - **Narrowed guarantee + singleton rule:** the "no oversized request" guarantee applies to **multi-constituent** descriptors only. A single-constituent descriptor whose body exceeds the limit (e.g., through large common EIDs/context) cannot be split further and **dispatches as-is — exactly the behavior an oversized solo call has today** (sent, possibly answered 413, resolving as a no-bid auction). Oversized-common-EIDs is an explicit test case. @@ -98,7 +98,7 @@ Nothing in this design may create impression, win, or billing signals for ad uni ### Lever B — GAM preconnect hint (opt-in) -**Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false` — plus required-when-enabled siblings that bind emission to a **validated, signed approval manifest** rather than an arbitrary id and date: `gam_preconnect_approval_id` (identifier of the manifest), `gam_preconnect_approval_hash` (digest of the signed manifest content), and `gam_preconnect_valid_until` (RFC 3339, mirrored from the manifest). At config load the server validates the manifest reference: signature valid, hash matches, the manifest's recorded **configuration version and jurisdiction scope match the effective configuration**, and server time is before `valid_until`. Any failure leaves the hint un-emitted with a warning (automatic disable, no redeploy needed) — an approval that does not describe the running configuration proves nothing and must not enable emission. +**Config:** `[integrations.gpt] gam_preconnect` — `bool`, default `false` — plus a required-when-enabled `[integrations.gpt.gam_preconnect_approval]` **embedded manifest envelope**, so validation needs no fetch and has no fetch-failure branch. The envelope carries the manifest content itself (id, configuration version, jurisdiction inventory, verified matrix summary, accepted tail, approver, `valid_until`), canonically encoded as **RFC 8785 (JCS) JSON**, with an Ed25519 signature over the canonical bytes. Trust and rotation: the verifying public keys ship in the binary's pinned trust set (two active keys during rotation overlap; a manifest verifies against either). At config load the server verifies: signature valid against a pinned key; the manifest's **configuration-version digest** (SHA-256 over the canonical encoding of the effective integration configuration, computed by the server at load) matches the digest the manifest records; and jurisdiction scope matches the deployed scope declaration. **Expiry is rechecked at every emission**, not only at load — GPT `head_inserts` builds synchronously from `Settings`, so the emission path compares current server time against `valid_until` per document; a long-lived adapter (Axum/Cloudflare hold startup state) therefore stops emitting at expiry without a reload. Any validation failure leaves the hint un-emitted with a warning — an approval that does not describe the running configuration proves nothing and must not enable emission. When enabled, GPT `head_inserts` emits `` **before the GPT bootstrap inserts** (asserted by a transformed-HTML ordering test), without `crossorigin` — ad requests are cookie-credentialed. Browsers may partially perform or skip hints; best-effort by nature. @@ -109,13 +109,13 @@ When enabled, GPT `head_inserts` emits `48 h late) reporting days extend the hold rather than pass it, with two consecutive inconclusive days escalating to manual review. +**Live control limits (per signal, fixed before enablement):** every live guardrail carries a written tuple — estimator, fixed threshold, **rollback-bound polarity**, minimum daily volume, and missing/stale-data action — so a rollback decision is reproducible from the recorded rules alone. The polarity is the opposite of acceptance: acceptance proved benefit via a lower bound clearing a threshold, but copying that rule live would let mere daily-sample **uncertainty** trigger rollback. A live metric triggers rollback only on **evidence of degradation** — its one-sided 95% **upper** bound (for benefit metrics) falls below the minimum acceptable level, or a conformance invariant fails (those act immediately, no statistics involved). **Batching by metric feasibility:** daily 10-pair batches evaluate only median-level and proportion metrics; p95-gated metrics are evaluated once on the accumulated hold pool (≥70 pairs by day 7) at the day-7/day-9 read. **Repeated-look control:** the seven daily looks share a Bonferroni-split alpha (0.05/7 per look) so the hold's family-wise false-rollback rate stays at 5%. Fill/revenue alerts require a minimum volume of 1,000 impressions per property-day — a below-volume day is **inconclusive**, and inconclusive or missing/stale (>48 h late) reporting days extend the hold rather than pass it, with two consecutive inconclusive days escalating to manual review. **Live guardrails and their sources:** @@ -216,11 +219,12 @@ Numeric gates: **Drain (Lever A):** three requirements make the bound hold end to end. -- **Injection implies the cache cap.** The 60 s freshness term is only valid for responses that actually carry it. Response classes that retain origin/CDN cache policy (bot, prefetch, consent-denied/unresolved, non-GET — today's request-scoped cache-cap skips) **must not inject the coalescing client config**; any HTML response that injects either lever's config carries a browser-cache policy of at most 60 s. A test asserts the implication in both directions. -- **Document lifetime uses dual expiry with resume guards.** The shim tracks a latched monotonic (`performance.now()`-based) document age **and** a wall-clock ceiling (`issuedAt + lifetime + 1 h skew allowance`); coalescing self-disables when **either** trips, re-checked on `pageshow`/`visibilitychange` so suspension gaps (monotonic clocks may exclude suspended time in Firefox/WebKit) are caught at resume. Over-disabling is safe; only under-disabling breaks the bound. +- **Injection implies a full-stack cache bound.** The 60 s freshness term is only valid when every cache layer honors it. Response classes that retain origin/CDN cache policy (bot, prefetch, consent-denied/unresolved, non-GET — today's request-scoped cache-cap skips) **must not inject either lever's client config**. Any HTML response that does inject it carries: browser `Cache-Control: max-age=60` (or stricter), **capped or removed CDN-targeted directives** (`Surrogate-Control`, `Fastly-Surrogate-Control`, `CDN-Cache-Control`, `Cloudflare-CDN-Cache-Control`), a removed `Expires`, and — critically — **no revivable validators**: `ETag`/`Last-Modified` are stripped (today conditional-header stripping applies only on ad-stack paths), because an origin `304` would otherwise revive previously transformed flag-carrying HTML past every `max-age`. A test asserts the implication in both directions across all named headers. +- **Emission stops before approval expiry can be outlived.** Lever B's per-emission expiry check uses `valid_until − (observed propagation + 60 s freshness)` as its effective cutoff, so no cached hint-carrying HTML can still be served when the approval lapses. +- **Document lifetime is enforced per call, fail-closed.** Per the lifetime contract above, expiry is evaluated on every wrapped call and flush against both the monotonic age and elapsed wall time since receipt, with backward wall-clock jumps disabling coalescing outright. Suspension gaps (monotonic clocks may exclude suspended time in Firefox/WebKit) are caught by the wall-time comparison at the next call; over-disabling is safe, and no path extends the lifetime past 24 h of provable elapsed time. - **Propagation is observed across the served scope.** After the disabling push, the effective **rendered** configuration must be observed across the POP scope actually serving the property (enumerated from recent traffic logs, or via platform config-version observability) — a fixed small probe count does not establish convergence for an eventually consistent config store; ~5 minutes is the expectation, the observation is the bound. -Drain completion is then the **additive** bound: observed propagation + 60 s HTML freshness + the dual-expiry document lifetime after the flag returns to `0`. Because auction telemetry is best-effort and can drop rows, zero merged-auction rows alone cannot prove drain: the check requires the `auction_coalescing_daily` rollup to show **zero logical merges and no anomalous invalid-metadata counts while overall `/auction` row volume for the property confirms pipeline liveness** (an ingestion-freshness check). Lever B rollback completion is the same observed propagation + 60 s HTML freshness. +Drain completion is then the **additive** bound: observed propagation + 60 s HTML freshness + the 24 h enforced document lifetime after the flag returns to `0` — and this **time bound is authoritative**. The telemetry check is confirmatory, not gating: because auction telemetry is best-effort and can drop rows, the `auction_coalescing_daily` rollup showing **zero logical merges while overall `/auction` row volume confirms pipeline liveness** corroborates drain; the invalid-metadata signal is compared against its **trailing pre-disable baseline** (a server-observed rate, not a value clients can set) and an anomaly **escalates to investigation without extending drain** — `coalesced` metadata is client-supplied, so a rule that let malformed traffic hold drain open indefinitely would hand an attacker a veto. Lever B rollback completion is the same observed propagation + 60 s HTML freshness. ## Rollout