feat(benchmark): publish bounded paid-agent aggregates

[abrichr]

What

Publishes bounded aggregate results for the paid claude-sonnet-5
computer-use agent arm across three pinned, synthetic reference environments,
while enforcing the open-core evidence boundary.

Vertical	Trials	Primary outcome	Over-halt	Silent incorrect	Mean list-price cost
Insurance / openIMIS	3	3/3 correct	0/3	0/3	$0.4793/run
Lending / Frappe Lending	6	6/6 correct writes; 5/6 clean	1/6 (post-write cost-cap halt)	0/6	$0.4240/run
Healthcare / local OpenEMR	6	0/6 correct; 6/6 missing write	0/6	0/6	$0.8901/run

Evidence boundary

• Public source retains the generic paid-run mechanism in
  openadapt_flow/benchmark/agent_baseline.py: explicit opt-in, action/cost
  caps, usage accounting, and self-report that is never trusted for scoring.
• Public docs retain only bounded multi-trial aggregates, methods, failure
  taxonomy, and caveats.
• Raw per-run JSON/JSONL, environment fingerprints, screenshots, detailed spend
  records, and the three application-specific driver/oracle recipes remain in
  private OpenAdaptAI/openadapt-corpus PR feat: record→compile→replay→heal reference implementation (v0) #1 with file-level provenance and
  SHA-256 hashes.
• The public branch is one clean commit; the private files and recipes are not
  retained in its history.
• The source/release guard rejects every per-system agent-arm/ artifact,
  raw rows.jsonl, detailed cost ledger, named driver recipe, or private
  provenance banner from public source, wheel, and sdist.

The public diff contains no raw row, screenshot, fingerprint, detailed cost
ledger, customer/deployment recipe, deployment-derived threshold/tuning, grown
corpus, or real-EMR dataset.

Honest scope

The paid-agent trials used newly provisioned baselines separate from the
earlier compiled/API subsets. N is 3-6 per environment. This is local
engineering evidence, not a matched three-arm matrix, publication result,
certification claim, or broad comparison against computer-use agents.

Exact reviewed state

• base: bd2a27caf300e58d2b19dcee3b1493c5ab6fa813 (current
  origin/main, including merged fix: enforce the public source artifact boundary #225 and feat(lending): add collateral-write fault class + single-surface arm for honest cross-domain comparability #219)
• head: a78a9dbd791451a448a22b5a7a5e262951a532c3
• commits: 1
• public-artifacts.json: byte-identical to current main. feat(benchmark): publish bounded paid-agent aggregates #216 adds no
  artifact-suffix payload, so the merged positive inventory remains
  authoritative.

Validation

• focused release contract: 30 passed
• benchmark environment registry/verification: 11 passed
• Ruff check and format check: passed
• source/release consistency: passed
• claims: 16 claims / 7 CI-supported, passed
• cross-file consistency: passed
• bounded broad non-E2E suite: 2,750 passed, 13 skipped, 1 deselected
  • excluded the unrelated real-screen openadapt-capture import to honor the
    no-local-GUI boundary
  • excluded the separate OCR stress module and the documented 900-second
    browser+OCR harness; neither is changed by this PR
• fresh wheel + sdist built and inspected with the release validator:
  • sdist SHA-256:
    4dcd50289db317e5e5d254940be2021232bed1e08f6cf96b58b6434ccafdbb02
  • wheel SHA-256:
    3ff345f5cc581ef78f3ba260dabc4a9041fea4ffea7e1c306877b5e2f8e34395
  • the bounded aggregate README and reviewed inventory are present
  • neither archive contains raw rows, ledgers, per-system agent-arm paths,
    named recipes, private-corpus signatures, openIMIS, AGPL, or third-party
    notice payload
• protected runs on this head at final preparation:
  • Paper 29852887248: success
  • Validate claims 29852887814: success
  • CI 29852887260: in progress; 9 applicable jobs green, with only the
    unit/coverage and browser-E2E jobs still running and no failures

No provider, paid run, deployment, release, local GUI, or input injection was
performed by this preparation.

Dependency and reconciliation order

1. Hold for the root-owned macOS exact-main gate.
2. Merge Flow docs(paper): honest disclosures for adversarial review #2 (closed-world, statistics, positioning, ethics) #217 first.
3. Refetch the resulting exact main, rebase feat(benchmark): publish bounded paid-agent aggregates #216 again if it advanced, and
   rerun the affected source/package/claims and deterministic fixture gates.
4. Merge feat(benchmark): publish bounded paid-agent aggregates #216 only after protected CI and the final source-boundary review are
   green.
5. Counterpart public claims then reconcile from the merged Flow evidence:
   OpenAdaptAI/openadapt-ops#57 and OpenAdaptAI/openadapt-web#264.
6. Flow feat(eligibility): API-first 270/271 eligibility waterfall (Stedi client, payer route map, document-hash verified artifacts) #147 and feat(openimis): effect-verified insurance eligibility-check reference workflow + showcase #145 are not dependencies of feat(benchmark): publish bounded paid-agent aggregates #216. If resumed, feat(eligibility): API-first 270/271 eligibility waterfall (Stedi client, payer route map, document-hash verified artifacts) #147's additive
   payer-route inventory entry should land before the broader feat(openimis): effect-verified insurance eligibility-check reference workflow + showcase #145 branch;
   feat(openimis): effect-verified insurance eligibility-check reference workflow + showcase #145 must then rebase last, preserve feat(benchmark): publish bounded paid-agent aggregates #216's aggregate prose/source guard and
   feat(eligibility): API-first 270/271 eligibility waterfall (Stedi client, payer route map, document-hash verified artifacts) #147's inventory entry, and regenerate/review the combined positive
   inventory. This sequencing preserves the functioning eligibility/reference
   capabilities rather than removing them to avoid overlap.

Do not merge this PR yet.