Skip to content

docs: publish bounded cross-vertical paid-agent evidence#57

Merged
abrichr merged 1 commit into
mainfrom
docs/agent-arm-verticals
Jul 21, 2026
Merged

docs: publish bounded cross-vertical paid-agent evidence#57
abrichr merged 1 commit into
mainfrom
docs/agent-arm-verticals

Conversation

@abrichr

@abrichr abrichr commented Jul 21, 2026

Copy link
Copy Markdown
Member

What changed

Publishes the bounded paid computer-use-agent aggregates from merged
OpenAdaptAI/openadapt-flow#216
(exact Flow main e740a7e4b68ed5c6461bdcd6fbd086e8ac75c8be) alongside the earlier compiled/API reference
evidence for the three pinned synthetic environments.

Vertical Trials Primary outcome Over-halt Silent incorrect Mean list-price cost
Insurance / openIMIS 3 3/3 correct 0/3 0/3 $0.4793/run
Lending / Frappe Lending 6 6/6 correct writes; 5/6 clean 1/6 post-write cost-cap halt 0/6 $0.4240/run
Healthcare / local OpenEMR 6 0/6 correct; 6/6 missing_write 0/6 0/6 $0.8901/run

The old duplicate cross-vertical section is consolidated, and stale
"agent arm omitted/not run" wording is removed from the current docs.

Dependency and honest scope

  • The public aggregate source is merged Flow #216 at exact main
    e740a7e4b68ed5c6461bdcd6fbd086e8ac75c8be.
  • Paid trials used separately provisioned baselines from the earlier
    compiled/API subsets. They are not matched head-to-head rows or a three-arm
    matrix.
  • N is 3-6 per environment. This is local engineering evidence, not a
    publication, certification, broad computer-use-agent claim, customer result,
    or design-partner result.
  • All paid tasks used the Browser (Playwright) Beta substrate. They are not
    Windows, RDP, Citrix, or VDI evidence.
  • The OpenEMR agent exhausted its action budget without the required effect;
    the oracle classification is missing_write, not over-halt.
  • Public docs retain bounded aggregates, methods, failure taxonomy, and
    caveats only. Raw per-run rows, screenshots, fingerprints, detailed cost
    ledgers, and application-specific driver/oracle recipes remain private.

Website/catalog counterpart: OpenAdaptAI/openadapt-web#264.

Exact prepared state

  • base: 3c3c71770d84c2969bc7219246647475d2b5c82d
  • head: 6a79700bab9c87a346e672f3071e44551166eed9
  • commits: 1

Validation

  • docs tests: 68 passed
  • mkdocs build --strict: passed
  • stale current-claim search: passed

@abrichr abrichr changed the title docs: add measured cross-vertical computer-use agent arm results docs: publish bounded cross-vertical paid-agent evidence Jul 21, 2026
Signed-off-by: abrichr <richard.abrich@mldsai.com>
@abrichr
abrichr force-pushed the docs/agent-arm-verticals branch from 736cbc5 to 6a79700 Compare July 21, 2026 18:04
@abrichr
abrichr merged commit 1f37a69 into main Jul 21, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant