Skip to content

Add execution error reporting#2880

Open
JohnPhamous wants to merge 3 commits into
mainfrom
johnphamous/eve-agent-runs-v1-workflow-client
Open

Add execution error reporting#2880
JohnPhamous wants to merge 3 commits into
mainfrom
johnphamous/eve-agent-runs-v1-workflow-client

Conversation

@JohnPhamous

Copy link
Copy Markdown
  • Adds a Workflow execution-error occurrence reporter for agent run observability.
  • Routes reports through world.executionErrors instead of a generic Workflow observability namespace, keeping the surface scoped to runtime failures.
  • Includes a changeset so the reporter can be published before Eve depends on it.

Signed-off-by: John Pham <john.pham@vercel.com>
Signed-off-by: John Pham <john.pham@vercel.com>
Signed-off-by: John Pham <john.pham@vercel.com>
@JohnPhamous JohnPhamous requested review from a team and ijjk as code owners July 11, 2026 00:57
@changeset-bot

changeset-bot Bot commented Jul 11, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 7fb11a5

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 20 packages
Name Type
@workflow/core Patch
@workflow/world Patch
@workflow/world-vercel Patch
@workflow/builders Patch
@workflow/cli Patch
@workflow/next Patch
@workflow/nitro Patch
@workflow/vitest Patch
@workflow/web-shared Patch
@workflow/web Patch
workflow Patch
@workflow/world-testing Patch
@workflow/world-local Patch
@workflow/world-postgres Patch
@workflow/astro Patch
@workflow/nest Patch
@workflow/nuxt Patch
@workflow/rollup Patch
@workflow/sveltekit Patch
@workflow/vite Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
example-nextjs-workflow-turbopack Ready Ready Preview, Comment Jul 11, 2026 1:00am
example-nextjs-workflow-webpack Ready Ready Preview, Comment Jul 11, 2026 1:00am
example-workflow Ready Ready Preview, Comment Jul 11, 2026 1:00am
workbench-astro-workflow Ready Ready Preview, Comment Jul 11, 2026 1:00am
workbench-express-workflow Ready Ready Preview, Comment Jul 11, 2026 1:00am
workbench-fastify-workflow Ready Ready Preview, Comment Jul 11, 2026 1:00am
workbench-hono-workflow Ready Ready Preview, Comment Jul 11, 2026 1:00am
workbench-nitro-workflow Ready Ready Preview, Comment Jul 11, 2026 1:00am
workbench-nuxt-workflow Ready Ready Preview, Comment Jul 11, 2026 1:00am
workbench-sveltekit-workflow Ready Ready Preview, Comment Jul 11, 2026 1:00am
workbench-tanstack-start-workflow Ready Ready Preview, Comment Jul 11, 2026 1:00am
workbench-vite-workflow Ready Ready Preview, Comment Jul 11, 2026 1:00am
workflow-docs Ready Ready Preview, Comment, Open in v0 Jul 11, 2026 1:00am
workflow-swc-playground Ready Ready Preview, Comment Jul 11, 2026 1:00am
workflow-tarballs Ready Ready Preview, Comment Jul 11, 2026 1:00am
workflow-web Ready Ready Preview, Comment Jul 11, 2026 1:00am

@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

Passed Failed Skipped Total
✅ ▲ Vercel Production 1453 0 230 1683
✅ 💻 Local Development 1617 0 219 1836
✅ 📦 Local Production 1617 0 219 1836
✅ 🐘 Local Postgres 1617 0 219 1836
✅ 🪟 Windows 153 0 0 153
✅ 📋 Other 894 0 177 1071
Total 7351 0 1064 8415

Details by Category

✅ ▲ Vercel Production
App Passed Failed Skipped
✅ astro 126 0 27
✅ example 126 0 27
✅ express 126 0 27
✅ fastify 126 0 27
✅ hono 126 0 27
✅ nextjs-turbopack 150 0 3
✅ nextjs-webpack 150 0 3
✅ nitro 126 0 27
✅ nuxt 126 0 27
✅ sveltekit 145 0 8
✅ vite 126 0 27
✅ 💻 Local Development
App Passed Failed Skipped
✅ astro-stable 128 0 25
✅ express-stable 128 0 25
✅ fastify-stable 128 0 25
✅ hono-stable 128 0 25
✅ nextjs-turbopack-canary 134 0 19
✅ nextjs-turbopack-stable 153 0 0
✅ nextjs-webpack-canary 134 0 19
✅ nextjs-webpack-stable 153 0 0
✅ nitro-stable 128 0 25
✅ nuxt-stable 128 0 25
✅ sveltekit-stable 147 0 6
✅ vite-stable 128 0 25
✅ 📦 Local Production
App Passed Failed Skipped
✅ astro-stable 128 0 25
✅ express-stable 128 0 25
✅ fastify-stable 128 0 25
✅ hono-stable 128 0 25
✅ nextjs-turbopack-canary 134 0 19
✅ nextjs-turbopack-stable 153 0 0
✅ nextjs-webpack-canary 134 0 19
✅ nextjs-webpack-stable 153 0 0
✅ nitro-stable 128 0 25
✅ nuxt-stable 128 0 25
✅ sveltekit-stable 147 0 6
✅ vite-stable 128 0 25
✅ 🐘 Local Postgres
App Passed Failed Skipped
✅ astro-stable 128 0 25
✅ express-stable 128 0 25
✅ fastify-stable 128 0 25
✅ hono-stable 128 0 25
✅ nextjs-turbopack-canary 134 0 19
✅ nextjs-turbopack-stable 153 0 0
✅ nextjs-webpack-canary 134 0 19
✅ nextjs-webpack-stable 153 0 0
✅ nitro-stable 128 0 25
✅ nuxt-stable 128 0 25
✅ sveltekit-stable 147 0 6
✅ vite-stable 128 0 25
✅ 🪟 Windows
App Passed Failed Skipped
✅ nextjs-turbopack 153 0 0
✅ 📋 Other
App Passed Failed Skipped
✅ e2e-local-dev-nest-stable 128 0 25
✅ e2e-local-dev-tanstack-start- 128 0 25
✅ e2e-local-postgres-nest-stable 128 0 25
✅ e2e-local-postgres-tanstack-start- 128 0 25
✅ e2e-local-prod-nest-stable 128 0 25
✅ e2e-local-prod-tanstack-start- 128 0 25
✅ e2e-vercel-prod-tanstack-start 126 0 27

📋 View full workflow run

@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit 7fb11a5 · Sat, 11 Jul 2026 01:17:30 GMT · run logs

Backend: vercel · app: nextjs-turbopack

Metric Scenario Avg (ms) P75 (ms) P90 (ms) P99 (ms) Samples
TTFS stream 1351 (-4.2%) 1732 🔴 1833 🔴 1863 🔴 30
TTFS hook + stream 1491 (-8.4%) 1928 🔴 2013 🔴 2126 🔴 30
STSO 1020 steps (1-20) 272 (-3.3%) 302 🔴 481 🔴 528 🔴 19
STSO 1020 steps (101-120) 409 (+1.3%) 444 🔴 479 🔴 521 🔴 19
STSO 1020 steps (1001-1020) 864 (+5.4%) 916 🔴 940 🔴 1029 🔴 19
WO stream 1351 (-4.2%) 1732 1833 1863 30
WO hook + stream 1491 (-8.4%) 1928 2013 2126 30
SL stream 4669 (-7.0%) 5613 🔴 5771 🔴 6192 🔴 30
SL hook + stream 5016 (+4.8%) 5722 🔴 5861 🔴 6164 🔴 30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body execution · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (time outside step bodies, client start → last step body exit) · SL: stream latency (first chunk write → visible to the reader)

Scenarios — stream: one step that streams chunks back to the client; no hooks, so the run stays in turbo mode · hook + stream: registers a hook before the same streaming step, which exits turbo mode · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

TTFS/WO compare client vs deployment clocks and SL compares the step runner’s clock vs the client’s (NTP-synced in CI). WO ends at the last step body exit, the closest observable proxy for the final step-completion request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant