(feature): add initial Runs dashboard#161
Merged
Merged
Conversation
ac7c667 to
e4307f2
Compare
There was a problem hiding this comment.
Pull request overview
Adds an initial “Run history” dashboard to the UI, backed by the durable Postgres run storage, and extends the backend + streaming pipeline so runs can be grouped and trended over time (including stable agent identity via service.name).
Changes:
- Add a new Runs dashboard view (list + charts + per-run detail) powered by
GET /api/runsandGET /api/runs/{id}/results. - Track and propagate a “golden/reference trace” so it’s plotted in performance charts but excluded from pass/fail scoring.
- Persist richer run summaries (per-metric aggregates + agent identity) and ensure
service.namesurvives streaming trace serialization.
Reviewed changes
Copilot reviewed 30 out of 30 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| ui/src/lib/types.ts | Adds persisted run-history types (Run, RunSummary, RunResultRow, etc.) and wires runs into ViewType. |
| ui/src/context/TraceProvider.tsx | Adds goldenTraceId to state and clears it on clearResults; exposes setGoldenTraceId. |
| ui/src/context/TraceContext.tsx | Extends TraceState/actions with goldenTraceId + setter. |
| ui/src/config.ts | Adds /api/runs endpoint URL. |
| ui/src/components/streaming/LiveStreamingView.tsx | Includes golden trace in uploaded traces and records goldenTraceId for later scoring/chart behavior. |
| ui/src/components/sidebar/Sidebar.tsx | Adds “Run History” navigation item and sidebar section mapping for the runs view. |
| ui/src/components/runs/RunsView.tsx | New Runs dashboard page: load runs, group/filter, show summary stats + trend charts + table + drill-in. |
| ui/src/components/runs/RunsHistoryTable.tsx | New table rendering run rows with status, group keys, pass rate bar, duration, and models. |
| ui/src/components/runs/runHistory.ts | New helpers for grouping, pass rate/duration formatting, frequency-based legends, and status/chart colors. |
| ui/src/components/runs/RunDetailView.tsx | New per-run detail view showing configuration, per-case results, and persisted tool-trajectory diffs. |
| ui/src/components/runs/PerMetricTrendChart.tsx | New chart.js per-metric avg score trends across runs. |
| ui/src/components/runs/PassRateTrendChart.tsx | New chart.js pass-rate trends (per agent line where available). |
| ui/src/components/dashboard/TraceTable.tsx | Adds trajectory-diff badge and expands rows to show tool-trajectory mismatch details. |
| ui/src/components/dashboard/PerformanceCharts.tsx | Labels/sorts the golden trace as “(reference)” and prioritizes it in chart ordering. |
| ui/src/components/dashboard/DashboardView.tsx | Excludes golden trace from scoring/table rows while still passing it to performance charts for plotting. |
| ui/src/App.tsx | Routes the new runs view to RunsView. |
| ui/src/api/client.ts | Adds listRuns, getRun, getRunResults plus a StorageUnavailableError for 503 durable-storage gating. |
| src/agentevals/trace_metrics.py | Extracts stable agent identity using service.name (with fallback) and adds it to extracted metadata. |
| src/agentevals/trace_attrs.py | Introduces OTEL_SERVICE_NAME = "service.name" constant. |
| src/agentevals/streaming/ws_server.py | Re-attaches service.name resource attribute onto each span when serializing JSONL traces. |
| src/agentevals/runner.py | Adds service_name/agent_name onto TraceResult using extract_agent_identity. |
| src/agentevals/run/result_builder.py | Ensures summary/result status mapping consistency; adds per-metric aggregates and agent list to persisted run summary. |
| src/agentevals/api/streaming_routes.py | Persists “evaluate sessions” streaming runs when durable storage is configured; reuses canonical trace serializer for /get-trace. |
| examples/zero-code-examples/strands/run.py | Sets OTEL_SERVICE_NAME by default for stable run grouping. |
| examples/zero-code-examples/pydantic-ai/run.py | Sets OTEL_SERVICE_NAME by default for stable run grouping. |
| examples/zero-code-examples/openai-agents/run.py | Sets OTEL_SERVICE_NAME by default for stable run grouping. |
| examples/zero-code-examples/ollama/run.py | Sets OTEL_SERVICE_NAME by default for stable run grouping. |
| examples/zero-code-examples/langchain/run.py | Sets OTEL_SERVICE_NAME by default for stable run grouping. |
| examples/zero-code-examples/adk/run.py | Sets OTEL_SERVICE_NAME by default for stable run grouping. |
| examples/dice_agent/agent.py | Updates the example agent model identifier. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a dashboard to be able to explore historical evaluations backed by PG.
Fixes #140