Skip to content

docs(TSP-1334): document new evals table features#715

Open
claude[bot] wants to merge 1 commit into
mainfrom
docs/TSP-1334
Open

docs(TSP-1334): document new evals table features#715
claude[bot] wants to merge 1 commit into
mainfrom
docs/TSP-1334

Conversation

@claude

@claude claude Bot commented Jul 10, 2026

Copy link
Copy Markdown

Summary

  • Added <Note> callout in the Scenario results section documenting column sorting (server-side, cross-page), column resizing, and the column visibility picker. Notes that the Test column is pinned and that visibility preferences are saved per view.
  • Extended the scenario results column table with four new optional cost breakdown columns: Scenario credits, Execution credits, Checks credits, and Checks actions — each noting they are toggled via the column visibility picker.
  • Expanded the Viewing dashboard insights section's runs list from a single bullet into a full column reference table, with a matching <Note> callout for table interaction features.
  • Updated the Cost and billing section to surface that the per-component breakdown is also available inline via the column visibility picker, not only through the click-to-expand modal.
  • Light update to the Runs bullet in the Evals sections overview to mention sorting and the column visibility picker.

Relates to: https://linear.app/relevance/issue/TSP-1334/

Test plan

  • Verify <Note> callouts render correctly in Mintlify preview (no attributes, single paragraph)
  • Confirm new column table rows display as expected
  • Check internal anchor links (#scenario-results, #viewing-dashboard-insights, #cost-and-billing) still resolve
  • Review sentence case on all headings and table column names

🤖 Generated with Claude Code

… column picker, cost breakdown columns)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@claude claude Bot added the docs-drafter Documentation drafted by Claude label Jul 10, 2026
@linear

linear Bot commented Jul 10, 2026

Copy link
Copy Markdown

TSP-1334

@mintlify

mintlify Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
relevanceai 🟢 Ready View Preview Jul 10, 2026, 5:02 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@github-actions

Copy link
Copy Markdown
Contributor

🎯 Vibe check

Reviewed: 1 file (1 with issues, 0 clean)

Scores

Dimension Score What's holding it back
🟡 Consistency 7/10 Repeated lowercase "agent" and "tool" in the Cost and billing section — these are Relevance AI product features and should be "Agent" / "Tool". Pattern appears across ~6 locations concentrated in lines 253, 363, 365, 369, 376. Two additional hits at lines 11 and 93.
🟢 Technical clarity 9/10 No issues found. Edge cases are well-covered (truncation behavior, plan-specific columns, Workforce-specific Tool simulation scoping). UI element names are exact.
🟡 Non-technical clarity 8/10 The intro frames Evals as "what it can do" before establishing "why you'd want it." A reader new to AI agent quality assurance has no hook for why testing matters before the feature walkthrough begins.
🟢 Structure 9/10 No issues found. Logical flow from concept → how-to → results → monitoring → publishing → cost → best practices → FAQ. Components are well-chosen for their content type.

Score key: 🟢 9–10, 🟡 6–8, 🔴 1–5. Scores are a single overall judgment about the whole PR.

Overall vibe: Solid, detailed feature doc with specific UI references, good edge-case coverage, and a clear billing breakdown that will save readers a lot of confusion. The only real problem is a capitalization pattern that breaks down in the Cost and billing section — "agent" and "tool" drift to lowercase throughout that section when they should stay "Agent" and "Tool" as Relevance AI product terms.

🔧 Issues (6)
  • build/agents/build-your-agent/evals.mdx:11 — "sub-agents and tools inside a Workforce" → "sub-agents and Tools inside a Workforce" (Tools are a Relevance AI product feature)

  • build/agents/build-your-agent/evals.mdx:93 — "a sub-agent or tool in the Workforce" and "a particular sub-agent used a given tool" — both instances of "tool" → "Tool" (the node and the thing being asserted are Relevance AI Tools)

  • build/agents/build-your-agent/evals.mdx:253 — "each Check plus the agent's tool runs" → "each Check plus the Agent's Tool runs"

  • build/agents/build-your-agent/evals.mdx:363 — "the agent's own run" → "the Agent's own run"

  • build/agents/build-your-agent/evals.mdx:365 — "the agent's own tool runs" → "the Agent's own Tool runs"

  • build/agents/build-your-agent/evals.mdx:369,376 — "the agent's own execution" (line 369), "The agent's own run during the test", "each tool the agent runs", "when the agent ran" (line 376) — all three occurrences of "agent" → "Agent" and "tool" → "Tool"

🧩 Component suggestions (1)
  • build/agents/build-your-agent/evals.mdx:383–402 — The best practices <CardGroup> is borderline per CLAUDE.md ("not appropriate for best practices that read naturally as flowing bullets"). The six cards here do have icons and a short description each, so they carry enough weight to justify the visual treatment — but if these best practices ever grow short (one-liners), consider converting to a bulleted list.
⚠️ Contradictions (1)
  • build/agents/build-your-agent/evals.mdx:11 says "Evals apply to both individual Agents and Workforces, including the sub-agents and tools inside a Workforce." The sibling build-overview.mdx lists the Build page sections (Prompt, Tools, Knowledge, Triggers, Alerts, Memory, Variables, Advanced) with no mention of the Evaluate tab. This isn't a direct factual contradiction, but build-overview.mdx will mislead readers into thinking they've seen the full Agent builder — it should eventually be updated to reference the Evaluate tab. Not a blocker for this PR but worth a follow-up ticket.
🔋 Credit usage
Item Count
Files reviewed 1
Context pages read 2
Total lines processed ~609

Files read: build/agents/build-your-agent/evals.mdx (445 lines), build/agents/build-your-agent/build-overview.mdx (25 lines), build/agents/build-your-agent/memory.mdx (164 lines)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs-drafter Documentation drafted by Claude

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant