Skip to content

feat(copilot): attribute tool metrics to invoking agents - #6266

Open
j15z wants to merge 6 commits into
stagingfrom
feat/track-subagent-tool-use
Open

feat(copilot): attribute tool metrics to invoking agents#6266
j15z wants to merge 6 commits into
stagingfrom
feat/track-subagent-tool-use

Conversation

@j15z

@j15z j15z commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Preserve the invoking agent across Sim tool execution and replay paths
  • Attribute tool call, latency, and span telemetry to bounded agent names
  • Fall back to main and collapse unknown labels to other
  • Coordinate with the companion Mothership PR: https://github.com/simstudioai/mothership/pull/399

Type of Change

  • New feature

Testing

  • bun run lint
  • Full ship audit suite
  • Focused Copilot tests: 51 passing

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Aug 4, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Aug 4, 2026 10:30pm

Request Review

@cursor

cursor Bot commented Aug 4, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches copilot streaming state and shared telemetry label keys; execution semantics are mostly unchanged aside from new optional fields and metric attributes.

Overview
Copilot Sim-side tool telemetry now records which agent invoked each tool, aligned with the companion Mothership work.

Stream handling adds optional agentId on ToolCallState, taken from event.scope.agentId (default main). Main and subagent tool registration paths set it on new calls; replays use ??= so the first attribution wins when partial frames repeat with different agents.

Metrics and traces extend recordSimToolMetric and withCopilotToolSpan with gen_ai.agent.name, normalized via normalizeToolAgentId against registered catalog subagent IDs plus main, collapsing anything else to other for bounded Prometheus cardinality. The tool executor forwards toolCall.agentId ?? 'main on success, error, and unexpected throw paths.

Reviewed by Cursor Bugbot for commit 560fa26. Bugbot is set up for automated code reviews on this repo. Configure here.

@greptile-apps

greptile-apps Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR preserves invoking-agent attribution through Copilot tool handling and applies a shared bounded normalization policy to tool metrics and spans.

  • Stores the invoking agent on tool-call state, including partial and replayed calls.
  • Adds normalized agent labels to Sim tool counters, duration histograms, and OpenTelemetry spans.
  • Adds focused coverage for propagation, replay behavior, fallback attribution, and unknown-agent normalization.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported unbounded span-label issue is fixed because both tool metrics and spans now use the same catalog-backed normalization.

Important Files Changed

Filename Overview
apps/sim/lib/copilot/request/handlers/tool.ts Propagates event-scope agent IDs into main and subagent tool-call state while retaining the initial attribution during replay.
apps/sim/lib/copilot/request/metrics.ts Adds a catalog-derived bounded agent-name normalizer and includes the normalized value in tool metrics.
apps/sim/lib/copilot/request/otel.ts Applies the shared agent-name normalization to Copilot tool spans, resolving the previously reported unbounded-label issue.
apps/sim/lib/copilot/request/tools/executor.ts Forwards stored agent attribution to both span and metric emission across success and error paths.

Reviews (2): Last reviewed commit: "fix(copilot): bound tool span agent labe..." | Re-trigger Greptile

Comment thread apps/sim/lib/copilot/request/otel.ts Outdated
@j15z
j15z force-pushed the feat/track-subagent-tool-use branch from b81c66e to 560fa26 Compare August 4, 2026 22:29
@j15z

j15z commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile

@j15z

j15z commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 560fa26. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants