Skip to content

feat: extend AiMetrics, MetricSummary, and TrackMetricsOf#300

Open
mattrmc1 wants to merge 1 commit into
mainfrom
mmccarthy/AIC-2727/extend-metrics-ai-metrics
Open

feat: extend AiMetrics, MetricSummary, and TrackMetricsOf#300
mattrmc1 wants to merge 1 commit into
mainfrom
mmccarthy/AIC-2727/extend-metrics-ai-metrics

Conversation

@mattrmc1

@mattrmc1 mattrmc1 commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Extends AiMetrics, MetricSummary, and TrackMetricsOf to bring the .NET AI SDK to spec parity with JS, Python, and Java for metric tracking. Addresses AIC-2727.

AiMetrics — new optional fields

public AiMetrics(bool success, Usage? tokens = null,
    IReadOnlyList<string> toolCalls = null, double? durationMs = null)

ToolCalls carries tool keys invoked during the operation (AIRUNNER §1.5.1). DurationMs provides an explicit duration override — when set, TrackMetricsOf uses it instead of the stopwatch measurement (AITRACK §1.1.13.2).

MetricSummary — new optional fields

public record struct MetricSummary(
    double? DurationMs,
    Feedback? Feedback,
    Usage? Tokens,
    bool? Success,
    double? TimeToFirstTokenMs,
    IReadOnlyList<string> ToolCalls = null,      // NEW
    string ResumptionToken = null                 // NEW
);

ToolCalls accumulates keys from TrackToolCall invocations. ResumptionToken surfaces the tracker's resumption token on the summary (AITRACK §1.1.17.2).

TrackMetricsOf — tool call auto-tracking & duration override

TrackMetricsOf now extracts ToolCalls from AiMetrics and calls TrackToolCalls automatically. It also honors DurationMs when present, falling back to the stopwatch value when null.

The error-handling semantics now match Java: if the operation throws, duration + error are tracked and the exception propagates. If the metrics extractor throws, duration is tracked but error is NOT — the AI operation itself succeeded.

TrackToolCall — accumulates keys on summary

TrackToolCall now appends to an internal list (lock-guarded) so that Summary.ToolCalls reflects all tool invocations. The existing per-call $ld:ai:tool_call event emission is unchanged.

How to test

dotnet test pkgs/sdk/server-ai/test/LaunchDarkly.ServerSdk.Ai.Tests.csproj --framework net8.0

9 new test cases covering backward compat, new field population, auto tool call tracking, duration override vs. stopwatch fallback, extractor failure semantics, and summary accumulation.

Known limitations

None. All changes are additive — existing constructors and call sites compile and behave identically.


Note

Low Risk
Additive API and observability behavior in the AI metrics SDK; the main behavioral change is clearer TrackMetricsOf error/duration semantics, which aligns with other language SDKs rather than changing security or data paths.

Overview
Brings the .NET server AI tracker in line with other SDKs by extending AiMetrics with optional ToolCalls and DurationMs, and MetricSummary with ToolCalls and ResumptionToken.

TrackMetricsOf now stops the timer before running the metrics extractor (so slow extractors do not inflate duration), uses DurationMs when provided, auto-emits tool-call events from AiMetrics.ToolCalls, and adjusts failure semantics: operation failures still record duration + generation error; extractor failures record duration only (no error), matching Java.

TrackToolCall appends keys to a lock-protected list so Summary.ToolCalls reflects all invocations; existing per-call $ld:ai:tool_call events are unchanged. Changes are additive for existing call sites; tests cover the new fields and behaviors.

Reviewed by Cursor Bugbot for commit b2c7bc2. Bugbot is set up for automated code reviews on this repo. Configure here.

…alls, duration override, and resumption token
@mattrmc1 mattrmc1 marked this pull request as ready for review June 29, 2026 22:57
@mattrmc1 mattrmc1 requested a review from a team as a code owner June 29, 2026 22:57

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b2c7bc2. Configure here.

IReadOnlyList<string> toolCalls;
lock (_toolCallKeys)
{
toolCalls = _toolCallKeys.Count > 0 ? _toolCallKeys.AsReadOnly() : null;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary ToolCalls live list alias

Medium Severity

Summary assigns ToolCalls from _toolCallKeys.AsReadOnly(), so the returned MetricSummary keeps a live view of the tracker’s internal list. Later TrackToolCall / TrackMetricsOf updates change a previously read summary’s tool list, and concurrent reads while tracking can throw during enumeration.

Fix in Cursor Fix in Web

Triggered by learned rule: Tracker methods must defensive-copy caller-provided collections

Reviewed by Cursor Bugbot for commit b2c7bc2. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant