feat: multi-stream OTEL support for VS Code Copilot token usage#1438
Open
svarlamov wants to merge 28 commits into
Open
feat: multi-stream OTEL support for VS Code Copilot token usage#1438svarlamov wants to merge 28 commits into
svarlamov wants to merge 28 commits into
Conversation
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…gration) Add a stream_type column to the sessions table, creating a compound primary key (session_id, stream_type). This enables the same session_id to have multiple stream entries (e.g., "transcript" and "otel_traces"). All existing callers pass "transcript" as the stream_type for backward compatibility. The v4 migration copies existing data with stream_type defaulting to 'transcript'. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…criptor Add PathResolverKind, StreamDescriptor types and streams()/default_transcript_format() methods to the Agent trait. Add OtelSqliteTraces variant to TranscriptFormat enum. All 12 existing agents implement default_transcript_format() returning their respective format. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds 10 unit tests covering incremental reading, batch pagination, watermark resumption, attribute/event denormalization, ID extraction, timestamp extraction, and fixture DB validation. Also fixes CAST issues for REAL-typed timestamp columns in the SQLite reader queries. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ms() override Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a checkpoint fires or sweep discovers a session, the worker now creates one ProcessingTask per stream declared by the agent's streams() method instead of a single hardcoded "transcript" task. The in_flight dedup set uses (PathBuf, String) keys so different stream_types can process concurrently. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds three end-to-end tests verifying the OTEL SQLite reader integrates correctly with CopilotAgent: span reading with event ID extraction, watermark resumption behavior, and stream descriptor declaration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…oad payload Add OtelTracePayload struct and route OTEL stream events from the transcript worker into a dedicated otel_traces array in the MetricsBatch upload body, instead of converting them to SessionEvent MetricEvents. This matches the server-side handler that reads body.otel_traces as a separate top-level array. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… payload OTEL trace events now flow through the standard MetricEvent pipeline (event_id=6) with full EventAttributes (repo_url, version, author, etc.) instead of a separate OtelTracePayload/otel_traces buffer. This removes the separate TelemetryEnvelope::OtelTraces variant, OtelTracePayload struct, submit_otel_traces_sync, and the stream_type branching in process_session_blocking. All streams are now processed uniformly through log_metrics(). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When multiple spans share the same end_time_ms at a batch boundary, the strict `>` comparison would skip remaining spans. Introduces TimestampCursorWatermark that tracks (timestamp_millis, last_span_id) and uses compound keyset pagination to guarantee no spans are lost. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Makes it clear this format is specific to GitHub Copilot's OTEL SQLite DB, consistent with other tool-specific format naming (CopilotSessionJson, CopilotEventStreamJsonl, etc). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…atermark Add "0|" to the set of known initial watermark strings so OTEL streams correctly detect first-event state. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
OtelTrace events now have their own Values type that returns MetricEventId::OtelTrace from event_id(). The transcript worker branches on stream_type to emit OtelTraceValues for OTEL streams and SessionEventValues for everything else. No shared types between the two event kinds. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When expanding agent.streams() into ProcessingTask entries, non-transcript streams (like otel_traces) need their own session records in the DB. Previously only "transcript" records were created, causing OTEL processing tasks to fail with "session not found". Also adds WatermarkType::create_initial_watermark() helper. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Multiple Copilot sessions resolve to the same shared OTEL DB. Without dedup during enqueue, a single sweep could queue N tasks for the same DB, each reading all spans from watermark=0. Track enqueued (path, stream_type) pairs within a sweep to ensure only the first session to claim a shared resource gets processed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bulk rename across all Rust source and test files. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ormat Every agent now directly implements streams() instead of relying on a default impl that called default_transcript_format(). Less indirection, clearer API. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove duplicate session record creation from daemon.rs (ensure_session_exists) and sweep_coordinator.rs (insert_new_session). The transcript worker's ensure_stream_session is now the sole owner of all session record creation, called uniformly for ALL streams (including transcript) without guards. - Add external_session_id/external_parent_session_id to CheckpointNotification and SessionToProcess so the worker has all info needed to create records - Remove stream_kind != "transcript" guards from run_sweep and handle_checkpoint_notification - Pass inferred_cwd from agent during sweep for repo_work_dir Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…cation Add `shared` field to StreamDescriptor. When true (e.g., Copilot's global agent-traces.db), the session_id used for the DB watermark record is derived from the canonical path rather than the triggering session. This ensures all Copilot sessions share a single watermark for the global OTEL DB instead of each session independently reprocessing from scratch. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add watermark_type_resolver to StreamDescriptor, allowing the effective watermark type to be determined from the resolved file path. Copilot's transcript stream uses this to return RecordIndex for .json files and ByteOffset for .jsonl files, preventing a Fatal downcast error when processing legacy session files discovered via sweep. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
For shared OTEL streams, each emitted MetricEvent now gets its session_id derived from the span's chat_session_id field using the same generate_session_id(chat_session_id, "github-copilot") formula used by the transcript stream. This ensures OTEL events and transcript events for the same Copilot session share an identical session_id, enabling correct linkage in the dataset. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…EL spans Spans without a chat_session_id now use conversation_id as the session identifier. These represent separate logical sessions (non-agent chat, older sessions) and will get their own distinct session_id. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Spans without any session identifier cannot be linked to a session and are now silently dropped rather than emitted with a synthetic path-derived session_id. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…sion_id - Add extract_event_session_id trait method (default returns None) - Copilot agent implements it: chat_session_id → conversation_id fallback - Worker uses the trait method generically — no copilot-specific knowledge - Add SQL WHERE filter in copilot_otel.rs to skip spans lacking both identifiers at the DB level (avoids reading useless rows) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
svarlamov
force-pushed
the
feat/vsc-copilot-token-usage-otel
branch
from
b22f1ff to
14cffb7
Compare
Two distinct transcript files sharing the same filename stem produced identical session_ids, causing the sweep to perpetually re-discover sessions and corrupt watermarks. Adding transcript_path to the composite PK (session_id, stream_kind, transcript_path) ensures each physical file gets its own watermark row. All query functions now include transcript_path in WHERE clauses to prevent ambiguous updates. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
streams()method on the Agent traitcopilot_otelmodule) that incrementally reads spans from VS Code Copilot'sagent-traces.dbotel_tracesstream, processed independently with separate watermarks(session_id, stream_type)to the sessions DB for independent watermark trackingotel_tracesarray in the metrics upload payload (separate fromsession_events)Architecture
Each agent can now declare multiple
StreamDescriptors viastreams(). Each stream has its own format, watermark type, and path resolver. The transcript worker creates per-stream processing tasks with dedup on(path, stream_type). The default implementation returns a single transcript stream, so existing agents are unaffected.For Copilot's OTEL stream, the path is resolved by navigating from the workspace transcript path to VS Code's
globalStorage/github.copilot-chat/agent-traces.db. An env varGIT_AI_COPILOT_OTEL_DB_PATHoverrides for testing.Test plan
task lint && task fmtclean🤖 Generated with Claude Code