Skip to content

fix(otel): skip disabled recording upload errors - #2107

Open
rosetta-livekit-bot[bot] wants to merge 5 commits into
mainfrom
sty-cellar-timorous
Open

fix(otel): skip disabled recording upload errors#2107
rosetta-livekit-bot[bot] wants to merge 5 commits into
mainfrom
sty-cellar-timorous

Conversation

@rosetta-livekit-bot

@rosetta-livekit-bot rosetta-livekit-bot Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Port of livekit/agents#6167 (AGT-3015).

Summary

  • add a process-wide upload gate that latches only on the known project data-recording-disabled 401 response
  • short-circuit Cloud trace, custom processor, OTLP/Pino log, session-report, and recording uploads after the gate disables, returning synthetic success without opening another request
  • reset the gate per Cloud tracer/session, restoring patched HTTP functions and isolating in-flight responses by generation
  • preserve OpenTelemetry 2.8 resources, processors, fanout lifecycle, and provider configuration from current main
  • retain an @livekit/agents patch changeset

Python parity

Compared the actual JS and Python PR patches and current implementations. The exact 401/body matcher, one-warning latch, supported upload paths, reset lifecycle, in-flight request semantics, ordinary 401 passthrough, and synthetic-success behavior match Python. The JS implementation adapts the shared gate across the repository's split trace, log, Pino, session-report, and multipart upload transports.

A generation token prevents a delayed disabled response from an older session disabling or warning for a newly reset session. The stale response is still rewritten to success for its own exporter.

Tests

  • strict red-green regression: custom processor initially made two requests; fixed behavior returns two successes with one server request
  • strict red-green reset regression: delayed old-generation 401 no longer relatches the new session
  • unrelated custom 401: failure then success, two requests, no gate latch
  • focused recording-disabled suite — 21 passed
  • telemetry suite — 44 passed
  • exact-head full agents suite — 110 files, 1,479 passed and 5 skipped
  • pnpm typecheck — 33/33 tasks passed
  • pnpm build — 40/40 tasks passed
  • changed-file lint, Prettier, IDE diagnostics, and git diff --check — clean; four pre-existing Pino warnings remain non-failing
  • independent final gates — Spec compliance: APPROVED and Code quality: APPROVED, with no findings

Deterministic HTTP verification

The behavior tests use the actual OpenTelemetry 2.8 HTTP export delegate and a local HTTP server:

  • known disabled 401: both export callbacks report success, server receives exactly one request
  • unrelated 401 control: first export fails, second succeeds, server receives two requests, gate remains enabled
  • reset isolation: an old delayed disabled response reports success without disabling/warning for the current generation, whose request still reaches the server
  • interceptor cleanup: reset restores the original Node HTTP request functions and clears registered targets

Cue voice is not applicable because it cannot observe telemetry transport request counts or response rewriting.


Ported from livekit/agents#6167

@rosetta-livekit-bot
rosetta-livekit-bot Bot requested a review from a team as a code owner July 24, 2026 10:19
@changeset-bot

changeset-bot Bot commented Jul 24, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 878e62d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 39 packages
Name Type
@livekit/agents Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-anthropic Patch
@livekit/agents-plugin-assemblyai Patch
@livekit/agents-plugin-azure Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-cerebras Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-did Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-fishaudio Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-hedra Patch
@livekit/agents-plugin-hume Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-krisp Patch
@livekit/agents-plugin-lemonslice Patch
@livekit/agents-plugin-liveavatar Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-minimax Patch
@livekit/agents-plugin-mistral Patch
@livekit/agents-plugin-mistralai Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-perplexity Patch
@livekit/agents-plugin-phonic Patch
@livekit/agents-plugin-protoface Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-runway Patch
@livekit/agents-plugin-sarvam Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugin-soniox Patch
@livekit/agents-plugin-tavus Patch
@livekit/agents-plugins-test Patch
@livekit/agents-plugin-trugen Patch
@livekit/agents-plugin-xai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

chenghao-mou and others added 4 commits July 24, 2026 13:54
Preserve the OpenTelemetry 2.8 provider/resource behavior while retaining the recording-disabled upload gate.

Co-authored-by: Cursor <cursoragent@cursor.com>
Short-circuit registered OTLP HTTP targets after recording is disabled and restore patched HTTP bindings when each session resets.

Co-authored-by: Cursor <cursoragent@cursor.com>
Prevent delayed disabled-recording responses from relatching the gate after a new telemetry session resets it.

Co-authored-by: Cursor <cursoragent@cursor.com>
@toubatbrian

Copy link
Copy Markdown
Contributor

Resolved the OpenTelemetry 2.8 conflicts and closed the remaining custom-processor suppression gap on exact head 878e62d7cb3cbb719b6cedd8709be6a9641e4147.

Verification includes strict red-green coverage for post-latch request suppression and cross-session reset isolation; 21 focused recording-disabled tests; 44 telemetry tests; full agents suite 1,479 passed/5 skipped; typecheck 33/33; build 40/40; deterministic real-HTTP evidence; independent Spec compliance: APPROVED and Code quality: APPROVED; exact-head CI green; zero unresolved threads; mergeable. Cue is inapplicable because the changed behavior is telemetry transport suppression.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants