feat(voice): Background Sessions — Async Background integrations for AgentSession

[toubatbrian]

Overview

Adds background sessions: a reusable abstraction for long-lived background third-party integrations (coding agents, research pipelines, monitors, ...) that communicate asynchronously and bidirectionally with a voice AgentSession. The voice agent stays the human-facing orchestrator/voice overlay; background work runs session-scoped, survives Agent handoffs, and can never outlive its AgentSession.

Background API

from livekit.agents import AgentSession, BackgroundContext, background

@background(name="claude")
async def claude_session(ctx: BackgroundContext) -> None:
    """Handles long-running coding and repository tasks."""
    async for message in ctx.message_stream():   # voice -> background (FIFO)
        ...
        await ctx.send(update)                    # background -> voice (async update)

session = AgentSession(background=[claude_session])

• @background(name=..., description=...) returns an immutable, reusable BackgroundDefinition; the description falls back to the docstring like @function_tool.
• One generated session-level tool, lk_background_send(background_session_id, content), lets the voice LLM route messages by ID; its description deterministically lists every configured session. The name is reserved — conflicting session/Agent tools fail fast.
• BackgroundContext.send() publishes a user-role update into the conversation, using the same eager context insertion, idle waiting, per-source coalescing, handoff retargeting, and deferred generate_reply(tool_choice="none") behavior as deferred async-tool updates. Templates are configurable per definition or per session via BackgroundHandlingOptions.
• Local background_message_updated events (BackgroundMessageReceived, BackgroundReplyUpdated with scheduled/completed/interrupted/skipped) expose the message lifecycle.
• Lifecycle: runtimes start inside AgentSession.start() after the current Agent exists, keep running across handoffs, and are cancelled on aclose().

Internal refactor

The deferred-reply machinery (pending buffering, idle wait, handoff retargeting until the target Agent is stable, tail detection, reply scheduling) is extracted from _ToolExecutor into a shared _ReplyScheduler, used by both async tools and background runtimes. Existing async-tool behavior, events, and public APIs are unchanged.

Examples

examples/voice_agents/background_claude_code.py — the simplest end-to-end usage: a Claude Code (Claude Agent SDK) session as a background session. One long-lived interactive ClaudeSDKClient per voice call; every relayed user message becomes a follow-up client.query() with full session context retained, and streamed assistant text is forwarded to the voice conversation via ctx.send().

@background(name="claude_code")
async def claude_code(ctx: BackgroundContext) -> None:
    """Runs coding tasks with Claude Code in this repository..."""
    async with ClaudeSDKClient(options=options) as client:
        async for message in ctx.message_stream():   # voice -> claude code (FIFO)
            await client.query(message)
            async for msg in client.receive_response():
                ...
                await ctx.send(block.text)            # claude code -> voice

examples/voice_agents/deep-research/ — a fuller voice-fronted deep-research pipeline:

• agent.py: thin voice overlay that relays clarifying questions/progress and forwards user answers and mid-run steering via lk_background_send.
• deep_research.py: long-lived, multi-round background session (clarify -> scope -> parallel search -> fetch/extract falsifiable claims -> 3-vote adversarial verify -> synthesize). Writes a dated markdown report per round, then keeps listening for follow-ups with the previous report as context. All LLM calls use llm.chat() with forced tool calls for structured output.

Verified end to end over live voice: clarify Q&A round-trips, mid-run steering, report delivery, and follow-up rounds re-using prior context.

Testing

• tests/test_background_session.py (new): API validation, routing, FIFO/isolation, template resolution, deferred delivery, handoff retargeting, lifecycle/close races, events, reserved tool names.
• tests/test_tools.py: scheduler-extraction regressions — existing async-tool semantics (first-update inline, coalescing, call-ID stability, cancellation/drain) are unchanged.
• Full hermetic unit suite: 1273 passed, 4 skipped. Strict mypy (625 files), ruff lint/format clean.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fb30127f68

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Recheck idle after retargeting the reply context

When a pending background/tool update is retargeted after wait_for_idle() has already returned, this awaited update_chat_ctx() can take a realtime-session round trip; if the user starts talking during that await, _deliver_reply() still proceeds to session.generate_reply() without checking idleness again. In realtime handoff cases this can schedule the deferred update over an active user turn, defeating the scheduler's idle-wait contract; re-wait or hold the idle state after any awaited retarget sync before generating.

Useful? React with 👍 / 👎.

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.