[Bug] call_llm spans not exported in multi-agent setups due to GeneratorExit breaking span.end() in _call_llm_async

[nappa0326]

Description

In multi-agent setups using transfer_to_agent, the call_llm spans for parent agents are created but never exported to Cloud Trace / OpenTelemetry backends. Only sub-agent spans are correctly exported.

The spans are referenced (via parent_span_id) by child spans, confirming they are created — but the span bodies themselves are absent from the trace backend (orphan spans).

Root Cause

_call_llm_async in base_llm_flow.py uses tracer.start_as_current_span('call_llm') as a context manager around an async for loop that yields responses:

async def _call_llm_with_tracing():
    with tracer.start_as_current_span('call_llm') as span:
        ...
        async for llm_response in agen:
            yield llm_response  # GeneratorExit raised here when transfer_to_agent occurs

When the LLM returns transfer_to_agent, the sub-agent runs (can take 10+ seconds), then the AsyncGenerator is closed, raising GeneratorExit inside the with block.

Inside OTel's start_as_current_span finally block:

finally:
    context.detach(token)  # raises ValueError: Token was created in a different Context
    span.end()             # NEVER REACHED

Because context.detach() raises ValueError (the contextvars token is invalid after the async context switch caused by sub-agent execution), span.end() is never called. Spans that are never ended are never exported.

Relationship to Existing Issues

This is the same root cause as #501 and #1670 (GeneratorExit + OTel contextvars in AsyncGenerator), both of which were fixed in base_agent.py (commit 955f477). However, base_llm_flow.py:_call_llm_async was not updated with the same fix.

Steps to Reproduce

1. Deploy a multi-agent system to Vertex AI Agent Engine with OpenTelemetry tracing enabled
2. Configure agents with transfer_to_agent tool usage
3. Query the parent agent so it transfers to a sub-agent
4. Check Cloud Trace or BigQuery linked dataset for call_llm spans

Expected: call_llm spans for all agents appear in the trace
Actual: Only sub-agent call_llm spans appear; parent agent call_llm spans are missing (but referenced as parent_span_id by child spans)

Proposed Fix

Replace start_as_current_span with explicit span management to ensure span.end() is always called even when GeneratorExit is raised:

span = tracer.start_span('call_llm')
ctx = trace.set_span_in_context(span)
token = context.attach(ctx)
try:
    async for llm_response in agen:
        yield llm_response
finally:
    try:
        context.detach(token)
    except ValueError:
        pass
    span.end()  # Always called, even on GeneratorExit

Environment

• ADK version: 1.23.0 (also reproduced on later versions)
• Deployment: Vertex AI Agent Engine
• Python: 3.11
• Related: opentelemetry-python#2606 (upstream OTel issue with AsyncGenerator context propagation)