CCM-33842: Improve LiteLLM trace quality for cost attribution

[shreyas70]

Summary

Improves liteLLM GenAI trace quality so token-bearin spans reliably expose model identity, token usage, and enough raw data to survive provider/library shape changes.

Verified against an exported trace set (data3.json, collected from llm-model-service in QA): every LLM trace carried two litellm_request spans (one enriched, one empty), and only ~50% of spans had gen_ai.response.model + input tokens.

data3.json

Changes

• Deduplicate spans. LiteLLM's async entry points (acompletion/aembedding) internally re-dispatch to their sync counterparts (completion/embedding) via run_in_executor with a copied context. Since all four functions are wrapped, the nested sync call created a second, empty litellm_request span. Added a contextvars re-entrancy guard so each provider call yields exactly one span.
• Guarantee gen_ai.response.model. Token-bearing spans now always set the response model, falling back to the request model when the provider omits it, so spans can join rate cards by provider + model.
• Bedrock execution model. Extract x-amzn-bedrock-model-id from LiteLLM _hidden_params (additional headers / ResponseMetadata.HTTPHeaders) into aws.bedrock.execution_model_id, distinguishing the requested inference profile from the actual executing model.
• Bounded raw usage fallback. Opt-in via HA_GEN_AI_RAW_CAPTURE_ENABLED (default off). Serializes the usage object into gen_ai.response.usage.raw (+ ..._truncated), bounded by HA_GEN_AI_RAW_CAPTURE_MAX_BYTES (default 8192) and sanitized to omit prompt/content/header/credential fields. Provides resilience if LiteLLM changes response labels.

Tests

Added regression tests in test/instrumentation/litellm/:

• Async embedding/completion emit exactly one litellm_request span.
• Object-style and dict-style embedding usage.
• Response-model fallback to request model.
• Bedrock execution model capture from _hidden_params.
• Raw usage capture opt-in enabled and disabled-by-default.

All 14 LiteLLM instrumentation tests pass.

Config

• HA_GEN_AI_RAW_CAPTURE_ENABLED (default false)
• HA_GEN_AI_RAW_CAPTURE_MAX_BYTES (default 8192)

Expected impact

• One litellm_request span per provider call (was two).
• gen_ai.response.model + input token usage present on all token-bearing spans.
• Bedrock inference-profile requests preserve both requested profile and actual execution model when Bedrock exposes it.

Made with Cursor

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}