fix(langchain): resolve request model from ls_model_name metadata for LangGraph tool spans#4316
Conversation
… LangGraph tool spans LangGraph tool-orchestrated chat models (e.g. init_chat_model with model_provider=bedrock_converse) don't expose the model in serialized kwargs or invocation_params, so set_request_params fell back to "unknown". LangChain still provides it via the callback metadata (ls_model_name), so thread that through and use it as a fallback, matching what the response side already does. Pure fallback; normal resolution is unchanged. Fixes traceloop#3098
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughAdds an optional ChangesMetadata-based model name fallback
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
When you run a LangChain chat model through LangGraph with a provider that
doesn't surface the model in its serialized kwargs — for example
init_chat_model(..., model_provider="bedrock_converse")bound with tools — theresulting spans come out with
gen_ai.request.modelandgen_ai.response.modelset to
unknown. The span dump in #3098 shows it nicely: everything else isthere (prompts, tool calls), but the model name is gone.
I traced it down to
set_request_params. It looks for the model in two spots:the callback's
invocation_params, andserialized["kwargs"](viaset_chat_request). For something likeChatOpenAIthe name lives inserialized["kwargs"]["model_name"], so it resolves fine and nobody notices.But for LangGraph tool-orchestrated models neither spot has it, so we just give
up and write
"unknown".The thing is, LangChain does hand us the model name — it's in the callback
metadataasls_model_name, which_get_ls_paramsfills in automatically forevery chat model. We were already receiving that metadata in
on_chat_model_startand passing it to_create_llm_span, just never down toset_request_params.So the fix is small: thread the metadata through and, when the usual lookups come
up empty, fall back to
ls_model_namebefore defaulting tounknown. It's thesame key we already trust on the response side (
_extract_model_name_from_association_metadata,from #3237), so this just brings the request side in line. When the model is
resolvable the normal way, nothing changes — it's purely a fallback.
For testing I added
tests/test_request_model_extraction.py. It drives thecallback handler directly against an in-memory exporter, so it needs no cassettes
and no API keys: one test reproduces the LangGraph/bedrock_converse case (it fails
on
mainwithassert 'unknown' == 'us.anthropic...'and passes here), andanother makes sure an explicit model in
serialized["kwargs"]still takesprecedence over the metadata fallback. The full langchain suite stays green (151
passed) and ruff is clean.
Fixes #3098
Summary by CodeRabbit
Improvements
Tests