-
Notifications
You must be signed in to change notification settings - Fork 3
Description
Summary
The Vercel AI SDK exports a stable rerank() core function for reranking documents by relevance, but neither the wrapper (wrapAISDK) nor the auto-instrumentation plugin instruments it. Calls to rerank() produce no Braintrust spans. Combined with the missing embed/embedMany instrumentation (#1625), the entire RAG retrieval pipeline (embed → search → rerank) is unobservable through Braintrust.
What is missing
- Wrapper (
js/src/wrappers/ai-sdk/ai-sdk.ts):wrapAISDKonly wrapsgenerateText,streamText,generateObject,streamObject, and Agent class methods. No handling ofrerank. - Auto-instrumentation config (
js/src/auto-instrumentations/configs/ai-sdk.ts): Only defines configs for text/object generation and Agent variants. No config forrerank. - Channels (
js/src/instrumentation/plugins/ai-sdk-channels.ts): No channel definition forrerank. - Plugin (
js/src/instrumentation/plugins/ai-sdk-plugin.ts): No handler for reranking calls. - Vendor types (
js/src/vendor-sdk-types/ai-sdk-common.ts):AISDKNamespaceBasedoes not includererank.
Upstream reference
- AI SDK
rerank()reference: https://ai-sdk.dev/docs/reference/ai-sdk-core/rerank - Stable core function (not experimental), exported from
'ai'module. import { rerank } from 'ai'- Parameters:
model(reranking model),documents(strings or JSON objects),query(search query), optionaltopN,maxRetries,abortSignal,headers,providerOptions. - Returns
{ originalDocuments, rerankedDocuments, ranking, response }whererankingcontains relevance scores per document.
Braintrust docs status
The Braintrust tracing guide lists "Vercel AI SDK" as a supported integration but does not enumerate which AI SDK functions are covered. rerank is not mentioned (not_found).
Precedent in this repo
OpenAI embeddings are instrumented in all three layers (wrapper, auto-instrumentation config, plugin). Reranking is the natural complement to embeddings in RAG pipelines and would benefit from the same observability: latency tracking, relevance score capture, model identification, and query/document logging.
Local files inspected
js/src/wrappers/ai-sdk/ai-sdk.tsjs/src/auto-instrumentations/configs/ai-sdk.tsjs/src/instrumentation/plugins/ai-sdk-channels.tsjs/src/instrumentation/plugins/ai-sdk-plugin.tsjs/src/vendor-sdk-types/ai-sdk-common.ts