Skip to content

Vercel AI SDK rerank() not instrumented (wrapper and auto-instrumentation) #1662

@braintrust-bot

Description

@braintrust-bot

Summary

The Vercel AI SDK exports a stable rerank() core function for reranking documents by relevance, but neither the wrapper (wrapAISDK) nor the auto-instrumentation plugin instruments it. Calls to rerank() produce no Braintrust spans. Combined with the missing embed/embedMany instrumentation (#1625), the entire RAG retrieval pipeline (embed → search → rerank) is unobservable through Braintrust.

What is missing

  • Wrapper (js/src/wrappers/ai-sdk/ai-sdk.ts): wrapAISDK only wraps generateText, streamText, generateObject, streamObject, and Agent class methods. No handling of rerank.
  • Auto-instrumentation config (js/src/auto-instrumentations/configs/ai-sdk.ts): Only defines configs for text/object generation and Agent variants. No config for rerank.
  • Channels (js/src/instrumentation/plugins/ai-sdk-channels.ts): No channel definition for rerank.
  • Plugin (js/src/instrumentation/plugins/ai-sdk-plugin.ts): No handler for reranking calls.
  • Vendor types (js/src/vendor-sdk-types/ai-sdk-common.ts): AISDKNamespaceBase does not include rerank.

Upstream reference

  • AI SDK rerank() reference: https://ai-sdk.dev/docs/reference/ai-sdk-core/rerank
  • Stable core function (not experimental), exported from 'ai' module.
  • import { rerank } from 'ai'
  • Parameters: model (reranking model), documents (strings or JSON objects), query (search query), optional topN, maxRetries, abortSignal, headers, providerOptions.
  • Returns { originalDocuments, rerankedDocuments, ranking, response } where ranking contains relevance scores per document.

Braintrust docs status

The Braintrust tracing guide lists "Vercel AI SDK" as a supported integration but does not enumerate which AI SDK functions are covered. rerank is not mentioned (not_found).

Precedent in this repo

OpenAI embeddings are instrumented in all three layers (wrapper, auto-instrumentation config, plugin). Reranking is the natural complement to embeddings in RAG pipelines and would benefit from the same observability: latency tracking, relevance score capture, model identification, and query/document logging.

Local files inspected

  • js/src/wrappers/ai-sdk/ai-sdk.ts
  • js/src/auto-instrumentations/configs/ai-sdk.ts
  • js/src/instrumentation/plugins/ai-sdk-channels.ts
  • js/src/instrumentation/plugins/ai-sdk-plugin.ts
  • js/src/vendor-sdk-types/ai-sdk-common.ts

Metadata

Metadata

Assignees

No one assigned

    Labels

    bot-automationIssues generated by an agent automation

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions