Skip to content

[bot] Missing instrumentation for Databricks SDK for Java (Model Serving execution) #148

Description

@braintrust-bot

Summary

The Braintrust Java SDK does not instrument the official Databricks SDK for Java (com.databricks:databricks-sdk-java). Databricks is already listed by Braintrust as a supported "cloud platform" for model access, but this repo has no instrumentation for the SDK's ServingEndpointsAPI, which is the execution surface Java applications use to invoke Databricks Model Serving / Foundation Model API endpoints (chat completions, text completions, and embeddings hosted or proxied by Databricks).

What is missing

The com.databricks:databricks-sdk-java SDK provides this execution-oriented surface, not instrumented anywhere in this repo:

API Class Description
query() ServingEndpointsAPI / ServingEndpointsService Invokes a Databricks Model Serving endpoint — covers chat-completions-style requests, legacy completions, and embeddings, including Databricks Foundation Model APIs (e.g. served Llama/DBRX/GTE models) and external-model endpoints proxying OpenAI/Anthropic/etc.
Streaming query responses ServingEndpointsAPI Server-sent-event style incremental chat responses from a serving endpoint

This is a genuine model-execution surface (request/response over a served model), distinct from the admin/CRUD operations the same SDK also exposes for creating or managing endpoints (out of scope by this audit's own criteria).

Why existing instrumentation does not cover this

  • None of the six existing instrumentation modules (openai_2_15_0, anthropic_2_2_0, genai_1_18_0, langchain_1_8_0, springai_1_0_0, aws_bedrock_2_30_0) reference Databricks classes or the com.databricks package.
  • The Databricks SDK uses its own HTTP client/transport layer, entirely separate from the OpenAI/Anthropic/Google/AWS clients this repo currently wraps, so no existing module incidentally covers it.

Braintrust docs status

  • Databricks is listed as a supported cloud platform at https://www.braintrust.dev/docs/integrations/ai-providers (models accessible through it), but there is no mention of Java SDK-level instrumentation, or of ServingEndpointsAPI/model-serving execution tracing, anywhere in Braintrust's docs: unclear (provider is nominally supported at the proxy/config level, but code-level Java instrumentation of the official SDK's execution calls is not_found)

Upstream sources

Local repo files inspected

  • settings.gradle (lines 28–33) — lists all instrumentation modules; no Databricks module present
  • braintrust-sdk/instrumentation/ — contains openai_2_15_0, anthropic_2_2_0, genai_1_18_0, langchain_1_8_0, springai_1_0_0, aws_bedrock_2_30_0; no Databricks directory
  • Full-repo grep for databricks (case-insensitive) — zero matches

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions