Dataset.fetchFromBraintrust generic signature is unsound — cases come back as LinkedHashMap, and there is no way to deserialize without losing experiment linking

[one-klink-to-rule-them-all]

Summary

dev.braintrust.eval.Dataset.<INPUT, OUTPUT>fetchFromBraintrust(client, project, name, version) declares a Dataset<INPUT, OUTPUT> return type, but at runtime the DatasetCase input() / expected() values are java.util.LinkedHashMap regardless of what INPUT / OUTPUT are. As a user there is currently no way to (a) get typed Java objects out of a Braintrust-backed dataset and (b) keep the experiment linked to that dataset in the Braintrust UI. Picking either one forces giving up the other.

Root cause

In braintrust-sdk/src/main/java/dev/braintrust/eval/Dataset.java (both fetchFromBraintrust overloads, lines 67–106), the <INPUT, OUTPUT> parameters are never plumbed to a deserializer — no Class, no Jackson TypeReference, no ObjectMapper. The call constructs new DatasetBrainstoreImpl<>(apiClient, datasetId, datasetVersion) and stops.

Inside DatasetBrainstoreImpl.BrainstoreCursor.next() (braintrust-sdk/src/main/java/dev/braintrust/eval/DatasetBrainstoreImpl.java, lines 100–118) the values are taken straight from the OpenAPI model and cast via erasure under @SuppressWarnings("unchecked"):

@SuppressWarnings("unchecked")
public Optional<DatasetCase<INPUT, OUTPUT>> next() {
    ...
    INPUT  input    = (INPUT)  event.getInput();      // Object -> LinkedHashMap at runtime
    OUTPUT expected = (OUTPUT) event.getExpected();
    ...
}

dev.braintrust.openapi.model.DatasetEvent#getInput() / #getExpected() are Object and arrive as LinkedHashMap from the generated OpenAPI client. The compile-time <INPUT, OUTPUT> is purely a hint; the runtime types do not match the declared signature.

Reproduction (sdk 0.3.7 → 0.3.10, unchanged on master @ e110611)

Dataset<MyInput, MyOutput> ds = Dataset.fetchFromBraintrust(client, project, name, null);
var first = ds.openCursor().next().orElseThrow();
System.out.println(first.input().getClass());   // java.util.LinkedHashMap
first.input().prompt();                          // ClassCastException
//   class java.util.LinkedHashMap cannot be cast to class com.example.MyInput

Why we can't fix this in user code

The natural workaround is to wrap the fetched dataset in a thin adapter that deserializes each row (Jackson, hand-rolled, whatever the user's serialization rules are). That works for getting typed cases out — but Eval.run() keys experiment ↔ dataset linking on a concrete-class check:

// braintrust-sdk/src/main/java/dev/braintrust/eval/Eval.java, line 71
if (dataset instanceof DatasetBrainstoreImpl<INPUT, OUTPUT>) {
    datasetVersion = cursor.version();
    datasetId = Optional.of(dataset.id());
}

Any wrapper around the fetched dataset (a user-side mapper, DatasetInMemoryImpl, an anonymous adapter) is not a DatasetBrainstoreImpl, so CreateExperiment.datasetId / datasetVersion are silently never set and the experiment shows as unlinked in the Braintrust UI — even when the wrapped dataset's id() returns a perfectly valid Brainstore UUID.

So today the choice is binary:

• Use fetchFromBraintrust directly → keep the experiment link, get LinkedHashMap and a CCE on first field access.
• Wrap it to deserialize → get typed cases, lose the experiment link.

Neither is acceptable for our use case (golden datasets used for typed agent evals where we also want the experiment to be navigable from the dataset in the UI).

Request

A path that lets a caller get both at once:

1. Typed Java objects out of Dataset cases sourced from fetchFromBraintrust, with deserialization rules under the caller's control (we have @JsonIgnore, custom modules, and promote-from-logs fixups in our dataset rows — so a single SDK-owned ObjectMapper isn't enough; we need a user-supplied row converter).
2. The resulting Dataset still drives experiment ↔ dataset linking in Eval.run() (dataset id + version stamped on the created experiment).

Happy to discuss API shape — we have a draft in a fork and are glad to open a PR once the design is acknowledged, per CONTRIBUTING.md's issue-first guidance.

Environment

• SDK versions affected: v0.3.7 → v0.3.10, also master @ e110611.
• JDK: 25