Skip to content

feat: Add bun test runner integration#1486

Closed
Abhijeet Prasad (AbhiPrasad) wants to merge 2 commits intomainfrom
abhi-bun-test-runner-integration
Closed

feat: Add bun test runner integration#1486
Abhijeet Prasad (AbhiPrasad) wants to merge 2 commits intomainfrom
abhi-bun-test-runner-integration

Conversation

@AbhiPrasad
Copy link
Copy Markdown
Member

@AbhiPrasad Abhijeet Prasad (AbhiPrasad) commented Mar 3, 2026

resolves #1479

AI Summary

Adds initBunTestSuite — a Braintrust experiment integration for Bun's built-in test runner. Each suite.test() call registers a real bun:test test that logs inputs, outputs, and scorer results as experiment spans.

Usage

import { test, describe, afterAll } from "bun:test";
import { initBunTestSuite } from "braintrust";

describe("Translation Evaluation", () => {
  const suite = initBunTestSuite({
    projectName: "my-project",
    afterAll, // auto-flush when suite finishes
    test,     // bun's test function
  });

  suite.test("translates hello", {
    input: { text: "hello", lang: "es" },
    expected: "hola",
    scorers: [({ output, expected }) => ({
      name: "exact_match",
      score: output === expected ? 1 : 0,
    })],
  }, async ({ input }) => {
    return await translate(input.text, input.lang);
  });

  // untracked tests work alongside suite tests
  test("sanity check", () => expect(1 + 1).toBe(2));
});

Data expansion uses loops, and all bun:test modifiers are supported:

for (const [i, record] of cases.entries()) {
  suite.test(`case [${i}]`, { ...record, scorers }, async ({ input }) => translate(input));
}

suite.test.skip("wip", config, fn);
suite.test.failing("expected error", config, fn);
suite.test.skipIf(!process.env.CI)("CI only", config, fn);
// also: .only, .todo, .concurrent, .serial, .if, .todoIf

How it differs from wrapVitest and initNodeTestSuite

All three integrations share the same core (EvalConfig, ScorerFunction, runTracedEval) but adapt to each runner's API:

wrapVitest initNodeTestSuite initBunTestSuite
Registration bt.test(name, config, fn) test(name, suite.eval(config, fn)) suite.test(name, config, fn)
Modifiers skip, only, concurrent, todo Inherited from node:test skip, only, todo, failing, concurrent, serial, if, skipIf, todoIf
Extra APIs logOutputs, logFeedback, expect wrapper

initBunTestSuite uses the suite.test(name, config, fn) shape because bun callbacks don't receive a context with .name — the test name must be captured at registration. This also makes modifier wrapping natural (suite.test.skip(...), suite.test.if(cond)(...)).

The test config parameter is generic (BunTestSuiteConfig<TTest>) so bun's actual Test<[]> type is forwarded without the SDK depending on bun's types. A runtime validateTestFunction() catches misconfiguration early. Modifiers like .only are bound lazily because bun's CI mode throws when test.only is even property-read.

Future extensions

  • test.each() support for more ergonomic parameterized tests
  • Custom bun reporter that captures results automatically without suite.test() wrappers
  • logOutputs / logFeedback helpers for parity with the vitest integration
  • Shared createTracedEvalFn to reduce duplication across runner integrations

@AbhiPrasad Abhijeet Prasad (AbhiPrasad) deleted the abhi-bun-test-runner-integration branch March 4, 2026 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bun test runner integration

1 participant