Feature request: worker-less mode or test mock for @powersync/node

[mikecfisher]

Hi team, I'm using PowerSync with TanStack DB in a Next.js app and run my tests with Bun.

The problem: @powersync/node uses comlink + worker_threads internally, and Bun's worker_threads implementation has serialization bugs that cause TypeError: Unserializable return value when trying to initialize a PowerSync database.

Bun's worker support is still experimental (tracked in oven-sh/bun#15964) and there's no workaround on their side in the near term.

What would help:

1. A worker-less mode for @powersync/node - even if slower, it would let us run integration tests
2. Or a lightweight test mock that implements AbstractPowerSyncDatabase using synchronous SQLite (like bun:sqlite or better-sqlite3 without workers)

I want to test our TanStack DB collections end-to-end without needing to spin up Vitest separately just for PowerSync tests. Happy to help if you point me in the right direction.

[kobiebotha]

Getting 500s from GH trying to convert this to a discussion 😅

Sorry about the radio silence on this once, busy checking in to see if we can point you in the right direction, or if it's something we'd need to implement ourselves.

[simolus3]

Apologies for the long wait on this. We could add a worker-less mode to @powersync/node, but that would essentially be a duplicate implementation (unless we can find some tricks to use a mock worker where postMessage just moves the object through some channel without serializing it).

Anyway, starting from our latest SDK versions released today, it has become much easier to define your own database adapers. Here's one I wrote specifically for Bun:

import { Database } from "bun:sqlite";
import path from "node:path";

import {
  BaseObserver,
  ColumnType,
  DBAdapterDefaultMixin,
  DBGetUtilsDefaultMixin,
  PowerSyncDatabase,
  Schema,
  Table,
  type ConnectionPool,
  type DBAdapter,
  type DBAdapterListener,
  type DBLockOptions,
  type LockContext,
  type QueryResult,
  type SqlExecutor,
} from "@powersync/node";
import { Mutex } from "async-mutex";
import { getPowerSyncExtensionFilename } from "@powersync/node/worker.js";

function loadPowerSyncSqliteCoreExtension(db: Database) {
  const filename = getPowerSyncExtensionFilename();
  const resolved = Bun.resolveSync("@powersync/node", import.meta.dir);

  const extensionPath = path.join(path.dirname(resolved), filename);
  db.loadExtension(extensionPath, "sqlite3_powersync_init");
}

if (navigator.platform.startsWith("Mac")) {
  // Use homebrew SQLite because the one from macOS can't load extensions.
  Database.setCustomSQLite("/opt/homebrew/opt/sqlite/lib/libsqlite3.dylib");
}

class BunConnectionPool
  extends BaseObserver<DBAdapterListener>
  implements ConnectionPool
{
  #mutex = new Mutex();

  constructor(readonly database: Database) {
    super();
    loadPowerSyncSqliteCoreExtension(database);
  }

  get name(): string {
    return this.database.filename;
  }

  close() {
    this.database.close();
  }

  async readLock<T>(
    fn: (tx: LockContext) => Promise<T>,
    options?: DBLockOptions,
  ): Promise<T> {
    // There's a single connection, so read and write locks are the same thing.
    return this.writeLock(fn, options);
  }

  async writeLock<T>(
    fn: (tx: LockContext) => Promise<T>,
    options?: DBLockOptions,
  ): Promise<T> {
    return await this.#mutex.runExclusive(async () => {
      const context = new BunLockContext(this);
      return await fn(context);
    });
  }

  async refreshSchema() {
    // Not necessary for a single connection
  }

  async initialize() {
    // Currently needed because the Node.JS SDK tries to call this method (it shouldn't).
  }

  collectQueryResult(): { changes: number; lastInsertRowId: number } {
    using stmt = this.database.query(
      "SELECT changes() AS changes, last_insert_rowid() AS lastInsertRowId",
    );
    return stmt.get() as { changes: number; lastInsertRowId: number };
  }
}

export class BunDatabaseAdapter
  extends DBAdapterDefaultMixin(BunConnectionPool)
  implements DBAdapter {}

class BunSqlExecutor implements SqlExecutor {
  constructor(private readonly pool: BunConnectionPool) {}

  async execute(query: string, params: any[] = []): Promise<QueryResult> {
    using stmt = this.pool.database.prepare(query);
    const resultSet = stmt.all(...params);
    const state = this.pool.collectQueryResult();
    return {
      rowsAffected: state.changes,
      insertId: state.lastInsertRowId,
      rows: {
        _array: resultSet,
        length: resultSet.length,
        item(idx) {
          return resultSet[idx];
        },
      },
    };
  }

  async executeRaw(query: string, params: any[] = []): Promise<any[][]> {
    using stmt = this.pool.database.prepare(query);
    return stmt.values(...params);
  }

  async executeBatch(
    query: string,
    params: any[][] = [],
  ): Promise<QueryResult> {
    using stmt = this.pool.database.prepare(query);
    let affectedRows = 0;
    let lastInsertId: number | undefined;
    for (const parameterSet of params) {
      stmt.all(...parameterSet);

      const { changes, lastInsertRowId } = this.pool.collectQueryResult();
      affectedRows += changes;
      lastInsertId = lastInsertRowId;
    }

    return { rowsAffected: affectedRows, insertId: lastInsertId };
  }
}

class BunLockContext
  extends DBGetUtilsDefaultMixin(BunSqlExecutor)
  implements LockContext {}

And then you can use this to open your database:

const db = new PowerSyncDatabase({
  schema: new Schema({
    users: new Table({
      name: { type: ColumnType.TEXT },
    }),
  }),
  database: new BunDatabaseAdapter(new Database(":memory:")),
});

await db.execute("INSERT INTO users (id, name) VALUES (uuid(), ?)", [
  "Test user",
]);

console.log(await db.get("SELECT * FROM users"));

I could potentially see a @powersync/node/synchronous.js entrypoint exporting something like this to make testing easier. But I hope that snippet helps you get started if you're still interested in this.