Add Vercel AI SDK and Mastra tool adapters (chdb/ai-sdk, chdb/mastra) by ShawnChen-Sirius · Pull Request #61 · chdb-io/chdb-node

ShawnChen-Sirius · 2026-06-26T02:37:51Z

What

First-class chDB integrations for the two main TypeScript agent frameworks, over one
shared, framework-agnostic executor core so behavior is identical across them.

Tools — chdb/ai-sdk and chdb/mastra (chdbTools() returns three tools so an
agent can discover then query, not guess SQL):

chdbListTables — tables/views in the session.
chdbDescribeSource — columns/types of a table or any table function
(s3()/file()/postgresql()/…), via ClickHouse DESCRIBE TABLE.
chdbQuery — read-only by default: reads run on a dedicated engine-level
read-only session (SET readonly = 1) bound to the same data path, so a write/DDL
is rejected by the engine, not just a prompt. allowWrite opts out.
Results are JSON (capped at maxRows), engine errors are handed back to the model
rather than thrown. chdbQueryTool() remains for the single-tool case.

ChDBVector (chdb/mastra) — a Mastra vector store backed by chDB-core's HNSW
vector_similarity index, so similarity search is index-accelerated ANN (verified via
EXPLAIN indexes=1), not a brute-force scan. Full MastraVector contract; cosine +
euclidean; idempotent upsert (ReplacingMergeTree + FINAL); flat metadata filters.

ChDBStore (chdb/mastra) — a Mastra storage adapter for the two columnar-friendly
domains: memory (threads/messages) and observability (AI tracing spans, so
trace/span data is analytically queryable with SQL). Records are stored as JSON +
indexed key columns in ReplacingMergeTree tables (upsert by id / (traceId,spanId), read
via FINAL). Other Mastra domains (workflows, scores, …) are point-update/OLTP-shaped and
left to the base — compose another backend for them.

Authored as ESM (.mjs + .d.mts) because both frameworks are ESM-only; exposed via the
chdb/ai-sdk and chdb/mastra subpath exports. ai, @mastra/core, and zod are
optional peer dependencies.

Tests

23 integration tests, all green: the toolset (incl. read-only enforcement and engine
errors), ChDBVector (ANN / filter / upsert-replace / describe / list / delete /
generated ids), and ChDBStore (thread+message round-trips and filters; span/trace
read-write; updateSpan merge; listTraces status; an analytical GROUP BY over spans).
Full v3 suite green (390 passed).

🤖 Generated with Claude Code

Note

Add Vercel AI SDK and Mastra tool adapters under `chdb/ai-sdk` and `chdb/mastra` subpaths

Adds integrations/chdb-tool-core.mjs with a framework-agnostic createChdbExecutor factory that provides query, listTables, and describeSource methods with read-only enforcement and capped row results (errors returned as strings, not thrown).
Adds integrations/ai-sdk.mjs exposing a chdbTools factory and chdbQueryTool helper for the Vercel AI SDK using ai.tool with Zod input schemas.
Adds integrations/mastra.mjs with equivalent Mastra-compatible tools via createTool, plus re-exports of ChDBVector (HNSW vector store) and ChDBStore (thread/message/span persistence).
integrations/chdb-vector.mjs implements a MastraVector-compatible store backed by chDB's HNSW index with upsert, ANN query, metadata filtering, and full index lifecycle management.
integrations/chdb-store.mjs implements Mastra memory (threads/messages) and observability (AI spans) storage using ReplacingMergeTree tables with paginated, filterable reads.
New subpath exports chdb/ai-sdk and chdb/mastra are published via package.json; @mastra/core, ai, and zod are added as optional peer dependencies.

^{Macroscope summarized eb7d0a9.}

ShawnChen-Sirius · 2026-06-26T04:08:49Z

@chibugai, review it

Copilot

Pull request overview

Adds first-class “agent tool” integrations for running ClickHouse SQL via chDB in two ESM-only TypeScript agent ecosystems (Vercel AI SDK and Mastra), sharing a single framework-agnostic executor to keep behavior consistent.

Changes:

Introduces a shared runChdbQuery() core (description + execution semantics) and two thin adapters (chdb/ai-sdk, chdb/mastra).
Exposes the adapters via new subpath exports and adds optional peer deps (ai, @mastra/core, zod).
Adds a Vitest integration test covering the AI SDK adapter end-to-end.

Reviewed changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
test/v3/integrations/ai-sdk.test.ts	Adds an end-to-end test for the Vercel AI SDK tool wrapper.
integrations/chdb-tool-core.mjs	Introduces shared tool description and `runChdbQuery()` executor used by both adapters.
integrations/ai-sdk.mjs	Adds the Vercel AI SDK `tool()` adapter wrapper.
integrations/ai-sdk.d.mts	Adds type declarations for the AI SDK adapter subpath.
integrations/mastra.mjs	Adds the Mastra `createTool()` adapter wrapper.
integrations/mastra.d.mts	Adds type declarations for the Mastra adapter subpath.
package.json	Adds `./ai-sdk` and `./mastra` exports, files whitelist entry, and optional peer/dev deps.
package-lock.json	Updates lockfile for new deps and version metadata.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

chibugai · 2026-06-26T04:14:48Z

Reviewed — OCR deep pass (opus-4-8, 5 source files) plus a manual read of the core and both adapters. No correctness or safety issues found; looks good to me.

What I specifically checked and was happy with:

chdb-tool-core.mjs → runChdbQuery — the input guard rejects non-string/empty SQL before touching the engine, truncation is correct (data.slice(0, maxRows) with the truncated flag reporting the cap was hit), and engine errors are returned as a value ({ error }) rather than thrown — the right call for agent loops, since a thrown tool error is opaque to most of them. The Array.isArray(parsed?.data) guard also keeps a malformed/empty result from blowing up.
Shared core — both adapters are thin wrappers over runChdbQuery, so the description, input contract, and execution semantics stay identical across the Vercel AI SDK and Mastra. Good for consistency.
mastra.mjs — the dual input-shape handling (input.context.sql ?? input.sql) covers both current and older Mastra versions, and if neither is present sql is undefined and falls cleanly through to the empty-string guard. outputSchema matches the actual return shape (elapsed/error correctly .optional()).

One non-blocking design note, just for awareness: the tool deliberately hands the model the full SQL surface, including table functions that reach external hosts and embed credentials (s3(), url(), postgresql(), remoteSecure(), …). That's by design, and the description already says "embed only safe literal values" — but since the SQL is model-generated it's worth keeping in mind for prompt-injection / data-exfiltration exposure. A future readonly / table-function allowlist option could be a nice hardening knob. Not a blocker for this PR.

ShawnChen-Sirius · 2026-06-30T02:28:30Z

@chibugai, review it again

Copilot

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 7 comments.

@ts-ignore

- chdb-vector: import randomUUID from node:crypto (require() is undefined in ESM, so upsert without ids would have thrown); drop the require() helper. - chdb-vector: give __chdb_vector_meta a version column and use ReplacingMergeTree(_updated) so re-creating an index resolves deterministically under FINAL; populate _updated on insert. - chdb-vector: normalize topK with Math.max(1, Math.floor(...)) instead of `| 0` (bitwise wraps/negatives are invalid for the UInt32 bind). - chdb-tool-core: coerce maxRows to a positive integer once so 0/negative/NaN can't make slice()/`truncated` misbehave. - ai-sdk/mastra .d.mts: type the tools' execute() results (ChdbQueryResult / ChdbListTablesResult / ChdbDescribeResult) instead of returning unknown. - tests: use @ts-ignore (not @ts-expect-error) for the .mjs imports so a future resolution change can't fail the suite with an unused-directive error; add a ChDBVector case that upserts without ids (exercises randomUUID). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@ts-ignore

chdb-vector: - import randomUUID from node:crypto (require() is undefined in ESM, so upsert without ids would have thrown); drop the require() helper. - idempotent upsert: the vector table is now ReplacingMergeTree(_v) ORDER BY id and reads use FINAL, so re-inserting an id resolves to the latest row — replaces the async, non-atomic lightweight-DELETE + INSERT (which could leave duplicates or lose data on a crash). The vector_similarity ANN index is still used under FINAL (verified via EXPLAIN indexes=1). - __chdb_vector_meta gains a version column (ReplacingMergeTree(_updated)) so a re-created index resolves deterministically; _updated populated on insert. - normalize topK with Math.max(1, Math.floor(...)) instead of `| 0`. - filterToSql escapes backslashes as well as quotes (ClickHouse interprets backslash escapes — otherwise a metadata value could corrupt the SQL). chdb-store: - esc() neutralizes backslashes and quotes and maps null/undefined to an empty literal (was quote-only — a backslash could break out of the literal). - updateSpan merges only `args.updates` (UpdateSpanArgs = {traceId, spanId, updates}), so extraneous top-level keys never leak into the persisted span. chdb-tool-core: - coerce maxRows to a positive integer once so 0/negative/NaN can't make slice()/`truncated` misbehave. ai-sdk/mastra .d.mts: type the tools' execute() results (ChdbQueryResult / ChdbListTablesResult / ChdbDescribeResult) instead of returning unknown. tests: use @ts-ignore (not @ts-expect-error) for the .mjs imports; add a ChDBVector case that upserts without ids (exercises randomUUID). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ore) A schema-aware agent integration for the two main TypeScript agent frameworks, over one shared, framework-agnostic executor core so behavior is identical. Tools (chdb/ai-sdk, chdb/mastra) — chdbTools() returns three tools so an agent can discover then query, instead of guessing SQL against an unknown schema: - chdbListTables — tables/views in the session - chdbDescribeSource — columns/types of a table or any table function (s3()/file()/postgresql()/…), via ClickHouse DESCRIBE TABLE - chdbQuery — read-only by default: reads run on a dedicated engine-level read-only session (SET readonly = 1) bound to the same data path, so a write/DDL the model emits is rejected by the engine, not just by a prompt. allowWrite opts out. Each returns rows/columns as JSON (capped at maxRows) and hands engine errors back to the model instead of throwing. chdbQueryTool() remains for the single-tool case. ChDBVector (chdb/mastra) — a Mastra vector store backed by chDB-core's HNSW vector index (vector_similarity), so similarity search is index-accelerated ANN, not a brute-force distance scan (verified via EXPLAIN indexes=1). Implements the full MastraVector contract (createIndex/upsert/query/describeIndex/listIndexes/ deleteIndex/updateVector/deleteVector/deleteVectors); metrics cosine + euclidean; true upsert (delete-then-insert); flat metadata equality filters. Authored as ESM (.mjs + .d.mts) since both frameworks are ESM-only. ai, @mastra/core, and zod are optional peer dependencies. Tests: 14 integration tests (toolset incl. read-only enforcement; ChDBVector ANN / filter / upsert-replace / describe / list / delete). Full v3 suite green (382 passed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Makes chDB a Mastra storage backend, scoped to the two domains where ClickHouse's columnar engine is the right tool, exported from chdb/mastra alongside chdbTools and ChDBVector: - memory — agent threads + messages - observability — AI tracing spans, so trace/span data (latencies, errors, span types, per-resource volume) is analytically queryable with plain SQL over the mastra_ai_spans table — the reason to back agent telemetry with ClickHouse. Storage model: each record is a full JSON blob plus indexed key columns (resourceId, threadId, spanType, startedAt, isRoot, …) in a ReplacingMergeTree keyed by id / (traceId, spanId) and versioned by an _updated column. This gives upsert-by-id semantics (the point updates ClickHouse otherwise handles awkwardly) and exact round-tripping of Mastra's nested record shapes; reads use FINAL so a re-saved id returns its latest version immediately. Implements MemoryStorage's required methods (save/get/update/delete thread, save/list/listById/update messages, listThreads with resourceId + metadata filters) and the ObservabilityStorage tracing path (createSpan, batchCreateSpans, updateSpan, getSpan, getRootSpan, getTrace, getSpans, listTraces with status + pagination). Other Mastra domains (workflows, scores, …) are intentionally left to the base — they are point-update/OLTP-shaped; compose another backend for them. Tests: 7 ChDBStore cases (thread/message round-trips + filters; span/trace read-write; updateSpan merge; listTraces status; an analytical GROUP BY over spans). Full v3 suite green (390 passed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@ts-ignore

chdb-vector: - import randomUUID from node:crypto (require() is undefined in ESM, so upsert without ids would have thrown); drop the require() helper. - idempotent upsert: the vector table is now ReplacingMergeTree(_v) ORDER BY id and reads use FINAL, so re-inserting an id resolves to the latest row — replaces the async, non-atomic lightweight-DELETE + INSERT (which could leave duplicates or lose data on a crash). The vector_similarity ANN index is still used under FINAL (verified via EXPLAIN indexes=1). - __chdb_vector_meta gains a version column (ReplacingMergeTree(_updated)) so a re-created index resolves deterministically; _updated populated on insert. - normalize topK with Math.max(1, Math.floor(...)) instead of `| 0`. - filterToSql escapes backslashes as well as quotes (ClickHouse interprets backslash escapes — otherwise a metadata value could corrupt the SQL). chdb-store: - esc() neutralizes backslashes and quotes and maps null/undefined to an empty literal (was quote-only — a backslash could break out of the literal). - updateSpan merges only `args.updates` (UpdateSpanArgs = {traceId, spanId, updates}), so extraneous top-level keys never leak into the persisted span. chdb-tool-core: - coerce maxRows to a positive integer once so 0/negative/NaN can't make slice()/`truncated` misbehave. ai-sdk/mastra .d.mts: type the tools' execute() results (ChdbQueryResult / ChdbListTablesResult / ChdbDescribeResult) instead of returning unknown. tests: use @ts-ignore (not @ts-expect-error) for the .mjs imports; add a ChDBVector case that upserts without ids (exercises randomUUID). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

readonly=1 rejects external table functions (file(), url(), s3()) with Code:164 READONLY, which breaks the tool's primary use case of querying external data. readonly=2 allows SELECT plus external table functions while still rejecting all writes/DDL, and cannot be escaped (lowering readonly is rejected by the engine, and INSERTs smuggled into multi-statement queries are rejected). Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>

- vector_similarity index used GRANULARITY 1, which builds one HNSW per 8192-row granule and prunes nothing, so ANN queries degraded to a full scan. Use a large GRANULARITY so one graph covers the whole part and granules are actually pruned (results are approximate ANN). - Metadata filters interpolated values into SQL and rendered numbers and booleans unquoted, so JSONExtractString(...) = 2024 threw Code 386 and the destructive deleteVectors path carried unbound values. Bind every filter value as a String param and compare against the JSON string form, which fixes numeric/boolean filters and keeps values out of the SQL text. - upsert accepted wrong-length vectors, silently corrupting the HNSW graph. Add CONSTRAINT check_dim CHECK length(vector) = <dim> so the engine rejects a dimension mismatch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

ShawnChen-Sirius · 2026-07-01T00:20:13Z

Thanks for the thorough pass, and for reproducing each of these on a real build. All four are fixed in eb7d0a9.

1. GRANULARITY. You're right, and this is the important one. GRANULARITY 1 builds one HNSW per granule so nothing prunes; it was effectively a brute-force scan wearing an index. Bumped to a large GRANULARITY so a single graph covers the whole part and granules actually get pruned. Updated the header comment to call the results approximate ANN rather than implying exactness.

2. Numeric/boolean filters. Fixed. JSONExtractString always yields the value's string form, so every filter now compares against that string (= '2024', = 'true') instead of a bare literal. Numeric and boolean filters work on both query() and deleteVectors().

3. Param binding. Filter values are now bound as String params through queryBindAsync rather than hand-escaped into the SQL, on both the read path and the destructive deleteVectors. The key stays identifier-validated and inlined.

4. Dimension guard. Added CONSTRAINT check_dim CHECK length(vector) = <dim> to the table, so a wrong-length vector is rejected by the engine (Code 469) instead of silently corrupting the graph.

Added tests for the numeric/boolean filter, delete-by-filter, and the dimension rejection (chdb-vector 9/9, integrations 26/26). Thanks again.

@dfrankland

…w integrations Additive minor release on top of 3.1.0 (2026-06-24). No breaking changes; existing 3.1.0 APIs continue to work unchanged. libchdb bits are unchanged (chdb-core v26.5.1-rc.1 packaged as @chdb/lib-* 26.5.2 — no republish needed, the subpackage-matrix job will idempotent-skip). ## What's new ### Layer 3 fluent query builder SDK (#55, #59) First-class typed fluent query API — Kysely-/Drizzle-style: const db = connect({ session }) const rows = await db .from('users') .where(r => r.age.gt(18)) .select('id', 'name') .execute() - `src/layer3/{builder,execute,compile,dialect,introspection,connect,types,codegen}/` - `chdb-gen-types` CLI — generates row types from a schema (Drizzle / Prisma / raw SQL sources) - `.stream()` terminal with server-side parameter binding - Arrow C Data Interface support: register JS columnar data as `arrowstream()` tables - 12 new test files under `test/v3/layer3/` ### Cross-language agent-tool contract + 2 framework adapters (#61, #66) - `chdb/ai-sdk` — Vercel AI SDK tools (`generateText`/`streamText`) - `chdb/mastra` — Mastra tools + `ChDBVector` (HNSW RAG store) + `ChDBStore` - Both are thin wrappers over `ChDBTool.call()` from `integrations/agents/`, which implements the cross-language contract vendored from `chdb-io/chdb` (`chdb/agents/CONTRACT.md`, chdb-io/chdb#597). - 7 canonical tools: `run_select_query` / `list_databases` / `list_tables` / `describe_table` / `get_sample_data` / `list_functions` / `attach_file` - 5 pillars enforced: engine-level readonly=2 (P1), value-vs-identifier separation (P2), truncation flag (P3), error envelope for model self-correction (P4), resource/source controls (P5) - Behavior verified against the shared `conformance/cases.jsonl` fixture ### `chdb/hypequery` DatabaseAdapter (#63, #65) Lets `@hypequery/clickhouse` query builder run on embedded chDB instead of a remote ClickHouse server. Uses hypequery's own `substituteParameters` (imported, not copied) so rendered SQL stays byte-identical to hypequery's built-in HTTP adapter. ### `chdb/connection` stabilization for @clickhouse/client 1.23.0 - `peerDependency: "@clickhouse/client: >=1.23.0"` (optional) — old client triggers a clear npm warning instead of a runtime TS error - Parity runner switched to track `ClickHouse/clickhouse-js` release tag `client-1.23.0` (per the sync-policy table; #52 shipped the initial hook) - Runner auto-detects upstream Vitest layout (pre-/post-clickhouse-js#931) - Runner re-synced when clickhouse-js reorganized into a monorepo (#62) ### AI-discovery docs (#60) - Top-level `AGENTS.md` + `llms-full.txt` - Per-subpath `AGENTS.md` (`src/layer3/`, `src/connection/`, etc.) - All packaged in `files` so agents crawling the installed tarball find them ## Fixes / infrastructure - Bind `null` / `undefined` query params as SQL `\N` (contributor @dfrankland, #68) — matches `@clickhouse/client` behavior - `close()`-mid-flight engine abort + ResultSet teardown race (#56) - `ChDBVector` HNSW granularity / filter binding / dimension guard (#61) - Release workflow: publish → `unverified` dist-tag → clean-install verify matrix → promote to `latest` (#64). A broken release now never reaches `latest` users. - CI: cache deps + harden npm registry fetch against transient resets ## User-facing installation matrix ``` npm install chdb # base + Layer 3 SDK npm install chdb @clickhouse/client # + chdb/connection npm install chdb @hypequery/clickhouse # + chdb/hypequery npm install chdb ai zod # + chdb/ai-sdk npm install chdb @mastra/core zod # + chdb/mastra ``` Native libchdb: chdb-core v26.5.1-rc.1 (packaged as @chdb/lib-* 26.5.2), unchanged from 3.1.0. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>

ShawnChen-Sirius requested a review from Copilot June 26, 2026 04:08

Copilot started reviewing on behalf of ShawnChen-Sirius June 26, 2026 04:09 View session

Copilot AI reviewed Jun 26, 2026

View reviewed changes

ShawnChen-Sirius force-pushed the feat/agent-tools branch 4 times, most recently from b930240 to 5320f33 Compare June 30, 2026 02:22

ShawnChen-Sirius requested a review from Copilot June 30, 2026 02:28

Copilot started reviewing on behalf of ShawnChen-Sirius June 30, 2026 02:28 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

chibugai reviewed Jun 30, 2026

View reviewed changes

Comment thread integrations/chdb-vector.mjs Outdated

Comment thread integrations/chdb-store.mjs Outdated

Comment thread integrations/chdb-tool-core.mjs

Comment thread integrations/chdb-store.mjs Outdated

Comment thread integrations/chdb-vector.mjs Outdated

ShawnChen-Sirius force-pushed the feat/agent-tools branch from a7fcf99 to fde921f Compare June 30, 2026 02:46

ShawnChen-Sirius commented Jun 30, 2026

View reviewed changes

Comment thread integrations/chdb-tool-core.mjs Outdated

ShawnChen-Sirius and others added 4 commits June 30, 2026 09:47

ShawnChen-Sirius force-pushed the feat/agent-tools branch from 23a7441 to d71344a Compare June 30, 2026 09:48

This comment has been minimized.

Sign in to view

wudidapaopao approved these changes Jul 1, 2026

View reviewed changes

wudidapaopao merged commit d64dda3 into chdb-io:main Jul 1, 2026
18 checks passed

chibugai mentioned this pull request Jul 1, 2026

test CI: async-stress lifecycle-race 120s hang on main (macos-14/20.x) since 09454213 #53

Closed

ShawnChen-Sirius mentioned this pull request Jul 7, 2026

release: chdb 3.2.0 — Layer 3 fluent SDK + agent-tool contract + 3 new integrations #69

Merged

Uh oh!

Uh oh!

Conversation

ShawnChen-Sirius commented Jun 26, 2026 • edited by macroscopeapp Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Tests

Add Vercel AI SDK and Mastra tool adapters under chdb/ai-sdk and chdb/mastra subpaths

Uh oh!

ShawnChen-Sirius commented Jun 26, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chibugai commented Jun 26, 2026

Uh oh!

ShawnChen-Sirius commented Jun 30, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

ShawnChen-Sirius commented Jul 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ShawnChen-Sirius commented Jun 26, 2026 •

edited by macroscopeapp Bot

Loading

Add Vercel AI SDK and Mastra tool adapters under `chdb/ai-sdk` and `chdb/mastra` subpaths