Skip to content

Add chdb/hypequery — run hypequery on embedded chDB#63

Merged
wudidapaopao merged 2 commits into
chdb-io:mainfrom
ShawnChen-Sirius:feat/hypequery-adapter
Jul 1, 2026
Merged

Add chdb/hypequery — run hypequery on embedded chDB#63
wudidapaopao merged 2 commits into
chdb-io:mainfrom
ShawnChen-Sirius:feat/hypequery-adapter

Conversation

@ShawnChen-Sirius

@ShawnChen-Sirius ShawnChen-Sirius commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Lets @hypequery/clickhouse run its type-safe
query builder against in-process chDB instead of a remote ClickHouse server, by passing a
chDB-backed adapter to hypequery's existing createQueryBuilder({ adapter }) hook.

Discussed in hypequery/hypequery#237 (maintainer is on board).

import { createQueryBuilder } from '@hypequery/clickhouse'
import { Session } from 'chdb'
import { chdbAdapter } from 'chdb/hypequery'

const db = createQueryBuilder({ adapter: chdbAdapter({ session: new Session('./db') }) })

How

hypequery renders final SQL itself (?-positional params), so the adapter just runs the
SQL on chDB and returns JSONEachRow rows. It implements the DatabaseAdapter contract:

  • query — rows
  • stream — chDB's chunked cursor wrapped as a ReadableStream<T[]>
  • render — mirrors hypequery's substituteParameters/escapeValue so generated SQL
    matches the built-in HTTP adapter exactly

No @clickhouse/client and no ClickHouse server involved. The adapter is duck-typed and
imports nothing from @hypequery/clickhouse at runtime, so it stays an optional, types-only
peer dependency. Shipped as the chdb/hypequery subpath.

It is self-contained against the currently published @hypequery/clickhouse (it does not
depend on any hypequery change). A small follow-up can switch the copied render helpers to
imports once hypequery exports them (hypequery/hypequery PR for that is separate).

Tests

A real @hypequery/clickhouse builder chain runs against embedded chDB: filter + aggregate

  • group + order, value escaping, rawQuery passthrough, and streaming.

🤖 Generated with Claude Code

Note

Add chdbAdapter to integrate hypequery with embedded chDB

  • Adds a new chdb/hypequery subpath export with an ESM entry (hypequery.mjs) and TypeScript declarations (hypequery.d.mts).
  • chdbAdapter(opts?) returns a hypequery-compatible DatabaseAdapter with query(), stream(), and render() methods backed by chDB's queryAsync and queryStream.
  • Includes substituteParameters and escapeValue helpers that inject typed JS values as SQL-safe literals and throw on placeholder/param count mismatch.
  • stream() wraps chDB's async iterator into a Web ReadableStream<T[]> but requires a bound session — throws at call time if none is provided.
  • Declares @hypequery/clickhouse >=2 as an optional peer dependency.

Macroscope summarized ebb0eda.

@ShawnChen-Sirius

Copy link
Copy Markdown
Contributor Author

@chibugai, review it

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new optional integration subpath (chdb/hypequery) so @hypequery/clickhouse can execute its rendered SQL against embedded chDB (no ClickHouse server / no @clickhouse/client required at runtime).

Changes:

  • Adds integrations/hypequery.mjs implementing a hypequery-shaped DatabaseAdapter (query/stream/render) backed by chDB.
  • Adds integrations/hypequery.d.mts types and exposes the ./hypequery subpath export.
  • Adds vitest coverage exercising builder execution, escaping behavior, rawQuery passthrough, and streaming.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
test/v3/integrations/hypequery.test.ts Adds integration tests validating hypequery builder + streaming against embedded chDB.
package.json Exposes chdb/hypequery, includes integration files in published package, and adds optional peer/dev dependency for hypequery.
package-lock.json Lockfile updates to include new dev dependency chain (notably @hypequery/clickhouse and its transitive deps).
integrations/hypequery.mjs Implements the chDB-backed hypequery adapter, including SQL placeholder substitution and stream wrapping.
integrations/hypequery.d.mts Provides a structural DatabaseAdapter type surface for consumers importing chdb/hypequery.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread integrations/hypequery.mjs
Comment thread integrations/hypequery.d.mts
Comment thread test/v3/integrations/hypequery.test.ts Outdated
ShawnChen-Sirius added a commit to ShawnChen-Sirius/chdb-node that referenced this pull request Jul 1, 2026
…dableStream import, session close

Three Copilot review-comment fixes:

### integrations/hypequery.mjs — `escapeValue` robustness

Previous version wrapped `JSON.stringify(value)` in single quotes with no
escaping, and had two silent failure modes:

  - Single quotes inside the JSON (e.g. `{name: "O'Reilly"}`) produced
    malformed SQL — the ClickHouse parser closes the string literal early
    at the embedded `'`.
  - `JSON.stringify(undefined)` returns `undefined` (not a string), so the
    template literal produced the literal SQL string `'undefined'`.
  - `JSON.stringify(BigInt(...))` throws `TypeError: Do not know how to
    serialize a BigInt`.

Fix:

  - `null` / `undefined` → SQL `NULL` (matches ClickHouse convention).
  - `bigint` → unquoted numeric literal (`n.toString()`).
  - Object/array path applies the same `\\` + `'` escaping the string
    branch uses, so JSON containing `'` becomes SQL-safe via ClickHouse's
    `''` doubling.

Verified against 9 targeted cases (null, undef, bool, number, bigint,
string with `'`, string with `\`, object with `'`, Date) — all pass.

### integrations/hypequery.d.mts — `ReadableStream` type source

`.d.mts` referenced `ReadableStream` without importing it. The repo's TS
config uses `lib: ["ES2022"]` (no DOM), so consumers with a similar
Node-only setup get `Cannot find name 'ReadableStream'` when consuming
`chdb/hypequery`'s types. Fix: import from `node:stream/web` (available in
Node 18+) so the type surface works without DOM libs.

### test/v3/integrations/hypequery.test.ts — session `close()` in afterEach

Test created a new `Session` per case in `beforeEach` but never closed it
explicitly. The global safety net in `test/v3/setup.ts` catches leaks but
localizing lifetime makes failures easier to reason about (and matches
the convention the other v3 tests follow). Added `afterEach(() => s?.close())`.

### Verification

  npm run build:ts                                                  → clean
  npx vitest run test/v3/integrations/hypequery.test.ts             → 4/4 ✓

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
wudidapaopao
wudidapaopao previously approved these changes Jul 1, 2026
ShawnChen-Sirius and others added 2 commits July 1, 2026 20:36
Lets @hypequery/clickhouse run its type-safe query builder against in-process chDB
instead of a remote ClickHouse server, by passing a chDB-backed adapter to
hypequery's existing createQueryBuilder({ adapter }) hook:

  import { createQueryBuilder } from '@hypequery/clickhouse'
  import { Session } from 'chdb'
  import { chdbAdapter } from 'chdb/hypequery'
  const db = createQueryBuilder({ adapter: chdbAdapter({ session: new Session('./db') }) })

hypequery renders final SQL itself (?-positional params), so the adapter just runs
the SQL on chDB and returns JSONEachRow rows. It implements the DatabaseAdapter
contract: query (rows), stream (chDB's chunked cursor -> ReadableStream<T[]>), and
render (mirrors hypequery's substituteParameters/escapeValue so generated SQL matches
the HTTP adapter exactly). No @clickhouse/client and no ClickHouse server involved.

The adapter is duck-typed and imports nothing from @hypequery/clickhouse at runtime;
it is an optional, types-only peer dependency. Shipped as the chdb/hypequery subpath.

Tests: a builder query (filter + aggregate + group + order), value escaping, rawQuery
passthrough, and streaming, all run against embedded chDB through real
@hypequery/clickhouse.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dableStream import, session close

Three Copilot review-comment fixes:

### integrations/hypequery.mjs — `escapeValue` robustness

Previous version wrapped `JSON.stringify(value)` in single quotes with no
escaping, and had two silent failure modes:

  - Single quotes inside the JSON (e.g. `{name: "O'Reilly"}`) produced
    malformed SQL — the ClickHouse parser closes the string literal early
    at the embedded `'`.
  - `JSON.stringify(undefined)` returns `undefined` (not a string), so the
    template literal produced the literal SQL string `'undefined'`.
  - `JSON.stringify(BigInt(...))` throws `TypeError: Do not know how to
    serialize a BigInt`.

Fix:

  - `null` / `undefined` → SQL `NULL` (matches ClickHouse convention).
  - `bigint` → unquoted numeric literal (`n.toString()`).
  - Object/array path applies the same `\\` + `'` escaping the string
    branch uses, so JSON containing `'` becomes SQL-safe via ClickHouse's
    `''` doubling.

Verified against 9 targeted cases (null, undef, bool, number, bigint,
string with `'`, string with `\`, object with `'`, Date) — all pass.

### integrations/hypequery.d.mts — `ReadableStream` type source

`.d.mts` referenced `ReadableStream` without importing it. The repo's TS
config uses `lib: ["ES2022"]` (no DOM), so consumers with a similar
Node-only setup get `Cannot find name 'ReadableStream'` when consuming
`chdb/hypequery`'s types. Fix: import from `node:stream/web` (available in
Node 18+) so the type surface works without DOM libs.

### test/v3/integrations/hypequery.test.ts — session `close()` in afterEach

Test created a new `Session` per case in `beforeEach` but never closed it
explicitly. The global safety net in `test/v3/setup.ts` catches leaks but
localizing lifetime makes failures easier to reason about (and matches
the convention the other v3 tests follow). Added `afterEach(() => s?.close())`.

### Verification

  npm run build:ts                                                  → clean
  npx vitest run test/v3/integrations/hypequery.test.ts             → 4/4 ✓

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
@ShawnChen-Sirius ShawnChen-Sirius force-pushed the feat/hypequery-adapter branch from f93ecdb to ebb0eda Compare July 1, 2026 08:39
@wudidapaopao wudidapaopao merged commit 27d43f6 into chdb-io:main Jul 1, 2026
31 of 32 checks passed
ShawnChen-Sirius added a commit that referenced this pull request Jul 7, 2026
…w integrations

Additive minor release on top of 3.1.0 (2026-06-24). No breaking changes;
existing 3.1.0 APIs continue to work unchanged. libchdb bits are unchanged
(chdb-core v26.5.1-rc.1 packaged as @chdb/lib-* 26.5.2 — no republish
needed, the subpackage-matrix job will idempotent-skip).

## What's new

### Layer 3 fluent query builder SDK (#55, #59)

First-class typed fluent query API — Kysely-/Drizzle-style:

    const db = connect({ session })
    const rows = await db
      .from('users')
      .where(r => r.age.gt(18))
      .select('id', 'name')
      .execute()

- `src/layer3/{builder,execute,compile,dialect,introspection,connect,types,codegen}/`
- `chdb-gen-types` CLI — generates row types from a schema (Drizzle /
  Prisma / raw SQL sources)
- `.stream()` terminal with server-side parameter binding
- Arrow C Data Interface support: register JS columnar data as
  `arrowstream()` tables
- 12 new test files under `test/v3/layer3/`

### Cross-language agent-tool contract + 2 framework adapters (#61, #66)

- `chdb/ai-sdk` — Vercel AI SDK tools (`generateText`/`streamText`)
- `chdb/mastra` — Mastra tools + `ChDBVector` (HNSW RAG store) + `ChDBStore`
- Both are thin wrappers over `ChDBTool.call()` from `integrations/agents/`,
  which implements the cross-language contract vendored from `chdb-io/chdb`
  (`chdb/agents/CONTRACT.md`, chdb-io/chdb#597).
- 7 canonical tools: `run_select_query` / `list_databases` / `list_tables` /
  `describe_table` / `get_sample_data` / `list_functions` / `attach_file`
- 5 pillars enforced: engine-level readonly=2 (P1), value-vs-identifier
  separation (P2), truncation flag (P3), error envelope for model
  self-correction (P4), resource/source controls (P5)
- Behavior verified against the shared `conformance/cases.jsonl` fixture

### `chdb/hypequery` DatabaseAdapter (#63, #65)

Lets `@hypequery/clickhouse` query builder run on embedded chDB instead
of a remote ClickHouse server. Uses hypequery's own `substituteParameters`
(imported, not copied) so rendered SQL stays byte-identical to hypequery's
built-in HTTP adapter.

### `chdb/connection` stabilization for @clickhouse/client 1.23.0

- `peerDependency: "@clickhouse/client: >=1.23.0"` (optional) — old client
  triggers a clear npm warning instead of a runtime TS error
- Parity runner switched to track `ClickHouse/clickhouse-js` release tag
  `client-1.23.0` (per the sync-policy table; #52 shipped the initial hook)
- Runner auto-detects upstream Vitest layout (pre-/post-clickhouse-js#931)
- Runner re-synced when clickhouse-js reorganized into a monorepo (#62)

### AI-discovery docs (#60)

- Top-level `AGENTS.md` + `llms-full.txt`
- Per-subpath `AGENTS.md` (`src/layer3/`, `src/connection/`, etc.)
- All packaged in `files` so agents crawling the installed tarball find them

## Fixes / infrastructure

- Bind `null` / `undefined` query params as SQL `\N` (contributor
  @dfrankland, #68) — matches `@clickhouse/client` behavior
- `close()`-mid-flight engine abort + ResultSet teardown race (#56)
- `ChDBVector` HNSW granularity / filter binding / dimension guard (#61)
- Release workflow: publish → `unverified` dist-tag → clean-install verify
  matrix → promote to `latest` (#64). A broken release now never reaches
  `latest` users.
- CI: cache deps + harden npm registry fetch against transient resets

## User-facing installation matrix

```
npm install chdb                                    # base + Layer 3 SDK
npm install chdb @clickhouse/client                 # + chdb/connection
npm install chdb @hypequery/clickhouse              # + chdb/hypequery
npm install chdb ai zod                             # + chdb/ai-sdk
npm install chdb @mastra/core zod                   # + chdb/mastra
```

Native libchdb: chdb-core v26.5.1-rc.1 (packaged as @chdb/lib-* 26.5.2),
unchanged from 3.1.0.

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants