Skip to content

Bind null query params as SQL NULL (\N), matching @clickhouse/client#68

Merged
ShawnChen-Sirius merged 2 commits into
chdb-io:mainfrom
dfrankland:fix/null-param-binding
Jul 7, 2026
Merged

Bind null query params as SQL NULL (\N), matching @clickhouse/client#68
ShawnChen-Sirius merged 2 commits into
chdb-io:mainfrom
dfrankland:fix/null-param-binding

Conversation

@dfrankland

@dfrankland dfrankland commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

What

formatParamValue (the {name:Type} server-side param serializer) now emits the
TSV null marker \N for a top-level null/undefined, instead of throwing
ChdbBindError. This makes null params bind as SQL NULL for Nullable
placeholders — the same behavior as @clickhouse/client-common's
formatQueryParams.

Nested nulls (inside an Array/Map param) were already rendered as the NULL
keyword by serializeValue, which also matches the reference client, so the only
change is the top-level branch.

Why

chdb-node is a byte-compatible @clickhouse/client façade (#49, #52). The
reference client serializes null as \N (or the NULL keyword when nested);
the server accepts that for Nullable columns. Throwing on null was a
divergence, not an engine limitation.

Fixes #67.

Behavior change

  • queryBind('SELECT {v:Nullable(String)}', { v: null }) → binds SQL NULL
    (was: ChdbBindError).
  • null against a non-Nullable placeholder is now rejected by the engine
    (a CHDB_* query error), exactly as the HTTP client + server behave — instead
    of a client-side ChdbBindError.
  • The literal string "NULL" is still bound as the string "NULL", not coerced
    to null (only \N is the marker).

This is the intended compatibility semantics, but it does change the error type
for the non-Nullable case (bind-time → query-time), so calling it out.

Verification

\N binding confirmed end-to-end against the native chdb_query_with_params
path:

param placeholder result
\N Nullable(String) null (IS NULL=1)
\N Nullable(Int64) null (IS NULL=1)
[1,NULL,3] Array(Nullable(Int64)) [1, null, 3]
"NULL" Nullable(String) "NULL" (not coerced)
\N Int64 engine error

The diff

 export function formatParamValue(value: unknown): string {
-  if (value === null || value === undefined) {
-    throw new ChdbBindError(
-      'null/undefined parameter values are not supported; omit the parameter or use a typed NULL in SQL',
-    )
-  }
+  if (value === null || value === undefined) return '\\N'
   if (typeof value === 'string') return tsvEscape(value)
   ...

Parity suite

This removes the two NULL-binding entries from tests/clickhouse-js/skip_list.json
and they now pass. Verified with the parity runner against @clickhouse/client
1.23.0 (ChdbConnection injected, :memory:):

✓ … select with query binding > NULL parameter binding > should work with nulls
✓ … select with query binding > NULL parameter binding > should with an explicit undefined
  Test Files  1 passed (1)
       Tests  20 passed | 9 skipped (0 failed)

(The remaining 9 skips in that spec are the unrelated Tuple/Map/nested-boolean
compound-type divergences — see Scope below.)

Tests

  • test/v3/serialize.test.ts: formatParamValue unit block — null/undefined
    \N, nested null → NULL keyword, TSV escaping, and non-finite/unsafe/
    unserializable values still throw.
  • test/v3/querybind.test.ts: split the old "rejects null" case into
    (a) a Nullable placeholder binds null as SQL NULL end-to-end, and
    (b) the engine rejects null against a non-Nullable placeholder.

All affected test/v3 suites pass (serialize.test.ts + querybind.test.ts: 26/26).

Scope / non-goals

This PR only covers a top-level scalar null/undefined. It does NOT touch
the adjacent compound-type parameter-serialization divergences still tracked in
skip_list.json, which have a different root cause (the engine parser rejecting
chdb-node's composite param form):

  • Map(K, Nullable(V)) — nested null already serialized as the NULL keyword
    before this PR; the Map param form itself is what the engine rejects.
  • Tuple params — chdb-node emits {'field':value}, the engine wants positional
    (v1, v2, …) (also affects Array(Tuple), Map(K, Tuple), and booleans
    nested inside tuples).

Explicitly out of scope as intentional design (see
docs/design/layer1-native-binding.md §3.3): rejecting NaN/Infinity,
unsafe integers, and invalid Dates is a deliberate "never silently lose
precision" choice, not a divergence to fix.

🤖 Generated with Claude Code

Note

Bind null/undefined query params as SQL NULL (\N) matching @clickhouse/client

  • formatParamValue in serialize.ts now returns '\N' for top-level null/undefined instead of throwing ChdbBindError, matching the behavior of the official @clickhouse/client.
  • The engine treats \N as SQL NULL for Nullable placeholders and rejects it for non-Nullable ones, surfacing an engine-level error rather than a client-side error.
  • Two null-binding tests are removed from the clickhouse-js skip list in skip_list.json as they now pass.
  • Behavioral Change: code that previously caught ChdbBindError on null params will no longer receive that error; binding null to a non-Nullable column now yields a CHDB_* engine error instead.

Macroscope summarized caf4f69.

@CLAassistant

CLAassistant commented Jul 2, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

formatParamValue rejected a top-level null/undefined with ChdbBindError.
The reference client (@clickhouse/client-common formatQueryParams) instead
emits the TSV null marker \N at the top level (and the NULL keyword when
nested), which the server binds as SQL NULL for Nullable columns. chdb-node
is a byte-compat @clickhouse/client facade (chdb-io#49, chdb-io#52), so throwing was a
divergence, not an engine limitation: the native chdb_query_with_params
path already accepts \N as NULL (verified for Nullable(String)/Int64).

Emit \N for a top-level null/undefined; nested nulls already render as the
NULL keyword via serializeValue. A null bound to a non-Nullable placeholder
is now rejected by the engine (a CHDB_* query error) rather than a
client-side ChdbBindError, matching the HTTP client + server.

Removes the two NULL-binding entries from the clickhouse-js parity skip
list; both now pass against @clickhouse/client 1.23.0.

Fixes chdb-io#67

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dfrankland dfrankland force-pushed the fix/null-param-binding branch from 4722e96 to b37dfc7 Compare July 2, 2026 18:58
@ShawnChen-Sirius

Copy link
Copy Markdown
Contributor

@dfrankland thanks for this PR. Pls fix the ci unexception, then we will merge it~

The null-param compat fix maps top-level null/undefined to the TSV null
marker \N (matching @clickhouse/client). Binding NULL to a non-Nullable
{x:UInt32} placeholder is rejected by the engine (CHDB_QUERY), not at
serialization (CHDB_BIND). Drop undefined from the unserializable-values
case and add a dedicated null/undefined -> \N assertion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dfrankland

Copy link
Copy Markdown
Contributor Author

@ShawnChen-Sirius thanks for reviewing. Just pushed a fix

@ShawnChen-Sirius ShawnChen-Sirius merged commit dbdaf64 into chdb-io:main Jul 7, 2026
17 checks passed
ShawnChen-Sirius added a commit that referenced this pull request Jul 7, 2026
…w integrations

Additive minor release on top of 3.1.0 (2026-06-24). No breaking changes;
existing 3.1.0 APIs continue to work unchanged. libchdb bits are unchanged
(chdb-core v26.5.1-rc.1 packaged as @chdb/lib-* 26.5.2 — no republish
needed, the subpackage-matrix job will idempotent-skip).

## What's new

### Layer 3 fluent query builder SDK (#55, #59)

First-class typed fluent query API — Kysely-/Drizzle-style:

    const db = connect({ session })
    const rows = await db
      .from('users')
      .where(r => r.age.gt(18))
      .select('id', 'name')
      .execute()

- `src/layer3/{builder,execute,compile,dialect,introspection,connect,types,codegen}/`
- `chdb-gen-types` CLI — generates row types from a schema (Drizzle /
  Prisma / raw SQL sources)
- `.stream()` terminal with server-side parameter binding
- Arrow C Data Interface support: register JS columnar data as
  `arrowstream()` tables
- 12 new test files under `test/v3/layer3/`

### Cross-language agent-tool contract + 2 framework adapters (#61, #66)

- `chdb/ai-sdk` — Vercel AI SDK tools (`generateText`/`streamText`)
- `chdb/mastra` — Mastra tools + `ChDBVector` (HNSW RAG store) + `ChDBStore`
- Both are thin wrappers over `ChDBTool.call()` from `integrations/agents/`,
  which implements the cross-language contract vendored from `chdb-io/chdb`
  (`chdb/agents/CONTRACT.md`, chdb-io/chdb#597).
- 7 canonical tools: `run_select_query` / `list_databases` / `list_tables` /
  `describe_table` / `get_sample_data` / `list_functions` / `attach_file`
- 5 pillars enforced: engine-level readonly=2 (P1), value-vs-identifier
  separation (P2), truncation flag (P3), error envelope for model
  self-correction (P4), resource/source controls (P5)
- Behavior verified against the shared `conformance/cases.jsonl` fixture

### `chdb/hypequery` DatabaseAdapter (#63, #65)

Lets `@hypequery/clickhouse` query builder run on embedded chDB instead
of a remote ClickHouse server. Uses hypequery's own `substituteParameters`
(imported, not copied) so rendered SQL stays byte-identical to hypequery's
built-in HTTP adapter.

### `chdb/connection` stabilization for @clickhouse/client 1.23.0

- `peerDependency: "@clickhouse/client: >=1.23.0"` (optional) — old client
  triggers a clear npm warning instead of a runtime TS error
- Parity runner switched to track `ClickHouse/clickhouse-js` release tag
  `client-1.23.0` (per the sync-policy table; #52 shipped the initial hook)
- Runner auto-detects upstream Vitest layout (pre-/post-clickhouse-js#931)
- Runner re-synced when clickhouse-js reorganized into a monorepo (#62)

### AI-discovery docs (#60)

- Top-level `AGENTS.md` + `llms-full.txt`
- Per-subpath `AGENTS.md` (`src/layer3/`, `src/connection/`, etc.)
- All packaged in `files` so agents crawling the installed tarball find them

## Fixes / infrastructure

- Bind `null` / `undefined` query params as SQL `\N` (contributor
  @dfrankland, #68) — matches `@clickhouse/client` behavior
- `close()`-mid-flight engine abort + ResultSet teardown race (#56)
- `ChDBVector` HNSW granularity / filter binding / dimension guard (#61)
- Release workflow: publish → `unverified` dist-tag → clean-install verify
  matrix → promote to `latest` (#64). A broken release now never reaches
  `latest` users.
- CI: cache deps + harden npm registry fetch against transient resets

## User-facing installation matrix

```
npm install chdb                                    # base + Layer 3 SDK
npm install chdb @clickhouse/client                 # + chdb/connection
npm install chdb @hypequery/clickhouse              # + chdb/hypequery
npm install chdb ai zod                             # + chdb/ai-sdk
npm install chdb @mastra/core zod                   # + chdb/mastra
```

Native libchdb: chdb-core v26.5.1-rc.1 (packaged as @chdb/lib-* 26.5.2),
unchanged from 3.1.0.

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

queryBind rejects null/undefined params instead of binding SQL NULL (diverges from @clickhouse/client)

3 participants