Verify the published package on clean installs before promoting to latest#64
Merged
wudidapaopao merged 4 commits intoJul 1, 2026
Merged
Conversation
Contributor
Author
|
@chibugai, review it |
There was a problem hiding this comment.
Pull request overview
Updates the release workflow to validate the published chdb artifact (as installed from npm) on clean machines before promoting a stable release to the latest dist-tag, preventing regressions like the absolute-rpath failure in #50.
Changes:
- Adds an installed-package Vitest suite that imports
chdbby package name to exercise optionalDependency resolution, native loading, and subpath exports. - Refactors the release workflow into publish → verify (matrix clean installs) → promote (move
latestafter verification), with stable releases published initially underunverified.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
test/pack/installed.test.mjs |
New installed-from-npm test suite to validate native loading, basic queries, Session round-trip, and chdb/connection export. |
.github/workflows/prebuild-publish.yml |
Adds verify/promote jobs and changes stable publish dist-tag to unverified until verified across OS/Node matrix. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
ShawnChen-Sirius
added a commit
to ShawnChen-Sirius/chdb-node
that referenced
this pull request
Jul 1, 2026
…t-connection races Addresses Copilot review comment on PR chdb-io#64 (test/pack/installed.test.mjs:17). The suite mixes the standalone default connection (`version()` / `query()` / `queryAsync()`) with a `Session` that binds its own temp data directory. libchdb's single-active-data-directory-per-process constraint means those two cannot coexist mid-flight — if Vitest ever runs the `it(...)` blocks concurrently (or someone enables `--sequence.shuffle`), the standalone call can race the Session's setup/teardown and either surface `version().libchdb === 'unknown'` or throw on the concurrent `query()`. Fix: replace `describe(...)` with `describe.sequential(...)` so the cases run one-at-a-time regardless of Vitest's default concurrency setting or shuffle flags. Coverage is unchanged; only the ordering guarantee tightens. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
wudidapaopao
previously approved these changes
Jul 1, 2026
ShawnChen-Sirius
added a commit
to ShawnChen-Sirius/chdb-node
that referenced
this pull request
Jul 1, 2026
…t-connection races Addresses Copilot review comment on PR chdb-io#64 (test/pack/installed.test.mjs:17). The suite mixes the standalone default connection (`version()` / `query()` / `queryAsync()`) with a `Session` that binds its own temp data directory. libchdb's single-active-data-directory-per-process constraint means those two cannot coexist mid-flight — if Vitest ever runs the `it(...)` blocks concurrently (or someone enables `--sequence.shuffle`), the standalone call can race the Session's setup/teardown and either surface `version().libchdb === 'unknown'` or throw on the concurrent `query()`. Fix: replace `describe(...)` with `describe.sequential(...)` so the cases run one-at-a-time regardless of Vitest's default concurrency setting or shuffle flags. Coverage is unchanged; only the ordering guarantee tightens. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
b6dd48a to
aa35c71
Compare
…omoting to latest The release workflow built the native addon and ran the suite in the build tree, so it only ever tested the in-place build, never the artifact users install. A binary with the build machine's absolute rpath (chdb-io#50) therefore passed CI and broke on every clean host. Fix: test what users actually get, then gate 'latest' on it. - test/pack/installed.test.mjs imports `chdb` BY NAME (never the source tree), so it exercises the packed files, the @chdb/lib-* binary resolved through optionalDependencies, its rpath at the installed location, and the subpath exports. The first `import 'chdb'` loads the native addon from where npm put it — exactly the chdb-io#50 failure point. - prebuild-publish.yml now publishes a stable release to a holding tag ('unverified', and prereleases stay on 'next'), then a `verify` matrix (every target platform x Node 18/20/22) installs the just-published chdb from npm into a clean dir and runs the installed-package tests, and only then does `promote` move 'latest'. If verify fails on any platform, users on 'latest' never see the broken release. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ndency + runner layout-detect
@clickhouse/client 1.23.0 was published to npm on 2026-06-29 (GitHub
release tag `client-1.23.0` at commit `70ad405`) — the first stable
release carrying the `connection` injection point from PR #879/#880.
This is the first end-to-end path where users can `npm install chdb
@clickhouse/client` and get a working chDB integration without pinning
a pre-release. Follow-ups:
Marked optional in `peerDependenciesMeta` so users who install `chdb`
without using `chdb/connection` are not forced to pull in
`@clickhouse/client`. Users who DO import from `chdb/connection` and
have an older `@clickhouse/client` will get a clear npm warning
instead of a runtime `Property 'connection' does not exist on type ...`
TypeScript error / `undefined` at construction.
Per the sync-policy table in this file ("After next release: latest
released tag"). `lastSyncedSha` bumped to `70ad405` (the release
tag's commit), `lastSyncedAt` to 2026-07-01.
`client-1.23.0` was cut BEFORE PR #931 (which merged 2026-06-30 as
commit `199f9946` and moved the Vitest config/setup into
`packages/client-node/`). So the release tag still has the OLD
repo-root layout (`vitest.node.setup.ts`, root `test:node:integration`
script), while `main` has the NEW per-package layout.
`runner.mjs` now probes for both:
- post-#931 (main, and any release cut after #931):
patch `packages/client-node/vitest.setup.ts`
run `npm --prefix packages/client-node run test:integration`
- pre-#931 (client-1.23.0 release):
patch `vitest.node.setup.ts` at the repo root
run `npm run test:node:integration -- --config vitest.node.config.ts`
Detection is done by probing which setup file exists on disk; cheaper
and more robust than parsing package.json scripts.
rm -rf .chdb-runner
npm run build:ts → clean
npx vitest run test/v3/connection/contract.test.ts → 13/13 ✓
node tests/clickhouse-js/runner.mjs → detected upstream layout: pre-931
patching /vitest.node.setup.ts
Test Files 26 passed / 2 skipped (28)
Tests 170 passed / 41 skipped / 0 failed (211)
The 10-test drop vs. the previous main run (180 → 170) is because the
pre-#931 `test:node:integration` script only invokes the client-node
integration suite (client-common's tests are exercised via a separate
script on that layout), while the post-#931 `packages/client-node`
script pulls the shared common tests in. Coverage of `ChdbConnection`
is identical; every previously-passing case that surfaces the adapter
still passes.
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
…t-connection races Addresses Copilot review comment on PR chdb-io#64 (test/pack/installed.test.mjs:17). The suite mixes the standalone default connection (`version()` / `query()` / `queryAsync()`) with a `Session` that binds its own temp data directory. libchdb's single-active-data-directory-per-process constraint means those two cannot coexist mid-flight — if Vitest ever runs the `it(...)` blocks concurrently (or someone enables `--sequence.shuffle`), the standalone call can race the Session's setup/teardown and either surface `version().libchdb === 'unknown'` or throw on the concurrent `query()`. Fix: replace `describe(...)` with `describe.sequential(...)` so the cases run one-at-a-time regardless of Vitest's default concurrency setting or shuffle flags. Coverage is unchanged; only the ordering guarantee tightens. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
The test/connection/sanitizer workflows installed with a plain `npm install --ignore-scripts` and no dependency cache, so every leg of every push re-fetched the whole tree from the registry with no retry cushion. That is the class behind the ubuntu-latest ECONNRESET seen on this PR: with ~18 legs each re-downloading, a single transient socket reset reddens a job. - actions/setup-node `cache: npm` on every job, so a warm cache serves deps offline and the registry is hit only on a cache miss. - Workflow-level npm fetch-retry/timeout env so a transient reset retries instead of failing the job. Kept `npm install` (not `npm ci`): the committed lockfile is not ci-clean because @mastra/core's nested ai@6.x / @ai-sdk / @opentelemetry subtree is not recorded in it, and the resolution differs across npm majors, so `npm ci` reds every leg. `npm install` reconciles the partial lock; the cache + retry are what remove the flake. Syncing the lockfile for `npm ci` is a separate follow-up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
aa35c71 to
79d3a39
Compare
ShawnChen-Sirius
added a commit
that referenced
this pull request
Jul 7, 2026
…w integrations Additive minor release on top of 3.1.0 (2026-06-24). No breaking changes; existing 3.1.0 APIs continue to work unchanged. libchdb bits are unchanged (chdb-core v26.5.1-rc.1 packaged as @chdb/lib-* 26.5.2 — no republish needed, the subpackage-matrix job will idempotent-skip). ## What's new ### Layer 3 fluent query builder SDK (#55, #59) First-class typed fluent query API — Kysely-/Drizzle-style: const db = connect({ session }) const rows = await db .from('users') .where(r => r.age.gt(18)) .select('id', 'name') .execute() - `src/layer3/{builder,execute,compile,dialect,introspection,connect,types,codegen}/` - `chdb-gen-types` CLI — generates row types from a schema (Drizzle / Prisma / raw SQL sources) - `.stream()` terminal with server-side parameter binding - Arrow C Data Interface support: register JS columnar data as `arrowstream()` tables - 12 new test files under `test/v3/layer3/` ### Cross-language agent-tool contract + 2 framework adapters (#61, #66) - `chdb/ai-sdk` — Vercel AI SDK tools (`generateText`/`streamText`) - `chdb/mastra` — Mastra tools + `ChDBVector` (HNSW RAG store) + `ChDBStore` - Both are thin wrappers over `ChDBTool.call()` from `integrations/agents/`, which implements the cross-language contract vendored from `chdb-io/chdb` (`chdb/agents/CONTRACT.md`, chdb-io/chdb#597). - 7 canonical tools: `run_select_query` / `list_databases` / `list_tables` / `describe_table` / `get_sample_data` / `list_functions` / `attach_file` - 5 pillars enforced: engine-level readonly=2 (P1), value-vs-identifier separation (P2), truncation flag (P3), error envelope for model self-correction (P4), resource/source controls (P5) - Behavior verified against the shared `conformance/cases.jsonl` fixture ### `chdb/hypequery` DatabaseAdapter (#63, #65) Lets `@hypequery/clickhouse` query builder run on embedded chDB instead of a remote ClickHouse server. Uses hypequery's own `substituteParameters` (imported, not copied) so rendered SQL stays byte-identical to hypequery's built-in HTTP adapter. ### `chdb/connection` stabilization for @clickhouse/client 1.23.0 - `peerDependency: "@clickhouse/client: >=1.23.0"` (optional) — old client triggers a clear npm warning instead of a runtime TS error - Parity runner switched to track `ClickHouse/clickhouse-js` release tag `client-1.23.0` (per the sync-policy table; #52 shipped the initial hook) - Runner auto-detects upstream Vitest layout (pre-/post-clickhouse-js#931) - Runner re-synced when clickhouse-js reorganized into a monorepo (#62) ### AI-discovery docs (#60) - Top-level `AGENTS.md` + `llms-full.txt` - Per-subpath `AGENTS.md` (`src/layer3/`, `src/connection/`, etc.) - All packaged in `files` so agents crawling the installed tarball find them ## Fixes / infrastructure - Bind `null` / `undefined` query params as SQL `\N` (contributor @dfrankland, #68) — matches `@clickhouse/client` behavior - `close()`-mid-flight engine abort + ResultSet teardown race (#56) - `ChDBVector` HNSW granularity / filter binding / dimension guard (#61) - Release workflow: publish → `unverified` dist-tag → clean-install verify matrix → promote to `latest` (#64). A broken release now never reaches `latest` users. - CI: cache deps + harden npm registry fetch against transient resets ## User-facing installation matrix ``` npm install chdb # base + Layer 3 SDK npm install chdb @clickhouse/client # + chdb/connection npm install chdb @hypequery/clickhouse # + chdb/hypequery npm install chdb ai zod # + chdb/ai-sdk npm install chdb @mastra/core zod # + chdb/mastra ``` Native libchdb: chdb-core v26.5.1-rc.1 (packaged as @chdb/lib-* 26.5.2), unchanged from 3.1.0. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The release workflow builds the native addon and runs the suite in the build tree, so
it only ever tests the in-place build, never the artifact users install. That is the gap
#50 fell through: the Linux
@chdb/lib-*binary had the build machine's absolute rpath(
/home/runner/work/chdb-node/chdb-node/libchdb.so) baked in, which loaded fine on the CIbuild box and threw
cannot open shared object fileon every clean host.First principles: what a user runs is the packed tarball, with
@chdb/lib-*resolvedthrough optionalDependencies and loaded by rpath at the installed location, on a clean
machine. None of that is exercised by "build then test in place." So: test what users get,
and gate
lateston it.Change
test/pack/installed.test.mjsimportschdbby name (never the source tree), soit exercises the packed files, the platform binary resolved via optionalDependencies, its
rpath at the installed location, and the subpath exports. The first
import 'chdb'loadsthe native addon from where npm put it — the exact why this path /home/runner/work/chdb-node/chdb-node/libchdb.so #50 failure point. It then runs
version(), a stateless query, an async query, a Session insert/query/stream round-trip,
and a
chdb/connectionsubpath resolution.prebuild-publish.ymlbecomes publish → verify → promote:unverified); prereleases stay onnext;verifymatrix (every target platform x Node 18/20/22) installs the just-publishedchdbfrom npm into a clean dir and runs the installed-package tests;promotemoveslatestonly after verify is green on every leg. If verify fails on anyplatform, users on
latestnever see the broken release.Validation
Locally:
npm pack-> cleannpm install ./chdb-*.tgz(real@chdb/lib-*pulled from npm)-> the installed-package tests pass 5/5. The same flow would have failed at
import 'chdb'on the #50 build.
Notes
test/v3, source correctness) still runs in-tree on PRs; the newinstalled-package suite is the release-time artifact gate. Separate concerns.
@chdb/lib-*subpackages publish before verify, but nobody installs them directly — theyare pulled transitively only by a
chdbversion, which is gated by thelatestpromotion, so a bad binary can't reach users via
latest.Fixes the class of bug behind #50.
Additional change: sync
@clickhouse/client1.23.0 release@clickhouse/client 1.23.0 was published to npm on 2026-06-29 (GitHub release tag
client-1.23.0at commit70ad405) — the first stable release carrying theconnectioninjection point from clickhouse-js#879/#880. This is the first end-to-end path where users cannpm install chdb @clickhouse/clientand get a working chDB integration without pinning a pre-release. Three follow-ups landed in the second commit of this PR:peerDependencies['@clickhouse/client']: ">=1.23.0"(optional). Marked optional inpeerDependenciesMetaso users installing chdb withoutchdb/connectionaren't forced to pull it in. Users who DO import fromchdb/connectionwith an older@clickhouse/clientnow get a clear npm warning instead of a runtime TS error / undefined field surprise.skip_list.json#syncedAgainst.ref:main→client-1.23.0. Follows the sync-policy table's third row ("After next release: latest released tag").lastSyncedShabumped to70ad405,lastSyncedAtto 2026-07-01.runner.mjsauto-detects the upstream Vitest layout.client-1.23.0was cut before clickhouse-js#931 (which merged 2026-06-30 and moved the config/setup intopackages/client-node/). The release tag still has the OLD repo-root layout (vitest.node.setup.tsat root,test:node:integrationscript at root); main has the NEW per-package layout.runner.mjsnow probes for both setup-file locations and picks the right npm-script incantation for each.Verification (clean env):
The 10-test drop vs. the previous main run (180 → 170) is because the pre-#931
test:node:integrationscript only invokes the client-node integration suite; the post-#931packages/client-nodescript pulls the shared common tests in as well.ChdbConnectioncoverage is identical across both — every previously-passing case that surfaces the adapter still passes.🤖 Generated with Claude Code
Note
Verify the published npm package on clean installs before promoting to latest
unverifieddist-tag instead oflatest; a newverifyjob in prebuild-publish.yml installs the package into a clean temp dir across 4 OSes and Node 18/20/22, runs the new installed.test.mjs suite, and retries up to 6 times for registry propagation.promotejob moves thelatestdist-tag to the verified version only after all matrix checks pass; prereleases continue to publish directly undernext.cache: npmare added to all CI workflows to reduce flakiness from transient registry errors.detectLayout()function to handle both pre- and post-#931 upstream repo layouts automatically.latest; consumers relying onnpm install chdbwill get the version only after cross-platform verification completes.Macroscope summarized 79d3a39.