agents: contract 0.2.0 — generate the tool surface from vendored descriptors

integrations/agents/CONTRACT.md

@@ -4,12 +4,16 @@
 > [`chdb-io/chdb` at `chdb/agents/CONTRACT.md`](https://github.com/chdb-io/chdb/blob/main/chdb/agents/CONTRACT.md).
 > This is a synced mirror so the chdb-node implementation and its conformance
 > runner sit next to the contract they satisfy. Do not edit it here; change it
-> upstream and re-sync. `conformance/cases.jsonl` and `conformance/fixtures/` are
-> vendored the same way.
+> upstream and re-sync. `conformance/cases.jsonl`, `conformance/fixtures/`, and
+> `descriptors.json` are vendored the same way.
 
 > **Status: beta / experimental (introduced in chdb 4.2.0).** The surface may
 > change in a minor release while it stabilizes across bindings; pin a version if
 > you depend on it.
+>
+> **Contract version: `0.2.0`** — exposed as `CONTRACT_VERSION` and via
+> `capabilities()` in every binding. Downstream consumers probe capabilities
+> instead of guessing from a package version (see *Versioning & capabilities*).
 
 One behavior, many bindings. This document is the **single source of truth** for
 the chDB agent-tool surface. `chdb.agents.ChDBTool` (Python) is the reference
@@ -37,6 +41,53 @@ Bindings expose these as native tools (Python methods on `ChDBTool`; TS
 `chdbTools()`; MCP tools in mcp-clickhouse) but the **capability set, argument
 meaning, error classification, and safety guarantees are identical**.
 
+## descriptors.json — the model-visible surface (single source)
+
+`descriptors.json` (next to this file; vendored byte-identical in every
+binding) is the single source of truth for the **model-visible** surface: tool
+names, description text, and argument schemas. Everything the model reads is
+generated from it:
+
+- `tool_specs(dialect)` / `toolSpecs(dialect)` render it per runtime family —
+  `anthropic` (`input_schema`), `openai` (`{type: "function", function}`),
+  `mcp` (`inputSchema`).
+- Framework adapters derive their native schema declarations from it (the TS
+  Vercel AI SDK / Mastra adapters generate zod schemas mechanically; schema-
+  declaring frameworks like CrewAI / smolagents generate their arg specs the
+  same way). Hand-copying a description into an adapter re-creates the drift
+  this file exists to kill.
+- Changing `descriptors.json` **is a contract change**: bump `contract_version`
+  (in the file, the code constant, and the conformance fixture header) and sync
+  every vendored copy.
+
+## Versioning & capabilities
+
+- `CONTRACT_VERSION` (semver string) names the contract revision a binding
+  implements. It changes when `descriptors.json`, `conformance/cases.jsonl`, or
+  normative text here changes. It is intentionally **not** the package version.
+- `capabilities()` returns `{contract_version, tools, features}`. `features`
+  flags the capability-gated parts of the contract, so a consumer asks *"does
+  this binding do X?"* instead of maintaining a version matrix:
+  `dataframe_query` (Python-only, co-located engine), `attachments`,
+  `file_allowlist`, `max_execution_time`, `async`, `streaming` (reserved,
+  currently `false` everywhere).
+- Conformance cases carrying `"requires": "<feature>"` run only on bindings
+  whose `capabilities().features[<feature>]` is true; all other cases are core
+  and every binding must pass them.
+
+## Async
+
+Engine calls are synchronous in-process; how a binding exposes async is part of
+the contract surface:
+
+- **Python** — `aquery()` / `acall()` run the sync call in a worker thread
+  (`asyncio.to_thread`), documented as such, never faked as native async. This
+  is what async-first frameworks (AutoGen, Pydantic AI) bind to, instead of
+  each re-implementing a thread-pool wrapper.
+- **TypeScript** — `query()` / `call()` are natively Promise-based.
+- The sync path (Python) remains the canonical one the conformance runner
+  drives; async forms must return identical results.
+
 **Language-optional capability (not in the cross-language conformance):**
 `dataframe_query` — query in-process DataFrames via the `Python()` table
 function. Meaningful only in languages/deployments where the agent runtime and
@@ -67,14 +118,29 @@ to TS, which has no in-process pandas). Bindings may omit it.
 - Every result is capped by `max_rows` (and a `max_bytes` guard). When the engine
   produced more than the cap, the result sets `truncated = true`. Truncation is
   **never silent**.
+- `max_bytes` counts **UTF-8 bytes of each row's compact JSON encoding** (no
+  spaces, non-ASCII kept raw). This is normative because it is model-visible:
+  counting UTF-16 units or ASCII-escaped characters instead moves the truncation
+  point by up to ~3× on non-ASCII data, splitting the corpus between bindings.
+- A non-numeric per-call cap (`max_rows`, `limit`) is a typed
+  `INVALID_ARGUMENT` error, never a silently disabled cap. On the model-facing
+  `call()` path a JSON `null` for an optional argument means "omitted".
 
 ### P4 — Errors reach the model
 - The **tool-dispatch** path (`call()` / the tool executor) returns a structured
   envelope `{ ok: false, error: { code, type, message } }` instead of throwing,
   so an agent can read the engine message and self-correct.
+- That includes caller mistakes on the dispatch path itself: an unknown tool
+  name and a non-object `arguments` payload both come back as envelopes
+  (`UNKNOWN_TOOL` / `INVALID_ARGUMENT`), never as a raised exception.
 - Direct library methods raise a typed error (`ChDBError` and subclasses).
 - Error classification is shared: parse `Code: N. DB::Exception: <msg>. (TYPE)`;
   map by code (`164→READONLY`, `62→SYNTAX`, `46/47/60/81/115→UNKNOWN_*`).
+- Binding-side validation failures (no engine round-trip) use `code: 0` with a
+  shared `type`: `INVALID_ARGUMENT` (bad argument value, e.g. a non-numeric
+  cap or an unknown `tool_specs` dialect), `ACCESS_DENIED` (allowlist),
+  `UNKNOWN_TOOL` (bad `call()` name), `TOOL_ERROR` (any other non-engine
+  failure surfaced through the envelope).
 
 ### P5 — Resource and source controls (normative, optional-per-deployment)
 - **Query timeout** — an optional `max_execution_time` (seconds) bounds runaway
@@ -99,7 +165,11 @@ future coercion must be opt-in and documented here.
 ## Conformance
 `conformance/cases.jsonl` is a language-neutral list of cases: setup-free inputs
 (they use `numbers()`, `file()` on `conformance/fixtures/`, `system.*`, params,
-and write-blocking DDL) plus an expected outcome. Each binding ships a thin
-runner (~30 LOC) that loads the fixture, invokes the named method, and asserts.
-A binding is "contract-conformant" when every case passes. See
+and write-blocking DDL) plus an expected outcome. Its first line is a header
+record `{"fixture": ..., "contract_version": ...}` (no `id`), which runners
+check against their binding's `CONTRACT_VERSION` and skip. A case with
+`"requires": "<feature>"` runs only where `capabilities()` reports that feature
+(everything else is core). Each binding ships a thin runner (~30 LOC) that
+loads the fixture, invokes the named method, and asserts. A binding is
+"contract-conformant" when every applicable case passes. See
 `conformance/README.md`.

integrations/agents/conformance/README.md

@@ -1,33 +1,48 @@
 # Cross-language conformance fixture (vendored)
 
-`cases.jsonl` is a language-neutral list of behaviors every chDB agent-tool
-binding must satisfy. It is the executable half of `../CONTRACT.md`.
-
 This is a **vendored mirror** of `chdb-io/chdb`'s
 `chdb/agents/conformance/`. The chdb-node runner loads *this* copy so the
 TypeScript binding verifies the **same** behaviors as the Python reference. When
 the upstream fixture changes, re-sync this directory.
 
-Each line is one case:
+`cases.jsonl` is a language-neutral list of behaviors every chDB agent-tool
+binding must satisfy. It is the executable half of `../CONTRACT.md`.
+
+The first line is a header record (no `id`):
 
 ```json
-{"id": "...", "pillar": "P1|P2|P3|P4|introspection|safety|catalog", "method": "query|call|list_databases|list_tables|describe|get_sample_data|list_functions", "args": {...}, "expect": {...}}
+{"fixture": "chdb-agents-conformance", "contract_version": "..."}
+```
+
+Runners assert `contract_version` equals their binding's `CONTRACT_VERSION`
+(so a stale vendored fixture fails loudly) and skip the record. Every other
+line is one case:
+
+```json
+{"id": "...", "pillar": "P1|P2|P3|P4|introspection", "method": "query|call|list_databases|list_tables|describe|get_sample_data|list_functions|dataframe_query", "args": {...}, "expect": {...}}
 ```
 
 `args` for `method: "call"` are `{name, arguments}` (the tool-dispatch path);
 for every other method they are the method's keyword arguments.
 
+A case may include `"requires": "<feature>"`: it runs only on bindings whose
+`capabilities().features[<feature>]` is true (e.g. `dataframe_query` is
+Python-only), and is skipped elsewhere. Cases without `requires` are core —
+every binding must pass them. For `method: "dataframe_query"`,
+`args.dataframes` maps each `Python(<name>)` reference to `{column: [values]}`,
+which the runner materializes as a real DataFrame.
+
 A case may include an optional `tool` object with constructor-level config
 (`max_execution_time`, `file_allowlist`, `attachments`, `read_only`, ...). When
 present, the runner builds a dedicated tool from it for that case; otherwise it
-uses a read-only tool. `{{fixtures}}` is substituted inside `tool` too.
+reuses a shared read-only tool. `{{fixtures}}` is substituted inside `tool` too.
 
 `expect` is one of:
 
 | key | meaning |
 |---|---|
 | `rows` | exact row list equality |
-| `error_type` | the method must fail with this error `type` (rejecting path) |
+| `error_type` | the method must fail with this error `type` (raising path) |
 | `truncated` + `row_count` | truncation flag and returned row count |
 | `row_count` | returned row count |
 | `contains_all` | every listed value present in the returned list |
@@ -40,12 +55,13 @@ uses a read-only tool. `{{fixtures}}` is substituted inside `tool` too.
 
 ## Running it
 
-- **Python** (reference, upstream): `python -m unittest tests.test_agents_conformance`.
-- **TypeScript** (`chdb-node`): `npx vitest run test/v3/integrations/agents-conformance.test.ts`
-  — a thin runner that loads this same `cases.jsonl`, maps each `method` to
-  `ChDBTool`, and asserts identically.
+- **Python** (reference): `python -m unittest tests.test_agents_conformance`
+  (loads this file, runs one read-only `ChDBTool`, asserts every case).
+- **TypeScript** (`chdb-node`): a ~30-line runner loads this same file, maps each
+  `method` to `chdbTools()` / the `ChDBVector`-adjacent query surface, and
+  asserts identically. The fixture is vendored (or git-submoduled) so both repos
+  test the *same* behaviors.
 
-Constructor keys in a case's `tool` object use the contract's snake_case
-(`read_only`, `max_execution_time`, `file_allowlist`); the TS runner maps them to
-the binding's camelCase options. A binding is **contract-conformant** when every
-case passes.
+A binding is **contract-conformant** when every case passes. To document an
+intentional per-language divergence, add a case with the divergent expectation
+and a `desc` explaining why — never diverge silently.

integrations/agents/conformance/cases.jsonl

@@ -1,3 +1,4 @@
+{"fixture": "chdb-agents-conformance", "contract_version": "0.2.0"}
 {"id": "p1_create_blocked", "pillar": "P1", "desc": "read-only blocks DDL at the engine", "method": "query", "args": {"sql": "CREATE TABLE conformance_t (a Int32) ENGINE = Memory"}, "expect": {"error_type": "READONLY"}}
 {"id": "p1_escape_blocked", "pillar": "P1", "desc": "read-only cannot be lowered (no escape)", "method": "query", "args": {"sql": "SET readonly = 0"}, "expect": {"error_type": "READONLY"}}
 {"id": "p1_file_allowed", "pillar": "P1", "desc": "CANARY: readonly=2 still allows file() (readonly=1 would reject)", "method": "query", "args": {"sql": "SELECT toInt32(count()) AS c FROM file('{{fixtures}}/sample.csv')"}, "expect": {"rows": [{"c": 2}]}}
@@ -14,3 +15,9 @@
 {"id": "safety_timeout", "pillar": "safety", "desc": "max_execution_time bounds a runaway query", "tool": {"max_execution_time": 1}, "method": "query", "args": {"sql": "SELECT count() FROM numbers(100000000000)"}, "expect": {"error_type": "TIMEOUT_EXCEEDED"}}
 {"id": "safety_allowlist", "pillar": "safety", "desc": "file_allowlist blocks file() paths outside the allowlist", "tool": {"file_allowlist": ["/nonexistent-prefix/"]}, "method": "query", "args": {"sql": "SELECT * FROM file('{{fixtures}}/sample.csv')"}, "expect": {"error_type": "ACCESS_DENIED"}}
 {"id": "catalog_attachments", "pillar": "catalog", "desc": "a read-only tool can query files attached at construction", "tool": {"read_only": true, "attachments": {"rep": "{{fixtures}}/sample.csv"}}, "method": "query", "args": {"sql": "SELECT toInt32(count()) AS c FROM rep"}, "expect": {"rows": [{"c": 2}]}}
+{"id": "p3_maxbytes_utf8", "pillar": "P3", "desc": "max_bytes counts UTF-8 bytes of compact JSON (not UTF-16 units or ASCII-escaped chars)", "tool": {"max_bytes": 60}, "method": "query", "args": {"sql": "SELECT '汉字汉字汉字' AS s FROM numbers(5)"}, "expect": {"truncated": true, "row_count": 2}}
+{"id": "p3_maxrows_non_numeric", "pillar": "P3", "desc": "a non-numeric per-call max_rows is a typed error, never a silently disabled cap", "method": "query", "args": {"sql": "SELECT 1", "max_rows": "lots"}, "expect": {"error_type": "INVALID_ARGUMENT"}}
+{"id": "p3_limit_non_numeric", "pillar": "P3", "desc": "a non-numeric get_sample_data limit is a typed error", "method": "get_sample_data", "args": {"target": "numbers(5)", "limit": "many"}, "expect": {"error_type": "INVALID_ARGUMENT"}}
+{"id": "p2_empty_database_rejected", "pillar": "P2", "desc": "an explicit empty-string database is rejected, not silently treated as unqualified", "method": "call", "args": {"name": "describe_table", "arguments": {"target": "tables", "database": ""}}, "expect": {"envelope_ok": false, "error_type": "TOOL_ERROR"}}
+{"id": "optional_dataframe_query", "pillar": "optional", "desc": "co-located DataFrame query via Python() (capability-gated: dataframe_query)", "requires": "dataframe_query", "method": "dataframe_query", "args": {"sql": "SELECT toInt32(sum(a)) AS s FROM Python(t)", "dataframes": {"t": {"a": [1, 2, 3]}}}, "expect": {"rows": [{"s": 6}]}}
+{"id": "p4_malformed_arguments", "pillar": "P4", "desc": "a non-object call() arguments payload returns a typed envelope, not a throw", "method": "call", "args": {"name": "run_select_query", "arguments": "SELECT 1"}, "expect": {"envelope_ok": false, "error_type": "INVALID_ARGUMENT"}}

integrations/agents/descriptors.d.mts

@@ -0,0 +1,40 @@
+export declare const CONTRACT_VERSION: string
+
+export type ToolSpecDialect = 'anthropic' | 'openai' | 'mcp'
+
+export interface ToolParamDescriptor {
+  name: string
+  type: 'string' | 'integer' | 'object'
+  required?: boolean
+  description?: string
+}
+
+export interface ToolDescriptor {
+  name: string
+  id: string
+  description: string
+  params: ToolParamDescriptor[]
+}
+
+export interface Descriptors {
+  contract_version: string
+  tools: ToolDescriptor[]
+}
+
+export interface Capabilities {
+  contract_version: string
+  tools: string[]
+  features: {
+    dataframe_query: boolean
+    attachments: boolean
+    file_allowlist: boolean
+    max_execution_time: boolean
+    async: boolean
+    streaming: boolean
+    [feature: string]: boolean
+  }
+}
+
+export declare function loadDescriptors(): Descriptors
+export declare function toolSpecs(dialect?: ToolSpecDialect): object[]
+export declare function capabilities(): Capabilities