feat: add bounded semantic SQL completion

README.md

@@ -10,7 +10,7 @@ Published as [`@marimo-team/codemirror-sql`](https://www.npmjs.com/package/@mari
 - **Document sessions** — open SQL documents, apply atomic text and context updates, and track opaque revisions
 - **Built-in dialects** — PostgreSQL, DuckDB, BigQuery, and Dremio
 - **Embedded regions** — mask non-SQL spans (for example notebook interpolations) in document coordinates
-- **Schema-aware completion** — complete relations, namespaces, physical columns, aliases, correlated scopes, and visible CTEs
+- **Schema-aware completion** — complete relations, namespaces, physical columns, inferred CTE/derived outputs, aliases, correlated scopes, set-operation arms, and DML targets
 - **CodeMirror integration** — cancellation-safe completion, disposable detail panels, atomic context updates, and an optional virtualized statement gutter
 - **Composable completion** — package results coexist with SQL keywords, functions, embedded-language sources, and host sources
 - **Isolated parsing** — optional browser-worker parser execution kept off the public API surface
@@ -72,9 +72,10 @@ pnpm install
 pnpm dev
 ```
 
-The playground exercises relation, namespace, physical-column, CTE, correlated
-scope, and `LATERAL` completion against an in-memory catalog. It also exposes
-provider latency and catalog invalidation controls.
+The playground exercises relation, namespace, physical-column, CTE-output,
+derived/set, `USING`, DML, correlated-scope, and `LATERAL` completion against
+an in-memory catalog. It also exposes provider latency and catalog invalidation
+controls.
 
 ## Development
 

demo/index.html

@@ -125,13 +125,41 @@ <h2 class="text-2xl font-bold mb-2">Completion scenarios</h2>
             <strong>CTE visibility</strong>
             <code class="block mt-2 text-sm text-blue-700">WITH recent AS (...) SELECT * FROM rec|</code>
           </button>
+          <button
+            class="example-btn rounded-lg border bg-white p-4 text-left hover:border-blue-400"
+            data-sql="WITH recent AS (SELECT id AS order_id, status FROM sales.orders) SELECT recent.| FROM recent"
+          >
+            <strong>CTE output columns</strong>
+            <code class="block mt-2 text-sm text-blue-700">WITH recent AS (...) SELECT recent.|</code>
+          </button>
           <button
             class="example-btn rounded-lg border bg-white p-4 text-left hover:border-blue-400"
             data-sql="SELECT * FROM main.users u, LATERAL (SELECT u.| FROM sales.orders o) x, sales.customers c"
           >
             <strong>LATERAL scope</strong>
             <code class="block mt-2 text-sm text-blue-700">..., LATERAL (SELECT u.| ...)</code>
           </button>
+          <button
+            class="example-btn rounded-lg border bg-white p-4 text-left hover:border-blue-400"
+            data-sql="SELECT * FROM main.users UNION ALL SELECT * FROM sa|"
+          >
+            <strong>Set-operation arm</strong>
+            <code class="block mt-2 text-sm text-blue-700">... UNION ALL SELECT * FROM sa|</code>
+          </button>
+          <button
+            class="example-btn rounded-lg border bg-white p-4 text-left hover:border-blue-400"
+            data-sql="SELECT * FROM main.users u JOIN main.posts p USING (|)"
+          >
+            <strong>JOIN USING intersection</strong>
+            <code class="block mt-2 text-sm text-blue-700">... JOIN posts USING (|)</code>
+          </button>
+          <button
+            class="example-btn rounded-lg border bg-white p-4 text-left hover:border-blue-400"
+            data-sql="MERGE INTO main.users USING sa| ON true"
+          >
+            <strong>DML relation completion</strong>
+            <code class="block mt-2 text-sm text-blue-700">MERGE INTO users USING sa|</code>
+          </button>
         </div>
       </section>
     </main>

docs/capability-charter.md

@@ -238,8 +238,8 @@ count and estimated retained bytes.
 
 Provisional compressed bundle budgets:
 
-- Framework-independent core with relation, namespace, and column completion:
-  50 KiB
+- Framework-independent core with relation, namespace, column, and local query
+  output completion: 54 KiB
 - Core plus CodeMirror adapter, excluding parser and dialect data: 75 KiB
 - Each ordinary dialect module: 25 KiB
 - Optional `node-sql-parser` chunk: no regression from its recorded baseline

docs/marimo-sql-migration.md

@@ -133,14 +133,13 @@ columns. standard now provides all three through separate bounded providers,
 including qualified and unqualified query-site completion, ambiguity handling,
 stable provenance, cancellation, and batched column work.
 
-The remaining feature gaps are:
-
-- the dialect coverage described above; and
-- output-column inference for CTEs and derived relations. standard completes CTE
-  relation names but deliberately does not send a visible CTE name to the
-  physical column provider. Keep the legacy column source for those sites, or
-  defer full source replacement, until projection/output-column inference
-  lands.
+The remaining cutover gap is the dialect coverage described above. standard
+now infers provable output names for CTEs, derived relations, and the first arm
+of set operations. Explicit CTE column lists are authoritative. Simple column
+references and explicit aliases are inferred locally; stars and unaliased
+expressions remain explicitly partial rather than receiving invented names.
+Visible CTE and derived relation names are never sent to the physical column
+provider.
 
 The fixture feeds marimo's immutable namespace projection—stable entity ID,
 scope, canonical identifier path, and namespace kind—through one public,
@@ -158,8 +157,8 @@ scoped namespace provider on the shared service.
 4. Compare relation and column results with the golden corpus, including
    quoted insert text, aliases, ambiguity, partial/loading/failure states, and
    cold epoch behavior.
-5. Cut over supported physical-relation sites while preserving variable,
-   keyword, and legacy CTE/derived-output column sources.
+5. Cut over supported physical and inferred local-relation sites while
+   preserving variable and keyword sources.
 6. Add dialect coverage, expand the router, and remove completion-only legacy
    schema code. Keep legacy schema data while hover or diagnostics still use
    it.
@@ -185,9 +184,9 @@ Library tests cover relation and physical-column completion for `FROM`, `JOIN`,
 `alias.`, unqualified projections and predicates, `USING`, correlated nested
 queries, quoted identifiers, ambiguous columns, provider
 loading/invalidations, bounded batching, and template barriers. CTE tests prove
-declaration-order relation visibility and that CTE names are not incorrectly
-resolved through the physical column provider; they do not prove CTE
-output-column inference.
+declaration-order relation visibility, conservative output-column inference
+including partial stars and unaliased expressions, and that CTE names are not
+incorrectly resolved through the physical column provider.
 
 Marimo integration tests cover:
 

docs/session-primitives.md

@@ -5,8 +5,18 @@ Import: `@marimo-team/codemirror-sql`
 
 This entry point provides document ownership, atomic text/context updates,
 opaque revisions, dialect registration, lifecycle management, and
-parser-independent relation completion. Diagnostics, hover, navigation, and
-general expression completion are not yet available.
+parser-independent relation and column completion. Relation sites include
+`SELECT` set-operation arms and bounded `INSERT`, `UPDATE`, `DELETE`, and
+`MERGE` targets. Column completion combines physical catalog columns with
+locally inferred CTE and derived-query outputs. Diagnostics, hover, navigation,
+and general expression completion are not yet available.
+
+Local output inference only publishes names it can prove: explicit CTE column
+lists, explicit projection aliases, and simple column references. Set
+operations take their output names from the first arm. Stars, unaliased
+expressions, opaque templates without an explicit alias, and resource-limited
+shapes make the result incomplete; the service does not guess engine-generated
+column names or query a physical provider with a CTE/derived relation name.
 
 CodeMirror consumers should use the separate
 [standard CodeMirror adapter](./codemirror-adapter.md).

implementation.md

@@ -0,0 +1,59 @@
+# SQL editor completion plan
+
+Status: active  
+Updated: 2026-07-27
+
+The standard language-service overhaul is delivered by PR #204. Two additional
+vertical PRs finish the remaining product roadmap without coupling CodeMirror
+to parser ASTs, database clients, language-server transports, or formatter
+implementations.
+
+## PR 1: semantic completion
+
+This PR closes the remaining marimo schema-source cutover blocker and improves
+the most common relation sites:
+
+- infer provable CTE and derived-query output names;
+- make explicit CTE column lists authoritative;
+- use the first set-operation arm for output names;
+- complete relations in later set-operation arms;
+- complete bounded DML target/source relation sites;
+- restrict `JOIN ... USING` completion to shared columns on the immediate
+  relation pair;
+- preserve explicit partial evidence for stars, unaliased expressions,
+  templates, malformed SQL, and resource limits;
+- add owned PostgreSQL, BigQuery, and DuckDB completion corpora;
+- exercise the behavior through sessions, CodeMirror browser tests, the demo,
+  and a warm 10 KiB p95 performance gate.
+
+Local relation names never cross the physical column-provider boundary.
+Completion provenance distinguishes inferred query outputs from catalog
+columns.
+
+## PR 2: language intelligence and release hardening
+
+The final PR adds the remaining feature methods and provider composition:
+
+- syntax, semantic, and host diagnostics;
+- hover;
+- definition, references, highlights, and rename;
+- document symbols and folding;
+- parameter, function, table-function, and snippet contracts;
+- formatting and code-action providers;
+- bounded native-engine and LSP composition seams;
+- deterministic property/fuzz infrastructure and a mutation pilot;
+- heap, listener, worker, and multi-editor leak gates;
+- Chromium, Firefox, and WebKit evidence;
+- public API and bundle regression checks;
+- a packed runtime marimo fixture.
+
+Optional DuckDB, language-server, and formatter integrations remain providers.
+They augment the fast local baseline and cannot delay or replace unrelated
+local evidence.
+
+## Quality gates
+
+Each PR must keep repository and changed-code coverage above 95%, strict type
+checking, zero-warning lint, package/SSR/worker placement, browser tests,
+security scanning, and explicit latency/bundle budgets green. Exactly two
+independent adversarial reviews run only after implementation is complete.

scripts/worker-placement.mjs

@@ -35,11 +35,11 @@ const PARSER_MARKERS = [
   "tableList",
 ];
 const BIGQUERY_GZIP_LIMIT = 50 * 1024;
-const CORE_TOTAL_GZIP_LIMIT = 50 * 1024;
-const CORE_TOTAL_RAW_LIMIT = 184 * 1024;
+const CORE_TOTAL_GZIP_LIMIT = 54 * 1024;
+const CORE_TOTAL_RAW_LIMIT = 200 * 1024;
 const POSTGRESQL_GZIP_LIMIT = 68 * 1024;
-const WORKER_TOTAL_GZIP_LIMIT = 160 * 1024;
-const WORKER_TOTAL_RAW_LIMIT = 700 * 1024;
+const WORKER_TOTAL_GZIP_LIMIT = 164 * 1024;
+const WORKER_TOTAL_RAW_LIMIT = 720 * 1024;
 const MIME_TYPES = new Map([
   [".css", "text/css; charset=utf-8"],
   [".html", "text/html; charset=utf-8"],