From 9d893d4a16ab10d687369e13a88edcde67c53ad3 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 1 Aug 2026 16:33:47 -0400 Subject: [PATCH 1/7] =?UTF-8?q?docs(#248):=20design=20spec=20=E2=80=94=20p?= =?UTF-8?q?ersistability=20derives=20from=20source=20presence,=20not=20sub?= =?UTF-8?q?type?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Full review (Fable): both halves of the fail-open — migrate-ts Pass 1 phantom CREATE TABLE for any sourceless object, and codegen-ts queries/routes/api-model emitting DB-bound artifacts (broken imports) for sourceless objects incl. plain object.value. Principle: an object participates in the DB iff it declares/inherits a source.* — the loader's already-published contract (validate-source-roles: "zero sources ⇒ not persisted"). npm-only PATCH. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01HLoJkFSyoticveo5ehMUAr --- ...e-248-persistability-from-source-design.md | 295 ++++++++++++++++++ 1 file changed, 295 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-01-issue-248-persistability-from-source-design.md diff --git a/docs/superpowers/specs/2026-08-01-issue-248-persistability-from-source-design.md b/docs/superpowers/specs/2026-08-01-issue-248-persistability-from-source-design.md new file mode 100644 index 000000000..cc4fecbf4 --- /dev/null +++ b/docs/superpowers/specs/2026-08-01-issue-248-persistability-from-source-design.md @@ -0,0 +1,295 @@ +# Persistability derives from source presence, never subtype (#248) — Design + +**Date:** 2026-08-01 · **Issue:** #248 · **Branch:** `fix/248-persistability-from-source` · **Status:** proposed + +## 1. Principle + +> **An object participates in the database iff it declares (or inherits via `extends`) a +> `source.*` child. No database decision — table, DDL, FK target, drift check, CRUD +> queries/routes — may key on the object's subtype name.** + +This is not a new rule. It is the **loader's already-declared contract**, which migrate-ts +(and parts of codegen-ts) violate: + +- `server/typescript/packages/metadata/src/persistence/source/validate-source-roles.ts:3-5,20,36` + — *"An object that declares ≥1 source MUST have exactly one with role 'primary'. + **Zero sources is allowed (object is not persisted).**"* A sourceless object of ANY + subtype loads clean and means "not backed by any store". +- `server/typescript/packages/codegen-ts/src/source-detect.ts:1-8` — the entity-file + composer already documents it verbatim: *"Pure metadata-driven, not a typeId + discriminator: any object subtype can opt out of Drizzle table emission simply by + omitting source.rdb."* + +Because the fix aligns implementations with an invariant the loader already publishes, +**this is a bugfix (PATCH), not a behavior contract change** — no flag, no deprecation +window. + +## 2. The bug (both halves, empirically confirmed) + +### 2a. migrate-ts — phantom `CREATE TABLE` (the reported half) + +`buildExpectedSchema` Pass 1 (`server/typescript/packages/migrate-ts/src/expected-schema.ts:139-168`) +skips exactly: non-objects (`:140`), abstracts (`:141`), the **hardcoded string compare** +`child.subType === "value"` (`:142`), TPH subtypes (`:145`), and read-only-source-only +projections (`:147-154`). An object with **no source at all falls through** and gets a +`TableDescriptor` with a **fabricated physical name** (`resolveTableName` falls back to +`pluralize(snake_case(name))`), and enters the FK-target maps (`:176-206`) under that fake +name. Every provider-registered `object.*` subtype fails **open**. + +Reported blast radius: a 155-object wire-protocol package (custom `object.message` +subtype, co-loaded with domain entities so messages can reference them) produced +`UP: 157 CREATE TABLE` — ~146 phantom tables — making `meta migrate` and +`meta verify --db` (which flows through the same `buildExpectedSchema`, via +`src/drift/drift.ts:70-86`) unusable. + +### 2b. codegen-ts — the same fail-open, as broken generated code + +The **table tier is already correct**: `templates/entity-file.ts:98-101` dispatches on +`hasWritableRdbSource` — a sourceless object of any subtype gets the value-object path +(interface + Zod, no Drizzle table). But the **queries/routes/API tier still filters by +subtype**, so it emits DB-bound artifacts against a table that was (correctly) never +emitted. Confirmed by direct `runGen` probe against a 3-object model (`Order` entity with +source; `Money` plain `object.value`; `Ghost` entity with **no** source): + +- `Ghost.queries.ts` emitted, importing `ghosts` (the Drizzle table const) and + `GhostUpdateSchema` from `./Ghost.js` — **neither export exists** (value-object path) + → TS2305 in generated output. +- `Ghost.routes.ts` emitted — same class of broken imports. +- `Money.routes.ts` emitted for a **plain value object** (`generators/routes-file.ts:26-28` + has no value skip at all), importing `moneys`, `MoneyFilterAllowlist`, + `MoneySortAllowlist` — none exist. A pre-existing fail-open of the same family. + +So for the reported model, fixing migrate alone leaves `meta gen` emitting ~146 broken +queries/routes files. Both halves ship in this fix. + +## 3. The derived rules + +Two predicates, both pure functions of the object's **resolving** children +(ADR-0039: an entity may inherit its `source.rdb` via `extends` — Pass 1 and +`hasWritableRdbSource` already read resolving for exactly this reason): + +- **R1 — table tier** (migrate Pass 1; drizzle table already conforms): + a non-abstract, non-TPH-subtype object gets a `TableDescriptor` **iff it declares or + inherits ≥1 WRITABLE source** (`MetaSource.isWritable()`, + `metadata/src/persistence/source/meta-source.ts:89-96`). The `@unmanaged`-writable + branch (→ `fkTargetOnly`, no descriptor) is unchanged and nested inside R1. +- **R2 — DB-artifact tier** (codegen queries/routes/API-doc CRUD): the artifact is + emitted **iff the object declares or inherits ≥1 `source.rdb` of ANY kind** + (new helper `hasAnyRdbSource`, sibling of `hasWritableRdbSource` in + `codegen-ts/src/source-detect.ts`). Writable → CRUD path; read-only-only → + the existing projection read-only path (template-internal `isProjection` dispatch, + unchanged). Zero sources → shape-only artifacts (entity file's value-object path), + no queries/routes. + +### 3a. Why R1 subsumes the old skips EXACTLY + +- **`subType === "value"`:** value purity (ADR-0028) bans sources on values, and the + TS enforcement iterates **resolving** `children()` + (`metadata/src/subtype-rules.ts:99-137`, `validateValuePurity` — `model.children()` + at `:100`, source ban at `:125-136`). So even a value that `extends` an entity "for + shape" cannot reach migrate/codegen with an effective source — it fails load with + `ERR_SUBTYPE_RULE_VIOLATION` first. For every loadable model, + `hasWritableSource(value) === false`. Subsumption is loader-enforced, not + convention. +- **Read-only-only projection skip:** read-only ⇒ not writable, so + `!hasWritableSource` covers it; the view-diff pipeline + (`codegen-ts/src/projection/build-projection-views.ts`, threaded via + `BuildExpectedSchemaOptions.views`) owns those objects, unchanged. +- **Write-through** (writable table + read-only view): has a writable source → table + emitted, view handled by Pass 4. Unchanged. +- **`@unmanaged` writable:** still enters the writable branch; own-source + `isUnmanaged` detection (`expected-schema.ts:161-167`) and + `collectUnmanagedNames` (`unmanaged.ts:32-43`) agree exactly as before. +- **NEW (the fix):** sourceless entity and sourceless custom subtype → skipped; they + also drop out of the `entities`/`fkTargetOnly` FK maps, removing the fabricated + physical FK-target name hazard. An FK whose `@references` target is sourceless now + resolves to nothing and is skipped (`buildForeignKeys`' existing + `if (!refTable) continue;` at `expected-schema.ts:831-832`) — the same behavior as + any unresolvable target today (see §9a for the follow-up loader validation). + +### 3b. Inheritance is deliberate + +A custom subtype (or entity) that `extends` a sourced entity **inherits the source and +is persisted** — `extends` is THE inheritance mechanism (ADR-0029/0039) and resolving +reads are the norm. An object that must never be persisted simply must not extend a +sourced base. (A subtype that should be *incapable* of persistence has an existing home: +its provider just doesn't license `source.*` children — ADR-0037 step 0 child-licensing — +see §5.) + +## 4. Site inventory (the "find them all" review) + +Schema/DDL is TS-owned (ADR-0015): Java/Python/C# emit no DDL, so DB-gating sites are +TS-only. Every site below was read; classification is **keep** (legitimately structural) +or **rederive** (subtype-as-persistence-proxy). + +### migrate-ts (`server/typescript/packages/migrate-ts`) + +| Site | Today | Verdict | +|---|---|---| +| `src/expected-schema.ts:142` `subType === "value"` | hardcoded subtype compare | **REDERIVE** — delete; subsumed by R1 | +| `src/expected-schema.ts:147-154` read-only-only skip | source-kind based, but leaves the sourceless fall-through | **REDERIVE** — collapse to `if (!hasWritableSource) continue;` (delete the now-dead `hasReadOnlySource`) | +| `src/expected-schema.ts:140` `type !== TYPE_OBJECT` | type-axis gate | **KEEP** — only objects can own sources/fields; not a persistence proxy | +| `src/expected-schema.ts:141` `isAbstract` | abstract template | **KEEP** — an abstract is a reusable declaration template, never an instance store, regardless of sources it hoists for concretes to inherit | +| `src/expected-schema.ts:145` `isTphSubtype` (+ `:392-426` TPH walk, `tphConcreteSubtypes`' `TYPE_OBJECT` scan) | discriminator topology | **KEEP** — structural inheritance fact: the subtype's storage IS the base's single table. The *whether-persisted* question is answered on the base by R1; a sourceless TPH hierarchy skips uniformly (subtypes via `:145`, base via R1) | +| `src/expected-schema.ts:161-167` `@unmanaged` own-source branch | source-derived | **KEEP** (now nested under R1) | +| `src/expected-schema.ts:176-206` FK maps / `resolveTargetTable` | populated from Pass 1 | **KEEP** — fixed transitively (phantom names no longer enter) | +| `src/unmanaged.ts:32-43` `collectUnmanagedNames` | iterates own sources | **KEEP** — already source-derived | +| `src/drift/drift.ts:70-86` `computeDrift[FromActual]` | composes `buildExpectedSchema` | **KEEP** — `meta verify --db` / `--d1` fixed transitively | +| `src/referential-actions.ts:40` `VALIDATOR_SUBTYPE_REQUIRED` etc.; field/validator subtype switches throughout `expected-schema.ts` | field/validator TYPE mapping | **KEEP** — type mapping, not persistence gating | +| `diff/ emit/ apply/ introspect/ snapshot/ verify/` | operate on `SchemaSnapshot`, post-metadata | **KEEP** — no metadata subtype awareness (grep-verified) | + +### codegen-ts (`server/typescript/packages/codegen-ts`) + +| Site | Today | Verdict | +|---|---|---| +| `src/templates/entity-file.ts:98-101` table-vs-shape dispatch | `hasWritableRdbSource` | **KEEP** — already conforms; this is the reference pattern | +| `src/reference/entity.ts:82` | same | **KEEP** | +| `src/source-detect.ts:20-31` `hasWritableRdbSource` | resolving, source-derived | **KEEP**; add sibling `hasAnyRdbSource` (any kind) exported via `src/index.ts` (`:97` exports the existing one) | +| `src/generators/queries-file.ts:15-22` `skipNonQueryable = subType !== OBJECT_SUBTYPE_VALUE && !isTphSubtype` | subtype proxy; sourceless falls open | **REDERIVE** → `hasAnyRdbSource(e) && !isTphSubtype(e)` | +| `src/reference/queries.ts:108-111` same filter (scaffold-and-own asset; `meta init` copies it verbatim via `reference-templates.ts` / `cli/src/commands/init.ts:275-293`) | same | **REDERIVE** (new scaffolds only; existing consumers own their copies — §7) | +| `src/generators/routes-file.ts:26-28` filter `@emitRoutes !== false && !isTphSubtype` | **no persistence gate at all** — values AND sourceless objects get broken routes files | **REDERIVE** → add `hasAnyRdbSource(e)` conjunct | +| `src/reference/routes.ts:40-44` same | same | **REDERIVE** | +| `src/generators/routes-file-hono.ts:34` filter `@emitRoutes !== false` only | same fail-open | **REDERIVE** → add `hasAnyRdbSource(e)` conjunct (its missing TPH handling is pre-existing and out of scope here) | +| `src/generators/api-model.ts:337-349` `isQueryable` ("mirror of the queries filter") | subtype proxy | **REDERIVE** in lockstep — api-docs must not document CRUD that no longer exists | +| `src/projection/projection-detector.ts`, `extract-view-spec.ts:256`, `build-projection-views.ts:163-269` | own read-only-kind sources | **KEEP** — already source-derived (own-source reads are the chartered C#-parity classification, ADR-0039 comments in situ) | +| `src/relation-resolver.ts:69` skips projections | source-derived | **KEEP** | +| `src/runner.ts:161,175` value-object emitted-name collision domain (ADR-0044) | subtype check | **KEEP** — file/type NAMING taxonomy, not a DB decision | +| `src/generators/docs-data-builder.ts:52,616`, `src/templates/mermaid-er.ts:115` | value labeling for docs/ER | **KEEP** — docs depict the MODEL taxonomy; persisted-ness already flows from `hasWritableRdbSource` (`:52`) | +| `src/templates/callable-file.ts:78` proc-args value lookup; `src/generators/prompt-render-file.ts:48` payload VOs; `src/generators/api-model.ts:765` payload-ref VO check | value-shape semantics | **KEEP** — taxonomy (what a value IS), not persistence | +| `src/templates/tph-discriminator.ts:213` `subType !== OBJECT_SUBTYPE_ENTITY` | TPH scan scoped to entities | **KEEP** — TPH is chartered as an entity-inheritance concept (`spec/metamodel/object.json`: `@discriminator`/`@discriminatorValue` registered on `object.entity` only) | +| `src/templates/drizzle-schema.ts:186-211` `buildFkMapForEntity` | no target-persisted check; also bare-name `findObject` | **DEFER** — an `@enforce`'d reference to a non-persisted target is a modeling error the LOADER should reject (§9a); the bare-name lookup is already tracked as a #228/#244-adjacent gap | +| `src/templates/barrel.ts` | exports entity modules only | **KEEP** — never references queries/routes, so filter changes can't strand it (verified) | + +### Elsewhere (verified clean) + +- `cli`: `commands/migrate.ts` / `verify.ts` compose `buildExpectedSchema` + + `buildProjectionViews` — no own subtype gating; fixed transitively. +- `runtime-ts`: `validator-runner.ts:229` (`OBJECT_SUBTYPE_VALUE`) is VO-validation + taxonomy; the rest is TYPE_OBJECT/identity lookups. **KEEP** — runtime binds to + tables the consumer passes in; it makes no persistability decision. +- `sdk`, client packages (`runtime-web`/`react`/`tanstack`): no object-subtype DB + gates (grep-verified). See §9c for the API-surface-parity follow-up. + +## 5. Decision 1 — explicit `persistence?: "never" | "bySource"` on `TypeDefinition`: **AGAINST** + +The issue's alternative would let a provider declare a subtype's persistence posture on +the type registration. Rejected: + +1. **Fully derivable** — the loader contract already encodes "zero sources ⇒ not + persisted" per OBJECT; a per-TYPE flag restates a subset of that and creates a + contradiction case (`persistence: "never"` + a declared source) needing a new error + for zero expressive gain. ADR-0023: never add vocabulary a rule can derive. +2. **Wrong axis per ADR-0007/0037** — physical/persistence concerns live on the + `source` child of the OBJECT, not on the type definition; a type-level flag is a + second source of truth for the same fact. +3. **The legitimate residual need already has a home** — "this subtype must be + *incapable* of persistence" is child-licensing: the provider simply does not + license `source.*` under its subtype (`childRules`, FR-033 fail-closed), and the + loader rejects a declared source with `ERR_CHILD_NOT_ALLOWED`. No new field. +4. **Cost** — `TypeDefinition` shape is mirrored in five ports and gated by + `registry-conformance`; a new field is a coordinated all-port change inside what + must stay an npm-only PATCH. + +## 6. Decision 2 — migrate/verify scope filter (`--entities` / package): **DEFER (separable follow-up)** + +`ResolvedGenConfig` has `entities: string[]` (`cli/src/lib/config.ts:25-28`, +`args.ts:90-96` — gen positionals); `ResolvedMigrateConfig` (`config.ts:37-59`) and +`VerifyFlags` (`args.ts:189-197`) have no scope. Recommendation: **not in this fix.** + +1. The core fix removes the *need* in the reported case (phantom tables gone) — the + remaining value is ergonomics (scoping a big multi-package model), not correctness. +2. Migrate scoping has real design surface gen scoping doesn't: a scoped expected + schema diffed against a full DB proposes **drops for every out-of-scope table** + unless the scope also filters the ACTUAL side (and the snapshot ledger, and the + FK-graph cascade keying from #241). That both-sides-scoped diff semantics deserves + its own design, not a rider on a PATCH. + +## 7. Backward compatibility & byte-identity guardrail + +**The norm (well-formed models — every table-owning object declares/inherits a source):** + +- `meta migrate` / `meta verify --db|--d1`: **byte-identical** expected schema, SQL, + and drift output. Only sourceless objects change classification, and the only + objects that were both sourceless and previously table-classified are the bug. +- `meta gen`: **no emitted file's content changes.** The only delta is file-SET + membership: files that stop being emitted are exactly those that imported + non-existent exports (`ghosts`, `MoneyFilterAllowlist`, …) and could never have + typechecked. Pinned by no-churn assertions + the golden gate (§8). + +**Previously-bitten models (had sourceless objects):** + +- A DB/snapshot that already **contains phantom tables** (from applying the buggy + migration) will see `DROP TABLE` proposals on the next migrate — correct (they are + phantoms), and destructive-gated behind the existing `--allow drop-table` policy; + called out in the CHANGELOG entry. +- Previously-written broken `*.queries.ts`/`*.routes.ts` files are **not pruned** — + `meta gen` never deletes; they simply stop regenerating (consumer deletes them). +- **Scaffold-and-own** (ADR-0034): existing consumers own their copied + `codegen/generators/{queries,routes}.ts` — the reference-template fix reaches new + `meta init` scaffolds only; existing copies keep the subtype filter until the + consumer re-syncs. Documented, acceptable: their owned copies are no *more* wrong + than before, and the engine-side generators most consumers use are fixed. + +## 8. Versioning + test strategy + +**npm-only PATCH** (next `0.20.x`). Changed packages: `migrate-ts`, `codegen-ts` +(`cli` untouched — it composes the fixed pieces; `metadata` untouched — the contract +already exists there). PyPI/NuGet/Maven: **no changed product file** — no other port +emits DDL (ADR-0015). Litmus (per the versioning policy): a `^prev` consumer with a +well-formed model runs `npm update && meta gen && meta migrate` and gets byte-identical +output. Cross-port codegen (Spring/Pydantic/Kotlin generators) is audited as a +follow-up (§9b), not folded into this release. + +**Tests (TDD; details in the plan):** + +1. `migrate-ts` regression (`test/expected-schema-persistability.test.ts`, new): + co-load persisted entity + sourceless entity + sourceless CUSTOM subtype + (`object.message` registered via `composeRegistry([...coreProviders, provider])`, + the pattern in `metadata/test/child-placement-enforcement.test.ts:142`) → exactly + one table; FK `@references` a sourceless object → no FK descriptor, no fabricated + name; diff vs empty DB → exactly one `create-table`. +2. `codegen-ts` regression (probe-style `runGen`): sourceless entity + value + entity + model → `Ghost.queries/routes`, `Money.routes` NOT emitted; `Order.*` content + byte-identical to a run without the sourceless co-loads; same assertions through + the reference (scaffold) generators. +3. Existing pins that must stay green (byte-identity): `expected-schema.test.ts` + (value skip at `:447`), TPH (`expected-schema-tph.test.ts`), `@unmanaged`/#208, + projection/view suites, full `migrate-ts` + `codegen-ts` + `cli` suites. + +**Known traps (from the project memory, planned around explicitly):** + +- **Golden gate lives outside the package suite** — run + `cd server/typescript/packages/codegen-ts && bun test test/golden/` explicitly. The + golden corpus has no top-level `object.value` and all its entities are sourced + (grep-verified), so expected result is green-unchanged; any diff is a stop signal. +- **`migrate-ts` changes can regress the SEPARATE + `server/typescript/packages/integration-tests`** (`view-lifecycle-{pg,sqlite}.test.ts`) + — only the slow CI lane runs them; run locally (Docker/Testcontainers) before done. +- **Migrate correctness gate** — the change only removes never-correct tables from the + expected side; the existing real-engine round-trips (`migrate-ts/test/integration/` + lifecycle/apply suites) pin emit→apply→introspect→re-diff-EMPTY for sourced models + and must pass unchanged. +- Never bare `bun test` at repo root; stage explicit paths, never `git add -A`. + +## 9. Out of scope / deferred follow-ups (to file as issues) + +- **9a. Loader validation: enforced reference/relationship → non-persisted target.** + Today an `@enforce`'d `identity.reference` (or relationship) from a persisted entity + to a sourceless object yields divergent downstream behavior: migrate silently skips + the FK (unresolvable target), while `drizzle-schema.ts:186-211` still emits a + `.references()` against a table const that doesn't exist (broken generated code). + The right fix is a LOADER error (cross-port, all four loaders + a conformance + fixture) — a physical FK needs a physical target. Too big for this PATCH. +- **9b. Cross-port codegen audit.** Java `codegen-spring`, Python, Kotlin, C# emit + repositories/controllers/DTOs — audit whether their emission gates are + subtype-proxies with the same sourceless fail-open (no DDL, so lower stakes; same + principle). Pattern-match to the #228 out-of-scope audit list. +- **9c. Client API-surface parity.** `codegen-ts-react` / `codegen-ts-tanstack` emit + hooks/forms for any object (compilable today — they bind types/schemas that ARE + emitted); once routes stop existing for sourceless objects those hooks dangle at + runtime. Align their filters with `hasAnyRdbSource` as an ergonomics follow-up. +- **9d. `meta migrate` / `meta verify` scope filter** (§6) — both-sides-scoped diff + design. +- **9e. `routes-file-hono` TPH gating + `drizzle-schema`/`relation-resolver` + bare-name `findObject`** — pre-existing gaps adjacent to, but independent of, this + fix (the latter already noted under the #228/#244 follow-up audits). From d87ba506e943144fa58503274fbc7314fe35d73e Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 1 Aug 2026 16:33:54 -0400 Subject: [PATCH 2/7] =?UTF-8?q?docs(#248):=20implementation=20plan=20?= =?UTF-8?q?=E2=80=94=20persistability=20from=20source=20(5=20tasks)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SDD plan: (1) migrate-ts Pass 1 collapse to `if (!hasWritableSource) continue;` + tests incl. custom-subtype provider; (2) codegen-ts hasAnyRdbSource + queries/ routes/hono/api-model filters + no-churn probe; (3) reference/scaffold templates; (4) integration/golden/typecheck gates; (5) CHANGELOG. Byte-identical for well-formed models. npm-only PATCH (migrate-ts + codegen-ts). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01HLoJkFSyoticveo5ehMUAr --- ...01-issue-248-persistability-from-source.md | 172 ++++++++++++++++++ 1 file changed, 172 insertions(+) create mode 100644 docs/superpowers/plans/2026-08-01-issue-248-persistability-from-source.md diff --git a/docs/superpowers/plans/2026-08-01-issue-248-persistability-from-source.md b/docs/superpowers/plans/2026-08-01-issue-248-persistability-from-source.md new file mode 100644 index 000000000..433a39a1d --- /dev/null +++ b/docs/superpowers/plans/2026-08-01-issue-248-persistability-from-source.md @@ -0,0 +1,172 @@ +# Persistability from source presence (#248) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax. + +**Goal:** Derive every "database thing" decision from declared/inherited `source.*` presence instead of object-subtype name compares: `buildExpectedSchema` Pass 1 stops emitting phantom `CREATE TABLE`/FK targets for sourceless objects (#248), and the codegen queries/routes/API-doc tier stops emitting broken DB-bound artifacts for them. + +**Architecture:** Two npm packages. (A) `migrate-ts`: Pass 1 collapses the `subType === "value"` compare + read-only-only skip into one rule — *table iff a writable source is declared/inherited* (`if (!hasWritableSource) continue;`); FK maps, drift, and `meta verify --db/--d1` are fixed transitively. (B) `codegen-ts`: new `hasAnyRdbSource` helper (sibling of `hasWritableRdbSource` in `source-detect.ts`); the queries/routes/hono/api-model filters gain it as a conjunct, replacing (queries/api-model) or supplementing (routes/hono, which had NO persistence gate) the subtype compare; the ADR-0034 reference templates get the same swap. Value objects are subsumed exactly: the loader's value-purity pass bans sources on the RESOLVING view (`metadata/src/subtype-rules.ts:99-137`), so no loadable value has an effective source. + +**Tech Stack:** TypeScript (bun test), `@metaobjectsdev/metadata` loader + `composeRegistry` test providers, Testcontainers PG/Docker for the integration gates. + +**Design spec:** `docs/superpowers/specs/2026-08-01-issue-248-persistability-from-source-design.md` + +## Global Constraints + +- **Byte-identity guardrail:** for a model where every table-owning object declares/inherits a source (the norm), `meta migrate`/`meta verify` output is byte-identical and NO emitted `meta gen` file's CONTENT changes — the only permitted delta is that files importing non-existent exports (never typechecked) stop being emitted. Any other output change is a task failure. +- **Versioning:** npm-only PATCH. Changed product packages: `migrate-ts`, `codegen-ts` ONLY. No `metadata` package change, no CLI flag changes, no cross-port files. +- **Named constants:** the fix DELETES the inline `"value"` literal in `expected-schema.ts:142`; do not introduce new inline metamodel strings anywhere. +- **ADR-0039:** the persistability predicates read RESOLVING `children()` (an entity may inherit `source.rdb` via `extends`); keep the existing own-source read for the `@unmanaged` branch and its comment. Any own/resolving choice carries its sanctioned-case comment. +- **Tests:** never a bare `bun test` at repo root. Scope: `cd server/typescript && bun test packages/` (picks up `server/typescript/bunfig.toml` preload). +- **Git:** stage explicit paths only, NEVER `git add -A`. Commit to branch `fix/248-persistability-from-source`. +- **Public-repo hygiene:** fixtures/tests use `acme::*` packages and generic names (`Order`, `Ghost`, `WireNote`); no private/other-project names, no absolute home paths in committed files or commit messages. + +--- + +### Task 1: migrate-ts — Pass 1 persistability from writable-source presence + +**Files:** +- Modify: `server/typescript/packages/migrate-ts/src/expected-schema.ts` (Pass 1, `:125-168`: the skip-list comment block, line `:142`, lines `:147-154`) +- Create: `server/typescript/packages/migrate-ts/test/expected-schema-persistability.test.ts` + +**Interfaces:** +- Consumes: `MetaSource.isWritable()` (already imported at `expected-schema.ts:13`); `MetaDataLoader` + `InMemoryStringSource` (test pattern: `test/expected-schema-schema-aware.test.ts:5-10`); `composeRegistry`/`coreProviders`/`TypeRegistry`/`TypeId`/`MetaObject` from `@metaobjectsdev/metadata` (custom-provider pattern: `metadata/test/child-placement-enforcement.test.ts:142`; `TypeDefinition`/`ChildRule` shapes: `metadata/src/registry.ts:24-120`). +- Produces: `buildExpectedSchema` that skips any object without a declared/inherited writable source — consumed transitively by `meta migrate`, `computeDrift[FromActual]` (`src/drift/drift.ts`), and Task 4's gates. No signature changes. + +- [ ] **Step 1: Write the failing tests.** In `expected-schema-persistability.test.ts`, load models via the `loadJson` helper pattern (default registry unless noted), all in package `acme::probe`: + 1. **Sourceless entity co-load:** `Order` (`field.long id`, `field.string ref`, `source.rdb @table:"orders"`, `identity.primary`) + `Ghost` (`field.long id`, `field.string note`, `identity.primary`, NO source). Assert `snapshot.tables.map(t => t.name)` is exactly `["orders"]`. + 2. **No fabricated FK target:** same model + on `Order` a `field.long ghostId` and `identity.reference` `@fields:["ghostId"] @references:"Ghost"`. Assert the `orders` table has `foreignKeys` length 0 (no FK against a fabricated `ghosts` name) and `buildExpectedSchema` does not throw. + 3. **Custom subtype (the reported scenario):** build a loader with `composeRegistry([...coreProviders, wireProvider])` where `wireProvider` (`id: "test-wire-messages"`, `dependencies: ["metaobjects-core-types"]`) registers `typeId: new TypeId(TYPE_OBJECT, "message")`, `factory: (t, n) => new MetaObject(t, n)`, `childRules: [{ childType: TYPE_FIELD, childSubType: "*", childName: "*" }]`, `attributes: []`. Load `Order` (as above) + `object.message` `WireNote` (two fields, no source). Assert tables are exactly `["orders"]`. (If the FR-033 child-side placement pass rejects `field.*` under `object.message` despite the parent-side childRule, extend the provider per the pattern in `metadata/test/child-placement-enforcement.test.ts` — the fixture's point is only "custom subtype, no source, no table".) + 4. **Inherited writable source still persists:** abstract `object.entity` `Base` carrying `source.rdb @table:"things"` + `identity.primary` + `field.long id`; concrete `Thing extends Base` with one own field. Assert exactly `["things"]` (resolving-children read; also pins that the ABSTRACT skip still precedes the source rule). +- [ ] **Step 2: Run to verify RED.** Run: `cd server/typescript && bun test packages/migrate-ts/test/expected-schema-persistability.test.ts`. Expected failures: test 1 receives `["orders", "ghosts"]`; test 2 finds an FK with `refTable: "ghosts"`; test 3 receives `["orders", "wire_notes"]`. Test 4 should already pass (pin). +- [ ] **Step 3: Implement.** In `expected-schema.ts` Pass 1: + - Delete line `:142` (`if (child.subType === "value") continue;`). + - Replace `:147-154` (the `hasReadOnlySource`/`hasWritableSource` pair + projection skip) with the single rule — keep the existing `hasWritableSource` computation, delete the now-dead `hasReadOnlySource`: + ```ts + // #248 — persistability derives from source presence, never subtype (loader + // contract: zero sources ⇒ not persisted — metadata validate-source-roles). + // Table iff a WRITABLE source is declared or inherited (ADR-0039 resolving). + // Subsumes the old `subType === "value"` skip (value purity bans sources on + // the resolving view) and the read-only-only projection skip (view pipeline + // owns those); closes the fail-open where a sourceless object (custom + // subtype or plain entity) got a phantom CREATE TABLE + fabricated FK name. + const hasWritableSource = child.children().some( + (c) => c instanceof MetaSource && c.isWritable(), + ); + if (!hasWritableSource) continue; + ``` + - Update the Pass 1 skip-list comment block (`:125-130`) to name the new rule. + - Leave untouched: `:140` (TYPE_OBJECT), `:141` (isAbstract), `:145` (isTphSubtype), `:155` (`resolveTableName`), `:161-167` (`@unmanaged` own-source branch). +- [ ] **Step 4: Run the new test — expect GREEN.** Same command as Step 2. +- [ ] **Step 5: Full migrate-ts unit/emit suite for byte-identity.** Run: `cd server/typescript && bun test packages/migrate-ts` (excludes nothing; Docker-dependent integration files skip if the engine is absent — they run in Task 4). Expected: all green — in particular the existing value-skip (`test/unit/expected-schema.test.ts:447`), TPH (`test/expected-schema-tph.test.ts`), `@unmanaged`/#208, and view/diff suites unchanged. +- [ ] **Step 6: Commit.** +```bash +git add server/typescript/packages/migrate-ts/src/expected-schema.ts \ + server/typescript/packages/migrate-ts/test/expected-schema-persistability.test.ts +git commit -m "fix(#248): migrate-ts derives table persistability from writable-source presence, not subtype" +``` + +--- + +### Task 2: codegen-ts — `hasAnyRdbSource` + queries/routes/hono/api-model filter rederivation + +**Files:** +- Modify: `server/typescript/packages/codegen-ts/src/source-detect.ts` (add `hasAnyRdbSource`) +- Modify: `server/typescript/packages/codegen-ts/src/index.ts` (`:97` — export it beside `hasWritableRdbSource`) +- Modify: `server/typescript/packages/codegen-ts/src/generators/queries-file.ts` (`:15-22` `skipNonQueryable`) +- Modify: `server/typescript/packages/codegen-ts/src/generators/routes-file.ts` (`:26-28` filter) +- Modify: `server/typescript/packages/codegen-ts/src/generators/routes-file-hono.ts` (`:34` filter) +- Modify: `server/typescript/packages/codegen-ts/src/generators/api-model.ts` (`:337-349` `isQueryable`) +- Create: `server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts` + +**Interfaces:** +- Consumes: `MetaSource`/`TYPE_SOURCE`/`SOURCE_SUBTYPE_RDB` (as `hasWritableRdbSource` does, `source-detect.ts:9-11`); `runGen` (probe pattern: config `{ outDir, dialect: "postgres", dbImport: "./db", generators: [...] }`, `write: false` is unsupported for content capture — write to a bun `tmpdir` and read files back, or assert on the returned `files[].path` set plus on-disk content). +- Produces: exported `hasAnyRdbSource(entity: MetaObject): boolean` — TRUE iff resolving `children()` contains ≥1 `source.rdb` `MetaSource` of ANY kind. Consumed by Task 3's reference templates. + +- [ ] **Step 1: Write the failing test.** In `sourceless-objects.test.ts`, load (default registry, package `acme::probe`): `Order` (sourced entity, as Task 1), `Money` (`object.value`, two fields), `Ghost` (entity, `identity.primary`, NO source). Run `runGen` with `[entityFile(), queriesFile(), routesFile(), routesFileHono()]` into a temp dir. Assert: + - Emitted path set INCLUDES `Order.ts`, `Order.queries.ts`, `Order.routes.ts`, the Order hono file, `Money.ts`, `Ghost.ts`. + - Emitted path set EXCLUDES `Ghost.queries.ts`, `Ghost.routes.ts`, `Money.queries.ts`, `Money.routes.ts`, and both hono files for `Money`/`Ghost`. + - **Content no-churn pin:** run `runGen` a second time on a model containing ONLY `Order`; assert `Order.ts` / `Order.queries.ts` / `Order.routes.ts` contents are byte-identical between the two runs. + - **Projection still queryable pin:** a read-only-source projection (model on an existing projection test fixture in `codegen-ts/test`) still gets its queries/routes files. +- [ ] **Step 2: Run to verify RED.** Run: `cd server/typescript && bun test packages/codegen-ts/test/sourceless-objects.test.ts`. Expected: EXCLUDES assertions fail — today `Ghost.queries.ts`, `Ghost.routes.ts`, `Money.routes.ts` and hono files ARE emitted (empirically confirmed during design). +- [ ] **Step 3: Implement.** + - `source-detect.ts`: add beside `hasWritableRdbSource` (same iteration, drop the `isWritable()` gate): + ```ts + /** True when the object declares (or inherits via extends — ADR-0039 resolving) + * at least one source.rdb child of ANY kind. Zero sources ⇒ not backed by any + * store (loader contract, validate-source-roles): no DB-bound artifacts. */ + export function hasAnyRdbSource(entity: MetaObject): boolean { ... } + ``` + - `index.ts:97`: export it. + - `queries-file.ts`: `skipNonQueryable = (e) => hasAnyRdbSource(e) && !isTphSubtype(e);` (drop the `OBJECT_SUBTYPE_VALUE` import if now unused); update the comment: values are subsumed (no source, loader-enforced), sourceless objects newly skipped. + - `routes-file.ts:26-28`: filter becomes `e.attr(CODEGEN_ATTR_EMIT_ROUTES) !== false && hasAnyRdbSource(e) && !isTphSubtype(e) && userFilter(e)`. + - `routes-file-hono.ts:34`: add the `hasAnyRdbSource(e)` conjunct (do NOT add TPH logic — pre-existing, out of scope). + - `api-model.ts` `isQueryable`: `hasAnyRdbSource(obj) && !isTphSubtype(obj)`; update its mirror comment (it documents itself as the queries-filter mirror). +- [ ] **Step 4: Run the new test — expect GREEN.** Same command as Step 2. +- [ ] **Step 5: Package suite + golden gate (byte-identity).** Run: `cd server/typescript && bun test packages/codegen-ts` AND explicitly `cd server/typescript/packages/codegen-ts && bun test test/golden/` (the golden gate lives outside default attention; the corpus has no sourceless objects/top-level values with routes, so expected result is green-UNCHANGED — any snapshot diff is a stop-and-review signal, not an auto-regen). +- [ ] **Step 6: Commit.** +```bash +git add server/typescript/packages/codegen-ts/src/source-detect.ts \ + server/typescript/packages/codegen-ts/src/index.ts \ + server/typescript/packages/codegen-ts/src/generators/queries-file.ts \ + server/typescript/packages/codegen-ts/src/generators/routes-file.ts \ + server/typescript/packages/codegen-ts/src/generators/routes-file-hono.ts \ + server/typescript/packages/codegen-ts/src/generators/api-model.ts \ + server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts +git commit -m "fix(#248): codegen-ts gates queries/routes/api-model on source presence, not subtype" +``` + +--- + +### Task 3: reference (scaffold-and-own) templates get the same derived filter + +**Files:** +- Modify: `server/typescript/packages/codegen-ts/src/reference/queries.ts` (`:108-111` `skipNonQueryable`) +- Modify: `server/typescript/packages/codegen-ts/src/reference/routes.ts` (`:40-44` filter) +- Modify: `server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts` (extend) + +**Interfaces:** +- Consumes: Task 2's exported `hasAnyRdbSource` — the reference templates import ONLY `@metaobjectsdev/codegen-ts` (ADR-0034: a copied file must work verbatim in a consumer repo), so both must import it from the package root, NOT a relative path. +- Produces: fixed scaffolds for future `meta init` runs (`cli/src/commands/init.ts:281-293` copies `src/reference/*.ts` verbatim via `readReferenceTemplate`). Existing consumers own their copies — no propagation, per the design spec §7. + +- [ ] **Step 1: Extend the failing test.** In `sourceless-objects.test.ts`, add a second `runGen` pass over the same Order/Money/Ghost model using the REFERENCE generators (`import { queriesFile as refQueries } from "../src/reference/queries.js"`, same for routes — they are importable TS in-repo even though excluded from the tsc build). Assert the same INCLUDES/EXCLUDES sets as Task 2 Step 1. +- [ ] **Step 2: Run to verify RED.** `cd server/typescript && bun test packages/codegen-ts/test/sourceless-objects.test.ts` — the reference-generator EXCLUDES assertions fail. +- [ ] **Step 3: Implement.** Swap both reference filters to `hasAnyRdbSource(e) && !isTphSubtype(e)` (queries) / add the `hasAnyRdbSource(e)` conjunct (routes), importing `hasAnyRdbSource` from `@metaobjectsdev/codegen-ts`; update the header comments (these files are consumer-facing documentation). +- [ ] **Step 4: Run — expect GREEN**, then the cli suite (init scaffolds + config wiring consume these assets): `cd server/typescript && bun test packages/cli`. +- [ ] **Step 5: Commit.** +```bash +git add server/typescript/packages/codegen-ts/src/reference/queries.ts \ + server/typescript/packages/codegen-ts/src/reference/routes.ts \ + server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts +git commit -m "fix(#248): reference scaffold generators derive queryability from source presence" +``` + +--- + +### Task 4: guardrail gates — integration suites, typecheck, workspace build + +**Files:** none modified (verification-only; any failure loops back into Tasks 1-3). + +**Interfaces:** consumes Docker (Testcontainers PG) locally; the slow-lane suites CI won't run on this PR. + +- [ ] **Step 1: migrate-ts real-engine round-trips** (emit → apply → introspect → re-diff EMPTY must hold unchanged): `cd server/typescript && bun test packages/migrate-ts/test/integration` (Docker running). Expected: green, byte-identical behavior for sourced models. +- [ ] **Step 2: the SEPARATE integration-tests package** (the known trap — only slow CI runs it): `cd server/typescript && bun test packages/integration-tests/test/view-lifecycle-pg.test.ts packages/integration-tests/test/view-lifecycle-sqlite.test.ts`. Then the package's remaining suite: `cd server/typescript && bun test packages/integration-tests`. +- [ ] **Step 3: remaining server packages that consume the changed ones:** `cd server/typescript && bun test packages/cli packages/runtime-ts packages/codegen-ts-react packages/codegen-ts-tanstack`. +- [ ] **Step 4: workspace typecheck + build** (bun test does NOT typecheck; the pre-push hook gates on this): from repo root, `bun run --filter '*' build && bun run --filter '*' typecheck`. Expected: green. +- [ ] **Step 5: no commit** (nothing changed); record pass/fail evidence in the task report. + +--- + +### Task 5: CHANGELOG + follow-up ledger + +**Files:** +- Modify: `CHANGELOG.md` (new `[Unreleased]` npm-only section, or fold into the current one if release scoping demands — coordinate with whatever is unreleased at execution time) + +**Interfaces:** none downstream; the design spec §9 holds the follow-up issue list (loader validation for enforced refs → non-persisted targets; cross-port codegen audit; client hook parity; migrate/verify scope filter) for the maintainer to file — do NOT file issues from this plan. + +- [ ] **Step 1: Write the CHANGELOG entry.** npm-only PATCH, `### Fixed` — cover: (a) phantom `CREATE TABLE`/FK-target for sourceless objects in `meta migrate`/`meta verify --db|--d1` (the 155-object custom-subtype scenario, genericized); (b) codegen no longer emits queries/routes (Fastify + Hono) or api-docs CRUD for a sourceless object — including the pre-existing value-object routes fail-open; (c) the byte-identity statement (well-formed models unchanged; only never-compilable files stop being emitted); (d) the migration note: a DB/snapshot already holding phantom tables will now correctly propose `DROP TABLE`, gated behind `--allow drop-table`; stale broken generated files are not auto-pruned. Reference the loader contract (zero sources ⇒ not persisted) as the pre-existing invariant this aligns to. +- [ ] **Step 2: Hygiene scan.** Re-read the staged diff for private names/absolute paths (`scripts/ci-local.sh --quick` covers the leak scan). +- [ ] **Step 3: Commit.** +```bash +git add CHANGELOG.md +git commit -m "docs(#248): changelog — persistability derives from source presence (npm-only patch)" +``` From 7916b53948aa4cb90b3ef592c96db9a3085120c8 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 1 Aug 2026 16:48:46 -0400 Subject: [PATCH 3/7] fix(#248): migrate-ts derives table persistability from writable-source presence, not subtype MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit buildExpectedSchema Pass 1 decided "is this a table" via a hardcoded `subType === "value"` compare, so any object with no source.* (a custom subtype, or a plain sourceless entity) fell through and got a phantom CREATE TABLE with a fabricated physical name (entering the FK-target maps under that fake name). Replace the subtype compare + the read-only-only projection skip with a single rule: skip an object unless it declares or inherits >=1 WRITABLE source (ADR-0039 resolving children), matching the loader's own already-published contract that zero sources means "not persisted" (validate-source-roles.ts). Subsumes both prior skips exactly (value purity bans sources on the resolving view; read-only-only implies !hasWritableSource) while closing the fail-open for sourceless objects of any subtype. Also adds a bare `source.rdb` to ~18 pre-existing migrate-ts test fixtures/inline models that declared no source at all — they only produced a table under the old subtype-based fallback. A source with no @table/name resolves its physical name via the same entity-name fallback the old sourceless path used, so table names are unchanged; this just makes the models well-formed under the corrected rule instead of leaving the suite exercising an empty schema. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01HLoJkFSyoticveo5ehMUAr --- .../migrate-ts/src/expected-schema.ts | 22 +- .../drift/compute-drift-from-actual.test.ts | 1 + .../expected-schema-persistability.test.ts | 195 ++++++++++++++++++ .../test/fixtures/single-entity.json | 1 + .../fixtures/trainer-website-entities.json | 7 + .../test/fixtures/two-entities-fk.json | 2 + .../test/integration/d1-cascade.test.ts | 2 +- .../test/integration/drift-sqlite.test.ts | 1 + .../sqlite-autoset-default.test.ts | 1 + .../sqlite-check-evolution.test.ts | 1 + .../sqlite-default-semantics.test.ts | 1 + .../integration/sqlite-fk-convergence.test.ts | 3 + .../integration/sqlite-index-escapes.test.ts | 1 + .../integration/sqlite-quoted-default.test.ts | 1 + .../sqlite-recreate-roundtrip.test.ts | 2 + .../expected-schema-isarray-scalars.test.ts | 1 + .../unit/expected-schema-lenient-inet.test.ts | 1 + .../test/unit/expected-schema.test.ts | 3 + .../unit/referential-actions-nullable.test.ts | 2 + .../test/unit/referential-actions.test.ts | 4 + 20 files changed, 241 insertions(+), 11 deletions(-) create mode 100644 server/typescript/packages/migrate-ts/test/expected-schema-persistability.test.ts diff --git a/server/typescript/packages/migrate-ts/src/expected-schema.ts b/server/typescript/packages/migrate-ts/src/expected-schema.ts index 3e2dc896b..5ed94c1bb 100644 --- a/server/typescript/packages/migrate-ts/src/expected-schema.ts +++ b/server/typescript/packages/migrate-ts/src/expected-schema.ts @@ -125,9 +125,10 @@ export function buildExpectedSchema( // Pass 1: collect entities + their resolved table names. // Skip: // - abstract objects (e.g., BaseEntity) - // - value objects (no table backing) - // - projections (read-only @kind source with no writable peer — handled by - // the view-diff pipeline, not the table diff) + // - TPH subtypes (their columns fold into the discriminator base's table) + // - any object with no declared/inherited WRITABLE source (#248 — covers + // value objects, projections, sourceless entities, and any sourceless + // custom subtype uniformly; see the rule comment below) const entities: { entity: MetaObject; tableName: string }[] = []; // #208 §7 — entities whose physical name must still resolve for FK TARGETING even // though we emit NO TableDescriptor for them: an @unmanaged table (Flyway/hand- @@ -139,19 +140,20 @@ export function buildExpectedSchema( for (const child of root.children()) { if (child.type !== TYPE_OBJECT) continue; if (child.isAbstract) continue; - if (child.subType === "value") continue; // FR-017 TPH: a subtype shares its discriminator base's single table, so it // emits no table of its own. Its own columns are folded into the base below. if (isTphSubtype(child)) continue; - // ADR-0039: effective children — an entity may inherit its source.rdb via extends. - const hasReadOnlySource = child.children().some( - (c) => c instanceof MetaSource && c.isReadOnly(), - ); + // #248 — persistability derives from source presence, never subtype (loader + // contract: zero sources ⇒ not persisted — metadata validate-source-roles). + // Table iff a WRITABLE source is declared or inherited (ADR-0039 resolving). + // Subsumes the old `subType === "value"` skip (value purity bans sources on + // the resolving view) and the read-only-only projection skip (view pipeline + // owns those); closes the fail-open where a sourceless object (custom + // subtype or plain entity) got a phantom CREATE TABLE + fabricated FK name. const hasWritableSource = child.children().some( (c) => c instanceof MetaSource && c.isWritable(), ); - // Projection: read-only and not write-through. - if (hasReadOnlySource && !hasWritableSource) continue; + if (!hasWritableSource) continue; const tableName = resolveTableName(child); // #208 §7 — an @unmanaged writable (table) source: emit no descriptor, but keep the // entity in entityToTable so an inbound FK resolves the physical name. OWN-source diff --git a/server/typescript/packages/migrate-ts/test/drift/compute-drift-from-actual.test.ts b/server/typescript/packages/migrate-ts/test/drift/compute-drift-from-actual.test.ts index f666c4cfe..7e10d083c 100644 --- a/server/typescript/packages/migrate-ts/test/drift/compute-drift-from-actual.test.ts +++ b/server/typescript/packages/migrate-ts/test/drift/compute-drift-from-actual.test.ts @@ -45,6 +45,7 @@ const META = JSON.stringify({ "object.entity": { name: "Gadget", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.string": { name: "label" } }, { "identity.primary": { name: "pk", "@fields": ["id"] } }, diff --git a/server/typescript/packages/migrate-ts/test/expected-schema-persistability.test.ts b/server/typescript/packages/migrate-ts/test/expected-schema-persistability.test.ts new file mode 100644 index 000000000..d7010803c --- /dev/null +++ b/server/typescript/packages/migrate-ts/test/expected-schema-persistability.test.ts @@ -0,0 +1,195 @@ +// #248 — buildExpectedSchema Pass 1 must derive table persistability from a +// declared/inherited WRITABLE source (metadata's own loader contract: zero +// sources ⇒ not persisted), never from `subType`. Before this fix, ANY object +// with no `source.*` — a plain sourceless entity, or an object of a custom +// provider-registered subtype — fell through the `subType === "value"` compare +// and got a phantom `TableDescriptor` (fabricated physical name, entered the FK- +// target maps). See docs/superpowers/specs/2026-08-01-issue-248-persistability- +// from-source-design.md §3/§3a for the derivation. + +import { describe, test, expect } from "bun:test"; +import { + MetaDataLoader, + InMemoryStringSource, + composeRegistry, + coreProviders, + MetaObject, + TypeId, + TYPE_OBJECT, + TYPE_FIELD, +} from "@metaobjectsdev/metadata"; +import type { MetaData, MetaDataTypeProvider, TypeRegistry } from "@metaobjectsdev/metadata"; +import { buildExpectedSchema } from "../src/expected-schema.js"; + +async function loadJson(json: unknown, registry?: TypeRegistry): Promise { + const result = await new MetaDataLoader(registry !== undefined ? { registry } : undefined).load([ + new InMemoryStringSource(JSON.stringify(json)), + ]); + if (result.errors.length > 0) { + throw new Error(`Loader errors:\n${result.errors.map((e) => e.message).join("\n")}`); + } + return result.root; +} + +describe("buildExpectedSchema — persistability derives from source presence (#248)", () => { + test("1. sourceless entity co-load: only the sourced entity gets a table", async () => { + const root = await loadJson({ + "metadata.root": { + package: "acme::probe", + children: [ + { + "object.entity": { + name: "Order", + children: [ + { "field.long": { name: "id" } }, + { "field.string": { name: "ref" } }, + { "source.rdb": { "@table": "orders" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + { + "object.entity": { + name: "Ghost", + children: [ + { "field.long": { name: "id" } }, + { "field.string": { name: "note" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, + }); + + const snapshot = buildExpectedSchema(root); + expect(snapshot.tables.map((t) => t.name)).toEqual(["orders"]); + }); + + test("2. no fabricated FK target: a reference to a sourceless object yields no FkDescriptor", async () => { + const root = await loadJson({ + "metadata.root": { + package: "acme::probe", + children: [ + { + "object.entity": { + name: "Order", + children: [ + { "field.long": { name: "id" } }, + { "field.string": { name: "ref" } }, + { "field.long": { name: "ghostId" } }, + { "source.rdb": { "@table": "orders" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + { + "identity.reference": { + name: "ref_ghost", + "@fields": ["ghostId"], + "@references": "Ghost", + }, + }, + ], + }, + }, + { + "object.entity": { + name: "Ghost", + children: [ + { "field.long": { name: "id" } }, + { "field.string": { name: "note" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, + }); + + expect(() => { + const snapshot = buildExpectedSchema(root); + const orders = snapshot.tables.find((t) => t.name === "orders"); + expect(orders?.foreignKeys.length).toBe(0); + }).not.toThrow(); + }); + + test("3. custom subtype (the reported scenario): a sourceless object.message gets no table", async () => { + const wireProvider: MetaDataTypeProvider = { + id: "test-wire-messages", + dependencies: ["metaobjects-core-types"], + registerTypes(registry: TypeRegistry): void { + registry.register({ + typeId: new TypeId(TYPE_OBJECT, "message"), + description: "Test-only wire message subtype — no source, never persisted.", + factory: (typeId, name) => new MetaObject(typeId, name), + childRules: [{ childType: TYPE_FIELD, childSubType: "*", childName: "*" }], + attributes: [], + }); + }, + }; + const registry = composeRegistry([...coreProviders, wireProvider]); + + const root = await loadJson( + { + "metadata.root": { + package: "acme::probe", + children: [ + { + "object.entity": { + name: "Order", + children: [ + { "field.long": { name: "id" } }, + { "field.string": { name: "ref" } }, + { "source.rdb": { "@table": "orders" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + { + "object.message": { + name: "WireNote", + children: [ + { "field.string": { name: "topic" } }, + { "field.string": { name: "payload" } }, + ], + }, + }, + ], + }, + }, + registry, + ); + + const snapshot = buildExpectedSchema(root); + expect(snapshot.tables.map((t) => t.name)).toEqual(["orders"]); + }); + + test("4. (pin) inherited writable source via extends still persists", async () => { + const root = await loadJson({ + "metadata.root": { + package: "acme::probe", + children: [ + { + "object.entity": { + name: "Base", + abstract: true, + children: [ + { "field.long": { name: "id" } }, + { "source.rdb": { "@table": "things" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + { + "object.entity": { + name: "Thing", + extends: "Base", + children: [{ "field.string": { name: "label" } }], + }, + }, + ], + }, + }); + + const snapshot = buildExpectedSchema(root); + expect(snapshot.tables.map((t) => t.name)).toEqual(["things"]); + }); +}); diff --git a/server/typescript/packages/migrate-ts/test/fixtures/single-entity.json b/server/typescript/packages/migrate-ts/test/fixtures/single-entity.json index c71189a4a..cc8b48233 100644 --- a/server/typescript/packages/migrate-ts/test/fixtures/single-entity.json +++ b/server/typescript/packages/migrate-ts/test/fixtures/single-entity.json @@ -5,6 +5,7 @@ "object.entity": { "name": "User", "children": [ + { "source.rdb": {} }, { "field.long": { "name": "id" } }, { "field.string": { "name": "email", "@required": true } }, { "field.string": { "name": "firstName" } }, diff --git a/server/typescript/packages/migrate-ts/test/fixtures/trainer-website-entities.json b/server/typescript/packages/migrate-ts/test/fixtures/trainer-website-entities.json index dbbd55dbd..5f52f416b 100644 --- a/server/typescript/packages/migrate-ts/test/fixtures/trainer-website-entities.json +++ b/server/typescript/packages/migrate-ts/test/fixtures/trainer-website-entities.json @@ -5,6 +5,7 @@ "object.entity": { "name": "Subscriber", "children": [ + { "source.rdb": {} }, { "field.long": { "name": "id" } }, { "field.string": { "name": "email", "@required": true } }, { "field.string": { "name": "firstName", "@required": true } }, @@ -21,6 +22,7 @@ "object.entity": { "name": "Program", "children": [ + { "source.rdb": {} }, { "field.long": { "name": "id" } }, { "field.string": { "name": "slug", "@required": true } }, { "field.string": { "name": "title", "@required": true } }, @@ -37,6 +39,7 @@ "object.entity": { "name": "Week", "children": [ + { "source.rdb": {} }, { "field.long": { "name": "id" } }, { "field.long": { "name": "programId", "@required": true } }, { "field.int": { "name": "weekNumber", "@required": true } }, @@ -51,6 +54,7 @@ "object.entity": { "name": "Workout", "children": [ + { "source.rdb": {} }, { "field.long": { "name": "id" } }, { "field.long": { "name": "weekId", "@required": true } }, { "field.int": { "name": "dayNumber", "@required": true } }, @@ -65,6 +69,7 @@ "object.entity": { "name": "Video", "children": [ + { "source.rdb": {} }, { "field.long": { "name": "id" } }, { "field.string": { "name": "bunnyVideoId", "@required": true } }, { "field.string": { "name": "title", "@required": true } }, @@ -86,6 +91,7 @@ "object.entity": { "name": "MediaAsset", "children": [ + { "source.rdb": {} }, { "field.uuid": { "name": "id" } }, { "field.string": { "name": "storageKey", "@required": true } }, { "field.string": { "name": "caption", "@default": "n/a (unknown)" } }, @@ -109,6 +115,7 @@ "object.entity": { "name": "Exercise", "children": [ + { "source.rdb": {} }, { "field.long": { "name": "id" } }, { "field.long": { "name": "workoutId", "@required": true } }, { "field.long": { "name": "videoId" } }, diff --git a/server/typescript/packages/migrate-ts/test/fixtures/two-entities-fk.json b/server/typescript/packages/migrate-ts/test/fixtures/two-entities-fk.json index 2d2ef4653..a89f24511 100644 --- a/server/typescript/packages/migrate-ts/test/fixtures/two-entities-fk.json +++ b/server/typescript/packages/migrate-ts/test/fixtures/two-entities-fk.json @@ -5,6 +5,7 @@ "object.entity": { "name": "Program", "children": [ + { "source.rdb": {} }, { "field.long": { "name": "id" } }, { "field.string": { "name": "slug", "@required": true } }, { "field.string": { "name": "title", "@required": true } }, @@ -17,6 +18,7 @@ "object.entity": { "name": "Week", "children": [ + { "source.rdb": {} }, { "field.long": { "name": "id" } }, { "field.long": { "name": "programId", "@required": true } }, { "field.int": { "name": "weekNumber", "@required": true } }, diff --git a/server/typescript/packages/migrate-ts/test/integration/d1-cascade.test.ts b/server/typescript/packages/migrate-ts/test/integration/d1-cascade.test.ts index 049264e43..f1ef049eb 100644 --- a/server/typescript/packages/migrate-ts/test/integration/d1-cascade.test.ts +++ b/server/typescript/packages/migrate-ts/test/integration/d1-cascade.test.ts @@ -112,7 +112,7 @@ async function insertIsRejected(stmt: string): Promise { // --- metadata + pipeline helpers ------------------------------------------- function entity(name: string, children: unknown[]): unknown { - return { "object.entity": { name, children } }; + return { "object.entity": { name, children: [{ "source.rdb": {} }, ...children] } }; } function rootMeta(children: unknown[]): string { return JSON.stringify({ "metadata.root": { package: "acme", children } }); diff --git a/server/typescript/packages/migrate-ts/test/integration/drift-sqlite.test.ts b/server/typescript/packages/migrate-ts/test/integration/drift-sqlite.test.ts index 1511eba2a..124483210 100644 --- a/server/typescript/packages/migrate-ts/test/integration/drift-sqlite.test.ts +++ b/server/typescript/packages/migrate-ts/test/integration/drift-sqlite.test.ts @@ -42,6 +42,7 @@ const META = JSON.stringify({ "object.entity": { name: "Widget", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.string": { name: "name" } }, { "field.string": { name: "color" } }, diff --git a/server/typescript/packages/migrate-ts/test/integration/sqlite-autoset-default.test.ts b/server/typescript/packages/migrate-ts/test/integration/sqlite-autoset-default.test.ts index 2de828031..c530de2fb 100644 --- a/server/typescript/packages/migrate-ts/test/integration/sqlite-autoset-default.test.ts +++ b/server/typescript/packages/migrate-ts/test/integration/sqlite-autoset-default.test.ts @@ -37,6 +37,7 @@ const META = JSON.stringify({ "object.entity": { name: "Event", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.string": { name: "title", "@required": true } }, { "field.timestamp": { name: "createdAt", "@autoSet": "onCreate", "@required": true } }, diff --git a/server/typescript/packages/migrate-ts/test/integration/sqlite-check-evolution.test.ts b/server/typescript/packages/migrate-ts/test/integration/sqlite-check-evolution.test.ts index a3cc989ac..d3566953e 100644 --- a/server/typescript/packages/migrate-ts/test/integration/sqlite-check-evolution.test.ts +++ b/server/typescript/packages/migrate-ts/test/integration/sqlite-check-evolution.test.ts @@ -37,6 +37,7 @@ function metaWithStatuses(values: string[]): string { "object.entity": { name: "Ticket", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.enum": { name: "status", "@values": values, "@required": true } }, { "identity.primary": { name: "id", "@fields": ["id"], "@generation": "increment" } }, diff --git a/server/typescript/packages/migrate-ts/test/integration/sqlite-default-semantics.test.ts b/server/typescript/packages/migrate-ts/test/integration/sqlite-default-semantics.test.ts index 936828e68..b46923a5f 100644 --- a/server/typescript/packages/migrate-ts/test/integration/sqlite-default-semantics.test.ts +++ b/server/typescript/packages/migrate-ts/test/integration/sqlite-default-semantics.test.ts @@ -44,6 +44,7 @@ const META = JSON.stringify({ "object.entity": { name: "Photo", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.boolean": { name: "isPrimary", "@default": false, "@required": true } }, { "field.boolean": { name: "isPublished", "@default": true, "@required": true } }, diff --git a/server/typescript/packages/migrate-ts/test/integration/sqlite-fk-convergence.test.ts b/server/typescript/packages/migrate-ts/test/integration/sqlite-fk-convergence.test.ts index e148163bb..436a08aa1 100644 --- a/server/typescript/packages/migrate-ts/test/integration/sqlite-fk-convergence.test.ts +++ b/server/typescript/packages/migrate-ts/test/integration/sqlite-fk-convergence.test.ts @@ -35,6 +35,7 @@ const META = JSON.stringify({ "object.entity": { name: "Region", children: [ + { "source.rdb": {} }, { "field.string": { name: "code", "@required": true } }, { "field.string": { name: "country", "@required": true } }, // Composite primary key — the referenced side of the composite FK. @@ -46,6 +47,7 @@ const META = JSON.stringify({ "object.entity": { name: "City", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.string": { name: "regionCode", "@required": true } }, { "field.string": { name: "regionCountry", "@required": true } }, @@ -66,6 +68,7 @@ const META = JSON.stringify({ "object.entity": { name: "Landmark", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.long": { name: "cityId", "@required": true } }, { "identity.primary": { name: "pk", "@fields": ["id"], "@generation": "increment" } }, diff --git a/server/typescript/packages/migrate-ts/test/integration/sqlite-index-escapes.test.ts b/server/typescript/packages/migrate-ts/test/integration/sqlite-index-escapes.test.ts index 7ffe24d98..da257afe5 100644 --- a/server/typescript/packages/migrate-ts/test/integration/sqlite-index-escapes.test.ts +++ b/server/typescript/packages/migrate-ts/test/integration/sqlite-index-escapes.test.ts @@ -43,6 +43,7 @@ const META = JSON.stringify({ "object.entity": { name: "Account", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.string": { name: "email", "@required": true } }, { "field.string": { name: "tags" } }, diff --git a/server/typescript/packages/migrate-ts/test/integration/sqlite-quoted-default.test.ts b/server/typescript/packages/migrate-ts/test/integration/sqlite-quoted-default.test.ts index ae79c3aeb..6d227abe8 100644 --- a/server/typescript/packages/migrate-ts/test/integration/sqlite-quoted-default.test.ts +++ b/server/typescript/packages/migrate-ts/test/integration/sqlite-quoted-default.test.ts @@ -31,6 +31,7 @@ const META = JSON.stringify({ "object.entity": { name: "Quip", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, // The quote-bearing literal default under test. { "field.string": { name: "greeting", "@default": "don't panic", "@required": true } }, diff --git a/server/typescript/packages/migrate-ts/test/integration/sqlite-recreate-roundtrip.test.ts b/server/typescript/packages/migrate-ts/test/integration/sqlite-recreate-roundtrip.test.ts index 9e1dfb9f1..dbdc94d17 100644 --- a/server/typescript/packages/migrate-ts/test/integration/sqlite-recreate-roundtrip.test.ts +++ b/server/typescript/packages/migrate-ts/test/integration/sqlite-recreate-roundtrip.test.ts @@ -58,6 +58,7 @@ describe("SQLite recreate-and-copy — data preservation", () => { "object.entity": { name: "Item", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.string": { name: "name", "@required": true } }, { "field.string": { name: "tag", "@default": "untagged" } }, @@ -125,6 +126,7 @@ describe("SQLite recreate-and-copy — data preservation", () => { "object.entity": { name: "Person", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.string": { name: "firstName", "@required": true } }, { "field.string": { name: "tag", "@default": "v1" } }, diff --git a/server/typescript/packages/migrate-ts/test/unit/expected-schema-isarray-scalars.test.ts b/server/typescript/packages/migrate-ts/test/unit/expected-schema-isarray-scalars.test.ts index 581ecf4c3..f1b4907f3 100644 --- a/server/typescript/packages/migrate-ts/test/unit/expected-schema-isarray-scalars.test.ts +++ b/server/typescript/packages/migrate-ts/test/unit/expected-schema-isarray-scalars.test.ts @@ -33,6 +33,7 @@ const META = JSON.stringify({ "object.entity": { name: "Sample", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.string": { name: "tags", isArray: true } }, { "field.string": { name: "codes", isArray: true, "@maxLength": 50 } }, diff --git a/server/typescript/packages/migrate-ts/test/unit/expected-schema-lenient-inet.test.ts b/server/typescript/packages/migrate-ts/test/unit/expected-schema-lenient-inet.test.ts index 80b11010b..d7e8c8754 100644 --- a/server/typescript/packages/migrate-ts/test/unit/expected-schema-lenient-inet.test.ts +++ b/server/typescript/packages/migrate-ts/test/unit/expected-schema-lenient-inet.test.ts @@ -16,6 +16,7 @@ const META = JSON.stringify({ "object.entity": { name: "Endpoint", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.inet": { name: "strictIp" } }, { "field.inet": { name: "lenientIp", "@lenient": true } }, diff --git a/server/typescript/packages/migrate-ts/test/unit/expected-schema.test.ts b/server/typescript/packages/migrate-ts/test/unit/expected-schema.test.ts index 4b52f8fef..6a772786c 100644 --- a/server/typescript/packages/migrate-ts/test/unit/expected-schema.test.ts +++ b/server/typescript/packages/migrate-ts/test/unit/expected-schema.test.ts @@ -112,6 +112,7 @@ describe("buildExpectedSchema — @autoSet timestamp default", () => { }; const doc = { "metadata.root": { package: "acme", children: [ { "object.entity": { name: "Event", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "identity.primary": { "name": "id", "@fields": "id" } }, stampField, @@ -158,6 +159,7 @@ describe("buildExpectedSchema — native array columns (derived from isArray)", async function arrayCols() { const doc = { "metadata.root": { package: "acme", children: [ { "object.entity": { name: "Bag", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "identity.primary": { "name": "id", "@fields": "id" } }, { "field.uuid": { name: "memberIds", isArray: true } }, @@ -180,6 +182,7 @@ describe("buildExpectedSchema — native array columns (derived from isArray)", test("field.uuid scalar → uuid; field.string scalar (no maxLength) → text", async () => { const doc = { "metadata.root": { package: "acme", children: [ { "object.entity": { name: "Scalars", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "identity.primary": { "name": "id", "@fields": "id" } }, { "field.uuid": { name: "ownerId" } }, diff --git a/server/typescript/packages/migrate-ts/test/unit/referential-actions-nullable.test.ts b/server/typescript/packages/migrate-ts/test/unit/referential-actions-nullable.test.ts index 9f298cd2c..62ab0e256 100644 --- a/server/typescript/packages/migrate-ts/test/unit/referential-actions-nullable.test.ts +++ b/server/typescript/packages/migrate-ts/test/unit/referential-actions-nullable.test.ts @@ -44,6 +44,7 @@ function makeDoc(options: { "object.entity": { name: "Program", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "identity.primary": { "name": "id", "@fields": "id" } }, ], @@ -53,6 +54,7 @@ function makeDoc(options: { "object.entity": { name: "Week", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.long": { diff --git a/server/typescript/packages/migrate-ts/test/unit/referential-actions.test.ts b/server/typescript/packages/migrate-ts/test/unit/referential-actions.test.ts index ae422a1be..18c13023f 100644 --- a/server/typescript/packages/migrate-ts/test/unit/referential-actions.test.ts +++ b/server/typescript/packages/migrate-ts/test/unit/referential-actions.test.ts @@ -24,10 +24,12 @@ async function loadDoc(doc: unknown) { function weekDoc(rel: Record | undefined) { return { "metadata.root": { package: "acme", children: [ { "object.entity": { name: "Program", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "identity.primary": { "name": "id", "@fields": "id" } }, ] } }, { "object.entity": { name: "Week", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.long": { name: "programId" } }, ...(rel ? [rel] : []), @@ -155,10 +157,12 @@ describe("end-to-end FK actions in emitted DDL", () => { function weekDocRef(refAttrs: Record, rel?: Record) { return { "metadata.root": { package: "acme", children: [ { "object.entity": { name: "Program", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "identity.primary": { "name": "id", "@fields": "id" } }, ] } }, { "object.entity": { name: "Week", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.long": { name: "programId" } }, ...(rel ? [rel] : []), From cbbd299af73839938eea74936f803cce6cb3e549 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 1 Aug 2026 17:12:01 -0400 Subject: [PATCH 4/7] fix(#248): codegen-ts gates queries/routes/api-model on source presence, not subtype MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit R2 (design spec §3): an object participates in DB-bound codegen iff it declares/inherits a source.rdb child, never by subtype. New hasAnyRdbSource (source-detect.ts) — sibling of the entity-file tier's hasWritableRdbSource, but ANY source.rdb kind (writable or read-only), so a read-only projection stays queryable. queries-file.ts, routes-file.ts, routes-file-hono.ts, and api-model.ts's isQueryable now gate on it instead of subType !== OBJECT_SUBTYPE_VALUE (queries) or no gate at all (routes/hono — the latter was a pre-existing fail-open: a plain object.value got a broken routes file importing exports that were never generated). Adapted three pre-existing tests/fixtures that only worked because of this bug (stashed-baseline-verified, per Task 1's carry-forward): - codegen-ts-tanstack multi-grid-entity.json: Program had no source.rdb at all; added a bare one (byte-identical physical naming) so the full-pipeline integration test's queries/routes files keep existing. - codegen-ts generators/queries-file.test.ts: entityWithPk() helper built entities with no source; added a bare source.rdb so "keeps every object.entity" fixtures stay queryable, and added a new case pinning the actual new behavior (a sourceless object.entity is excluded, not just object.value). - codegen-ts reference-byte-identical.test.ts: carved out one precisely identified, tracked, temporary divergence (Triple.routes.ts in cross-package-vo.json) — the reference (scaffold-and-own) generators get the same fix in the very next commit (#248 Task 3); until then the built-in/reference composers diverge by exactly the DB-bound artifact this fix correctly stops emitting for a sourceless value object. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01HLoJkFSyoticveo5ehMUAr --- .../test/fixtures/multi-grid-entity.json | 1 + .../codegen-ts/src/generators/api-model.ts | 14 +- .../codegen-ts/src/generators/queries-file.ts | 19 +- .../src/generators/routes-file-hono.ts | 8 +- .../codegen-ts/src/generators/routes-file.ts | 7 +- .../packages/codegen-ts/src/index.ts | 2 +- .../packages/codegen-ts/src/source-detect.ts | 20 ++ .../test/generators/queries-file.test.ts | 31 ++- .../test/reference-byte-identical.test.ts | 29 ++- .../test/sourceless-objects.test.ts | 197 ++++++++++++++++++ 10 files changed, 308 insertions(+), 20 deletions(-) create mode 100644 server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts diff --git a/server/typescript/packages/codegen-ts-tanstack/test/fixtures/multi-grid-entity.json b/server/typescript/packages/codegen-ts-tanstack/test/fixtures/multi-grid-entity.json index 4f0f1f5af..de2fdaa9c 100644 --- a/server/typescript/packages/codegen-ts-tanstack/test/fixtures/multi-grid-entity.json +++ b/server/typescript/packages/codegen-ts-tanstack/test/fixtures/multi-grid-entity.json @@ -5,6 +5,7 @@ "object.entity": { "name": "Program", "children": [ + { "source.rdb": {} }, { "field.long": { "name": "id", diff --git a/server/typescript/packages/codegen-ts/src/generators/api-model.ts b/server/typescript/packages/codegen-ts/src/generators/api-model.ts index 0e5adb2c1..796c8a79b 100644 --- a/server/typescript/packages/codegen-ts/src/generators/api-model.ts +++ b/server/typescript/packages/codegen-ts/src/generators/api-model.ts @@ -110,6 +110,7 @@ import { getPkInfo } from "../templates/queries.js"; import { isTphSubtype } from "../templates/zod-validators.js"; import { isTphDiscriminatorBase } from "../templates/tph-discriminator.js"; import { isCallableEntity } from "../templates/callable-file.js"; +import { hasAnyRdbSource } from "../source-detect.js"; import { CODEGEN_ATTR_EMIT_ROUTES } from "../constants.js"; import { resourcePath } from "../templates/entity-constants.js"; import { isProjection } from "../projection/projection-detector.js"; @@ -334,10 +335,13 @@ function stripTs(path: string): string { // --------------------------------------------------------------------------- /** Mirror of the queries generator's filter (queries-file.ts `skipNonQueryable` - * = `subType !== OBJECT_SUBTYPE_VALUE && !isTphSubtype`). A queryable entity is - * any non-value, non-TPH-subtype object: - * • Value objects have no primary identity → the queries/routes/validation - * generators emit no CRUD for them. + * = `hasAnyRdbSource(e) && !isTphSubtype(e)`, #248 R2). A queryable object is + * any source-backed, non-TPH-subtype object: + * • An object with no declared/inherited source.rdb (of ANY kind) isn't + * backed by any store → the queries/routes/validation generators emit no + * CRUD for it. Value objects are subsumed here: value purity (ADR-0028) + * bans sources on values, loader-enforced, so no loadable value ever has + * hasAnyRdbSource === true. * • TPH subtypes (@discriminatorValue under a @discriminator base) emit no * standalone queries/routes file — their surface lives in the discriminator * BASE's polymorphic file (routes-file.ts:27 + queries-file.ts:21-22). @@ -345,7 +349,7 @@ function stripTs(path: string): string { * per-subtype polymorphic helpers + subpaths are a documented deferral — see * the module header.) */ function isQueryable(obj: MetaObject): boolean { - return obj.subType !== OBJECT_SUBTYPE_VALUE && !isTphSubtype(obj); + return hasAnyRdbSource(obj) && !isTphSubtype(obj); } /** Whether the routes generator emits REST routes for this entity. It filters diff --git a/server/typescript/packages/codegen-ts/src/generators/queries-file.ts b/server/typescript/packages/codegen-ts/src/generators/queries-file.ts index 000262929..1275d0368 100644 --- a/server/typescript/packages/codegen-ts/src/generators/queries-file.ts +++ b/server/typescript/packages/codegen-ts/src/generators/queries-file.ts @@ -1,7 +1,8 @@ -import { OBJECT_SUBTYPE_VALUE, type MetaObject } from "@metaobjectsdev/metadata"; +import type { MetaObject } from "@metaobjectsdev/metadata"; import { perEntity, type Generator, type GeneratorFactory } from "../generator.js"; import { renderQueriesFile } from "../templates/queries-file.js"; import { isTphSubtype } from "../templates/zod-validators.js"; +import { hasAnyRdbSource } from "../source-detect.js"; import { formatTs } from "../format.js"; import { entityOutputPath } from "../import-path.js"; @@ -10,16 +11,20 @@ export interface QueriesFileOpts { target?: string; } -// object.value records have no primary identity, so the rendered queries module -// emits findById/updateById/deleteById against a non-existent column. Skipping -// value subtypes is unconditional — the user-supplied filter (if any) is applied -// on top via boolean AND. +// #248 R2: persistability derives from declared/inherited source, never +// subtype. An object with no source.rdb (of ANY kind) isn't backed by any +// store — the rendered queries module would emit findById/updateById/deleteById +// against Drizzle table/schema exports the entity-file generator never emits +// for it. object.value is subsumed here too: value purity (ADR-0028) bans +// sources on values, loader-enforced (ERR_SUBTYPE_RULE_VIOLATION), so no +// loadable value ever has hasAnyRdbSource === true — no separate value check +// needed. Skipping non-source objects is unconditional — the user-supplied +// filter (if any) is applied on top via boolean AND. // // FR-017 Tier 2: TPH subtypes are ALSO skipped — they emit no standalone // queries file. Their per-subtype CRUD helpers live in the discriminator // base's queries file (which targets the single shared table). -const skipNonQueryable = (e: MetaObject): boolean => - e.subType !== OBJECT_SUBTYPE_VALUE && !isTphSubtype(e); +const skipNonQueryable = (e: MetaObject): boolean => hasAnyRdbSource(e) && !isTphSubtype(e); export const queriesFile = function queriesFile(opts?: QueriesFileOpts): Generator { const userFilter = opts?.filter; diff --git a/server/typescript/packages/codegen-ts/src/generators/routes-file-hono.ts b/server/typescript/packages/codegen-ts/src/generators/routes-file-hono.ts index bf7e4f3b0..133ba9a6d 100644 --- a/server/typescript/packages/codegen-ts/src/generators/routes-file-hono.ts +++ b/server/typescript/packages/codegen-ts/src/generators/routes-file-hono.ts @@ -1,6 +1,7 @@ import type { MetaObject } from "@metaobjectsdev/metadata"; import { perEntity, type Generator, type GeneratorFactory } from "../generator.js"; import { renderRoutesFileHono } from "../templates/routes-file-hono.js"; +import { hasAnyRdbSource } from "../source-detect.js"; import { formatTs } from "../format.js"; import { entityOutputPath } from "../import-path.js"; import { CODEGEN_ATTR_EMIT_ROUTES } from "../constants.js"; @@ -22,6 +23,10 @@ export interface RoutesFileHonoOpts { * * Per-entity opt-out via `@emitRoutes: false` is honored. If the user * supplies their own filter, both must pass (AND). + * + * #248 R2: an object with no declared/inherited source.rdb (of ANY kind) isn't + * backed by any store — gated by `hasAnyRdbSource` (does NOT add TPH handling; + * that gap is pre-existing and out of scope here). */ export const routesFileHono = function routesFileHono(opts?: RoutesFileHonoOpts): Generator { const userFilter = opts?.filter ?? (() => true); @@ -31,7 +36,8 @@ export const routesFileHono = function routesFileHono(opts?: RoutesFileHonoOpts) // `ctx.config.includeHonoRoutes` and api-docs auto-documents the Hono surface. emitsHonoRoutes: true, // ADR-0039: resolving — a concrete entity may inherit @emitRoutes via extends. - filter: (e: MetaObject) => e.attr(CODEGEN_ATTR_EMIT_ROUTES) !== false && userFilter(e), + filter: (e: MetaObject) => + e.attr(CODEGEN_ATTR_EMIT_ROUTES) !== false && hasAnyRdbSource(e) && userFilter(e), generate: perEntity(async (entity, ctx) => { if (!ctx.renderContext) { throw new Error("routes-file-hono: renderContext is required (provided by runGen)"); diff --git a/server/typescript/packages/codegen-ts/src/generators/routes-file.ts b/server/typescript/packages/codegen-ts/src/generators/routes-file.ts index 3b6ff0918..7a69c84ec 100644 --- a/server/typescript/packages/codegen-ts/src/generators/routes-file.ts +++ b/server/typescript/packages/codegen-ts/src/generators/routes-file.ts @@ -2,6 +2,7 @@ import type { MetaObject } from "@metaobjectsdev/metadata"; import { perEntity, type Generator, type GeneratorFactory } from "../generator.js"; import { renderRoutesFile } from "../templates/routes-file.js"; import { isTphSubtype } from "../templates/zod-validators.js"; +import { hasAnyRdbSource } from "../source-detect.js"; import { formatTs } from "../format.js"; import { entityOutputPath } from "../import-path.js"; import { CODEGEN_ATTR_EMIT_ROUTES } from "../constants.js"; @@ -15,6 +16,10 @@ export interface RoutesFileOpts { * Per-entity opt-out via `@emitRoutes: false` is honored. If the user supplies * their own filter, both must pass (AND). * + * #248 R2: an object with no declared/inherited source.rdb (of ANY kind) isn't + * backed by any store — routes against it would import Drizzle table/allowlist + * exports the entity-file generator never emits. Gated by `hasAnyRdbSource`. + * * FR-017 Tier 2: TPH subtypes get no standalone routes file — their per-subtype * route set lives in the discriminator base's routes file. */ @@ -25,7 +30,7 @@ export const routesFile = function routesFile(opts?: RoutesFileOpts): Generator // Always set: AND-composes metadata opt-out with optional user filter. filter: (e: MetaObject) => // ADR-0039: resolving — a concrete entity may inherit its @emit* opt-out flag via extends. - e.attr(CODEGEN_ATTR_EMIT_ROUTES) !== false && !isTphSubtype(e) && userFilter(e), + e.attr(CODEGEN_ATTR_EMIT_ROUTES) !== false && hasAnyRdbSource(e) && !isTphSubtype(e) && userFilter(e), generate: perEntity(async (entity, ctx) => { if (!ctx.renderContext) { throw new Error("routes-file: renderContext is required (provided by runGen)"); diff --git a/server/typescript/packages/codegen-ts/src/index.ts b/server/typescript/packages/codegen-ts/src/index.ts index 163d7900c..598285877 100644 --- a/server/typescript/packages/codegen-ts/src/index.ts +++ b/server/typescript/packages/codegen-ts/src/index.ts @@ -94,7 +94,7 @@ export { isTphSubtype, tphDiscriminatorPin } from "./templates/zod-validators.js // package-internal relative path. These are the assembly pieces the built-in // entity/queries composers use; the reference templates relocate that assembly. export { renderTphDiscriminatorUnion } from "./templates/tph-discriminator.js"; -export { hasWritableRdbSource } from "./source-detect.js"; +export { hasWritableRdbSource, hasAnyRdbSource } from "./source-detect.js"; export { renderSharedEnumsFile, SHARED_ENUMS_BASENAME } from "./templates/enums-file.js"; // ADR-0034 scaffold-and-own — reader for the copyable reference generators in diff --git a/server/typescript/packages/codegen-ts/src/source-detect.ts b/server/typescript/packages/codegen-ts/src/source-detect.ts index 9074b2b55..3081acfa5 100644 --- a/server/typescript/packages/codegen-ts/src/source-detect.ts +++ b/server/typescript/packages/codegen-ts/src/source-detect.ts @@ -29,3 +29,23 @@ export function hasWritableRdbSource(entity: MetaObject): boolean { } return false; } + +/** + * True when the object declares (or inherits via extends — ADR-0039 resolving) + * at least one source.rdb child of ANY kind (writable OR read-only). Zero + * sources means "not backed by any store" (loader contract, + * validate-source-roles: zero sources is allowed, means not persisted) — the + * DB-artifact tier (queries/routes/api-model) must emit nothing for it, exactly + * as the table tier already refuses to emit a Drizzle table for it + * (hasWritableRdbSource, above — this is its any-kind sibling). + */ +export function hasAnyRdbSource(entity: MetaObject): boolean { + // ADR-0039: resolving — same rationale as hasWritableRdbSource. + for (const child of entity.children()) { + if (child.type !== TYPE_SOURCE) continue; + if (child.subType !== SOURCE_SUBTYPE_RDB) continue; + if (!(child instanceof MetaSource)) continue; + return true; + } + return false; +} diff --git a/server/typescript/packages/codegen-ts/test/generators/queries-file.test.ts b/server/typescript/packages/codegen-ts/test/generators/queries-file.test.ts index 4da24dcb2..5a241d9d5 100644 --- a/server/typescript/packages/codegen-ts/test/generators/queries-file.test.ts +++ b/server/typescript/packages/codegen-ts/test/generators/queries-file.test.ts @@ -13,7 +13,26 @@ async function loadRoot(children: unknown[]) { return res.root; } +// #248 R2: persistability derives from declared source, never subtype — a +// "normal, queryable entity" fixture must declare a source.rdb (any kind; bare +// is enough, physical naming is unaffected — see source-detect.ts). function entityWithPk(name: string) { + return { + "object.entity": { + name, + children: [ + { "source.rdb": {} }, + { "field.string": { name: "id" } }, + { "identity.primary": { "name": "id", "@fields": "id" } }, + ], + }, + }; +} + +// A sourceless object.entity — loads clean (zero sources = "not persisted", +// per validate-source-roles), but must NOT be queryable (the actual #248 bug: +// a queries file importing Drizzle table exports that were never generated). +function sourcelessEntityWithPk(name: string) { return { "object.entity": { name, @@ -48,13 +67,23 @@ describe("queriesFile() factory", () => { expect(filtered.map((e) => e.name)).toEqual(["Post"]); }); - test("default filter keeps every object.entity", async () => { + test("default filter keeps every source-backed object.entity", async () => { const root = await loadRoot([entityWithPk("Post"), entityWithPk("Comment"), valueShape("Stamp")]); const gen = queriesFile(); const filtered = root.objects().filter(gen.filter!); expect(filtered.map((e) => e.name).sort()).toEqual(["Comment", "Post"]); }); + // #248 R2: persistability derives from source presence, not subtype — a + // sourceless object.entity is excluded exactly like a value object, even + // though it declares a primary identity. + test("default filter excludes a sourceless object.entity (not just object.value)", async () => { + const root = await loadRoot([entityWithPk("Post"), sourcelessEntityWithPk("Ghost"), valueShape("Stamp")]); + const gen = queriesFile(); + const filtered = root.objects().filter(gen.filter!); + expect(filtered.map((e) => e.name)).toEqual(["Post"]); + }); + test("user-supplied filter is composed with the value-skip default via AND", async () => { const root = await loadRoot([entityWithPk("Post"), entityWithPk("Comment"), valueShape("Stamp")]); const gen = queriesFile({ filter: (e) => e.name === "Comment" }); diff --git a/server/typescript/packages/codegen-ts/test/reference-byte-identical.test.ts b/server/typescript/packages/codegen-ts/test/reference-byte-identical.test.ts index 6810b7c77..5471f8eec 100644 --- a/server/typescript/packages/codegen-ts/test/reference-byte-identical.test.ts +++ b/server/typescript/packages/codegen-ts/test/reference-byte-identical.test.ts @@ -43,6 +43,24 @@ async function gen(dir: string, generators: ReturnType[], return out; } +// #248 R2 (Task 2 of 2 for the codegen-ts half): the built-in engine generators +// (src/generators/*) now gate queries/routes on hasAnyRdbSource — a sourceless +// object no longer gets a broken queries/routes file. The scaffold-and-own +// reference templates (src/reference/*) get the SAME fix in the very next +// commit (Task 3); until then they still use the old subtype-based filter, so +// they diverge from the now-fixed built-ins by exactly the DB-bound artifacts +// a sourceless/value object should never have gotten in the first place. Known, +// tracked, temporary — remove KNOWN_PENDING_DIVERGENCE once Task 3 lands (it +// makes src/reference/{queries,routes}.ts derive from hasAnyRdbSource too). +const KNOWN_PENDING_DIVERGENCE: Record = { + // cross-package-vo.json's "Triple" is an object.value (no source, value + // purity) — reference/routes.ts has no value/source skip at all (design + // spec §4, "reference/routes.ts:40-44 same" REDERIVE row) and still emits + // Triple.routes.ts; the built-in routes-file.ts (fixed here) correctly does + // not. + "cross-package-vo.json": ["Triple.routes.ts"], +}; + describe("ADR-0034 — reference templates are byte-identical to built-ins", () => { for (const fixture of FIXTURES) { test(fixture, async () => { @@ -55,10 +73,13 @@ describe("ADR-0034 — reference templates are byte-identical to built-ins", () try { const a = await gen(aDir, [builtinEntity(), builtinQueries(), builtinRoutes(), builtinBarrel()], result.root); const b = await gen(bDir, [refEntity(), refQueries(), refRoutes(), refBarrel()], result.root); - // same set of files - expect(Object.keys(b).sort()).toEqual(Object.keys(a).sort()); - // byte-identical contents - for (const k of Object.keys(a)) { + const pending = new Set(KNOWN_PENDING_DIVERGENCE[fixture] ?? []); + const aKeys = Object.keys(a).filter((k) => !pending.has(k)).sort(); + const bKeys = Object.keys(b).filter((k) => !pending.has(k)).sort(); + // same set of files (excluding the known-pending Task 3 divergence) + expect(bKeys).toEqual(aKeys); + // byte-identical contents for every file both sides agree on emitting + for (const k of aKeys) { expect(`${k}:\n${b[k]}`).toBe(`${k}:\n${a[k]}`); } } finally { diff --git a/server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts b/server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts new file mode 100644 index 000000000..13d0c0a94 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts @@ -0,0 +1,197 @@ +// #248 R2 — DB-artifact tier (queries/routes/hono) must gate on SOURCE presence, +// never subtype. Before this fix, queries-file/routes-file/routes-file-hono +// filtered on `subType !== OBJECT_SUBTYPE_VALUE` (queries) or nothing at all +// (routes/hono) — so a sourceless entity ("Ghost") got a queries/routes file +// importing Drizzle table/allowlist exports the entity-file generator (correctly) +// never emits (hasWritableRdbSource gates THAT tier already), and a plain +// object.value ("Money") got a broken routes file with no persistence layer at +// all. Both are TS2305/TS2724-class breakage in generated output. +// +// The fix: `hasAnyRdbSource` (any source.rdb, not just writable) — the queries/ +// routes/hono/api-model tier's sibling of the entity-file tier's +// `hasWritableRdbSource` — gates DB-bound artifact emission. A read-only-source +// projection still gets queries/routes (the existing read-only path); a +// zero-source object of ANY subtype gets neither. + +import { describe, test, expect, beforeEach, afterEach } from "bun:test"; +import { mkdtempSync, rmSync, readFileSync, readdirSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { runGen, defineConfig } from "../src/index.js"; +import { entityFile, queriesFile, routesFile, routesFileHono } from "../src/generators/index.js"; + +let tmp: string; +beforeEach(() => { + tmp = mkdtempSync(join(tmpdir(), "codegen-sourceless-")); +}); +afterEach(() => { + rmSync(tmp, { recursive: true, force: true }); +}); + +async function loadRoot(children: unknown[]) { + const result = await new MetaDataLoader().load([ + new InMemoryStringSource(JSON.stringify({ "metadata.root": { package: "acme::probe", children } })), + ]); + if (result.errors.length > 0) { + throw new Error(`Loader errors:\n${result.errors.map((e) => e.message).join("\n")}`); + } + return result.root; +} + +// Order — a normal, sourced entity (the well-formed baseline, Task 1's fixture shape). +const ORDER = { + "object.entity": { + name: "Order", + children: [ + { "source.rdb": { "@table": "orders" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "reference" } }, + { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment" } }, + ], + }, +}; + +// Money — a plain object.value (two fields, no identity, no source — value +// purity, ADR-0028). Before the fix, routes-file.ts had NO value skip at all. +const MONEY = { + "object.value": { + name: "Money", + children: [ + { "field.long": { name: "cents" } }, + { "field.string": { name: "currency" } }, + ], + }, +}; + +// Ghost — an entity with identity but NO source.* child at all. Loads clean +// (validate-source-roles: zero sources = "not persisted", not an error). +const GHOST = { + "object.entity": { + name: "Ghost", + children: [ + { "field.long": { name: "id" } }, + { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment" } }, + ], + }, +}; + +function genConfig(outDir: string) { + return defineConfig({ + outDir, + extStyle: "none", + dbImport: "~/server/db", + dialect: "postgres", + generators: [entityFile(), queriesFile(), routesFile(), routesFileHono()], + }); +} + +describe("#248 R2 — sourceless objects get no DB-bound artifacts", () => { + test("Order (sourced) gets entity+queries+routes+hono; Money (value) and Ghost (sourceless entity) get neither", async () => { + const root = await loadRoot([ORDER, MONEY, GHOST]); + const out = await runGen({ config: genConfig(tmp), metadata: root }); + expect(out.warnings).toEqual([]); + + // runGen reports files.path as absolute (outDir-joined) paths. + const paths = new Set(out.files.map((f) => f.path)); + const at = (name: string) => join(tmp, name); + + // INCLUDES — the sourced entity gets the full DB-bound surface; the + // sourceless/value objects still get their shape-only entity file. + expect(paths.has(at("Order.ts"))).toBe(true); + expect(paths.has(at("Order.queries.ts"))).toBe(true); + expect(paths.has(at("Order.routes.ts"))).toBe(true); + expect(paths.has(at("Order.routes.hono.ts"))).toBe(true); + expect(paths.has(at("Money.ts"))).toBe(true); + expect(paths.has(at("Ghost.ts"))).toBe(true); + + // EXCLUDES — no DB-bound artifact for a plain value or a sourceless entity; + // this is the actual bug (queries/routes previously emitted here, importing + // Drizzle table/allowlist exports that were never generated). + expect(paths.has(at("Ghost.queries.ts"))).toBe(false); + expect(paths.has(at("Ghost.routes.ts"))).toBe(false); + expect(paths.has(at("Ghost.routes.hono.ts"))).toBe(false); + expect(paths.has(at("Money.queries.ts"))).toBe(false); + expect(paths.has(at("Money.routes.ts"))).toBe(false); + expect(paths.has(at("Money.routes.hono.ts"))).toBe(false); + + // Nothing beyond the on-disk files reported by runGen either. + const onDisk = new Set(readdirSync(tmp)); + expect(onDisk.has("Ghost.queries.ts")).toBe(false); + expect(onDisk.has("Money.routes.ts")).toBe(false); + }); + + test("content no-churn: Order.* is byte-identical whether or not Money/Ghost are co-loaded", async () => { + const mixedRoot = await loadRoot([ORDER, MONEY, GHOST]); + const mixedOut = tmp; + await runGen({ config: genConfig(mixedOut), metadata: mixedRoot }); + + const soloDir = mkdtempSync(join(tmpdir(), "codegen-sourceless-solo-")); + try { + const soloRoot = await loadRoot([ORDER]); + await runGen({ config: genConfig(soloDir), metadata: soloRoot }); + + for (const name of ["Order.ts", "Order.queries.ts", "Order.routes.ts"]) { + const mixed = readFileSync(join(mixedOut, name), "utf-8"); + const solo = readFileSync(join(soloDir, name), "utf-8"); + expect(mixed).toBe(solo); + } + } finally { + rmSync(soloDir, { recursive: true, force: true }); + } + }); + + test("a read-only-source projection is still queryable (queries+routes emitted)", async () => { + // Same Program/Week/ProgramSummary shape as + // test/projection/queries-file.test.ts — a writable-source base entity plus + // a read-only (@kind: view) projection derived from it via origin.aggregate. + const root = await loadRoot([ + { + "object.entity": { + name: "Program", + children: [ + { "source.rdb": { "@table": "programs" } }, + { "field.int": { name: "id" } }, + { "field.string": { name: "title" } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + { "relationship.association": { name: "weeks", "@objectRef": "Week", "@cardinality": "many" } }, + ], + }, + }, + { + "object.entity": { + name: "Week", + children: [ + { "source.rdb": { "@table": "weeks" } }, + { "field.int": { name: "id" } }, + { "field.int": { name: "programId" } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + { "identity.reference": { name: "ref_program", "@fields": "programId", "@references": "Program" } }, + ], + }, + }, + { + "object.projection": { + name: "ProgramSummary", + children: [ + { "source.rdb": { "@kind": "view", "@table": "v_program_summary" } }, + { "field.int": { name: "id" } }, + { + "field.int": { + name: "weekCount", + children: [{ "origin.aggregate": { "@agg": "count", "@of": "Week.id", "@via": "Program.weeks" } }], + }, + }, + ], + }, + }, + ]); + + const out = await runGen({ config: genConfig(tmp), metadata: root }); + expect(out.warnings).toEqual([]); + const paths = new Set(out.files.map((f) => f.path)); + expect(paths.has(join(tmp, "ProgramSummary.ts"))).toBe(true); + expect(paths.has(join(tmp, "ProgramSummary.queries.ts"))).toBe(true); + expect(paths.has(join(tmp, "ProgramSummary.routes.ts"))).toBe(true); + }); +}); From f21c677b39ac637b0363690288cc1f9675834741 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 1 Aug 2026 17:26:54 -0400 Subject: [PATCH 5/7] fix(#248): reference scaffold generators derive queryability from source presence Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01HLoJkFSyoticveo5ehMUAr --- .../codegen-ts/src/reference/queries.ts | 20 ++++++--- .../codegen-ts/src/reference/routes.ts | 9 +++- .../test/reference-byte-identical.test.ts | 25 ++--------- .../test/sourceless-objects.test.ts | 44 +++++++++++++++++++ 4 files changed, 69 insertions(+), 29 deletions(-) diff --git a/server/typescript/packages/codegen-ts/src/reference/queries.ts b/server/typescript/packages/codegen-ts/src/reference/queries.ts index c497617da..8bf80409d 100644 --- a/server/typescript/packages/codegen-ts/src/reference/queries.ts +++ b/server/typescript/packages/codegen-ts/src/reference/queries.ts @@ -4,7 +4,9 @@ // // use-when: you want generated typed CRUD finders (findById, lists, create/update/delete) // over Drizzle. Drop it if you hand-write your data access. -// emits: /.queries.ts per write-through entity. +// emits: /.queries.ts per source-backed object (any source.rdb kind, +// incl. read-only projections) — skipped for sourceless objects (incl. every +// object.value, source-less by value purity) and TPH subtypes (#248 R2). // customize: the vanilla CRUD assembly below is OWNED — reorder, drop verbs (e.g. no delete), // change the Db type alias, add your own finders. The renderFn primitives emit // each block; call your own instead to change a verb's body. @@ -15,7 +17,7 @@ // out of the package source. The vanilla path here is byte-identical to the built-in. import { code, joinCode, type Code } from "ts-poet"; -import { OBJECT_SUBTYPE_VALUE, type MetaObject } from "@metaobjectsdev/metadata"; +import type { MetaObject } from "@metaobjectsdev/metadata"; import { perEntity, type Generator, @@ -33,6 +35,7 @@ import { isProjection, isWriteThrough, isTphSubtype, + hasAnyRdbSource, renderQueriesFile, // engine composer — used for the delegated variants formatTs, entityOutputPath, @@ -105,10 +108,15 @@ export interface QueriesFileOpts { target?: string; } -// value objects have no identity (findById/updateById would target a non-existent column), -// and TPH subtypes emit no standalone queries file — both are skipped unconditionally. -const skipNonQueryable = (e: MetaObject): boolean => - e.subType !== OBJECT_SUBTYPE_VALUE && !isTphSubtype(e); +// #248 R2: persistability derives from declared/inherited source, never subtype. +// An object with no source.rdb (of ANY kind) isn't backed by any store — the +// rendered queries module would emit findById/updateById/deleteById against +// Drizzle table/schema exports the entity file never emits for it (value +// objects are subsumed here too: value purity bans sources on them, so no +// loadable value ever has hasAnyRdbSource === true). TPH subtypes emit no +// standalone queries file either — their per-subtype CRUD helpers live in the +// discriminator base's queries file (which targets the single shared table). +const skipNonQueryable = (e: MetaObject): boolean => hasAnyRdbSource(e) && !isTphSubtype(e); export const queriesFile = function queriesFile(opts?: QueriesFileOpts): Generator { const userFilter = opts?.filter; diff --git a/server/typescript/packages/codegen-ts/src/reference/routes.ts b/server/typescript/packages/codegen-ts/src/reference/routes.ts index 6fa2b5872..08219d0ec 100644 --- a/server/typescript/packages/codegen-ts/src/reference/routes.ts +++ b/server/typescript/packages/codegen-ts/src/reference/routes.ts @@ -6,6 +6,9 @@ // if you need bespoke endpoints — or keep it and add handlers via .extra.ts. // emits: /.routes.ts — full CRUD for write-through entities, read-only // (GET list + GET :id) for projections, polymorphic + per-subtype for TPH bases. +// Skipped for any sourceless object (incl. every object.value, source-less by +// value purity) and for TPH subtypes — no source.rdb means no table/allowlist +// for a routes file to import (#248 R2). // customize: this generator (filter, output path, per-entity @emitRoutes opt-out, target) is // YOURS — edit it freely. The route *composition* itself is richer than the others // (M:N junction traversal, TPH per-subtype route sets), so it stays in the engine via @@ -23,6 +26,7 @@ import { type GeneratorFactory, renderRoutesFile, isTphSubtype, + hasAnyRdbSource, formatTs, entityOutputPath, CODEGEN_ATTR_EMIT_ROUTES, @@ -39,9 +43,12 @@ export const routesFile = function routesFile(opts?: RoutesFileOpts): Generator name: "routes-file", // per-entity opt-out via `@emitRoutes: false`; TPH subtypes get no standalone routes // file (their routes live in the discriminator base's); AND-composed with your filter. + // #248 R2: an object with no declared/inherited source.rdb (of ANY kind) isn't + // backed by any store — routes against it would import Drizzle table/allowlist + // exports the entity file never emits. Gated by hasAnyRdbSource. filter: (e: MetaObject) => // ADR-0039: resolving — a concrete entity may inherit its @emit* opt-out flag via extends. - e.attr(CODEGEN_ATTR_EMIT_ROUTES) !== false && !isTphSubtype(e) && userFilter(e), + e.attr(CODEGEN_ATTR_EMIT_ROUTES) !== false && hasAnyRdbSource(e) && !isTphSubtype(e) && userFilter(e), generate: perEntity(async (entity, ctx) => { if (!ctx.renderContext) { throw new Error("routes-file: renderContext is required (provided by runGen)"); diff --git a/server/typescript/packages/codegen-ts/test/reference-byte-identical.test.ts b/server/typescript/packages/codegen-ts/test/reference-byte-identical.test.ts index 5471f8eec..2476951ae 100644 --- a/server/typescript/packages/codegen-ts/test/reference-byte-identical.test.ts +++ b/server/typescript/packages/codegen-ts/test/reference-byte-identical.test.ts @@ -43,24 +43,6 @@ async function gen(dir: string, generators: ReturnType[], return out; } -// #248 R2 (Task 2 of 2 for the codegen-ts half): the built-in engine generators -// (src/generators/*) now gate queries/routes on hasAnyRdbSource — a sourceless -// object no longer gets a broken queries/routes file. The scaffold-and-own -// reference templates (src/reference/*) get the SAME fix in the very next -// commit (Task 3); until then they still use the old subtype-based filter, so -// they diverge from the now-fixed built-ins by exactly the DB-bound artifacts -// a sourceless/value object should never have gotten in the first place. Known, -// tracked, temporary — remove KNOWN_PENDING_DIVERGENCE once Task 3 lands (it -// makes src/reference/{queries,routes}.ts derive from hasAnyRdbSource too). -const KNOWN_PENDING_DIVERGENCE: Record = { - // cross-package-vo.json's "Triple" is an object.value (no source, value - // purity) — reference/routes.ts has no value/source skip at all (design - // spec §4, "reference/routes.ts:40-44 same" REDERIVE row) and still emits - // Triple.routes.ts; the built-in routes-file.ts (fixed here) correctly does - // not. - "cross-package-vo.json": ["Triple.routes.ts"], -}; - describe("ADR-0034 — reference templates are byte-identical to built-ins", () => { for (const fixture of FIXTURES) { test(fixture, async () => { @@ -73,10 +55,9 @@ describe("ADR-0034 — reference templates are byte-identical to built-ins", () try { const a = await gen(aDir, [builtinEntity(), builtinQueries(), builtinRoutes(), builtinBarrel()], result.root); const b = await gen(bDir, [refEntity(), refQueries(), refRoutes(), refBarrel()], result.root); - const pending = new Set(KNOWN_PENDING_DIVERGENCE[fixture] ?? []); - const aKeys = Object.keys(a).filter((k) => !pending.has(k)).sort(); - const bKeys = Object.keys(b).filter((k) => !pending.has(k)).sort(); - // same set of files (excluding the known-pending Task 3 divergence) + const aKeys = Object.keys(a).sort(); + const bKeys = Object.keys(b).sort(); + // same set of files expect(bKeys).toEqual(aKeys); // byte-identical contents for every file both sides agree on emitting for (const k of aKeys) { diff --git a/server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts b/server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts index 13d0c0a94..f334f388e 100644 --- a/server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts +++ b/server/typescript/packages/codegen-ts/test/sourceless-objects.test.ts @@ -20,6 +20,8 @@ import { join } from "node:path"; import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; import { runGen, defineConfig } from "../src/index.js"; import { entityFile, queriesFile, routesFile, routesFileHono } from "../src/generators/index.js"; +import { queriesFile as refQueriesFile } from "../src/reference/queries.js"; +import { routesFile as refRoutesFile } from "../src/reference/routes.js"; let tmp: string; beforeEach(() => { @@ -86,6 +88,22 @@ function genConfig(outDir: string) { }); } +// Task 3 — the ADR-0034 scaffold-and-own reference templates (src/reference/*) +// must gate queries/routes emission on the same hasAnyRdbSource signal as the +// engine generators above. `entityFile` is reused unchanged (its table-vs-shape +// dispatch was already correct — see design spec §4 "KEEP" row); only the +// queries/routes reference generators are under test here. There is no +// reference hono generator, so routesFileHono is intentionally omitted. +function refGenConfig(outDir: string) { + return defineConfig({ + outDir, + extStyle: "none", + dbImport: "~/server/db", + dialect: "postgres", + generators: [entityFile(), refQueriesFile(), refRoutesFile()], + }); +} + describe("#248 R2 — sourceless objects get no DB-bound artifacts", () => { test("Order (sourced) gets entity+queries+routes+hono; Money (value) and Ghost (sourceless entity) get neither", async () => { const root = await loadRoot([ORDER, MONEY, GHOST]); @@ -121,6 +139,32 @@ describe("#248 R2 — sourceless objects get no DB-bound artifacts", () => { expect(onDisk.has("Money.routes.ts")).toBe(false); }); + test("reference (scaffold-and-own) generators: same gating as the engine — Order gets queries+routes; Money/Ghost get neither", async () => { + const root = await loadRoot([ORDER, MONEY, GHOST]); + const out = await runGen({ config: refGenConfig(tmp), metadata: root }); + expect(out.warnings).toEqual([]); + + const paths = new Set(out.files.map((f) => f.path)); + const at = (name: string) => join(tmp, name); + + // INCLUDES + expect(paths.has(at("Order.ts"))).toBe(true); + expect(paths.has(at("Order.queries.ts"))).toBe(true); + expect(paths.has(at("Order.routes.ts"))).toBe(true); + expect(paths.has(at("Money.ts"))).toBe(true); + expect(paths.has(at("Ghost.ts"))).toBe(true); + + // EXCLUDES — the same bug, in the reference (copy-and-own) template. + expect(paths.has(at("Ghost.queries.ts"))).toBe(false); + expect(paths.has(at("Ghost.routes.ts"))).toBe(false); + expect(paths.has(at("Money.queries.ts"))).toBe(false); + expect(paths.has(at("Money.routes.ts"))).toBe(false); + + const onDisk = new Set(readdirSync(tmp)); + expect(onDisk.has("Ghost.queries.ts")).toBe(false); + expect(onDisk.has("Money.routes.ts")).toBe(false); + }); + test("content no-churn: Order.* is byte-identical whether or not Money/Ghost are co-loaded", async () => { const mixedRoot = await loadRoot([ORDER, MONEY, GHOST]); const mixedOut = tmp; From 4766e83a2e005070468eab0d1b6b897acba21b49 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 1 Aug 2026 17:45:11 -0400 Subject: [PATCH 6/7] fix(#248): adapt sourceless test fixtures in consuming packages to source-derived persistability Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01HLoJkFSyoticveo5ehMUAr --- .../packages/cli/test/integration/gen-multi-target.test.ts | 1 + .../packages/cli/test/integration/verify-db-drift.test.ts | 2 ++ server/typescript/packages/cli/test/unit/migrate-d1.test.ts | 1 + server/typescript/packages/cli/test/unit/verify-d1.test.ts | 1 + .../cli/test/unit/wrangler-local-state-warning.test.ts | 1 + .../packages/integration-tests/test/view-lifecycle-pg.test.ts | 4 ++++ 6 files changed, 10 insertions(+) diff --git a/server/typescript/packages/cli/test/integration/gen-multi-target.test.ts b/server/typescript/packages/cli/test/integration/gen-multi-target.test.ts index 963bc12b3..077cbd021 100644 --- a/server/typescript/packages/cli/test/integration/gen-multi-target.test.ts +++ b/server/typescript/packages/cli/test/integration/gen-multi-target.test.ts @@ -20,6 +20,7 @@ function setupRepo(): string { name: "Program", package: "mikes::commerce", children: [ + { "source.rdb": {} }, { "field.long": { name: "id", children: [ { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } }, ] } }, diff --git a/server/typescript/packages/cli/test/integration/verify-db-drift.test.ts b/server/typescript/packages/cli/test/integration/verify-db-drift.test.ts index e9d5e8728..988fae078 100644 --- a/server/typescript/packages/cli/test/integration/verify-db-drift.test.ts +++ b/server/typescript/packages/cli/test/integration/verify-db-drift.test.ts @@ -17,6 +17,7 @@ import { run } from "../../src/index.js"; function metaJson(withColor: boolean): string { const widgetChildren: Record[] = [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.string": { name: "name", "@column": "name" } }, ]; @@ -144,6 +145,7 @@ describe("meta verify --db — schema-drift gate", () => { package: "acme::drift", children: [ { "object.entity": { name: "Widget", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.string": { name: "name", "@column": "name" } }, { "identity.primary": { name: "pk", "@fields": ["id"] } }, diff --git a/server/typescript/packages/cli/test/unit/migrate-d1.test.ts b/server/typescript/packages/cli/test/unit/migrate-d1.test.ts index 54e6f9a6a..56f892b65 100644 --- a/server/typescript/packages/cli/test/unit/migrate-d1.test.ts +++ b/server/typescript/packages/cli/test/unit/migrate-d1.test.ts @@ -37,6 +37,7 @@ describe("migrate command with --dialect d1", () => { "object.entity": { "name": "User", "children": [ + { "source.rdb": {} }, { "field.long": { "name": "id" } }, { "field.string": { "name": "email" } }, { "identity.primary": { "name": "id", "@fields": ["id"], "@generation": "increment" } } diff --git a/server/typescript/packages/cli/test/unit/verify-d1.test.ts b/server/typescript/packages/cli/test/unit/verify-d1.test.ts index eb275919b..c06822ea4 100644 --- a/server/typescript/packages/cli/test/unit/verify-d1.test.ts +++ b/server/typescript/packages/cli/test/unit/verify-d1.test.ts @@ -86,6 +86,7 @@ describe("meta verify --dialect d1 — schema-drift gate", () => { "object.entity": { name: "User", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.string": { name: "email" } }, { "identity.primary": { name: "id", "@fields": ["id"], "@generation": "increment" } }, diff --git a/server/typescript/packages/cli/test/unit/wrangler-local-state-warning.test.ts b/server/typescript/packages/cli/test/unit/wrangler-local-state-warning.test.ts index 17905df1a..74198dbb2 100644 --- a/server/typescript/packages/cli/test/unit/wrangler-local-state-warning.test.ts +++ b/server/typescript/packages/cli/test/unit/wrangler-local-state-warning.test.ts @@ -44,6 +44,7 @@ function metaJson(): string { "object.entity": { name: "Widget", children: [ + { "source.rdb": {} }, { "field.long": { name: "id" } }, { "field.string": { name: "name", "@column": "name" } }, { "identity.primary": { name: "pk", "@fields": ["id"] } }, diff --git a/server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts b/server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts index e41213c21..d0c80c23d 100644 --- a/server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts +++ b/server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts @@ -42,6 +42,7 @@ function meta(opts: { summaryFields: string[] }): string { "package": "acme", "children": [ { "object.entity": { "name": "Program", "children": [ + { "source.rdb": {} }, { "field.long": { "name": "id" } }, { "field.string": { "name": "title", "@required": true } }, { "field.string": { "name": "status", "@required": true } }, @@ -49,6 +50,7 @@ function meta(opts: { summaryFields: string[] }): string { { "relationship.aggregation": { "name": "weeks", "@cardinality": "many", "@objectRef": "Week" } } ] } }, { "object.entity": { "name": "Week", "children": [ + { "source.rdb": {} }, { "field.long": { "name": "id" } }, { "field.long": { "name": "programId", "@required": true } }, { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } }, @@ -84,6 +86,7 @@ function metaSql(): string { "package": "acme", "children": [ { "object.entity": { "name": "Program", "children": [ + { "source.rdb": {} }, { "field.long": { "name": "id" } }, { "field.string": { "name": "title", "@required": true } }, { "field.string": { "name": "status", "@required": true } }, @@ -106,6 +109,7 @@ function metaUnmanagedView(): string { return `{ "metadata.root": { "package": "acme", "children": [ { "object.entity": { "name": "Program", "children": [ + { "source.rdb": {} }, { "field.long": { "name": "id" } }, { "field.string": { "name": "title", "@required": true } }, { "identity.primary": { "name": "id", "@fields": "id", "@generation": "increment" } } From 5c0d6939d0442c02b216dc6552b4d5eaa549a81f Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 1 Aug 2026 17:51:57 -0400 Subject: [PATCH 7/7] =?UTF-8?q?docs(#248):=20changelog=20=E2=80=94=20persi?= =?UTF-8?q?stability=20derives=20from=20source=20presence=20(npm-only=20pa?= =?UTF-8?q?tch)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01HLoJkFSyoticveo5ehMUAr --- CHANGELOG.md | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index f9acf8b29..b89f4e30a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -61,6 +61,44 @@ metamodel vocabulary (ADR-0023 unaffected). See [ADR-0044](spec/decisions/ADR-0044-payload-record-naming-cross-package-collision.md), whose Consequences section now marks this recurrence closed. +### Fixed — persistability derives from a declared/inherited source, never object subtype (#248) + +**npm-only** (`migrate-ts` + `codegen-ts`; PyPI / NuGet / Maven Central unchanged — schema +migrations and this codegen tier are TS-owned artifacts, ADR-0015, and no other port emits +DDL). Both `migrate-ts`'s expected-schema builder and `codegen-ts`'s query/route/api-doc +emitters decided "is this object persisted?" against a hardcoded `subType === "value"` +compare (migrate) or a subtype allowlist (codegen), instead of asking whether the object +declares — or inherits via `extends` — a `source.*` child, which is the loader's own +already-published contract (`validate-source-roles`: an object with zero sources loads clean +and means "not persisted"). Any OTHER provider-registered `object` subtype with no source fell +through both gates and was silently treated as persisted. + +- **`meta migrate` / `meta verify --db|--d1`:** a sourceless object no longer produces a + phantom `CREATE TABLE` (with a fabricated physical table name) and is no longer eligible as + a foreign-key target. Reported blast radius: a package of roughly 150 wire-protocol message + objects modeled as a registered custom `object` subtype, co-loaded with domain entities so + messages could reference them, produced well over a hundred phantom `CREATE TABLE` + statements — making `meta migrate` and `meta verify --db` unusable against that model. +- **`meta gen`:** queries/routes (both the Fastify and Hono generators) and api-doc CRUD are + no longer emitted for a sourceless object, gated on the presence of a `source.rdb` of any + kind (a new `hasAnyRdbSource` check) — matching the Drizzle table tier's existing + `hasWritableRdbSource` gate. This also closes a pre-existing fail-open where a plain + `object.value` got a broken `*.routes.ts` file, importing a table const and + filter/sort allowlists that don't exist for a value object. + +This is a bugfix, not a behavior-contract change — it aligns both tiers to an invariant the +loader already enforces. `meta gen` / `meta migrate` output is **byte-identical** for +well-formed models (every table-owning object declares or inherits a source, the norm); the +only delta is that files which could never have typechecked (importing exports that don't +exist) stop being emitted. + +**Migration note for models already bitten by the bug:** a database or migration snapshot +that already contains phantom tables (created by applying a pre-fix migration) will correctly +propose `DROP TABLE` for them on the next `meta migrate` — destructive-gated behind the +existing `--allow drop-table` policy, so nothing drops without explicit opt-in. +Previously-generated broken `*.queries.ts` / `*.routes.ts` files are **not** auto-pruned +(`meta gen` never deletes existing files) — remove them by hand. + ## [0.20.9] — 2026-07-28 **npm-only** — `migrate-ts` + `codegen-ts` (schema migrations and projection-view codegen