fix(seed-loader): support composite externalId so join-table seeds dedupe on replay (#3434)#3442
Merged
Merged
Conversation
…dupe on replay (#3434) A junction / join table has no single-field natural key — the PAIR of its foreign keys is what's unique — so its seed could only run `mode: 'insert'`, which re-inserts every row on each replay boot with no existing-row check (`decideWriteAction`'s `insert` case returned `insert` unconditionally). The table duplicated on every restart: the showcase `showcase_project_membership` fixture (3 rows) grew 3 → 6 → 9. It stayed hidden until #3415 let the master-detail parents seed at all. - `SeedSchema.externalId` now accepts a list of field names (`externalId: ['team', 'project']`) as well as a single field name, declaring a composite natural key. Default stays 'name'. - `SeedLoaderService` builds the uniqueness key from all listed fields (joined with a NUL separator that can't occur in a natural-key value) via a single `externalIdKey` helper threaded through every key-building call site. Reference key fields are compared by their RESOLVED parent ids — which the existing DB row already stores — so a composite of foreign keys matches across restarts. A partial key (any component absent) is treated as no key, falling back to insert, exactly as a missing single-field key already did. Single-field behavior is byte-for-byte unchanged. - A composite-key target does not participate in single-value reference resolution (a reference is one natural-key string), so such objects keep the 'name' default when referenced by another dataset. The showcase membership fixture switches to `mode: 'ignore'` + `externalId: ['team', 'project']`, so replay boots leave the three rows untouched instead of duplicating them. Tests: new composite-key replay test (insert-once then skip-all on replay, and a new pair still inserts while sharing rows don't block it) plus a spec schema test; existing single-field seed-loader suites unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqZsDkKaCkpQGpJdEMEWDF
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
github-actions
Bot
added
documentation
Contributor
📓 Docs Drift CheckThis PR changes 2 package(s): 104 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
`pnpm --filter @objectstack/spec gen:schema && gen:docs` after the `SeedSchema.externalId` union change. These files are generated from the Zod schema (Prime Directive #1) and the check:docs CI gate flagged them stale. Only the externalId type row changes (`string` → `string | string[]`) across seed, seed-loader, and manifest reference tables. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqZsDkKaCkpQGpJdEMEWDF
os-zhuang
marked this pull request as ready for review
July 24, 2026 15:04
os-zhuang
added a commit
that referenced
this pull request
Jul 24, 2026
…osite externalId (#3460) Follow-up to #3442 (which closed #3434). - lint: add validateSeedReplaySafety (@objectstack/lint, a pure (stack) => Finding[] rule per ADR-0019) and wire it into os validate / os lint. It flags every data[] seed declared mode:'insert' — the one non-idempotent mode, which duplicates its table on every replay boot — and points the author at ignore/upsert plus an externalId (single field, or a composite list for a join table). Catches the #3434 footgun at authoring time. - docs: correct the seed insert-mode row in seed-data.mdx, add an insert replay-safety warning, and add a "Composite externalId — Join / Junction Tables" section plus best-practice and API-reference updates. Verified on a real better-sqlite3 file DB with the real showcase ProjectMembership object: mode:'insert' reproduces 3 -> 6 -> 9 across reboots, while mode:'ignore' + externalId:['team','project'] stays 3 -> 3 -> 3. Refs #3434, #3442 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqZsDkKaCkpQGpJdEMEWDF
os-zhuang
added a commit
that referenced
this pull request
Jul 24, 2026
…3433 follow-up) (#3471) Follow-ups to the #3433 seed / state-machine exemption: - lint: `validateSeedStateMachine` — an author-time `os validate` / `os lint` warning when a seed record's `state_machine`-governed field carries a value the machine does not declare (initialStates ∪ transition keys ∪ targets). #3433 exempts seeds from the FSM at write time, so this re-adds the "is this even a known state?" safety net a typo would otherwise slip past. Advisory, symmetric with the #3434 replay-safety rule. Rule id `seed-value-outside-state-machine`. - test(cloud-connection): a marketplace-install integration test that drives the real install handler → runInlineSeed → SeedLoaderService against an engine stub reproducing the #3165 initialStates guard, proving a template whose seed spans the whole pipeline (prospecting → closed_won) lands every row on install. Red without the #3433 exemption (verified). - docs: note the seed exemption in the state-machine, validation, and seed-data guides — seeds are established facts, not lifecycle events; every other validation still runs; `os lint` catches an unknown-state typo before boot. The remaining #3433 follow-up — #3434 (mode:'insert' seeds duplicate on replay) — was already closed by #3442 (composite externalId) + #3460 (replay-safety lint), so it needs no change here. Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
mode: 'insert'的 dataset 在每次 boot 重放时无条件重插——decideWriteAction的insertcase 直接return { action: 'insert' },从不查 existing。join 表(junction table)没有单字段天然唯一键——唯一的只有两个外键的组合——所以它只能用insert,于是每次重启都重复入库:showcase 的showcase_project_membership(3 条)实测 3 → 6 → 9。此前被 #3415 掩盖(membership 的 master-detail 父项目缺失、0 行入库),#3415 修好后立即显形。Issue 给了两个方向:方向 1(引擎支持组合键,更根治,任何 join 表种子都会遇到)与方向 2(给对象加一个显式
name字段当 externalId,一行可落)。本 PR 采用方向 1——按 Prime Directive #5(不做临时补丁,采用可持续方案)。给纯 join 对象塞一个冗余name字段只是为了迁就 seed loader,属于 workaround;junction 对象本就应以组合外键为天然键,seed loader 应当支持。改动
packages/spec/src/data/seed.zod.ts—SeedSchema.externalId现同时接受单字段名与字段名列表(externalId: ['team', 'project']),用于声明组合天然键;默认仍为'name'。packages/metadata-protocol/src/seed-loader.ts— 新增单一的externalIdKey辅助方法,用 NUL(``,天然键值不可能包含)拼接各字段值构成唯一键,并贯穿所有构键调用点(loadExistingRecords/ `decideWriteAction` / `writeRecord` / 批量插入 `extIdOf` / deferred updates / `buildWriteError`)。'name'默认。examples/app-showcase/src/data/seed/index.ts— membership fixture 改为mode: 'ignore'+externalId: ['team', 'project'],重放 boot 不再复制这三行。测试
seed-loader-composite-external-id.test.ts(faithful engine):首 boot 三个 (team, project) 组合各插一次;重放 boot 全部 skip(仍是 3 行,不再 3→6→9);第三个 boot 仍 3 行;共享同一 team/project 的新组合仍会插入,不会被误判为重复。externalId,拒绝空数组。metadata-protocol(16)、specdata/seed(60)、runtimeseed-loader(41,含mode:'insert')、objectqlseed-loader(7)、app-showcaseseed(4)、app-plugin.seed+ 真实驱动 bulk-write(9)全部通过。pnpm build(71/71)通过。Closes #3434
🤖 Generated with Claude Code
https://claude.ai/code/session_01JqZsDkKaCkpQGpJdEMEWDF
Generated by Claude Code