Skip to content

feat(data): add Seed Loader Protocol for relationship resolution & dependency ordering#875

Draft
Copilot wants to merge 2 commits intomainfrom
copilot/support-dynamic-relationship-resolution
Draft

feat(data): add Seed Loader Protocol for relationship resolution & dependency ordering#875
Copilot wants to merge 2 commits intomainfrom
copilot/support-dynamic-relationship-resolution

Conversation

Copy link
Contributor

Copilot AI commented Mar 5, 2026

The engine's seed data loader has no protocol for automatically resolving lookup/master_detail references via externalId, determining insert order from object dependencies, or handling circular references with multi-pass loading. This makes seed/demo/config scenarios fragile and manual.

New schemas (seed-loader.zod.ts)

  • ReferenceResolutionSchema — maps a lookup/master_detail field to its target object and match field
  • ObjectDependencyNodeSchema / ObjectDependencyGraphSchema — object dependency graph with topological insert order and circular dependency detection
  • ReferenceResolutionErrorSchema — actionable error: source object, field, target object, target field, attempted value, record index, message
  • SeedLoaderConfigSchemadryRun, haltOnError, multiPass, defaultMode, batchSize, transaction, env
  • DatasetLoadResultSchema — per-object stats: inserted/updated/skipped/errored, references resolved/deferred
  • SeedLoaderResultSchema — aggregated result with dependency graph, per-object results, errors, summary
  • SeedLoaderRequestSchema — combines DatasetSchema[] with loader config

Usage

import { SeedLoaderRequestSchema } from '@objectstack/spec/data';

const request = SeedLoaderRequestSchema.parse({
  datasets: [
    { object: 'account', externalId: 'name', records: [{ name: 'Acme Corp' }] },
    { object: 'contact', records: [{ name: 'John', account_id: 'Acme Corp' }] },
  ],
  config: { dryRun: true, multiPass: true },
});
// request.config.dryRun === true
// request.config.defaultMode === 'upsert' (applied via defaults)

Notes

  • Uses z.preprocess for the nested config field — Zod's .default({}) does not apply inner field defaults
  • Builds on existing DatasetSchema / DatasetMode; exported from data/index.ts
  • 40 tests covering all schemas, full suite (6665 tests) passing
Original prompt

This section details on the original issue you should resolve

<issue_title>[Meta-driven Seed Loader] Engine must support dynamic relationship resolution & dependency ordering for seed data initialization</issue_title>
<issue_description>## Problem
ObjectStack currently allows objects and their fields to be defined and loaded entirely via metadata, but the engine's seed data loader does not automatically resolve relationships (lookup/master_detail fields) between records during initialization. Instead, it expects either pre-populated IDs, manual foreign-key mapping, or a fully correct insert order.

This makes onboarding, demo, and config scenarios fragile, and does not meet the self-driving data onboarding expectation of metadata-driven platforms.

Detail Checklist

  • Engine seed loader must, during insert/upsert of seed data, automatically resolve lookup/master_detail references using target object's externalId (usually 'name'). This should happen if the field value is not already an internal ID, but is a string matching a unique key in the referenced object.
  • Topological sort must be performed to determine import order based on object dependency graph, so parent objects are inserted before child objects that reference them.
  • If circular or delayed references exist, loader must perform a second pass to fill unresolved relationships after all records are inserted.
  • Engine should respect DatasetSchema's mode: upsert, not just blindly insert objects; upsert logic should be honored based on externalId for idempotency.
  • Any reference resolution failure should generate actionable error messages with: field name, target object, externalId, attempted value, and possible missing referent.
  • Loader should support dry-run for validation (check relationships before write) to surface broken references before failures.
  • All logic should be metadata-driven and use SchemaRegistry, supporting hot-reload and dynamic object changes.

Acceptance Criteria

  • Seed data can be written entirely using references via externalId (natural key).
  • Demo and onboarding environments require no manual ID prep or table pre-sorting.
  • All new objects with relationships work immediately from metadata alone, regardless of schema changes.
  • Error handling and reporting matches the ease-of-use of platforms like Salesforce, Airtable, ServiceNow, etc.

Impact

This makes ObjectStack truly declarative, metadata-first for data onboarding, and closes a gap in platform intelligence compared to major low-code engines.</issue_description>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

@vercel
Copy link

vercel bot commented Mar 5, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
objectstack-play Ready Ready Preview, Comment Mar 5, 2026 4:28am
spec Ready Ready Preview, Comment Mar 5, 2026 4:28am

Request Review

…ependency ordering

Add seed-loader.zod.ts with Zod schemas for:
- ReferenceResolutionSchema: field reference resolution via externalId
- ObjectDependencyNodeSchema/GraphSchema: topological sort for insert order
- ReferenceResolutionErrorSchema: actionable error reporting
- SeedLoaderConfigSchema: dry-run, multi-pass, batch, transaction config
- DatasetLoadResultSchema: per-object load statistics
- SeedLoaderResultSchema: complete load result with summary
- SeedLoaderRequestSchema: request combining datasets + config

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Copilot AI changed the title [WIP] Add dynamic relationship resolution for seed data initialization feat(data): add Seed Loader Protocol for relationship resolution & dependency ordering Mar 5, 2026
@hotlong
Copy link
Contributor

hotlong commented Mar 5, 2026

@copilot 我是想实现API和运行时环境,评估这个协议是否有必要

Copy link
Contributor Author

Copilot AI commented Mar 5, 2026

@copilot 我是想实现API和运行时环境,评估这个协议是否有必要

这个协议对API和运行时实现是有价值的,但不是必须先行的。主要作用:

  1. Runtime ValidationSeedLoaderRequestSchema / SeedLoaderResultSchema 可直接用于API入参和返回值的运行时校验
  2. Type Safety — 运行时实现可以 z.infer<typeof ...> 获取完整类型,减少手写interface
  3. Error ContractReferenceResolutionErrorSchema 定义了标准化的错误报告格式,API层可以直接序列化返回
  4. JSON Schema — 自动生成API文档和IDE提示

但如果你更倾向于在实现过程中迭代定义协议(而非先定义后实现),我可以revert这些schema,让你在runtime包中边实现边定义。这取决于你更偏好 spec-first 还是 implementation-first 的工作流。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Meta-driven Seed Loader] Engine must support dynamic relationship resolution & dependency ordering for seed data initialization

2 participants