Skip to content

fix(rest): validate required fields in import dry-run to match the real insert#2839

Merged
os-zhuang merged 2 commits into
mainfrom
fix/import-dryrun-required-validation
Jul 11, 2026
Merged

fix(rest): validate required fields in import dry-run to match the real insert#2839
os-zhuang merged 2 commits into
mainfrom
fix/import-dryrun-required-validation

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Problem

POST /data/:object/import with dryRun:true reports success for rows that then FAIL the real insert on NOT NULL / required-field constraints. The dry-run branch in import-runner.ts only runs cell coercion (coerceRow), then marks every coercible CREATE row ok — it never checks that required fields have a value. The real insert goes through the engine, where a required (⇒ NOT NULL) column with no default and no value is rejected.

Live-observed (2026-07-11), AI-built object mx1n_member (required member_name; status — a select active/frozen/lost_contact/archived, no default):

  • dryRun:true{total:5, ok:5, errors:0} (says all 5 will succeed)
  • dryRun:false (same rows, missing status) → {ok:0, errors:5}, each NOT NULL constraint failed: mx1n_member.status

The ImportWizard (objectui) shows the dry-run result, so it promised imports that then failed.

Fix

A required-field pre-check in the shared import runner (runImport), applied to CREATE rows, mirroring the engine's insert-time validation (objectql/record-validator.ts + applyFieldDefaults):

  • A required field is unsatisfied only when it has no value AND no default. system/readonly/autonumber and the engine-owned lifecycle columns (id, created_at, …) are exempt — exactly the engine's own exemptions.
  • ExportFieldMeta gains required/system/readonly/hasDefault, populated by buildFieldMetaMap from the object schema (unused by the export path).
  • New firstMissingRequiredField(data, metaMap) in import-coerce.ts.
  • Applied to both dry-run and the real path (this module keeps them identical), so the dry run predicts the real insert exactly and a real insert returns a readable status is required instead of a raw driver error. Skipped when runAutomations is set — a beforeInsert hook may legitimately populate a required field, so we defer to the engine's own validation there.

Tests

import-integration.test.ts (real engine + protocol) gains a member fixture (required member_name; required-no-default status; required-but-defaulted tier) and cases:

  • required no-default field missing → dry-run and real insert both fail with code:'required'; the complete row still succeeds;
  • a required field with a schema default is satisfied without being mapped;
  • required text fields flagged too; a blank cell counts as missing;
  • update-mode rows are exempt (only touched fields matter).

Full rest import suite green (44/44 locally against the previously-pinned SHA; branch rebased onto current main, and export-format.ts cleanly merged with the xlsx-color change #2761).

Companion (cloud)

objectstack-ai/cloud#807 (merged) gives AI-built objects a sensible default for required select fields — the root-cause fix for AI-built objects. This PR is the general safety net that makes the dry run honest for every object, including required text fields that can't be auto-defaulted. Once merged, cloud's .framework-sha should be bumped to pick it up.

🤖 Generated with Claude Code

https://claude.ai/code/session_01P5MRtrxWenRExebx2WrnFt


Generated by Claude Code

…al insert

The bulk-import dry run (`POST /data/:object/import`, dryRun:true) only ran cell
coercion, then reported every coercible CREATE row as ok — so a row missing a
required NOT-NULL field with no default was green-lit, then died on the real
insert with `NOT NULL constraint failed` (live: mx1n_member.status). The
ImportWizard shows the dry-run result, so it promised imports that then failed.

Add a required-field pre-check to the shared import runner (CREATE rows only),
mirroring the engine's insert-time validation (objectql record-validator.ts +
applyFieldDefaults): a required field is unsatisfied only when it has no value
AND no default; system/readonly/autonumber and engine-owned lifecycle columns
are exempt. Carried on ExportFieldMeta (required/system/readonly/hasDefault),
populated by buildFieldMetaMap. Applied to both dry-run and real paths so they
stay identical and the real insert returns a readable `<field> is required`
instead of a raw driver error; skipped when runAutomations is set (a
beforeInsert hook may populate the field).
@vercel

vercel Bot commented Jul 11, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
spec Ready Ready Preview, Comment Jul 11, 2026 2:04pm

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/rest.

8 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/api/error-catalog.mdx (via @objectstack/rest)
  • content/docs/api/error-handling-server.mdx (via @objectstack/rest)
  • content/docs/api/index.mdx (via @objectstack/rest)
  • content/docs/permissions/authorization.mdx (via packages/rest)
  • content/docs/plugins/index.mdx (via @objectstack/rest)
  • content/docs/plugins/packages.mdx (via @objectstack/rest)
  • content/docs/releases/implementation-status.mdx (via @objectstack/rest)
  • content/docs/releases/v12.mdx (via @objectstack/rest)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actions github-actions Bot added documentation Improvements or additions to documentation tooling labels Jul 11, 2026
@os-zhuang
os-zhuang marked this pull request as ready for review July 11, 2026 13:56
@os-zhuang
os-zhuang merged commit 4d9dd7b into main Jul 11, 2026
15 of 16 checks passed
@os-zhuang
os-zhuang deleted the fix/import-dryrun-required-validation branch July 11, 2026 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants