Flow lookup-field template resolution (`{record.account.name}`) needs a read-identity design decision (follow-up to #3426)

[os-zhuang]

Follow-up to #3426. PR #3472 closed the formula half of {record.<path>} template resolution (via the #3445 re-read) and added a build-time warning for unresolvable paths. The remaining piece — single-hop lookup traversal like {record.account.name} — is deferred here because it hinges on a design decision, not plumbing.

Problem

In a record-change flow, a node template {record.account.name} (where account is a lookup) renders as an empty string. The seeded flow record carries the relation as a scalar foreign-key id, not an expanded object, so the template walk .name on a string id yields nothing — silently. Same for a notify node's title/body, start conditions, etc.

Scoping — the plumbing is easy (no spec/engine/automation changes)

• FlowTriggerBinding.config already carries the start node's config by reference (service-automation/src/engine.ts resolveTriggerBinding), so a config.expand opt-in surfaces on the binding with no automation-engine change.
• IDataEngine.findOne already accepts an expand: Record<string, QueryAST> map, and expandRelatedRecords (objectql/src/engine.ts) infers the target object from the field's reference — passing expand: { account: {} } is enough; no nested object needed.
• The flow start node's config is an open z.record, so an expand key validates today with no spec schema change.

The blocker — a read-identity / authorization decision

The trigger-record-change re-read that hydrates the record runs as an elevated system principal (so formula virtuals aren't FLS-masked). If we expanded a relation there, the expand sub-read would also run as system — bypassing the triggering user's RLS/FLS on the referenced object. For a runAs:'user' flow, {record.account.secret} in a notify template would then leak fields the user can't otherwise see.

Reading the expand as the user is not feasible in the trigger: the ObjectQL hook session carries only userId, not the user's resolved positions/permissions — the automation engine resolves those only at flow-run setup (see #3356). So the trigger structurally lacks the identity needed to scope a safe expand.

Proposed direction (needs a small ADR)

Do the lookup expansion inside the automation engine's flow run, which already resolves the triggering user's full grants for runAs:'user', rather than in the trigger's system re-read. Alternatively, a lazy template-time resolve that uses the flow's execution identity. Either keeps the RLS/FLS check intact.

Design points to settle:

• Where expand is declared (flow/start-node config.expand: string[] vs per-node) and its shape.
• Merge semantics: an opted-in relation becomes an object on the record, so {record.account} (bare id) and [P2] record-change context does not hydrate multi-lookup (junction) fields #1872 multi-lookup arrays must stay untouched for non-expanded relations.
• Read identity for the expand (must respect the triggering user under runAs:'user'; system only for runAs:'system').

Meanwhile

• os validate now warns on these paths (flow-template-lookup-traversal, added in PR fix(#3426): build-time warning for unresolvable flow templates + guard the formula re-read #3472), so authors get a design-time signal instead of a silent blank.
• Workaround: add a formula field on the source object that projects the related value (e.g. account_name) and reference that, or recompose from local scalar columns.

Acceptance

• {record.account.name} resolves in flow templates/conditions under runAs:'user' without exposing fields the triggering user could not read directly.
• Opt-in per relation; bare {record.<lookup>} ids and multiple lookup arrays ([P2] record-change context does not hydrate multi-lookup (junction) fields #1872) are unaffected for relations not expanded.
• The flow-template-lookup-traversal lint warning is suppressed for a relation once expansion is declared.

[os-zhuang]

Implementation plan (verified against main @ 169b58a7a)

Mapping the automation engine changed the picture in the issue's favour: the triggering user's grants are already resolved engine-side, and there is already a user-scoped read path. So safe lookup expansion is a self-contained engine-side change — no trigger change, no ObjectQL change, no spec-schema change.

Key facts (file:line)

• User identity is resolved in the engine, not the trigger. AutomationEngine.resolveRunContext() (service-automation/src/engine.ts:1522-1574) stamps runAs and, for runAs:'user', resolves the triggering user's full positions/permissions via the injected userGrantsResolver (engine.ts:1539-1563; the service-automation: runAs:'user' runs data ops with a credential-less user — trigger context never carries the actor's permission sets/positions (follow-up to #1888) #3356 resolver, injected by setUserGrantsResolver engine.ts:1135, wired to core's resolveUserAuthzGrants in plugin.ts:508-525). The resolved runContext is threaded into every node executor (engine.ts:1683 → executeNode → executor.execute(node, variables, context) engine.ts:2543/2548).
• A user-scoped data read already exists. resolveRunDataContext(runContext) (service-automation/src/runtime-identity.ts:46-60) maps the run context to { isSystem, userId, positions, permissions, tenantId }; get_record uses it: data.findOne(objectName, { where, fields, context: dataCtx }) (builtin/crud-nodes.ts:107). runAs:'system' → { isSystem:true }; runAs:'user' → the triggering user. Reusing this makes the expand read respect the user's RLS/FLS automatically.
• One record, shared by all interpolators. execute() seeds variables.set('record'/'$record', context.record) at engine.ts:1619-1625; every node interpolates {record.<...>} against that same map (per-node interpolate call sites in builtin/*-nodes.ts; resolver template.ts:47/84-92). runContext.record === context.record (shallow clone, engine.ts:1523), so enriching that object in place once makes {record.account.name} resolve in every node with zero per-node changes.
• ObjectQL.findOne already supports expand and applies the referenced object's RLS+FLS on the sub-read (objectql/src/engine.ts expandRelatedRecords, __expandRead path); the target object is inferred from the field's reference, so expand: { account: {} } is enough.
• The start-node config is an open passthrough (spec/automation/flow.zod.ts:102, z.record), and the engine already reads the start node (engine.ts:1655, startNode.config at 1666-1669). So a config.expand key round-trips with no schema change and is readable engine-side — without plumbing it onto AutomationContext (which deliberately carries only trigger data + identity, spec/contracts/automation-service.ts:23-69).

Approach — expand engine-side, as the triggering user

Do the lookup expansion in execute() after identity is resolved, reusing the user-scoped read path. This is the fix for the security blocker: the trigger's own re-read is system-elevated (record-change-trigger.ts:441) and intentionally does not expand (:344-355) — that path stays formula-only; expansion moves to where the user's grants live.

Formula hydration (trigger, system-scoped, own-row) and lookup expansion (engine, user-scoped, cross-object) stay separate layers that compose on the same record object. Do not unify them into one read — a user-scoped whole-record re-read would FLS-mask formula fields the trigger currently returns unmasked (a regression).

Phase 1 — Engine: inject a record expander + enrich the record

Mirror the #3356 userGrantsResolver injection to keep AutomationEngine decoupled from IDataEngine:

1. Add setRecordExpander(fn) + a RecordExpander type on the engine (beside setUserGrantsResolver, engine.ts:1135):

   type RecordExpander = (
     objectName: string, id: unknown, expandFields: string[], runContext: AutomationContext,
   ) => Promise<Record<string, unknown> | undefined>;

2. In resolveRunContext (engine.ts:1522-1574), after grants resolution and only for a record-change run with an id:
   • Read const expandFields = asStringArray((startNode?.config as any)?.expand) (fetch startNode here; resolveRunContext already receives flow).
   • If expandFields.length and this.recordExpander and runContext.record?.id:

     const full = await this.recordExpander(runContext.object, runContext.record.id, expandFields, runContext);
     if (full) for (const f of expandFields) if (f in full) (runContext.record as any)[f] = full[f];

   • Graft only the declared expand keys (keeps formula fields and [P2] record-change context does not hydrate multi-lookup (junction) fields #1872 multi-lookup arrays intact for non-expanded relations; raw-wins otherwise). Any expander failure → leave record unexpanded (never break the flow), matching the trigger's fail-safe posture.
   • Because runContext.record is the same object referenced by the variables map, no re-set/re-flatten is needed for the dotted {record.account.name} path.
3. Cover both execution paths. Putting the enrichment inside resolveRunContext covers execute() (engine.ts:1651) and, if it also routes through resolveRunContext, the no-retry twin executeWithoutRetry (engine.ts:2955, seeds $record at 2977). Verify the twin calls resolveRunContext; if not, factor a shared enrichRecordWithExpansions() and call it from both.

Phase 2 — Plugin bridge: implement the expander with the data engine

In service-automation/src/plugin.ts (beside the grants-resolver wiring at plugin.ts:508-525):

this.engine.setRecordExpander(async (objectName, id, expandFields, runContext) => {
  const dataCtx = resolveRunDataContext(runContext);           // user- or system-scoped by runAs
  const expand = Object.fromEntries(expandFields.map((f) => [f, {}]));
  return (await engineQl.findOne(objectName, { where: { id }, expand, context: dataCtx })) ?? undefined;
});

engineQl is already in scope here (it feeds the grants resolver). resolveRunDataContext lives in the same package (runtime-identity.ts). This is the single query per run when expand is declared (opt-in ⇒ zero cost otherwise).

Phase 3 — Lint consistency (@objectstack/lint, shipped in #3472)

Update validateFlowTemplatePaths (packages/lint/src/validate-flow-template-paths.ts) so flow-template-lookup-traversal is suppressed when the flow's start-node config.expand covers that relation head. Read expand off the start node (the validator already resolves boundObjectOf/start config), build a Set, and skip the finding when expandSet.has(head). Add a test asserting the warning disappears once expand: ['crm_account'] is declared.

Phase 4 (optional, discoverability) — typed start-node schema

There is currently no StartNodeConfigSchema (start uses the generic config). For Studio/AI discoverability, optionally introduce one with objectName/triggerType/condition/expand: string[]. Not required for function (the passthrough already works) — separate, low-priority follow-up.

Semantics & security (acceptance)

• Opt-in per relation. Only config.expand field names become nested objects; bare {record.<lookup>} ids and multiple lookup arrays ([P2] record-change context does not hydrate multi-lookup (junction) fields #1872) are untouched for relations not listed.
• A declared relation becomes an object — {record.account} now yields {…}, {record.account.name} resolves. Author opted in; document this.
• RLS/FLS respected. runAs:'user' ⇒ resolveRunDataContext supplies the user's grants ⇒ both the outer read and the expanded sub-read (__expandRead) enforce the triggering user's row/field security. A field the user can't read is masked — closing the leak that made the trigger-side approach unsafe. runAs:'system' ⇒ system-scoped, as intended.

Test plan

• service-automation engine: record-change flow with config.expand:['account'] + notify {record.account.name} → resolves; assert the injected expander was called with the run context and that under runAs:'user' the mapped data context carries the user's positions/permissions (not isSystem). Non-declared lookup stays a scalar id; multiple lookup array unaffected. runAs:'system' → expander receives a system context.
• Fail-safe: expander throws → flow still runs with an unexpanded (scalar-id) record.
• lint: flow-template-lookup-traversal suppressed when expand covers the relation; still fires otherwise.
• regression: formula hydration (formula-context.test.ts) unchanged — expansion must not alter formula-field values on record.

Out of scope

• Multi-hop (a.b.c) — single hop only (pass one expand level).
• Formula hydration's own system-read FLS question (tracked separately; not regressed here).