From 448700c52f5b28afb8022cdfbad47c34aa2d7906 Mon Sep 17 00:00:00 2001 From: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Date: Mon, 27 Jul 2026 22:26:01 +0800 Subject: [PATCH] docs(liveness): stop counting authoring/preview renderers as runtime consumers (#1878) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A 2026-06 sweep that "missed objectui" over-corrected and marked 13 properties `live` on the evidence of a metadata-admin previews/*Preview.tsx panel — a surface that only echoes what the author typed. `live` is supposed to mean "authoring this changes runtime behaviour". Three of the 13 were re-verified in the §3 naming-drift recheck; all three were wrong or misleading: - skill.permissions -> dead + authorWarn. The cloud SkillRegistry reads only active/triggerConditions/tools; nothing gates invocation by this list. Mirrors the identical tool.permissions entry. (Cloud documents the intent in a comment without implementing it.) - agent.knowledge -> dead + authorWarn. search_knowledge takes sourceIds from the LLM's tool-call arguments, never from the agent record, so a declared knowledge scope constrains nothing (and an agent can search sources it was never scoped to). - action.disabled: verdict kept (it IS live now) but evidence corrected to the six rendering surfaces. At ledger-writing time it was enforced on ONE of six, so the preview-based verdict hid a real five-surface silent no-op — fixed in objectui#2863. Also: sync the hand-maintained count table (agent 14/5/0 -> 13/5/1, skill 10/-/- -> 9/-/1) and add a "an authoring/preview renderer is NOT a runtime consumer" section naming the ten still-unverified preview-only claims, with the tie-break rule: when in doubt prefer dead + authorWarn — a warning on a property that works costs nothing; silence on a property that does nothing ships a bug. check:liveness green (exit 0); the 13 unregistered-proof warnings are pre-existing on main. Co-Authored-By: Claude Fable 5 --- .../liveness-ledger-ai-scope-honesty.md | 12 +++++++++ packages/spec/liveness/README.md | 27 +++++++++++++++++-- packages/spec/liveness/action.json | 4 +-- packages/spec/liveness/agent.json | 8 +++--- packages/spec/liveness/skill.json | 8 +++--- 5 files changed, 49 insertions(+), 10 deletions(-) create mode 100644 .changeset/liveness-ledger-ai-scope-honesty.md diff --git a/.changeset/liveness-ledger-ai-scope-honesty.md b/.changeset/liveness-ledger-ai-scope-honesty.md new file mode 100644 index 0000000000..4a62181494 --- /dev/null +++ b/.changeset/liveness-ledger-ai-scope-honesty.md @@ -0,0 +1,12 @@ +--- +--- + +docs(liveness): correct three ledger entries that counted an authoring/preview renderer as a runtime consumer (#1878 §3 recheck) + +`skill.permissions` and `agent.knowledge` were marked `live` citing only a +metadata-admin PREVIEW panel — which echoes what the author typed and proves +nothing about enforcement. Both are corrected to `dead` + `authorWarn` with an +actionable hint. `action.disabled`'s evidence is corrected to the six real +rendering surfaces (its verdict was right for the wrong reason and hid a +five-surface silent no-op, since fixed in objectui#2863). Ledger-only; releases +nothing. diff --git a/packages/spec/liveness/README.md b/packages/spec/liveness/README.md index 79097046fd..4c12057b77 100644 --- a/packages/spec/liveness/README.md +++ b/packages/spec/liveness/README.md @@ -46,6 +46,29 @@ Resolution per property: **ledger entry → spec `.describe()` marker → UNCLAS Framework provenance/lock fields (`_lock*`, `_provenance`, `_packageId/Version`, `protection` — ADR-0010) are auto-classified `live`. +### ⚠️ An authoring/preview renderer is NOT a runtime consumer + +`live` means **authoring the property changes runtime behaviour**. A Studio +`*.form.ts` input or a `metadata-admin/previews/*Preview.tsx` panel merely +*echoes back what the author typed* — it proves the property round-trips, never +that anything acts on it. A 2026-06 sweep that "missed objectui" over-corrected +and marked **13 properties `live` citing only a preview renderer**; of those, +three were re-verified in 2026-07 and **all three were wrong or misleading**: + +- `skill.permissions` — no gate anywhere (corrected → `dead` + `authorWarn`) +- `agent.knowledge` — retrieval scope comes from the LLM's tool args (corrected → `dead`) +- `action.disabled` — enforced on **one of six** rendering surfaces at the time; + the verdict was right for the wrong reason and hid a five-surface silent + no-op (evidence corrected; the gap itself fixed in objectui#2863) + +The remaining ten preview-only claims (`action.execute`/`shortcut`/`bulkEnabled`, +`flow.status`/`active`, `skill.triggerPhrases`, `tool.category`/ +`requiresConfirmation`/`active`/`builtIn`) are **unverified** — treat them as +suspect until someone cites a real runtime reader. When in doubt, the honest +status is `dead` + `authorWarn`: an author who gets a warning for a property +that turns out to work loses nothing; an author who gets silence for a property +that does nothing ships a bug. + ## Runtime proofs — prove-it-runs (ADR-0054) `live` today means only *a static pointer to a consumer* — proof that something @@ -207,9 +230,9 @@ EOF | hook | 11 | – | 2 | – | model-healthy; only label/description dead (benign) | | permission | 32 | – | 0 | – | CRUD/FLS/RLS live; dead `contextVariables` REMOVED (ADR-0105 D11 — RLS resolves only the `current_user.*` built-ins plus runtime-staged `rlsMembership` sets) | | position | 4 | – | – | – | (role's ADR-0090 successor) fully live | -| agent | 14 | 5 | 0 | – | dead `tenantId` + `planning.strategy`/`allowReplan` REMOVED (#2377) — only `planning.maxIterations` live; autonomy tier experimental | +| agent | 13 | 5 | 1 | – | dead `tenantId` + `planning.strategy`/`allowReplan` REMOVED (#2377) — only `planning.maxIterations` live; autonomy tier experimental; `knowledge` CORRECTED to dead 2026-07 — `search_knowledge` takes `sourceIds` from the LLM's tool-call args, never from the agent record (#1878 §3 recheck) | | tool | 9 | 1 | 1 | – | `permissions` dead — tool invocation not permission-gated by it | -| skill | 10 | – | – | – | fully live | +| skill | 9 | – | 1 | – | `permissions` CORRECTED to dead 2026-07 — the cloud SkillRegistry reads only `active`/`triggerConditions`/`tools`; mirrors the identical `tool.permissions` entry (#1878 §3 recheck) | | dataset | 19 | – | 0 | – | `measures.certified` (declared-but-unenforced governance flag) REMOVED in 16.0 (#2377) | | page | 16 | – | – | 1 | fully live + one planned | | view | 70 | 0 | 5 | – | list/form drilled via `children` (#2998 Track B); dead = list.{responsive,performance} + form.{data,defaultSort,aria}, all but aria authorWarn'd; form.{buttons,defaults} now live — objectui ObjectForm folds them onto its flat props (framework#1894 / #2998); audit-era DEAD lines superseded by re-verification (submitBehavior, sharing.lockedBy, list ViewData providers, and the ADR-0021 chart shape — all live now); level-2 dead residue (userActions.buttons, addRecord.mode/formView, tabs[].order) noted on parents — one drill level only | diff --git a/packages/spec/liveness/action.json b/packages/spec/liveness/action.json index 7965e896ee..9ecb3e23e6 100644 --- a/packages/spec/liveness/action.json +++ b/packages/spec/liveness/action.json @@ -101,8 +101,8 @@ }, "disabled": { "status": "live", - "evidence": "objectui: packages/app-shell/src/views/metadata-admin/previews/ActionPreview.tsx:169", - "note": "LIVE via objectui metadata-admin authoring UI — the 2026-06 audit missed objectui; verified by reading the file." + "evidence": "objectui: components action-button/-group/-icon/-menu + app-shell DeclaredActionsBar + plugin-detail record-quick-actions — all six rendering surfaces gate the Button on the predicate", + "note": "EVIDENCE CORRECTED 2026-07 (#1878 §3 recheck): previously cited the metadata-admin PREVIEW renderer, which only echoes the authored value. At that time the predicate was actually enforced on ONE of six surfaces (#1885 wired action:button alone) — the 'live' verdict was right for the wrong reason and hid a real silent no-op on the other five. objectui#2863 wired the rest; showcase specimen `showcase_archive_task` (#3643) dogfoods it." }, "shortcut": { "status": "live", diff --git a/packages/spec/liveness/agent.json b/packages/spec/liveness/agent.json index 2d8be5ce38..8ec2682972 100644 --- a/packages/spec/liveness/agent.json +++ b/packages/spec/liveness/agent.json @@ -65,9 +65,11 @@ "note": "evaluateAgentAccess() — caller must hold ALL required permissions/roles; enforced at the chat route. #1884." }, "knowledge": { - "status": "live", - "evidence": "objectui: packages/app-shell/src/views/metadata-admin/previews/AgentPreview.tsx:69 (d.knowledge; KnowledgeSummary renders sources/indexes)", - "note": "LIVE via objectui metadata-admin authoring UI — the 2026-06 audit missed objectui; verified by reading the file." + "status": "dead", + "evidence": "agent.form.ts + objectui AgentPreview.tsx:69 (both authoring-side echo only); the RAG path never reads it", + "authorWarn": true, + "authorHint": "Declaring `knowledge.sources` does NOT scope retrieval — the `search_knowledge` tool takes `sourceIds` from the LLM's tool-call arguments (cloud service-ai/src/tools/knowledge-tools.ts:96), never from the agent record. Restrict retrieval at the knowledge-service / source level instead.", + "note": "CORRECTED 2026-07 (#1878 §3 recheck): was marked live on the evidence of the metadata-admin PREVIEW renderer — an echo of what the author typed, not runtime consumption. The `topics`→`sources` NAMING drift is genuinely fixed (#1891 alias fold); the block is nonetheless behaviourally inert. Wiring it would also close a scope gap (an agent can search sources it was never scoped to) — tracked separately." }, "lifecycle": { "status": "experimental", diff --git a/packages/spec/liveness/skill.json b/packages/spec/liveness/skill.json index 673b95cd51..a57aad0c45 100644 --- a/packages/spec/liveness/skill.json +++ b/packages/spec/liveness/skill.json @@ -47,9 +47,11 @@ "note": "LIVE via objectui metadata-admin authoring UI — the 2026-06 audit missed objectui; verified by reading the file." }, "permissions": { - "status": "live", - "evidence": "objectui: packages/app-shell/src/views/metadata-admin/previews/SkillPreview.tsx:48", - "note": "LIVE via objectui metadata-admin authoring UI — the 2026-06 audit missed objectui; verified by reading the file." + "status": "dead", + "evidence": "skill.form.ts + objectui SkillPreview.tsx:48 (both authoring-side echo only); no gate in the cloud SkillRegistry", + "authorWarn": true, + "authorHint": "Skill invocation is NOT permission-gated by this list — the registry reads only `active`/`triggerConditions`/`tools`. Agent-level `access`/`permissions` ARE enforced (#1884), so gate at the agent, or gate the underlying tools' actions via permission sets (ADR-0066).", + "note": "CORRECTED 2026-07 (#1878 §3 recheck): was marked live on the evidence of the metadata-admin PREVIEW renderer — an echo of what the author typed, not runtime consumption. Mirrors the identical `tool.permissions` entry. Cloud documents the intent ('Layer permissions via Skill.permissions independent of the agent's permissions', service-ai-studio/src/skills/metadata-authoring-skill.ts:15) without implementing it — enforce-or-prune tracked separately." } } }