Share one prose matcher between both gates

docs/wiki/INDEX.md

@@ -13,6 +13,7 @@ Ideas about building software that have aged well across fifty years, for design
 - [Hide what's likely to change](engineering/information-hiding.md): how to split code into modules, and the wrong-abstraction trap. Reach for it before extracting shared code or designing an interface.
 - [Theory and documentation](engineering/theory-and-documentation.md): why the code alone can't carry a project's reasoning, and what to write down instead. Reach for it when deciding what a plan or record should say.
 - [Context engineering](engineering/context-engineering.md): keeping what the agent reads lean, honest, and re-tunable. Reach for it when a rules file grows, the model generation changes, or the process feels heavier than the work it governs.
+- [Clear technical writing](engineering/clear-technical-writing.md): how to explain technical work in plain English without deleting the terms the reader needs. Reach for it when writing for someone who doesn't share your context, or when a review flags unexplained language.
 
 ## Design fundamentals
 

docs/wiki/engineering/clear-technical-writing.md

@@ -0,0 +1,43 @@
+# Clear technical writing
+
+How to explain technical work in plain English without deleting the terms the reader genuinely needs. This page is for agents and people writing FOR someone who does not share their context: a project owner reading a work summary, a teammate catching up, a future maintainer reading a decision record.
+
+The problem it solves: technical writers default to compression. Compression is cheap for the writer and expensive for the reader, and when the reader is learning the system, an unexplained term doesn't just slow them down; it teaches them that they can't follow this project's writing. Clarity is a product feature of every explanation, not a style preference.
+
+## Where these ideas come from, and where they stop
+
+Six of the principles below are informed by ASD-STE100 (ASD Simplified Technical English), a controlled language developed for aerospace maintenance documentation, and by the plain-language and cognitive-accessibility guidance published by W3C and the UK Department for Education. A 1995 study by Shubert and colleagues found that complex aircraft procedures rewritten in Simplified English were easier to understand and to search, with essentially no change in task time; that is evidence for clearer procedural writing, not proof that any rule transfers everywhere.
+
+Two boundaries, stated plainly:
+
+- This page reproduces nothing from ASD-STE100: no rule text, no examples, no dictionary entries, no rule ordering. It claims no compliance, certification, approval, or endorsement. The standard itself says it was designed for procedural and descriptive technical documentation, not general correspondence. Treat these principles as borrowed judgment, not a checklist that makes writing "compliant."
+- Conversation is not a maintenance manual. Audience-aware vocabulary, answer-first structure, explaining a term the first time it appears, and formatting that helps a reader scan are all older and broader plain-language ideas; they come first, and the six principles below supplement them.
+
+## The six principles
+
+**1. Explain a necessary technical term, then use one stable name for it.** Plain language does not mean deleting the term the reader will meet again; it means paying for it once. Introduce the term with a short explanation in the reader's words, then call it by the same name every time after. What breaks understanding is not the term; it's three different names for one thing, or one name silently reused for two things.
+
+**2. Name the actor and the action.** "The cache is invalidated when the file changes" hides who does the invalidating. "The build script clears the cache when the file changes" tells the reader where to look when it goes wrong. Sentences where a named thing does a named action survive being read fast; sentences where things merely happen do not.
+
+**3. Put a condition first when the reader must know it before acting.** If an instruction only applies in some situation, the situation comes before the instruction: "If the server is still running, stop it before pulling." A reader executing steps top to bottom has already acted by the time a trailing condition arrives.
+
+**4. One main idea per sentence.** A sentence carrying two decisions invites the reader to keep one. Splitting is not dumbing down; it is choosing where the full stops go so each idea gets its own.
+
+**5. One topic per paragraph.** A paragraph is a promise that its sentences belong together. When a paragraph drifts from what happened to why to what's next, the reader loses the thread and rereads. Three short paragraphs beat one that braids three topics.
+
+**6. State the consequence of a warning.** "Don't run this against production" is weaker than "Don't run this against production: it rewrites every player record and there is no undo." A warning without its consequence reads as style; a warning with its consequence reads as physics.
+
+## What NOT to import
+
+Controlled languages come with machinery that does not transfer to conversational or explanatory writing, and importing it flattens the writer's voice for no comprehension gain:
+
+- Hard caps on sentence or paragraph length as pass/fail rules. Length advice is only worth having when measured against real writing from the project, and even then as advice, not as a gate. Readability formulas measure ease of decoding, not whether the reader understood the system.
+- Restricted vocabularies and banned grammatical forms (contractions, phrasal verbs, passive voice everywhere). These serve translation pipelines and regulatory review, not a reader learning a codebase from a trusted narrator.
+- Automated certainty. A checker can find a listed phrase; it cannot judge whether a sentence makes sense. The official guidance around Simplified Technical English itself warns that automated checkers are noisy assistants, not authorities. Whether the actor is named, whether the term was explained, whether the warning carries its consequence: these stay writing responsibilities.
+
+## Further reading
+
+- ASD-STE100 official site and FAQ: https://www.asd-ste100.org/STE_faq.html (scope and intent, in the maintainers' own words)
+- Shubert, Spyridakis, Holmback, Coney (1995), "The comprehensibility of Simplified English in procedures": https://journals.sagepub.com/doi/10.2190/WG69-D74B-4DLL-2WBK
+- W3C, writing for cognitive accessibility: https://www.w3.org/WAI/WCAG2/supplemental/objectives/o3-clear-content/
+- UK Department for Education, plain-language content design: https://design.education.gov.uk/content-design/plain-language

scripts/check-jargon.js

@@ -22,6 +22,10 @@
 // command source folder; plus comment text (line and block) in
 // scripts/ and tests/ JavaScript. String and template literals are
 // code, not prose: test fixtures quote listed phrases on purpose.
+// One markdown exception: a document wrapped whole in a single outer
+// ```markdown fence is a copy wrapper (handoff blocks travel this way),
+// so its contents are unwrapped and scanned as prose. The mechanics live
+// in scripts/prose-matcher.js, shared with the advisory voice gate.
 //
 // Known limits, on purpose: this is a best-effort comment scanner, not
 // a full JavaScript lexer. Two exotic constructs can fool it. First, a
@@ -44,44 +48,28 @@
 import { readdirSync, readFileSync, existsSync } from 'node:fs';
 import { dirname, join } from 'node:path';
 import { fileURLToPath } from 'node:url';
+import { loadPhraseList, compilePhrases, scanProse, matchLine } from './prose-matcher.js';
 
 // CHECK_ROOT lets the tests point the gate at a fixture tree.
 const root = process.env.CHECK_ROOT ?? join(dirname(fileURLToPath(import.meta.url)), '..');
 
-function loadListStrict() {
-  const listPath = join(root, 'scripts', 'phrase-list.json');
-  let parsed;
-  try {
-    parsed = JSON.parse(readFileSync(listPath, 'utf8'));
-  } catch (err) {
-    console.error(`[jargon] BROKEN LIST: scripts/phrase-list.json is unreadable or not valid JSON (${err.message}). The gate refuses to pass with a damaged config.`);
-    process.exit(1);
-  }
-  if (!Array.isArray(parsed)) {
-    console.error('[jargon] BROKEN LIST: scripts/phrase-list.json must be a flat array of {"bad", "good"} entries.');
-    process.exit(1);
-  }
-  const malformed = parsed.filter(
-    (e) => !e || typeof e !== 'object' || typeof e.bad !== 'string' || typeof e.good !== 'string'
-      || e.bad.trim() === '' || e.good.trim() === '' || Object.keys(e).length !== 2
-  );
-  if (malformed.length > 0) {
-    console.error(`[jargon] BROKEN LIST: ${malformed.length} entr(y/ies) not shaped {"bad": string, "good": string}:`);
-    for (const e of malformed) console.error(`  ${JSON.stringify(e)}`);
-    process.exit(1);
-  }
-  return parsed;
+// Matching semantics live in scripts/prose-matcher.js, shared with the
+// advisory voice gate. This gate's posture: a broken list fails LOUD
+// (exit 1) because a blocking gate degrading to silence would disable
+// enforcement with nobody told.
+let compiled;
+try {
+  compiled = compilePhrases(loadPhraseList(join(root, 'scripts', 'phrase-list.json')));
+} catch (err) {
+  console.error(`[jargon] BROKEN LIST: scripts/phrase-list.json — ${err.message} The gate refuses to pass with a damaged config.`);
+  process.exit(1);
 }
 
-const list = loadListStrict();
 const hits = [];
 
 function scanText(file, line, text) {
-  const lower = text.toLowerCase();
-  for (const { bad, good } of list) {
-    if (lower.includes(bad.toLowerCase())) {
-      hits.push(`${file}:${line}: "${bad}" -> try: "${good}"`);
-    }
+  for (const { bad, good } of matchLine(text, compiled)) {
+    hits.push(`${file}:${line}: "${bad}" -> try: "${good}"`);
   }
 }
 
@@ -115,19 +103,15 @@ function listJsFiles() {
   return files;
 }
 
-// Markdown: prose is what's outside fenced blocks and inline spans.
+// Markdown: prose is what's outside fenced blocks and inline spans; a
+// whole-document ```markdown copy wrapper (a handoff block) is unwrapped
+// and its contents scanned as prose. Both rules live in the shared
+// matcher so the advisory gate agrees.
 function scanMarkdown(file) {
   const content = readFileSync(join(root, file), 'utf8');
-  let inFence = false;
-  content.split('\n').forEach((text, idx) => {
-    if (/^\s*(```|~~~)/.test(text)) {
-      inFence = !inFence;
-      return;
-    }
-    if (inFence) return;
-    const prose = text.replace(/`[^`]*`/g, ' ');
-    scanText(file, idx + 1, prose);
-  });
+  for (const { line, bad, good } of scanProse(content, compiled)) {
+    hits.push(`${file}:${line}: "${bad}" -> try: "${good}"`);
+  }
 }
 
 // JavaScript: comment text only. States: code, line comment, block
@@ -196,4 +180,4 @@ if (hits.length > 0) {
   console.error(`[jargon] ${hits.length} listed phrase(s) in committed prose. Rewrite the prose; never widen an exclusion to dodge the gate.`);
   process.exit(1);
 }
-console.log(`[jargon] OK: no listed phrases in committed prose (${list.length} phrases checked).`);
+console.log(`[jargon] OK: no listed phrases in committed prose (${compiled.length} phrases checked).`);

scripts/prose-matcher.js

@@ -0,0 +1,155 @@
+// Shared prose matcher. One home for how Foundry's two gates find listed
+// phrases in human-facing text: the advisory draft gate (voice-gate.js)
+// and the blocking committed-prose gate (check-jargon.js) both build on
+// these helpers, so matching semantics cannot drift between them.
+//
+// The semantics, in plain terms:
+//
+//   - Matching is case-insensitive.
+//   - A phrase that starts or ends with a letter or digit only matches at
+//     word edges: a listed "zorbly flux" is found in "the zorbly flux
+//     case" but not inside "mezorbly fluxative". A phrase whose edge is
+//     punctuation (the em dash entry, for example) matches as a plain
+//     substring, because its neighbours are legitimately letters.
+//   - Markdown code is not prose. Fenced blocks and inline `code` spans
+//     are skipped. One exception: a document wrapped WHOLE in a single
+//     outer ```markdown (or ```md) fence is a copy wrapper, not a code
+//     sample, so its contents are unwrapped and scanned normally
+//     (handoff blocks travel this way). Fences inside the unwrapped body
+//     count as real code again.
+//   - Every hit reports its 1-based line number in the ORIGINAL text,
+//     so a hit inside an unwrapped handoff still points at the real line.
+//
+// List loading is strict and throws: an unreadable file, a non-array
+// root, or a malformed entry raises an Error naming the problem. Each
+// caller decides what failing closed means for its posture (the blocking
+// gate exits 1; the advisory gate reports the broken list loudly and
+// never prints a clean verdict).
+
+import { readFileSync } from 'node:fs';
+
+// ─── list loading ────────────────────────────────────────────────────────
+
+export function assertValidPhraseList(parsed) {
+  if (!Array.isArray(parsed)) {
+    throw new Error('phrase list must be a flat array of {"bad", "good"} entries.');
+  }
+  const malformed = parsed.filter(
+    (e) => !e || typeof e !== 'object' || typeof e.bad !== 'string' || typeof e.good !== 'string'
+      || e.bad.trim() === '' || e.good.trim() === '' || Object.keys(e).length !== 2
+  );
+  if (malformed.length > 0) {
+    const listing = malformed.map((e) => JSON.stringify(e)).join('\n  ');
+    throw new Error(
+      `${malformed.length} entr(y/ies) not shaped {"bad": string, "good": string}:\n  ${listing}`
+    );
+  }
+  return parsed;
+}
+
+export function loadPhraseList(path) {
+  let parsed;
+  try {
+    parsed = JSON.parse(readFileSync(path, 'utf8'));
+  } catch (err) {
+    throw new Error(`phrase list is unreadable or not valid JSON (${err.message}).`);
+  }
+  return assertValidPhraseList(parsed);
+}
+
+// ─── phrase compilation ──────────────────────────────────────────────────
+
+const RE_SPECIALS = /[.*+?^${}()|[\]\\]/g;
+const WORD_EDGE = /^[\p{L}\p{N}]/u;
+const WORD_EDGE_END = /[\p{L}\p{N}]$/u;
+
+// Word boundaries only guard edges that are themselves word characters;
+// a punctuation-edged phrase keeps substring semantics on that side.
+export function compilePhrases(list) {
+  return list.map(({ bad, good }) => {
+    const escaped = bad.replace(RE_SPECIALS, '\\$&');
+    const lead = WORD_EDGE.test(bad) ? '(?<![\\p{L}\\p{N}])' : '';
+    const tail = WORD_EDGE_END.test(bad) ? '(?![\\p{L}\\p{N}])' : '';
+    return { bad, good, re: new RegExp(`${lead}${escaped}${tail}`, 'iu') };
+  });
+}
+
+// ─── markdown region extraction ──────────────────────────────────────────
+
+const FENCE_LINE = /^\s*(`{3,}|~{3,})(.*)$/;
+const OUTER_OPEN = /^(`{3,}|~{3,})\s*(markdown|md)\s*$/i;
+const OUTER_CLOSE = /^(`{3,}|~{3,})\s*$/;
+
+// A whole-document ```markdown wrapper is a copy wrapper, not code.
+// Detection is deliberately narrow: the FIRST non-empty line opens the
+// fence with a markdown/md info string, and the LAST non-empty line
+// closes it with the same character and at least the same run length.
+// Anything less exact stays a normal code fence.
+export function unwrapOuterMarkdownFence(text) {
+  const lines = text.split('\n');
+  let first = 0;
+  while (first < lines.length && lines[first].trim() === '') first += 1;
+  let last = lines.length - 1;
+  while (last >= 0 && lines[last].trim() === '') last -= 1;
+  if (first >= last) return { lines, offset: 0, unwrapped: false };
+  const open = lines[first].trim().match(OUTER_OPEN);
+  const close = lines[last].trim().match(OUTER_CLOSE);
+  if (
+    open && close
+    && close[1][0] === open[1][0]
+    && close[1].length >= open[1].length
+  ) {
+    return { lines: lines.slice(first + 1, last), offset: first + 1, unwrapped: true };
+  }
+  return { lines, offset: 0, unwrapped: false };
+}
+
+// Returns [{ line, text }] for prose lines only, with inline code spans
+// blanked (spaces preserve column positions). `line` is 1-based against
+// the original text, including any unwrapped outer fence.
+export function extractProseLines(text) {
+  const { lines, offset } = unwrapOuterMarkdownFence(text);
+  const out = [];
+  let fence = null;
+  lines.forEach((raw, i) => {
+    const m = raw.match(FENCE_LINE);
+    if (m) {
+      if (!fence) {
+        fence = { char: m[1][0], len: m[1].length };
+      } else if (m[1][0] === fence.char && m[1].length >= fence.len && m[2].trim() === '') {
+        fence = null;
+      }
+      return;
+    }
+    if (fence) return;
+    out.push({
+      line: offset + i + 1,
+      text: raw.replace(/`[^`]*`/g, (s) => ' '.repeat(s.length)),
+    });
+  });
+  return out;
+}
+
+// ─── matching ────────────────────────────────────────────────────────────
+
+// One hit per phrase per line: enough to point at the line, and it keeps
+// output stable when a phrase repeats within a line.
+export function matchLine(lineText, compiled) {
+  const hits = [];
+  for (const { bad, good, re } of compiled) {
+    if (re.test(lineText)) hits.push({ bad, good });
+  }
+  return hits;
+}
+
+// Markdown-aware scan: unwrap, strip code, match. Returns
+// [{ line, bad, good }] with original 1-based line numbers.
+export function scanProse(text, compiled) {
+  const hits = [];
+  for (const { line, text: prose } of extractProseLines(text)) {
+    for (const hit of matchLine(prose, compiled)) {
+      hits.push({ line, ...hit });
+    }
+  }
+  return hits;
+}