Skip to content

Commit d6ebb4b

Browse files
brannnclaude
andcommitted
Strengthen planner system prompts to prevent common linter errors
Fixes recurring issues where generated specs had: - Landmarks (RULES, DONE_WHEN, etc.) appearing outside FUNCTION blocks - Undefined DATA types referenced in FUNCTION signatures Changes: 1. Added CRITICAL STRUCTURE RULE emphasizing landmarks must be inside FUNCTION blocks 2. Made DATA type definition requirement explicit (must define before use) 3. Clarified top-level vs FUNCTION-level landmark hierarchy 4. Improved refinement prompt to explain Simplex structure upfront 5. Added reminder in brownfield fragment about DATA type definitions The prompt now explicitly states: - Top-level: only DATA and CONSTRAINT - Inside FUNCTION: all required landmarks (RULES, DONE_WHEN, EXAMPLES, ERRORS) - Custom types in signatures require DATA blocks defined first This should eliminate W001 (landmarks outside FUNCTION) and W006 (undefined DATA type) errors in planner-generated specifications. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
1 parent f0dd527 commit d6ebb4b

1 file changed

Lines changed: 40 additions & 27 deletions

File tree

web/cmd/server/static/js/simplex-spec.js

Lines changed: 40 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -307,18 +307,24 @@ ${COMPLEX_EXAMPLE}
307307
308308
Based on the user's description and any refinement conversation, generate a complete Simplex specification that:
309309
310-
1. Contains at least one FUNCTION block with all required landmarks (RULES, DONE_WHEN, EXAMPLES, ERRORS)
311-
2. Uses DATA blocks when custom types would clarify the specification
312-
3. Includes CONSTRAINT blocks for important invariants
313-
4. Provides enough EXAMPLES to cover every conditional branch in RULES (this is a strict requirement — the linter will reject specs where the number of examples is less than the number of conditional branches)
314-
5. Always includes a catch-all error handler: "any unhandled condition → fail with descriptive message"
315-
6. Stays within complexity limits:
310+
1. **CRITICAL STRUCTURE RULE**: All required landmarks (RULES, DONE_WHEN, EXAMPLES, ERRORS) MUST appear INSIDE a FUNCTION block, immediately after the FUNCTION declaration. They are NOT top-level landmarks. See the reference examples above for correct structure.
311+
312+
2. **Top-level landmarks**: Only DATA and CONSTRAINT appear at the top level (outside FUNCTION blocks). Everything else goes inside FUNCTION blocks.
313+
314+
3. **DATA types**: When using custom types in FUNCTION signatures (like "updated cart" or "PolicyRule"), you MUST define them with a DATA block BEFORE the FUNCTION that uses them. Never reference undefined types.
315+
316+
4. **Example coverage**: Provide enough EXAMPLES to cover every conditional branch in RULES. The linter will reject specs where example count is less than conditional branch count.
317+
318+
5. **Error handling**: Always include the catch-all error handler: "any unhandled condition → fail with descriptive message"
319+
320+
6. **Complexity limits**:
316321
- Maximum 15 RULES items per function
317322
- Maximum 6 function inputs
318323
- Maximum 200 characters per single rule item
319324
- If more complex, decompose into multiple functions
320325
321-
Generate ONLY the specification. No explanations, no markdown code blocks, no surrounding text. Output the raw Simplex spec text directly.`;
326+
**Output format**: Generate ONLY the specification. No explanations, no markdown code blocks, no surrounding text. Output the raw Simplex spec text directly.`;
327+
322328

323329
// Appended to BASE_SYSTEM_PROMPT when brownfield project type is selected
324330
const BROWNFIELD_PROMPT_FRAGMENT = `
@@ -327,12 +333,12 @@ const BROWNFIELD_PROMPT_FRAGMENT = `
327333
328334
This is an EVOLUTIONARY specification for an existing codebase. You MUST include:
329335
330-
1. **BASELINE** landmark with all three required fields:
336+
1. **BASELINE** landmark (inside FUNCTION block) with all three required fields:
331337
- reference: description or pointer to the current/prior state being evolved
332338
- preserve: list of behaviors, contracts, or APIs that must NOT regress
333339
- evolve: list of capabilities being added or changed
334340
335-
2. **EVAL** landmark with all three required fields:
341+
2. **EVAL** landmark (inside FUNCTION block) with all three required fields:
336342
- preserve: pass^k threshold (e.g., pass^3 means all 3 trials must pass - for regression testing)
337343
- evolve: pass@k threshold (e.g., pass@5 means at least 1 of 5 must pass - for capability testing)
338344
- grading: code | model | outcome
@@ -341,6 +347,8 @@ This is an EVOLUTIONARY specification for an existing codebase. You MUST include
341347
- Preserved behaviors (regression tests) - these verify nothing breaks
342348
- Evolved capabilities (capability tests) - these verify new functionality
343349
350+
4. **Remember**: Define any custom types (like "AuthSystem") with DATA blocks BEFORE using them in FUNCTION signatures.
351+
344352
Here is an evolutionary specification example:
345353
\`\`\`
346354
${EVOLUTION_EXAMPLE}
@@ -349,24 +357,29 @@ ${EVOLUTION_EXAMPLE}
349357
// System prompt for refinement (used with temperature 0.7, max_tokens 1024)
350358
const REFINE_SYSTEM_PROMPT = `You are a helpful assistant that helps users refine their software requirements into clear, testable specifications for the Simplex specification (v0.5).
351359
352-
Simplex specs have these required landmarks:
353-
- FUNCTION: The name, inputs, and return type
354-
- RULES: Business logic and behavioral constraints
355-
- DONE_WHEN: Observable completion criteria
356-
- EXAMPLES: Input/output pairs showing expected behavior
357-
- ERRORS: Error conditions and how they should be handled
358-
359-
Optional landmarks include:
360-
- DATA: Type definitions (use when custom types clarify the spec)
361-
- CONSTRAINT: Global invariants
362-
- BASELINE/EVAL: For evolutionary specs (modifying existing systems)
363-
- DETERMINISM: Output variance control (level: strict | structural | semantic)
364-
365-
Your job is to ask clarifying questions to gather enough information to generate a complete spec. Focus on:
366-
1. Data types and structures needed
367-
2. Business rules and constraints
368-
3. Edge cases and error conditions
369-
4. Concrete examples of expected behavior
360+
## Simplex Structure
361+
362+
A Simplex spec has this hierarchy:
363+
- **Top-level**: DATA (type definitions) and CONSTRAINT (global invariants)
364+
- **FUNCTION blocks** contain:
365+
- FUNCTION: name(inputs) → return_type
366+
- RULES: behavioral specification (outcomes, not steps)
367+
- DONE_WHEN: observable completion criteria
368+
- EXAMPLES: concrete input/output pairs (must cover every RULES branch)
369+
- ERRORS: error conditions and responses
370+
- Optional: READS, WRITES, TRIGGERS, NOT_ALLOWED, HANDOFF, UNCERTAIN, DETERMINISM
371+
372+
For evolutionary specs (modifying existing systems):
373+
- BASELINE: reference (prior state), preserve (unchanged behaviors), evolve (new capabilities)
374+
- EVAL: preserve (pass^k), evolve (pass@k), grading (code/model/outcome)
375+
376+
## Your Job
377+
378+
Ask clarifying questions to gather enough information to generate a complete spec. Focus on:
379+
1. **What types are needed?** (Will become DATA blocks if not obvious primitive types)
380+
2. **What are the behavioral rules?** (Outcomes, not implementation steps)
381+
3. **What are the edge cases?** (Every conditional needs an example)
382+
4. **How should errors be handled?** (Specific conditions and responses)
370383
371384
Be conversational but focused. Ask 2-3 questions at a time. When you have enough information (usually after 3-4 exchanges), let the user know they can proceed to generate the spec.
372385

0 commit comments

Comments
 (0)