Split system prompt into static/dynamic blocks for better caching

[sestinj]

Summary

• Restructure constructSystemMessage() to return an array of SystemMessageBlock[] instead of a single string, separating static content (core identity/instructions) from dynamic content (environment info, git status)
• Static content in block 1 is identical across all users/projects, enabling Anthropic prompt cache hits globally. Semi-static user rules go in block 2, and per-session dynamic content goes in block 3
• The Anthropic API adapter and AnthropicCachingStrategies.ts already handle system as an array of blocks with independent cache_control, so this change automatically improves cache utilization without API-side changes

Test plan

• All 25 systemMessage.test.ts tests pass, including new tests for block structure
• Updated all downstream consumers (streamChatResponse, compaction helpers, subagent executor, serve command) to handle the new array format
• Updated all test mocks to use block format
• Pre-existing test failures (yaml module resolution) confirmed to exist on main branch

🤖 Generated with Continue

---

Continue Tasks: ❌ 7 failed — View all

---

Summary by cubic

Split the system prompt into static, rules, and dynamic blocks to improve Anthropic prompt cache hits and reduce prompt tokens. Removed the directory tree context, added prompt cache telemetry, and updated services/tests to handle blocks.

• New Features

  • constructSystemMessage returns SystemMessageBlock[] (static → user rules → dynamic).
  • Added flattenSystemMessage for places that need a single string.
  • convertFromUnifiedHistoryWithSystemMessage accepts a string or block array.
  • PostHog telemetry now includes cache_read_tokens, cache_write_tokens, and a prompt_cache_metrics event.

• Refactors

  • Updated CLI services, streaming, compaction helpers, subagent executor, and tests to use blocks; flatten to string for token counting and subagents.
  • Removed directory tree from the prompt; kept git status and environment info.
  • Fixed auto compaction nullish check and lint rule violations; cleaned up unused directives and formatting.
  • Updated Vitest global mocks to return blocks and expose flattenSystemMessage.

^{Written for commit 400c329. Summary will update on new commits.}

[cubic-dev-ai]

1 issue found across 17 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="extensions/cli/src/stream/streamChatResponse.autoCompaction.ts">

<violation number="1" location="extensions/cli/src/stream/streamChatResponse.autoCompaction.ts:167">
P2: The new truthy check skips an explicitly provided empty system message. This changes behavior from the previous nullish-coalescing logic and can override a deliberate empty string with the default system message. Use a nullish check instead.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}