System Prompt Lost During Context Compaction

[AndyLee1024]

Summary

The context compaction feature discards the system prompt when compressing conversation history, causing the agent to lose its core instructions after compaction occurs.

Root Cause

In bu_agent_sdk/agent/compaction/service.py, the check_and_compact method replaces the entire message history with only a summary UserMessage:

async def check_and_compact(
    self,
    messages: list[BaseMessage],
    llm: BaseChatModel | None = None,
) -> tuple[list[BaseMessage], CompactionResult]:
    # ...omitted code...
    
    result = await self.compact(messages, llm)

    # Replace entire history with summary as a user message
    # This matches the Anthropic SDK behavior
    new_messages: list[BaseMessage] = [
        UserMessage(content=result.summary or ""),
    ]

    return new_messages, result

Problem: This discards the SystemMessage that was added during initialization.

Impact

What Works ✅

• The system prompt is included when generating the summary (because it's in the messages list passed to the LLM)

What's Broken ❌

• The system prompt is lost from the message history after compaction
• All subsequent conversations lack the system prompt's guidance
• The agent's behavior changes unexpectedly after compaction triggers

Reproduction Flow

1. Agent Initialization (bu_agent_sdk/agent/service.py):

async def query(self, message: str) -> str:
    # Add system prompt on first message
    if not self._messages and self.system_prompt:
        self._messages.append(SystemMessage(content=self.system_prompt, cache=True))

2. Normal Operation:

   • Messages: [SystemMessage, UserMessage, AssistantMessage, ...]

3. After Compaction Triggers:

async def _check_and_compact(self, response: ChatInvokeCompletion) -> bool:
    # ...
    new_messages, result = await self._compaction_service.check_and_compact(
        self._messages,  # Contains SystemMessage
        self.llm,
    )

    if result.compacted:
        self._messages = list(new_messages)  # ⚠️ SystemMessage is lost here!
        return True

4. Result:
   • Messages: [UserMessage(summary)]
   • SystemMessage is gone!

Proposed Fix

Preserve SystemMessage and DeveloperMessage during compaction:

async def check_and_compact(
    self,
    messages: list[BaseMessage],
    llm: BaseChatModel | None = None,
) -> tuple[list[BaseMessage], CompactionResult]:
    model = llm or self.llm
    if model is None:
        return messages, CompactionResult(compacted=False)

    if not await self.should_compact(model.model):
        return messages, CompactionResult(compacted=False)

    result = await self.compact(messages, llm)

    # Preserve system/developer messages from the original history
    system_messages = [
        msg for msg in messages 
        if isinstance(msg, (SystemMessage, DeveloperMessage))
    ]
    
    # Combine preserved system messages with summary
    new_messages: list[BaseMessage] = [
        *system_messages,
        UserMessage(content=result.summary or ""),
    ]

    return new_messages, result

Additional Considerations

1. Message Import: Need to import DeveloperMessage in service.py:

from bu_agent_sdk.llm.messages import (
    AssistantMessage,
    BaseMessage,
    DeveloperMessage,  # Add this
    SystemMessage,
    UserMessage,
)

2. Alternative Approach: If system messages should be re-added by the Agent class rather than preserved during compaction, the logic could be moved to _check_and_compact in agent/service.py.

Files Affected

• bu_agent_sdk/agent/compaction/service.py - Primary fix location
• bu_agent_sdk/agent/service.py - Uses compaction service