Skip to content

fix: reorder fake tool call messages at tail to correct timing - #9451

Open
Rail1bc wants to merge 1 commit into
AstrBotDevs:masterfrom
Rail1bc:fix/fake-tool-call-reorder
Open

fix: reorder fake tool call messages at tail to correct timing#9451
Rail1bc wants to merge 1 commit into
AstrBotDevs:masterfrom
Rail1bc:fix/fake-tool-call-reorder

Conversation

@Rail1bc

@Rail1bc Rail1bc commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Motivation / 动机

通过向 req.contexts 尾部注入 assistant(tool_calls) → tool(result) 消息对,可以强制 LLM 认为自己已经调用了某个工具并拿到了结果,从而:

  • 跳过 LLM 自身的工具调用决策,节省 token
  • 降低延迟(不需要等 LLM 生成工具调用)
  • 增加确定性(固定或规则触发,不受模型输出波动影响,增强确定性)

这是一种通用手法,已在 LivingMemory 插件中长期记忆注入中使用,未来可能被更多插件用于各种确定性工具调用场景。

但在 OpenAI_Provider._prepare_chat_payload() 中,处理顺序是:

  1. deepcopy(contexts) —— 此时尾部已含伪造工具调用对
  2. append(new_record) —— 追加当前用户消息

最终发给 LLM API 的 messages 序列为:

... assistant(tool_calls=...), tool(result), user("...")

LLM 视角下,assistant 在用户提问之前就"预知"了查询内容。正确顺序应为:

... user("..."), assistant(tool_calls=...), tool(result)

该问题影响所有使用该手法的插件,以及所有走 OpenAI_Provider 的 LLM 路径

Refs: #9450

Modifications / 改动点

astrbot/core/provider/sources/openai_source.py

_sanitize_assistant_messages() 末尾新增检测与重排逻辑:

  1. 从尾部弹出 user 消息
  2. 用 while 循环收集所有 assistant(tool_calls) + tool 成对消息(验证 tool_call_id 匹配)
  3. 重排为 user → assistant₁, tool₁ → ... → assistant_N, tool_N

支持多插件各自注入多轮伪造对的场景。

  • 不依赖具体插件实现,通用模式检测
  • 不改动 req 对象,不破坏其他 on_llm_request handler
  • 覆盖 _query / _query_streaming 全部路径
  • 真实工具调用场景下 tool 后不会直接跟 user,不会误杀

此为轻量妥协修复。未来若实现专用的上下文操作钩子或伪造工具调用钩子,可考虑移除此方法。

  • This is NOT a breaking change. / 这不是一个破坏性变更。

Test Results / 测试结果

逻辑验证(手动):

  • 无伪造对:不触发,不变
  • 单对:..., asst(tc), tool, user..., user, asst(tc), tool
  • 多对:..., asst₁, tool₁, asst₂, tool₂, user..., user, asst₁, tool₁, asst₂, tool₂
  • 真实工具调用:..., asst(tc), tool, asst(content), user → 不变,不误杀

Checklist / 检查清单

  • 😊 新功能已通过 Issue 与作者讨论
  • 👀 我的更改经过了充分测试,测试结果已在上方提供
  • 🤓 无新依赖引入
  • 😮 无恶意代码

Summary by Sourcery

Ensure OpenAI provider sends user messages before any injected fake tool-call assistant/tool pairs to maintain correct conversation chronology.

Bug Fixes:

  • Fix misordered tail messages where injected assistant(tool_calls) and tool results appeared before the latest user message, causing incorrect tool-call timing perception by the LLM.

Enhancements:

  • Add a generic post-processing step that detects and reorders trailing assistant(tool_calls)/tool message pairs so they follow the latest user message, supporting multiple plugins and multiple injected pairs without affecting real tool calls.

…nAI provider

When plugins inject assistant(tool_calls) + tool(result) pairs into
contexts to force tool invocation (fake tool call),
_prepare_chat_payload deepcopies contexts (with pairs at tail) then
appends user_msg, resulting in:

    ..., assistant(tc), tool, user

LLM sees assistant "predicting" user query. Reorder to:

    ..., user, assistant(tc), tool

Co-Authored-By: deepseek-v4-flash <deepseek-ai@claude-code-best.win>
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. area:provider The bug / feature is about AI Provider, Models, LLM Agent, LLM Agent Runner. labels Jul 30, 2026
@dosubot

dosubot Bot commented Jul 30, 2026

Copy link
Copy Markdown

📄 Knowledge review

Dosu skipped reviewing this PR because your organization has used its 200 included credits for the month. Your usage will reset on 2026-08-01. To have Dosu review this PR before then, ask your organization admin to upgrade to a pro account.


Leave Feedback Ask Dosu about AstrBot Add Dosu to your team

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="astrbot/core/provider/sources/openai_source.py" line_range="534" />
<code_context>
+        ProviderOpenAIOfficial._reorder_tailing_tool_call_user(payloads)
+
+    @staticmethod
+    def _reorder_tailing_tool_call_user(payloads: dict) -> None:
+        """重排因伪造工具调用导致尾部 assistant(tc) → tool → user 乱序的消息。
+
</code_context>
<issue_to_address>
**issue (complexity):** Consider refactoring `_reorder_tailing_tool_call_user` to use an index-based scan with a separate pair-validation helper instead of pop-based mutation to clarify its control flow and data handling.

You can simplify `_reorder_tailing_tool_call_user` by making it index-based and separating pair validation from mutation. This reduces control-flow nesting, avoids repeated `pop()`/reverse operations, and makes the behavior easier to reason about.

### 1. Extract a small validator helper

```python
@staticmethod
def _is_valid_tool_pair(asst_msg: dict, tool_msg: dict) -> bool:
    if asst_msg.get("role") != "assistant" or tool_msg.get("role") != "tool":
        return False

    tool_calls = asst_msg.get("tool_calls")
    if not tool_calls:
        return False

    tc_ids = {tc.get("id") for tc in tool_calls if isinstance(tc, dict)}
    return tool_msg.get("tool_call_id") in tc_ids
```

### 2. Use indices + slicing instead of pops + reverse

```python
@staticmethod
def _reorder_tailing_tool_call_user(payloads: dict) -> None:
    messages = payloads.get("messages")
    if not isinstance(messages, list) or len(messages) < 2:
        return

    # 必须以 user 结尾
    if messages[-1].get("role") != "user":
        return

    # 从尾部向前扫描匹配的 assistant/tool 成对消息
    end = len(messages) - 1  # index of tailing user
    i = end - 1
    while i >= 1:
        asst_msg = messages[i - 1]
        tool_msg = messages[i]
        if not ProviderOpenAIOfficial._is_valid_tool_pair(asst_msg, tool_msg):
            break
        i -= 2  # 每次向前跳过一对

    # 没有成对工具调用,保持原样
    if i == end - 1:
        return

    # 现在 messages 结构为: [ ... prefix ..., pair_start..pair_end, user ]
    prefix = messages[:i + 1]
    pairs = messages[i + 1:end]  # contiguous assistant/tool pairs
    user_msg = messages[end]

    # 重排:prefix + [user] + pairs
    payloads["messages"] = prefix + [user_msg] + pairs
```

This keeps all behavior but removes the mutable stack-like operations and nested breaks, making the tail reordering logic more straightforward and testable.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread astrbot/core/provider/sources/openai_source.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:provider The bug / feature is about AI Provider, Models, LLM Agent, LLM Agent Runner. size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant