fix(agent-core): strip unsupported media only for explicit capability matrices - #2707
fix(agent-core): strip unsupported media only for explicit capability matrices#2707686f6c61 wants to merge 3 commits into
Conversation
downgradeUnsupportedMedia previously left image/video/audio parts intact when capability was UNKNOWN_CAPABILITY. Uncatalogued or custom OpenAI-compatible text-only models then received history with image_url parts, returned 400, and poisoned every subsequent turn (MoonshotAI#2669). Honor image_in/video_in/audio_in flags for all known matrices including UNKNOWN (all false). Only skip when capability is undefined. Fixes MoonshotAI#2669
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 44969a335a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // OpenAI-compatible provider returns 400 and poisons every subsequent turn | ||
| // (#2669). UNKNOWN was previously exempt, which left custom providers that | ||
| // declare no vision (or are uncatalogued) unprotected. | ||
| if (capability === undefined) return [...messages]; |
There was a problem hiding this comment.
When a host uses KimiForCodingProvider or constructs the public SingleModelProvider without its optional capability argument, Agent passes UNKNOWN_CAPABILITY here. That marker means the model is uncatalogued, not that it is text-only, so it can represent a multimodal custom model; this change then silently replaces every image, video, and audio part—including the current user attachment—before generate() sees it. Keep unknown capabilities distinct from explicitly unsupported modalities, or make these providers supply an accurate matrix while applying the conservative behavior only to the affected text-only configuration.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in 5229cd2: UNKNOWN_CAPABILITY is left alone again (uncatalogued ≠ text-only). Stripping only runs for explicit matrices with image_in / video_in / audio_in false. Covered by preserves media for UNKNOWN_CAPABILITY and strips images when an explicit matrix has no image_in in packages/agent-core/test/agent/kosong-llm.test.ts.
Document the user-facing CLI patch for stripping unsupported media when the model capability matrix has no media input (including uncatalogued custom providers).
Codex review: UNKNOWN means uncatalogued, not text-only. Multimodal custom models and SingleModelProvider defaults use that marker; stripping would drop legitimate attachments. Keep stripping only for explicit capability matrices with image_in / video_in / audio_in false (ProviderManager-resolved declared caps, MoonshotAI#2669).
686f6c61
changed the title
Codex P1 addressed (5229cd2)Agreed with the review: Stripping still applies only to explicit matrices with Also dropped the changeset — this is a behavioral guard + tests, not a user-facing CLI changelog entry on its own. |
1 participant
Summary
downgradeUnsupportedMediamust not treatUNKNOWN_CAPABILITYas text-only: that marker means uncatalogued (e.g.SingleModelProviderdefault) and can still represent a multimodal model.image_in/video_in/audio_infalse — the shapeProviderManagerproduces from declared config such ascapabilities = ["thinking", "tool_use"].Related to #2669 (text-only custom providers with an explicit no-vision matrix).
Test plan
vitest run test/agent/kosong-llm.test.tsinpackages/agent-core(18 passed)UNKNOWN_CAPABILITYimage_in