Skip to content

Commit a94e839

Browse files
zikajkzikajk
committed
add configurable reasoning preservation for openai completions
Standard behavior: discard reasoning before last user message (based on OpenAI SDK). Model-level config enables preservation (e.g. GLM-4.7 with preserved thinking). Changes: - prune-history: add keep-history-reasoning parameter - Tests: cover both pruning and preservation - Docs: add keepHistoryReasoning to model schema
1 parent 68b59f5 commit a94e839

6 files changed

Lines changed: 90 additions & 38 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
## Unreleased
44

5+
- (OpenAI Chat) - Configurable reasoning history via `keepHistoryReasoning` (model-level, default: prune)
6+
57
## 0.92.0
68

79
- Fix Gemini (OpenAI compatible). #247

‎docs/configuration.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -673,7 +673,8 @@ To configure, add your OTLP collector config via `:otlp` map following [otlp aut
673673
thinkTagEnd?: string;
674674
models: {[key: string]: {
675675
modelName?: string;
676-
extraPayload?: {[key: string]: any}
676+
extraPayload?: {[key: string]: any};
677+
keepHistoryReasoning?: boolean;
677678
}};
678679
}};
679680
defaultModel?: string;

‎docs/models.md‎

Lines changed: 39 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -61,19 +61,20 @@ You just need to add your provider to `providers` and make sure add the required
6161

6262
Schema:
6363

64-
| Option | Type | Description | Required |
65-
|-------------------------------|--------|--------------------------------------------------------------------------------------------------------------|----------|
66-
| `api` | string | The API schema to use (`"openai-responses"`, `"openai-chat"`, or `"anthropic"`) | Yes |
67-
| `url` | string | API URL (with support for env like `${env:MY_URL}`) | No* |
68-
| `key` | string | API key (with support for `${env:MY_KEY}` or `{netrc:api.my-provider.com}` | No* |
69-
| `completionUrlRelativePath` | string | Optional override for the completion endpoint path (see defaults below and examples like Azure) | No |
70-
| `thinkTagStart` | string | Optional override the think start tag tag for openai-chat (Default: "<think>") api | No |
71-
| `thinkTagEnd` | string | Optional override the think end tag for openai-chat (Default: "</think>") api | No |
72-
| `httpClient` | map | Allow customize the http-client for this provider requests, like changing http version | No |
73-
| `models` | map | Key: model name, value: its config | Yes |
74-
| `models <model> extraPayload` | map | Extra payload sent in body to LLM | No |
75-
| `models <model> modelName` | string | Override model name, useful to have multiple models with different configs and names that use same LLM model | No |
76-
| `fetchModels` | boolean | Enable automatic model discovery from `/models` endpoint (OpenAI-compatible providers) | No |
64+
| Option | Type | Description | Required |
65+
|---------------------------------------|---------|--------------------------------------------------------------------------------------------------------------|----------|
66+
| `api` | string | The API schema to use (`"openai-responses"`, `"openai-chat"`, or `"anthropic"`) | Yes |
67+
| `url` | string | API URL (with support for env like `${env:MY_URL}`) | No* |
68+
| `key` | string | API key (with support for `${env:MY_KEY}` or `{netrc:api.my-provider.com}` | No* |
69+
| `completionUrlRelativePath` | string | Optional override for the completion endpoint path (see defaults below and examples like Azure) | No |
70+
| `thinkTagStart` | string | Optional override the think start tag tag for openai-chat (Default: "<think>") api | No |
71+
| `thinkTagEnd` | string | Optional override the think end tag for openai-chat (Default: "</think>") api | No |
72+
| `httpClient` | map | Allow customize the http-client for this provider requests, like changing http version | No |
73+
| `models` | map | Key: model name, value: its config | Yes |
74+
| `models <model> extraPayload` | map | Extra payload sent in body to LLM | No |
75+
| `models <model> modelName` | string | Override model name, useful to have multiple models with different configs and names that use same LLM model | No |
76+
| `models <model> keepHistoryReasoning` | boolean | Keep `reason` messages in conversation history. Default: `false` | No |
77+
| `fetchModels` | boolean | Enable automatic model discovery from `/models` endpoint (OpenAI-compatible providers) | No |
7778

7879
_* url and key will be searched as envs `<provider>_API_URL` and `<provider>_API_KEY`, they require the env to be found or config to work._
7980

@@ -120,6 +121,30 @@ Examples:
120121

121122
This way both will use gpt-5 model but one will override the reasoning to be high instead of the default.
122123

124+
=== "History reasoning"
125+
126+
`keepHistoryReasoning` preserves reasoning in conversation history. Set for specific models:
127+
128+
```javascript title="~/.config/eca/config.json"
129+
{
130+
"providers": {
131+
"z-ai": {
132+
"api": "openai-chat",
133+
"url": "https://api.z.ai/api/paas/v4/",
134+
"key": "your-api-key",
135+
"models": {
136+
"GLM-4.7": {
137+
"keepHistoryReasoning": true, // Preserves reasoning
138+
"extraPayload": {"clear_thinking": false} // Preserved thinking (see https://docs.z.ai/guides/capabilities/thinking-mode#preserved-thinking)
139+
}
140+
}
141+
}
142+
}
143+
}
144+
```
145+
146+
Default: `false`.
147+
123148
=== "Dynamic model discovery"
124149

125150
For OpenAI-compatible providers, set `fetchModels: true` to automatically discover available models:
@@ -211,7 +236,7 @@ Notes:
211236
3. Type the chosen method
212237
4. Authenticate in your browser, copy the code.
213238
5. Paste and send the code and done!
214-
239+
215240
=== "Codex / Openai"
216241

217242
1. Login to Openai via the chat command `/login`.

‎src/eca/llm_api.clj‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -206,6 +206,7 @@
206206
(let [url-relative-path (:completionUrlRelativePath provider-config)
207207
think-tag-start (:thinkTagStart provider-config)
208208
think-tag-end (:thinkTagEnd provider-config)
209+
keep-history-reasoning (:keepHistoryReasoning model-config)
209210
http-client (:httpClient provider-config)]
210211
(handler
211212
{:model real-model
@@ -221,6 +222,7 @@
221222
:url-relative-path url-relative-path
222223
:think-tag-start think-tag-start
223224
:think-tag-end think-tag-end
225+
:keep-history-reasoning keep-history-reasoning
224226
:http-client http-client
225227
:api-url api-url
226228
:api-key api-key}

‎src/eca/llm_providers/openai_chat.clj‎

Lines changed: 21 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -383,19 +383,24 @@
383383
(reset! reasoning-state* {:id nil :type nil :content "" :buffer ""})))
384384

385385
(defn ^:private prune-history
386-
"Ensure DeepSeek-style reasoning_content is discarded from history but kept for the active turn.
387-
Only drops 'reason' messages WITH :delta-reasoning? before the last user message.
388-
Think-tag based reasoning (without :delta-reasoning?) is preserved and transformed to assistant messages."
389-
[messages]
390-
(if-let [last-user-idx (llm-util/find-last-user-msg-idx messages)]
391-
(->> messages
392-
(keep-indexed (fn [i m]
393-
(when-not (and (= "reason" (:role m))
394-
(get-in m [:content :delta-reasoning?])
395-
(< i last-user-idx))
396-
m)))
397-
vec)
398-
messages))
386+
"Discard reasoning messages from history.
387+
Reasoning with :delta-reasoning? is preserved in the same turn (as required by Deepseek).
388+
This corresponds to the implementation standard. However, it can be change it at the model level configuration.
389+
Parameters:
390+
- messages: the conversation history
391+
- keep-history-reasoning: if true, preserve all reasoning in history"
392+
[messages keep-history-reasoning]
393+
(if keep-history-reasoning
394+
messages
395+
(if-let [last-user-idx (llm-util/find-last-user-msg-idx messages)]
396+
(->> messages
397+
(keep-indexed (fn [i m]
398+
(when-not (and (= "reason" (:role m))
399+
(or (< i last-user-idx)
400+
(not (get-in m [:content :delta-reasoning?]))))
401+
m)))
402+
vec)
403+
messages)))
399404

400405
(defn chat-completion!
401406
"Primary entry point for OpenAI chat completions with streaming support.
@@ -405,14 +410,14 @@
405410
Compatible with OpenRouter and other OpenAI-compatible providers."
406411
[{:keys [model user-messages instructions temperature api-key api-url url-relative-path
407412
past-messages tools extra-payload extra-headers supports-image?
408-
think-tag-start think-tag-end http-client]}
413+
think-tag-start think-tag-end keep-history-reasoning http-client]}
409414
{:keys [on-message-received on-error on-prepare-tool-call on-tools-called on-reason on-usage-updated] :as callbacks}]
410415
(let [think-tag-start (or think-tag-start "<think>")
411416
think-tag-end (or think-tag-end "</think>")
412417
stream? (boolean callbacks)
413418
system-messages (when instructions [{:role "system" :content instructions}])
414419
;; Pipeline: prune history -> normalize -> merge adjacent assistants -> filter
415-
all-messages (prune-history (vec (concat past-messages user-messages)))
420+
all-messages (prune-history (vec (concat past-messages user-messages)) keep-history-reasoning)
416421
messages (vec (concat
417422
system-messages
418423
(normalize-messages all-messages supports-image? think-tag-start think-tag-end)))
@@ -472,7 +477,7 @@
472477
tool-calls))
473478
on-tools-called-wrapper (fn on-tools-called-wrapper [tools-to-call on-tools-called handle-response]
474479
(when-let [{:keys [new-messages]} (on-tools-called tools-to-call)]
475-
(let [pruned-messages (prune-history new-messages)
480+
(let [pruned-messages (prune-history new-messages keep-history-reasoning)
476481
new-messages-list (vec (concat
477482
system-messages
478483
(normalize-messages pruned-messages supports-image? think-tag-start think-tag-end)))

‎test/eca/llm_providers/openai_chat_test.clj‎

Lines changed: 24 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -259,7 +259,7 @@
259259
{:role "assistant" :reasoning_content "Thinking..."}])))))
260260

261261
(deftest prune-history-test
262-
(testing "Drops reason messages WITH :delta-reasoning? before the last user message (DeepSeek)"
262+
(testing "Drops all reason messages before the last user message by default"
263263
(is (match?
264264
[{:role "user" :content "Q1"}
265265
{:role "assistant" :content "A1"}
@@ -272,28 +272,45 @@
272272
{:role "assistant" :content "A1"}
273273
{:role "user" :content "Q2"}
274274
{:role "reason" :content {:text "r2" :delta-reasoning? true}}
275-
{:role "assistant" :content "A2"}]))))
275+
{:role "assistant" :content "A2"}]
276+
false))))
276277

277-
(testing "Preserves reason messages WITHOUT :delta-reasoning? (think-tag based)"
278+
(testing "Preserves reason messages (without :delta-reasoning?) before last user message"
278279
(is (match?
279280
[{:role "user" :content "Q1"}
280-
{:role "reason" :content {:text "thinking..."}}
281281
{:role "assistant" :content "A1"}
282282
{:role "user" :content "Q2"}
283-
{:role "reason" :content {:text "more thinking..."}}
284283
{:role "assistant" :content "A2"}]
285284
(#'llm-providers.openai-chat/prune-history
286285
[{:role "user" :content "Q1"}
287286
{:role "reason" :content {:text "thinking..."}}
288287
{:role "assistant" :content "A1"}
289288
{:role "user" :content "Q2"}
290289
{:role "reason" :content {:text "more thinking..."}}
291-
{:role "assistant" :content "A2"}]))))
290+
{:role "assistant" :content "A2"}]
291+
false))))
292+
293+
(testing "Preserves all reasoning when keep-history-reasoning is true (Bedrock)"
294+
(is (match?
295+
[{:role "user" :content "Q1"}
296+
{:role "reason" :content {:text "r1"}}
297+
{:role "assistant" :content "A1"}
298+
{:role "user" :content "Q2"}
299+
{:role "reason" :content {:text "r2"}}
300+
{:role "assistant" :content "A2"}]
301+
(#'llm-providers.openai-chat/prune-history
302+
[{:role "user" :content "Q1"}
303+
{:role "reason" :content {:text "r1"}}
304+
{:role "assistant" :content "A1"}
305+
{:role "user" :content "Q2"}
306+
{:role "reason" :content {:text "r2"}}
307+
{:role "assistant" :content "A2"}]
308+
true))))
292309

293310
(testing "No user message leaves list unchanged"
294311
(let [msgs [{:role "assistant" :content "A"}
295312
{:role "reason" :content {:text "r"}}]]
296-
(is (= msgs (#'llm-providers.openai-chat/prune-history msgs))))))
313+
(is (= msgs (#'llm-providers.openai-chat/prune-history msgs false))))))
297314

298315
(deftest valid-message-test
299316
(testing "Tool messages are always kept"

0 commit comments

Comments
 (0)