Skip to content

Commit 8e3e3f2

Browse files
committed
Replace keepHistoryReasoning boolean with reasoningHistory
Introduce a more granular control for reasoning retention in requests: "all" (default, send everything) "turn" (current turn only) "off (discard all) Both delta-reasoning (reasoning_content) and think-tag reasoning are handled uniformly. DB storage is unaffected. Reasoning is always persisted for UI display. This setting only controls what gets sent back to the model.
1 parent 1cbfcd0 commit 8e3e3f2

8 files changed

Lines changed: 74 additions & 57 deletions

File tree

docs/configuration.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -694,7 +694,7 @@ To configure, add your OTLP collector config via `:otlp` map following [otlp aut
694694
models: {[key: string]: {
695695
modelName?: string;
696696
extraPayload?: {[key: string]: any};
697-
keepHistoryReasoning?: boolean;
697+
reasoningHistory?: "all" | "turn" | "off";
698698
}};
699699
}};
700700
defaultModel?: string;

docs/models.md

Lines changed: 10 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ Schema:
7373
| `models` | map | Key: model name, value: its config | Yes |
7474
| `models <model> extraPayload` | map | Extra payload sent in body to LLM | No |
7575
| `models <model> modelName` | string | Override model name, useful to have multiple models with different configs and names that use same LLM model | No |
76-
| `models <model> keepHistoryReasoning` | boolean | Keep `reason` messages in conversation history. Default: `false` | No |
76+
| `models <model> reasoningHistory` | string | Controls reasoning in conversation history: `"all"` (default), `"turn"`, or `"off"` | No |
7777
| `fetchModels` | boolean | Enable automatic model discovery from `/models` endpoint (OpenAI-compatible providers) | No |
7878

7979
_* url and key will be searched as envs `<provider>_API_URL` and `<provider>_API_KEY`, they require the env to be found or config to work._
@@ -121,32 +121,18 @@ Examples:
121121

122122
This way both will use gpt-5 model but one will override the reasoning to be high instead of the default.
123123

124-
=== "History reasoning"
125-
`keepHistoryReasoning` - Determines whether the model's internal reasoning chain is persisted in the conversation history for subsequent turns.
124+
=== "Reasoning in conversation history"
125+
`reasoningHistory` - Controls whether and how the model's reasoning (thinking blocks, reasoning_content) is included in conversation history sent to the model.
126126

127-
- **Standard Behavior**: Most models expect reasoning blocks (e.g., `<think>` tags or `reasoning_content`) to be removed in subsequent requests to save tokens and avoid bias.
128-
- **Usage**: Enable this for models that explicitly support "preserved thinking," or if you want to experiment with letting the model see its previous thought process (with XML-based reasoning).
129-
- **Example**: See [GLM-4.7 with Preserved thinking](https://docs.z.ai/guides/capabilities/thinking-mode#preserved-thinking).
127+
**Available modes:**
130128

131-
```javascript title="~/.config/eca/config.json"
132-
{
133-
"providers": {
134-
"z-ai": {
135-
"api": "openai-chat",
136-
"url": "https://api.z.ai/api/paas/v4/",
137-
"key": "your-api-key",
138-
"models": {
139-
"GLM-4.7": {
140-
"keepHistoryReasoning": true, // Preserves reasoning
141-
"extraPayload": {"clear_thinking": false} // Preserved thinking (see https://docs.z.ai/guides/capabilities/thinking-mode#preserved-thinking)
142-
}
143-
}
144-
}
145-
}
146-
}
147-
```
129+
- **`"all"`** (default, safe choice) - Send all reasoning blocks back to the model. The model can see its full chain of thought from previous turns. This is the safest option.
130+
- **`"turn"`** - Send only reasoning from the current conversation turn (after the last user message). Previous reasoning is discarded before sending to the API.
131+
- **`"off"`** - Never send reasoning blocks to the model. All reasoning is discarded before API calls.
132+
133+
**Note:** Reasoning is always shown to you in the UI and stored in chat history—this setting only controls what gets sent to the model in API requests.
148134

149-
Default: `false`.
135+
Default: `"all"`.
150136

151137
=== "Dynamic model discovery"
152138

integration-test/integration/chat/github_copilot_test.clj

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -168,7 +168,7 @@
168168
(match-content chat-id "system" {:type "progress" :state "finished"})
169169
(is (match?
170170
{:input [{:role "user" :content [{:type "input_text" :text "hello!"}]}
171-
{:role "assistant" :content [{:type "output_text" :text "hello there!"}]}
171+
{:role "assistant" :content [{:type "output_text" :text "<think>I should say hello</think>\nhello there!"}]}
172172
{:role "user" :content [{:type "input_text" :text "how are you?"}]}]
173173
:instructions (m/pred string?)}
174174
(llm.mocks/get-req-body :reasoning-1)))))))

integration-test/integration/chat/google_test.clj

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -167,7 +167,7 @@
167167
(match-content chat-id "system" {:type "progress" :state "finished"})
168168
(is (match?
169169
{:input [{:role "user" :content [{:type "input_text" :text "hello!"}]}
170-
{:role "assistant" :content [{:type "output_text" :text "hello there!"}]}
170+
{:role "assistant" :content [{:type "output_text" :text "<thought>I should say hello</thought>\nhello there!"}]}
171171
{:role "user" :content [{:type "input_text" :text "how are you?"}]}]
172172
:instructions (m/pred string?)}
173173
(llm.mocks/get-req-body :reasoning-1)))))))

src/eca/config.clj

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -336,7 +336,8 @@
336336
{:kebab-case-key
337337
[[:providers]]
338338
:keywordize-val
339-
[[:providers :ANY :httpClient]]
339+
[[:providers :ANY :httpClient]
340+
[:providers :ANY :models :ANY :reasoningHistory]]
340341
:stringfy-key
341342
[[:behavior]
342343
[:providers]

src/eca/llm_api.clj

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,7 @@
9797
provider-config (get-in config [:providers provider])
9898
model-config (get-in provider-config [:models model])
9999
extra-payload (:extraPayload model-config)
100+
reasoning-history (or (:reasoningHistory model-config) :all)
100101
[auth-type api-key] (llm-util/provider-api-key provider provider-auth config)
101102
api-url (llm-util/provider-api-url provider config)
102103
{:keys [handler]} (provider->api-handler provider config)
@@ -123,6 +124,7 @@
123124
:web-search web-search
124125
:extra-payload (merge {:parallel_tool_calls true}
125126
extra-payload)
127+
:reasoning-history reasoning-history
126128
:api-url api-url
127129
:api-key api-key
128130
:auth-type auth-type}
@@ -157,6 +159,7 @@
157159
:tools tools
158160
:extra-payload (merge {:parallel_tool_calls true}
159161
extra-payload)
162+
:reasoning-history reasoning-history
160163
:api-url api-url
161164
:api-key api-key
162165
:extra-headers {"openai-intent" "conversation-panel"
@@ -179,6 +182,7 @@
179182
:tools tools
180183
:think-tag-start "<thought>"
181184
:think-tag-end "</thought>"
185+
:reasoning-history reasoning-history
182186
:extra-payload (merge {:parallel_tool_calls false}
183187
(when reason?
184188
{:extra_body {:google {:thinking_config {:include_thoughts true}}}})
@@ -206,7 +210,6 @@
206210
(let [url-relative-path (:completionUrlRelativePath provider-config)
207211
think-tag-start (:thinkTagStart provider-config)
208212
think-tag-end (:thinkTagEnd provider-config)
209-
keep-history-reasoning (:keepHistoryReasoning model-config)
210213
http-client (:httpClient provider-config)]
211214
(handler
212215
{:model real-model
@@ -222,7 +225,7 @@
222225
:url-relative-path url-relative-path
223226
:think-tag-start think-tag-start
224227
:think-tag-end think-tag-end
225-
:keep-history-reasoning keep-history-reasoning
228+
:reasoning-history reasoning-history
226229
:http-client http-client
227230
:api-url api-url
228231
:api-key api-key}

src/eca/llm_providers/openai_chat.clj

Lines changed: 22 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -384,24 +384,27 @@
384384
(reset! reasoning-state* {:id nil :type nil :content "" :buffer ""})))
385385

386386
(defn ^:private prune-history
387-
"Discard reasoning messages from history.
388-
Reasoning with :delta-reasoning? is preserved in the same turn (as required by Deepseek).
389-
This corresponds to the implementation standard. However, it can be change it at the model level configuration.
387+
"Discard reasoning messages from history based on reasoning-history mode.
388+
390389
Parameters:
391390
- messages: the conversation history
392-
- keep-history-reasoning: if true, preserve all reasoning in history"
393-
[messages keep-history-reasoning]
394-
(if keep-history-reasoning
395-
messages
396-
(if-let [last-user-idx (llm-util/find-last-user-msg-idx messages)]
397-
(->> messages
398-
(keep-indexed (fn [i m]
399-
(when-not (and (= "reason" (:role m))
400-
(or (< i last-user-idx)
401-
(not (get-in m [:content :delta-reasoning?]))))
402-
m)))
403-
vec)
404-
messages)))
391+
- reasoning-history: controls reasoning retention
392+
- :all - preserve all reasoning in history (safe default)
393+
- :turn - preserve reasoning only in the current turn (after last user message)
394+
- :off - discard all reasoning messages"
395+
[messages reasoning-history]
396+
(case reasoning-history
397+
:all messages
398+
:off (filterv #(not= "reason" (:role %)) messages)
399+
:turn (if-let [last-user-idx (llm-util/find-last-user-msg-idx messages)]
400+
(->> messages
401+
(keep-indexed (fn [i m]
402+
(when-not (and (= "reason" (:role m))
403+
(< i last-user-idx))
404+
m)))
405+
vec)
406+
messages)
407+
messages))
405408

406409
(defn chat-completion!
407410
"Primary entry point for OpenAI chat completions with streaming support.
@@ -411,14 +414,14 @@
411414
Compatible with OpenRouter and other OpenAI-compatible providers."
412415
[{:keys [model user-messages instructions temperature api-key api-url url-relative-path
413416
past-messages tools extra-payload extra-headers supports-image?
414-
think-tag-start think-tag-end keep-history-reasoning http-client]}
417+
think-tag-start think-tag-end reasoning-history http-client]}
415418
{:keys [on-message-received on-error on-prepare-tool-call on-tools-called on-reason on-usage-updated] :as callbacks}]
416419
(let [think-tag-start (or think-tag-start "<think>")
417420
think-tag-end (or think-tag-end "</think>")
418421
stream? (boolean callbacks)
419422
system-messages (when instructions [{:role "system" :content instructions}])
420423
;; Pipeline: prune history -> normalize -> merge adjacent assistants -> filter
421-
all-messages (prune-history (vec (concat past-messages user-messages)) keep-history-reasoning)
424+
all-messages (prune-history (vec (concat past-messages user-messages)) reasoning-history)
422425
messages (vec (concat
423426
system-messages
424427
(normalize-messages all-messages supports-image? think-tag-start think-tag-end)))
@@ -478,7 +481,7 @@
478481
tool-calls))
479482
on-tools-called-wrapper (fn on-tools-called-wrapper [tools-to-call on-tools-called handle-response]
480483
(when-let [{:keys [new-messages]} (on-tools-called tools-to-call)]
481-
(let [pruned-messages (prune-history new-messages keep-history-reasoning)
484+
(let [pruned-messages (prune-history new-messages reasoning-history)
482485
new-messages-list (vec (concat
483486
system-messages
484487
(normalize-messages pruned-messages supports-image? think-tag-start think-tag-end)))

test/eca/llm_providers/openai_chat_test.clj

Lines changed: 32 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -259,7 +259,7 @@
259259
{:role "assistant" :reasoning_content "Thinking..."}])))))
260260

261261
(deftest prune-history-test
262-
(testing "Drops all reason messages before the last user message by default"
262+
(testing "reasoningHistory \"turn\" drops all reason messages before the last user message"
263263
(is (match?
264264
[{:role "user" :content "Q1"}
265265
{:role "assistant" :content "A1"}
@@ -273,13 +273,14 @@
273273
{:role "user" :content "Q2"}
274274
{:role "reason" :content {:text "r2" :delta-reasoning? true}}
275275
{:role "assistant" :content "A2"}]
276-
false))))
276+
:turn))))
277277

278-
(testing "Preserves reason messages (without :delta-reasoning?) before last user message"
278+
(testing "reasoningHistory \"turn\" also drops think-tag reasoning before last user message"
279279
(is (match?
280280
[{:role "user" :content "Q1"}
281281
{:role "assistant" :content "A1"}
282282
{:role "user" :content "Q2"}
283+
{:role "reason" :content {:text "more thinking..."}}
283284
{:role "assistant" :content "A2"}]
284285
(#'llm-providers.openai-chat/prune-history
285286
[{:role "user" :content "Q1"}
@@ -288,9 +289,9 @@
288289
{:role "user" :content "Q2"}
289290
{:role "reason" :content {:text "more thinking..."}}
290291
{:role "assistant" :content "A2"}]
291-
false))))
292+
:turn))))
292293

293-
(testing "Preserves all reasoning when keep-history-reasoning is true (Bedrock)"
294+
(testing "reasoningHistory \"all\" preserves all reasoning"
294295
(is (match?
295296
[{:role "user" :content "Q1"}
296297
{:role "reason" :content {:text "r1"}}
@@ -305,12 +306,35 @@
305306
{:role "user" :content "Q2"}
306307
{:role "reason" :content {:text "r2"}}
307308
{:role "assistant" :content "A2"}]
308-
true))))
309+
:all))))
309310

310-
(testing "No user message leaves list unchanged"
311+
(testing "reasoningHistory \"off\" removes all reasoning messages"
312+
(is (match?
313+
[{:role "user" :content "Q1"}
314+
{:role "assistant" :content "A1"}
315+
{:role "user" :content "Q2"}
316+
{:role "assistant" :content "A2"}]
317+
(#'llm-providers.openai-chat/prune-history
318+
[{:role "user" :content "Q1"}
319+
{:role "reason" :content {:text "r1" :delta-reasoning? true}}
320+
{:role "assistant" :content "A1"}
321+
{:role "user" :content "Q2"}
322+
{:role "reason" :content {:text "r2"}}
323+
{:role "assistant" :content "A2"}]
324+
:off))))
325+
326+
(testing "No user message - reasoningHistory \"turn\" leaves list unchanged"
311327
(let [msgs [{:role "assistant" :content "A"}
312328
{:role "reason" :content {:text "r"}}]]
313-
(is (= msgs (#'llm-providers.openai-chat/prune-history msgs false))))))
329+
(is (= msgs (#'llm-providers.openai-chat/prune-history msgs :turn)))))
330+
331+
(testing "No user message - reasoningHistory \"off\" removes reason"
332+
(is (match?
333+
[{:role "assistant" :content "A"}]
334+
(#'llm-providers.openai-chat/prune-history
335+
[{:role "assistant" :content "A"}
336+
{:role "reason" :content {:text "r"}}]
337+
:off)))))
314338

315339
(deftest valid-message-test
316340
(testing "Tool messages are always kept"

0 commit comments

Comments
 (0)