[Feature Request]: Adding request model reasoning

[roseodimm]

Description

Can you add request model reasoning like in the SillyTavern? So the user can read what the model think and debug the prompt.

I using deepseek paid API and the R1 model tend to produce a blank response, the same API work fine with Sillytavern's think process in the same hour.

Proposed Solution

No response

[Vali-98]

I'm not 100% sure what this means, does deepseek not provide reasoning context as well?

[roseodimm]

I can't see or find a way to read how the model thinks, like in SillyTavern. It only shows three dots bouncing up and down. This is the 0.8.6 stable version.

In SillyTavern the model will show how it think before output the reply.

If I remember correctly, the 0.8.7 beta 4 is showing only three dots as well. (I stop using Beta because it crashes mid-generation.)

[Mabuse1337]

Let me jump in on this one. I'm pretty sure that "request model reasoning" in SillyTavern just adds the enable_thinking=True flag to the chat completions payload which is usually not mandatory. A handful of models let you include a flag in the payload to force reasoning, but from what I've seen unfortunately it's a different flag from one model to the next.

The real problem is that even on thinking models we don't always see the model's reasoning block, and that's because of some quirks in how certain models' reasoning interacts with something like Llama.cpp - mainly the problem that ChatterUI seems to expect think blocks with the usual tags, but in a lot of cases llama.cpp will separate the thinking into 'reasoning_content' and then the rest of the output goes into 'content' and all we ever see is what's in content. I had this issue with both GLM 4.5 and Qwen 3 Thinking.

In SillyTavern, I can force the response to begin with a think tag, if I do that, either of those models will use a traditional think block. If I don't do that I get the 'reasoning_content' section. With Qwen 3 at least when I start llama.cpp I can use the flag --reasoning-content deepseek-legacy to make it default back to think blocks, but GLM just won't - I had to write a python proxy script to intercept the stream and change the reasoning content into a normal think block.

I'll add that if anyone needs that script in the meantime I have it in a repo. https://github.com/Mabuse1337/LLM-API-Proxy

[Vali-98]

Thanks for the detailed response, I actually tackled this a while back with Open Router since reasoning was given within a different object, hence required a new API field in reasoningParsePattern.

Currently the original issue isn't really resolved as I have no implemented this for DeepSeek, however its now just a matter of testing the reasoning response formats of each provider.

[mratsim]

vllm, sglang, llama.cpp all use reasoning_content AFAIK. The only framework that doesn't is TabbyAPI, yet theroyallab/tabbyAPI#295

[mratsim]

Actually things might not be as clear-cut: vllm-project/vllm#27755 as OpenAI is suggesting a reasoning field, uses a reasoning_details field as well.

[mratsim]

I'll add that if anyone needs that script in the meantime I have it in a repo. https://github.com/Mabuse1337/LLM-API-Proxy

@Mabuse1337, I've added some feedback to it here Mabuse1337/LLM-API-Proxy#1, one thing is that your script buffers. On long reasoning content we can wait for over a minute before receiving an answer and there is no way to differentiate between buffering or a real problem.

I've implemented a proxy as well in Rust with no buffering and binary or docker images at https://github.com/mratsim/llm-reasoning-proxy.

[Vali-98]

This should be resolved with a recent addition to the reasoning parser.

[mratsim]

Note that vLLM has switched to reasoning and so did OpenAI. So you need to support both.

It's a big mess

• RFC: [RFC]: reasoning_content -> reasoning vllm-project/vllm#27755
• Deletion: Remove deprecated reasoning_content message field vllm-project/vllm#33402
• Deletion followup (pending): Remove deprecated reasoning_content message field(part-2) vllm-project/vllm#37480
• Regression issue: [Bug]: reasoning_content to reasoning rename caused regression in chat templates expecting reasoning_content vllm-project/vllm#33616
• Reinstatement for chat template: [Bugfix] Interleaved thinking keeps compatibility with reasoning_content vllm-project/vllm#33635
• Regression on downstream and vLLM contradicting its document on backward compat (pending): [Bugfix] Add reasoning_content backward compat to DeltaMessage for streaming vllm-project/vllm#34030