Problem
Some models' chat template are like this: "{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\n<think>\n' }}". But API client won't see that <think> tag thus breaking parsing and UI.
Solution
Maybe make it optional, but add the ability to detect this and "compensate" that tag at the start of the response.
Acknowledgements
Problem
Some models' chat template are like this: "{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\n<think>\n' }}". But API client won't see that <think> tag thus breaking parsing and UI.
Solution
Maybe make it optional, but add the ability to detect this and "compensate" that tag at the start of the response.
Acknowledgements