Nemotron-3-Nano-30B-A3B special reasoning tags are not being outputted.

[vevi33]

Describe the Issue
I am using https://huggingface.co/unsloth/Nemotron-3-Nano-30B-A3B-GGUF
"< think > and < /think > are separate tokens, so use --special if needed."
More info: https://unsloth.ai/docs/models/nemotron-3

It can't be enabled in the GUI, thus it won't format or even output and tags.
Can you add an option somewhere to enable these special tokens from the GUI? Or is there any alternative solution?

Thanks in advance!

[vevi33]

When I use "render special tags" I get this:

[vevi33]

I can force thinking though, but then with regex I gotta remove the end tag.

Also, even without thinking the smart cache is broken. You wrote an example in a different topic with this exact model. Yeah it is working for the first few prompts and reuses the prompt stage, however after 2-3 conversations it always reloads the first memory state, which is often thousands of tokens away. It will always keep that first and only load that one. Tested with an empty conversation with 5 and 10 number of memory cache settings . It keeps creating new ones but never loads them, which makes the model even less usable. I did not use author's note or anything either.

[LostRuins]

Try render special tags

[vevi33]

I used that already, which is working but the smart cache is broken compared to LM studio

[AdamJ8]

It's not just Nemotron v3, there are quite a few models that recently were released without a forced generation of the opening reasoning token. I made the following changes to the koboldcpp.py to accept the "enable_thinking" = true API parameter, then pass it to the JINJA processor AND enforce a standard opening tag being issued so I can use it with Nemotron v3:

In format_jinja() near the bottom I replaced the jinja calling with:

# NEW: Extract enable_thinking from genparams and pass to Jinja
        thinking_val = genparams.get("enable_thinking", True) # Default to True to match your template's logic
        text = None
        if tools and len(tools)>0:
            text = jinja_compiled_template.render(
                messages=messages, 
                tools=tools, 
                add_generation_prompt=True, 
                enable_thinking=thinking_val, # Pass the variable here
                bos_token="", 
                eos_token=""
            )
        else:
            text = jinja_compiled_template.render(
                messages=messages, 
                add_generation_prompt=True, 
                enable_thinking=thinking_val, # Pass the variable here
                bos_token="", 
                eos_token=""
            )
        return text if text else None

and replace the main method line with:

def format_jinja(messages, tools, genparams):

In transform_genparams() benethe the oaiml = line I added this:

        if "enable_thinking" in genparams: # NEW: case correct if param is not lower
            genparams["enable_thinking"] = str(genparams["enable_thinking"]).lower() == "true"

and just below the jinjatools = genparams.get('tools', []) change to call jinja with genparams:

            if use_jinja and cached_chat_template:
                jinja_output = format_jinja(messages_array, jinjatools, genparams)

In generate_text() I replaced the recvtxt = with:

        recvtxt = genout['text']        
        # NEW: PREPEND THINK TAG FOR NON-STREAMING
        if api_format == 4 and genparams.get("enable_thinking", False) and recvtxt:
            if not recvtxt.strip().startswith("<think>"):
                recvtxt = "<think>\n" + recvtxt

In handle_sse_stream() beneath incomplete_token_buffer = line:

        # NEW: Logic for Thinking Tag Injection
        injected_think_tag = False
        enable_thinking_flag = genparams.get("enable_thinking", False)

and ABOVE the SECOND if tokenStr!="" or streamDone: line:

                        # NEW: Prepend <think> tag to the first available content token
                        if tokenStr != "" and api_format == 4 and enable_thinking_flag and not injected_think_tag:
                            if not tokenStr.strip().startswith("<think>"):
                                tokenStr = "<think>\n" + tokenStr
                            injected_think_tag = True

[henk717]

Is this still an issue on the latest koboldcpp? We automatically detect the common think tags and change them back to regular tokens so they don't get hidden. But we hardcode the tags, so if its "< think >" this won't work, if its the usual "" or "[THINK]" then it will. If its still an issue for this particular model we have to add "< think >" to the list.

As for forced thinking, the in scope way of doing it is adding it to the chat adapter or doing it in the UI option since we don't even want to force thinking on people in a hardcoded way. In the next release GLM-4.7 will have part of the chat adapter by default to mimick how the jinja does it. But it won't show that, just like in the jinja version. UI's should not expect it as thats no longer correct for modern models.

[henk717]

Can confirm nemotron's tags can be sent to the UI:
load: setting token '' (12) attribute to USER_DEFINED (16), old attributes: 8

load: setting token '[TOOL_CALLS]' (9) attribute to USER_DEFINED (16), old attributes: 8

load: setting token '' (13) attribute to USER_DEFINED (16), old attributes: 8

It changed it from the incorrect 8 to the correct 16.

[henk717]

Can also now verify the model behaves correctly, its a model that has optional thinking and chooses on its own if it wishes to think which is good behavior. https://lite.koboldai.net/?dp=6GE4GTU26

It correctly could think when it needed to and correctly used the closing think tag. The UI correctly detected the thinking.