LTX2.3 `invalid tokenizer` Error

[ConnoisseurProtege]

I'm getting this error with the workflows when I set the LTXV Audio Text Encoder Loader to the Gemma3 listed in the README. I've made sure that the hashes for the model match. Here is the full error -

ValueError: invalid tokenizer


  File "~\ComfyUI\execution.py", line 525, in execute
    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "~\ComfyUI\execution.py", line 334, in get_output_data
    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "~\ComfyUI\execution.py", line 308, in _async_map_node_over_list
    await process_inputs(input_dict, i)

  File "~\ComfyUI\execution.py", line 296, in process_inputs
    result = f(**inputs)

  File "~\ComfyUI\comfy_api\internal\__init__.py", line 149, in wrapped_func
    return method(locked_class, **inputs)

  File "~\ComfyUI\comfy_api\latest\_io.py", line 1777, in EXECUTE_NORMALIZED
    to_return = cls.execute(*args, **kwargs)

  File "~\ComfyUI\comfy_extras\nodes_lt_audio.py", line 209, in execute
    clip = comfy.sd.load_clip(ckpt_paths=[clip_path1, clip_path2], embedding_directory=folder_paths.get_folder_paths("embeddings"), clip_type=clip_type, model_options=model_options)

  File "~\ComfyUI\comfy\sd.py", line 1204, in load_clip
    clip = load_text_encoder_state_dicts(clip_data, embedding_directory=embedding_directory, clip_type=clip_type, model_options=model_options, disable_dynamic=disable_dynamic)

  File "~\ComfyUI\comfy\sd.py", line 1553, in load_text_encoder_state_dicts
    clip = CLIP(clip_target, embedding_directory=embedding_directory, parameters=parameters, tokenizer_data=tokenizer_data, state_dict=clip_data, model_options=model_options, disable_dynamic=disable_dynamic)

  File "~\ComfyUI\comfy\sd.py", line 236, in __init__
    self.tokenizer = tokenizer(embedding_directory=embedding_directory, tokenizer_data=tokenizer_data)
                     ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "~\ComfyUI\comfy\text_encoders\lt.py", line 81, in __init__
    super().__init__(embedding_directory=embedding_directory, tokenizer_data=tokenizer_data, name="gemma3_12b", tokenizer=Gemma3_12BTokenizer)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "~\ComfyUI\comfy\sd1_clip.py", line 690, in __init__
    setattr(self, self.clip, tokenizer(embedding_directory=embedding_directory, tokenizer_data=tokenizer_data))
                             ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "~\ComfyUI\comfy\text_encoders\lt.py", line 76, in __init__
    super().__init__(tokenizer, pad_with_end=False, embedding_size=3840, embedding_key='gemma3_12b', tokenizer_class=SPieceTokenizer, has_end_token=False, pad_to_max_length=False, max_length=99999999, min_length=1024, pad_left=True, disable_weights=True, tokenizer_args={"add_bos": True, "add_eos": False, "special_tokens": special_tokens}, tokenizer_data=tokenizer_data)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "~\ComfyUI\comfy\sd1_clip.py", line 490, in __init__
    self.tokenizer = tokenizer_class.from_pretrained(tokenizer_path, **tokenizer_args)
                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "~\ComfyUI\comfy\text_encoders\spiece_tokenizer.py", line 7, in from_pretrained
    return SPieceTokenizer(path, **kwargs)

  File "~\ComfyUI\comfy\text_encoders\spiece_tokenizer.py", line 21, in __init__
    raise ValueError("invalid tokenizer")

[ConnoisseurProtege]

Seems to be looking for the tokenizer.model in the wrong folder?

[ConnoisseurProtege]

Ended up grabbing the text_encoders here - https://huggingface.co/Comfy-Org/ltx-2/tree/main/split_files/text_encoders

[aelfwyne]

Sadly, no amount of downloading encoders has fixed this for me. I have downloaded everything listed in the model, as well as on the github main page as far as requirements. This error persists.

[ConnoisseurProtege]

Sadly, no amount of downloading encoders has fixed this for me. I have downloaded everything listed in the model, as well as on the github main page as far as requirements. This error persists.

The node that they're using seems to only accept singular safetensor files, did the Comfy-Org ones not work for you?

[JackyChan]

I encountered the same problem. Later, I switched to singular safetensor files, and then it showed "Reconnecting". My system is M5 Max macOS 26.4.

[starymarvel]

Jacky Chan (@JackyChan) what encoder did u use couldu tell me where u got it