Summary
Split from #332 — add GLM4 causal LM model support only.
Tasks
- Add GLM4 model config adapter (
csrc/models/glm4/)
- Add partial rotary support (
llama_utils.hpp, llama_attention.*)
- Add
forward_naive + GLM4 post-norm support (llama_decoder_layer.*)
- Add RoPE algo selection for GLM4 (
llama_model.cpp)
- Add GLM4 special model creation path (
rank_worker.cpp)
- Register
"glm4" in config_factory.cpp classic_models list
- Register
"glm4" in python/infinilm/auto_config.py
- Add GLM4 weight remapping (
gate_up_proj split) in modeling_utils.py
Depends on: #348 (Baichuan)
Parent issue: #332
Summary
Split from #332 — add GLM4 causal LM model support only.
Tasks
csrc/models/glm4/)llama_utils.hpp,llama_attention.*)forward_naive+ GLM4 post-norm support (llama_decoder_layer.*)llama_model.cpp)rank_worker.cpp)"glm4"inconfig_factory.cppclassic_models list"glm4"inpython/infinilm/auto_config.pygate_up_projsplit) inmodeling_utils.pyDepends on: #348 (Baichuan)
Parent issue: #332