[CUDA] Enable FP4 QMoE by default

[Tianlei Wu (tianleiwu)]

This change enables FP4 QMoE kernel instantiations by default for CUDA builds. Non-CUDA builds remain disabled, and CUDA users can opt out with -Donnxruntime_USE_FP4_QMOE=OFF.

The QMoE documentation now describes the default, dependency, and override behavior.

Validation

• cmake -S /home/tianlei/git/onnxruntime/cmake -B /tmp/ort_fp4_qmoe_check -Donnxruntime_USE_CUDA=OFF -Donnxruntime_BUILD_UNIT_TESTS=OFF -Donnxruntime_BUILD_SHARED_LIB=OFF
• cmake -S /home/tianlei/git/onnxruntime/cmake -B /tmp/ort_fp4_qmoe_cuda_check -Donnxruntime_USE_CUDA=ON -Donnxruntime_BUILD_UNIT_TESTS=OFF -Donnxruntime_BUILD_SHARED_LIB=OFF -Donnxruntime_USE_TENSORRT=OFF -DCMAKE_CUDA_COMPILER=/home/tianlei/cuda13.0/bin/nvcc -Donnxruntime_CUDNN_HOME=/home/tianlei/cudnn_9.23_cuda13
• git diff --check

The CUDA configure cache records onnxruntime_USE_FP4_QMOE:BOOL=ON.

[Copilot]

Pull request overview

Enables CUDA FP4 QMoE kernels by default and aligns quick-build instantiations with dispatcher requirements.

Changes:

• Defaults onnxruntime_USE_FP4_QMOE on for CUDA builds.
• Updates quick-build FP16/BF16 generated kernel coverage.
• Documents default and opt-out behavior.

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
cmake/CMakeLists.txt	Changes the FP4 QMoE CUDA default.
docs/contrib_ops/cuda/moe_qmoe.md	Documents defaults and opt-out behavior.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/generate_moe_gemm_tma_ws_sm90_fp4.py	Updates quick-build generation rules.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_fp16_m128_n64_k256_cm1_cn1_pp.generated.cu	Excludes FP16 PP specialization from quick builds.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_fp16_m128_n64_k256_cm1_cn1_pp_finalize.generated.cu	Excludes finalized FP16 PP specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_fp16_m128_n64_k256_cm1_cn1_co.generated.cu	Includes FP16 cooperative specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_fp16_m128_n64_k256_cm1_cn1_co_finalize.generated.cu	Includes finalized FP16 cooperative specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_fp16_m128_n64_k128_cm1_cn1_pp.generated.cu	Excludes FP16 PP specialization from quick builds.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_fp16_m128_n64_k128_cm1_cn1_pp_finalize.generated.cu	Excludes finalized FP16 PP specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_fp16_m128_n64_k128_cm1_cn1_co.generated.cu	Includes FP16 cooperative specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_fp16_m128_n64_k128_cm1_cn1_co_finalize.generated.cu	Includes finalized FP16 cooperative specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n64_k256_cm1_cn1_co.generated.cu	Includes BF16 cooperative specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n64_k256_cm1_cn1_co_finalize.generated.cu	Includes finalized BF16 cooperative specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n64_k128_cm1_cn1_co.generated.cu	Includes BF16 cooperative specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n64_k128_cm1_cn1_co_finalize.generated.cu	Includes finalized BF16 cooperative specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n32_k256_cm1_cn1_co.generated.cu	Includes BF16 cooperative specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n32_k256_cm1_cn1_co_finalize.generated.cu	Includes finalized BF16 cooperative specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n32_k128_cm1_cn1_co.generated.cu	Includes BF16 cooperative specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n32_k128_cm1_cn1_co_finalize.generated.cu	Includes finalized BF16 cooperative specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n16_k256_cm1_cn1_pp.generated.cu	Includes BF16 PP specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n16_k256_cm1_cn1_pp_finalize.generated.cu	Includes finalized BF16 PP specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n16_k128_cm1_cn1_pp.generated.cu	Includes BF16 PP specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n16_k128_cm1_cn1_pp_finalize.generated.cu	Includes finalized BF16 PP specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n128_k256_cm1_cn1_pp.generated.cu	Includes BF16 PP specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n128_k256_cm1_cn1_pp_finalize.generated.cu	Includes finalized BF16 PP specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n128_k128_cm1_cn1_pp.generated.cu	Includes BF16 PP specialization.
onnxruntime/contrib_ops/cuda/llm/moe_gemm/launchers/moe_gemm_tma_ws_sm90_fp4_bf16_m128_n128_k128_cm1_cn1_pp_finalize.generated.cu	Includes finalized BF16 PP specialization.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.