perf(dflash): enable grouped MMID on RDNA

[cheese-cakee]

Summary

Extends the grouped MMID (MUL_MAT_ID) batch gate to AMD RDNA3 and RDNA4 while keeping the HIP path opt-in and default-off. Adds portable grouped-versus-legacy dispatch and numerical coverage for CUDA and HIP, including the DS4 gate/up activation graph.

Changes

• Extend get_mmvq_mmid_max_batch() to admit RDNA3/RDNA4 for grouped MMID.
• Keep NVIDIA Turing+ enabled by default and AMD HIP gated by DFLASH_MMID_GROUPED=1.
• Exercise Q4_K, Q6_K, Q4_0, Q8_0, and Q5_K at widths 2, 4, 8, 9, and 16.
• Cover both bare MUL_MAT_ID and the production pair of gate/up MUL_MAT_ID nodes feeding SWIGLU_DS4.
• Preserve GLU parameters and DS4/OAI activation semantics in the grouped kernel.
• Avoid grouped MMID/GLU fusion and compose the two validated grouped MMIDs with the standalone activation instead.
• Compare 50 output slices: byte-exact when both arms use the same MMVQ reduction, otherwise against the legacy graph with the repository's existing MMQ NMSE tolerance.
• Verify 75 grouped dispatch records and zero grouped records in the legacy arm.
• Build and run the regression in the NVIDIA GPU and gfx1151 ROCm CI jobs.

How it works

The grouped kernel sorts (token, slot) pairs by routed expert so each unique expert weight stream is consumed once per MoE batch. The gate admits supported types and bounded widths/pair counts on NVIDIA Turing+ or AMD RDNA3/RDNA4. HIP remains default-off until performance is measured.

For grouped DS4 graphs, fusion selection deliberately falls back to two grouped MMIDs plus the existing standalone DS4 activation. This retains the grouped matmul optimization while preserving activation semantics and a stable numerical oracle.

Verification

• CUDA / RTX 4050 (sm_89, CUDA 12.6): deepseek4_mmid_grouped_cuda passed.
  • compared_cases=50
  • exact_cases=25
  • tolerant_cases=25
  • legacy_grouped=0
  • grouped_grouped=75
  • parity=PASS
• AMD Strix Halo (gfx1151, ROCm HIP): five-test gate passed:
  • rocmfp4_reference
  • rocmfpx_reference
  • rocmfp4_hip_tail
  • rocmfpx_mmq
  • deepseek4_mmid_grouped_cuda
• The CUDA and HIP grouped regression also pass when the parent process starts with GGML_CUDA_DISABLE_FUSION=1, proving the grouped child clears inherited test state.
• Current-main integration with open PRs fix(ds4): preserve exact speculative verification #548 and feat(ds4): add default-off HIP GPU phase profiler #554: a 32-token DS4 non-regression smoke produced identical token IDs with the flag off/on. This IQ2XXS/Q2_K model is outside the grouped kernel's supported quant types, so the 75-dispatch synthetic regression—not this smoke—is the grouped-path proof.
• git diff --check passes.

Limitations

• HIP performance has not yet been measured, so the HIP path remains opt-in and default-off.
• Grouped dispatch covers Q4_K, Q6_K, Q4_0, Q8_0, and Q5_K. DS4 ROCmFP2/FP3 specializations are outside this change.
• Compute Sanitizer was attempted locally but could not initialize the WSL/WDDM debugger interface for this device; it is not counted as a passing check.

[cubic-dev-ai]

All reported issues were addressed across 3 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>