MiniMax-M3 H100 AgentX EAGLE3 tuning on vLLM v0.27.1 - #2564
Conversation
…n-AL synthetic acceptance Add minimaxm3-fp8-h100-vllm-agentic-mtp, the spec-decoding=mtp variant of the MiniMax-M3 MXFP8 H100 agentic-coding recipe: Inferact/MiniMax-M3-EAGLE3 draft head, 3 speculative tokens, FLASH_ATTN drafter, and synthetic acceptance pinned to the committed golden AL 2.83 (minimaxm3_eagle3.yaml, thinking_on, K=3). EVAL_ONLY keeps real target verification. Serve shape, Mooncake offload and search-space arms mirror the non-MTP sibling, trimmed at the top concurrencies. 中文：新增 minimaxm3-fp8-h100-vllm-agentic-mtp，即 MiniMax-M3 MXFP8 H100 智能体 编码配方的投机解码（spec-decoding=mtp）变体：使用 Inferact/MiniMax-M3-EAGLE3 草稿头、3 个投机 token、drafter 固定为 FLASH_ATTN，并将合成接受长度固定为已提交 的黄金 AL 2.83（minimaxm3_eagle3.yaml，thinking_on，K=3）。EVAL_ONLY 场景保留真实 目标验证。服务参数、Mooncake KV 卸载与搜索空间与非 MTP 版本保持一致，仅在高并发端 做了裁剪。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
cquil11
added
full-sweep-enabled
agentx-fast
|
Claude finished @cquil11's task in 6m 26s —— View job PR Review
SummaryOne 🟡 warning posted inline: the new comment block above One note I could not verify offline: every other synthetic-acceptance MTP recipe (dsv4, kimik3, minimaxm3 fp4) pins a vLLM nightly image, while this is the first to use the stable |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| # EAGLE3 speculative-decoding (spec-decoding: mtp) variant of | ||
| # minimaxm3-fp8-h100-vllm-agentic, pairing MiniMaxAI/MiniMax-M3-MXFP8 with the | ||
| # Inferact/MiniMax-M3-EAGLE3 draft head (3 speculative tokens, FLASH_ATTN | ||
| # drafter) and pinning synthetic acceptance to the golden AL 2.83 | ||
| # (golden_al_distribution/minimaxm3_eagle3.yaml, thinking_on, K=3). Same TP8-only | ||
| # layout and KV arms as the non-MTP entry so the spec-decode delta is readable at | ||
| # equal concurrency, trimmed at the extreme-conc end: the draft head plus its KV | ||
| # eat into the same HBM budget that already puts the GPU-resident cliff near | ||
| # conc 6 on 80 GB H100s. |
There was a problem hiding this comment.
🟡 WARNING: This comment block contradicts what the recipe actually runs, in three places:
- The script pins
DRAFT_MODEL="Inferact/MiniMax-M3-EAGLE3-GQA"(benchmarks/single_node/agentic/minimaxm3_fp8_h100_mtp.sh:28), notInferact/MiniMax-M3-EAGLE3. - The script pins
SYNTHETIC_ACCEPT_LEN=2.78, which isgolden_al_distribution/minimaxm3_eagle3_gqa.yamlthinking_on[3] — not 2.83 fromminimaxm3_eagle3.yaml(that's the non-GQA head's table). The perf-changelog entry in this PR also says 2.78. - "Same TP8-only layout and KV arms as the non-MTP entry" — this entry adds two DEP8 (
dp-attn: true) arms that the non-MTP entry doesn't have.
Why it matters: this comment is what reviewers and future submitters use to check which acceptance target a submission is pinned to; citing 2.83/minimaxm3_eagle3.yaml invites someone to "fix" the script to the wrong golden AL for the GQA draft.
Fix:
| # EAGLE3 speculative-decoding (spec-decoding: mtp) variant of | |
| # minimaxm3-fp8-h100-vllm-agentic, pairing MiniMaxAI/MiniMax-M3-MXFP8 with the | |
| # Inferact/MiniMax-M3-EAGLE3 draft head (3 speculative tokens, FLASH_ATTN | |
| # drafter) and pinning synthetic acceptance to the golden AL 2.83 | |
| # (golden_al_distribution/minimaxm3_eagle3.yaml, thinking_on, K=3). Same TP8-only | |
| # layout and KV arms as the non-MTP entry so the spec-decode delta is readable at | |
| # equal concurrency, trimmed at the extreme-conc end: the draft head plus its KV | |
| # eat into the same HBM budget that already puts the GPU-resident cliff near | |
| # conc 6 on 80 GB H100s. | |
| # EAGLE3 speculative-decoding (spec-decoding: mtp) variant of | |
| # minimaxm3-fp8-h100-vllm-agentic, pairing MiniMaxAI/MiniMax-M3-MXFP8 with the | |
| # Inferact/MiniMax-M3-EAGLE3-GQA draft head (3 speculative tokens, FLASH_ATTN | |
| # drafter) and pinning synthetic acceptance to the golden AL 2.78 | |
| # (golden_al_distribution/minimaxm3_eagle3_gqa.yaml, thinking_on, K=3). Keeps the | |
| # non-MTP entry's TP8/TEP8 layout and KV arms (plus DEP8 dp-attn arms) so the | |
| # spec-decode delta is readable at equal concurrency, trimmed at the | |
| # extreme-conc end: the draft head plus its KV eat into the same HBM budget | |
| # that already puts the GPU-resident cliff near conc 6 on 80 GB H100s. |
There was a problem hiding this comment.
Looks good — the only finding is a documentation-comment nit in the new config entry (wrong draft-head name/AL cited in the comment, with zero runtime impact since the script's actual constants are correct).
Extended reasoning...
Overview
This PR adds a new AgentX MTP (speculative decoding) benchmark recipe for MiniMax-M3 MXFP8 on H100, mirroring the existing non-MTP sibling (minimaxm3_fp8_h100.sh) and other MiniMax-M3 MTP recipes already in the repo (e.g. minimaxm3_fp4_b200_mtp.sh, minimaxm3_fp4_b300_mtp.sh). Changes are confined to a new benchmark script, a new config matrix entry in nvidia-master.yaml, and a perf-changelog.yaml entry — no production serving code, auth, or shared library logic is touched.
Security risks
None identified. This is benchmark/tooling infrastructure operating on trusted internal config; no user input, network-facing auth, or credential handling is introduced. The Mooncake KV-offload wiring and hf-download retry loop follow the same patterns already used elsewhere in the repo.
Level of scrutiny
Low-to-moderate is appropriate: this is a new but highly templated benchmark recipe, structurally near-identical to several already-merged siblings. The one bug found (comment citing the wrong draft-head name and golden AL in nvidia-master.yaml) is inert documentation — the script's actual DRAFT_MODEL=Inferact/MiniMax-M3-EAGLE3-GQA and SYNTHETIC_ACCEPT_LEN=2.78 are correct and consistent with the perf-changelog entry, so nothing incorrect actually executes.
Other factors
Two additional candidate issues (missing FULL_DECODE_ONLY cudagraph mode setting, and duplicated hf-download retry logic) were raised and refuted by verifier agents as not real bugs. The PR description reports clean bash/YAML validation and 231 passing matrix/schema tests. Given the nit is cosmetic-only and the rest of the change follows established conventions, I don't think this needs to block on human review.
|
|
||
| # EAGLE3 speculative-decoding (spec-decoding: mtp) variant of | ||
| # minimaxm3-fp8-h100-vllm-agentic, pairing MiniMaxAI/MiniMax-M3-MXFP8 with the | ||
| # Inferact/MiniMax-M3-EAGLE3 draft head (3 speculative tokens, FLASH_ATTN | ||
| # drafter) and pinning synthetic acceptance to the golden AL 2.83 | ||
| # (golden_al_distribution/minimaxm3_eagle3.yaml, thinking_on, K=3). Same TP8-only | ||
| # layout and KV arms as the non-MTP entry so the spec-decode delta is readable at | ||
| # equal concurrency, trimmed at the extreme-conc end: the draft head plus its KV | ||
| # eat into the same HBM budget that already puts the GPU-resident cliff near |
There was a problem hiding this comment.
🟡 The comment block above the new minimaxm3-fp8-h100-vllm-agentic-mtp entry (configs/nvidia-master.yaml:7108-7116) names the wrong draft head and golden AL: it says Inferact/MiniMax-M3-EAGLE3 and AL 2.83 from minimaxm3_eagle3.yaml, but the script it drives (minimaxm3_fp8_h100_mtp.sh) actually uses Inferact/MiniMax-M3-EAGLE3-GQA with AL 2.78 from minimaxm3_eagle3_gqa.yaml.
Extended reasoning...
The new minimaxm3-fp8-h100-vllm-agentic-mtp config entry at configs/nvidia-master.yaml:7108-7116 carries a doc comment describing the recipe's speculative-decoding setup. It states the recipe pairs the target with the Inferact/MiniMax-M3-EAGLE3 draft head and pins synthetic acceptance to golden AL 2.83 from golden_al_distribution/minimaxm3_eagle3.yaml (thinking_on, K=3).
That doesn't match the script this config entry actually drives, benchmarks/single_node/agentic/minimaxm3_fp8_h100_mtp.sh. Line 27 sets DRAFT_MODEL="Inferact/MiniMax-M3-EAGLE3-GQA" (the GQA variant, not the plain one named in the comment), and line 144 sets SYNTHETIC_ACCEPT_LEN=2.78, with the script's own adjacent comment correctly attributing that value to minimaxm3_eagle3_gqa.yaml thinking_on[3].
Step-by-step proof:
golden_al_distribution/minimaxm3_eagle3.yaml→minimax-m3.thinking_on[3] = 2.83(non-GQA head).golden_al_distribution/minimaxm3_eagle3_gqa.yaml→minimax-m3.thinking_on[3] = 2.78(GQA head).- The script sets
DRAFT_MODEL=Inferact/MiniMax-M3-EAGLE3-GQAandSYNTHETIC_ACCEPT_LEN=2.78— this is the GQA value from file (2), not the 2.83 from file (1). perf-changelog.yaml's new entry for this PR independently confirms this, stating "EAGLE3-GQA synthetic golden AL 2.78".- Yet the
nvidia-master.yamlcomment cites the non-GQA model name, the non-GQA file, and the non-GQA AL value (2.83) — all three details are inconsistent with what the script and changelog actually implement.
The two golden_al_distribution files hold distinct measured acceptance-length values for genuinely different draft heads (GQA vs non-GQA attention in the EAGLE3 head), so this isn't a rounding or trivial wording slip — it names a different model checkpoint and cites a different measurement entirely. Nothing in the YAML schema or config loader cross-checks free-text comments against the script's runtime constants, so there's no existing mechanism that would have caught this drift, and it will keep misleading anyone reading the config to reconcile it with what's actually benchmarked.
There is no runtime impact: the comment is inert documentation and the script's own hardcoded DRAFT_MODEL/SYNTHETIC_ACCEPT_LEN values are what actually execute. The fix is simply to update the comment at configs/nvidia-master.yaml:7108-7116 to say Inferact/MiniMax-M3-EAGLE3-GQA, cite golden_al_distribution/minimaxm3_eagle3_gqa.yaml, and use golden AL 2.78, matching both the script and the new perf-changelog entry.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31540120174 |
恢复基础设施丢失点，并在 H100 MiniMax-M3 拐点附近测试 vLLM SimpleCPUOffloadConnector。
…agentx-v27 # Conflicts: # perf-changelog.yaml
将 vLLM SimpleCPUOffloadConnector 接入实际的 MiniMax-M3 MTP 运行时，按检查点与模型预留计算主机内存，并排除已确认 NVML 故障的 H100 节点。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31558793046 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31564355779 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31564355779 |
2 participants
Summary
Refreshes MiniMax-M3 MXFP8 AgentX on H100 with current stable vLLM, the EAGLE3-GQA draft, strict server-metric validation, and a broad fast-search matrix across TP8, TEP8, DEP8, and Mooncake DRAM offload.
The initial
agentx-fastsweep will prune dominated or nonfunctional arms before the final full-duration Pareto sweep. Single-node TP8 is retained because the official recipe supports 8×H100 and prior green Mooncake runs already cover the capacity tier; a two-node aggregate topology will only be added if the fast results show a missing viable region.Supersedes the stale/conflicting experiment in #2423.
Initial search
Validation