Add DSV4 B200 disaggregated Dynamo SGLang MTP configuration / 新增 DSV4 B200 分离式 Dynamo SGLang MTP 配置 - #2554
Add DSV4 B200 disaggregated Dynamo SGLang MTP configuration / 新增 DSV4 B200 分离式 Dynamo SGLang MTP 配置#2554RohitNagraj wants to merge 2 commits into
Conversation
新增 DSV4 B200 nscale UCX 验证配置。
328eff9 to
0555089
Compare
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
Claude finished @RohitNagraj's task in 6m 51s —— View job PR Review — DSV4 FP4 B200 nscale Dynamo-SGLang MTP UCX canary
SummaryNo blocking issues found. Two 🟡 warnings posted inline, both on the new recipe: Checks that passed: Caveat: this sandbox has no network access, so I could not verify externally that the Docker Hub tag |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| # Keep RC for inter-node transport, but stage CUDA buffers through host memory. | ||
| UCX_TLS: "rc,cuda_ipc,cuda_copy,self" | ||
| UCX_IB_GPU_DIRECT_RDMA: "no" | ||
| UCX_PROTO_EMULATION_ENABLE: "y" |
There was a problem hiding this comment.
🟡 WARNING: UCX_PROTO_EMULATION_ENABLE does not appear to be a real UCX configuration variable
Why it matters: Mainline UCX exposes UCX_PROTO_ENABLE / UCX_PROTO_INFO, and proto-v2 "emulation" protocols are selected automatically when a transport lacks a capability — I can find no PROTO_EMULATION_ENABLE knob in the UCX config table. Unknown UCX_* vars are silently ignored (at most a "unused env var" warning), so the run will proceed, but the canary's stated third knob ("enable host-staged protocol emulation") would be a no-op and conclusions drawn from this arm could be misattributed. Host staging is still forced here by UCX_IB_GPU_DIRECT_RDMA=no + cuda_copy, so the experiment isn't broken — it just may not test what the PR description claims. (Same variable appears in the decode block at line 77.)
Fix: Verify the variable exists in the container's UCX build (ucx_info -c | grep -i proto), or rely on UCX_PROTO_INFO=y output (already enabled) to confirm which protocol was actually selected, and drop the variable if it's not recognized. Note: I could not verify against UCX source/docs from this sandbox (network access blocked), so please double-check before drawing conclusions from the run.
| sbatch_directives: | ||
| cpus-per-task: "144" | ||
| mem: "0" |
There was a problem hiding this comment.
🟡 WARNING: cpus-per-task: "144" looks copied from GB200/B300 recipes and may be unsatisfiable on nscale B200 nodes
Why it matters: 144 matches GB200/GB300 Grace nodes (2×72 cores) — see e.g. disagg-gb200-mid-curve-1p1d-dep8-dep16-mtp.yaml, which this recipe otherwise mirrors. The B200 nscale nodes are x86 (the sibling dsv4-fp4-b200-dynamo-vllm-mtp recipes use the vllm-x86_64 image), and none of the existing B200 nscale recipes set cpus-per-task (they use sbatch_directives: segment: "1" instead, e.g. benchmarks/multi_node/srt-slurm-recipes/vllm/deepseek-v4/8k1k/disagg-b200-1p1d-dep8-dep8-mtp.yaml:15). If the nscale nodes have fewer than 144 cores available per task, sbatch rejects the job with "Requested node configuration is not available" and the canary never starts.
Fix: Confirm the nscale B200 node core count supports --cpus-per-task=144, or drop the directive (and consider whether segment: "1" from the existing nscale recipes is needed) so Slurm uses its defaults.
There was a problem hiding this comment.
LGTM. This is a "[Do Not Merge]" canary PR whose content is byte-for-byte identical to commit 0555089, already merged to main separately — the same recipe, master-config entry, changelog entry, and launcher gating were already landed and validated. No new risk introduced.
What was reviewed: the new SGLang recipe's UCX/host-staging env vars for prefill vs. decode, the launcher's expanded framework/spec-decoding gate logic in launch_b200-nscale-slurm.sh, and the srt-slurm checkout/recipe-copy branching for the dynamo-sglang path. The flagged missing SGLANG_OPT_USE_CUSTOM_ALL_REDUCE_V2=1 in prefill_environment was checked against sibling multi-node DSv4 recipes — CAR_V2 is single-node only and correctly omitted here since prefill runs on 1 node with DEP8.
Extended reasoning...
Overview
The PR adds one new benchmark recipe YAML for a DeepSeek-V4-Pro FP4 Dynamo-SGLang MTP run on B200 nscale (1P/2D DEP8, host-staged UCX transport), a corresponding entry in configs/nvidia-master.yaml, a perf-changelog.yaml entry, and an extension of runners/launch_b200-nscale-slurm.sh to route this new framework/spec-decoding combination and pin the srt-slurm revision used for it.
Security risks
None. This is CI/benchmark infrastructure config — no auth, crypto, or user-facing surface. The launcher changes only affect environment-variable gating and which pinned public repo revision is checked out for benchmark execution.
Level of scrutiny
Low-to-moderate is appropriate: this is a benchmark-recipe addition following an extremely well-established pattern in this repo (dozens of similar recipes exist), and it is explicitly marked '[Do Not Merge]' — a canary intended only to validate the config via CI, not to be merged into main as-is. Notably, the entire diff (all 4 files) is byte-for-byte identical to commit 0555089, which already landed on main from the same author. That means this exact change has already been reviewed/merged through a separate path, materially reducing residual risk here.
Other factors
The bug-hunting pass found no issues, and the one candidate concern raised (missing CAR_V2 flag in the new prefill_environment) was checked against sibling recipes and confirmed to be correct as written — CAR_V2 is intentionally single-node-only and prefill here runs on a single node. The launcher's new gating conditional was traced against SPEC_DECODING, which is a standard workflow-exported env var used identically across many other launcher scripts, so no wiring risk there either.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31449085188 |
扩展 DSV4 B200 Dynamo-SGLang MTP 配置，补充五个分离式拓扑并保持配置与 recipe 一致。
RohitNagraj
changed the title
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31451666262 |
1 participant
Description
Add a DSV4 B200 disaggregated Dynamo SGLang MTP configuration for 8k/1k.
Validation:
中文说明
新增 DSV4 在 B200 上的分离式 Dynamo SGLang MTP 8k/1k 配置。
验证：
Related Issue
N/A
Type of Change
Checklist
perf-changelog.yamland have not edited historical entriesOWNER/MEMBER/COLLABORATOR) has commented/reuse-sweep-runon this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the sweep label will no longer automatically kick off new sweeps. Remove and re-add the label to force one.