Skip to content

Update DSv4 B200 TRT image to 2dd03e6 (non-MTP + MTP) #1664

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Oseltamivir wants to merge 6 commits into from
+12 −5
Closed

Update DSv4 B200 TRT image to 2dd03e6 (non-MTP + MTP) #1664

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
700e5ec
Use official TRT-LLM image (1.3.0rc15.post1) for DSv4 B200 TRT
Oseltamivir Jun 4, 2026
c8c6b3c
Switch B200 DSv4 TRT image to 2dd03e6
Oseltamivir Jun 4, 2026
7baa914
Merge branch 'main' into cap-dsv4-b200-trt-conc
Oseltamivir Jun 5, 2026
9ff8446
Disable overlap scheduler for DSv4 B200 TRT (pin max_num_requests=256)
Oseltamivir Jun 5, 2026
e9268da
Merge branch 'main' into cap-dsv4-b200-trt-conc
Oseltamivir Jun 5, 2026
80aed07
Use official TRT-LLM image (1.3.0rc15.post1) for DSv4 B200 TRT
Oseltamivir Jun 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1801,7 +1801,7 @@ dsv4-fp4-b200-vllm-agentic:
- { tp: 8, ep: 8, dp-attn: true, offloading: cpu, conc-list: [64, 128, 256] }

dsv4-fp4-b200-trt:
image: ghcr.io#semianalysisai/trtllm-deepseek-v4:feat-deepseek_v4-9aa3715
image: nvcr.io#nvidia/tensorrt-llm/release:1.3.0rc15.post1
model: deepseek-ai/DeepSeek-V4-Pro
model-prefix: dsv4
runner: b200-dsv4
Expand All @@ -1814,15 +1814,15 @@ dsv4-fp4-b200-trt:
osl: 1024
search-space:
- { tp: 8, conc-start: 1, conc-end: 32 }
- { tp: 8, ep: 8, dp-attn: true, conc-start: 32, conc-end: 2048 }
- { tp: 8, ep: 8, dp-attn: true, conc-start: 32, conc-end: 128 }
- isl: 8192
osl: 1024
search-space:
- { tp: 8, conc-start: 1, conc-end: 32 }
- { tp: 8, ep: 8, dp-attn: true, conc-start: 32, conc-end: 1024 }
- { tp: 8, ep: 8, dp-attn: true, conc-start: 32, conc-end: 256 }

dsv4-fp4-b200-trt-mtp:
image: ghcr.io#semianalysisai/trtllm-deepseek-v4:feat-deepseek_v4-9aa3715
image: nvcr.io#nvidia/tensorrt-llm/release:1.3.0rc15.post1
model: deepseek-ai/DeepSeek-V4-Pro
model-prefix: dsv4
runner: b200-dsv4
Expand All @@ -1835,7 +1835,7 @@ dsv4-fp4-b200-trt-mtp:
osl: 1024
search-space:
- { tp: 8, conc-start: 1, conc-end: 32, spec-decoding: mtp }
- { tp: 8, ep: 8, dp-attn: true, conc-start: 32, conc-end: 512, spec-decoding: mtp }
- { tp: 8, ep: 8, dp-attn: true, conc-start: 32, conc-end: 128, spec-decoding: mtp }
- isl: 8192
osl: 1024
search-space:
Expand Down
7 changes: 7 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3459,6 +3459,13 @@
- "Add 1k1k/8k1k FP8 recipe set under benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m2.5-gb200-fp8/"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1648

- config-keys:
- dsv4-fp4-b200-trt
- dsv4-fp4-b200-trt-mtp
description:
- "Update B200 DeepSeek-V4-Pro TRT image to the official nvcr.io#nvidia/tensorrt-llm/release:1.3.0rc15.post1 (non-MTP and MTP), replacing the older ghcr.io semianalysis 9aa3715 build. The official release uses the V1 KV-cache manager (use_kv_cache_manager_v2=False), avoiding the custom feat/deepseek_v4 build's V2 max_num_requests=2x doubling that OOM'd conc-256 dpa=true on B200, and runs the overlap scheduler natively (mirrors the B300 setup in PR #1636 / run 26999118817)."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1664

- config-keys:
- minimaxm2.5-fp8-b300-dynamo-vllm
description:
Expand Down
Loading