Skip to content

Fix Kimi K3 B200 AIPerf metrics configuration #2569

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
cquil11 wants to merge 38 commits into
base: main
Choose a base branch
from
+16 −41
Open

Fix Kimi K3 B200 AIPerf metrics configuration #2569

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
f94cabe
perf(agentx): add B200 Kimi K3 DSpark refresh
cquil11 Aug 3, 2026
256df24
chore: link B200 DSpark refresh PR
cquil11 Aug 3, 2026
9515848
feat: add b200 kimi k3 agentx profiles
adibarra Aug 3, 2026
c5fa8da
fix: bypass unsupported b200 custom collectives
adibarra Aug 3, 2026
88d3bb7
fix: correct b200 dep launch accounting
adibarra Aug 3, 2026
3cb4971
fix: force pynccl for cross-node tp
adibarra Aug 3, 2026
94d677a
fix: disable cross-node latent tail fusion
adibarra Aug 3, 2026
349ddfd
fix: disable cross-node allreduce fusion
adibarra Aug 3, 2026
48dffa1
fix: use viable b200 tp8pp2 profiles
adibarra Aug 3, 2026
4508808
fix: keep dspark on pp-free tp16
adibarra Aug 3, 2026
f862b86
chore: merge main into B200 branch
adibarra Aug 10, 2026
3bf7616
fix: test B200 with modern Kimi nightly
adibarra Aug 10, 2026
ef1ffac
fix: disable fused Kimi latent tail
adibarra Aug 10, 2026
eac9a89
fix: force portable Kimi collectives
adibarra Aug 10, 2026
08ca238
fix: install Kimi compatibility setup
adibarra Aug 10, 2026
48ea8e6
fix: bypass unsupported Kimi workspaces
adibarra Aug 10, 2026
6c86aff
fix: force PyNCCL for Kimi
adibarra Aug 10, 2026
0d8d71b
fix: terminate Kimi changelog entry
adibarra Aug 10, 2026
cdcd4a7
refactor: remove obsolete DEP accounting changes
adibarra Aug 10, 2026
c4f8ad5
fix: reserve sufficient Kimi offload memory
adibarra Aug 10, 2026
b37142c
chore: remove stale DEP launcher comment
adibarra Aug 10, 2026
5167128
fix: cap offload at stable concurrency
adibarra Aug 10, 2026
629d447
fix: retain accepted offload capacity endpoint
adibarra Aug 10, 2026
4107dc6
chore: merge main and resolve conflicts
adibarra Aug 11, 2026
72b321e
fix: preserve changelog newline after merge
adibarra Aug 11, 2026
0a2c7eb
chore: merge main and resolve conflicts [skip-sweep]
adibarra Aug 11, 2026
977edcc
fix: use upstream kimi k3 b200 path
adibarra Aug 11, 2026
3be29f2
fix: run b200 tep in eager mode
adibarra Aug 11, 2026
7abf1db
fix: keep b200 tensor parallel node local
adibarra Aug 11, 2026
02bed65
chore: merge main and preserve changelog
adibarra Aug 11, 2026
a476441
ci: shorten multi-node eval artifact names
adibarra Aug 11, 2026
db438d5
fix: enforce Kimi K3 metrics collection
cquil11 Aug 11, 2026
2bb6eea
perf: probe Kimi K3 offload crossover
cquil11 Aug 11, 2026
5c1e3de
Merge origin/main into Kimi K3 B200 tuning
cquil11 Aug 12, 2026
0c06e9b
fix: pass Kimi K3 metrics through supported env
cquil11 Aug 12, 2026
22ae693
Merge current main for Kimi K3 metrics fix
cquil11 Aug 12, 2026
258c6e4
chore: document Kimi K3 metrics fix
cquil11 Aug 12, 2026
5b132ba
perf: retain measured Kimi K3 B200 frontier
cquil11 Aug 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...i_node/srt-slurm-recipes/vllm/kimi-k3/agentic/agg-b200-tp8dp2-latency-dspark-agentic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,12 +95,12 @@ srun_options:

benchmark:
type: custom
aiperf_server_metrics: true
command: bash /infmax-workspace/benchmarks/multi_node/agentic_srt.sh
env:
INFMAX_CONTAINER_WORKSPACE: "/infmax-workspace"
AIPERF_TRACE_IDLE_GAP_CAP_SECONDS: "300"
Comment on lines 98 to 101

@claude claude Bot Aug 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 This PR modifies 4 Kimi K3 B200 recipe YAMLs (swapping benchmark.aiperf_server_metrics for AIPERF_SERVER_METRICS_URLS) but does not append a new perf-changelog.yaml entry, violating the AGENTS.md non-negotiable invariant that every recipe addition or modification requires one. Please append a new entry for config-key kimik3-fp4-b200-dynamo-vllm-agentic-dspark with a pr-link to this PR (#2569), describing the AIPerf metrics-URL fix, at the tail of the file.

Extended reasoning...

AGENTS.md § "Non-negotiable benchmark invariants" (line 21) states explicitly: "Every change that can affect benchmark performance and every recipe addition or modification requires a new perf-changelog.yaml entry. The file is append-only and byte-sensitive. Preserve all existing bytes and separator whitespace, and append only at the tail." This PR modifies four recipe YAMLs under benchmarks/multi_node/srt-slurm-recipes/vllm/kimi-k3/agentic/, removing the unsupported benchmark.aiperf_server_metrics: true field and adding AIPERF_SERVER_METRICS_URLS to the benchmark.env block. This is unambiguously a "recipe modification" under the stated rule, so a new changelog entry is required regardless of whether the change is purely a config-key fix rather than a topology change.

I confirmed directly against the repository that perf-changelog.yaml (root of repo, ~5820 lines) is not among the PR's 4 changed files, and its tail still ends with two entries for PR #2475 under config-key kimik3-fp4-b200-dynamo-vllm-agentic-dspark — the exact config-key that covers these four recipes. There is no entry referencing PR #2569 (this PR's number) anywhere in the file.

This omission is not merely a style nit: the changelog is the append-only audit trail the repo uses to track exactly which PR touched which recipe and why, and prior modifications to these same recipes were always accompanied by a changelog entry — e.g. PR #2475 (which originally added aiperf_server_metrics: true) appended two entries at what are now lines 5807-5820, and PR #2364 shows a directly analogous precedent, adding a changelog entry specifically for a server-metrics-prefix config change ("Enable SGLang engine metrics and require the sglang: server-metric prefix..."). This PR's change is the vLLM equivalent of that exact kind of metrics-config fix, yet appends nothing.

Step-by-step proof:

  1. Read AGENTS.md line 21 — invariant requires a changelog entry for every recipe modification, not just performance-affecting ones (the two clauses are joined by "and", not "or", but both apply here as a metrics-related change affects observability of benchmark performance).
  2. List this PR's changed files — only the 4 recipe YAMLs, perf-changelog.yaml is absent.
  3. tail -40 perf-changelog.yaml — file ends at PR [AgentX] Tune Kimi K3 DSpark on B200 #2475 and [NV] Add H200 DeepSeek-V4-Pro AgentX recipes / [NV] 添加 H200 DeepSeek-V4-Pro AgentX 配方 #2364 entries; no PR Fix Kimi K3 B200 AIPerf metrics configuration #2569 entry exists.
  4. grep -n "kimik3-fp4-b200-dynamo-vllm-agentic-dspark" perf-changelog.yaml — shows only the pre-existing [AgentX] Tune Kimi K3 DSpark on B200 #2475 entries covering the recipes this PR touches, confirming these recipes are tracked by the changelog and thus require an update when modified.
  5. Conclusion: the invariant is violated as written.

Fix: Append a new entry at the tail of perf-changelog.yaml (preserving existing bytes/whitespace) with config-keys: [kimik3-fp4-b200-dynamo-vllm-agentic-dspark], a description summarizing the AIPerf metrics-URL fix (replacing the unsupported aiperf_server_metrics field with AIPERF_SERVER_METRICS_URLS env var), and pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2569.

This is flagged as an explicit, non-negotiable repository invariant (not subjective style preference), and the repo's own history shows every prior touch of these recipes carried a changelog entry, so it should be addressed before merge.

AIPERF_LIVE_FAILED_REQUEST_THRESHOLD: "0.25"
AIPERF_SERVER_METRICS_URLS: "http://localhost:8000/metrics"
AIPERF_REQUIRED_SERVER_METRIC_PREFIX: "vllm:"
RESULT_DIR: "/logs/agentic"
PORT: "8000"
Expand Down
2 changes: 1 addition & 1 deletion ...e/srt-slurm-recipes/vllm/kimi-k3/agentic/agg-b200-tp8dp2-latency-dspark-eval-agentic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,12 +93,12 @@ srun_options:

benchmark:
type: custom
aiperf_server_metrics: true
command: bash /infmax-workspace/benchmarks/multi_node/agentic_srt.sh
env:
INFMAX_CONTAINER_WORKSPACE: "/infmax-workspace"
AIPERF_TRACE_IDLE_GAP_CAP_SECONDS: "300"
AIPERF_LIVE_FAILED_REQUEST_THRESHOLD: "0.25"
AIPERF_SERVER_METRICS_URLS: "http://localhost:8000/metrics"
AIPERF_REQUIRED_SERVER_METRIC_PREFIX: "vllm:"
RESULT_DIR: "/logs/agentic"
PORT: "8000"
Expand Down
2 changes: 1 addition & 1 deletion ...lurm-recipes/vllm/kimi-k3/agentic/agg-b200-tp8dp2-vllm-simple-offload-dspark-agentic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,12 +96,12 @@ srun_options:

benchmark:
type: custom
aiperf_server_metrics: true
command: bash /infmax-workspace/benchmarks/multi_node/agentic_srt.sh
env:
INFMAX_CONTAINER_WORKSPACE: "/infmax-workspace"
AIPERF_TRACE_IDLE_GAP_CAP_SECONDS: "300"
AIPERF_LIVE_FAILED_REQUEST_THRESHOLD: "0.25"
AIPERF_SERVER_METRICS_URLS: "http://localhost:8000/metrics"
AIPERF_REQUIRED_SERVER_METRIC_PREFIX: "vllm:"
RESULT_DIR: "/logs/agentic"
PORT: "8000"
Expand Down
2 changes: 1 addition & 1 deletion ...recipes/vllm/kimi-k3/agentic/agg-b200-tp8dp2-vllm-simple-offload-dspark-eval-agentic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,12 +94,12 @@ srun_options:

benchmark:
type: custom
aiperf_server_metrics: true
command: bash /infmax-workspace/benchmarks/multi_node/agentic_srt.sh
env:
INFMAX_CONTAINER_WORKSPACE: "/infmax-workspace"
AIPERF_TRACE_IDLE_GAP_CAP_SECONDS: "300"
AIPERF_LIVE_FAILED_REQUEST_THRESHOLD: "0.25"
AIPERF_SERVER_METRICS_URLS: "http://localhost:8000/metrics"
AIPERF_REQUIRED_SERVER_METRIC_PREFIX: "vllm:"
RESULT_DIR: "/logs/agentic"
PORT: "8000"
Expand Down
42 changes: 5 additions & 37 deletions configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8051,9 +8051,9 @@ kimik3-fp4-b200-dynamo-vllm-agentic-dspark:
agentic-coding:
- dram-utilization: 0.63
search-space:
# Low-latency and high-interactivity points.
# Retained resident latency curve from the completed broad fast sweep.
- spec-decoding: mtp
conc-list: [1, 2, 4]
conc-list: [2, 4, 8, 12]
prefill:
num-worker: 1
tp: 8
Expand All @@ -8067,44 +8067,12 @@ kimik3-fp4-b200-dynamo-vllm-agentic-dspark:
tp: 8
ep: 16
dp-attn: true
# Balanced medium-concurrency points.
- spec-decoding: mtp
conc-list: [8, 12, 16]
prefill:
num-worker: 1
tp: 8
ep: 16
dp-attn: true
additional-settings:
- "CONFIG_FILE=recipes/vllm/kimi-k3/agentic/agg-b200-tp8dp2-latency-dspark-agentic.yaml"
- "EVAL_CONFIG_FILE=recipes/vllm/kimi-k3/agentic/agg-b200-tp8dp2-latency-dspark-eval-agentic.yaml"
decode:
num-worker: 0
tp: 8
ep: 16
dp-attn: true
# GPU-resident throughput points around the prior c16-c32 KV cliff.
- spec-decoding: mtp
conc-list: [20, 24, 28, 32]
prefill:
num-worker: 1
tp: 8
ep: 16
dp-attn: true
additional-settings:
- "CONFIG_FILE=recipes/vllm/kimi-k3/agentic/agg-b200-tp8dp2-latency-dspark-agentic.yaml"
- "EVAL_CONFIG_FILE=recipes/vllm/kimi-k3/agentic/agg-b200-tp8dp2-latency-dspark-eval-agentic.yaml"
decode:
num-worker: 0
tp: 8
ep: 16
dp-attn: true
# CPU KV-offload crossover and capacity points. Keep the resident points
# above so the same concurrency can be compared with one variable changed.
# Retain the measured SimpleCPUOffloadConnector knee, immediate boundary,
# and high-capacity controls from the completed broad fast sweep.
- spec-decoding: mtp
kv-offloading: dram
kv-offload-backend: { name: vllm-simple, version: "13c59a3" }
conc-list: [8, 12, 16, 20, 24, 28, 32, 48, 64]
conc-list: [8, 12, 16, 28, 32, 48]
prefill:
num-worker: 1
tp: 8
Expand Down
7 changes: 7 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5818,3 +5818,10 @@
description:
- "Extend the SimpleCPUOffloadConnector grid to c8/c12/c16/c20/c24/c28/c32/c48/c64 to locate its crossover against the resident curve"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2475

- config-keys:
- kimik3-fp4-b200-dynamo-vllm-agentic-dspark
description:
- "Fix AIPerf server-metrics configuration for the pinned Kimi K3 srt-slurm renderer by passing the aggregate vLLM endpoint through the supported custom-benchmark environment contract"
- "After the complete 19-point AgentX-fast search, retain resident c2/c4/c8/c12 and SimpleCPUOffloadConnector c8/c12/c16/c28/c32/c48 for the full-duration latency curve, knee, cliff boundary, and high-capacity controls"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2569
Loading