bench(bw): sweep decode batch size to measure cross-token expert dedup

[gdevenyi]

What this adds

ft bench bw measures one decode step at batch 1. Cross-token expert dedup only does anything when several tokens in a batch route to the same expert, and at batch 1 no expert can repeat. The existing benchmark therefore cannot show whether dedup works.

--batch 1,8,32,64 sweeps the decode batch size and reports, per size, the route count, the distinct expert count, the resulting reuse factor, and the step cost with dedup on and off:

  bs  routes  uniq  reuse   dedup off   dedup on  speedup
   1       8     8  1.00x      1.09ms     0.98ms    1.11x
   8      64    52  1.23x      7.95ms     6.55ms    1.21x
  16     128    82  1.56x     13.67ms     9.70ms    1.41x
  32     256   112  2.29x     22.95ms    13.18ms    1.74x
  64     512   126  4.06x     39.49ms    19.10ms    2.07x

The routing draw is made once per batch size and replayed for both arms, so the two sides do identical work. Each arm gets its own executor, because the dedup decision is cached per executor and a shared one would measure the same path twice.

Independent of the dedup work

This is one commit on main. It touches benchbw.py and its test, and nothing else, so it can be reviewed and merged on its own.

It does not require the cpu-moe dedup PRs to be useful. Without them the dedup off and dedup on columns report the same path, and the sweep still shows the route count, the distinct expert count and the reuse factor, which is the part that stands alone.

Testing

tests/moe/test_benchbw_batch.py covers the routing replay, the reuse arithmetic, and the per-arm executor. On main with this PR alone: 16 passed.

[gdevenyi]

Merged this alongside my dedup series (#46/#47/#49) — they answer the same question from opposite ends, and together they close it.

This PR measures cross-token expert dedup as a property of the workload: how much reuse exists at a given batch size. My PRs exploit that reuse in the CPU MoE kernel by restructuring pass 1 and pass 2 around (unique expert, row block) so each expert's rows are read from DRAM once per step instead of once per route.

Measured on 2× Xeon Gold 6526Y (E=256, top_k=8, H=2048, I=768, uniform-random routing):

bs	routes	unique	reuse	dedup off	dedup on	delta
16	128	104	1.23×	14.76 ms	11.34 ms	+30%
32	256	156	1.64×	29.92 ms	17.21 ms	+74%
64	512	213	2.40×	59.10 ms	22.45 ms	+163%

The speedups track 1 / (2/3 / reuse + 1/3) to within 2%, so the reuse factor this PR measures is directly predictive of the achievable gain — which makes the two changes complementary rather than redundant. Having the sweep in ft bench bw means a user can see whether their batch sizes are in the regime where dedup pays before turning anything on.

Merge conflicts with my series were purely additive (both add keys to the same entry dict and parameters to _bench_format); resolved by keeping both in deploy/chatdnp-v2, no semantic conflict.

One suggestion from having chased this: report the reuse factor in the printed table, not just the JSON. The single most useful number here is "routes / unique", and when I was debugging my own dedup a printed reuse factor is what caught a toggle that had silently disabled itself — a static-cached env read made an A/B measure the same path twice and report "+1.4%, no effect" for a change actually worth +163%.