Skip to content

perf(pd): reduce qwen3next prefill linear-att state memory - #1421

Open
sufubao wants to merge 1 commit into
ModelTC:mainfrom
sufubao:perf-pd-prefill-linear-att-state
Open

perf(pd): reduce qwen3next prefill linear-att state memory#1421
sufubao wants to merge 1 commit into
ModelTC:mainfrom
sufubao:perf-pd-prefill-linear-att-state

Conversation

@sufubao

@sufubao sufubao commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

PD 分离部署下,qwen3next 的 linear-att(conv/ssm)状态缓冲按 mtp_step + 1 个槽/请求分配。但 MTP verify 只在 decode 节点发生,prefill 节点只做 prefill 并把 KV 交给 decode,从不运行 verify,所以第 1..mtp_step 个槽在 prefill 节点对每个请求都空着。

引入 get_linear_att_state_mtp_size(args)run_mode == "prefill" 返回 1,其余(normal / decode)仍为 mtp_step + 1。prefill 节点上:

  • ssm 状态缓冲:(max_req + 1) * (mtp_step + 1)(max_req + 1) * 1,约 mtp_step + 1 倍缩减;
  • conv 状态最后一维:(kernel - 1) + mtp_step(kernel - 1)

normal 与 decode 路径不变。

正确性:KV 搬运页 kv_move_buffer 每个请求只承载一份 ssm 状态(ssm_shape 来自单个状态,与 mtp 无关);_get_req_state_indexes 用 stride 只是在各自节点的 gpu ssm 缓冲里定位每个请求的第 0 行(canonical 槽)参与搬运——prefill(stride=1)与 decode(stride=mtp_step+1)取到的是同一份状态,两侧页格式一致。kernel copy_linear_att_state_to_kv_buffermtp_step 形参相应改为 ssm_state_stride,传入实际步长。

只做了 py_compile / 静态检查;E2E PD 显存实测待补。

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant