Skip to content

QVAC-21981 fix: compose the walk attention mask explicitly (old-ggml CUDA compat) - #27

Open
DmitryMalishev wants to merge 1 commit into
2026-07-03from
fix/abot-masked-softmax-old-ggml
Open

QVAC-21981 fix: compose the walk attention mask explicitly (old-ggml CUDA compat)#27
DmitryMalishev wants to merge 1 commit into
2026-07-03from
fix/abot-masked-softmax-old-ggml

Conversation

@DmitryMalishev

@DmitryMalishev DmitryMalishev commented Aug 4, 2026

Copy link
Copy Markdown

What

Removes the ABot walk's dependency on ggml's fused masked soft-max, whose CUDA kernel in the registry's sanctioned 2026-07-03 ggml line (50cf5630) rejects the walk's 2D-mask head broadcast — the first block aborted with ggml_cuda_compute_forward: SOFT_MAX failed, "invalid argument". That incompatibility is the only reason the monorepo addon carries a ggml overlay repin and why qvac-registry-vcpkg#284 initially proposed a ggml REF change (rejected in review — the ltx-lineage REF is not the diffusion line, and the branch tip carries an embargoed commit).

The change (one call-site, ABot-only): WanSelfAttention::forward routes the masked case through forward_kv with no cached context — the KV-cache formulation that composes scale → add(mask) → soft_max from primitive ops supported by every ggml revision in play. Only the ABot walk ever passes a mask; every other Wan model uses the nullptr default and is untouched. forward_kv was built during the KV-cache work to mirror the fused path op-for-op, so this is a switch between two already-gated formulations, not new math.

Validation (all against ggml 50cf5630, the registry REF)

  • CPU golden replay (F16 DiT, 4 blocks, recompute path — the changed one) vs the PyTorch reference: WALK PASS at cosine 0.999951 / 0.998994 / 0.997401 / 0.992991 — identical to the fused path's gated results, i.e. the two formulations are numerically equivalent.
  • CUDA (RTX 4050): recompute and KV walks both complete with coherent frames on the exact ggml revision that previously aborted in SOFT_MAX on block 0.
  • Engine builds cleanly against 50cf5630 on CPU (MinGW) and CUDA (MSVC/nvcc) — first builds of this engine against the registry ggml line.

Regression safety: provably ABot-only

⚠️ TL;DR for review: the diff is +12 lines / 0 deletions in one function, inside if (mask != nullptr) — and only the ABot walk ever passes a mask. Every other workflow executes a byte-identical instruction path.

Verified call-graph containment:

  1. WanSelfAttention::forward(…, mask = nullptr) receives a mask argument in exactly one place: WanAttentionBlock::forward (wan.hpp:1744), whose attn_mask is also defaulted nullptr.
  2. WanAttentionBlock::forward receives that argument in exactly one place in the tree: abot_world.hpp:389 (the walk). The standard Wan generation path (wan.hpp:2223 — Wan 2.1 T2V/I2V, Wan 2.2 TI2V/A14B, VACE) omits it → nullptr → unchanged path.
  3. SD/SDXL/SD3/FLUX/LTX/Ideogram/ESRGAN never instantiate Wan attention; other self_attn->forward matches in the repo (clip.hpp, llm.hpp, ernie_image.hpp, anima.hpp) are unrelated classes.

Even for ABot this is not a behavior change: the golden replay on the routed formulation reproduced the fused path's gates at identical cosines to six decimals. The monorepo's full 73-test GPU suite (incl. real Wan 2.1/2.2 and LTX generations) re-runs against the merged tip automatically when the registry port (qvac-registry-vcpkg#284) is adopted by tetherto/qvac#3352, before anything reaches consumers.

Effect downstream

  • qvac-registry-vcpkg can publish stable-diffusion-cpp 2026-07-03#6 pointing at this branch's merge tip while keeping ggml at 50cf5630 (PR LYCORIS Model leejet/stable-diffusion.cpp#284 will repoint its sd REF here once merged).
  • The monorepo addon can then delete both of its overlay ports (stable-diffusion-cpp and the ggml repin) and run entirely on registry ports.

🤖 Generated with Claude Code

The ABot walk's recompute path reached ggml's fused masked soft-max, whose
CUDA kernel in the registry's 2026-07-03 ggml line (50cf5630) rejects the
2D-mask head broadcast: the walk aborted on its first block with
"ggml_cuda_compute_forward: SOFT_MAX failed, invalid argument". The engine
submodule's newer ggml accepts it, which is why the addon has carried a
ggml overlay repin - blocking the registry from staying on its sanctioned
ggml REF.

WanSelfAttention::forward now routes the masked case (only the ABot walk
ever passes a mask; every other Wan model uses the nullptr default) through
forward_kv with no cached context - the KV-cache formulation that composes
scale -> add(mask) -> soft_max explicitly from primitive ops supported by
every ggml revision in play. forward_kv was built to mirror the fused path
op-for-op and both formulations were already gated against the goldens.

Validated against ggml 50cf5630 (the registry REF):
- CPU golden replay, recompute path, 4 blocks vs the PyTorch reference:
  WALK PASS at cosine 0.999951 / 0.998994 / 0.997401 / 0.992991 -
  identical to the fused path's gated results, i.e. the two formulations
  are numerically equivalent
- CUDA (RTX 4050): recompute and KV walks both complete with coherent
  frames - previously the first block aborted in SOFT_MAX on this exact
  ggml revision

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant