QVAC-21981 fix: compose the walk attention mask explicitly (old-ggml CUDA compat) - #27
Open
DmitryMalishev wants to merge 1 commit into
Open
QVAC-21981 fix: compose the walk attention mask explicitly (old-ggml CUDA compat)#27DmitryMalishev wants to merge 1 commit into
DmitryMalishev wants to merge 1 commit into
Conversation
The ABot walk's recompute path reached ggml's fused masked soft-max, whose CUDA kernel in the registry's 2026-07-03 ggml line (50cf5630) rejects the 2D-mask head broadcast: the walk aborted on its first block with "ggml_cuda_compute_forward: SOFT_MAX failed, invalid argument". The engine submodule's newer ggml accepts it, which is why the addon has carried a ggml overlay repin - blocking the registry from staying on its sanctioned ggml REF. WanSelfAttention::forward now routes the masked case (only the ABot walk ever passes a mask; every other Wan model uses the nullptr default) through forward_kv with no cached context - the KV-cache formulation that composes scale -> add(mask) -> soft_max explicitly from primitive ops supported by every ggml revision in play. forward_kv was built to mirror the fused path op-for-op and both formulations were already gated against the goldens. Validated against ggml 50cf5630 (the registry REF): - CPU golden replay, recompute path, 4 blocks vs the PyTorch reference: WALK PASS at cosine 0.999951 / 0.998994 / 0.997401 / 0.992991 - identical to the fused path's gated results, i.e. the two formulations are numerically equivalent - CUDA (RTX 4050): recompute and KV walks both complete with coherent frames - previously the first block aborted in SOFT_MAX on this exact ggml revision Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Removes the ABot walk's dependency on ggml's fused masked soft-max, whose CUDA kernel in the registry's sanctioned
2026-07-03ggml line (50cf5630) rejects the walk's 2D-mask head broadcast — the first block aborted withggml_cuda_compute_forward: SOFT_MAX failed, "invalid argument". That incompatibility is the only reason the monorepo addon carries a ggml overlay repin and why qvac-registry-vcpkg#284 initially proposed a ggml REF change (rejected in review — the ltx-lineage REF is not the diffusion line, and the branch tip carries an embargoed commit).The change (one call-site, ABot-only):
WanSelfAttention::forwardroutes the masked case throughforward_kvwith no cached context — the KV-cache formulation that composesscale → add(mask) → soft_maxfrom primitive ops supported by every ggml revision in play. Only the ABot walk ever passes a mask; every other Wan model uses thenullptrdefault and is untouched.forward_kvwas built during the KV-cache work to mirror the fused path op-for-op, so this is a switch between two already-gated formulations, not new math.Validation (all against ggml
50cf5630, the registry REF)SOFT_MAXon block 0.50cf5630on CPU (MinGW) and CUDA (MSVC/nvcc) — first builds of this engine against the registry ggml line.Regression safety: provably ABot-only
Verified call-graph containment:
WanSelfAttention::forward(…, mask = nullptr)receives a mask argument in exactly one place:WanAttentionBlock::forward(wan.hpp:1744), whoseattn_maskis also defaultednullptr.WanAttentionBlock::forwardreceives that argument in exactly one place in the tree:abot_world.hpp:389(the walk). The standard Wan generation path (wan.hpp:2223— Wan 2.1 T2V/I2V, Wan 2.2 TI2V/A14B, VACE) omits it → nullptr → unchanged path.self_attn->forwardmatches in the repo (clip.hpp,llm.hpp,ernie_image.hpp,anima.hpp) are unrelated classes.Even for ABot this is not a behavior change: the golden replay on the routed formulation reproduced the fused path's gates at identical cosines to six decimals. The monorepo's full 73-test GPU suite (incl. real Wan 2.1/2.2 and LTX generations) re-runs against the merged tip automatically when the registry port (qvac-registry-vcpkg#284) is adopted by tetherto/qvac#3352, before anything reaches consumers.
Effect downstream
qvac-registry-vcpkgcan publishstable-diffusion-cpp 2026-07-03#6pointing at this branch's merge tip while keeping ggml at50cf5630(PR LYCORIS Model leejet/stable-diffusion.cpp#284 will repoint its sd REF here once merged).stable-diffusion-cppand the ggml repin) and run entirely on registry ports.🤖 Generated with Claude Code