Support scalar (non-channelwise) decay in the GDN kernels - #21844
Support scalar (non-channelwise) decay in the GDN kernels#21844JakeStevens wants to merge 1 commit into
Conversation
Summary: `channelwise_gated_delta_rule` and `gated_delta_net` only accepted a per-channel decay gate (`[B, Hkv, T, K]`). Qwen3-Next-style GatedDeltaNet produces one gate per head (`[B, Hkv, T]`), so it had no path to the optimized kernels and fell back to a Python token-by-token recurrence. This offers a replacement for the previous reverted implentation that hooks into our optimized per-channel version. Differential Revision: D114738016
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21844
Note: Links to docs will display an error until the docs builds have been completed. ❌ 2 New FailuresAs of commit 6021dff with merge base ea3641d ( NEW FAILURES - The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@JakeStevens has exported this pull request. If you are a Meta employee, you can view the originating Diff in D114738016. |
This PR needs a
|
Summary:
channelwise_gated_delta_ruleandgated_delta_netonly accepted a per-channel decay gate ([B, Hkv, T, K]).Qwen3-Next-style GatedDeltaNet produces one gate per head (
[B, Hkv, T]), so it had no path to the optimized kernels and fell back to a Python token-by-token recurrence.This offers a replacement for the previous reverted implentation that hooks into our optimized per-channel version.
Differential Revision: D114738016