Skip to content

Clean up splash attention imports #4606

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
huytransformer wants to merge 1 commit into
base: main
Choose a base branch
from
+2 −2
Open

Clean up splash attention imports #4606

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/maxtext/layers/attention_op.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,6 @@
from maxtext.kernels.attention import tokamax_ring_attention
from maxtext.kernels.attention.ragged_attention import ragged_gqa
from maxtext.kernels.attention.ragged_attention import ragged_mha
from maxtext.kernels.tokamax_splash_attention import splash_attention_kernel as tokamax_splash_kernel
from maxtext.kernels.tokamax_splash_attention import splash_attention_mask as tokamax_splash_mask
from maxtext.layers import nnx_wrappers
from maxtext.layers.initializers import variable_to_logically_partitioned
from maxtext.layers.quantizations import AqtQuantization as Quant
Expand All @@ -77,6 +75,8 @@
import numpy as np
from tokamax._src.ops.attention import base as tokamax_attention_base
from tokamax._src.ops.attention import pallas_triton as tokamax_pallas_triton
from tokamax._src.ops.experimental.tpu.splash_attention import splash_attention_kernel as tokamax_splash_kernel
from tokamax._src.ops.experimental.tpu.splash_attention import splash_attention_mask as tokamax_splash_mask

@NuojCheng NuojCheng Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect an if-else path, e.g. if using ring attention we take forked splash otherwise tokamax path. If the changes are simple like this, why we need to fork all files from tokamax at the beginning?

@huytransformer huytransformer Jul 24, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ring never needed this, it uses the tokamax_ring_attention. These were originally intended for consistency of splash paths

# pylint: disable=line-too-long, g-doc-args, g-doc-return-or-yield, bad-continuation, g-inconsistent-quotes
# pytype: disable=attribute-error

Expand Down
Loading