Skip to content

[Fix] Wire cudac bindings#105

Merged
icavan merged 2 commits into
inclusionAI:mainfrom
cherhh:fix-cudac-bindings
Jul 20, 2026
Merged

[Fix] Wire cudac bindings#105
icavan merged 2 commits into
inclusionAI:mainfrom
cherhh:fix-cudac-bindings

Conversation

@cherhh

@cherhh cherhh commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

📌 Description

This PR fixes the Python bindings used by setup.py builds for the SM90 and SM100/SM103 prefill extensions.

#86 added two optional prefill arguments and moved the binding definitions into csrc/api/pybind.cu, where their names and default values are defined. However, that new binding file was not wired into the CUDA extension sources, so source builds still used the legacy unnamed bindings. As a result:

  • the basic wrapper failed when the optional arguments were omitted;
  • the optimized and CP paths could not pass those arguments by keyword.

This PR wires the shared binding file into both architecture-specific extensions, enables the SM100/SM103 binding guards from setup.py, and removes the duplicate legacy PYBIND11_MODULE blocks.

No kernel computation or routing logic is changed.

🔍 Related Issues

Follow-up to #86.

🚀 Pull Request Checklist

Thank you for contributing to cuLA! Before we review your pull request, please make sure the following items are complete.

✅ Pre-commit Checks

  • I have installed pre-commit by running pip install pre-commit (or used my preferred method).
  • I have installed the hooks with pre-commit install.
  • I have run the hooks manually with pre-commit run --all-files and fixed any reported issues.

pre-commit run --all-files and git diff --check both pass.

🧪 Tests

  • Tests have been added or updated as needed.
  • All tests are passing.

Validation performed on an H100:

  • Rebuilt the SM90 extension from source.
  • Verified both affected call styles reach the kernel:
    • basic wrapper: positional call with optional arguments omitted;
    • optimized/CP wrappers: keyword calls with the optional arguments.
  • Ran the SM90 C++ test suites: 113 passed, 1 failed.

The remaining failure is pre-existing and hardware-dependent: test_cp_bypass_matches_basic[128K + 5×1K, H=8]. On a 132-SM H100, the CP heuristic engages, while the test table expects a bypass. The assertion fails in a pure-Python precondition before any kernel runs, so it is independent of this binding change.

⚡ Performance

Not applicable. This PR only fixes extension binding and build wiring; no kernel computation or routing logic is changed.

Reviewer Notes

The SM90 and SM100/SM103 extensions now use the shared bindings in csrc/api/pybind.cu instead of separate legacy PYBIND11_MODULE blocks.

@cherhh
cherhh requested review from icavan and tongke6 July 17, 2026 16:09

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors Pybind11 module definitions by removing them from the individual SM100 and SM90 CUDA source files and consolidating them into a shared csrc/api/pybind.cu file. However, sharing the exact same source file across multiple CUDAExtension modules in setup.py causes a critical build collision and caching issue in setuptools due to differing preprocessor definitions. To prevent linker errors and incorrect bindings, it is recommended to use separate lightweight wrapper files for each extension that include the shared pybind file.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread setup.py Outdated
Comment thread setup.py Outdated
@tongke6

tongke6 commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Hi @cherhh, please test the fat package build command to make sure it doesn't break it.

# Build fat wheel (SM90 + SM100 + SM103):

CULA_BUILD_ALL_ARCHS=1 python -m build --wheel --no-isolation

@cherhh

cherhh commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator Author

Hi @cherhh, please test the fat package build command to make sure it doesn't break it.

# Build fat wheel (SM90 + SM100 + SM103):

CULA_BUILD_ALL_ARCHS=1 python -m build --wheel --no-isolation

Tested with:

CULA_BUILD_ALL_ARCHS=1 python -m build --wheel --no-isolation

Key output:

CULA_BUILD_ALL_ARCHS=1: enabling all SM targets (sm90a, sm100a, sm103a)
Compiling using NVCC 13.0
building 'cula._cudac_sm100' extension
... pybind_sm100.cu -o .../pybind_sm100.o
... -gencode arch=compute_100a,code=sm_100a -DCULA_SM100_ENABLED
... -gencode arch=compute_103a,code=sm_103a -DCULA_SM103_ENABLED
... kda_sm100.o ... pybind_sm100.o ... kda_fwd_sm100.o
... -o .../cula/_cudac_sm100.cpython-312-x86_64-linux-gnu.so
building 'cula._cudac_sm90' extension
... pybind_sm90.cu -o .../pybind_sm90.o
... -gencode arch=compute_90a,code=sm_90a -DCULA_SM90A_ENABLED
... kda_sm90.o ... pybind_sm90.o ... kda_fwd_sm90.o ... kda_fwd_sm90_safe_gate.o
... -o .../cula/_cudac_sm90.cpython-312-x86_64-linux-gnu.so

Successfully built cuda_linear_attention-0.1.1.dev46-cp312-cp312-linux_x86_64.whl

The resulting wheel contains:

- cula/_cudac_sm100.cpython-312-x86_64-linux-gnu.so
- cula/_cudac_sm90.cpython-312-x86_64-linux-gnu.so

Both extension modules import successfully and expose the expected bindings. The separate pybind_sm100.o and pybind_sm90.o objects confirm that the object-file reuse issue is fixed.

@tongke6 tongke6 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@icavanyu icavanyu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@icavan
icavan merged commit 326a307 into inclusionAI:main Jul 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants