From 09a922cf3cd253dac5beab658b5b6e4cfc02d333 Mon Sep 17 00:00:00 2001 From: yuxihong Date: Sun, 2 Aug 2026 11:11:55 -0400 Subject: [PATCH] fix: pin CUDA_PATH to the CUDA 12.8 runtime wheel in CuPy CI The self-hosted CuPy job fails at test collection with: cupy.cuda.compiler.CompileException: .../libcudacxx/cuda/std/ __floating_point/storage.h(137): error: incomplete type "__nv_fp8_e8m0" is not allowed NVRTC comes from the nvidia-cuda-nvrtc-cu12==12.8.93 wheel, so CuPy 14's bundled CCCL enables _CCCL_HAS_NVFP8_E8M0() (gated on CTK >= 12.8) and forward-declares __nv_fp8_e8m0. But CuPy also passes -I$(get_cuda_path())/include, and get_cuda_path() reads $CUDA_PATH first. When the runner inherits CUDA_PATH=/usr/local/cuda from its shell, that directory is searched before the wheel include dir, so resolves to the system CUDA 12.2 header which has no __nv_fp8_e8m0. Every CCCL-using kernel then fails to compile; test_poststack.py just happens to trigger the first one at import time via filtfilt/convolve1d. The step already rebuilds CUDA_HOME, LD_LIBRARY_PATH and NUMBA_CUDA_NVVM from the uv wheels -- CUDA_PATH was the one that got missed. Pointing it at the 12.8 cuda_runtime wheel makes the header and NVRTC versions agree. Verified on the self-hosted runner: 2183 passed, 910 skipped in 6m18s. Co-Authored-By: Claude Opus 5 --- .github/workflows/buildcupy.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/buildcupy.yaml b/.github/workflows/buildcupy.yaml index 5ebb952d..c5b65efe 100644 --- a/.github/workflows/buildcupy.yaml +++ b/.github/workflows/buildcupy.yaml @@ -64,6 +64,7 @@ jobs: ln -sf "$CUDA_HOME/lib64/libnvrtc.so.12" "$CUDA_HOME/lib64/libnvrtc.so" echo "CUDA_HOME=$CUDA_HOME" >> "$GITHUB_ENV" + echo "CUDA_PATH=$CUDA_RUNTIME_ROOT" >> "$GITHUB_ENV" echo "CUDA_NVVM_LIB=$CUDA_HOME/nvvm/lib64" >> "$GITHUB_ENV" echo "done!" - name: Test with pytest