Skip to content

Add conda build option and CUDA target includes#6316

Open
JanuszL wants to merge 1 commit into
NVIDIA:mainfrom
JanuszL:incorporate_conda_changes
Open

Add conda build option and CUDA target includes#6316
JanuszL wants to merge 1 commit into
NVIDIA:mainfrom
JanuszL:incorporate_conda_changes

Conversation

@JanuszL
Copy link
Copy Markdown
Contributor

@JanuszL JanuszL commented Apr 27, 2026

Adds a conda packaging build option and switches CUDA-header consumers to
the CMake CUDAToolkit target directory.

It absorbs https://github.com/conda-forge/nvidia-dali-python-feedstock/tree/main/recipe/patches.

Category:

Other (Build / Configuration)

Description:

This change adds BUILD_FOR_CONDA, enables it from the conda native-libs and
Python-bindings recipes, and keeps the default source and wheel build behavior
unchanged when the option is disabled.

When BUILD_FOR_CONDA=ON, the build:

  • uses conda's lib layout for nvImageCodec runtime lookup and RPATH;
  • prefers the shared libtar library instead of forcing libtar.a;
  • emits conda install guidance for dynamically loaded CUDA toolkit libraries;
  • keeps conda-provided CUDA architecture settings intact.

The change also requires CMake's CUDAToolkit package and uses
CUDAToolkit_TARGET_DIR/include directly for CUDA-family generated stubs and
include directories. This covers the CUDA, cuFile, cuFFT, NPP, nvImageCodec,
NVML, and nvCUVID stub generation paths without depending on the trimmed
CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES value.

For the split conda package build, bundled FFTS and cocoapi static targets are
not rebuilt when PREBUILD_DALI_LIBS=ON.

Additional information:

Affected modules and functionalities:

  • CMake configuration: CMakeLists.txt
  • Dependency lookup: cmake/Dependencies.common.cmake
  • Conda recipes:
    • conda/dali_native_libs/recipe/build.sh
    • conda/dali_python_bindings/recipe/build.sh
  • CUDA-header stub generation and include paths:
    • dali/core/CMakeLists.txt
    • dali/kernels/signal/fft/CMakeLists.txt
    • dali/npp/CMakeLists.txt
    • dali/nvimgcodec/CMakeLists.txt
    • dali/operators/video/dynlink_nvcuvid/CMakeLists.txt
    • dali/util/CMakeLists.txt
  • Dynamic loader diagnostics:
    • dali/core/dynlink_cufile.cc
    • dali/core/dynlink_nvcomp.cc
    • dali/kernels/signal/fft/cufft_wrap.cc
    • dali/npp/npp_wrap.cc
    • dali/nvimgcodec/nvimgcodec_wrap.cc
    • dali/operators/operators.cc
  • Minor typo fix in dali/util/nvml_wrap.cc

Key points relevant for the review:

  • BUILD_FOR_CONDA defaults to OFF; default builds should retain existing
    behavior.
  • The option is propagated through propagate_option(BUILD_FOR_CONDA), which
    produces FOR_CONDA_ENABLED for the C++ loader diagnostics.
  • find_package(CUDAToolkit REQUIRED) is now used so the build can consistently
    reference CUDAToolkit_TARGET_DIR/include.
  • The obsolete nvJPEG2K conda-forge patch was intentionally not carried over.

Tests:

  • Existing tests apply
  • New tests added
    • Python tests
    • GTests
    • Benchmark
    • Other
  • N/A

Previously configured successfully with:

  • default reduced CMake configuration
  • BUILD_FOR_CONDA=ON
  • BUILD_FOR_CONDA=ON with explicit CUDAToolkit_TARGET_DIR

For this metadata refresh, git diff --check upstream/main...HEAD passes.

Checklist

Documentation

  • Existing documentation applies
  • Documentation updated
    • Docstring
    • Doxygen
    • RST
    • Jupyter
    • Other
  • N/A

DALI team only

Requirements

  • Implements new requirements
  • Affects existing requirements
  • N/A

REQ IDs: N/A

JIRA TASK: DALI-4681

@JanuszL JanuszL marked this pull request as draft April 27, 2026 12:13
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Apr 27, 2026

Greptile Summary

This PR adds a BUILD_FOR_CONDA CMake option (defaulting to OFF) to support conda-forge packaging while keeping all default wheel/source builds unchanged. When enabled, it switches nvimgcodec to lib instead of lib64, prefers shared libtar, honours an explicit CUDAToolkit_TARGET_DIR include path, and emits conda-specific install guidance from the dynamic-loader diagnostics.

  • All stub generators are migrated from the ad-hoc CUDA_TOOLKIT_INCLUDE_MAJOR_DIRECTORY / CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES_DIRECTIVE pattern to ${CUDAToolkit_TARGET_DIR}/include, made possible by adding find_package(CUDAToolkit REQUIRED) unconditionally.
  • The FOR_CONDA_ENABLED preprocessor define (produced by propagate_option(BUILD_FOR_CONDA)) gates the new error messages in six .cc files; the non-conda paths are fully preserved.
  • dali/util/nvml_wrap.cc fixes a pre-existing "dirver" typo.

Confidence Score: 5/5

The change is additive and off-by-default; default builds are unaffected except for the transparent adoption of CUDAToolkit_TARGET_DIR for stub generation.

All new code paths are gated behind BUILD_FOR_CONDA=OFF. The unconditional migration of stub generators to CUDAToolkit_TARGET_DIR is backward-compatible on standard CUDA installations, and find_package(CUDAToolkit REQUIRED) succeeds in every environment where project(... CUDA ...) already compiles. The two findings are unused-variable cleanup nits with no functional impact.

No files require special attention.

Important Files Changed

Filename Overview
CMakeLists.txt Adds find_package(CUDAToolkit REQUIRED), BUILD_FOR_CONDA option, and conda-specific RPATH handling; removes the CUDA_TOOLKIT_INCLUDE_MAJOR_DIRECTORY if/else but leaves its SEMICOLON_POS guard as dead code on line 49.
cmake/Dependencies.common.cmake Guards libtar static-preference and FFTS/CocoAPI builds behind BUILD_FOR_CONDA and PREBUILD_DALI_LIBS checks; straightforward and correct.
dali/core/CMakeLists.txt cuda.h and cufile.h stub generators updated to CUDAToolkit_TARGET_DIR; nvcomp stub (line 108) still uses the old CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES_DIRECTIVE, so that computation block remains necessary.
dali/util/CMakeLists.txt nvml.h stub updated to CUDAToolkit_TARGET_DIR; the SEMICOLON_POS / CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES_DIRECTIVE computation block (lines 84-93) is now dead code.
dali/nvimgcodec/nvimgcodec_wrap.cc Correctly toggles lib/lib64 path via FOR_CONDA_ENABLED macro and provides conda install guidance on dlopen failure.
dali/operators/operators.cc Adds conda-specific version-requirement error message with single-quoted conda spec; non-conda branch restructured with inline CUDA_VERSION computation.

Fix All in Claude Code

Reviews (3): Last reviewed commit: "Add conda build option and CUDA target i..." | Re-trigger Greptile

Comment thread dali/operators/operators.cc Outdated
@JanuszL JanuszL force-pushed the incorporate_conda_changes branch from 5666d1a to c985677 Compare April 27, 2026 12:20
@JanuszL
Copy link
Copy Markdown
Contributor Author

JanuszL commented Apr 27, 2026

@greptile review

@JanuszL JanuszL force-pushed the incorporate_conda_changes branch from c985677 to 71390d0 Compare April 27, 2026 16:53
@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [49621159]: BUILD STARTED

@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [49621159]: BUILD FAILED

@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [49621159]: BUILD PASSED

@JanuszL JanuszL force-pushed the incorporate_conda_changes branch from 71390d0 to 832f595 Compare April 27, 2026 21:21
@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [49643983]: BUILD STARTED

@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [49643983]: BUILD FAILED

@JanuszL JanuszL force-pushed the incorporate_conda_changes branch from 832f595 to 2b83881 Compare April 28, 2026 06:41
@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [49679151]: BUILD STARTED

@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [49679151]: BUILD FAILED

@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [49679151]: BUILD PASSED

@JanuszL JanuszL force-pushed the incorporate_conda_changes branch from 2b83881 to 8a6a7ef Compare May 29, 2026 18:03
@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [53088284]: BUILD STARTED

@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [53088284]: BUILD FAILED

@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [53103732]: BUILD STARTED

@dali-automaton
Copy link
Copy Markdown
Collaborator

CI MESSAGE: [53103732]: BUILD PASSED

@JanuszL JanuszL marked this pull request as ready for review June 1, 2026 11:19
Add BUILD_FOR_CONDA and pass it from the conda recipes so conda-forge
packaging can opt into its own dependency layout without changing default
builds.

When enabled, prefer shared libtar, use conda's nvImageCodec lib directory
in runtime lookup and RPATH, and emit conda install guidance for dynamically
loaded CUDA toolkit libraries.

Require CUDAToolkit package discovery and use CUDAToolkit_TARGET_DIR/include
directly for generated CUDA-family stubs and CUDA include directories. This
avoids relying on the trimmed CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES value
while still honoring explicit CUDA target directories.

Also skip bundled FFTS and cocoapi static targets when prebuilt DALI libs are
used, and fix a typo in the NVML loader diagnostic.

Signed-off-by: Janusz Lisiecki <jlisiecki@nvidia.com>
@JanuszL JanuszL force-pushed the incorporate_conda_changes branch from 8a6a7ef to e2a4ea1 Compare June 1, 2026 11:24
@JanuszL JanuszL changed the title Add conda-forge build option Add conda build option and CUDA target includes Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants