Move from custom DALI conda recipe to upstream nvidia-dali-feedstock

[JanuszL]

• uses nvidia-dali-feedstock recipe to build DALI in CI
• removes DALI maintained recipe

Category:

Other (e.g. Documentation, Tests, Configuration)

Description:

• uses nvidia-dali-feedstock recipe to build DALI in CI
• removes DALI maintained recipe

Additional information:

Affected modules and functionalities:

• conda build process

Key points relevant for the review:

• NA

Tests:

• Existing tests apply
  • all test *_conda
• New tests added
  • Python tests
  • GTests
  • Benchmark
  • Other
• N/A

Checklist

Documentation

• Existing documentation applies
• Documentation updated
  • Docstring
  • Doxygen
  • RST
  • Jupyter
  • Other
• N/A

DALI team only

Requirements

• Implements new requirements
• Affects existing requirements
• N/A

REQ IDs: N/A

JIRA TASK: DALI-4680

[dali-automaton]

CI MESSAGE: [55681719]: BUILD STARTED

[greptile-apps]

Greptile Summary

This PR removes the DALI-maintained conda recipe (two packages, config YAML, and activate/deactivate hooks) and replaces it with a thin orchestration script that clones the upstream conda-forge/nvidia-dali-python-feedstock, optionally pins it to a caller-supplied SHA, applies a single local patch, and drives rattler-build instead of conda mambabuild.

• build_conda_packages.sh is rewritten end-to-end: strict input validation for CUDA_VERSION_MAJOR, DALI_VERSION, DALI_TIMESTAMP, and NVIDIA_BUILD_ID; sed patches with follow-up grep guards to verify each recipe edit; and rattler-build invocation with an artifact copy step.
• nvidia-dali-python-feedstock.patch carries all DALI-specific recipe deltas against the upstream feedstock: rm -rf guards before symlink creation, BUILD_FOR_CONDA/flavor/timestamp CMake flags, re-enabled tests and benchmarks, and a documented nvCOMP 5.2 minimum.
• All previously maintained recipe files (meta.yaml, build.sh, activate/deactivate hooks) are deleted.

Confidence Score: 5/5

Safe to merge. The migration is well-structured with input validation, patch pre-checks, and sed verification guards throughout. The only weakness is the else-clause fall-through for the CUDA major version selector, which is benign for currently supported CUDA 12 and 13 but would silently produce a misconfigured build for any other major version.

The change is a build-system migration — no runtime logic, no data paths affected. The new script is more defensively written than the one it replaces: explicit env-var guards, git apply --check before applying, and grep verifications on each sed substitution. The CUDA_VERSION_MAJOR else-clause maps every non-13 value to the CUDA 12.9 YAML without an explicit unknown-version guard, but all currently supported CUDA versions (12 and 13) are handled correctly.

The CUDA_VERSION_MAJOR branch in conda/build_conda_packages.sh (lines 12–16) deserves a second look: consider making the else branch explicit for 12 and adding a hard-failure path for unrecognised major versions.

Important Files Changed

Filename	Overview
conda/build_conda_packages.sh	Core orchestration script fully rewritten: clones the upstream feedstock, optionally pins to a SHA, applies a local patch, and drives rattler-build. Input validation and sed-patch verification are solid; the CUDA_VERSION_MAJOR else-branch silently falls through to the CUDA 12.9 YAML for unrecognised major versions.
conda/nvidia-dali-python-feedstock.patch	Adds rm-rf guards before symlink creation to prevent 'target exists' failures, injects BUILD_FOR_CONDA / flavor / timestamp CMake flags, re-enables tests and benchmarks, documents the nvCOMP 5.2 minimum, and wires DALI_VERSION / NVIDIA_BUILD_ID into the recipe context. Tightly coupled to specific line numbers in the upstream build.sh, so a feedstock refactor may cause git apply --check to catch and fail cleanly.
conda/config/conda_build_config.yaml	Deleted; equivalent config now lives in the upstream feedstock's conda_build_config.yaml, extended via the patch file.
conda/dali_native_libs/recipe/meta.yaml	Deleted as part of migrating to the upstream feedstock recipe.
conda/dali_python_bindings/recipe/meta.yaml	Deleted as part of migrating to the upstream feedstock recipe.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[build_conda_packages.sh] --> B{Validate env vars}
    B -->|invalid| FAIL1[exit 1 with message]
    B -->|valid| C{CUDA_VERSION_MAJOR}
    C -->|== 13| D[METAL_YAML: cuda13.0]
    C -->|else| E[METAL_YAML: cuda12.9]
    D --> F[git clone nvidia-dali-python-feedstock]
    E --> F
    F --> G{DALI_FEEDSTOCK_SHA set?}
    G -->|yes| H[git checkout SHA]
    G -->|no| I[use HEAD]
    H --> J[git apply --check patch]
    I --> J
    J -->|fail| FAIL2[exit 1]
    J -->|ok| K[git apply patch]
    K --> L[sed: replace tarball URL with path: /opt/dali]
    L --> M{grep verify}
    M -->|fail| FAIL3[exit 1]
    M -->|ok| N[sed: add use_gitignore: false]
    N --> O{grep verify}
    O -->|fail| FAIL4[exit 1]
    O -->|ok| P[rattler-build build with METAL_YAML]
    P --> Q[cp *.conda to artifacts/]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[build_conda_packages.sh] --> B{Validate env vars}
    B -->|invalid| FAIL1[exit 1 with message]
    B -->|valid| C{CUDA_VERSION_MAJOR}
    C -->|== 13| D[METAL_YAML: cuda13.0]
    C -->|else| E[METAL_YAML: cuda12.9]
    D --> F[git clone nvidia-dali-python-feedstock]
    E --> F
    F --> G{DALI_FEEDSTOCK_SHA set?}
    G -->|yes| H[git checkout SHA]
    G -->|no| I[use HEAD]
    H --> J[git apply --check patch]
    I --> J
    J -->|fail| FAIL2[exit 1]
    J -->|ok| K[git apply patch]
    K --> L[sed: replace tarball URL with path: /opt/dali]
    L --> M{grep verify}
    M -->|fail| FAIL3[exit 1]
    M -->|ok| N[sed: add use_gitignore: false]
    N --> O{grep verify}
    O -->|fail| FAIL4[exit 1]
    O -->|ok| P[rattler-build build with METAL_YAML]
    P --> Q[cp *.conda to artifacts/]

Loading

_{Reviews (12): Last reviewed commit: "Rebase feedstock patch on current recipe" | Re-trigger Greptile}

[dali-automaton]

CI MESSAGE: [55681719]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [55685089]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [55685089]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [55688559]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [55688559]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [55693916]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [55694466]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [55696184]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [57832399]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [57832399]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [57957810]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [57957810]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [57967556]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [57967556]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [58002631]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [58002631]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [58097163]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [58097163]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [58105433]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [58105433]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [58114829]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [58114829]: BUILD PASSED