Skip to content

Conversation

@jameslamb
Copy link
Member

@jameslamb jameslamb commented Jan 27, 2026

Description

Proposes a batch of miscellaneous build / packaging / CI changes.

Changes

Tightens wheel size limits

Contributes to rapidsai/build-planning#219

CUDA 13 wheels can be as much as 50% smaller than the equivalent CUDA 12 wheels, because of some new compression features in nvcc.

To ensure surprising package-size growth is caught in CI, this PR tightens the limits in the following ways:

  • setting different limits based on CUDA major version where package size differs significantly
  • setting libcuopt limits to {compressed_size} + 50Mi, rounded to the nearest 5Mi
  • setting all other limits to {compressed_size} + 10Mi, rounded to the nearest 5Mi

Expands CI-skipping logic

Contributes to rapidsai/build-planning#243

Tries to avoid unnecessary CI runs by making the CI-skipping rules finer-grained. For example, PRs that only touch .pre-commit-config.yaml should now not require any runners with GPUs 😁

Removes some reliance on pypi.nvidia.com

Contributes to rapidsai/build-planning#241

git grep -i -E 'pypi\.nvidia\.com'
git grep -i -E 'rapids\-init\-pip'

And removed/updated all relevant references. This project does not need any wheels from pypi.nvidia.com at build-time or runtime, it can safely avoid searching that index.

Its own packages are not available on pypi.org though, so uses of pypi.nvidia.com in docs are preserved here.

Enforces PEP 639 license metadata in pyproject.toml

Contributes to rapidsai/pre-commit-hooks#95

Checklist

  • I am familiar with the Contributing Guidelines.
  • Testing
    • New or existing tests cover these changes
  • Documentation
    • The documentation is up to date with these changes

Summary by CodeRabbit

  • New Features

    • Added a public CLI entrypoint (cuopt_cli).
  • Chores

    • More granular CI gating so jobs run based on changed files.
    • Updated pre-commit hooks to rev 1.3.3 and added license verification.
    • Removed NVIDIA PyPI extra-index usage and introduced an opt-out env flag for CI pip init.
    • Tightened and standardized compressed package size limits and improved packaging validation.

✏️ Tip: You can customize this high-level summary in your review settings.

@copy-pr-bot

This comment was marked as resolved.

@jameslamb jameslamb changed the title WIP: [NOT READY FOR REVIEW] try removing pypi.nvidia.com WIP: tighten wheel size limits, expand CI-skipping logic, other small build changes Jan 29, 2026
needs: [conda-python-build, changed-files, compute-matrix-filters]
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@main
#if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python_conda
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These skips have all been disabled for the last year: https://github.com/rapidsai/cuopt/pull/2244

From that PR, I'm really not sure why or what the issue was. But either way, that's a year of running more CI than necessary, which is a waste of money and time. Hopefully we can re-enable this now... it'd help get changes integrated faster and reduce cuopt's demands on our shared CI runners.

echo "installing 'cvxpy' with cuopt"
python -m pip install \
--constraint "${PIP_CONSTRAINT}" \
--extra-index-url=https://pypi.nvidia.com \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't ever be necessary.

The only packages installed in this command that aren't on pypi.org are cuOpt packages themselves, which should all have been downloaded from the same CI run before this runs.

@jameslamb
Copy link
Member Author

/ok to test

with:
files_yaml: |
test_cpp:
build_docs:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs builds use GPU runners. Those runners are scarce, so it's worth skipping those jobs when they're not necessary (like when a PR only changes the Helm chart).

- '!ci/test_wheel*.sh'
- '!container-builder/**'
- '!helm-chart/**'
- '!helmchart/**'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This directory is called helmchart/ (no -). This was wrong before.

Comment on lines -128 to -130
- '!omniverse/**'
- '!regression/**'
- '!resources/**'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These directories no longer exist in the repo.

- '!python/cuopt_self_hosted/**'
- '!python/cuopt_server/**'
- '!python/nvcf_client/**'
test_python_cuopt_server:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm hoping the test_python_wheels: group is granular enough that we don't need test_python_cuopt_server separately.

But please do break it down into more-specific groups if you feel that's doable! The more unnecessary CI runs we can save, the better 😁

libcuopt = "libcuopt"

[project.scripts]
cuopt_cli = "libcuopt._cli_wrapper:main"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needed to be moved up to work around this: rapidsai/pre-commit-hooks#105

packages:
# pip recognizes the index as a global option for the requirements.txt file
- --extra-index-url=https://pypi.nvidia.com
- --extra-index-url=https://pypi.anaconda.org/rapidsai-wheels-nightly/simple
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

every package in this group is available from pypi.org

@jameslamb jameslamb changed the title WIP: tighten wheel size limits, expand CI-skipping logic, other small build changes tighten wheel size limits, expand CI-skipping logic, other small build changes Jan 30, 2026
@jameslamb jameslamb marked this pull request as ready for review January 30, 2026 03:53
@jameslamb jameslamb requested a review from a team as a code owner January 30, 2026 03:53
@coderabbitai
Copy link

coderabbitai bot commented Jan 30, 2026

📝 Walkthrough

Walkthrough

Updates GitHub Actions changed-file gating and exclusions; adds/export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX and removes NVIDIA extra-index usage in CI scripts; tightens pyproject pydistcheck size limits; adds CUDA-aware size logic to ci/validate_wheel.sh; bumps pre-commit hooks and adds a verify-pyproject-license hook.

Changes

Cohort / File(s) Summary
Workflows
​.github/workflows/pr.yaml
Renames/repurposes changed-file gating groups (e.g., test_cppbuild_docs), enables many previously-commented gating if clauses, updates job dependencies/sequencing, and adjusts/relocates exclusion patterns for CI/docs paths.
Pre-commit config
.pre-commit-config.yaml
Bumps rapidsai/pre-commit-hooks rev v1.2.1v1.3.3 and adds verify-pyproject-license hook with an exclude regex for top-level pyproject.toml.
CI scripts — init/pip index and env var
ci/build_wheel.sh, ci/build_wheel_cuopt*.sh, ci/build_wheel_libcuopt.sh, ci/test_self_hosted_service.sh, ci/test_wheel_*.sh, ci/test_wheel_cuopt_server.sh, ci/thirdparty-testing/run_cvxpy_tests.sh, ci/test_self_hosted_service.sh
Adds/export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX="true" before sourcing rapids-init-pip; removes --extra-index-url=https://pypi.nvidia.com from pip invocations; minor header copyright year updates.
Validate & dependency config
ci/validate_wheel.sh, dependencies.yaml
ci/validate_wheel.sh: adds RAPIDS_CUDA_MAJOR, PYDISTCHECK_ARGS array, conditional --max-allowed-size-compressed based on package_dir/CUDA major, and explicit error on unknown package_dir. dependencies.yaml: removes multiple NVIDIA --extra-index-url entries.
Python packaging — pydistcheck size limits
python/cuopt/pyproject.toml, python/cuopt_self_hosted/pyproject.toml, python/cuopt_server/pyproject.toml, python/cuopt/cuopt/linear_programming/pyproject.toml
Tightens or changes [tool.pydistcheck].max_allowed_size_compressed values (commonly reduced to 15Mi or to smaller Mi thresholds) and updates comments.
libcuopt packaging & CLI
python/libcuopt/pyproject.toml
Removes max_allowed_size_compressed from [tool.pydistcheck] and (re)adds [project.scripts] entry cuopt_cli = "libcuopt._cli_wrapper:main"; adds comment that size is handled via CLI args.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main changes: tightening wheel size limits, expanding CI-skipping logic, and other miscellaneous build modifications, all of which are well-represented in the changeset.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Fix all issues with AI agents
In @.github/workflows/pr.yaml:
- Around line 218-254: The test_python_wheels group currently excludes the
python/** path pattern (the '!python/**' entry in the test_python_wheels file
list), which prevents wheel tests from running on Python source changes; remove
the '!python/**' exclusion from the test_python_wheels list so python/** changes
will trigger the wheel job (make the test_python_wheels include python/** the
same way test_python_conda does).

In `@ci/validate_wheel.sh`:
- Around line 11-12: Add an explicit guard before computing RAPIDS_CUDA_MAJOR to
fail fast with a clear message when RAPIDS_CUDA_VERSION is unset or empty: check
if the variable RAPIDS_CUDA_VERSION is set/non-empty (e.g., using parameter
expansion or an if [ -z "${RAPIDS_CUDA_VERSION:-}" ] test), and if it is missing
echo a descriptive error and exit 1; then proceed to set
RAPIDS_CUDA_MAJOR="${RAPIDS_CUDA_VERSION%%.*}".

@rgsl888prabhu
Copy link
Collaborator

@jameslamb Thank you for this awesome PR, now turnaround time for PR runs would be less for so many of those.

@rgsl888prabhu
Copy link
Collaborator

@jameslamb I run few tests that are there in docs https://github.com/NVIDIA/cuopt/tree/main/docs/cuopt/source/cuopt-python/lp-qp-milp/examples in

./ci/test_doc_examples.sh

May be we need to add paths to those example files so they will trigger tests, else we might need to move that test to different workflow.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In @.github/workflows/pr.yaml:
- Line 125: Update the four exclusion patterns that currently use the misspelled
filename string '.coderabbit_review_guid.md' in the GitHub Actions workflow to
the correct '.coderabbit_review_guide.md'; locate occurrences of the literal
'.coderabbit_review_guid.md' in the workflow (seen in the exclusion list
entries) and replace each with '.coderabbit_review_guide.md' so the exclusions
match the .coderabbit.yaml reference.

@jameslamb
Copy link
Member Author

I run few tests that are there in docs https://github.com/NVIDIA/cuopt/tree/main/docs/cuopt/source/cuopt-python/lp-qp-milp/examples in

Ah thanks, hadn't noticed that!

For now, I just pushed 0726dec removing docs/** entirely from the test_python_wheels group.

That's probably too broad, but it's still an improvement over the current state on main. You could update that however you want based on your knowledge of those tests... it's ok if you want to push more commits here, or do it in a follow-up PR.

@rgsl888prabhu
Copy link
Collaborator

Thank you @jameslamb I will work on a follow-up PR to combine them under one tree so it easier to maintain

@jameslamb
Copy link
Member Author

It seems that some recent changes (maybe #805) added 10s of MB to the package sizes for libcuopt and cuopt wheels... I've had to adjust the thresholds a few times since the one I set when I first opened this PR, which were based on a recent nightly build.

Not a complaint, just letting you know @rgsl888prabhu

Merging this PR will make such binary-size growth more obvious in the future so it's not a surprise.

@rgsl888prabhu
Copy link
Collaborator

It seems that some recent changes (maybe #805) added 10s of MB to the package sizes for libcuopt and cuopt wheels... I've had to adjust the thresholds a few times since the one I set when I first opened this PR, which were based on a recent nightly build.

Not a complaint, just letting you know @rgsl888prabhu

Merging this PR will make such binary-size growth more obvious in the future so it's not a surprise.

Thank you @jameslamb, this would definitely help. There might be additional growth in size due the grpc issue I was talking to you. In case of conda I think I have to build it as static and in case of wheel, I might need to bundle the .so with wheel.

@jameslamb
Copy link
Member Author

Oh yeah that's fine! The limit being enforced in CI doesn't mean you can never change it, it's just nice because then you have a line in the diff and reviewers can have a conversation about whether everyone is OK with the size growing.

@rgsl888prabhu
Copy link
Collaborator

/merge

@rapids-bot rapids-bot bot merged commit 894b035 into NVIDIA:main Jan 31, 2026
90 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improves an existing functionality non-breaking Introduces a non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants