Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 3 additions & 7 deletions .github/ISSUE_TEMPLATE/release_checklist.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-FileCopyrightText: Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# SPDX-License-Identifier: Apache-2.0

Expand All @@ -10,7 +10,7 @@ body:
- type: markdown
attributes:
value: |
This checklist is for cuda-core releases as well as cuda-bindings patches. Please go through this checklist and ensure all tasks are completed.
This checklist is for cuda-core releases as well as cuda-bindings patches. Please go through this checklist and ensure all tasks are completed. See [RELEASE-core.md](../RELEASE-core.md) for detailed guidance on each step (cuda.core).
- type: checkboxes
id: subpackage-tasks
Expand All @@ -20,17 +20,13 @@ body:
- label: File an internal nvbug to communicate test plan & release schedule with QA
- label: Ensure all pending PRs are reviewed, tested, and merged
- label: Check (or update if needed) the dependency requirements
- label: Create a public rc tag
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed w/ @mdboom internally last week, I think we'll still want this step.

- label: "Point QA to fetch public artifacts (wheels) from the GHA run ID, example: `gh run download 12323257563 -p \"cuda-core*\" -R NVIDIA/cuda-python`"
- label: Wait for QA reports and fix any issues found
- label: "Finalize the doc update, including release notes (\"Note: Touching docstrings/type annotations in code is OK during code freeze, apply your best judgement!\")"
- label: Update the docs for the new version
- label: Create a public release tag
- label: Wait for the tag-triggered CI run to complete, and use that run ID for release workflows
- label: If any code change happens, rebuild the wheels from the new tag
- label: Update the conda recipe & release conda packages
- label: Upload conda packages to nvidia channel
- label: Upload wheels to PyPI
- label: Update the conda recipe & release conda packages
- label: Post-release QA
- label: Finalize the announcement update
- label: Send out the announcement internally
Expand Down
286 changes: 286 additions & 0 deletions .github/RELEASE-core.md
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This document is huge. Are we expecting this to help avoid mistakes? How are we going to make sure this document is kept up to date?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assumed this is mainly for LLMs? It'd be great to call that out near the top.

Copy link
Contributor Author

@Andy-Jost Andy-Jost Feb 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a companion to the release checklist. Here's the 0.6.0 checklist: #1634

Each item has a section in this file. So, for example, the item "Update the docs for the new version" has a detailed explanation in this document.

I see two ways to use it:

  1. For a human, read these notes to get more information about a step (what does updating the docs entail, exactly?).
  2. For prompting, provide context for prompts such as "Help me update the docs. Outline the tasks we need to complete."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This document is huge.

To me, the size is reasonable. It's a cross-reference for items in the release checklist, not a document one would necessarily read straight from start to end.

Are we expecting this to help avoid mistakes?

It captures every issue I ran into while managing the release. Much of the value is in providing definitions for items in the release checklist.

How are we going to make sure this document is kept up to date?

Use it as reference when managing releases -> update sections based on learnings.

Original file line number Diff line number Diff line change
@@ -0,0 +1,286 @@
<!-- SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is specific to cuda-core, which is great, but it's currently not reflected in the filename. How about:

RELEASE-core.md

Ideally, the ISSUE_TEMPLATE would also have core in the name, e.g. release-core-checklist.yml (or with underscores instead of dashes).

Copy link
Contributor Author

@Andy-Jost Andy-Jost Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thought was that we should unify the release documentation. So, for this document we could just add sections. To me it's simpler than keeping separate RELEASE-*.md docs.

I think the bar should be quite low for this change. It puts something in place that we can improve in any way we like.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The three releases we have are vastly different. In order of complexity:

  • very complex: cuda-bindings & cuda-python, about half of the time tied to our private repo, full documentation cannot live here
  • medium complex: cuda-core — this doc seems great, assuming the intent is that LLMs pick up from here
  • super easy: cuda-pathfinder — @cpcloud's new release-cuda-pathfinder.yml automates that almost completely, except for the Conda part, which has been easy in the past

I think if the goal is to enable easy jump-start with LLMs, keeping the RELEASE docs cleanly separated will be most helpful. Mixing in cuda-bindings & cuda-python would be highly distracting (and isn't really possible b/o private repo). For cuda-pathfinder we only need a note about Conda somewhere, everything else is already automatic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I captured this in the latest revision. Renamed to RELEASE-core.md and added scoping to the introduction.

<!-- SPDX-License-Identifier: Apache-2.0 -->

# cuda.core Release Process

This document covers the `cuda.core` release process. For other packages:
`cuda-bindings` and `cuda-python` involve a private repository and are not
documented here; `cuda-pathfinder` is largely automated by the
[release-cuda-pathfinder.yml](workflows/release-cuda-pathfinder.yml)
workflow.

Each section below provides detailed guidance for a step in the
[Release Checklist](ISSUE_TEMPLATE/release_checklist.yml). To start a
release, create a new issue from that template and work through it item by
item, referring back here as needed.

---

## File an internal nvbug

Create an nvbug to request that SWQA begin post-release validation. Issues
identified by that process are typically addressed in a patch release. To find
the template, search for a previous release's nvbug (e.g. by title "Release of
cuda.core") and create a new bug from the same template.

Example:

> **Title:** Release of cuda.core v0.6.0
>
> **Description:**
>
> Requesting SWQA validation for the cuda.core v0.6.0 release. Please test
> the following SW combinations on all listed platforms and report any
> issues found.
>
> **SW Combinations**
> - cuda.core 0.6.0 / cuda.bindings 12.9 / CTK 12.9 / CUDA 12.9 driver
> - cuda.core 0.6.0 / cuda.bindings 13.0 / CTK 13.0 / CUDA 13.0 driver
> - cuda.core 0.6.0 / cuda.bindings 13.1 / CTK 13.1 / CUDA 13.1 driver
Comment on lines +36 to +39
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copying/pasting my offline note on this for posterity:

We want to specify the test matrix as follows (for CTK & driver)

  • latest major, latest minor
  • latest major, earliest minor
  • previous major, latest minor
  • previous major, earliest minor (this was dropped due to resource constraint)

so for 0.6.0 it’d be

  • CTK/driver 13.1 (the most recent public one as of today)
  • CTK/driver 13.0
  • CTK/driver 12.9

cuda.bindings should follow the same major.minor rules (and let patch float)

ideally we’d also like to test the case of mix-n-match, ex: cuda.bindings 12.9, CTK 12.3, driver 12.6, but again we can't, due to resource constraint 😄 so the maximum mix-n-match they’ll allow is

  • cuda.bindings 13.1, CTK/driver 13.1
  • cuda.bindings 13.1, CTK/driver 13.0
  • cuda.bindings 12.9, CTK/driver 12.9

i.e. always use the latest minor (but same major) for cuda.bindings

>
> **Platforms**
> - Linux x86-64
> - Linux arm64
> - Windows x86-64 (TCC and WDDM)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TCC / WDDM / MCDM

> - WSL
>
> **Test Plan**
>
> Functional tests as described in the cuda.core test plan.
>
> **Release Milestones**
> - Pre-release QA (this request)
> - GitHub release tag and posting
> - PyPI wheel upload
> - Post-release validation
Update the version, SW combinations (check with the release owner), and
platforms as appropriate for each release.

---

## Check (or update if needed) the dependency requirements

Review `cuda_core/pyproject.toml` and verify that all dependency
requirements are current.

---

## Finalize the doc update, including release notes

Review every PR included in the release. For each one, check whether new
functions, classes, or features were added and whether they have complete
docstrings. Add or edit docstrings as needed — touching docstrings and
type annotations in code is OK during code freeze.

Write the release notes in `cuda_core/docs/source/release/`. Look at
historical release notes for guidance on format and structure. Balance all
entries for length, specificity, tone, and consistency. Highlight a few
notable items in the highlights section, keeping their full entries in the
appropriate sections below.

---

## Update the docs for the new version

Add the new version to
`cuda_core/docs/nv-versions.json`. This file drives the version
switcher on the documentation site. Add an entry for the new version
after `"latest"`, following the existing pattern. The docs themselves are
built and deployed automatically by the release workflow.

---

## Create a public release tag

**Warning:** Pushing a tag is a potentially irrevocable action. Be absolutely
certain the tag points to the correct commit before pushing.

Tags should be GPG-signed. The tag name format is `cuda-core-v<VERSION>`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we've ever signed with GPG but it's not a bad idea

(e.g. `cuda-core-v0.6.0`). The tag must point to a commit on `main`.

```bash
git checkout main
git pull origin main
git tag -s cuda-core-v0.6.0 -m "cuda-core v0.6.0"
git push origin cuda-core-v0.6.0
```
Comment on lines +102 to +107
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't be needed once #1610 lands


---

## Wait for the tag-triggered CI run to complete

Pushing the tag triggers a CI run automatically. Monitor it in the
**Actions** tab on GitHub.

- **All CI tests should succeed.** If any fail, investigate and rerun as
needed.
- Note the **run ID** of the successful tag-triggered run. The release
workflow can auto-detect it from the tag, but you can also provide it
explicitly.

---

## Upload wheels to PyPI

This is a two-stage process: first publish to TestPyPI, verify, then
publish to PyPI.

### Stage 1: TestPyPI

1. Go to **Actions > CI: Release** and run the workflow with:
- **Component**: `cuda-core`
- **The release git tag**: `cuda-core-v0.6.0`
- **The GHA run ID that generated validated artifacts**: This is the
run ID of the successful tag-triggered CI run from the previous step.
You can find it in the URL when viewing the run in the Actions tab
(e.g. `https://github.com/NVIDIA/cuda-python/actions/runs/123456789`
— the run ID is `123456789`).
Comment on lines +134 to +138
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step is optional

- **build-ctk-ver**: the `cuda.build.version` from
[`ci/versions.yml`](../ci/versions.yml) (e.g. `13.1.1`)
- **Which wheel index to publish to**: `testpypi`

2. Wait for the workflow to complete.

3. Verify the TestPyPI upload by installing and running tests from a
checked-out copy of the repository:

```bash
pip install -i https://test.pypi.org/simple/ \
--extra-index-url https://pypi.org/simple/ \
cuda-core==0.6.0
cd cuda_core/tests && pytest
```

### Stage 2: PyPI

Once TestPyPI verification passes, rerun the same workflow with:
- **Which wheel index to publish to**: `pypi`

After completion, verify:

```bash
pip install cuda-core==0.6.0
```

---

## Update the conda recipe & release conda packages

The conda-forge feedstock builds from the GitHub Release source archive
(not from PyPI). There are three approaches to updating the feedstock,
from least effort to most control.

### Approach A: Wait for the bot

The `regro-cf-autotick-bot` periodically scans for new releases and opens
a PR automatically. If nothing has changed in the build requirements, the
bot's PR may be sufficient — review it and ask a feedstock maintainer
to merge. However, the bot only
updates the version and sha256. If build dependencies, import paths, or
other recipe fields have changed, the bot's PR will be incomplete and CI
will fail.

### Approach B: Request a bot update

If the bot hasn't opened a PR, you can request one explicitly. Go to the
feedstock's Issues tab and create a new "Bot commands" issue:

- **Title**: `@conda-forge-admin, please update version`
- **Body**: (leave empty)

This triggers the bot to create a version-bump PR. As with approach A,
review the PR and push additional fixes if needed.

### Approach C: Manual PR

For full control — or when the bot's PR needs extensive fixes — open a
PR manually from a fork.

**Fork and clone** (one-time setup):

```bash
gh repo fork conda-forge/cuda-core-feedstock --clone
cd cuda-core-feedstock
```
Comment on lines +200 to +205
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For feedstock maintainers this is even simpler: Launch an empty/draft PR via @conda-forge-admin, please rerender, and the bot will create a PR (from its fork). Maintainers can edit any PR sent to the feedstock.


**Create a branch and edit `recipe/meta.yaml`:**

```bash
git checkout -b update-v0.6.0 origin/main
```

Update the following fields:

1. **`version`**: Set to the new version (e.g. `0.6.0`).
2. **`number`** (build number): Reset to `0` for a new version.
3. **`sha256`**: The SHA-256 of the source archive from the GitHub
Release. Download it and compute the hash:

```bash
curl -sL https://github.com/NVIDIA/cuda-python/releases/download/cuda-core-v0.6.0/cuda-python-cuda-core-v0.6.0.tar.gz \
| sha256sum
Comment on lines +217 to +222
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW we also attach the checksum as one of the GitHub release artifacts. With recent GitHub it is also shown in the web UI so that a simple copy/paste is sufficient without curl.

```

4. **Host dependencies**: Ensure all build-time dependencies are listed.
For example, v0.6.0 added a Cython C++ dependency on `nvrtc.h`,
requiring `cuda-nvrtc-dev` in both `host` requirements and
`ignore_run_exports_from`.

5. **Test commands and descriptions**: Update any import paths or
descriptions that changed (e.g. `cuda.core.experimental` ->
`cuda.core`).

**Open a PR:**

```bash
git add recipe/meta.yaml
git commit -m "Update cuda-core to 0.6.0"
git push <your-github-username> update-v0.6.0

gh pr create \
--repo conda-forge/cuda-core-feedstock \
--head <your-github-username>:update-v0.6.0 \
--title "Update cuda-core to 0.6.0" \
--body "Update cuda-core to version 0.6.0."
```

### Notes

The feedstock CI (Azure Pipelines) triggers automatically on the PR.
Monitor it for build failures — common issues include missing build-time
header dependencies. Feedstock maintainers (listed in
`recipe/meta.yaml` under `extra.recipe-maintainers`) can merge the PR.

---

## Post-release QA

*TBD*

---

## Finalize the announcement update

The release workflow creates a draft GitHub Release. To publish it:

1. Go to the repository on GitHub, click **Tags**, then switch to the
**Releases** tab.
2. Find the draft release for the new tag and click **Edit**.
3. Copy the body from a previous release as a starting point. It
typically links to the release notes in the documentation (e.g.
`https://nvidia.github.io/cuda-python/cuda-core/latest/release/0.6.0-notes.html`).
4. Update the link and any version references, then click
**Publish release**.

---

## Send out the announcement internally

The release owner will prepare and send the announcement.

---

## Send out the announcement externally (GitHub Release -> Announcement)

*TBD*
Comment on lines +284 to +286
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer needed

Loading