Skip to content

cuda.core: introduce copy options for Buffer.copy_{to/from} - #2636

Draft
juenglin wants to merge 1 commit into
NVIDIA:mainfrom
juenglin:copy-with-attributes
Draft

cuda.core: introduce copy options for Buffer.copy_{to/from}#2636
juenglin wants to merge 1 commit into
NVIDIA:mainfrom
juenglin:copy-with-attributes

Conversation

@juenglin

@juenglin juenglin commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds an optional options keyword argument to Buffer.copy_to and Buffer.copy_from, exposing the same CopyOptions dataclass introduced by copy_batch` (#2593) on the per-buffer path. Closes #2365.

Public API

from cuda.core.utils import CopyOptions, MemcpySrcAccessOrder, MemcpyOverlapMode
from cuda.core import Device, Host

# Source access ordering
buf.copy_to(dst, stream=stream, options=CopyOptions(src_access_order=MemcpySrcAccessOrder.ANY))

# Overlap hint
buf.copy_to(dst, stream=stream, options=CopyOptions(overlap_mode=MemcpyOverlapMode.PREFER_OVERLAP_WITH_COMPUTE))

# Location hints (honored for managed memory; silently ignored otherwise)
buf.copy_to(dst, stream=stream, options=CopyOptions(
    src_access_order=MemcpySrcAccessOrder.STREAM,
    src_location_hint=Device(0),
    dst_location_hint=Host(),
))

options=None (the default) selects the existing cuMemcpyAsync path with no change in behaviour.

Behaviour

  • When options is set, cuMemcpyWithAttributesAsync is used only when all three conditions hold: cuda.bindings ≥ 13.2, driver ≥ CUDA 13.2, and the stream is neither in graph-capture mode nor a default-stream token (cuMemcpyWithAttributesAsync rejects those with CUDA_ERROR_INVALID_VALUE`).
  • When any condition is not met, a UserWarning is emitted and the copy falls back to cuMemcpyAsync. Data is always transferred correctly on both paths.
  • options=None never warns and always uses cuMemcpyAsync, so existing code is unaffected.
  • dst=None auto-allocation remains valid with options.

Relation to copy_batch

Uses the same CopyOptions dataclass and _to_cu_memcpy_attributes conversion helper (now in the shared _copy_attributes module extracted in the prerequisite refactor PR). The single-copy path diverges from copy_batch in two ways:

  • Default-stream tokens are accepted (warn+fallback) rather than hard-rejected.
  • Graph-capture streams warn+fallback rather than raising TypeError; the fallback cuMemcpyAsync call is captured as a graph node, so the copy is still usable in graphs.

Implementation notes

The prerequisite refactor PR (the _copy_attributes leaf module) extracted _to_cu_memcpy_attributes and the new _with_attributes_available CUDA 13.2 gate so that both _buffer and _copy_ops can cimport them without circular dependency.

Tests

tests/memory/test_copy_single_options.py covers:

  • Data correctness for copy_to and copy_from with options=None and each of the three MemcpySrcAccessOrder values.
  • overlap_mode=PREFER_OVERLAP_WITH_COMPUTE does not corrupt data.
  • UserWarning + correct fallback under graph capture (both methods).
  • UserWarning + correct fallback with default-stream tokens.
  • Device + host location hints with managed memory (src_location_hint, dst_location_hint).
  • NUMA location hints (Host(numa_id=...), Host.numa_current()), skipped if the system reports no NUMA topology.
  • dst=None auto-allocation with options.

@copy-pr-bot

copy-pr-bot Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the cuda.core Everything related to the cuda.core module label Aug 14, 2026
@juenglin juenglin changed the title Ocuda.core: introduce copy options for Buffer.copy_{to/from} cuda.core: introduce copy options for Buffer.copy_{to/from} Aug 14, 2026
@juenglin

Copy link
Copy Markdown
Contributor Author

/ok to test d835ecf

@juenglin juenglin added the enhancement Any code-related improvements label Aug 14, 2026
@juenglin juenglin added this to the cuda.core 1.2.0 milestone Aug 14, 2026
@juenglin juenglin self-assigned this Aug 14, 2026
@juenglin
juenglin requested review from Andy-Jost and leofang August 14, 2026 22:06
@github-actions

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cuda.core Everything related to the cuda.core module enhancement Any code-related improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEA] cuda.core: support copies with attributes (CUDA 13.2)

1 participant