Skip to content

Add __array__ and DLPack protocols to OrtValue#27980

Merged
tianleiwu merged 3 commits intomicrosoft:mainfrom
Rishi-Dave:rishidave/feat/ortvalue-array-dlpack-protocols-v2
Apr 7, 2026
Merged

Add __array__ and DLPack protocols to OrtValue#27980
tianleiwu merged 3 commits intomicrosoft:mainfrom
Rishi-Dave:rishidave/feat/ortvalue-array-dlpack-protocols-v2

Conversation

@Rishi-Dave
Copy link
Copy Markdown
Contributor

@Rishi-Dave Rishi-Dave commented Apr 4, 2026

Summary

  • Add __array__, __dlpack__, __dlpack_device__, and from_dlpack to the public OrtValue class
  • Enable standard Python interoperability protocols (numpy array protocol + DLPack) on OrtValue
  • Auto-detect boolean dtype from source objects in from_dlpack to avoid the uint8/bool ambiguity in older DLPack versions

Motivation

Fixes #24071

The C-level C.OrtValue already supports __dlpack__, __dlpack_device__, and from_dlpack, but the public Python wrapper OrtValue class does not expose them. Users currently have to access the private _ortvalue attribute (e.g. ortvalue._ortvalue.__dlpack__()) for DLPack interop. Similarly, np.asarray(ortvalue) doesn't work because __array__ is not implemented.

This makes OrtValue a well-behaved tensor type that works out of the box with:

  • np.asarray(ortvalue) / np.array(ortvalue) via __array__
  • torch.from_dlpack(ortvalue) via __dlpack__ / __dlpack_device__
  • OrtValue.from_dlpack(torch_tensor) via the from_dlpack classmethod

Changes

onnxruntime/python/onnxruntime_inference_collection.py:

  • __array__(dtype, copy): Delegates to self.numpy() with optional dtype conversion. Supports numpy 2.0 copy semantics while remaining compatible with older numpy versions.
  • __dlpack__(*, stream): Thin wrapper over the C-level __dlpack__.
  • __dlpack_device__(): Thin wrapper over the C-level __dlpack_device__.
  • from_dlpack(data): Classmethod that accepts any __dlpack__-compatible object or raw DLPack capsule. Detects boolean dtype from the source object's dtype attribute or data_type() method, avoiding the uint8/bool false-positive that is_dlpack_uint8_tensor would produce on genuine uint8 data.

onnxruntime/test/python/onnxruntime_test_python.py:

  • test_ort_value_array_protocol: Tests np.asarray/np.array with float32, int64, bool dtypes, and dtype conversion.
  • test_ort_value_dlpack_protocol: Tests __dlpack__ and __dlpack_device__ on the public class.
  • test_ort_value_from_dlpack_protocol_object: Tests from_dlpack with numpy arrays and OrtValue-to-OrtValue round-trip, verifying zero-copy (shared memory).
  • test_ort_value_from_dlpack_bool: Tests bool round-trip and verifies uint8 is not falsely detected as bool.

Test Plan

  • ruff check passes on both modified files
  • ruff format --check passes on both modified files
  • lintrunner reports no issues
  • Existing test_ort_value_dlpack test continues to pass
  • All logic paths verified against C-level bindings (bool detection, dtype conversion, shared memory)
  • CI: new tests pass against a full build with DLPack enabled

@Rishi-Dave
Add __array__, __dlpack__, __dlpack_device__, and from_dlpack to OrtV…
77c27fb 
…alue

Expose standard Python protocols on the public OrtValue class so it
interoperates with numpy, PyTorch, JAX, and other frameworks without
requiring users to reach into the private _ortvalue attribute.

- __array__(dtype, copy): supports np.asarray(ortvalue) and np.array()
  following the numpy array protocol, with numpy 2.0 copy semantics.
- __dlpack__(stream): returns a DLPack capsule for zero-copy sharing.
- __dlpack_device__(): returns (device_type, device_id) tuple.
- from_dlpack(data): classmethod accepting any __dlpack__-compatible
  object or raw capsule, with automatic bool dtype detection from the
  source object's dtype to avoid the uint8/bool ambiguity in older
  DLPack versions.

Fixes microsoft#24071
@tianleiwu tianleiwu requested a review from Copilot April 4, 2026 17:24
Copilot AI reviewed Apr 4, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds standard Python interoperability protocols to the public onnxruntime.OrtValue wrapper so it can participate in NumPy’s __array__ protocol and the DLPack exchange protocol without users reaching into private _ortvalue bindings (addressing #24071).

Changes:

  • Add OrtValue.__array__ to support np.asarray(ortvalue) / np.array(ortvalue) with optional dtype conversion and NumPy 2.0 copy semantics.
  • Expose DLPack protocol methods on OrtValue: __dlpack__, __dlpack_device__, and from_dlpack with bool dtype auto-detection for protocol objects.
  • Add Python unit tests covering __array__, public DLPack methods, from_dlpack object/capsule inputs, and bool vs uint8 behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
onnxruntime/python/onnxruntime_inference_collection.py Adds __array__ and DLPack protocol wrappers (__dlpack__, __dlpack_device__, from_dlpack) to the public OrtValue class.
onnxruntime/test/python/onnxruntime_test_python.py Adds tests validating NumPy array protocol behavior and public DLPack interop on OrtValue.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread onnxruntime/python/onnxruntime_inference_collection.py Outdated
Comment thread onnxruntime/python/onnxruntime_inference_collection.py Outdated
Comment thread onnxruntime/test/python/onnxruntime_test_python.py Outdated
@Rishi-Dave
fix: preserve zero-copy in __array__ dtype path, add shared-memory as…
9fef016 
…sertions

Use np.asarray instead of np.array for the dtype-only branch in
__array__, so that when the requested dtype already matches the
underlying buffer, no unnecessary copy is made.

Add data_ptr/ctypes.data assertions in the __array__ tests to verify
zero-copy behavior for both the default and same-dtype paths.
tianleiwu
tianleiwu reviewed Apr 4, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@tianleiwu tianleiwu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — PR #27980

Summary: Clean addition of __array__, __dlpack__, __dlpack_device__, and from_dlpack to the public OrtValue class. The wrapper pattern correctly delegates to the C-level bindings. Tests cover the main use cases including bool/uint8 disambiguation.

Python Interop Surface (onnxruntime_inference_collection.py)

Positive: The DLPack protocol wrapper is thin and correct — it forwards to proven C-level implementations without adding unnecessary logic.

Positive: Boolean auto-detection in from_dlpack addresses a real DLPack limitation (bool → uint8 encoding). Inspecting the source object's dtype before consuming the capsule is the right approach.

See inline comments for two suggestions.

Tests (onnxruntime_test_python.py)

Positive: Good coverage across float32, int64, bool, and uint8 dtypes. The uint8-not-falsely-detected-as-bool test is particularly important.

Positive: Shared-memory assertion via data_ptr() comparison in test_ort_value_from_dlpack_protocol_object directly validates zero-copy semantics.

Comment thread onnxruntime/python/onnxruntime_inference_collection.py
Comment thread onnxruntime/python/onnxruntime_inference_collection.py
@Rishi-Dave
Use dtype.name for more portable bool detection in from_dlpack
b05210c 
str(data.dtype) returns framework-specific representations that
don't always match simple string comparisons (e.g. TensorFlow
returns "<dtype: 'bool'>"). Use the .name attribute when available
(numpy, cupy, tensorflow) and fall back to str() for frameworks
like PyTorch where .name is absent.
@tianleiwu
Copy link
Copy Markdown
Contributor

tianleiwu commented Apr 4, 2026

/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline

@azure-pipelines
Copy link
Copy Markdown

azure-pipelines bot commented Apr 4, 2026

Azure Pipelines successfully started running 4 pipeline(s).

@tianleiwu tianleiwu merged commit b73b3dd into microsoft:main Apr 7, 2026
98 of 100 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@tianleiwu tianleiwu tianleiwu approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Feature Request] Make OrtValue compatible with numpy __array__ and dlpack protocols

3 participants

@Rishi-Dave @tianleiwu