[ET Device Support] DeviceAllocator interface and DeviceAllocatorRegistry by Gasoonjia · Pull Request #17535 · pytorch/executorch

Gasoonjia · 2026-02-18T19:25:54Z

Stack from ghstack (oldest at bottom):

-> [ET Device Support] DeviceAllocator interface and DeviceAllocatorRegistry #17535
[ET Device Support] Annotate device attributes of CUDA backend IO tensors cuda device #18080
[ET Device Support] Propagate device info from TensorSpec into serialized Tensor #18079
[executorch] Propagate device metadata from partitioner result onto TensorSpecs #18078
[ET Device Support] TensorImpl carries device info #17534
[ET Device Support] Schema changes: device info on Tensor and buffer-level device array #17533

This diff introduces the DeviceAllocator abstract interface and DeviceAllocatorRegistry for device-specific memory allocation. This is a foundational abstraction that enables the runtime to dispatch memory operations to the appropriate device backend other than CPU (CUDA, etc.).

DeviceAllocator interface provides:

init_buffer() - Initialize memory buffer pools for memory-planned tensors
get_offset_address() - Get pointer to offset within pre-allocated buffer
allocate() / deallocate() - Dynamic device memory allocation
copy_host_to_device() / copy_device_to_host() - Data transfer between host and device
device_type() - Returns the device type this allocator handles

DeviceAllocatorRegistry provides:

Singleton registry mapping DeviceType → DeviceAllocator
register_allocator() / get_allocator() methods
Fixed-size array indexed by device type (no dynamic allocation, embedded-friendly)

Design notes:

Registry stores raw pointers (non-owning) - allocators are expected to be singletons with static lifetime
Follows ExecuTorch's embedded-first philosophy (no std::unique_ptr, no heap allocation in registry)
Convenience free functions register_device_allocator() and get_device_allocator() for ease of use

Differential Revision: D93635656

…stry This diff introduces the `DeviceAllocator` abstract interface and `DeviceAllocatorRegistry` for device-specific memory allocation. This is a foundational abstraction that enables the runtime to dispatch memory operations to the appropriate device backend other than CPU (CUDA, etc.). **DeviceAllocator interface provides:** - `init_buffer()` - Initialize memory buffer pools for memory-planned tensors - `get_offset_address()` - Get pointer to offset within pre-allocated buffer - `allocate()` / `deallocate()` - Dynamic device memory allocation - `copy_host_to_device()` / `copy_device_to_host()` - Data transfer between host and device - `device_type()` - Returns the device type this allocator handles **DeviceAllocatorRegistry provides:** - Singleton registry mapping DeviceType → DeviceAllocator - `register_allocator()` / `get_allocator()` methods - Fixed-size array indexed by device type (no dynamic allocation, embedded-friendly) **Design notes:** - Registry stores raw pointers (non-owning) - allocators are expected to be singletons with static lifetime - Follows ExecuTorch's embedded-first philosophy (no std::unique_ptr, no heap allocation in registry) - Convenience free functions `register_device_allocator()` and `get_device_allocator()` for ease of use Differential Revision: [D93635656](https://our.internmc.facebook.com/intern/diff/D93635656/) [ghstack-poisoned]

pytorch-bot · 2026-02-18T19:25:58Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17535

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 43 New Failures

As of commit 76d7f72 with merge base 5db6f22 ():

NEW FAILURES - The following jobs have failed:

pull / unittest / linux / linux-job (gh)
exir/tests/test_serde.py::TestSerde::test_delegate_xnnpack
pull / unittest / macos / macos-job (gh)
exir/tests/test_serde.py::TestSerde::test_delegate_xnnpack
pull / unittest / windows / windows-job (gh)
backends/xnnpack/test/recipes/test_xnnpack_recipes.py::TestXnnpackRecipes::test_8a4w_recipe
pull / unittest-editable / linux / linux-job (gh)
exir/tests/test_serde.py::TestSerde::test_delegate_xnnpack
pull / unittest-editable / macos / macos-job (gh)
exir/tests/test_serde.py::TestSerde::test_delegate_xnnpack
pull / unittest-editable / windows / windows-job (gh)
backends/xnnpack/test/recipes/test_xnnpack_recipes.py::TestXnnpackRecipes::test_8a4w_recipe
Test CUDA Builds / test-cuda-pybind (gemma3-4b, --quantize, google-gemma-3-4b-it-cuda-quantized-int4-tile-packed) / linux-job (gh)
RuntimeError: Command docker exec -t 4339fe478f3674527142a4c45b5fd214f5dbad369b195eaecc17a25d1e856db5 /exec failed with exit code 1
Test CUDA Builds / test-cuda-pybind (gemma3-4b, google-gemma-3-4b-it-cuda-non-quantized) / linux-job (gh)
RuntimeError: Command docker exec -t fd66df3c0a21d4bfd992516a0362c1cb5fcfdbddf93d1a8e1faf60006168e003 /exec failed with exit code 1
Test CUDA Builds / test-cuda-pybind (qwen3-0.6b, --quantize, Qwen-Qwen3-0.6B-cuda-quantized-int4-tile-packed) / linux-job (gh)
RuntimeError: Command docker exec -t 4e570f57c2d0b54d76f4ed5c65ee00309aba628e4c8f7525bdf4f507b5e10822 /exec failed with exit code 1
Test CUDA Builds / test-cuda-pybind (qwen3-0.6b, Qwen-Qwen3-0.6B-cuda-non-quantized) / linux-job (gh)
RuntimeError: Command docker exec -t 935a6f252e16eed8d8b86d99a070f6b1f4ab6a8f8b872ffd1179a85fa69f89eb /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (google, gemma-3-4b-it, non-quantized) / linux-job (gh)
RuntimeError: Command docker exec -t 62ddc1d7ce0f15ad9cfe2a5acad98bba699e7cdbc0733fb31f35af348fedc9e5 /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (google, gemma-3-4b-it, quantized-int4-tile-packed) / linux-job (gh)
RuntimeError: Command docker exec -t fd8ce686e344eebde334f61e7df16ff319dd0afe82be1a4ed4c79b2699580085 /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (mistralai, Voxtral-Mini-3B-2507, non-quantized) / linux-job (gh)
RuntimeError: Command docker exec -t 295992c8b3091fac8d0dacd9264e6d92dc89238e33bfef06293c9ff7b610b150 /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (mistralai, Voxtral-Mini-3B-2507, quantized-int4-tile-packed) / linux-job (gh)
RuntimeError: Command docker exec -t 5733691c087bdcea655158c9339ec9ea2a0bfb1c4bf613b43ea23a970c95dd7b /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (mistralai, Voxtral-Mini-3B-2507, quantized-int4-weight-only) / linux-job (gh)
RuntimeError: Command docker exec -t dbf6ced9719dcbda0faa2965f53bf6bbb6038d349c86473e09cec084c5a36eac /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (mistralai, Voxtral-Mini-4B-Realtime-2602, quantized-int4-tile-packed) / linux-job (gh)
RuntimeError: Command docker exec -t e653f542211ff906031879d1f1dfb37cb7e7b674786cb47a5d3a2a3c02b89fbc /exec failed with exit code 134
Test CUDA Builds / test-model-cuda-e2e (nvidia, diar_streaming_sortformer_4spk-v2, non-quantized) / linux-job (gh)
RuntimeError: Command docker exec -t 7df4225b74cd22da3792b219b430afeb9c6b077e4b4ff58ce3455bef2053dd20 /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (nvidia, parakeet-tdt, non-quantized) / linux-job (gh)
RuntimeError: Command docker exec -t e6fe3056f42b9bb938d6fddd216f502c2b3e1bb1373a7a20f82019483114dd15 /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (nvidia, parakeet-tdt, quantized-int4-tile-packed) / linux-job (gh)
RuntimeError: Command docker exec -t e45def5104a0d5d67aab32e5c40fadd358fc8f3c87697e413838e7e3cb04a77a /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (nvidia, parakeet-tdt, quantized-int4-weight-only) / linux-job (gh)
RuntimeError: Command docker exec -t 39bf39375d983a81a9cd9b695213b01370ce26a18a181c9fd13487390947e957 /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (openai, whisper-large-v3-turbo, non-quantized) / linux-job (gh)
RuntimeError: Command docker exec -t 6ef5533019850e59159a054e58921f299d5e6c01a10e63cbeccbf54ffca1802c /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (openai, whisper-large-v3-turbo, quantized-int4-tile-packed) / linux-job (gh)
RuntimeError: Command docker exec -t 84b3682dde3b3b51d2eaa23b9cbb698d60284b9b658f1d3d5e5ba7483b9e6ff9 /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (openai, whisper-large-v3-turbo, quantized-int4-weight-only) / linux-job (gh)
RuntimeError: Command docker exec -t c99589bf25781ee544a60c7b06042304e9f11401d136d34b6ff541464b1156e8 /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (openai, whisper-small, non-quantized) / linux-job (gh)
RuntimeError: Command docker exec -t 6975ba9b8aaef3cb849d19aeca3abe1daa6b06879686ec07ea3041c81c674c11 /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (openai, whisper-small, quantized-int4-tile-packed) / linux-job (gh)
RuntimeError: Command docker exec -t 948e133d39e830bf9d03e8948f3c76cfc1fe4d549200667a39b0da4501db1caf /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (openai, whisper-small, quantized-int4-weight-only) / linux-job (gh)
RuntimeError: Command docker exec -t 4052e34651a91d9c050bb313a6e8e8c320cef51b41e31dfdc39afb3752d39e11 /exec failed with exit code 1
Test CUDA Builds / test-models-cuda (add_mul) / linux-job (gh)
RuntimeError: Command docker exec -t 3cbe271c15d1053e58380b0a9b565abfb47898eaffa1fa2f45d41734ad3296ac /exec failed with exit code 134
Test CUDA Builds / test-models-cuda (add) / linux-job (gh)
RuntimeError: Command docker exec -t 1cd18795fe58071a979adb4c84f1ad43b036f93c7ea7acc91d560a3b3cf4cb9a /exec failed with exit code 134
Test CUDA Builds / test-models-cuda (conv1d) / linux-job (gh)
RuntimeError: Command docker exec -t 7042fd3f916d2135abbe212ff340e169d959f618b2a8bc3b2c102465c25f7f29 /exec failed with exit code 134
Test CUDA Builds / test-models-cuda (linear) / linux-job (gh)
RuntimeError: Command docker exec -t 814abac5a75eb7c9696a59458a0a9adcab9b0809531ea7b86f67ce8d097843ba /exec failed with exit code 134
Test CUDA Builds / test-models-cuda (mv2) / linux-job (gh)
RuntimeError: Command docker exec -t c64c31db916c77bf4c04e2fd6c3c1e7d531d309be20e50203abeeddc32963668 /exec failed with exit code 134
Test CUDA Builds / test-models-cuda (mv3) / linux-job (gh)
RuntimeError: Command docker exec -t 4fa7160f1b1ed7dd0ffc291c0d23360b30da1b14d65134727d0bfd1fb8c113ae /exec failed with exit code 134
Test CUDA Builds / test-models-cuda (resnet18) / linux-job (gh)
RuntimeError: Command docker exec -t f623710afbcbd713ea30cb65f692409dfb6b8c0ce1de1efad9d296e1bd62c75b /exec failed with exit code 134
Test CUDA Builds / test-models-cuda (sdpa) / linux-job (gh)
RuntimeError: Command docker exec -t ffcd793f36a1607dca98e6a917fdc3139958c7395c68741a93c5e9b4d0fcb611 /exec failed with exit code 134
Test CUDA Builds / unittest-cuda / linux-job (gh)
RuntimeError: Command docker exec -t 73bf84710a20a19219e51dde44c58da5e3c8d5173ffd79cfad1e6ef4c61ad85d /exec failed with exit code 135
Test CUDA Windows Export and E2E / test-model-cuda-windows-e2e (nvidia, diar_streaming_sortformer_4spk-v2, non-quantized) / windows-job (gh)
Process completed with exit code 1.
Test Metal Backend / test-model-metal-e2e (mistralai, Voxtral-Mini-3B-2507, non-quantized) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
Test Metal Backend / test-model-metal-e2e (mistralai, Voxtral-Mini-3B-2507, quantized-int4-metal) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
Test Metal Backend / test-model-metal-e2e (mistralai, Voxtral-Mini-4B-Realtime-2602, quantized-int4-metal) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 134
Test Metal Backend / test-model-metal-e2e (openai, whisper-large-v3-turbo, non-quantized) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
Test Metal Backend / test-model-metal-e2e (openai, whisper-large-v3-turbo, quantized-int4-metal) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
Test Metal Backend / test-model-metal-e2e (openai, whisper-small, non-quantized) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
Test Metal Backend / test-model-metal-e2e (openai, whisper-small, quantized-int4-metal) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

…stry This diff introduces the `DeviceAllocator` abstract interface and `DeviceAllocatorRegistry` for device-specific memory allocation. This is a foundational abstraction that enables the runtime to dispatch memory operations to the appropriate device backend other than CPU (CUDA, etc.). **DeviceAllocator interface provides:** - `init_buffer()` - Initialize memory buffer pools for memory-planned tensors - `get_offset_address()` - Get pointer to offset within pre-allocated buffer - `allocate()` / `deallocate()` - Dynamic device memory allocation - `copy_host_to_device()` / `copy_device_to_host()` - Data transfer between host and device - `device_type()` - Returns the device type this allocator handles **DeviceAllocatorRegistry provides:** - Singleton registry mapping DeviceType → DeviceAllocator - `register_allocator()` / `get_allocator()` methods - Fixed-size array indexed by device type (no dynamic allocation, embedded-friendly) **Design notes:** - Registry stores raw pointers (non-owning) - allocators are expected to be singletons with static lifetime - Follows ExecuTorch's embedded-first philosophy (no std::unique_ptr, no heap allocation in registry) - Convenience free functions `register_device_allocator()` and `get_device_allocator()` for ease of use Differential Revision: [D93635656](https://our.internmc.facebook.com/intern/diff/D93635656/) ghstack-source-id: 342367956 Pull Request resolved: #17535

github-actions · 2026-02-18T19:27:27Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

…locatorRegistry" This diff introduces the `DeviceAllocator` abstract interface and `DeviceAllocatorRegistry` for device-specific memory allocation. This is a foundational abstraction that enables the runtime to dispatch memory operations to the appropriate device backend other than CPU (CUDA, etc.). **DeviceAllocator interface provides:** - `init_buffer()` - Initialize memory buffer pools for memory-planned tensors - `get_offset_address()` - Get pointer to offset within pre-allocated buffer - `allocate()` / `deallocate()` - Dynamic device memory allocation - `copy_host_to_device()` / `copy_device_to_host()` - Data transfer between host and device - `device_type()` - Returns the device type this allocator handles **DeviceAllocatorRegistry provides:** - Singleton registry mapping DeviceType → DeviceAllocator - `register_allocator()` / `get_allocator()` methods - Fixed-size array indexed by device type (no dynamic allocation, embedded-friendly) **Design notes:** - Registry stores raw pointers (non-owning) - allocators are expected to be singletons with static lifetime - Follows ExecuTorch's embedded-first philosophy (no std::unique_ptr, no heap allocation in registry) - Convenience free functions `register_device_allocator()` and `get_device_allocator()` for ease of use Differential Revision: [D93635656](https://our.internmc.facebook.com/intern/diff/D93635656/) [ghstack-poisoned]

…stry Pull Request resolved: #17535 This diff introduces the `DeviceAllocator` abstract interface and `DeviceAllocatorRegistry` for device-specific memory allocation. This is a foundational abstraction that enables the runtime to dispatch memory operations to the appropriate device backend other than CPU (CUDA, etc.). **DeviceAllocator interface provides:** - `init_buffer()` - Initialize memory buffer pools for memory-planned tensors - `get_offset_address()` - Get pointer to offset within pre-allocated buffer - `allocate()` / `deallocate()` - Dynamic device memory allocation - `copy_host_to_device()` / `copy_device_to_host()` - Data transfer between host and device - `device_type()` - Returns the device type this allocator handles **DeviceAllocatorRegistry provides:** - Singleton registry mapping DeviceType → DeviceAllocator - `register_allocator()` / `get_allocator()` methods - Fixed-size array indexed by device type (no dynamic allocation, embedded-friendly) **Design notes:** - Registry stores raw pointers (non-owning) - allocators are expected to be singletons with static lifetime - Follows ExecuTorch's embedded-first philosophy (no std::unique_ptr, no heap allocation in registry) - Convenience free functions `register_device_allocator()` and `get_device_allocator()` for ease of use Differential Revision: [D93635656](https://our.internmc.facebook.com/intern/diff/D93635656/) ghstack-source-id: 342371816

…locatorRegistry" This diff introduces the `DeviceAllocator` abstract interface and `DeviceAllocatorRegistry` for device-specific memory allocation. This is a foundational abstraction that enables the runtime to dispatch memory operations to the appropriate device backend other than CPU (CUDA, etc.). **DeviceAllocator interface provides:** - `init_buffer()` - Initialize memory buffer pools for memory-planned tensors - `get_offset_address()` - Get pointer to offset within pre-allocated buffer - `allocate()` / `deallocate()` - Dynamic device memory allocation - `copy_host_to_device()` / `copy_device_to_host()` - Data transfer between host and device - `device_type()` - Returns the device type this allocator handles **DeviceAllocatorRegistry provides:** - Singleton registry mapping DeviceType → DeviceAllocator - `register_allocator()` / `get_allocator()` methods - Fixed-size array indexed by device type (no dynamic allocation, embedded-friendly) **Design notes:** - Registry stores raw pointers (non-owning) - allocators are expected to be singletons with static lifetime - Follows ExecuTorch's embedded-first philosophy (no std::unique_ptr, no heap allocation in registry) - Convenience free functions `register_device_allocator()` and `get_device_allocator()` for ease of use Differential Revision: [D93635656](https://our.internmc.facebook.com/intern/diff/D93635656/) [ghstack-poisoned]

…stry Pull Request resolved: #17535 This diff introduces the `DeviceAllocator` abstract interface and `DeviceAllocatorRegistry` for device-specific memory allocation. This is a foundational abstraction that enables the runtime to dispatch memory operations to the appropriate device backend other than CPU (CUDA, etc.). **DeviceAllocator interface provides:** - `allocate()` / `deallocate()` - Dynamic device memory allocation - `copy_host_to_device()` / `copy_device_to_host()` - Data transfer between host and device - `device_type()` - Returns the device type this allocator handles **DeviceAllocatorRegistry provides:** - Singleton registry mapping DeviceType → DeviceAllocator - `register_allocator()` / `get_allocator()` methods - Fixed-size array indexed by device type (no dynamic allocation, embedded-friendly) **Design notes:** - Registry stores raw pointers (non-owning) - allocators are expected to be singletons with static lifetime - Follows ExecuTorch's embedded-first philosophy (no std::unique_ptr, no heap allocation in registry) - Convenience free functions `register_device_allocator()` and `get_device_allocator()` for ease of use ghstack-source-id: 350691519 Differential Revision: [D93635656](https://our.internmc.facebook.com/intern/diff/D93635656/)

Gasoonjia requested review from JacobSzwejbka and lucylq as code owners February 18, 2026 19:25

Gasoonjia mentioned this pull request Feb 18, 2026

[ET Device Support] Schema changes: device info on Tensor and buffer-level device array #17533

Open

Gasoonjia mentioned this pull request Feb 18, 2026

[ET Device Support] TensorImpl carries device info #17534

Open

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 18, 2026

meta-codesync bot added fb-exported meta-exported labels Feb 18, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ET Device Support] DeviceAllocator interface and DeviceAllocatorRegistry#17535

[ET Device Support] DeviceAllocator interface and DeviceAllocatorRegistry#17535
Gasoonjia wants to merge 3 commits intogh/gasoonjia/124/basefrom
gh/gasoonjia/124/head

Gasoonjia commented Feb 18, 2026 •

edited

Loading

Uh oh!

pytorch-bot bot commented Feb 18, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Feb 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Gasoonjia commented Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17535

❌ 43 New Failures

Uh oh!

github-actions bot commented Feb 18, 2026

This PR needs a release notes: label

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Gasoonjia commented Feb 18, 2026 •

edited

Loading

pytorch-bot bot commented Feb 18, 2026 •

edited

Loading

This PR needs a `release notes:` label