Skip to content

Vulkan: accept a SymInt delegate arg as a plain Int EValue - #21848

Open
msluszniak wants to merge 1 commit into
pytorch:mainfrom
msluszniak:ms/vulkan-symint-evalue
Open

Vulkan: accept a SymInt delegate arg as a plain Int EValue#21848
msluszniak wants to merge 1 commit into
pytorch:mainfrom
msluszniak:ms/vulkan-symint-evalue

Conversation

@msluszniak

@msluszniak msluszniak commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Summary

VulkanBackend::execute required every SymInt input to arrive as a scalar
tensor:

VK_CHECK_COND(
    args[i]->isTensor(),
    "Cannot handle symint arg to graph that is not derived from a "
    "scalar tensor at the moment.");

That holds when the value was lifted from a scalar tensor, but not when the
Method's schema declares the argument as SymInt — the emitter then passes a
plain Int EValue and the delegate rejects it. A dynamic-shape export whose
sequence length crosses the delegate boundary as a SymInt hits this.

This PR accepts both forms on the way in, and adds the mirrored handling on the
way out, which was missing entirely: the output loop knew about tensors and
TensorRefs but not SymInts, so a graph returning one fell through to
"Could not handle output with type". An output SymInt is now written into the
caller's slot whether that slot is a scalar tensor (Int or Long) or an Int
EValue.

Behaviour for the existing scalar-tensor path is unchanged.

Test plan

  • Existing backends/vulkan/test/test_vulkan_delegate.py suite.
  • On device (Galaxy S26 Ultra, Adreno): Gemma 4 and LFM2.5-VL exports whose
    decoder takes a dynamic sequence length pass the SymInt across the delegate
    boundary as a plain Int and generate correctly. Without this change the
    delegate aborts at the VK_CHECK_COND above during the first execute().

Originally written by @Nklockiewicz and Mateusz Kopciński as part of Gemma 4
Vulkan enablement; extracted here as a standalone change.

cc @SS-JIA @manuelcandales @digantdesai @cbilgin

VulkanBackend::execute required every SymInt input to arrive as a scalar
tensor:

    VK_CHECK_COND(
        args[i]->isTensor(),
        "Cannot handle symint arg to graph that is not derived from a "
        "scalar tensor at the moment.");

That holds when the value was lifted from a scalar tensor, but not when the
Method's schema declares the argument as SymInt -- the emitter then passes a
plain Int EValue and the delegate rejects it. A dynamic-shape export whose
sequence length crosses the delegate boundary as a SymInt hits this.

Accept both forms on the way in, and add the mirrored handling on the way out,
which was missing entirely: the output loop knew about tensors and TensorRefs
but not SymInts, so a graph returning one fell through to "Could not handle
output with type". An output SymInt is now written into the caller's slot
whether that slot is a scalar tensor (Int or Long) or an Int EValue.

Co-authored-by: Norbert Klockiewicz <Nklockiewicz12@gmail.com>
Co-authored-by: Mateusz Kopcinski <mateusz.kopcinski@swmansion.com>
@msluszniak
msluszniak requested a review from SS-JIA as a code owner August 14, 2026 17:40
@pytorch-bot pytorch-bot Bot added the module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/ label Aug 14, 2026
@pytorch-bot

pytorch-bot Bot commented Aug 14, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21848

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 13 Awaiting Approval

As of commit 18d9278 with merge base 33f68d1 (image):

AWAITING APPROVAL - The following workflows need approval before CI can run:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 14, 2026
@linux-foundation-easycla

Copy link
Copy Markdown

CLA Missing ID

  • ✅ login: msluszniak / name: Mateusz Słuszniak (18d9278)
  • ❌ The email address for the commit (18d9278, 18d9278) is not linked to the GitHub account, preventing the EasyCLA check. Consult this Help Article and GitHub Help to resolve. (To view the commit's email address, add .patch at the end of this PR page's URL.) For further assistance with EasyCLA, please visit our EasyCLA portal and chat with our support bot.

One or more co-authors of this pull request were not found. You must specify co-authors in commit message trailer via:

Co-authored-by: name <email>

Supported Co-authored-by: formats include:

  1. Anything <id+login@users.noreply.github.com> - it will locate your GitHub user by id part.
  2. Anything <login@users.noreply.github.com> - it will locate your GitHub user by login part.
  3. Anything <public-email> - it will locate your GitHub user by public-email part. Note that this email must be made public on Github.
  4. Anything <other-email> - it will locate your GitHub user by other-email part but only if that email was used before for any other CLA as a main commit author.
  5. login <any-valid-email> - it will locate your GitHub user by login part, note that login part must be at least 3 characters long.

Alternatively, if the co-author should not be included, remove the Co-authored-by: line from the commit message.

Please update your commit message(s) by doing git commit --amend and then git push [--force] and then request re-running CLA check via commenting on this pull request:

/easycla

@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants