[VitisAI] Recognize native-endian in-memory address tag in process_ext_address#29248
Open
mingyueliuh wants to merge 1 commit into
Open
[VitisAI] Recognize native-endian in-memory address tag in process_ext_address#29248mingyueliuh wants to merge 1 commit into
mingyueliuh wants to merge 1 commit into
Conversation
…t_address ORT marks initializers whose data lives in an in-process memory buffer with one of two location sentinels: the original little-endian tag "*/_ORT_MEM_ADDR_/*" (kTensorProtoLittleEndianMemoryAddressTag) and the newer native-endian tag "*/_ORT_NATIVE_ENDIAN_MEM_ADDR_/*" (kTensorProtoNativeEndianMemoryAddressTag). Since microsoft#27404, TensorToTensorProto() emits the native-endian tag by default, but the VitisAI EP's process_ext_address only matched the little-endian tag. As a result, in-memory initializers (e.g. quantized weights moved out of the TensorProto during graph optimization) are no longer detected by model_clone, fall through to the verbatim-copy branch, and carry the in-memory marker into the cloned model protobuf. After microsoft#28709 added an explicit ORT_ENFORCE in Graph::Graph rejecting in-memory address markers in deserialized protobufs, this now aborts model compilation with: Initializer '...' references an ORT in-memory address marker, which is not allowed in a model protobuf. Both tags encode the buffer address in the 'offset' field; on little-endian hosts (the only platform this EP targets) the byte layout is identical, so they are decoded the same way. Recognizing both tags restores the pre-1.27 behavior of converting these initializers to a lightweight reference during model_clone.
3f8fd5c to
a91086f
Compare
Contributor
Author
|
@adrianlizarraga @yuslepukhin please take a look when you are available. Thanks. |
edgchen1
approved these changes
Jun 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
vaip::process_ext_address(in the VitisAI EP) decodes the in-memory"external data" address marker that ORT plants on initializers whose data lives
in an in-process buffer. It only matched the little-endian tag
"*/_ORT_MEM_ADDR_/*"(kTensorProtoLittleEndianMemoryAddressTag). This PRmakes it also recognize the native-endian tag
"*/_ORT_NATIVE_ENDIAN_MEM_ADDR_/*"(kTensorProtoNativeEndianMemoryAddressTag).Motivation and Context
Two existing changes combined to break VitisAI model compilation:
S390x test fixes #27404 split the single in-memory marker into a little-endian and a
native-endian variant, and switched
TensorToTensorProto(use_tensor_buffer=true)to emit the native-endian tag by default. Both
HasExternalDataInMemoryand the data readers treat the two tags equivalently.
Address external path references in loaded models #28709 added an explicit
ORT_ENFORCE(!utils::HasExternalDataInMemory(tensor), ...)in
Graph::Graph, rejecting any in-memory address marker found in adeserialized model protobuf.
Because
process_ext_addressstill only recognized the little-endian tag,in-memory initializers (e.g. quantized weights moved out of the
TensorProtoduring graph optimization) are no longer detected in
vaip::model_clone. Theyfall through to the verbatim-copy branch and the native-endian marker is carried
into the cloned model proto. When ORT then constructs the cloned
Graph, thenew enforce fires and aborts compilation:
This regression is observed on 1.27.0, where both changes are present (1.25.x /
1.26.x have the native-endian tag but not the enforce, so the marker leaked
silently without crashing).
Both tags encode the buffer address in the
offsetfield; on little-endianhosts (the only platform this EP targets) the byte layout is identical, so they
are decoded the same way. Recognizing both tags restores the previous behavior
of converting these initializers into a lightweight reference inside
model_clone, so the in-memory marker never reachesGraph::Graph.