Skip to content

[VitisAI] Recognize native-endian in-memory address tag in process_ext_address#29248

Open
mingyueliuh wants to merge 1 commit into
microsoft:mainfrom
mingyueliuh:fix/vitisai-clone-native-endian-mem-addr
Open

[VitisAI] Recognize native-endian in-memory address tag in process_ext_address#29248
mingyueliuh wants to merge 1 commit into
microsoft:mainfrom
mingyueliuh:fix/vitisai-clone-native-endian-mem-addr

Conversation

@mingyueliuh

Copy link
Copy Markdown
Contributor

Description

vaip::process_ext_address (in the VitisAI EP) decodes the in-memory
"external data" address marker that ORT plants on initializers whose data lives
in an in-process buffer. It only matched the little-endian tag
"*/_ORT_MEM_ADDR_/*" (kTensorProtoLittleEndianMemoryAddressTag). This PR
makes it also recognize the native-endian tag
"*/_ORT_NATIVE_ENDIAN_MEM_ADDR_/*" (kTensorProtoNativeEndianMemoryAddressTag).

if (file == "*/_ORT_MEM_ADDR_/*" || file == "*/_ORT_NATIVE_ENDIAN_MEM_ADDR_/*") {
  auto addr = reinterpret_cast<const char*>(offset);
  return {addr, size};
}

Motivation and Context

Two existing changes combined to break VitisAI model compilation:

  1. S390x test fixes #27404 split the single in-memory marker into a little-endian and a
    native-endian variant, and switched TensorToTensorProto(use_tensor_buffer=true)
    to emit the native-endian tag by default. Both HasExternalDataInMemory
    and the data readers treat the two tags equivalently.

  2. Address external path references in loaded models #28709 added an explicit ORT_ENFORCE(!utils::HasExternalDataInMemory(tensor), ...)
    in Graph::Graph, rejecting any in-memory address marker found in a
    deserialized model protobuf.

Because process_ext_address still only recognized the little-endian tag,
in-memory initializers (e.g. quantized weights moved out of the TensorProto
during graph optimization) are no longer detected in vaip::model_clone. They
fall through to the verbatim-copy branch and the native-endian marker is carried
into the cloned model proto. When ORT then constructs the cloned Graph, the
new enforce fires and aborts compilation:

Initializer 'onnx::Conv_1575_quantized' references an ORT in-memory address
marker, which is not allowed in a model protobuf.

This regression is observed on 1.27.0, where both changes are present (1.25.x /
1.26.x have the native-endian tag but not the enforce, so the marker leaked
silently without crashing).

Both tags encode the buffer address in the offset field; on little-endian
hosts (the only platform this EP targets) the byte layout is identical, so they
are decoded the same way. Recognizing both tags restores the previous behavior
of converting these initializers into a lightweight reference inside
model_clone, so the in-memory marker never reaches Graph::Graph.

…t_address


ORT marks initializers whose data lives in an in-process memory buffer with one
of two location sentinels: the original little-endian tag "*/_ORT_MEM_ADDR_/*"
(kTensorProtoLittleEndianMemoryAddressTag) and the newer native-endian tag
"*/_ORT_NATIVE_ENDIAN_MEM_ADDR_/*" (kTensorProtoNativeEndianMemoryAddressTag).
Since microsoft#27404, TensorToTensorProto() emits the native-endian tag by default, but
the VitisAI EP's process_ext_address only matched the little-endian tag.

As a result, in-memory initializers (e.g. quantized weights moved out of the
TensorProto during graph optimization) are no longer detected by model_clone,
fall through to the verbatim-copy branch, and carry the in-memory marker into
the cloned model protobuf. After microsoft#28709 added an explicit ORT_ENFORCE in
Graph::Graph rejecting in-memory address markers in deserialized protobufs, this
now aborts model compilation with:

  Initializer '...' references an ORT in-memory address marker, which is not
  allowed in a model protobuf.

Both tags encode the buffer address in the 'offset' field; on little-endian
hosts (the only platform this EP targets) the byte layout is identical, so they
are decoded the same way. Recognizing both tags restores the pre-1.27 behavior
of converting these initializers to a lightweight reference during model_clone.
@mingyueliuh mingyueliuh force-pushed the fix/vitisai-clone-native-endian-mem-addr branch from 3f8fd5c to a91086f Compare June 24, 2026 10:54
@mingyueliuh

Copy link
Copy Markdown
Contributor Author

@adrianlizarraga @yuslepukhin please take a look when you are available. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants