Skip to content

[Matrix] Add comparison operator tests - #1447

Open
farzonl wants to merge 6 commits into
llvm:mainfrom
farzonl:task/issue-1446
Open

[Matrix] Add comparison operator tests#1447
farzonl wants to merge 6 commits into
llvm:mainfrom
farzonl:task/issue-1446

Conversation

@farzonl

@farzonl farzonl commented Aug 18, 2026

Copy link
Copy Markdown
Member

resolves #1446

cover cmp operators across matrix sizes and base types

assisted by co-pilot MAI model

@farzonl farzonl added the test-all When applied to a PR this will opt-in to additional pre-merge test configurations.. label Aug 18, 2026
@farzonl farzonl changed the title Task/issue 1446 [Matrix] Add comparison operator tests Aug 18, 2026
@farzonl

farzonl commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

So I don't have to come back and remove a bunch of xfails lets wait on this PR until llvm/llvm-project#216791 merges

@farzonl

farzonl commented Aug 19, 2026

Copy link
Copy Markdown
Member Author

the tests that have 64/16 bit requires are not as exhaustive with operator testing as the 32bit tests.

@kmpeng
kmpeng self-requested a review August 19, 2026 17:22
@farzonl

farzonl commented Aug 20, 2026

Copy link
Copy Markdown
Member Author

Failures to investigate:

DXC Metal

FAIL: OffloadTest-mtl :: Basic/Matrix/matrix_comparison_operators.uint32.test (5 of 621)
******************** TEST 'OffloadTest-mtl :: Basic/Matrix/matrix_comparison_operators.uint32.test' FAILED ********************
Exit Code: -6

Command Output (stdout):
--
# RUN: at line 85
/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/split-file /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/OffloadTest/test/Basic/Matrix/matrix_comparison_operators.uint32.test /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp
# executed command: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/split-file /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/OffloadTest/test/Basic/Matrix/matrix_comparison_operators.uint32.test /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp
# RUN: at line 86
"/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/DXC/build/bin/dxc-3.7" -T cs_6_0 -Fo /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/source.hlsl
# executed command: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/DXC/build/bin/dxc-3.7 -T cs_6_0 -Fo /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/source.hlsl
# RUN: at line 87
/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/offloader -debug-layer /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/pipeline.yaml /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o
# executed command: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/offloader -debug-layer /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/pipeline.yaml /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o
# .---command stdout------------
# | Using Metal API
# | Descriptor heap created with 2 descriptors.
# | Creating UAV: { Size = 48, Register = u0, Space = 0, HasCounter = 0 }
# `-----------------------------
# .---command stderr------------
# | Validation Warning: [ mvk-warn ]
# | VK_ERROR_FEATURE_NOT_PRESENT: Metal does not support buffer robustness.
# | 
# | Unsupported Resource format specified
# | UNREACHABLE executed at /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/OffloadTest/lib/API/MTL/MTLDevice.cpp:128!
# | PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace and instructions to reproduce the bug.
# | Stack dump:
# | 0.	Program arguments: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/offloader -debug-layer /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/pipeline.yaml /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o
# | Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
# | 0  offloader                0x0000000102b5d208 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56
# | 1  offloader                0x0000000102b5af50 llvm::sys::RunSignalHandlers() + 204
# | 2  offloader                0x0000000102b5dd08 SignalHandler(int, __siginfo*, void*) + 320
# | 3  libsystem_platform.dylib 0x000000018a843744 _sigtramp + 56
# | 4  libsystem_pthread.dylib  0x000000018a839888 pthread_kill + 296
# | 5  libsystem_c.dylib        0x000000018a73e850 abort + 124
# | 6  offloader                0x0000000102b26318 llvm::install_out_of_memory_new_handler() + 0
# | 7  offloader                0x0000000102ba483c (anonymous namespace)::MTLDevice::createResource(offloadtest::Resource&, unsigned long) + 912
# | 8  offloader                0x0000000102b9d500 (anonymous namespace)::MTLDevice::createBuffers(offloadtest::Pipeline&, (anonymous namespace)::MTLDevice::InvocationState&) + 928
# | 9  offloader                0x0000000102b967e0 (anonymous namespace)::MTLDevice::executeProgram(offloadtest::Pipeline&) + 1080
# | 10 offloader                0x0000000102b00998 main + 2072
# | 11 dyld                     0x000000018a471d54 start + 7184
# `-----------------------------
# error: command failed with exit status: -6

Clang DXC Metal

******************** TEST 'OffloadTest-clang-mtl :: Basic/Matrix/matrix_comparison_operators.uint32.test' FAILED ********************
Exit Code: -6

Command Output (stdout):
--
# RUN: at line 85
/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/split-file /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/OffloadTest/test/Basic/Matrix/matrix_comparison_operators.uint32.test /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp
# executed command: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/split-file /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/OffloadTest/test/Basic/Matrix/matrix_comparison_operators.uint32.test /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp
# RUN: at line 86
/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/clang-dxc --dxv-path=/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/DXC/build/bin -T cs_6_0 -Fo /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/source.hlsl
# executed command: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/clang-dxc --dxv-path=/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/DXC/build/bin -T cs_6_0 -Fo /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/source.hlsl
# .---command stdout------------
# | Signed DxilContainer written to "/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o"
# | Validation succeeded.
# `-----------------------------
# RUN: at line 87
/Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/offloader -debug-layer /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/pipeline.yaml /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o
# executed command: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/offloader -debug-layer /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/pipeline.yaml /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o
# .---command stdout------------
# | Using Metal API
# | Descriptor heap created with 2 descriptors.
# | Creating UAV: { Size = 48, Register = u0, Space = 0, HasCounter = 0 }
# `-----------------------------
# .---command stderr------------
# | Validation Warning: [ mvk-warn ]
# | VK_ERROR_FEATURE_NOT_PRESENT: Metal does not support buffer robustness.
# | 
# | Unsupported Resource format specified
# | UNREACHABLE executed at /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/OffloadTest/lib/API/MTL/MTLDevice.cpp:128!
# | PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace and instructions to reproduce the bug.
# | Stack dump:
# | 0.	Program arguments: /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/bin/offloader -debug-layer /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp/pipeline.yaml /Users/admin/actions-runner/_work/offload-test-suite/offload-test-suite/llvm-project/build/tools/OffloadTest/test/clang-mtl/Basic/Matrix/Output/matrix_comparison_operators.uint32.test.tmp.o
# | Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
# | 0  offloader                0x00000001007ed208 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56
# | 1  offloader                0x00000001007eaf50 llvm::sys::RunSignalHandlers() + 204
# | 2  offloader                0x00000001007edd08 SignalHandler(int, __siginfo*, void*) + 320
# | 3  libsystem_platform.dylib 0x000000018a843744 _sigtramp + 56
# | 4  libsystem_pthread.dylib  0x000000018a839888 pthread_kill + 296
# | 5  libsystem_c.dylib        0x000000018a73e850 abort + 124
# | 6  offloader                0x00000001007b6318 llvm::install_out_of_memory_new_handler() + 0
# | 7  offloader                0x000000010083483c (anonymous namespace)::MTLDevice::createResource(offloadtest::Resource&, unsigned long) + 912
# | 8  offloader                0x000000010082d500 (anonymous namespace)::MTLDevice::createBuffers(offloadtest::Pipeline&, (anonymous namespace)::MTLDevice::InvocationState&) + 928
# | 9  offloader                0x00000001008267e0 (anonymous namespace)::MTLDevice::executeProgram(offloadtest::Pipeline&) + 1080
# | 10 offloader                0x0000000100790998 main + 2072
# | 11 dyld                     0x000000018a471d54 start + 7184
# `-----------------------------
# error: command failed with exit status: -6

The issue in Metal seems to be offloader specific. We don't have a uint32 Format.
lib/API/MTL/MTLDevice.cpp:128

static MTL::PixelFormat getMTLFormat(DataFormat Format, int Channels) {
  switch (Format) {
  case DataFormat::Int32:
    MTLFormats(32Sint) break;
  case DataFormat::Float32:
    MTLFormats(32Float) break;
  case DataFormat::UInt64:
  case DataFormat::Int64:
    if (Channels == 1)
      return MTL::PixelFormatRG32Uint;
    if (Channels == 2)
      return MTL::PixelFormatRGBA32Uint;
    llvm_unreachable("Unsupported channel count for 64-bit format");

  default:
    llvm_unreachable("Unsupported Resource format specified");
  }
  return MTL::PixelFormatInvalid;
}

@farzonl farzonl removed the test-all When applied to a PR this will opt-in to additional pre-merge test configurations.. label Aug 20, 2026
Comment thread test/Basic/Matrix/matrix_comparison_operators.fp16.test Outdated
@damyanp
damyanp requested review from Icohedron and alsepkow August 25, 2026 17:23

@Icohedron Icohedron left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why the 32-bit tests have to be the only comprehensive ones.
The 32-bit tests aren't even that much bigger than the other tests in line count either.

e.g. the fp16 test is 72 lines while the fp32 test is 105 lines


[numthreads(1, 1, 1)]
void main() {
half4x3 HalfEQLHS = HalfIn[0], HalfEQRHS = HalfIn[1];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION] You should do the same as commit 8ebdcbd and make all the tests use unique matrix elements to confirm that the compares are actually done elementwise. Otherwise the comparisons collapse to single fcmp instructions.

...
#--- end

# Unimplemented https://github.com/microsoft/DirectXShaderCompiler/issues/8796

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[QUESTION] I'm wondering why these tests don't carry the same platform exclusion as the bool-matrix tests.

matrix_bool_and_operator.test, matrix_bool_and_operator_thread_group.test and
matrix_bool_or_operator.test all end with:

  # Note: Metal does not support boolean matrix types. For
  # Vulkan KosmicKrisp and MoltenVK the output is always False.
  # UNSUPPORTED: Metal || KosmicKrisp || MoltenVK

Every test in this PR produces boolNxM values from the comparison and indexes them.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The failures don't happen when you produce a boolean matrix only when you try and do an operation on them. At least for metal. I have not tested the mac vulkan runtimes.

@@ -0,0 +1,71 @@
#--- source.hlsl

RWBuffer<int64_t> Int64In : register(u0);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[ISSUE] RWBuffer<int64_t> has a known issue in Lavapipe.

The existing typed 64-bit test, test/Feature/TypedBuffer/64bit-scalar.test has this XFAIL:

  # Driver bug https://gitlab.freedesktop.org/mesa/mesa/-/work_items/15866
  # XFAIL: Lavapipe

So either switch to a RWStructuredBuffer<uint64_t> or add the same XFAIL.

Buffers:
- Name: UintIn
Format: UInt32
Data: [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 ]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION] The int and uint tests should exercise values where the sign bit is actually set. As-is, there is no observable difference if the compiler used signed/unsigned operations where it should have been unsigned/signed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Matrix] Add elementwise comparison operator tests

4 participants