arm64: Optimize ARM64 compare mask ExtractMostSignificantBits consumers#129688
arm64: Optimize ARM64 compare mask ExtractMostSignificantBits consumers#129688jonathandavies-arm wants to merge 9 commits into
Conversation
|
Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch |
|
Isn't it the same as #126790 ? |
@tannergooding Optimized all We still have a few places where we call Compare + EMSB instead of explicit IndexOf APIs, that is what my other PR tries to do: #126841 There will be a few places where we won't be able to replaces EMSB with IndexOf (e.g. iterators over multiple matches), though. But I don't think this PR does a good job for them either judging by the diffs - Basically, +600 LOC of JIT changes for 1 use-case (context) in benchmarks. |
|
If you want to make this transformation more useful, you need to borrow the assertionprop.cpp changes from #126790 - |
Teach assertion propagation on ARM64 to recognize Vector64/128 ExtractMostSignificantBits inputs whose value numbers represent per-element boolean masks. The helper recognizes comparison masks, all-zero/all-bits constants, boolean-preserving operations, and reaching PHI values, and marks the EMSB node with a HW intrinsic flag. Consume the flag in rationalization so existing ExtractMostSignificantBits rewrites can handle mask values that have flowed through locals, while keeping unsupported element types filtered out. Add coverage for a comparison mask stored in a local before PopCount(ExtractMostSignificantBits()).
|
A lot bigger diffs now. |
- Share the ARM64 compare-mask intrinsic helper between assertion propagation and rationalization. - Document the widened unsigned compare-mask base type check and missing assertion-prop compiler argument. - Include the zero-or-all-bits-set hardware intrinsic flag in GenTree comparison. - Reuse the shuffle index-type helper when normalizing compare-mask base types. - Extend the compare-mask ExtractMostSignificantBits zero-count rewrite to cover LeadingZeroCount. - Add ARM64 ExtractMostSignificantBits LeadingZeroCount regression coverage. Change-Id: I074f573d5fabfe8b99ec6679267fe485f2209fd7
EgorBo
left a comment
There was a problem hiding this comment.
LGTM, thanks. Perhaps @tannergooding @dotnet/jit-contrib want to take a look at the rationilizer impl?
|
@jonathandavies-arm CI build failures are related |
| // Return Value: | ||
| // True if every SIMD element is known to be either all-bits-set or zero. | ||
| // | ||
| static bool AllComponentsEitherZeroOrAllBitsSet(Compiler* comp, ValueNum vn, var_types baseType) |
There was a problem hiding this comment.
This should really be reusing IsVectorPerElementMask, which is designed exactly for this purpose and handles a few additional scenarios this new method is missing.
The biggest "gotcha" is that Arm64 has a different meaning of the HW_Flag_ReturnsPerElementMask flag and may need that part ifdef'd out, but the ideal is that we fix that to be consistent eventually and use a different flag for any special SVE mask annotation that's required; so that we don't have an ambiguous meaning across platforms.
There was a problem hiding this comment.
I've changed it to use IsVectorPerElementMask
I haven't done anything about the 2nd part. Does it need to be done in this PR?

See discussion at #121981 (comment)
SuperPMI ASM diffs: ExtractMostSignificantBits
Base JIT:
artifacts/asmdiff/builds/59979e64/core_root/libclrjit.soDiff JIT:
artifacts/tests/coreclr/linux.arm64.Checked/Tests/Core_Root/libclrjit.soMCH:
/tmp/ExtractMostSignificantBits_final.mchBase commit:
59979e6401a(origin/main)Diff commit:
e37c3840564Short summary
Diffs are based on 39 contexts (0 MinOpts, 39 FullOpts).
Overall (-480 bytes)
FullOpts (-480 bytes)
SuperPMI summary
Diffs are based on 39 contexts (0 MinOpts, 39 FullOpts).
Overall (-480 bytes)
FullOpts (-480 bytes)
Example diffs
ExtractMostSignificantBits_final.mch
-44 (-52.38%) : 17.dasm - TestExtractMostSignificantBits.Program:CountGreaterThanOrEqualByte(System.Runtime.Intrinsics.Vector128`1[byte],byte):int (FullOpts)
-32 (-47.06%) : 12.dasm - TestExtractMostSignificantBits.Program:CountLessThanInt32(System.Runtime.Intrinsics.Vector128`1[int],int):int (FullOpts)
-32 (-47.06%) : 27.dasm - TestExtractMostSignificantBits.Program:CountLessThanInt3264(System.Runtime.Intrinsics.Vector64`1[int],int):int (FullOpts)
-8 (-13.33%) : 33.dasm - TestExtractMostSignificantBits.Program:IndexOfFirstGreaterThanOrEqualByte64(System.Runtime.Intrinsics.Vector64`1[byte],byte):int (FullOpts)
-8 (-13.33%) : 23.dasm - TestExtractMostSignificantBits.Program:IndexOfFirstGreaterThanOrEqualUInt1664(System.Runtime.Intrinsics.Vector64`1[ushort],ushort):int (FullOpts)
-8 (-13.33%) : 8.dasm - TestExtractMostSignificantBits.Program:IndexOfFirstGreaterThanOrEqualUInt16(System.Runtime.Intrinsics.Vector128`1[ushort],ushort):int (FullOpts)
Details
Size improvements/regressions per collection
PerfScore improvements/regressions per collection
Context information
jit-analyze output
ExtractMostSignificantBits_final.mch
Detail diffs
Jit-analyze summary
Detail diffs
Generated artifacts
artifacts/spmi/superpmi.27.logartifacts/spmi/diff_short_summary.11.mdartifacts/spmi/diff_summary.11.mdartifacts/spmi/asm.ExtractMostSignificantBits_final/summary.mdartifacts/spmi/asm.ExtractMostSignificantBits_final/base/artifacts/spmi/asm.ExtractMostSignificantBits_final/diff/