Fix BITCOUNT ... BIT throwing PlatformNotSupportedException on ARM64 - #2008
Open
hexonal wants to merge 1 commit into
Open
Fix BITCOUNT ... BIT throwing PlatformNotSupportedException on ARM64#2008hexonal wants to merge 1 commit into
hexonal wants to merge 1 commit into
Conversation
BitIndexCount(byte, int, int) ends in Popcnt.X64.PopCount, an x86-only hardware intrinsic with no IsSupported guard. It is reachable only from the four-argument BITCOUNT key start end BIT form, so on a host without the x86 POPCNT instruction -- any ARM64 host -- that command throws PlatformNotSupportedException out of ProcessMessages, which terminates the RESP session: the client gets no reply and the connection is dropped. The server process itself stays up and other sessions are unaffected. The plain byte-offset forms are unaffected. Use BitOperations.PopCount instead. That is what __scalar_popc in this same file already does, since microsoft#1034 made the scalar path platform independent; this call site was missed. The counted value is (mask & payload) with both operands byte, so it is always in [0, 255] and the 32-bit overload returns the same count -- verified exhaustively over all 256 x 9 x 9 (payload, startBitOffset, endBitOffset) inputs. On x86-64 with POPCNT this remains a single POPCNT instruction (32-bit operand rather than 64-bit); on ARM64 the JIT emits an AdvSimd CNT/ADDV sequence instead of throwing. The SIMD helpers in this file also call Popcnt.X64, but their only call site is behind Avx2.IsSupported / Ssse3.IsSupported checks that are both false on ARM64, so they are unreachable there and left alone. No new test: GarnetBitmapTests already covers this. On ARM64, BitmapBitCountSimpleTest, BitmapBitCountBitBoundaryUnderflowTest and BitmapBitCountLongBitOffsetParsingTest fail without this change and pass with it (358/358 for the class). They cannot fail in CI because every test matrix is ubuntu-latest/windows-latest, both x64.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes an ARM64 portability bug in Garnet’s BITCOUNT ... BIT path by removing an unguarded x86-only POPCNT intrinsic call and replacing it with a platform-independent popcount implementation.
Changes:
- Replaced
Popcnt.X64.PopCount(...)withBitOperations.PopCount(...)inBitmapManagerBitCount.BitIndexCount(byte, int, int)to avoidPlatformNotSupportedExceptionon ARM64. - Keeps behavior equivalent for the masked-byte popcount while enabling correct execution on non-x86 platforms.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to the portability gap I flagged at the end of #1971 and deliberately kept out of scope there.
Garnet publishes
linux/arm64images (docker-linux.yml:78) and buildslinux-arm64/linux-musl-arm64/win-arm64natives, so this reproduces on a shipped configuration.The bug
BitIndexCount(byte, int, int)ends inPopcnt.X64.PopCount, an x86-only hardware intrinsic with noIsSupportedguard. It is reachable only from the four-argumentBITCOUNT key start end BITform, so on any ARM64 host that command throws:Correcting my own earlier wording: in #1971 I described this as crashing the server process. It doesn't — I measured it properly this time. The exception unwinds out of
ProcessMessagesand terminates the RESP session: the issuing client gets no reply and a reset connection, while the server process stays up and other sessions are unaffected. I verified that directly — a second concurrent connection kept answeringPINGthroughout, and a new connection opened fine afterwards.Repro on macOS arm64, against an unmodified build of
main:Redis returns
1for that last command. EveryBITCOUNT key start end BITfails this way on ARM64, so the BIT-index form is unusable there.The fix
One line: use
BitOperations.PopCount, which is what__scalar_popcin this same file already does. #1034 ("Platform independent bitcount operation") made the scalar path portable; this call site was missed.The counted value is
(mask & payload)with both operandsbyte, so it is always in[0, 255]and the 32-bit overload returns the same count. I verified that exhaustively rather than by inspection: all 65536(mask, payload)pairs and all 20736 reachable(payload, startBitOffset, endBitOffset)triples — including the realreverse()table — produce identical results under both expressions, zero mismatches.On x86-64 with POPCNT this stays a single
POPCNTinstruction (32-bit operand rather than 64-bit); on ARM64 the JIT emits an AdvSimdCNT/ADDVsequence instead of throwing.The SIMD helpers in this file also call
Popcnt.X64, but their only call site is behindAvx2.IsSupported/Ssse3.IsSupported, both false on ARM64, so they are unreachable there and left alone. For what it's worth this was the only unguarded x86 intrinsic reachable on ARM64 anywhere inlibs/— theBmi2uses inGeoHash.cs,Sse42inRespCommandHashLookup.cs(which already has anArm.Crc32.Arm64branch) andSse.Prefetch0inTsavorite.csare all properly guarded.Relationship to #1994
@vazois — #1994 fixed the mask arithmetic in
BitIndexCount(byte*, long, long). This touches the final popcount in theBitIndexCount(byte, int, int)overload that code calls into; on ARM64 it throws before the corrected mask is ever counted. With both in, theBitmapBitCountBitBoundaryUnderflowTestyou added in #1994 passes on ARM64 too.Testing
No new test —
GarnetBitmapTestsalready covers this. On ARM64,BitmapBitCountSimpleTest,BitmapBitCountBitBoundaryUnderflowTestandBitmapBitCountLongBitOffsetParsingTestfail without this change and pass with it. (#1034 likewise shipped without a new test.)GarnetBitmapTests355/358 → 358/3580 -1 BIT→ 1,16 23 BIT→ 1,0 0 BIT→ 0,0 19 BIT→ 0,20 20 BIT→ 1, session stays alivenet8.0builds clean (0 warnings,TreatWarningsAsErrorsis repo-wide); its tests were not run — I only have the .NET 10 runtime here. Tests were run on net10.0.dotnet format --verify-no-changescleanThis can't regress-test in CI: every test matrix is
ubuntu-latest/windows-latest, both x64, so those three tests pass vacuously there with or without the fix. If you'd want coverage, anubuntu-24.04-armleg on the standalone test job would catch this class of bug — happy to send that as a separate PR rather than widen this one.release/v1has the identical line; happy to send the backport alongside #1998 if you want it.Unrelated and pre-existing, mentioning it only because I noticed it while checking the guards:
__simd_popcX128is entered onSsse3.IsSupportedalone but its tail callsPopcnt.X64, and SSSE3 (Core 2, 2006) predates POPCNT (Nehalem, 2008) — so an SSSE3-without-POPCNT x64 CPU would hit the same exception there for payloads ≥ 128 bytes. Happy to file or fix that separately.