Skip to content

Vectorize Forward/Reflected CRC-32 and Forward CRC-64#124832

Open
bartonjs wants to merge 8 commits intodotnet:mainfrom
bartonjs:crc_vec3
Open

Vectorize Forward/Reflected CRC-32 and Forward CRC-64#124832
bartonjs wants to merge 8 commits intodotnet:mainfrom
bartonjs:crc_vec3

Conversation

@bartonjs
Copy link
Member

This provides a vector implementation of CRC-32, and forward-only for CRC-64.

Reflected CRC-64 theoretically requires UInt128 to hold the 65-bit constants. That's not "it can't be done", but it's complicating the algorithm, so it's being pushed to a followup.

Intel's isa-l repository does have a reflected CRC-64 that only uses the xmm registers, and seems to be using only ulong for the constants (e.g. rk20: DQ 0x1b1ab00000000001), so it does seem to be possible, but the tooling isn't coming to that conclusion automatically.

Contributes to #124576.

@bartonjs bartonjs added this to the 11.0.0 milestone Feb 24, 2026
@bartonjs bartonjs self-assigned this Feb 24, 2026
Copilot AI review requested due to automatic review settings February 24, 2026 23:08
@dotnet-policy-service
Copy link
Contributor

Tagging subscribers to this area: @dotnet/area-system-io-hashing, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request implements vectorized CRC computation for CRC-32 (both forward and reflected variants) and CRC-64 (forward only). The changes refactor the existing vectorization code from the Crc32 and Crc64 classes into the respective ParameterSet classes, enabling vectorization support for custom CRC polynomials.

Changes:

  • Introduces abstract base classes (ForwardCrc32, ReflectedCrc32, ForwardCrc64) that handle vectorization using a partial method pattern
  • Adds CrcPolynomialHelper class with a UInt640 struct for computing folding and Barrett constants for arbitrary polynomials
  • Migrates vectorization logic from Crc32/Crc64 static classes to parameter set implementations

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated no comments.

Show a summary per file
File Description
CrcPolynomialHelper.cs New helper class for computing CRC folding/Barrett constants using 640-bit arithmetic
Crc32ParameterSet.cs Adds ReflectedCrc32 and ForwardCrc32 abstract base classes with partial method hooks for vectorization
Crc32ParameterSet.Vectorized.cs Implements vectorized CRC-32 computation for both reflected and forward variants
Crc32ParameterSet.WellKnown.cs Updates IEEE 802.3 and CRC-32C to use new base classes with ARM intrinsics support
Crc32ParameterSet.Table.cs Updates table-based implementations to inherit from new base classes
Crc32.cs Removes partial modifier, delegates to parameter set's Update method
Crc32.Vectorized.cs Deleted - logic moved to ParameterSet
Crc32.Table.cs Deleted - table moved to ParameterSet.WellKnown
Crc32.Arm.cs Deleted - ARM intrinsics moved to ParameterSet.WellKnown
Crc64ParameterSet.cs Adds ForwardCrc64 abstract base class with vectorization support
Crc64ParameterSet.Vectorized.cs Implements vectorized forward CRC-64 computation
Crc64ParameterSet.WellKnown.cs Updates ECMA-182 to use ForwardCrc64 base class
Crc64ParameterSet.Table.cs Updates table-based forward implementation to inherit from ForwardCrc64
Crc64.cs Removes partial modifier, delegates to parameter set's Update method
Crc64.Vectorized.cs Deleted - logic moved to ParameterSet
Crc64.Table.cs Deleted - table moved to ParameterSet.WellKnown
System.IO.Hashing.csproj Updates file references to reflect deletions and additions
Crc32Tests_ParameterSet_Custom.cs Adds test driver for forward CRC-32 with non-standard polynomial

@bartonjs
Copy link
Member Author

Perf-wise, it's basically unchanged from the current vectorized implementation for CRC-32/IEEE and CRC-64/ECMA.


BenchmarkDotNet v0.15.8, Windows 11 (10.0.26200.7840/25H2/2025Update/HudsonValley2)
Intel Core i7-8700 CPU 3.20GHz (Max: 3.19GHz) (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET SDK 11.0.100-preview.1.26104.118
  [Host]     : .NET 10.0.3 (10.0.3, 10.0.326.7603), X64 RyuJIT x86-64-v3
  Job-QMWUGV : .NET 11.0.0 (11.0.0-dev, 42.42.42.42424), X64 RyuJIT x86-64-v3
  .NET 10.0  : .NET 10.0.3 (10.0.3, 10.0.326.7603), X64 RyuJIT x86-64-v3

Runtime=.NET 10.0  

Method Job Toolchain size Mean Error StdDev Allocated
Crc32_Prebuilt Job-QMWUGV CoreRun 0 2.5973 ns 0.0234 ns 0.0218 ns -
Crc32_Prebuilt .NET 10.0 Default 0 2.3792 ns 0.0178 ns 0.0139 ns -
Crc64_Prebuilt Job-QMWUGV CoreRun 0 0.7268 ns 0.0303 ns 0.0283 ns -
Crc64_Prebuilt .NET 10.0 Default 0 0.7403 ns 0.0070 ns 0.0062 ns -
Crc32_Prebuilt Job-QMWUGV CoreRun 10 9.6688 ns 0.0179 ns 0.0149 ns -
Crc32_Prebuilt .NET 10.0 Default 10 8.8438 ns 0.0230 ns 0.0204 ns -
Crc64_Prebuilt Job-QMWUGV CoreRun 10 8.3621 ns 0.0231 ns 0.0205 ns -
Crc64_Prebuilt .NET 10.0 Default 10 8.3531 ns 0.0239 ns 0.0223 ns -
Crc32_Prebuilt Job-QMWUGV CoreRun 100 13.3525 ns 0.0356 ns 0.0333 ns -
Crc32_Prebuilt .NET 10.0 Default 100 14.3781 ns 0.0298 ns 0.0279 ns -
Crc64_Prebuilt Job-QMWUGV CoreRun 100 14.8436 ns 0.0454 ns 0.0403 ns -
Crc64_Prebuilt .NET 10.0 Default 100 14.7429 ns 0.0191 ns 0.0149 ns -
Crc32_Prebuilt Job-QMWUGV CoreRun 1000 56.0132 ns 0.0887 ns 0.0740 ns -
Crc32_Prebuilt .NET 10.0 Default 1000 55.7654 ns 0.0795 ns 0.0744 ns -
Crc64_Prebuilt Job-QMWUGV CoreRun 1000 64.1417 ns 0.0877 ns 0.0778 ns -
Crc64_Prebuilt .NET 10.0 Default 1000 63.6507 ns 0.1179 ns 0.1045 ns -
Crc32_Prebuilt Job-QMWUGV CoreRun 10000 347.9694 ns 0.5755 ns 0.5383 ns -
Crc32_Prebuilt .NET 10.0 Default 10000 348.6138 ns 0.6414 ns 0.5999 ns -
Crc64_Prebuilt Job-QMWUGV CoreRun 10000 444.4825 ns 0.4317 ns 0.4038 ns -
Crc64_Prebuilt .NET 10.0 Default 10000 443.5430 ns 0.6813 ns 0.6373 ns -
Crc32_Prebuilt Job-QMWUGV CoreRun 100000 3,444.3479 ns 5.1116 ns 4.2684 ns -
Crc32_Prebuilt .NET 10.0 Default 100000 3,444.2575 ns 3.7278 ns 3.1128 ns -
Crc64_Prebuilt Job-QMWUGV CoreRun 100000 4,402.8019 ns 8.5030 ns 7.9537 ns -
Crc64_Prebuilt .NET 10.0 Default 100000 4,401.7223 ns 7.0760 ns 6.2727 ns -
Crc32_Prebuilt Job-QMWUGV CoreRun 1000000 36,523.5935 ns 296.2984 ns 277.1577 ns -
Crc32_Prebuilt .NET 10.0 Default 1000000 36,608.2642 ns 236.4533 ns 221.1786 ns -
Crc64_Prebuilt Job-QMWUGV CoreRun 1000000 44,802.2878 ns 72.3593 ns 67.6849 ns -
Crc64_Prebuilt .NET 10.0 Default 1000000 44,824.7990 ns 63.7654 ns 56.5264 ns -
Crc32_Prebuilt Job-QMWUGV CoreRun 1048576 38,287.0065 ns 210.2360 ns 186.3687 ns -
Crc32_Prebuilt .NET 10.0 Default 1048576 38,268.0418 ns 279.9785 ns 261.8921 ns -
Crc64_Prebuilt Job-QMWUGV CoreRun 1048576 47,016.1375 ns 106.4478 ns 99.5714 ns -
Crc64_Prebuilt .NET 10.0 Default 1048576 47,013.3135 ns 74.0468 ns 65.6406 ns -

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants