[WIP] Robin Hood Hashing

[aterenin]

This PR implements Robin Hood hashing, through introducing a new probing scheme, as per #817. An initial-pass implementation for get and insert, and tests, are done.

The implementation works by adding if constexpr logic throughout to handle Robin Hood specifics as they come up:

• The big question: is this the right structure? As an alternative, I could refactor the code to write separate insert methods for Robin Hood vs. others, and dispatch according to table type.

Some more details about this question, with an example, are given at the very end of #817.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[PointKernel]

/ok to test 1f3d64e

[PointKernel]

@aterenin Just wanted to let you know we haven't forgotten about this work! @sleeepyjack and I are discussing the new design and will align on a concrete proposal before moving forward to minimize unnecessary iterations. We'll keep you posted.

[aterenin]

Sounds good!

The current WIP implements everything needed, along with tests. I've carefully gone over all the core logic, and I think is reasonably close to ready-to-review.

The main remaining item I was planning to think about more is whether there is a more maintenance-friendly way to organize and structure the code, which is what the "big question" in the PR centers on. I also have a list of a bunch of subtleties worth mentioning, but these should wait until a review.

If discussion would help, I'd be happy to jump on a Zoom call - just let me know!

[PointKernel]

We've had a chance to evaluate the proposal and would be interested in discussing it further over a video call.

@aterenin Could you send your availability and preferred contact details to yunsongw@nvidia.com? Wednesday through Friday next week would work well on our end. Once we have a few time slots, we can coordinate and send out a meeting invite.

[aterenin]

Email headed your way!

[aterenin]

@PointKernel / @sleeepyjack Would you mind giving an OK to test on 628889a?

(Additional note: this is not quite ready for review yet - stay tuned for that)

[PointKernel]

/ok to test 628889a

[aterenin]

@PointKernel And can we try again with another OK to test?

(Sorry for long delay - just got back from conference travel.)

[sleeepyjack]

/ok to test 40187dd

[aterenin]

I finally have access to my GPU box again, and have ran some initial benchmarks. The goal here was to successfully reproduce the results from the GPURHH repo in a manner fully contained inside cuCollections, and I believe this has been successful.

Setup:

• 4090
• static_map_insert_unique_occupancy
• 1GB hash table
• Occupancy: 0.75, 0.85, 0.95, 0.99
• NumInputs: calculated from occupancy to ensure table is the correct target size
• MatchingRate: calculated to be approximately-equivalent to GPURHH's original test, see below.

Note that the resulting MatchingRate is set relatively small. This needs some explanation: the GPURHH benchmark does independent uniform insertion and retrieval. In cuCo, a different setup is used: first, the system first generates a list of N keys by shuffling 0,..,N-1 into a random order. The same stream is reused for insertion and retrieval, except that, with probability 1 - MatchingRate, for retrieval each key is replaced with a new one that re-drawn from N,..,max_value. I have set MatchingRate to ensure the number of collisions is approximately the same as GPURHH in expectation. The effects here are big enough that I think the fine details here are unlikely to affects results much.

The exact numbers are 0.023, 0.027, 0.03, 0.031 depending on load factor. As a result, for small tables, most keys will not be present, which is a pretty favorable evaluation from Robin Hood's perspective - if I had thought more carefully about this at blog post time, I might have picked something a bit more balanced. Though, there are certainly use cases out there where this will likely be realistic.

This makes me think I should benchmark some more instances to get a broader view - I'd be curious what other benchmarking scenarios you'd potentially like to see? (@PointKernel / @sleeepyjack)

Otherwise, I think now would be a good time to speak again on Zoom. The current implementation should mostly match what was discussed, with the main caveat that the hard-wire required quite a bit of stuff to be commented out, because either they don't make sense for RHH (different atomic pattern, etc.) or for related reasons. A diff against the non-Robin-Hood open_addressing_ref_impl should be pretty informative of the state of the code. On my end, I'll probably take a minor clean-up pass before meeting to fully remind myself what's there, but do not anticipate I will change much if anything.