Update DiskANN FFI

[kevin-montrose]

Making what are (hopefully) the final big FFI changes before stabilization.

1. Add nint logCallback to create_index
   • logCallback is:
     • void LogCallbackUnmanaged(ulong context, nint logMessage, nuint logMessageLength)
     • context low bits are used to enrich log line on the Garnet side
     • logMessage is utf8 encoded text
     • logMessageLength is the byte count of logMessage
2. Add valueLengthHint to readCallback
   • Its OK for this to be wrong, but it will impact performance
   • Better to overestimate than underestimate
   • This replaces the ActiveReadGeometry estimates on the Garnet side - allowing DiskANN to own record size information
3. Use continue_search(ulong context, nint index, nint continuation, nint output_ids, nuint output_ids_len, nint output_distances, nuint output_distances_len, nint new_continuation)
   • For when search_xxx(...) function can't fit all of the results into the provided output_ids buffer
   • DiskANN can assume we'll call this once per non-null continuation
   • Invoked until we get a null continuation - should be null if an error is encountered
4. Add byte random_members(ulong context, nint index, uint count, nint output_ids, nuint output_ids_len)
   • Used to implement VRANDMEMBER
   • Gets count random (external) ids and places them in output_ids, length prefixed
   • returns 1 on success, and anything else on failure (will be logged)
5. Add int search_neighbors(ulong context, nint index, nint id_data, nuint id_len, nint output_ids, nuint output_ids_len, nint output_distances, nuint output_distances_len, nint continuation)
   • This implements VLINKS which is basically "give me the neighbors of an element + distances to each"
   • If buffer is too small, continuation will be set and search can continue with continue_search(...)
6. Change filter_callback to accept data in place rather than the parsed internal id
7. Add result return to backfill_quant_vectors
   • Returning 1 is success, anything else is failure (and is logged)
8. Requires all keys passed to callbacks (from DiskANN to Garnet) must be multiples of 4-bytes in length
9. Asserts for 8 & that values passed to DiskANN dataCallbacks are 4-byte aligned

Code is naturally mostly untested since the DiskANN side isn't ready yet, but everything has a sketched out implementation so the basics can be validated and proper tests written once we have binary to test with.