Zero-copy ByteBuffer-backed vectors, no float[] materialization

[eolivelli]

The original problem

I'm using jvector from HerdDB (https://github.com/diennea/herddb). HerdDB already holds its
vectors as ByteBuffer (or byte[]) in memory and on disk — it's how the storage layer
naturally represents fixed-width columnar data. But to feed a vector to jvector today you
have to go through float[]:

ByteBuffer source = herdDBVector;              // native layout, already little-endian
float[] tmp = new float[dim];                  // ALLOCATION — dim × 4 bytes
source.asFloatBuffer().get(tmp);               // COPY
VectorFloat<?> v = vts.createFloatVector(tmp);
builder.addGraphNode(ordinal, v);

For a per-row insertion or a per-query vector, that's a wasted allocation + copy every
single time. Multiply by millions of rows or thousands of queries and it becomes a real
pressure point: GC churn, dirty cache lines, extra memory bandwidth.

The ask: let HerdDB (and any integrator in the same position) feed vectors to jvector as
ByteBuffer directly — no float[] materialization on the hot path — while preserving
SIMD everywhere jvector already uses it.

Why this fits jvector cleanly

jvector's public API is already VectorFloat<?>-based, not float[]-based.
GraphIndexBuilder.addGraphNode, GraphSearcher.search, RandomAccessVectorValues.getVector,
VectorSimilarityFunction.compare, and every VectorUtil distance kernel already speak
VectorFloat<?>. The work is therefore not a rewrite of the API — it's:

1. Adding a zero-copy ByteBuffer-backed VectorFloat<?> implementation.
2. Teaching the SIMD backends to operate on it without casting to ArrayVectorFloat.
3. Adding ByteBuffer-accepting factories, a ByteBuffer-backed RAVV, and ByteBuffer query
   overloads on the high-level API.
4. Fixing internal leaks that would have forced materialization anyway
   (MMapRandomAccessVectorValues used to allocate float[dim] per getVector; PQ training's
   ProductQuantization.getSubVector allocated a float[dim/M] per training-vector × subspace).

Baseline: JDK 22+

HerdDB runs on JDK 25+, so this fork baselines on JDK 22. That's where
java.lang.foreign.MemorySegment became stable — the API we need to make Panama SIMD work on
BufferVectorFloat views — and it matches jvector-native's existing target. CI now tests
exactly JDK 22 (and JDK 24 for AVX-512); the old jdk11 / jdk20 Maven profiles are removed.
Version is bumped to 4.0.0-rc.9-herddb-SNAPSHOT.

Changes

Index build + search path (the original ByteBuffer zero-copy work)

• BufferVectorFloat (new, jvector-base) — VectorFloat<ByteBuffer> view over a
  caller-owned buffer. Slices once at construction via ByteBuffer.slice(int, int) so
  subsequent element access and SIMD dispatch are allocation-free, and caller
  position()/limit() mutation doesn't disturb the view.
• ByteBufferRandomAccessVectorValues (new, jvector-base) — bulk input RAVV over a single
  concatenated ByteBuffer of N × dimension × 4 bytes.
• VectorTypeSupport.wrapFloatVector(ByteBuffer[, floatOffset, floatLength]) — typed
  zero-copy factory; MemorySegmentVectorProvider overrides it to return a MemorySegment-backed
  view directly so the native SIMD path stays zero-copy.
• MemorySegmentVectorFloat.wrap(ByteBuffer) — zero-copy static factory, companion to the
  legacy copying ctor. Also fixes a latent bug in MemorySegmentVectorFloat.get(int) that threw
  on off-heap segments (now falls back to segment.getAtIndex when heapBase() is empty).
• PanamaVectorUtilSupport — the four protected fromVectorFloat/intoVectorFloat
  helpers gain polymorphic dispatch: BufferVectorFloat → FloatVector.fromMemorySegment( MemorySegment.ofBuffer(bb), byteOffset, order). Full SIMD, no float[] materialization.
• NativeVectorUtilSupport — falls through to super for non-MemorySegment vectors, so
  BufferVectorFloat works under the native backend too.
• DefaultVectorUtilSupport — scalar kernels made polymorphic: if both operands are
  ArrayVectorFloat, the existing unrolled float[] fast path runs; otherwise the generic
  .get(i) loop.
• GraphSearcher.search(ByteBuffer, …) and GraphIndexBuilder.addGraphNode(int, ByteBuffer)
  overloads — thin wrappers that call wrapFloatVector internally.
• MMapRandomAccessVectorValues — rewritten to delegate to ByteBufferRandomAccessVectorValues
  over a MappedByteBuffer. Drops the per-call float[dimension] scratch.

PQ codebook training (follow-up)

Flamegraphs of HerdDB's indexing service showed float[] allocations in
ProductQuantization.getSubVector — one fresh float[dim/M] per training-vector × subspace
during codebook construction. For 100k training vectors × M=8 that's ~800k small allocations.

• VectorFloat.subview(int floatOffset, int floatLength) — new default method on the
  interface; fallback materializes, concrete types override for zero-copy.
• ArraySliceVectorFloat (new, jvector-base) — VectorFloat<float[]> that references a
  root float[] at arrayOffset with its own logical length. Companion to
  ArraySliceByteSequence.
• Zero-copy subview overrides on ArrayVectorFloat (→ ArraySliceVectorFloat),
  BufferVectorFloat (→ another BufferVectorFloat over the same ByteBuffer), and
  MemorySegmentVectorFloat (→ segment.asSlice(...)).
• Panama SIMD dispatch recognizes ArraySliceVectorFloat:
  FloatVector.fromArray(SPEC, asv.get(), asv.arrayOffset() + offset) — SIMD preserved.
• ArrayVectorFloat.copyFrom fast-pathed for ArraySliceVectorFloat source.
• ProductQuantization.getSubVector rewritten to vector.subview(offset, length) — one-liner.

Measured on dim=64, M=8, K=16, N=2000 training vectors:
37 B allocated per training vector (down from ~448 B when subvectors were materialized).
KMeans centroid storage and distance arrays — the algorithmic remainder — are unchanged.

Verification

All tests green locally, on all relevant test targets:

• mvn test -pl jvector-tests (SIMD profile): 225 tests, 0 failures, 2 skipped
• mvn test -pl jvector-native: 6 tests green (incl. new MemorySegmentVectorFloatWrapTest)
• mvn test -pl jvector-examples: 109 tests green (incl. new MMapRandomAccessVectorValuesTest)
• mvn -B verify (full reactor): green

New test classes:

• BufferVectorFloatTest — element access, endianness, zero-copy on direct buffers,
  position/limit independence, slice correctness (18 cases)
• VectorTypeSupportByteBufferTest — typed factory across active + Default provider (6)
• ByteBufferRandomAccessVectorValuesTest — parity with ListRAVV, concurrent
  threadLocalSupplier correctness (6)
• BuildFromByteBufferEquivalenceTest — builds the same graph from float[] vs ByteBuffer,
  asserts structural + search equivalence across EUCLIDEAN / DOT_PRODUCT / COSINE (4)
• TestSearchWithByteBufferQuery — ByteBuffer overloads produce same results as
  VectorFloat<?> overloads (2)
• MMapRandomAccessVectorValuesTest — rewritten mmap RAVV round-trip (1)
• MemorySegmentVectorFloatWrapTest — wrap vs legacy copying ctor, endianness, alignment (4)
• SearchAllocationProfileTest — per-query allocation comparison float[] vs ByteBuffer (1)
• VectorFloatSubviewTest — subview aliases source, nested subview, SIMD distance/dot/cosine
  equivalence with materialized copies (7)
• PQTrainingAllocationTest — getSubVector returns a live view; PQ training
  per-vector allocation bounded (2)

Existing tests pass — notably TestProductQuantization (9 cases) confirms the view-based
subvector extraction produces codebooks identical to the prior materialization path.

HerdDB integration, before and after

Before:

ByteBuffer source = herdDBVector;
float[] tmp = new float[dim];
source.asFloatBuffer().get(tmp);
VectorFloat<?> v = vts.createFloatVector(tmp);
builder.addGraphNode(ordinal, v);

After:

ByteBuffer source = herdDBVector;
builder.addGraphNode(ordinal, source);            // zero-copy wrap inside
// — or for bulk build —
var ravv = new ByteBufferRandomAccessVectorValues(allVectors, N, dim);
builder.build();
// — for queries —
GraphSearcher.search(source, topK, ravv, vsf, graph, Bits.ALL);

The HerdDB-side upgrade instructions are tracked in
eolivelli/herddb#174.

🤖 Generated with Claude Code