feat: promote RankQuantFastscan to public API + .ovfs persistence

[Fieldnote-Echo]

Stacked on #230 (the .ov* format). Promotes the FastScan b=2 path from #[doc(hidden)] to a documented public type and gives it persistence in the OV* convention #230 establishes.

What

• Un-hide RankQuantFastscan — it was #[doc(hidden)] on both the crate-root re-export and the struct definition. Now a stable, documented type, framed as a specialized b=2 minimum-latency scan path (block-32 nibble-LUT; AVX-512 → AVX2 → scalar) — not the headline retrieval surface (that's RankQuant / Bitmap / two-stage). Added a discoverable crate-root mention.
• Persistence (net-new) — RankQuantFastscan::write / load via a new .ovfs format (magic OVFS): a 13-byte header (OVFS | version | dim | n_vectors) + the opaque block-32 packed payload. rank_io gains write_fastscan / load_fastscan / fastscan_payload_bytes (pub(crate)). The loader validates magic, dim % 4 == 0, n_vectors, and exact payload length (no trailing bytes), with the MAX_PAYLOAD guard enforced before File::create. OVFS is new in the ordvec format — no legacy TV* counterpart (FastScan never had persistence under turbovec).
• Tests — write/load round-trip scans byte-identically; empty-index round-trip; OVFS-magic-on-write; rejects wrong magic / trailing bytes / dim % 4 != 0.
• Fuzz — new load_fastscan libFuzzer target (parity with the four existing loader targets).

Not in scope

• Kernels are unchanged — FastScan was already AVX-512/AVX2/scalar-optimized; this is promotion + persistence only.
• Probe support for .ovfs is deferred: probe_index_metadata returns IndexKind, and adding a Fastscan variant to the non-#[non_exhaustive] IndexKind enum would be a breaking change → tracked for the 0.8.0 API re-arch (0.8.0 staging: public API re-architecture & streamlining (breaking) #232). A .ovfs file currently probes as 'unknown magic'.

Validation

cargo fmt --all --check; clippy --all-targets --all-features -D warnings; RUSTDOCFLAGS=-D warnings cargo doc; full test suites (experimental + default, 18 fastscan tests incl. 6 new persistence); cargo +nightly fuzz build load_fastscan.

Follow-up: PR3 docs refresh (surfaces FastScan + .ovfs across README / bindings / C-ABI / Go).

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Context used

✅ Tickets: 🎫 0.8.0 staging: public API re-architecture & streamlining (breaking)

1. Write path can panic ✓ Resolved 🐞 Bug ☼ Reliability

Description

rank_io::write_fastscan uses assert_eq!(packed_fs.len(), payload_bytes), so the new public
RankQuantFastscan::write can panic (abort the caller) if an internal invariant is ever violated,
even though it returns io::Result<()>. This is a robustness footgun for the newly-public
persistence API (panic instead of an error).

Code

src/rank_io.rs[R895-903]

+    let payload_bytes = fastscan_payload_bytes(dim, n_vectors)?;
+    check_payload_bytes(payload_bytes)?;
+    assert_eq!(packed_fs.len(), payload_bytes);
+    let mut f = BufWriter::new(File::create(path)?);
+    f.write_all(OVFS_MAGIC)?;
+    f.write_all(&[VERSION])?;
+    f.write_all(&(dim as u32).to_le_bytes())?;
+    f.write_all(&(n_vectors as u32).to_le_bytes())?;
+    f.write_all(packed_fs)?;

Evidence

The public API RankQuantFastscan::write forwards directly to rank_io::write_fastscan, which
contains an unconditional assert_eq! on the payload length; if triggered, it panics rather than
returning an io::Error.

src/fastscan.rs[645-654]
src/rank_io.rs[886-905]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`write_fastscan()` currently uses `assert_eq!(packed_fs.len(), payload_bytes)`. Because `RankQuantFastscan::write()` is public and returns `io::Result<()>`, a panic here is surprising and turns what should be an error return into a process crash.

### Issue Context
Even if today the invariant is established by construction, keeping the write path panic-free makes the new public persistence surface more robust against future refactors/in-crate callers that might accidentally violate the invariant.

### Fix Focus Areas
- src/rank_io.rs[895-904]

### Suggested change
Replace the `assert_eq!` with a checked branch that returns `io::ErrorKind::InvalidInput`/`InvalidData` (consistent with other loader/writer validation), e.g.:
- if `packed_fs.len() != payload_bytes` => `return Err(invalid("OVFS packed payload length mismatch"))`

(Optionally keep a `debug_assert_eq!` as an internal development aid.)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[qodo-code-review]

PR Summary by Qodo

Promote RankQuantFastscan to public API and add .ovfs persistence
✨ Enhancement 🧪 Tests ⚙️ Configuration changes 🕐 40+ Minutes

Walkthroughs

Description

• Make RankQuantFastscan a documented, stable public type for b=2 low-latency scans
• Add .ovfs (OVFS) write/load persistence with strict header and payload validation
• Extend test + fuzz coverage for OVFS round-trips and malformed-file rejection

Diagram

graph TD
  A["Public API: RankQuantFastscan"] --> B["rank_io fastscan I/O"] --> C[(".ovfs file (OVFS)")]
  D["Tests: fastscan persistence"] --> A
  E["Fuzz target: load_fastscan"] --> A

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Embed FastScan payload into an existing OV* container (e.g., OVRQ)

• ➕ Fewer file extensions/magics to support
• ➕ Could reuse existing probing/metadata flows if already implemented for OV*
• ➖ More complex container format (optional sections, versioning, offsets)
• ➖ Higher risk of breaking compatibility expectations for existing OV* formats
• ➖ Harder to guarantee exact payload-length invariants without a dedicated spec

2. Add `.ovfs` probing support now (IndexKind::Fastscan)

• ➕ Better UX: probe can identify .ovfs instead of “unknown magic”
• ➕ Allows tooling to route files correctly without opening/parsing fully
• ➖ Requires changing a non-#[non_exhaustive] enum; breaking API change
• ➖ Pushes API churn earlier than planned (conflicts with stated 0.8.0 re-arch)

Recommendation: Keep the dedicated .ovfs/OVFS format as implemented: it cleanly isolates a specialized layout, enables strict length validation, and avoids entangling legacy formats. Deferring probe support is reasonable given the non-exhaustive enum breaking-change constraint; revisit during the planned API re-architecture.

File Changes

Enhancement (2)

fastscan.rs Promote RankQuantFastscan docs and add write/load API +34/-8

Promote RankQuantFastscan docs and add write/load API

• Removes doc-hiding and reframes 'RankQuantFastscan' as a stable but specialized b=2 scan path. Adds 'write'/'load' methods that persist and restore the opaque block-32 packed payload via the '.ovfs' format.

src/fastscan.rs

---

rank_io.rs Add OVFS (.ovfs) format support for FastScan persistence +81/-1

Add OVFS (.ovfs) format support for FastScan persistence

• Introduces OVFS magic and implements 'write_fastscan'/'load_fastscan' with a fixed 13-byte header and strict payload-length validation (including 'dim % 4 == 0' and no trailing bytes). Adds payload sizing logic with overflow checks and enforces MAX_PAYLOAD before file creation to avoid truncating existing files.

src/rank_io.rs

---

Tests (2)

load_fastscan.rs Add fuzz target that exercises RankQuantFastscan::load +27/-0

Add fuzz target that exercises RankQuantFastscan::load

• Introduces a libFuzzer harness that writes arbitrary bytes to a scratch file and calls the public 'ordvec::RankQuantFastscan::load' entry point. The goal is to ensure malformed inputs produce Ok/Err without panics/aborts/OOB reads.

fuzz/fuzz_targets/load_fastscan.rs

---

fastscan.rs Add OVFS persistence round-trip and validation tests +118/-0

Add OVFS persistence round-trip and validation tests

• Adds round-trip tests verifying identical search results after write/load and correct behavior for empty indexes. Adds negative tests rejecting wrong magic, trailing bytes, and 'dim' values not divisible by 4.

tests/index/fastscan.rs

---

Documentation (1)

lib.rs Document and publicly re-export RankQuantFastscan +12/-5

Document and publicly re-export RankQuantFastscan

• Adds crate-level documentation positioning FastScan as a b=2 latency-optimized companion to RankQuant/Bitmap. Removes the crate-root '#[doc(hidden)]' re-export so the type becomes discoverable in the public API.

src/lib.rs

---

Other (1)

Cargo.toml Register new libFuzzer target for FastScan loader +7/-0

Register new libFuzzer target for FastScan loader

• Adds a new fuzz binary target 'load_fastscan' pointing at 'fuzz_targets/load_fastscan.rs' so the OVFS loader path is fuzzed like the other index loaders.

fuzz/Cargo.toml

---

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[gemini-code-assist]

Code Review

This pull request introduces persistence support for the RankQuantFastscan index, enabling saving and loading using the new .ovfs file format. It includes the implementation of the write and load methods, a new fuzz target, and comprehensive unit tests. The review feedback suggests validating dim and n_vectors prior to writing to prevent silent truncation and potential file corruption when casting n_vectors to a u32.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[Fieldnote-Echo]

Remediated in 1bbdb8f (+ merged main = #237/#240):

• gemini + qodo (write_fastscan): write_fastscan (backing the now-public RankQuantFastscan persistence) now validates dim (range + multiple-of-4), n_vectors, payload size, and packed-buffer length before File::create — returning a clean io::Error instead of (a) silently truncating via as u32 or (b) panicking via assert_eq! in a Result fn. A rejected write never creates/truncates a file. Added regression test write_fastscan_validates_and_never_panics (valid round-trip + Err-not-panic on bad dim/length).
• CI (ffi header drift): regenerated ordvec-ffi/include/ordvec.h with cbindgen 0.29.3 (the committed header had drifted from the loader doc-comment).

Verified: cbindgen --verify ok, fmt + clippy clean, full default suite green.