Research writeup: ordvec ordinal-routing analysis — characterized mechanism + clean negative (not a crate change)

[toadkicker]

Exploratory research into ordvec's density behavior and whether prime / spectral /
oblivious structure can improve training-free routing. All work lives in
experiments/ordinal-routing-research/ (probes *.rs next to the findings *.md
that produced them) — no changes to the ordvec crate or its public API. The
directory is package.excluded: it ships with the source tree but not the published
crate. Self-reviewed by three adversarial agents plus the PR bots; findings are tiered
by what survived scrutiny.

Bottom line — a characterized mechanism and a clean negative

Research, not a feature. The prime / spectral / permutation / oblivious-structure
ideas for dense-region retrieval do not beat the boring baseline. The honest verdict:
spend the bits — use b=4 raw rank codes. Every constructed alternative was
pre-registered with pass/fail thresholds before running, and every one came back
negative or inert.

Note on this description. An earlier version of this PR led with
"density collapse is recoverable from permutation order" and "nomic intrinsic
dim ≈ 13." Both were later overturned by the experiments below and are corrected
here rather than quietly edited away — the matched-bytes bake-off showed the tau
signal is inert (b=4 wins), and ID ≈ 13 turned out to be a corpus property
(the repo corpus), not an encoder constant (fiqa reads ~24 on the same encoder).
The integrity record is in ADVERSARIAL_REVIEW.md.

SOUND — proven or real-data confirmed

• The decisive negative (the verdict). The b=2 density-collapse tau signal is
  real but does not beat b=4 at matched bytes: real embeddings b4 R@10 0.942 vs
  b2 0.898 vs tau-rerank 0.597 (fp32-rerank 1.000). The b=2 candidate pool contains
  every true neighbour, but the ~0.04 tau gap is too weak to order them — it scores
  below b=2's own ordering. Real-but-inert. (tau_rerank_bakeoff_results.md)
• Mechanism. RankQuant b=2 density collapse = Hamming-near codes the scorer can't
  separate; among those lookalikes true neighbours have lower intra-code Kendall-tau
  (gap ≈ 0.04, CI > 0). Small but real. (density_collapse_results.md)
• The oblivious-directions arc (round 2). Data-oblivious low-discrepancy
  directions (golden-angle / Sobol / Kronecker) do not beat iid-random for
  training-free routing — across 5 encoders (nomic, bge-m3, bge-large,
  snowflake-arctic-v2, harrier-oss) at real intrinsic dim 18–24. CLASS-DEAD,
  pre-registered, replicated (the one mid-ladder flicker failed to replicate). The one
  robust positive: data-aligned (PCA) directions lead at higher ID — the lever is
  data-alignment, which training-free forbids. (oblivious_directions_results.md)
• CRT vernier seam theorem — exhaustive finite proof: lcm spacing, one
  coincidence/period, capped density ∏ min(2t+1,m_i)/m_i. Lean 4 formalization is
  sorry-free in the companion repo
  ordvec-formalization#17.
  (crt_seam_oracle_results.md)
• Shard recall — random phase offsets add nothing vs aligned grids (post
  RNG-desync fix). (shard_recall_results.md)

Deployment-robustness sub-arc — RESOLVED (negative across every synthetic pathology)

Does training-free b=4 routing degrade on messy enterprise-lake geometry? Three
pre-registered fears, all unfounded:

• Phase B — multi-domain "lake" (fiqa+nq+quora = 281,729 docs, + templated-hub
  injection). Union ID is lower (8.2 vs ~24/domain), global centering still works,
  and b=4 R@10 is immune to templated hubs (flat −0.002 through 15% prevalence).
  (oblivious_directions_results.md, Phase B section)
• Path B — chunk-length-mixture lake (same 57.6k fiqa docs embedded at 4 chunk
  lengths {128,256,512,1100} → 230k-doc lake). b=4 raw R@10 immune: 0.823 → 0.825
  (+0.002, CR@100 = 1.0). Bonus: the "chunk length is a third geometry axis" claim
  is real but small and co-axial — R̄ spreads only 0.705→0.723 over an 8.6× length
  range, cone axes ≥0.986 aligned. (length_mixture_lake_results.md)

Mechanism for the robustness: per-vector rank codes discard the magnitude a hub or a
chunk-length tightness-shift moves, and there's no global IDF/frequency term to poison.
Training-free + rank-based is the property that confers the immunity. The only un-run
test left needs real dirty data (OCR / multilingual S3 sludge), uncapturable from
clean embeddings.

THEORY — directionally right, restated honestly

• Routing key is not number-variance-rigid (Thm 2/3, binomial L(1−L/n)); the
  over-broad "quantile optimal over all partitions" claim is retracted as a
  non-sequitur. (rigidity_impossibility_proofs.md)
• Citations verified by direct fetch; a few subagent confabulations caught and
  corrected. (conjecture_citation_audit.md)
• TwoNN ID: chord-metric fix is sound (sphere-validated to ID ~12); OLS-through-origin
  estimator is biased. Real-corpus ID (~18–24) is established in the directions arc,
  not here. (twonn_id_results.md)

WITHDRAWN — see withdrawn/

Number-variance "super-Poisson" finding: its unfold is uncalibrated (a salvage attempt
inverted the result). Kept for the record, not as a claim. The theory above does not
depend on it.

Conjecture verdict

Prime / Li(x) / Sacks-spiral constructions don't help retrieval: they act on the index
(ℕ) and carry no corpus information. The exploitable structure lives on the
permutohedron S_D — the data's own order — which is the density-collapse result, and
that turned out inert against b=4. Closed: research, not a feature.

The discipline that held the line: pre-register pass/fail thresholds in each probe's
source header before running, and require replication across ≥2 corpora (this caught
a seed-1 false positive and a bge-m3 flicker). Reviewer's guide and full tiering:
experiments/ordinal-routing-research/README.md and ADVERSARIAL_REVIEW.md.

🤖 Generated with Claude Code

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (4) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0)

1. Unsafe RankQuant bucketing 🐞 Bug ≡ Correctness

Description

examples/density_collapse.rs computes bucket ids via bucket_w = dim / (1<<bits) then `rank /
bucket_w, which panics when dim < 2^bits` (division by zero) and does not match ordvec’s
rank_to_bucket mapping unless dim happens to divide evenly into 2^bits buckets.

Code

examples/density_collapse.rs[R214-223]

+    let nbuckets = 1usize << cfg.bits;
+    let bucket_w = d / nbuckets;
+    let codes: Vec<Vec<u8>> = (0..cfg.n)
+        .into_par_iter()
+        .map(|i| {
+            let ranks = rank_transform(&corpus[i * d..(i + 1) * d]);
+            ranks
+                .iter()
+                .map(|&r| (r as usize / bucket_w).min(nbuckets - 1) as u8)
+                .collect()

Evidence

The probe’s bucketing uses bucket_w = d / nbuckets and divides by bucket_w, which can become
zero and is a different formula than ordvec’s canonical rank_to_bucket bucketing (`rank *
n_buckets / d`) with explicit domain guards.

examples/density_collapse.rs[211-225]
src/rank.rs[74-110]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`examples/density_collapse.rs` is intended to analyze RankQuant b-bit bucket behavior, but it reimplements bucketing as `rank / (d / 2^bits)`. This has two problems:
- It can panic at runtime when `d / 2^bits == 0`.
- It’s not equivalent to ordvec’s RankQuant bucketing for non-divisible dimensions.

### Issue Context
ordvec already provides a guarded, dimension-correct mapping in `ordvec::rank::rank_to_bucket(rank, d, bits)`.

### Fix Focus Areas
- examples/density_collapse.rs[24-27]
- examples/density_collapse.rs[211-225]

### Suggested fix
1. Change `Cfg.bits` to a `u8` (or validate `cfg.bits <= 7` before use).
2. Replace the custom `nbuckets/bucket_w` logic with:
  - `let bits: u8 = cfg.bits.try_into().expect(...)` (or similar)
  - `rank_to_bucket(r, d, bits)` for each rank.
3. (Optional) If you need `nbuckets` for reporting, compute it as `1usize << bits` after validation.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. R sweep changes projections 🐞 Bug ≡ Correctness

Description

examples/shard_recall.rs seeds its direction/phase RNGs with r (number of projections), so
sweeping r_values changes the entire projection set between R values; this confounds interpreting
recall changes as being caused by “adding more projections”.

Code

examples/shard_recall.rs[R274-283]

+fn build_projs(cfg: &Cfg, arm: Arm, r: usize, base_width: f32) -> Vec<Proj> {
+    let d = cfg.dim;
+    // Bug L fix: directions and phases come from SEPARATE, fixed-seed RNGs that
+    // do not depend on the arm. Every arm therefore sees the IDENTICAL R
+    // projection directions (and identical phase draws where used), so the arms
+    // differ ONLY in width/phase policy — the clean ablation. Previously the
+    // RandomOffset arm consumed an extra value from one shared stream, desyncing
+    // the directions so arms were compared on different projections.
+    let mut dir_rng = ChaCha8Rng::seed_from_u64(SEED ^ 0xD132_0000 ^ (r as u64));
+    let mut phase_rng = ChaCha8Rng::seed_from_u64(SEED ^ 0x9DA5E_000 ^ (r as u64));

Evidence

build_projs incorporates r into RNG seeds, and run_arms iterates for &r in &cfg.r_values, so
each R value produces a different direction set rather than reusing the first projections and adding
more.

examples/shard_recall.rs[274-308]
examples/shard_recall.rs[356-372]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The experiment sweeps `R` but `build_projs` seeds RNGs with `r`, so `R=8` and `R=16` use unrelated direction sets. That makes the R-sweep non-nested and confounds attribution of improvements to increased projection count.

### Issue Context
The bug-L fix correctly de-couples arms from RNG consumption; this issue is specifically about keeping the *same first k projections* when increasing R.

### Fix Focus Areas
- examples/shard_recall.rs[274-308]
- examples/shard_recall.rs[367-372]

### Suggested fix
- Remove `r` from the RNG seeds.
- Seed per-projection index `i` instead, e.g. inside the `(0..r).map(|i| ...)` loop:
 - `let mut dir_rng = ChaCha8Rng::seed_from_u64(hash(SEED, "dir", i));`
 - `let mut phase_rng = ChaCha8Rng::seed_from_u64(hash(SEED, "phase", i));`
 where `hash` is a simple SplitMix64-style mixer.
This preserves:
- identical directions across arms for fixed `i`
- nested direction sets across increasing `R`

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. NPY v2 header OOB 🐞 Bug ☼ Reliability

Description

The new .npy loaders treat any major != 1 as v2 and read bytes[10]/bytes[11] without first
ensuring bytes.len() >= 12, so malformed/truncated files can panic with out-of-bounds and v3+
files will be misparsed.

Code

examples/spectral_probe.rs[R123-130]

+    let bytes = std::fs::read(path).expect("read npy");
+    assert!(bytes.len() >= 10 && &bytes[..6] == b"\x93NUMPY", "not a numpy file");
+    let major = bytes[6];
+    let (hlen, hstart) = if major == 1 {
+        (u16::from_le_bytes([bytes[8], bytes[9]]) as usize, 10)
+    } else {
+        (u32::from_le_bytes([bytes[8], bytes[9], bytes[10], bytes[11]]) as usize, 12)
+    };

Evidence

In spectral_probe the non-v1 branch reads 4 header-length bytes (bytes[8..=11]) but the only
precondition is bytes.len() >= 10, which is insufficient for those indices and will panic on short
inputs.

examples/spectral_probe.rs[122-131]
examples/density_collapse.rs[162-170]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`load_npy_f32` only checks `bytes.len() >= 10`, but if `major != 1` it unconditionally indexes `bytes[10]` and `bytes[11]` to read a u32 header length. This can panic on short/corrupt files and also incorrectly treats unsupported major versions as v2.

### Issue Context
This loader pattern appears in multiple examples. These examples accept user-supplied `.npy` files, so they should fail with clear errors rather than panicking due to OOB indexing.

### Fix Focus Areas
- examples/spectral_probe.rs[122-146]
- examples/shard_recall.rs[79-105]
- examples/twonn_id.rs[65-90]
- examples/density_collapse.rs[162-185]

### Suggested fix
- Explicitly handle only major versions 1 and 2.
- Add length guards:
 - if major==1 require `bytes.len() >= 10`
 - if major==2 require `bytes.len() >= 12`
- Before slicing the header, validate `hstart + hlen <= bytes.len()`.
- Consider returning `Result<...>` with informative messages (even in examples), or at least use `panic!("...")` with version/length details.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

4. TwoNN metric comment mismatch 🐞 Bug ⚙ Maintainability

Description

examples/twonn_id.rs’s top-level comment says the probe uses cosine distance (1−cos), but the
implementation uses chord distance sqrt(2−2cos); this is misleading documentation for readers
adapting the estimator.

Code

examples/twonn_id.rs[R8-10]

+//! Metric: embeddings are L2-normalized and retrieval is ANGULAR, so NN
+//! distances here are cosine distance (1 - cos). Measuring Euclidean ID on
+//! normalized vectors would estimate the ID of the wrong metric.

Evidence

The header comment explicitly mentions cosine distance, while nn_ratios computes chord distance
via sqrt(2 - 2*cos).

examples/twonn_id.rs[8-10]
examples/twonn_id.rs[180-185]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The `twonn_id` example now correctly uses chord distance, but the file header still claims cosine distance is used.

### Issue Context
The code later explains why chord distance is required (locally linear in angle); the header should match.

### Fix Focus Areas
- examples/twonn_id.rs[8-10]

### Suggested fix
Update the doc comment to state that the estimator uses chord/Euclidean distance between L2-normalized vectors (`sqrt(2 - 2cos)`), and optionally mention that cosine distance `1-cos` is squared-angle and biases TwoNN.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[qodo-code-review]

PR Summary by Qodo

Add ordinal-routing research probes, CRT vernier oracle, and writeups (examples/ + benchmarks/)
✨ Enhancement 📝 Documentation 🕐 40+ Minutes

Walkthroughs

Description

• Add research-grade routing probes and generators under examples/ (no ordvec API changes)
• Document CRT vernier seam theorem, density-collapse tau signal, and rigidity bounds
• Include real-corpus runbook, citation audit, and Lean skeleton for CRT formalization

Diagram

graph TD
  U["Researcher"] --> E["embed_ollama.py"] --> N[(".npy embeddings")]
  U --> G["gen_corpus.rs"] --> N
  N --> P["Rust probes (examples)"] --> R["benchmarks/*.md findings"]
  P --> L["Lean skeleton"] --> R

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Factor shared .npy reader/writer into a single helper module

• ➕ Eliminates duplicated parsing/normalization logic across multiple examples
• ➕ Reduces risk of subtle format drift or inconsistent validation checks
• ➕ Makes it easier to harden IO (header parsing, errors, large-file streaming)
• ➖ Slightly less copy-pasteable examples if the goal is standalone probes
• ➖ Adds an internal module that might be mistaken for supported library surface

2. Use an existing npy/ndarray crate for IO

• ➕ More robust format coverage and less bespoke parsing code
• ➕ Potentially clearer intent and fewer correctness concerns
• ➖ Introduces dependencies contrary to the repo’s 'dependency-light' posture
• ➖ May complicate build targets (WASM, minimal environments) for examples

3. Promote key probes into a dedicated research harness crate/workspace

• ➕ Cleaner separation between library code and exploratory research artifacts
• ➕ Easier CI gating, feature flags, and reproducibility scripting
• ➖ Higher repo complexity and more scaffolding for a branch explicitly marked exploratory
• ➖ May be premature until the probes stabilize

Recommendation: Current approach (keeping everything in examples/ and benchmarks/ with explicit tiering/withdrawals) is appropriate for exploratory research and avoids impacting ordvec’s API. The main improvement worth considering is consolidating the repeated .npy IO + row-normalization code into a small internal helper to reduce duplication-driven bugs (similar to the RNG-desync and geometry-sharing issues already called out and fixed).

File Changes

Enhancement (8)

VernierSeamDensity.lean Add Lean 4 skeleton for CRT seam-density theorem with corrected NeZero requirement +141/-0

Add Lean 4 skeleton for CRT seam-density theorem with corrected NeZero requirement

• Introduces a draft Lean formalization of the seam-density count, including an explicit '[∀ i, NeZero (m i)]' precondition and marked 'sorry's for the CRT equivalence and supporting lemmas.

benchmarks/lean/VernierSeamDensity.lean

---

crt_seam_oracle.rs Add exhaustive CRT seam oracle enumerating Z/M for vernier theorem checks +164/-0

Add exhaustive CRT seam oracle enumerating Z/M for vernier theorem checks

• Implements a full enumeration over Z/M for coprime periods to verify lcm spacing, unique coincidence per period, capped near-seam density, and phase-dependent floor behavior.

examples/crt_seam_oracle.rs

---

density_collapse.rs Add density-collapse probe: Hamming near-collisions and Kendall-tau signal +345/-0

Add density-collapse probe: Hamming near-collisions and Kendall-tau signal

• Builds b-bit bucket codes from rank transforms, reports near-collision structure via Hamming distance, and tests whether top-k Kendall-tau separates true FP32 neighbors from b=2 lookalikes; supports real corpora via .npy.

examples/density_collapse.rs

---

embed_ollama.py Add Ollama embedding generator to produce .npy corpora for real-data runs +104/-0

Add Ollama embedding generator to produce .npy corpora for real-data runs

• Queries local Ollama nomic-embed-text embeddings in batches and writes NumPy v1 .npy files compatible with the Rust probes’ minimal readers.

examples/embed_ollama.py

---

gen_corpus.rs Add corpus zoo generator emitting multiple synthetic geometries as .npy +279/-0

Add corpus zoo generator emitting multiple synthetic geometries as .npy

• Generates clustered/isotropic/anisotropic/rogue/manifold/lattice-style corpora plus optional queries, fixes corpus/query geometry sharing by seeding a dedicated geometry RNG, and writes a consistent .npy format.

examples/gen_corpus.rs

---

shard_recall.rs Add shard-recall router evaluation with clean arm ablations (RNG desync fixed) +434/-0

Add shard-recall router evaluation with clean arm ablations (RNG desync fixed)

• Implements a candidates-scanned fair envelope comparison across aligned, random-offset, coprime, and combined arms; fixes RNG coupling so arms share identical projection directions and differ only by phase/width policy.

examples/shard_recall.rs

---

spectral_probe.rs Add number-variance probe with unfolding modes and rigidity self-test +375/-0

Add number-variance probe with unfolding modes and rigidity self-test

• Computes unfolded 1-D key positions from random projections and reports number variance across window sizes; includes explicit self-test and multiple unfolding strategies (Gaussian, exact-rank, smooth-knot).

examples/spectral_probe.rs

---

twonn_id.rs Add TwoNN intrinsic dimension estimator using chord distance on unit vectors +235/-0

Add TwoNN intrinsic dimension estimator using chord distance on unit vectors

• Implements TwoNN via exact NN searches for sampled anchors, fixes the metric to chord distance (locally linear in angle), supports .npy corpora, and reports an intrinsic dimension estimate.

examples/twonn_id.rs

---

Documentation (12)

README.md Add Motivation section explaining ordvec’s niche vs PQ/HNSW +12/-1

Add Motivation section explaining ordvec’s niche vs PQ/HNSW

• Introduces a new 'Motivation' section describing why ordvec is training-free and data-oblivious, and highlights footprint predictability and SIMD-friendly deployment.

README.md

---

ADVERSARIAL_REVIEW.md Record adversarial reviewer findings and tier claims (SOUND/CORRECTED/WITHDRAWN) +84/-0

Record adversarial reviewer findings and tier claims (SOUND/CORRECTED/WITHDRAWN)

• Adds a consolidated critique log from three hostile reviews, including corrected CRT density form, Lean preconditions, withdrawn number-variance result, and clarified rigidity theorem statements.

benchmarks/ADVERSARIAL_REVIEW.md

---

REAL_CORPUS_RUNBOOK.md Document a unified real-corpus workflow for all probes via shared .npy +48/-0

Document a unified real-corpus workflow for all probes via shared .npy

• Defines a single .npy format contract and provides reproducible commands for TwoNN ID, number variance, shard recall, and the existing bench_rank on real embeddings.

benchmarks/REAL_CORPUS_RUNBOOK.md

---

conjecture_citation_audit.md Add citation audit and corrections for prime/RH/geometry conjecture claims +41/-0

Add citation audit and corrections for prime/RH/geometry conjecture claims

• Separates verified vs qualitative vs corrected/unverified citations and documents specific confabulation fixes discovered during review.

benchmarks/conjecture_citation_audit.md

---

corpus_zoo_results.md Retain (withdrawn) corpus-zoo number-variance results with prominent banner +66/-0

Retain (withdrawn) corpus-zoo number-variance results with prominent banner

• Adds a withdrawal notice explaining unfold calibration issues while preserving the historical measurements and the reproducibility recipe as a record.

benchmarks/corpus_zoo_results.md

---

crt_seam_oracle_results.md Write up CRT vernier seam oracle theorem and corrected capped density form +83/-0

Write up CRT vernier seam oracle theorem and corrected capped density form

• Documents the exhaustive CRT seam enumeration results, corrects the density closed form to ∏ min(2t+1,m_i)/m_i, and explicitly refutes a generic phase-tunable floor claim.

benchmarks/crt_seam_oracle_results.md

---

density_collapse_results.md Describe RankQuant b=2 density-collapse mechanism and tau-based tie-break signal +94/-0

Describe RankQuant b=2 density-collapse mechanism and tau-based tie-break signal

• Explains near-collision via Hamming-close b=2 codes and reports a Kendall-tau separation signal that survives on real nomic-embed-text embeddings; outlines next experiments (tau rerank vs b=4).

benchmarks/density_collapse_results.md

---

README.md Add Lean formalization overview and explicit proof debt for CRT theorem +59/-0

Add Lean formalization overview and explicit proof debt for CRT theorem

• Describes module placement rationale, what is established vs open, and enumerates the remaining Lean proof obligations (crtEquiv, card lemmas, congruence transport).

benchmarks/lean/README.md

---

rigidity_impossibility_proofs.md Add corrected rigidity impossibility proofs (binomial variance, mixture hypothesis) +121/-0

Add corrected rigidity impossibility proofs (binomial variance, mixture hypothesis)

• Provides proofs that unfolded i.i.d. keys are binomial with Σ²(L)=L(1−L/n) and that mixture variance can only increase Σ²; retracts an over-broad quantile-optimality claim.

benchmarks/rigidity_impossibility_proofs.md

---

shard_recall_results.md Report shard-recall rerun after RNG-desync fix and clarify coprime geometry confound +69/-0

Report shard-recall rerun after RNG-desync fix and clarify coprime geometry confound

• Adds a results writeup noting the corrected clean ablation (aligned vs random-offset tied) and explains why coprime arms plateau due to parameterization and geometry mismatch.

benchmarks/shard_recall_results.md

---

spectral_probe_results.md Mark number-variance probe results as withdrawn due to uncalibrated unfolding +54/-0

Mark number-variance probe results as withdrawn due to uncalibrated unfolding

• Adds a prominent withdrawal banner describing why the probe does not measure the intended quantity; retains prior numbers as failure record and points to theory as independent.

benchmarks/spectral_probe_results.md

---

twonn_id_results.md Document TwoNN intrinsic-dimension probe and chord-vs-cosine metric correction +61/-0

Document TwoNN intrinsic-dimension probe and chord-vs-cosine metric correction

• Explains the metric bug (cosine distance as squared distance), provides sphere validation, and notes estimator bias caveats raised in adversarial review.

benchmarks/twonn_id_results.md

---

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive suite of benchmarks, mathematical proofs, and experimental probes to investigate the 'prime mile-marker / spectral index' conjecture for training-free routing in ordvec. It includes a Lean formalization of the vernier CRT seam-density theorem, Python scripts for generating real embeddings, and several Rust examples analyzing intrinsic dimension, number variance, density collapse, and shard recall. The review feedback highlights several opportunities to improve performance and robustness: precomputing knot values and removing redundant sorting in spectral_probe.rs, avoiding unnecessary HashSet allocations and preventing out-of-bounds panics in shard_recall.rs, handling potential NaN values during sorting in density_collapse.rs and twonn_id.rs, and adding a missing NeZero instance argument in the Lean proof to ensure successful compilation.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[toadkicker]

Reframed as a research writeup — no src/ / API changes, nothing to ship to the crate.

The decisive bake-off (tau_rerank_bakeoff_results.md)
settled the one path that could have been a feature: a Kendall-tau rerank of b=2
survivors does not beat b=4 (it scores below b=2's own ordering, even at the
ceiling). So the density-collapse signal is real-but-inert — "just use b=4."

What this branch is, then: a characterized mechanism + a clean negative + verified
citations, plus two withdrawn/corrected items kept honestly on record. Start at
benchmarks/README.md.

The Lean theorem work has moved to its proper home:
ordvec-formalization#17 (draft).

Suggested disposition: merge as a labeled research/analysis artifact if the
project wants the negative on record, or close — either is fine; nothing here
touches the library.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[toadkicker]

Closeout — the research line is complete

Final commit 1792349 (Path B, chunk-length-mixture lake) closes the last
experiment that was capturable without real dirty data. Refreshed the PR description
to the final honest state; summary of where the whole arc landed:

Verdict: research, not a feature. Every constructed alternative to "spend the bits,
b=4 raw ranks" was pre-registered and came back negative or inert:

• Density-collapse tau signal — real but inert; b=4 wins the matched-bytes
  bake-off (b4 R@10 0.942 vs tau-rerank 0.597).
• Oblivious low-discrepancy directions (golden/Sobol/Kronecker) — CLASS-DEAD across
  5 encoders at real ID 18–24; the only lever at high ID is data-alignment (PCA), which
  training-free forbids.
• Deployment robustness — b=4 routing is immune to every synthetic lake pathology:
  multi-domain cones + templated hubs (Phase B, −0.002 through 15%) and now a
  chunk-length mixture (Path B, +0.002, CR@100=1.0).

Two earlier headline claims were overturned and corrected in place rather than
edited away: density collapse is not exploitably recoverable (the bake-off), and
"nomic ID≈13" was a corpus property, not an encoder constant (fiqa reads ~24 on the
same encoder). Integrity record in ADVERSARIAL_REVIEW.md.

The only non-derivative test remaining needs real dirty data — an actual S3
sample with OCR-garbage / mixed-language / broken-encoding noise, which clean
embeddings can't model. Everything else re-confirms the negative. Closing the arc here.

Reviewer's guide: experiments/ordinal-routing-research/README.md.

🤖 Generated with Claude Code