Field report: CONCURRENTLY double-insert (HNSW AM), SONA dim hardcode, ef_search GUC question#716
Draft
jjohare wants to merge 2096 commits into
Draft
Field report: CONCURRENTLY double-insert (HNSW AM), SONA dim hardcode, ef_search GUC question#716jjohare wants to merge 2096 commits into
jjohare wants to merge 2096 commits into
Conversation
Published ruvector-postgres@2.0.4 to crates.io with SPARQL parser backtrack fix, executor memory leak fix, and catch_unwind safety. Co-Authored-By: claude-flow <ruv@ruv.net>
Built from commit 3147d1e Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions
Built from commit 059cb2a Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions
Co-Authored-By: claude-flow <ruv@ruv.net>
Built from commit d772890 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions
Integrate 5 workspace crates (ruvector-solver, ruvector-math, ruvector-attention, sona, ruvector-domain-expansion) as 6 feature-gated modules exposing solver, math distances, TDA, extended attention, Sona learning, and domain expansion — bringing total to 143 SQL functions. Docker image verified with all functions passing. Co-Authored-By: claude-flow <ruv@ruv.net>
Update function counts (143 SQL functions, 46 attention mechanisms), add v0.3.0 highlights section, document 6 new modules (Solver, Math, TDA, Extended Attention, Sona, Domain Expansion), update Docker tags, feature flags, and capabilities table (49 features). Co-Authored-By: claude-flow <ruv@ruv.net>
Co-Authored-By: claude-flow <ruv@ruv.net>
- Run cargo fmt --all to fix formatting in 362 files across the entire workspace - Add PGDG repository for PostgreSQL 17 in CI test-all-features and benchmark jobs - Add missing rvf dependency crates to standalone Dockerfile for domain-expansion - Add sona-learning and domain-expansion features to standalone Dockerfile build - Create npu.rs stub for ruvector-sparse-inference (fixes rustfmt resolution error) Co-Authored-By: claude-flow <ruv@ruv.net>
- Add #[allow(unreachable_code)] for NEON fallback in distance/mod.rs (ARM always returns before the Scalar fallback, causing clippy error on macOS) - Restructure standalone Dockerfile to use workspace layout so dependency crates with workspace inheritance (edition.workspace, version.workspace) can resolve correctly during Docker builds Co-Authored-By: claude-flow <ruv@ruv.net>
…and sona - ruvector-postgres: Add EdgeType import in mincut tests, remove incorrect Some() wrapping on pgrx default!() test params - ruvllm: Make ane_ops module available on all platforms (not just macOS) so tests can reference it unconditionally; fix unused variable warnings - sona: Add explicit lifetime annotations on RwLockReadGuard/WriteGuard to fix clippy mismatched_lifetime_syntaxes errors Co-Authored-By: claude-flow <ruv@ruv.net>
- Fix clippy -D warnings across 3 crates that blocked Code Quality CI - ruvector-core: fix unused imports, or_insert_with→or_default, div_ceil, field_reassign_with_default, iterator patterns, abs_diff - sona: fix unused imports, iterator patterns, range contains, unused fields, Default derives, factory struct init - ruvllm: add crate-level allows for pervasive style lints, fix or_insert_with→or_default in 4 files, allow clippy::all in test files - Change missing_docs from warn to allow in all 3 crates (116+ items) - Bump cargo-pgrx from 0.12.0 to 0.12.9 in postgres-extension-ci.yml Co-Authored-By: claude-flow <ruv@ruv.net>
- postgres-extension-ci.yml: bump cargo-pgrx 0.12.0→0.12.9 (4 locations) - ruvector-postgres-ci.yml: bump PGRX_VERSION 0.12.6→0.12.9 - Run cargo fmt to reformat multi-attribute #![allow(...)] lines Co-Authored-By: claude-flow <ruv@ruv.net>
…doc link errors The pgrx test steps used --no-default-features without passing the pg17 feature, causing linker failures against PostgreSQL symbols. Also escape bracket notation in doc comments to prevent unresolved intra-doc link errors. Co-Authored-By: claude-flow <ruv@ruv.net>
…ion-upgrade feat: ruvector-postgres v0.3.0 — 43 new SQL functions (ADR-044)
Built from commit cc5ab24 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions
…lation Implement ADR-014 Health Biomarker Analysis Architecture: - biomarker.rs: Composite risk scoring engine with 17-SNP weight matrix, gene-gene interaction modifiers (COMT×OPRM1, MTHFR compound, BRCA1×TP53), 64-dim HNSW-aligned profile vectors, clinical reference ranges for 12 biomarkers, and deterministic synthetic population generation - biomarker_stream.rs: Streaming biomarker simulator with generic RingBuffer, configurable noise/drift/anomaly injection, z-score anomaly detection, linear regression trend analysis, and exponential moving averages - 35 unit tests + 15 integration tests (168 total, 0 failures) - Criterion benchmark suite targeting ADR-014 performance budgets https://claude.ai/code/session_014FpaYVohmyLH5dcBZTgmSY
… halve ring buffer memory - Fix snp_idx silent fallback: unwrap_or(0) masked missing SNPs with incorrect index-0 lookups; now returns Option<usize> - RingBuffer: eliminate Option<T> wrapper, halving per-slot memory for f64 (8 bytes vs 16); use T::Default instead - window_mean_std: replace two-pass sum+variance with single-pass Welford's online algorithm (2x fewer cache misses) - compute_risk_scores: pre-compute category max scores via category_meta() to avoid re-scanning SNP_WEIGHTS per call; use &str keys in intermediate HashMap to reduce String allocations - HashMap capacity hints throughout (StreamProcessor, genotypes, biomarker_values, cat_scores) to eliminate rehashing - generate_synthetic_population: hoist APOE lookup out of inner loop, reserve biomarker_values capacity upfront - All 48 tests pass (33 unit + 15 integration), benchmark compiles https://claude.ai/code/session_014FpaYVohmyLH5dcBZTgmSY
…fehacks clinical data Evidence-based adjustments from geneticlifehacks.com research articles: - MTHFR C677T (rs1801133): het weight 0.30→0.35 to match documented 40% enzyme activity decrease - MTHFR A1298C (rs1801131): het 0.15→0.10, hom_alt 0.35→0.25 to match documented ~20% enzyme decrease - Homocysteine reference range: 4-12→5-15 μmol/L (clinical consensus), critical_high 50→30 (moderate hyperhomocysteinemia threshold) - Add MTHFR A1298C × COMT interaction (1.25x Neurological): A1298C homozygous + COMT slow = amplified depression risk - Add DRD2/ANKK1 × COMT interaction (1.2x Neurological): rs1800497 × Val158Met working memory interaction - Guard vector encoding with .take(4) so expanded interaction table (now 6 entries) doesn't overflow dims 56-59 Sources: - geneticlifehacks.com/mthfr/ (enzyme activity percentages) - geneticlifehacks.com/mthfr-c677t/ (MTHFR-COMT depression data) - geneticlifehacks.com/understanding-homocysteine-levels/ (ref ranges) - geneticlifehacks.com/dopamine-receptor-genes/ (DRD2×COMT interaction) All 48 tests pass (33 unit + 15 integration), benchmark compiles. https://claude.ai/code/session_014FpaYVohmyLH5dcBZTgmSY
Evidence-based refinements from peer-reviewed clinical research: - TP53 rs1042522 (Pro72Arg): hom_ref 0.10→0.00 — CC/Pro/Pro is not independently risk-associated; prior non-zero baseline was unjustified - BRCA2 rs11571833 (K3326X): het 0.25→0.20 — aligned with iCOGS meta-analysis OR 1.28 for breast cancer (Meeks et al., JNCI 2016, 76,637 cases / 83,796 controls) - NQO1 rs1800566 (Pro187Ser): het 0.20→0.15, hom_alt 0.45→0.30 — aligned with comprehensive meta-analysis OR 1.18 for TT vs CC (Lajin & Alachkar, Br J Cancer 2013, 92 studies, 21,178 cases); larger 2022 meta-analysis (43,736 cases) found no overall association Validated unchanged weights against SOTA evidence: - APOE rs429358: OR 3-4x het, 8-15x hom (Belloy JAMA Neurology 2023) - SLCO1B1 rs4363657: OR 4.5/allele, 16.9 hom (SEARCH/NEJM; CPIC 2022) - COMT×OPRM1 interaction: confirmed p=0.037 (orthopedic trauma study) All 48 tests pass (33 unit + 15 integration). https://claude.ai/code/session_014FpaYVohmyLH5dcBZTgmSY
…tion, and interaction tests - Add gene→biomarker correlations in synthetic population: APOE e4→lower HDL/higher triglycerides, MTHFR→lower B12, NQO1 null→higher CRP - Add CUSUM changepoint detection algorithm to StreamProcessor for detecting sustained biomarker shifts beyond simple anomaly detection - Add 4 new integration tests: MTHFR×COMT interaction, DRD2×COMT interaction, APOE→HDL population correlation, CUSUM changepoint detection - Remove unused variant_categories import - All 172 tests pass, all ADR-014 performance targets exceeded https://claude.ai/code/session_014FpaYVohmyLH5dcBZTgmSY
- Add Health Biomarker Engine section to rvDNA README with usage examples for composite risk scoring, streaming processing, and synthetic populations - Add biomarker.rs and biomarker_stream.rs to Modules table - Update test count from 102 to 172 (added biomarker tests) - Add biomarker benchmark results to Speed table - Add Welford, CUSUM, and PRS to Published Algorithms table - Update root README Genomics & Health capabilities (49 → 51 features) - Add health biomarker engine and streaming biomarkers to root feature table - Update rvDNA details section with risk scoring and streaming capabilities https://claude.ai/code/session_014FpaYVohmyLH5dcBZTgmSY
…eaming Structural improvements from deep code review: - Consolidate 5 parallel arrays (SNP_WEIGHTS, HOM_REF, HOM_ALT, HET, ALLELE_FREQS) into single SnpDef struct array — eliminates entire class of parallel-array misalignment bugs - Cache category_meta() with LazyLock — avoids per-call Vec allocation (critical in generate_synthetic_population hot path) - Hoist Normal::new out of inner loop in generate_readings — pre-compute distributions per biomarker instead of per-step*per-biomarker - Add clinically meaningful lower bounds: LDL normal_low 0→50 mg/dL (critical_low 25), Triglycerides normal_low 0→35 mg/dL (critical_low 20) - Optimize RingBuffer::clear from O(capacity) to O(1) — head/len reset is sufficient since push overwrites before read - Use NUM_SNPS const for vector encoding bounds instead of magic number 51 All 172 tests pass, zero clippy warnings for rvdna. https://claude.ai/code/session_014FpaYVohmyLH5dcBZTgmSY
…ence Add rs10455872 (OR 1.6-1.75/allele CHD) and rs3798220 (OR 1.49-1.54/allele) from 2024 LPA meta-analyses. Include Lp(a) biomarker reference (0-75 nmol/L) and gene-biomarker correlation in population model. Separate NUM_ONEHOT_SNPS (17) from NUM_SNPS (19) to preserve 64-dim vector layout with LPA encoded in summary dimension 63. https://claude.ai/code/session_014FpaYVohmyLH5dcBZTgmSY
Add PCSK9 R46L loss-of-function variant (NEJM 2006: OR 0.77 CHD, 0.40 MI) as a protective cardiovascular SNP with negative weights. Include PCSK9→LDL-C biomarker correlation (15-21% lower LDL in carriers). Refactor gene-biomarker correlations from match to additive if-chain so multiple gene effects can stack on the same biomarker (e.g., APOE raises LDL while PCSK9 R46L lowers it). Panel expanded to 20 SNPs. https://claude.ai/code/session_014FpaYVohmyLH5dcBZTgmSY
Update all references from 17 SNPs to 20 SNPs reflecting the addition of LPA rs10455872/rs3798220 and PCSK9 rs11591147. Document new gene-biomarker correlations (LPA→Lp(a), PCSK9→LDL) in synthetic population section. Update module table line counts. https://claude.ai/code/session_014FpaYVohmyLH5dcBZTgmSY
…nd benchmarks ADR-015: Pure-JS biomarker engine mirroring Rust biomarker.rs and biomarker_stream.rs exactly. Includes: - src/biomarker.js: 20-SNP composite risk scoring, 6 gene-gene interactions, 64-dim L2-normalized profile vectors, synthetic population generation with Mulberry32 PRNG - src/stream.js: RingBuffer, StreamProcessor with Welford online stats, CUSUM changepoint detection, z-score anomaly detection, linear regression trend analysis, batch reading generation - tests/test-biomarker.js: 35 tests + 5 benchmarks covering all classification levels, risk scoring, vector encoding, population generation, streaming, anomaly/trend detection - index.d.ts: Full TypeScript definitions for all biomarker APIs - package.json: Bump to v0.3.0, add biomarker keywords Benchmark results (Node.js): computeRiskScores: 7.33 us/op encodeProfileVector: 9.51 us/op RingBuffer push+iter: 3.32 us/op https://claude.ai/code/session_014FpaYVohmyLH5dcBZTgmSY
Optimizations (1.7-2x speedup across all hot paths): - biomarker.js: Replace O(n) findIndex with pre-built RSID_INDEX Map for O(1) SNP lookups; cache LPA SNP references to avoid repeated array iteration in vector encoding and population generation - stream.js: Add RingBuffer.pushPop() returning evicted value; replace O(n) windowMeanStd buffer scan with O(1) incremental windowed Welford algorithm in StreamProcessor Benchmark improvements (before → after): computeRiskScores: 7.33 → 3.70 us/op (1.98x) encodeProfileVector: 9.51 → 5.25 us/op (1.81x) StreamProcessor.processReading: 220 → 110 us/op (2.00x) generateSyntheticPopulation(100): 1090 → 595 us/op (1.83x) Real-data integration tests (25 new tests): - 4 realistic 23andMe fixture files (29 SNPs each) covering: high-risk cardio, low-risk baseline, multi-risk, PCSK9-protective - End-to-end pipeline: parse 23andMe → biomarker scoring → streaming - Clinical scenarios: APOE e4/e4, BRCA1 carrier, MTHFR compound het, COMT×OPRM1 pain, DRD2×COMT, PCSK9 protective - Cross-validation: 8 JS↔Rust parity assertions on tables, z-scores, classification, vector layout, risk thresholds - Population correlations: APOE→HDL, LPA→Lp(a), score distribution, clinical biomarker range validation (500 subjects) - Full pipeline benchmark: 220 us end-to-end https://claude.ai/code/session_014FpaYVohmyLH5dcBZTgmSY
…-ESZy4 Reviewed: all CI checks pass, 48 Rust tests + 60 JS tests pass, code review clean. Publishing rvdna crate 0.2.0 and @ruvector/rvdna 0.3.0.
- ExoTransferOrchestrator.package_as_rvf(): serializes all TransferPriors, PolicyKernels, and CostCurves into a 64-byte-aligned RVF byte stream - ExoTransferOrchestrator.save_rvf(path): convenience write-to-file method - Enable ruvector-domain-expansion rvf feature in exo-backend-classical - 3 new RVF tests: empty packager, post-cycle magic verification, save-to-file - substrate.rs: fill pattern field from returned search vector (r.vector.map(Pattern::new)) - README: document 5-phase transfer pipeline, RVF packaging, updated architecture diagram, 4 new Key Discoveries, 3 new Practical Applications All 0 failures across full workspace test suite. https://claude.ai/code/session_019Lt11HYsW1265X7jB7haoC
New `rvf` feature flag enables the `ruvf::RoboticsRvf` wrapper that bridges point clouds, scene graphs, trajectories, Gaussian splats, and obstacles into the RuVector Format (.rvf) for persistence and similarity search. RoboticsRvf supports: - pack_point_cloud (dim 3) - pack_scene_objects / pack_scene_graph (dim 9) - pack_trajectory (dim 3) - pack_gaussians (dim 7) — converts PointCloud→GaussianSplatCloud→RVF - pack_obstacles (dim 6) - query_nearest (kNN via HNSW index) - open/open_readonly/close lifecycle 9 unit tests covering create, ingest, query, reopen, dimension mismatch, and empty data rejection. Also fixes unused import warnings in integration tests. All 290 tests pass across default, domain-expansion, and rvf features. https://claude.ai/code/session_01H1GkTK5z9ppVVQDQukjBsY
Implements energy-driven computation with Landauer dissipation and Langevin/Metropolis noise. Key components: - State: activation vector + cumulative dissipated-joules counter - EnergyModel trait + Ising (Hopfield) + SoftSpin (double-well) Hamiltonians - Couplings: zeros, ferromagnetic ring, Hopfield memory factories - Params: inverse temperature β, Langevin step η, Landauer cost per irreversible flip - step_discrete: Metropolis-Hastings spin-flip with Boltzmann acceptance - step_continuous: overdamped Langevin (central-difference gradient + FDT noise) - anneal_discrete / anneal_continuous: traced annealing helpers - inject_spikes: Poisson kick noise, clamp-aware - Metrics: magnetisation, Hopfield overlap, binary entropy, free energy, Trace - Motifs: IsingMotif (ring, fully-connected, Hopfield), SoftSpinMotif (random) - 19 correctness tests: energy invariants, Metropolis, Langevin, Hopfield retrieval - 4 Criterion benchmark groups: step, 10k-anneal, Langevin, energy eval - GitHub Actions CI: fmt + clippy + test (ubuntu/macos/windows) + bench compile https://claude.ai/code/session_019Lt11HYsW1265X7jB7haoC
ruvector-dither (new crate):
- GoldenRatioDither: additive φ-sequence with best 1-D equidistribution
- PiDither: cyclic 256-entry π-byte table for deterministic weight dithering
- quantize_dithered / quantize_slice_dithered: drop-in pre-quantization offset
- quantize_to_code: integer-code variant for packed-weight use
- ChannelDither: per-channel pool seeded by (layer_id, channel_id) pairs
- DitherSource trait for generic dither composition
- 15 unit tests + 3 doctests; 4 Criterion benchmark groups
exo-backend-classical integration:
- ThermoLayer (thermo_layer.rs): Ising motif coherence gate using thermorust
- Runs Metropolis steps on clamped activations
- Returns ThermoSignal { lambda, magnetisation, dissipation_j, energy_after }
- λ-signal = −ΔE/|E₀|: positive means pattern is settling toward coherence
- DitheredQuantizer (dither_quantizer.rs): wraps ruvector-dither for exo tensors
- GoldenRatio or Pi kind, per-layer seeding, reset support
- Supports 3/5/7/8-bit quantization with ε-LSB dither amplitude
- 8 new unit tests across both modules; all 74 existing tests still pass
https://claude.ai/code/session_019Lt11HYsW1265X7jB7haoC
…erception SpatialIndex: replace Vec<Vec<f32>> with flat Vec<f32> buffer for cache locality and zero per-point heap allocation; use squared Euclidean distance in kNN/radius search (defer sqrt to final k results); fuse cosine distance into single loop. Clustering: add union-by-rank to union-find preventing tree degeneration (O(α(n)) amortized); add #[inline] on hot helpers. A* planning: add closed set (HashSet) to avoid re-expanding nodes; reuse neighbor buffer to eliminate per-expansion Vec allocation; pre-allocate HashMap capacity; add #[inline] on helpers. Perception: defer sqrt in bounding_sphere (compare squared distances, one sqrt at end); defer sqrt in scene graph edge construction (filter on squared threshold); add #[inline] on dist_3d. Sensor fusion: pre-allocate merged vectors from total eligible cloud size. Anomaly detection: fuse distance + statistics into single pass using Welford's online algorithm (eliminates one full data pass). All 281 tests pass. https://claude.ai/code/session_01H1GkTK5z9ppVVQDQukjBsY
Implements ADR-057 with 7 modules (2,940 lines, 54 tests): - types: 4 new segment types (FederatedManifest 0x33, DiffPrivacyProof 0x34, RedactionLog 0x35, AggregateWeights 0x36) - pii_strip: 3-stage pipeline (detect, redact, attest) with 12 regex rules - diff_privacy: Gaussian/Laplace noise, RDP accountant, gradient clipping - federation: ExportBuilder + ImportMerger with version-aware conflict resolution - aggregate: FedAvg, FedProx, Byzantine-tolerant weighted averaging - policy: FederationPolicy for selective sharing with allow/deny lists - error: 15 typed error variants Also updates rvf-types with 4 new segment discriminants (0x33-0x36), workspace Cargo.toml, and root README (crate count, segment count, federated learning code example). Co-Authored-By: claude-flow <ruv@ruv.net>
feat: rvf-federation crate for federated transfer learning
Built from commit c550fff Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions
Clippy fixes (8 warnings → 0): - Replace 5 manual Default impls with #[derive(Default)] - Use .clamp() instead of .min().max() chain - Use .is_some_and() instead of .map_or(false, ...) - Add type alias for complex return type in scene_graph_to_adjacency P0 correctness fixes from code review: - Fix NaN panic: use unwrap_or(Ordering::Equal) in cognitive_core think() - Fix integer overflow: use checked_mul in OccupancyGrid::new - Fix potential unwrap: use map_or in domain_expansion score_avoidance Co-Authored-By: claude-flow <ruv@ruv.net>
Add repository, homepage, keywords, categories for crates.io listing. Pin optional dependency versions (ruvector-domain-expansion 2.0.4, rvf-runtime 0.2, rvf-types 0.2) required for cargo publish. Co-Authored-By: claude-flow <ruv@ruv.net>
…egration-VOZu2 Add ruvector-robotics: unified cognitive robotics platform
Built from commit 85df6b9 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions
- Replace debug_assert with assert for bits bounds in quantize functions - Guard ChannelDither against 0 channels and invalid bits - Handle non-finite beta/rate in Langevin/Poisson noise (return 0) - Remove unused itertools dependency from thermorust - Fix partial_cmp().unwrap() NaN panics across 7 exo-ai files - Fix SystemTime unwrap() in transfer_crdt (use unwrap_or_default) - Fix domain ID mismatch (exo_retrieval → exo-retrieval) in orchestrator - Update tests to match corrected domain IDs Co-Authored-By: claude-flow <ruv@ruv.net>
Required for crates.io publishing. Co-Authored-By: claude-flow <ruv@ruv.net>
…README - Add ruvector-dither to Advanced Math & Inference section - Add thermorust to Neuromorphic & Bio-Inspired Learning section - Add collapsed Cognitive Robotics section for ruvector-robotics Co-Authored-By: claude-flow <ruv@ruv.net>
…rsion deps - Run cargo fmt across entire workspace - Create README.md files for all 9 EXO-AI crates - Convert path dependencies to crates.io version dependencies for publishing - Add [patch.crates-io] to exo workspace for local development Co-Authored-By: claude-flow <ruv@ruv.net>
Published to crates.io: - exo-core v0.1.1 - exo-temporal v0.1.1 - exo-hypergraph v0.1.1 - exo-manifold v0.1.1 - exo-federation v0.1.1 - exo-exotic v0.1.1 - exo-backend-classical v0.1.1 Changes from v0.1.0: - Fix NaN panics in all partial_cmp().unwrap() calls - Fix domain ID mismatch (underscores → hyphens) - Fix SystemTime unwrap → unwrap_or_default - Add README.md for all crates - Gate rvf feature behind feature flag in exo-backend-classical - Convert path dependencies to crates.io version dependencies Co-Authored-By: claude-flow <ruv@ruv.net>
…ity-review-LjcVx # Conflicts: # Cargo.toml
ADR-029: Multi-paradigm integration architecture for EXO-AI
Built from commit 75fa1c4 Platforms updated: - linux-x64-gnu - linux-x64-musl - linux-arm64-gnu - linux-arm64-musl - darwin-x64 - darwin-arm64 - win32-x64-msvc - wasm Generated by GitHub Actions
Built from commit 75fa1c4 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions
…hmarks Replace unsupported headline figures with the repo's own measured data (bench_results/latency_benchmark.csv, comparison_benchmark.md): - "150x-12,500x faster" -> ~100x vs brute force (10K/384D), the measured ceiling; the larger figure is an aspirational target, not a result. - "61us p50" -> ~0.8ms p50 (10K/384D). - "80K QPS on 8 cores" -> ~3.3K QPS on 8 cores (50K/384D). - GNN "+5-8% / +12.4% Recall@10" -> qualitative, flagged not yet benchmark-validated (no supporting run exists in benches/ or bench_results/). Co-Authored-By: jjohare <github@thedreamlab.uk>
AgenticDB::new() silently wired the non-semantic HashEmbedding placeholder; the ecosystem-wide "MiniLM-L6-v2 semantic search" claim is therefore not met by the default path (the Candle real-embeddings path is a stub that errors). - Emit a one-shot process-wide WARN (tracing + stderr) on first use of the hash fallback: "semantic embeddings DEGRADED — hash placeholder in use, not MiniLM; HNSW recall is non-semantic". - Document the degraded default and the api-embeddings/real-embeddings features in the crate README; note the runtime warning in the AgentDB API guide. Silent fake-success is banned (closeout F8). Co-Authored-By: jjohare <github@thedreamlab.uk>
The default feature set (`full`) ships forgeable HMAC/HKDF-SHA256 placeholders for ML-DSA-65 / ML-KEM-768; `production-crypto` is not default-on. A default build of any QuDagIdentity/DAG-signature consumer gets forgeable auth silently. - Emit a one-shot process-wide WARN (tracing + stderr) on first use of the placeholder ML-DSA and ML-KEM paths: "NOT cryptographically secure ... signatures forgeable from the public key alone". - Crate docs: add a Security section stating the degraded default plainly and requiring `production-crypto` for real post-quantum security; qualify the "quantum-resistant" QuDAG feature line. Feature defaults left unchanged (making production-crypto default-on is a breaking decision for the crate owner). Co-Authored-By: jjohare <github@thedreamlab.uk>
… signals Several pg_extern paths reported success (or fabricated data) while doing nothing. Replace fake success with explicit errors where the return type allows, and with unmissable one-shot WARNINGs where the ABI/return type does not (closeout F8: silent fake success is banned). - integrity::get_current_mincut: return an explicit "not implemented" error instead of the fabricated lambda_cut=10.0; the gated_transformer caller already fails open with the diagnostic, so gating no longer decides on fake state. - ruvector_hybrid_update_stats: return success:false / not_implemented instead of reusing existing stats and reporting success; BM25 stats never refreshed. - dag query-analysis (analysis.rs): mark the module experimental and emit a one-shot per-backend WARNING — every function returns placeholder data, not real per-query analysis (EXPLAIN output is discarded, SONA feedback unwired). - register_healing_worker / register_background_worker: WARN that no bgworker is registered (self-healing never runs) instead of a success-implying log. - ivfflat_aminsert: WARN once per backend that inserted vectors are NOT indexed (C ABI forces a bool return, so no error can be surfaced); documented as a non-functional stub. Co-Authored-By: jjohare <github@thedreamlab.uk>
The SolverOrchestrator Ax=b router dispatches Forward Push, Backward Push, Hybrid Random Walk, BMSSP (and GaussSeidel) to solve_jacobi_fallback, which tagged the result with the *requested* Algorithm enum — misleading callers about the method and its complexity (STS-002/STS-SOTA claim all 7 "Complete"). - Label fallback results Algorithm::Jacobi (the algorithm that actually ran) and emit a WARN when a non-Jacobi algorithm degrades. No reimplementation. - ADR-STS-SOTA and ADR-STS-002: add closeout caveats stating the four graph- native algorithms are available only via their standalone modules, the Ax=b router degrades to Jacobi, and the Tier-3 SONA adaptive router is disabled (enabled=false) with no adaptive code present. Co-Authored-By: jjohare <github@thedreamlab.uk>
- ADR-045 (lean-agentic), ADR-057 (rvf-federation), ADR-040 (Causal Atlas): Proposed -> Implemented; each cites the crate/examples that realise it (ADR-040 as a demonstrator, not a productised runtime). - ADR-044 (postgres v0.3): "Accepted — Implementation in progress" -> Implemented (ships at 0.3.0 with the v0.3 SQL surface); notes remaining function stubs. - ADR-036 (AGI container): correct wire-format counts 972/24 -> 962/28 and restate that phases 2-4 are unbuilt. - ADR-007 (security/debt tracker): marked Stale with a dated banner — TD-004/005/ 006 point at moved/absent paths and are resolved; counts unreliable pending re-audit. - Duplicate ADR numbers 017/029/031: add docs/adr/README.md disambiguation index (a/b suffixes, no renumber) plus an in-file note on each of the six files. Co-Authored-By: jjohare <github@thedreamlab.uk>
- Cargo.toml: mark the workspace-excluded HNSW-variant crates (micro-hnsw-wasm,
ruvector-hyperbolic-hnsw{,-wasm}) and the rvf tree as EXPERIMENTAL/UNSUPPORTED
members with no CI gate — the closeout FREEZE decision. Manifest still parses.
- ADR-042 (AIDefence TEE): Status "Accepted (spec) — TEE hardware layer not
implemented"; add a note that no SGX/SEV-SNP/TDX/CCA enclave backend exists in
code (only software proof-attestation ships), so Layer 1 is design intent.
Co-Authored-By: jjohare <github@thedreamlab.uk>
The full workspace suite (cargo fmt/clippy/test --workspace --all-features) previously ran only on release-tag push via release.yml, so it had not executed since v0.1.16 despite active core changes. New .github/workflows/ci.yml runs it on every PR and push to main. An advisory (continue-on-error) matrix job exercises the workspace-EXCLUDED crates (ruvector-hyperbolic-hnsw and its wasm variant, micro-hnsw-wasm, and the rvf/ tree) directly via --manifest-path so they are no longer silently untested, while respecting their frozen experimental status by not blocking merges. Co-Authored-By: jjohare <github@thedreamlab.uk>
…NA dim hardcode, ef_search GUC question Three live-verified findings from a production 178k-row agent-memory deployment of ruvector-postgres 2.0.5, each with reproduction SQL: 1. HNSW AM CREATE INDEX CONCURRENTLY inserts every tuple twice (silent recall degradation; non-concurrent build is correct). 2. SONA engine ignores trajectory dimension — hardcoded 256; 384-dim learns return status:learned but accumulate nothing (fresh-scope learn-first reproduced). 3. ruvector.hnsw_ef_search sweep 40..400 has no effect on recall or latency — GUC unread or overridden by dynamic adjustment. Plus a field observation: HNSW graphs degrade under churn; rebuild recovered self-recall 141->177/200. Offer to test candidate fixes against our recall harness. Co-Authored-By: jjohare <github@thedreamlab.uk>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Production field report from a 178k-row agent-memory deployment of
ruvector-postgres:2.0.5(PG 17.9, 384-dim client embeddings, HNSW cosine). Full report with reproduction SQL:docs/reports/2026-07-22-agentbox-field-report.md(this PR's single file).Three live-verified findings:
CREATE INDEX CONCURRENTLYdouble-inserts every tuple into the HNSW graph — every k-NN result row appears exactly twice (silent recall degradation; halves effective k). Non-concurrent build of the same data is correct. Severity: high, silent.embedding_dim: 256. A 384-dimruvector_sona_learnon a fresh scope, learn-first returnsstatus:learnedyet buffers/stores nothing. 405 real judged trajectories (8,855 steps) accepted-and-discarded. Request: propagate detected dim (or config param) and error on mismatch rather than silent no-op.ruvector.hnsw_ef_searchsweep 40→400 has no effect on recall or latency on a degraded graph (question: GUC unread, or overridden by the documented dynamic adjustment? mirrors theVectorDB::search ignores ef_searchnote inruvector-sota-bench).Plus a field observation worth documenting: HNSW graph quality degrades under churn (year of writes + 132k bulk ingest + 2M bulk delete → self-recall@10 fell to 141/200); a non-concurrent rebuild (m=16, ef_construction=128, 4m51s) recovered 177/200.
We run a fixture-based recall-regression harness against this live corpus and are happy to test candidate fixes.
🤖 Generated by Claude Code