Feat: blockwise quantization recall restore

.gitignore

@@ -67,13 +67,22 @@ docs/reports/*.html
 docs/perf/ondevice-query-profiler/GOALS-P5-*.md
 docs/perf/ondevice-query-profiler/PR-P5-[2-9]*.html
 docs/perf/ondevice-query-profiler/artifacts/
+docs/architecture/managed-local-rag-final.png
+docs/architecture/managed-local-rag-final.svg
+docs/promo/flutter-rag-to-llm-workflow-part*-draft.*
+docs/promo/package_guide_android_staging/
+docs/promo/package_guide_staging/
+docs/promo/package_guide_sample_document.txt
+docs/promo/package_guide_validation_*.md
+docs/promo/screenshots/
 example/assets/corpus/
 example/assets/evalsets/
 example/assets/sample_data/*.pdf
 !example/assets/sample_data/sample_eng.pdf
 !example/assets/sample_data/sample_kor.pdf
 
 # Example evaluation/performance runners (local only)
+example/lib/onnx_runtime_smoke.dart
 example/lib/*_runner.dart
 example/lib/profiling/relevance_evalset.dart
 example/lib/profiling/relevance_metrics.dart

CHANGELOG.md

@@ -5,6 +5,15 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 0.20.0
+* **Block-wise Quantization**:
+  - Integrated block-wise scalar quantization (Q8_0 style, 32-dim blocks) for on-device exact-scans.
+  - Implemented backward-compatible dynamic fallback in distance similarity logic to search both legacy 768-byte uniform blobs and new 864-byte packed block-wise blobs.
+* **Retrieval Quality**:
+  - Optimized HNSW index rebuild loops to load original high-precision f32 database embeddings directly, resolving the compound distortion loop and restoring semantic query recall to baseline parity.
+* **Compatibility**:
+  - Bumped native dependency constraint to `rag_engine_flutter: ^0.19.2`.
+
 ## 0.19.1
 * **Packaging**:
   - Fixed the publish archive so `lib/models/*.dart` is included again.

README.md

@@ -10,7 +10,7 @@ round-trip for retrieval.
 ![pub package](https://img.shields.io/pub/v/mobile_rag_engine)
 ![flutter](https://img.shields.io/badge/Flutter-3.9%2B-blue)
 ![rust](https://img.shields.io/badge/Core-Rust-orange)
-![platform](https://img.shields.io/badge/Platform-iOS%20|%20Android%20|%20macOS-lightgrey)
+![platform](https://img.shields.io/badge/Platform-iOS%20%7C%20Android%20%7C%20macOS%20%7C%20Windows%20%7C%20Linux-lightgrey)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
 Use it when you need a **Flutter local RAG engine** for private notes, document
@@ -25,7 +25,7 @@ stay on the device.
 
 **You do NOT need to install Rust, Cargo, or Android NDK.**
 
-This package includes **pre-compiled binaries** for iOS, Android, and macOS. Just `pub add` and run.
+This package includes **pre-compiled binaries** for iOS, Android, macOS, Windows, and Linux. Just `pub add` and run.
 
 ### Performance
 
@@ -91,6 +91,8 @@ Data never leaves the user's device. Perfect for privacy-focused apps (journals,
 | **iOS** | 16.0+ |
 | **Android** | API 21+ (Android 5.0 Lollipop) |
 | **macOS** | 14.0+ |
+| **Windows** | Windows 10+ (x64) |
+| **Linux** | glibc 2.31+ (x64) |
 
 > ONNX Runtime is provided through [`flutter_onnxruntime`](https://pub.dev/packages/flutter_onnxruntime). CocoaPods iOS builds require static framework linkage (`use_frameworks! :linkage => :static`), and Android release builds should keep ONNX Runtime classes in ProGuard/R8 rules.
 
@@ -102,7 +104,7 @@ Data never leaves the user's device. Perfect for privacy-focused apps (journals,
 
 ```yaml
 dependencies:
-  mobile_rag_engine: ^0.19.0
+  mobile_rag_engine: ^0.20.0
 ```
 
 ### 2. Download Model Files

docs/guides/quick_start.md

@@ -7,7 +7,7 @@ Get started with `mobile_rag_engine` in 5 minutes.
 ## Prerequisites
 
 - Flutter 3.9+
-- iOS 16.0+ / Android API 21+ / macOS 14.0+
+- iOS 16.0+ / Android API 21+ / macOS 14.0+ / Windows 10+ / Linux
 
 ---
 
@@ -16,7 +16,7 @@ Get started with `mobile_rag_engine` in 5 minutes.
 ```yaml
 # pubspec.yaml
 dependencies:
-  mobile_rag_engine: ^0.19.0
+  mobile_rag_engine: ^0.20.0
 ```
 
 ```bash

docs/perf/mimalloc-allocator-ab/README.md

@@ -0,0 +1,112 @@
+# Mimalloc Allocator A/B Runbook
+
+This runbook validates whether `allocator_mimalloc` should remain feature-gated,
+become a default-enable candidate, or be dropped.
+
+## Principle
+
+Compare two builds from the same instrumentation commit:
+
+- System allocator: native Rust features without `allocator_mimalloc`.
+- Mimalloc: same features plus `allocator_mimalloc`.
+
+Do not compare `main` against an instrumented branch. The allocator feature must
+be the only intended difference between the two measured variants.
+
+## Scope
+
+`#[global_allocator]` affects Rust global allocations in this native crate and
+Rust dependencies. It does not replace Dart VM, Flutter engine, ONNX Runtime,
+SQLite C pager/cache, OS file cache, or arbitrary C/C++ allocators.
+
+RSS metrics are full-process directional evidence only.
+
+## Required Variants
+
+System allocator expected values:
+
+```bash
+--dart-define=EXPECTED_NATIVE_ALLOCATOR=system
+--dart-define=EXPECTED_RUST_FEATURES=vector_faer,vector_quant_i8
+```
+
+Mimalloc expected values:
+
+```bash
+--dart-define=EXPECTED_NATIVE_ALLOCATOR=mimalloc
+--dart-define=EXPECTED_RUST_FEATURES=vector_faer,vector_quant_i8,allocator_mimalloc
+```
+
+## Measurement Commands
+
+Query profile:
+
+```bash
+cd example
+flutter drive \
+  --driver=test_driver/integration_test.dart \
+  --target=integration_test/query_profile_measure_test.dart \
+  --profile -d <device-id> \
+  --dart-define=EXPECTED_NATIVE_ALLOCATOR=<system|mimalloc> \
+  --dart-define=EXPECTED_RUST_FEATURES=<exact-feature-list>
+```
+
+Allocator-sensitive indexing macro:
+
+```bash
+cd example
+flutter drive \
+  --driver=test_driver/integration_test.dart \
+  --target=integration_test/allocator_indexing_measure_test.dart \
+  --profile -d <device-id> \
+  --dart-define=EXPECTED_NATIVE_ALLOCATOR=<system|mimalloc> \
+  --dart-define=EXPECTED_RUST_FEATURES=<exact-feature-list> \
+  --dart-define=ALLOCATOR_INDEXING_TEXT_MB=5,10,25
+```
+
+`ALLOCATOR_INDEXING_TEXT_MB` controls generated text scale in decimal MB. The
+current macro uses 500-char chunks, 30-char overlap metadata, and 384-dim stub
+embeddings. A 5/10/25 MB run maps to 10k/20k/50k generated chunks.
+
+Android arm64 build smoke:
+
+```bash
+flutter build apk --profile --target-platform android-arm64
+```
+
+## Run Order
+
+Use paired, counterbalanced runs on the same physical device and OS:
+
+```text
+system, mimalloc, system, mimalloc, system, mimalloc
+```
+
+Collect at least three valid runs per variant.
+
+## Required Evidence
+
+- Device model, OS version, battery/thermal state.
+- Git SHA and native artifact hash.
+- APK/app/framework size.
+- `native_allocator` and exact `rust_features`.
+- Query profile CSV/JSON exports.
+- `INDEXING_PROFILE` log rows. In older run artifacts, treat the `docs` field
+  as chunk/vector-point count, not source-document count.
+- Crash, test failure, or FRB content-hash mismatch notes.
+
+## Decision Threshold
+
+`DEFAULT_ENABLE_CANDIDATE` requires all of the following:
+
+- Android physical-device and iOS physical-device profile runs complete.
+- No crash, no FRB content-hash mismatch, no test regression.
+- At least one allocator-sensitive win:
+  - activation p50 improves by at least 10%, or
+  - large indexing/reindexing macro time improves by at least 10%, or
+  - peak RSS improves by at least 8%.
+- Warm `search` and `hydrate` p95 do not regress by more than 5%.
+- Binary size delta is acceptable for the package release.
+
+If results are noise-level, keep the feature gate but do not default-enable
+mimalloc.

docs/perf/mimalloc-allocator-ab/RESULTS.md

@@ -0,0 +1,115 @@
+# Mimalloc Allocator A/B Results
+
+Status: feature-gated candidate, not default-enable.
+
+## Scope Correction
+
+The current allocator indexing macro is text-scale based:
+
+- `ALLOCATOR_INDEXING_TEXT_MB=5,10,25`
+- 500-char generated chunks.
+- 30-char overlap metadata.
+- 384-dim stub embeddings.
+
+That maps to 10k/20k/50k generated chunks. Older run artifacts in this folder
+used a legacy `docs` field with 32-dim embeddings and roughly 50-byte synthetic
+chunks. In those older artifacts, `docs` means chunk/vector-point count, not
+source-document count.
+
+## Decision
+
+Keep `allocator_mimalloc` behind a feature gate. The repeated iOS and macOS
+legacy macro runs show a consistent rebuild-time win, but peak RSS rises with
+mimalloc and those runs used the older 32-dim synthetic corpus.
+
+Do not default-enable mimalloc until the current realistic macro has paired
+physical-device results:
+
+- 5/10/25 MB text scale, mapping to 10k/20k/50k chunks.
+- Physical iOS and Android profile runs.
+- Warm query/search/hydrate regression guard.
+
+## Legacy Repeated Indexing Macro Medians
+
+Each row uses 5 runs per variant. These are legacy synthetic runs retained as
+allocator-pressure evidence, not final product-scale evidence.
+
+### iPad 10, iOS 26.5
+
+Run folder:
+`docs/perf/mimalloc-allocator-ab/runs/ios-ipad10-repeat-20260629-034322/`
+
+| Chunks | System total ms | Mimalloc total ms | Delta | System peak RSS MiB | Mimalloc peak RSS MiB | Peak delta |
+| ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| 10,000 | 1345.418 | 1158.844 | -13.9% | 83.66 | 94.50 | +13.0% |
+| 20,000 | 3816.418 | 3217.003 | -15.7% | 113.55 | 133.63 | +17.7% |
+| 50,000 | 10483.004 | 8393.048 | -19.9% | 182.02 | 218.25 | +19.9% |
+
+### macOS
+
+Run folder:
+`docs/perf/mimalloc-allocator-ab/runs/macos-repeat-20260629-032925/`
+
+| Chunks | System total ms | Mimalloc total ms | Delta | System peak RSS MiB | Mimalloc peak RSS MiB | Peak delta |
+| ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| 10,000 | 1006.692 | 881.308 | -12.5% | 139.11 | 146.61 | +5.4% |
+| 20,000 | 2997.313 | 2796.225 | -6.7% | 175.25 | 181.44 | +3.5% |
+| 50,000 | 7975.187 | 6565.609 | -17.7% | 253.64 | 268.45 | +5.8% |
+
+## Interpretation
+
+The timing win is concentrated in HNSW rebuild. That makes mimalloc a valid
+candidate for activation/indexing/reindexing pressure, but the RSS tradeoff is
+real enough that the branch should not convert it into a default release choice
+yet.
+
+The safer near-term optimization is to reduce peak allocation in the rebuild
+pipeline itself instead of relying only on allocator behavior.
+
+### HNSW Streaming Platform Gate
+
+The realistic system-allocator HNSW A/B changed the HNSW streaming conclusion:
+it is not a universal package optimization.
+
+Benchmark shape: 384-dim f32 stub embeddings, 500-char chunks, 10k/20k/50k
+chunk counts, five-run median, app/profile mode, system allocator only.
+
+macOS matched the expected memory model. Removing the intermediate
+`Vec<(i64, Vec<f32>)>` lowered peak RSS by 4.5% / 5.4% / 8.6% at
+10k / 20k / 50k chunks, with runtime neutral to slightly better.
+
+iPad 10 did not match that model. Streaming was slower by 10.4% / 1.5% / 7.4%
+and peak RSS increased at all measured scales, including a 50k peak increase
+from 740.9 MiB to 821.9 MiB. Treat this as platform-specific interaction among
+row iteration, blob decode, per-row allocation, HNSW insertion timing, allocator
+retention, SQLite/cache accounting, and iOS RSS sampling.
+
+Decision: keep HNSW streaming as a macOS default memory-pressure optimization
+and as an opt-in experiment elsewhere via the `hnsw_streaming_rebuild` Rust
+feature. Keep iOS, Android, and other unvalidated targets on the existing
+collect-based rebuild path by default. Do not describe HNSW streaming as a
+mobile-wide or package-wide improvement.
+
+## Safe Optimization Shortlist
+
+1. Keep HNSW rebuild streaming platform-gated.
+   macOS can stream rows by default because the realistic system-allocator A/B
+   showed lower peak RSS. iOS, Android, and unvalidated targets should keep the
+   collect-based path unless `hnsw_streaming_rebuild` is explicitly enabled for
+   an experiment.
+
+2. Stream BM25 rebuild rows instead of collecting `Vec<(i64, String)>`.
+   Add each document to the in-memory BM25 index as rows are read from SQLite.
+
+3. Reduce BM25 tokenization allocation.
+   Build per-document term frequencies without first materializing a full
+   `Vec<String>` and then cloning tokens into a second map.
+
+4. Run the current realistic indexing macro.
+   The harness now emits text scale, chunk count, 500-char chunk size, 384-dim
+   embedding payload, allocator label, feature label, rebuild timings, and RSS.
+
+5. Evaluate SQLite PRAGMAs as a separate track.
+   `mmap_size`, `cache_size`, `temp_store`, WAL, and checkpoint behavior should
+   be A/B tested separately. Do not wire a SQLite custom allocator to mimalloc
+   in this branch; it is process-global and higher risk on Apple platforms.