From e3b6a2b4304632b2b86a4a61897dfd8d252ce05f Mon Sep 17 00:00:00 2001 From: "Brian.oh" <49855381+dev07060@users.noreply.github.com> Date: Wed, 3 Jun 2026 04:47:59 +0900 Subject: [PATCH 1/5] feat(package) : exclude profiler artifacts from pub archive --- .pubignore | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.pubignore b/.pubignore index fc0579b..e323b5d 100644 --- a/.pubignore +++ b/.pubignore @@ -54,6 +54,8 @@ refrag-develop.md # 테스트 드라이버 test_driver/ integration_test/ +example/test_driver/ +example/integration_test/ # 플랫폼별 디렉토리 (rust_builder에서 제공, 루트 레벨만 제외) /android/ @@ -70,3 +72,5 @@ example/assets/evalsets/ # Example evaluation/performance runners (local only) example/lib/*_runner.dart +example/lib/profiling/ +example/test/profiling/ From a4969b299fd103928a8bc99fbc1dda4af0b7b1ff Mon Sep 17 00:00:00 2001 From: "Brian.oh" <49855381+dev07060@users.noreply.github.com> Date: Wed, 3 Jun 2026 04:48:05 +0900 Subject: [PATCH 2/5] feat(parser) : use English OCR fallback message --- lib/services/document_parser.dart | 4 ++-- test/unit/document_parser_error_helper_test.dart | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/services/document_parser.dart b/lib/services/document_parser.dart index 925e051..da0e735 100644 --- a/lib/services/document_parser.dart +++ b/lib/services/document_parser.dart @@ -11,8 +11,8 @@ class DocumentParser { /// User-facing message for PDFs that contain no extractable text. static const scannedPdfOcrRequiredMessage = - '이 PDF는 텍스트를 추출할 수 없습니다.\n' - '스캔본 또는 이미지 기반 PDF일 수 있어 OCR 처리가 필요합니다.'; + 'This PDF does not contain extractable text.\n' + 'It may be scanned or image-based and require OCR processing.'; /// Returns true when a PDF extraction error indicates a scanned/image-only /// document — the kind OCR can recover. diff --git a/test/unit/document_parser_error_helper_test.dart b/test/unit/document_parser_error_helper_test.dart index 03a3b79..8ef7c4b 100644 --- a/test/unit/document_parser_error_helper_test.dart +++ b/test/unit/document_parser_error_helper_test.dart @@ -16,8 +16,8 @@ void main() { ); expect( DocumentParser.scannedPdfOcrRequiredMessage, - '이 PDF는 텍스트를 추출할 수 없습니다.\n' - '스캔본 또는 이미지 기반 PDF일 수 있어 OCR 처리가 필요합니다.', + 'This PDF does not contain extractable text.\n' + 'It may be scanned or image-based and require OCR processing.', ); }); From 2f227fe096deae611d956c3b5ad25e60ad8f5960 Mon Sep 17 00:00:00 2001 From: "Brian.oh" <49855381+dev07060@users.noreply.github.com> Date: Wed, 3 Jun 2026 04:48:12 +0900 Subject: [PATCH 3/5] feat(release) : prepare 0.18.6 final polishing --- CHANGELOG.md | 14 ++++++++++++++ README.md | 17 ++++++++++++++--- docs/guides/faq.md | 2 +- docs/guides/quick_start.md | 6 +++++- docs/guides/troubleshooting.md | 23 +++++++++++++++++++++++ pubspec.yaml | 4 ++-- rust_builder/CHANGELOG.md | 11 +++++++++++ rust_builder/pubspec.yaml | 2 +- 8 files changed, 71 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 91fea54..e6b719a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,20 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## 0.18.6 +* **PDF extraction UX**: + - Added OCR-needed classification helpers for scanned/image-only PDF extraction errors. + - Keeps mixed scanned + partially failed PDFs on the OCR-recoverable path while leaving fully failed/corrupt PDFs as raw extraction errors. + - Clarified production support boundaries: text-layer PDFs are production-ready, scanned PDFs require app-provided OCR, and DOCX remains beta. +* **Release-path validation**: + - Added i8 hot-path recall/fidelity safety nets for the shipped `vector_quant_i8` path. + - Runs the shipped `vector_faer,vector_quant_i8` native feature combo in CI with fail-closed checks for renamed or skipped safety nets. + - Logs corrupt f32 embedding blobs with row ids during HNSW rebuild instead of silently dropping them. +* **Packaging**: + - Excludes profiler-only example harnesses and tests from the published archive. +* **Compatibility**: + - Bumped dependency constraint to `rag_engine_flutter: ^0.18.4` so consumers receive the matching native hardening patch. + ## 0.18.5 * **PDF extraction quality**: - Bumped dependency constraint to `rag_engine_flutter: ^0.18.3` so consumers receive the native PDF extraction improvements. diff --git a/README.md b/README.md index c74969f..40b78f2 100644 --- a/README.md +++ b/README.md @@ -50,7 +50,7 @@ Data never leaves the user's device. Perfect for privacy-focused apps (journals, | Category | Features | |:---------|:---------| -| **Document Input** | PDF, DOCX, Markdown, Plain Text with smart dehyphenation; file-path and UTF-8 ingest fast paths | +| **Document Input** | Text-layer PDF, Markdown, Plain Text, and beta DOCX support; file-path and UTF-8 ingest fast paths | | **Chunking** | Plain-text paragraph/line chunking with heading-aware split and tokenizer hard guard; Markdown structure-aware chunking with header-path metadata | | **Search** | HNSW vector + BM25 keyword hybrid search with RRF fusion; metadata-first search with explicit context/chunk hydration | | **Storage** | SQLite persistence, HNSW Index persistence (fast startup), connection pooling, resumable indexing | @@ -58,6 +58,8 @@ Data never leaves the user's device. Perfect for privacy-focused apps (journals, | **Performance** | Rust core, 10x faster tokenization, thread control, memory optimized | | **Context** | Engine-tokenizer exact context budget, adjacent chunk expansion, single source mode | +**Support boundaries:** text-layer PDFs are production-ready. Scanned or image-only PDFs should be routed through an OCR layer before indexing. DOCX extraction is available for early adopters, but complex DOCX layouts such as tables, headers, and footnotes should be treated as beta. + --- ## Requirements @@ -179,8 +181,17 @@ try { lower.endsWith('.markdown')) { await MobileRag.instance.addDocumentUtf8(bytes, name: fileName); } else { - final text = await DocumentParser.parse(bytes); - await MobileRag.instance.addDocument(text, name: fileName); + try { + final text = await DocumentParser.parse(bytes); + await MobileRag.instance.addDocument(text, name: fileName); + } catch (error) { + if (DocumentParser.isOcrRequiredPdfExtractionError(error)) { + throw UnsupportedError( + DocumentParser.userMessageForExtractionError(error), + ); + } + rethrow; + } } } ``` diff --git a/docs/guides/faq.md b/docs/guides/faq.md index 11904ae..ef051c6 100644 --- a/docs/guides/faq.md +++ b/docs/guides/faq.md @@ -109,7 +109,7 @@ await MobileRag.instance.rebuildIndex(); ### Q: Memory usage is high -- **Prefer file-path ingest for large local files**: Use `addDocumentFromFile(path)` for PDF, DOCX, Markdown, and text files when you have a stable path. +- **Prefer file-path ingest for large local files**: Use `addDocumentFromFile(path)` for text-layer PDFs, Markdown, and text files when you have a stable path. DOCX is beta, and scanned/image-only PDFs need OCR before indexing. - **Use UTF-8 ingest when bytes are already loaded**: Use `addDocumentUtf8(bytes)` for text/Markdown bytes instead of converting them to a Dart `String` first. - **Limit document count**: 10K+ may cause degradation. - **Use INT8 models**: Smaller models reduce memory pressure. diff --git a/docs/guides/quick_start.md b/docs/guides/quick_start.md index e6cc18d..88a0085 100644 --- a/docs/guides/quick_start.md +++ b/docs/guides/quick_start.md @@ -111,13 +111,17 @@ await MobileRag.instance.addDocument( 'Flutter is Google\'s UI toolkit for building beautiful apps.', ); -// Add PDF/DOCX or other local files by path. +// Add text-layer PDFs, beta DOCX files, or other local files by path. // This lets Rust read and chunk the file directly. await MobileRag.instance.addDocumentFromFile( 'document.pdf', name: 'document.pdf', ); +// Scanned/image-only PDFs require OCR before indexing. If extraction fails, +// use DocumentParser.isOcrRequiredPdfExtractionError(error) to route the file +// to your app's OCR flow. DOCX extraction is beta for complex layouts. + // If a stable local path is not available, fall back to bytes/text ingestion. final bytes = await File('notes.md').readAsBytes(); await MobileRag.instance.addDocumentUtf8(bytes, name: 'notes.md'); diff --git a/docs/guides/troubleshooting.md b/docs/guides/troubleshooting.md index 410d3b8..c1bf527 100644 --- a/docs/guides/troubleshooting.md +++ b/docs/guides/troubleshooting.md @@ -154,6 +154,29 @@ await MobileRag.instance.rebuildIndex(); --- +### "PDF requires OCR" + +**Symptom:** PDF ingestion fails with a message that the file may be scanned or image-based. + +**Cause:** The PDF does not contain enough extractable text for semantic indexing. Mobile RAG Engine does not run OCR internally. + +**Solution:** Route the file through your app's OCR flow, then index the recognized text: +```dart +try { + await MobileRag.instance.addDocumentFromFile(file.path, name: fileName); +} catch (error) { + if (DocumentParser.isOcrRequiredPdfExtractionError(error)) { + // Replace this with your app-specific OCR implementation. + final recognizedText = await runYourAppOcrPipeline(file); + await MobileRag.instance.addDocument(recognizedText, name: fileName); + } else { + rethrow; + } +} +``` + +--- + ### "Out of memory" on large documents **Symptom:** App crashes when processing large PDFs diff --git a/pubspec.yaml b/pubspec.yaml index ec282a1..5cb13f4 100644 --- a/pubspec.yaml +++ b/pubspec.yaml @@ -1,6 +1,6 @@ name: mobile_rag_engine description: A high-performance, on-device RAG (Retrieval-Augmented Generation) engine for Flutter. Run semantic search completely offline on iOS, Android, and macOS with HNSW vector indexing. -version: 0.18.5 +version: 0.18.6 homepage: https://github.com/dev07060/mobile_rag_engine repository: https://github.com/dev07060/mobile_rag_engine issue_tracker: https://github.com/dev07060/mobile_rag_engine/issues @@ -20,7 +20,7 @@ dependencies: flutter: sdk: flutter flutter_rust_bridge: ^2.11.1 - rag_engine_flutter: ^0.18.3 + rag_engine_flutter: ^0.18.4 path_provider: ^2.1.5 onnxruntime: ^1.4.1 freezed_annotation: ^3.1.0 diff --git a/rust_builder/CHANGELOG.md b/rust_builder/CHANGELOG.md index d46a8ca..aa73ae6 100644 --- a/rust_builder/CHANGELOG.md +++ b/rust_builder/CHANGELOG.md @@ -1,5 +1,16 @@ # Changelog +## 0.18.4 +* **PDF extraction UX**: + - Preserves the `scanned/image-only` marker for mixed scanned PDFs that also contain page-level extraction failures so host apps can route them to OCR. + - Omits the OCR marker when every page fails to extract, avoiding misleading OCR guidance for fully corrupt/unsupported PDFs. +* **Release-path validation**: + - Added i8 cosine, recall@10 floor, and cosine-fidelity safety nets for the shipped `vector_quant_i8` hot path. + - Added fail-closed CI guards for the shipped `vector_faer,vector_quant_i8` feature combo. + - Added bench-only i8 wrappers and measured i8 scan/hot-kernel baselines without changing production kernels. +* **Diagnostics**: + - Logs corrupt f32 embedding blobs with row ids during index rebuild instead of silently skipping them. + ## 0.18.3 * **PDF extraction quality**: - Preserved paragraph boundaries while normalizing PDF text so downstream chunking can split on semantic breaks instead of one long flattened body. diff --git a/rust_builder/pubspec.yaml b/rust_builder/pubspec.yaml index 8a7176d..93d1844 100644 --- a/rust_builder/pubspec.yaml +++ b/rust_builder/pubspec.yaml @@ -2,7 +2,7 @@ name: rag_engine_flutter description: > Native Rust FFI plugin for mobile_rag_engine. Provides high-performance tokenization and HNSW vector indexing for on-device RAG. -version: 0.18.3 +version: 0.18.4 homepage: https://github.com/dev07060/mobile_rag_engine repository: https://github.com/dev07060/mobile_rag_engine issue_tracker: https://github.com/dev07060/mobile_rag_engine/issues From 1937920b75dc51edf3f912d637d46244b61ba465 Mon Sep 17 00:00:00 2001 From: "Brian.oh" <49855381+dev07060@users.noreply.github.com> Date: Wed, 3 Jun 2026 04:48:19 +0900 Subject: [PATCH 4/5] feat(docs) : close 0.18.6 perf polishing notes --- docs/perf/ondevice-query-profiler/PLAN.md | 2 +- docs/perf/ondevice-query-profiler/PR-P2.md | 2 +- docs/perf/ondevice-query-profiler/PR-P3.md | 2 +- docs/perf/ondevice-query-profiler/PR-P4.md | 7 +++---- docs/perf/ondevice-query-profiler/PR-P5-1.html | 4 ++-- docs/perf/ondevice-query-profiler/README.md | 2 +- docs/perf/vector-math-refactor/PR1.md | 4 ++-- docs/perf/vector-math-refactor/PR2.md | 6 +++--- .../PR6-plan-i8-measure-parity-net.md | 16 ++++++++-------- docs/perf/vector-math-refactor/PR6.md | 2 +- docs/perf/vector-math-refactor/README.md | 2 +- docs/perf/vector-math-refactor/risk-register.md | 10 +++++----- 12 files changed, 29 insertions(+), 30 deletions(-) diff --git a/docs/perf/ondevice-query-profiler/PLAN.md b/docs/perf/ondevice-query-profiler/PLAN.md index 7dbe2e2..6abe242 100644 --- a/docs/perf/ondevice-query-profiler/PLAN.md +++ b/docs/perf/ondevice-query-profiler/PLAN.md @@ -20,7 +20,7 @@ | P2 | example integration_test wiring + A/B fixture builder | LOC-67 | device | | P3 | Segment timing loop + 3 scenarios + query_metrics snapshot | LOC-68 | device | | P4 | JSON/CSV export to app-docs + structured log + run metadata | LOC-69 | device + host(serialize) | -| P5 | (conditional) Phase-2 drill-down per dominant bucket | LOC-70 | TBD by P3/P4 data | +| P5 | Phase-2 quality/latency drill-down | LOC-70 | P5-① recall complete; P5-②~④ deferred after 0.18.6 | Each PR: branch from main, commit (no Claude attribution), open PR, CI green (`cargo test -- --test-threads=1` unaffected — Dart-only), **user merges**. Fill `PRn.md` + README row before merge. diff --git a/docs/perf/ondevice-query-profiler/PR-P2.md b/docs/perf/ondevice-query-profiler/PR-P2.md index 30755cd..6e7ad55 100644 --- a/docs/perf/ondevice-query-profiler/PR-P2.md +++ b/docs/perf/ondevice-query-profiler/PR-P2.md @@ -2,7 +2,7 @@ - 브랜치: `feat/loc-67-profiler-fixture` - Linear: [LOC-67](https://linear.app/loceract/issue/LOC-67) -- 상태: 🟦 진행 (PR 열림, 기기 스모크 green) +- 상태: 🟩 머지(#71, 기기 스모크 green) ## 스코프 실 ONNX 자산이 있는 example 앱에서 구동하는 디바이스 프로파일러의 기반. diff --git a/docs/perf/ondevice-query-profiler/PR-P3.md b/docs/perf/ondevice-query-profiler/PR-P3.md index 214dc27..182dfad 100644 --- a/docs/perf/ondevice-query-profiler/PR-P3.md +++ b/docs/perf/ondevice-query-profiler/PR-P3.md @@ -2,7 +2,7 @@ - 브랜치: `feat/loc-68-profiler-timing` - Linear: [LOC-68](https://linear.app/loceract/issue/LOC-68) -- 상태: 🟦 진행 (PR 열림 예정, iPhone 실기 profile 런 green) +- 상태: 🟩 머지(#72, iPhone 실기 profile 런 green) ## 스코프 실 ONNX 자산 example 앱에서 쿼리 단계를 **격리 계측**하는 디바이스 프로파일러 본체. diff --git a/docs/perf/ondevice-query-profiler/PR-P4.md b/docs/perf/ondevice-query-profiler/PR-P4.md index 1fa6512..a45ee1a 100644 --- a/docs/perf/ondevice-query-profiler/PR-P4.md +++ b/docs/perf/ondevice-query-profiler/PR-P4.md @@ -2,7 +2,7 @@ - 브랜치: `feat/loc-69-profiler-export` (P3 위에 스택) - Linear: [LOC-69](https://linear.app/loceract/issue/LOC-69) -- 상태: 🟦 진행 (PR 열림 예정, iPhone 실기 profile 런 green) — **baseline 산출 = 1차 목표 달성** +- 상태: 🟩 머지(#75 rescue, iPhone 실기 profile 런 green) — **baseline 산출 = 1차 목표 달성** ## 스코프 - `example/lib/profiling/profile_export.dart` — 리포트를 앱 documents dir에 `query_profile_.json/.csv`로 flush 기록 + 실행당 `PROFILE` 로그 1줄 + dir/파일명 로그(추출용). 물리기기 추출은 Xcode Download Container 또는 `xcrun devicectl`(simctl은 시뮬전용). @@ -26,7 +26,7 @@ I/O(query_metrics): full_hydrate_rows pure_warm 300 / filtered 90 / pure_cold 10 ## ⚠️ 측정 범위 (헤드라인 정정 — latency ≠ quality) - 본 baseline은 **지연(latency) 분해**만 측정한다. **검색 품질(recall)은 측정하지 않는다.** - LOC-64의 `recall@10 = 0.997`은 **i8 exact-scan vs f32 전수조사의 수학적 충실도**(양자화 커널 정확도)일 뿐, **실제 출시 경로의 HNSW 그래프 e2e 하이브리드 리콜이 아니다** — HNSW는 `dequantize_i8_to_f32`로 복원한 찌그러진 벡터 공간 위에서 그래프를 탐색하므로 별개. 즉 "i8가 빠르다 + 0.997"을 "출시 검색 품질이 우수하다"로 읽으면 안 된다. -- **실제 프로덕션 e2e 하이브리드 리콜은 미측정** → **P5(LOC-70)의 1순위 타깃**(폰에서 `[f32 순수 원본 전수조사 top-K]` 대비 `[i8-HNSW + BM25 RRF top-K]` 교집합률 산출). 90% 미만이면 모바일용 `M`/`ef_search` 상향 튜닝 필요. +- **후속 업데이트:** P5-①에서 실제 프로덕션 e2e 리콜을 측정했고, vector-only recall@10=1.00으로 HNSW/i8 품질 게이트를 통과했다. hybrid recall@10=0.08은 순수-vector GT 대비 BM25/RRF reorder 진단으로 해석한다. ## 받은 피드백 / 한계 - 어드버서리얼 리뷰 HIGH: 초기 추출 안내가 simctl(시뮬전용)이었음 → 물리기기는 Xcode Download Container / `devicectl`로 정정. @@ -35,5 +35,4 @@ I/O(query_metrics): full_hydrate_rows pure_warm 300 / filtered 90 / pure_cold 10 ## 리스크 / 롤백 / 다음 - 동작 코드 변경 없음(example 프로파일링 export). 롤백: PR revert. -- ⚠️ **스택 PR**: P3(#LOC-68) 위에 스택. P3 먼저 머지 후 본 PR을 main으로 retarget(아니면 orphan). -- 다음: **P5(LOC-70)** Phase-2 — warm은 embed(ONNX) 지배라 Rust 벡터 작업 불요; cold가 중요하면 activate(bm25_rebuild vs hnsw_load) 분해. 데이터 게이트 충족. +- P5-① 품질 게이트 완료 후 남은 P5-②~④는 0.18.6 이후의 측정 심화 작업으로 보류. diff --git a/docs/perf/ondevice-query-profiler/PR-P5-1.html b/docs/perf/ondevice-query-profiler/PR-P5-1.html index 85df771..cfd8f06 100644 --- a/docs/perf/ondevice-query-profiler/PR-P5-1.html +++ b/docs/perf/ondevice-query-profiler/PR-P5-1.html @@ -291,8 +291,8 @@

Recommended Next Steps

Next - Proceed to P5-2 activate breakdown. - The vector-only quality gate passed, while cold activate remains the latency gate. + Defer P5-2 activate breakdown until after 0.18.6. + The vector-only quality gate passed; cold activate remains the next latency gate, but not a release blocker. Later diff --git a/docs/perf/ondevice-query-profiler/README.md b/docs/perf/ondevice-query-profiler/README.md index 1f84021..0124369 100644 --- a/docs/perf/ondevice-query-profiler/README.md +++ b/docs/perf/ondevice-query-profiler/README.md @@ -27,7 +27,7 @@ vector_math 커널 슬라이스는 거의 최적임을 확인했으나 **온디 | P3 | 세그먼트 타이밍 + 3시나리오 + metrics 스냅샷 | [LOC-68](https://linear.app/loceract/issue/LOC-68) | 🟩 머지(#72, [PR-P3.md](PR-P3.md), 기기 green) | | P4 | JSON/CSV export + 로그 + 메타 (baseline 산출) | [LOC-69](https://linear.app/loceract/issue/LOC-69) | 🟩 머지(#75 rescue, [PR-P4.md](PR-P4.md), 기기 green) | | P5-① | e2e hybrid recall@10 — 품질 | [LOC-70](https://linear.app/loceract/issue/LOC-70) | 🟩 완료([PR-P5-1.html](PR-P5-1.html), vector-only=1.00 / hybrid=0.08) | -| P5-②~④ | activate 분해 / 동시성 / SQLite scale | [LOC-70](https://linear.app/loceract/issue/LOC-70) | ⏭ 다음 순서 | +| P5-②~④ | activate 분해 / 동시성 / SQLite scale | [LOC-70](https://linear.app/loceract/issue/LOC-70) | ⏸ 0.18.6 이후 측정 심화 | ## 규약 (프로젝트 공통) - CI: `cargo test -- --test-threads=1`. 커밋/PR에 Claude 귀속 미포함. PR은 열고 CI green까지만, 머지는 본인. diff --git a/docs/perf/vector-math-refactor/PR1.md b/docs/perf/vector-math-refactor/PR1.md index 030bbea..ba36819 100644 --- a/docs/perf/vector-math-refactor/PR1.md +++ b/docs/perf/vector-math-refactor/PR1.md @@ -2,7 +2,7 @@ - 브랜치: `feat/loc-59-vector-math-bench` - Linear: [LOC-59](https://linear.app/loceract/issue/LOC-59) -- 상태: 🟦 진행 (PR 열림, CI green 대기) +- 상태: 🟩 머지(#64) ## 스코프 (무엇을/왜) faer 삭제 **전에** 성능 베이스라인을 캡처하고, fused가 faer와 등가임을 박아 PR2를 안전화하기 위함. @@ -42,5 +42,5 @@ faer 삭제 **전에** 성능 베이스라인을 캡처하고, fused가 faer와 - 벤치/패리티는 CI에서 안 돎(`bench`·`vector_faer` 미설정) → 로컬 실행/기록이 본 PR의 산출. ## 결정 로그 -- **PR2 전제 반증** → PR2를 “faer 제거”에서 **“faer 유지 + N2(CI 커버리지) [+ 선택 N1 무할당화]”로 피벗** 후보. 사용자 결정 대기. +- **PR2 전제 반증** → PR2를 “faer 제거”에서 **“faer 유지 + N2(CI 커버리지)”로 피벗**했고, PR2(#65)에서 완료. - 디바이스 캐비엇: 위 수치는 개발기(Apple Silicon). 방향(faer 우위)은 스칼라-vs-SIMD 차이라 폰 NEON에서도 견고할 것으로 예상하나, 크기는 온디바이스 프로파일로 확인 권장. diff --git a/docs/perf/vector-math-refactor/PR2.md b/docs/perf/vector-math-refactor/PR2.md index c4dcb54..519eacf 100644 --- a/docs/perf/vector-math-refactor/PR2.md +++ b/docs/perf/vector-math-refactor/PR2.md @@ -2,7 +2,7 @@ - 브랜치: `feat/loc-60-faer-ci-coverage` - Linear: [LOC-60](https://linear.app/loceract/issue/LOC-60) -- 상태: 🟦 진행 (PR 열림, CI green 대기) +- 상태: 🟩 머지(#65) ## 재설계 배경 PR1([PR1.md](PR1.md)) 벤치가 원안("faer 제거 → fused 통일")의 전제를 반증 — faer가 2–8× 빠름. @@ -22,10 +22,10 @@ PR1([PR1.md](PR1.md)) 벤치가 원안("faer 제거 → fused 통일")의 전제 - N2 갭: **닫힘** — 출시 faer+quant 경로가 CI에서 빌드 + 테스트됨. - 로컬 검증: `bash -n` OK; faer 테스트 4개(패리티 포함) green, fail-closed PASS; `cargo build --release --features vector_faer,vector_quant_i8` 성공(50s, 기존 dead_code 경고만). -- CI: (PR #__ green 후 갱신) +- CI: PR #65 green 후 머지. ## 받은 피드백 (리뷰) -- (PR 리뷰 후 갱신) +- faer 제거 대신 출시 faer 백엔드를 CI에서 검증하는 방향으로 피벗했고, PR1 측정 결과와 일치. ## 리스크 / 롤백 - CI native 잡 시간 증가(faer 2회 컴파일: test+release 프로파일). 게이트 가치 대비 수용. diff --git a/docs/perf/vector-math-refactor/PR6-plan-i8-measure-parity-net.md b/docs/perf/vector-math-refactor/PR6-plan-i8-measure-parity-net.md index e7adc22..0b2d19a 100644 --- a/docs/perf/vector-math-refactor/PR6-plan-i8-measure-parity-net.md +++ b/docs/perf/vector-math-refactor/PR6-plan-i8-measure-parity-net.md @@ -550,27 +550,27 @@ git commit -m "ci(vector_quant): run i8 ε/recall/fidelity nets on shipped faer+ - [ ] **Step 1: Create `PR6.md` with the measured results** -Create `docs/perf/vector-math-refactor/PR6.md` (fill `<...>` from Task 2 Step 5 and Task 3 Step 4): +Create `docs/perf/vector-math-refactor/PR6.md` with the measured values from Task 2 Step 5 and Task 3 Step 4: ```markdown # PR6 — i8 출시 핫패스 측정 + ε/recall/fidelity 안전망 (N: 측정 먼저) - 브랜치: `feat/loc-64-i8-measure-parity-net` - Linear: [LOC-64](https://linear.app/loceract/issue/LOC-64) -- 상태: 🟦 진행 (PR 열림, CI green 대기) +- 상태: 🟩 머지(#68) - 설계: [PR6-spec-i8-measure-parity-net.md](PR6-spec-i8-measure-parity-net.md) ## 스코프 (비파괴 — 커널/양자화 0줄 변경) 출시 핫패스(i8 `cosine_with_query_norm_i8_blob`)에 PR1 패턴 적용: 측정 + 수치 ε 네트 + recall@k floor + 코사인 fidelity 네트 + CI fail-closed. -## 결과 (측정) -- **i8 핫커널 마이크로벤치** (dim별, ns): 384=<...> / 768=<...> / 1024=<...> / 1536=<...> -- **스캔(2000×768) 비교**: `exact_scan[faer]`(f32 decode+cosine)=<...> µs vs `exact_scan_i8`(i8 blob)=<...> µs → i8가 f32-faer 대비 **<...>×**. +## 결과 (측정, dev arm64) +- **i8 핫커널 마이크로벤치** (ns): 384=7.87 / 768=14.97 / 1024=21.12 / 1536=31.28 +- **스캔(2000×768) 비교**: `exact_scan[faer]`(f32 decode+cosine)=452.82 µs vs `exact_scan_i8`(i8 blob)=29.98 µs → i8가 f32-faer 대비 **≈15.1×**. - **수치 ε 네트**: 차원 {1,2,3,16,384,768,1024,1536}에서 kernel ≈ f64 참조, ε=1e-4 green. - **핵심 발견**: i8 per-vector 양자화는 768d에서 **recall@10 ≈ 0.997**(=319/320, 거의 무손실) — '민감 밴드'는 출시 설정에서 도달 불가이며 강제 시 비대표적. 따라서 게이트는 이 높은 baseline을 잠금. - **recall@k floor 네트**: N=2000, Q=32, dim=768, k=10, clusters=16 → 측정 recall@10 = **0.996875**(dev arm64, CI 확인), FLOOR = **0.98** (= floor(X−0.02)). GT는 f64(플랫폼 jitter 제거), 전순서 `(score desc, index asc)`. - **코사인 fidelity 네트**: `max|cosine_i8 − cosine_f32_true|` = **0.00121**, 게이트 = **0.005** (≈4× baseline). 완전 결정론적·민감. -- **CI**: `--features "vector_quant_i8,vector_faer" -- --test-threads=1` fail-closed + 3개 네트 이름별 가드 (N=<...> passed). +- **CI**: `--features "vector_quant_i8,vector_faer" -- --test-threads=1` fail-closed + 3개 네트 이름별 가드, 7 passed. ## 받은 피드백 (리뷰 / 사전검증) - 설계 리뷰: N=2000/k=10, 전순서 타이브레이크, 출시 트리(faer+quant) CI. @@ -591,7 +591,7 @@ Create `docs/perf/vector-math-refactor/PR6.md` (fill `<...>` from Task 2 Step 5 In `docs/perf/vector-math-refactor/README.md`, add after the PR5 row line: ```markdown -| PR6 | i8 출시 핫패스 **측정 + ε/recall/fidelity 안전망** | i8 검증갭 | 낮음(비파괴) | main(#67) | [LOC-64](https://linear.app/loceract/issue/LOC-64) | 🟦 진행([PR6.md](PR6.md)) | +| PR6 | i8 출시 핫패스 **측정 + ε/recall/fidelity 안전망** | i8 검증갭 | 낮음(비파괴) | main(#67) | [LOC-64](https://linear.app/loceract/issue/LOC-64) | 🟩 머지(#68, [PR6.md](PR6.md)) | ``` - [ ] **Step 3: Commit** @@ -649,6 +649,6 @@ Expected: all checks pass. **Do NOT merge** — report "PR opened, CI green" and ## Self-Review (filled by plan author) - **Spec coverage**: §3 bench → Task 3; §4 ε net → Task 1; §5 quality net → Task 2 (recall floor + fidelity, per the approved Net-2 redesign); §6 CI → Task 4; §7 tracking → Task 5 + issue created; §8 acceptance → Tasks 1–6; non-goal (0 kernel changes) → only `mod tests`, `bench_api`, `benches`, CI, docs touched. ✅ (Spec §5/§8/§9/§10 updated to the recall-floor + fidelity design.) -- **Placeholders**: `MIN_RECALL`/`MAX_COS_ERR`/bench numbers are *measure-first outputs* with exact derivation + compile-time `const _` guards (Task 2 Steps 4–5), not vague TODOs. PR6.md `<...>` are explicitly "fill from measured results." ✅ +- **Placeholders**: `MIN_RECALL`/`MAX_COS_ERR`/bench numbers are *measure-first outputs* with exact derivation + compile-time `const _` guards (Task 2 Steps 4–5), not vague TODOs. The final PR6 journal uses measured values, not placeholders. ✅ - **Type consistency**: `order_desc` used on both `(usize,f64)` (GT) and `(usize,f32)` (i8); `cosine_f64_true`/`ref_cosine_i8_f64`/`clustered_corpus`/`quantize_f32_to_i8`/`l2_norm_i8`/`i8_blob_from_slice`/`cosine_with_query_norm_i8_blob` match real `vector_quant.rs` signatures (verified). bench_api wrappers match. ✅ - **Verification-driven fixes applied**: f64 GT (jitter), recall-floor not forced-band (saturation), const guards + CI name guards (vacuous gate), `pub(crate) mod` wording, flat `X−0.02` floor. ✅ diff --git a/docs/perf/vector-math-refactor/PR6.md b/docs/perf/vector-math-refactor/PR6.md index 2a3588e..8dd6805 100644 --- a/docs/perf/vector-math-refactor/PR6.md +++ b/docs/perf/vector-math-refactor/PR6.md @@ -2,7 +2,7 @@ - 브랜치: `feat/loc-64-i8-measure-parity-net` - Linear: [LOC-64](https://linear.app/loceract/issue/LOC-64) -- 상태: 🟦 진행 (PR 열림, CI green 대기) +- 상태: 🟩 머지(#68) - 설계: [PR6-spec-i8-measure-parity-net.md](PR6-spec-i8-measure-parity-net.md) · 계획: [PR6-plan-i8-measure-parity-net.md](PR6-plan-i8-measure-parity-net.md) ## 스코프 (비파괴 — 커널/양자화 0줄 변경) diff --git a/docs/perf/vector-math-refactor/README.md b/docs/perf/vector-math-refactor/README.md index 679a201..604b8b0 100644 --- a/docs/perf/vector-math-refactor/README.md +++ b/docs/perf/vector-math-refactor/README.md @@ -29,7 +29,7 @@ | PR3 | decode 버퍼 재사용 | Claim1 | 낮음~중 | 벤치/N3 게이트 | [LOC-61](https://linear.app/loceract/issue/LOC-61) | ❌ 폐기(출시 i8 빌드서 f32 decode 비핫, 코드검증) | | PR4 | ~~다중 누산기 언롤~~ | — | — | — | [LOC-62](https://linear.app/loceract/issue/LOC-62) | ❌ 폐기(faer 유지로 무의미) | | PR5 | 위생: 손상 로깅(N6) + 엔디안 문서화(N5) | N6, N5 | 낮음(독립) | — | [LOC-63](https://linear.app/loceract/issue/LOC-63) | 🟩 머지(#66, [PR5.md](PR5.md)) | -| PR6 | i8 출시 핫패스 **측정 + ε/recall/fidelity 안전망** | i8 검증갭 | 낮음(비파괴) | main(#67) | [LOC-64](https://linear.app/loceract/issue/LOC-64) | 🟦 진행([PR6.md](PR6.md)) | +| PR6 | i8 출시 핫패스 **측정 + ε/recall/fidelity 안전망** | i8 검증갭 | 낮음(비파괴) | main(#67) | [LOC-64](https://linear.app/loceract/issue/LOC-64) | 🟩 머지(#68, [PR6.md](PR6.md)) | 종료 회고: [RETRO.md](RETRO.md) · PR3([LOC-61](https://linear.app/loceract/issue/LOC-61)) ❌ 폐기 확정(RETRO §5) · 잔여(선택): 온디바이스 벤치 / encode 헬퍼 dedup — 프로젝트 핸드오프 노트 참조. diff --git a/docs/perf/vector-math-refactor/risk-register.md b/docs/perf/vector-math-refactor/risk-register.md index eb8a20e..ea473d5 100644 --- a/docs/perf/vector-math-refactor/risk-register.md +++ b/docs/perf/vector-math-refactor/risk-register.md @@ -9,12 +9,12 @@ | R2 | release 수치 ~1e-6 변동으로 **결과 순서** 변동 | PR2 | 회귀 테스트에서 top_k 순서 차이 | 코드근거상 컷오프 의존 0 (분석 §3). PR1 패리티(ε)로 등가 증명 | PR2 revert | 🟩 닫힘 — faer 유지로 백엔드 교체 없음 | | R3 | 고차원·대량 linear scan에서 **스칼라 throughput-bound** | PR2 후 | 벤치에서 1024+ 차원 회귀 | PR4 다중 누산기 언롤 | PR4 미적용 시 기존 fused 유지 | 🟥 PR1 확정(fused가 faer 대비 2–8× 느림) | | **R9** | **PR2 전제 반증**: faer 제거가 핫 패스를 회귀시킴(faer가 2–8× 빠름) | PR1 | PR1 벤치 (exact_scan 2.8×, dot 최대 7.8×) | PR2를 “faer 유지 + N2”로 피벗(완료) | — | 🟩 닫힘 — faer 유지 확정, PR2=N2 CI 커버리지 | -| **N2** | 출시 faer+quant 백엔드가 **CI에서 빌드/테스트 0회** | 상시(출시) | release만 faer 활성, CI는 default/debug | PR2: faer 테스트 + 출시 조합 릴리스 빌드 게이트 | test_ci.sh revert | 🟨 PR2에서 닫는 중 | -| R4 | PR4 **언롤 새 버그**(remainder/lane 합산 off-by-one) | PR4 | 패리티 테스트 ε 초과 / 단위 테스트 실패 | 스칼라 대비 property/ε 테스트 필수 | PR4 revert(커널은 PR2 상태로) | ⬜ 열림 | -| R5 | **스택 PR 고아화** (PR4/PR5 base=PR2) | PR2 머지 | PR4/PR5 diff에 PR2 변경분이 섞여 보임 | PR2 머지 직후 base→main retarget | 재타깃 PR | ⬜ 열림 | -| R6 | PR3 핫 루프 수정으로 **동작 변경** | PR3 | 검색 결과 diff | 결과 동일성 테스트 + N3로 적용 여부 게이트 | PR3 revert | ⬜ 열림 | +| **N2** | 출시 faer+quant 백엔드가 **CI에서 빌드/테스트 0회** | 상시(출시) | release만 faer 활성, CI는 default/debug | PR2: faer 테스트 + 출시 조합 릴리스 빌드 게이트 | test_ci.sh revert | 🟩 닫힘(PR2) | +| R4 | PR4 **언롤 새 버그**(remainder/lane 합산 off-by-one) | PR4 | 패리티 테스트 ε 초과 / 단위 테스트 실패 | 스칼라 대비 property/ε 테스트 필수 | PR4 revert(커널은 PR2 상태로) | 🟩 닫힘 — PR4 폐기 | +| R5 | **스택 PR 고아화** (PR4/PR5 base=PR2) | PR2 머지 | PR4/PR5 diff에 PR2 변경분이 섞여 보임 | PR2 머지 직후 base→main retarget | 재타깃 PR | 🟩 닫힘 — main 기반 PR로 회피 | +| R6 | PR3 핫 루프 수정으로 **동작 변경** | PR3 | 검색 결과 diff | 결과 동일성 테스트 + N3로 적용 여부 게이트 | PR3 revert | 🟩 닫힘 — PR3 폐기 | | R7 | N5 엔디안 변경이 **기존 DB 호환성** 깨짐 | PR5 | 기존 인덱스 로드 실패 | LE 기기선 ne==le 바이트 동일 → 무마이그레이션 확인 후만 적용 | 변경 revert | 🟩 닫힘 — PR5는 포맷 미변경(문서화만), 호환 리스크 없음 | | N6 | 손상 임베딩 블롭이 **무음 드롭** | 상시 | 손상 DB에서 인덱스 누락이 로그 없이 발생 | PR5: `decode_f32_embedding_or_warn`로 `log::warn!` 가시화(동작 보존) | 헬퍼 revert | 🟩 닫힘(PR5) | -| R8 | CI 병렬 실행으로 **공유 SQLite 테스트 플레이크** | 전체 | cargo test 간헐 실패 | `-- --test-threads=1` 고정 | — | ⬜ 열림 | +| R8 | CI 병렬 실행으로 **공유 SQLite 테스트 플레이크** | 전체 | cargo test 간헐 실패 | `-- --test-threads=1` 고정 | — | 🟨 관찰중 — 테스트 명령 고정으로 완화 | 상태 범례: ⬜ 열림 · 🟨 관찰중 · 🟩 닫힘(해소) · 🟥 발생함 From 36a6f620ae28968d075732d7c9422c7f92d8b2b2 Mon Sep 17 00:00:00 2001 From: "Brian.oh" <49855381+dev07060@users.noreply.github.com> Date: Wed, 3 Jun 2026 11:33:17 +0900 Subject: [PATCH 5/5] feat(deps) : lock published rag_engine_flutter 0.18.4 --- example/pubspec.lock | 4 ++-- pubspec.lock | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/example/pubspec.lock b/example/pubspec.lock index b978a9b..f916b67 100644 --- a/example/pubspec.lock +++ b/example/pubspec.lock @@ -229,7 +229,7 @@ packages: path: ".." relative: true source: path - version: "0.18.5" + version: "0.18.6" onnxruntime: dependency: transitive description: @@ -324,7 +324,7 @@ packages: path: "../rust_builder" relative: true source: path - version: "0.18.3" + version: "0.18.4" sky_engine: dependency: transitive description: flutter diff --git a/pubspec.lock b/pubspec.lock index 3ac7826..02e87a7 100644 --- a/pubspec.lock +++ b/pubspec.lock @@ -479,10 +479,10 @@ packages: dependency: "direct main" description: name: rag_engine_flutter - sha256: e4e4481242648807058b51bad34e891d9593cf0f720e5f77f2c26deef0d4b23a + sha256: ab4d9bcf9e63d5b6a83ffb25a6db9a240caa9b19b18d667d08913d8d293cc645 url: "https://pub.dev" source: hosted - version: "0.18.3" + version: "0.18.4" shelf: dependency: transitive description: