OpenIntelligence

Documentation status: Verified for OpenIntelligence v4.1 on June 14, 2026. Scope: Describes shipped behavior for on-device Apple Intelligence RAG architecture.

Local-first document intelligence for macOS and iOS, featuring an entirely on-device Retrieval-Augmented Generation (RAG) pipeline and native Apple Foundation Models integration.

OpenIntelligence is an exploratory, privacy-obsessed document query assistant built natively for Apple platforms. It proves that you can run production-grade document ingestion, vector indexing, lexical retrieval, and generative AI entirely on device without sacrificing privacy or relying on third-party cloud wrappers.

---

📚 Rigorous Engineering Documentation

OpenIntelligence is backed by extensive, rigorous engineering documentation. If you want to understand how we achieved reliable, hallucination-resistant on-device RAG using Apple's 4K-token local context windows, start here:

Core Architecture & Systems

• System Architecture: The high-level view of the decoupled import-time and query-time pipelines.
• Retrieval Pipeline (RETRIEVAL_PIPELINE.md): Deep dive into our hybrid search engine (BM25 + Core ML Vector) and Reciprocal Rank Fusion implementation.
• Ingestion Pipeline (INGESTION_PIPELINE.md): How we chunk documents, run local Vision OCR fallbacks, and structure semantic data.
• Privacy & Routing (PRIVACY_AND_ROUTING.md): Our strict local-first data guarantees and routing protocols.

Apple Intelligence Engineering Specs

• Apple Foundation Models Specs: How we optimize for macOS/iOS 26.x and WWDC26 betas, managing 4K token budgets and SystemLanguageModel sessions.
• Apple Document Intelligence: Detailed usage of Vision, PDFKit, and CoreText for semantic extraction.
• Private Cloud Compute (PCC): Documentation on Apple's PCC architecture constraints and secure enclave routing.
• RAG Technical Specifications: Hard math behind our vector dimensions (384), semantic evidence overlap index (0.70), and confidence gates.
• Storage & Pipeline Trace: How SQLite FTS5 and BNNS-accelerated vectors run at the metal level.

Audits & Constraints

• Hard Limits: A transparent look at what the engine can't do and where memory bottlenecks occur.
• Current State & Gaps: Ongoing challenges with local LLM hallucination and pipeline latency.
• Evaluation Framework: How we run local scripts/run_rag_benchmarks.py to continuously validate output quality.

---

⚙️ Technical Architecture Overview

The runtime operates in two decoupled phases:

flowchart TD
  subgraph INGEST[Import-Time Pipeline]
    A1[Import files]
    A2[Extract & normalize (Vision OCR)]
    A3[Semantic Chunking]
    A4[Build FTS5 & BNNS vector indexes]
    A1 --> A2 --> A3 --> A4
  end

  subgraph QUERY[Query-Time Pipeline]
    B1[User query]
    B2[Analyze intent & HyDE expansion]
    B3[Hybrid Retrieval & RRF merge]
    B4[Cross-encoder reranking]
    B5[Verification Gates]
    B6[Generative LLM Response]
    B1 --> B2 --> B3 --> B4 --> B5 --> B6
  end

  A4 --> B3

Loading

---

🗺️ Codebase Map

Module	Core Files	Responsibility
Ingestion	DocumentProcessor.swift, LayoutAwareExtractor.swift	Document content extraction, Vision OCR fallback, semantic structure recovery.
Chunking	SemanticChunker.swift, ContentTaggingService.swift	Context-aware document chunking, entity resolution, NLP metadata enrichment.
Indexing	SQLiteFullTextService.swift, BNNSVectorDatabase.swift	Blazing-fast SQLite FTS5 lexical storage and local BNNS-accelerated vector indexing.
Retrieval	HybridSearchService.swift, ContextPackingService.swift	BM25 + Vector hybrid merging, parent-chunk reconstruction, exact token packing.
Orchestration	LLMService.swift, RAGService.swift	Execution coordination with the local SystemLanguageModel and evaluation loops.
Shortcuts	RAGAppIntents.swift	Siri integration and entity-native App Intents for OS-level query capabilities.

---

🛠️ Placeholders & Scaffolding Warnings

To maintain codebase transparency, please note:

• Core AI Integration: Disabled via #if false directives in CoreAISentenceEmbeddingProvider.swift. The project currently runs on reliable local CoreMLSentenceEmbeddingProvider implementations until the OS 27 beta stabilizes.
• Private Cloud Compute (PCC): Routed locally using a fallback system language model wrapper in EngineSDKCompatibility.swift to ensure compilability on current public SDKs.

---

🚀 Build & Verification

Requirements:

• macOS Tahoe (26.x) with Xcode 26+
• iOS 26.0+ SDK target support
• Apple Silicon (M1+ / A17 Pro+) for adequate Neural Engine throughput

Instructions:

1. Clear macOS extended attributes to prevent codesign failure:

   /usr/bin/xattr -cr /Users/gunnarhostetler/Documents/GitHub/OpenIntelligence-Public/OpenIntelligence

2. Compile the simulator smoke target:

   ./scripts/build_simulator_smoke.sh

3. Execute the local RAG pipeline validation harness:

   python3 scripts/run_rag_benchmarks.py

---

License

OpenIntelligence is open-source software. See LICENSE for details.