PDF Chatbot

A full-stack application for conversational Q&A over PDF documents, built with a hybrid RAG (Retrieval-Augmented Generation) pipeline.

Upload a PDF, and the system chunks it, builds dual search indexes (BM25 + FAISS), reranks results with a Cross-Encoder, and streams answers from an LLM — all with persistent conversation history.

Architecture

flowchart LR
    User([Browser]) --> Frontend

    subgraph Frontend ["Frontend (Svelte 5 / Vite)"]
        UI[Chat UI]
    end

    subgraph Backend ["Backend (FastAPI)"]
        API[REST API]
        Ingest[Ingestion]
        Retrieval[Hybrid Retrieval]
        Chain[LangChain Pipeline]
    end

    subgraph Storage
        SQLite[(SQLite)]
        FAISS[(FAISS)]
        BM25[(BM25)]
    end

    subgraph Models
        Embed["bge-small-en-v1.5\n(Local Embeddings)"]
        Rerank["ms-marco-MiniLM-L-6-v2\n(Local Reranker)"]
        LLM["Llama 3.3 70B\n(Groq API)"]
    end

    Frontend <-->|REST| API
    API --> Ingest --> Embed --> FAISS
    Ingest --> BM25
    API --> Retrieval --> FAISS
    Retrieval --> BM25
    Retrieval --> Rerank
    Retrieval --> Chain
    API --> SQLite
    Chain <--> LLM

Loading

Features

• Hybrid Retrieval — Combines BM25 keyword search with FAISS vector search (MMR) via EnsembleRetriever (0.6/0.4 weighting)
• Cross-Encoder Reranking — Reranks merged results using ms-marco-MiniLM-L-6-v2 before sending to the LLM
• Streaming Responses — Token-by-token streaming from Groq via StreamingResponse
• Conversation Persistence — Chat history stored in SQLite via SQLModel, survives restarts
• Performance Logging — Request timing middleware, TTFT (Time to First Token), and total generation time logged per request
• Docker Ready — Multi-stage frontend build, CPU-optimized backend, single docker compose up

Tech Stack

Layer	Technology
Frontend	Svelte 5, Vite, Tailwind CSS
Backend	FastAPI, SQLModel, SQLite
Orchestration	LangChain
Embeddings	HuggingFace BAAI/bge-small-en-v1.5 (local)
Reranker	HuggingFace Cross-Encoder ms-marco-MiniLM-L-6-v2 (local)
LLM	Groq API (Llama 3.3 70B Versatile)
Vector Store	FAISS (local, persisted to disk)
Keyword Search	BM25 via rank-bm25 (serialized with pickle)

Project Structure

PdfChatbot/
├── backend/
│   ├── src/
│   │   ├── app.py           # FastAPI routes, middleware, streaming
│   │   ├── chains.py        # LangChain LCEL pipeline construction
│   │   ├── config.py        # Model initialization (LLM, embeddings, reranker)
│   │   ├── db.py            # SQLModel engine and session dependency
│   │   ├── ingestion.py     # PDF chunking, FAISS/BM25 indexing, retriever assembly
│   │   ├── models.py        # Conversation and Message database models
│   │   └── prompts.py       # Prompt templates
│   ├── main.py              # Dev entry point (uvicorn with reload)
│   ├── Dockerfile
│   ├── requirements.txt
│   └── .env.example
├── frontend/
│   ├── src/
│   │   ├── App.svelte       # Root layout, SPA routing
│   │   ├── lib/
│   │   │   ├── Chat.svelte  # Message display, streaming, auto-scroll
│   │   │   ├── Sidebar.svelte
│   │   │   ├── Navbar.svelte
│   │   │   ├── Upload.svelte # Drag-and-drop PDF upload
│   │   │   └── config.js    # API base URL configuration
│   │   └── app.css
│   ├── Dockerfile
│   └── package.json
├── docker-compose.yaml
└── README.md

Getting Started

Prerequisites

• Python 3.12+
• Node.js 20+ with pnpm
• A Groq API key

Local Development

Backend:

cd backend
python -m venv .venv
.venv\Scripts\activate        # Linux/macOS: source .venv/bin/activate
pip install -r requirements.txt

Create backend/.env:

GROQ_API_KEY=your_key_here

Start the server:

python main.py
# API available at http://localhost:8000

Frontend:

cd frontend
pnpm install
pnpm run dev
# UI available at http://localhost:5173

Docker

# Set your API key
export GROQ_API_KEY=your_key_here   # PowerShell: $env:GROQ_API_KEY="your_key_here"

docker compose up --build
# Frontend at http://localhost, Backend at http://localhost:8080

Note on PyTorch

The backend Dockerfile installs PyTorch CPU-only wheels (--index-url https://download.pytorch.org/whl/cpu). This keeps the image ~1.5 GB smaller than the default CUDA-enabled package. The Cross-Encoder and embedding models run efficiently on CPU for this workload.

If you are running locally with a GPU and want faster reranking, install the default PyTorch package instead:

pip install torch   # Installs with CUDA support if available

API Endpoints

Method	Endpoint	Description
GET	/	List all conversations (sorted by creation date)
POST	/upload	Upload a PDF and create a conversation
GET	/chat/{id}	Retrieve messages for a conversation
POST	/chat/{id}	Send a message (streams response as text/plain)

Future Improvements

• Multi-document chat — query across multiple PDFs in one conversation
• User authentication — per-user conversation isolation
• Cloud storage — migrate uploads from local filesystem to S3/GCS
• Managed vector database — replace FAISS with Pinecone, Weaviate, or Qdrant
• GPU inference — deploy reranker and embeddings on GPU for lower latency

License

MIT