RAG Chunking Experiments

Exploration of structurally and semantically aware chunking strategies for Retrieval-Augmented Generation (RAG) systems.

Implemented Methods

• Naive sentence chunking
• Semantic valley chunking

Planned Methods

• Spectral graph segmentation
• Heat kernel segmentation
• Information bottleneck chunking

Goals

Evaluate how chunking strategies affect retrieval quality in RAG pipelines.

Stack

• Python
• Sentence Transformers
• NumPy
• HuggingFace datasets