|
| 1 | +# VIBE ANNS Benchmark Reproduction with DEG |
| 2 | + |
| 3 | +This example reproduces the Approximate Nearest Neighbor Search (ANNS) experiments from the **[VIBE (Vector Index Benchmark for Embeddings)](https://vector-index-bench.github.io/)** project using the Dynamic Exploration Graph (DEG). |
| 4 | + |
| 5 | +VIBE provides realistic embedding datasets covering in-distribution and out-of-distribution (OOD) search across text, vision, and multi-modal models with exact pre-computed Top-100 ground truth neighbors. |
| 6 | + |
| 7 | +## Supported VIBE Datasets |
| 8 | + |
| 9 | +| Dataset Key | VIBE Name | Type | Size ($N$) | Dimension ($D$) | Metric | |
| 10 | +| :--- | :--- | :--- | :--- | :--- | :--- | |
| 11 | +| `agnews-mxbai` | AGNews-mxbai | In-Distribution | 120,000 | 1,024 | L2 (Euclidean) | |
| 12 | +| `arxiv-nomic` | ArXiv-nomic | In-Distribution | 2,000,000 | 768 | Inner Product | |
| 13 | +| `landmark-dino` | Landmark-dino | In-Distribution | 760,757 | 768 | Cosine | |
| 14 | +| `msmarco-qwen` | MSMARCO-qwen | In-Distribution | 8,841,823 | 1,024 | Inner Product | |
| 15 | +| `gooaq-distilroberta`| GooAQ-distilroberta | In-Distribution | 1,471,375 | 768 | Inner Product | |
| 16 | +| `laion-clip` | LAION-clip | Out-of-Distribution | 1,000,000 | 512 | Inner Product | |
| 17 | +| `imagenet-align` | ImageNet-align | Out-of-Distribution | 1,281,167 | 640 | Inner Product | |
| 18 | +| `imagenet-clip` | ImageNet-clip | In-Distribution | 1,281,167 | 512 | Inner Product | |
| 19 | +| `yandex` | Yandex-200 | Out-of-Distribution | 1,000,000 | 200 | Cosine | |
| 20 | +| `yahoo-minilm` | Yahoo-minilm | In-Distribution | 677,305 | 384 | Inner Product | |
| 21 | + |
| 22 | +## Quick Start |
| 23 | + |
| 24 | +### 1. Environment Setup |
| 25 | + |
| 26 | +```bash |
| 27 | +cd examples/vibe |
| 28 | +uv sync |
| 29 | +``` |
| 30 | + |
| 31 | +### 2. Run Benchmark |
| 32 | + |
| 33 | +Datasets are automatically downloaded on demand directly from the official [VIBE Hugging Face repository](https://huggingface.co/datasets/vector-index-bench/vibe) and stored in `D:/Data/VIBE` (or `~/.cache/vibe` / `VIBE_CACHE_DIR`). |
| 34 | + |
| 35 | +```bash |
| 36 | +# Run on AGNews-mxbai (compact dataset ~120k vectors) |
| 37 | +uv run main.py --dataset agnews-mxbai |
| 38 | + |
| 39 | +# Run on Yahoo-MiniLM (677k vectors) |
| 40 | +uv run main.py --dataset yahoo-minilm |
| 41 | + |
| 42 | +# Run without GUI plot popups |
| 43 | +uv run main.py --dataset arxiv-nomic --no-show |
| 44 | +``` |
| 45 | + |
| 46 | +### 3. Command-Line Options |
| 47 | + |
| 48 | +```bash |
| 49 | +uv run main.py --help |
| 50 | +``` |
| 51 | + |
| 52 | +- `--dataset`, `-d`: Dataset name to benchmark (e.g. `agnews-mxbai`, `arxiv-nomic`, `landmark-dino`, `msmarco-qwen`, `gooaq-distilroberta`, `laion-clip`, `imagenet-align`, `imagenet-clip`, `yandex`, `yahoo-minilm`). |
| 53 | +- `--cache-dir`, `-c`: Custom directory for dataset files and saved `.deg` graphs. |
| 54 | +- `--build-threads`, `-t`: Number of CPU threads for graph building (default: half of CPU cores, `threads // 2`). |
| 55 | +- `--k`: Graph degree $k$ (out-degree per vertex). |
| 56 | +- `--extend-k`: Exploration width during graph building. |
| 57 | +- `--eps`: Build $\varepsilon$ parameter. |
| 58 | +- `--anns-k`: Number of nearest neighbors to evaluate (default: 100). |
| 59 | +- `--rebuild-graph`: Force graph re-construction even if a cached graph file exists. |
| 60 | +- `--no-show`: Do not open interactive matplotlib plot window. |
0 commit comments