From a36f6da11ead17d6ba8ac87d078bf7c3be1c3219 Mon Sep 17 00:00:00 2001 From: Dianna Hoober Date: Tue, 3 Mar 2026 16:00:19 -0800 Subject: [PATCH 01/10] Transferred doc content to Markdown doc --- .../databases/vector-db-benchmarking/index.md | 377 ++++++++++++++++++ 1 file changed, 377 insertions(+) create mode 100644 docs/guides/databases/vector-db-benchmarking/index.md diff --git a/docs/guides/databases/vector-db-benchmarking/index.md b/docs/guides/databases/vector-db-benchmarking/index.md new file mode 100644 index 00000000000..d6335e3eb9a --- /dev/null +++ b/docs/guides/databases/vector-db-benchmarking/index.md @@ -0,0 +1,377 @@ +--- +slug: vector-db-benchmarking +title: "Guide to Vector Database Benchmarking" +description: "Adopting a vector database can be a significant investment in infrastructure costs and operational overhead" +author: ['Akamai'] +contributors: ['Akamai'] +published: false +keywords: ['vector databases','benchmarking','approximate nearest neighbor','ANN','recall','latency','throughput','embeddings','similarity search'] +license: "[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)" +--- + +Vector databases are becoming increasingly important for AI-powered applications, ranging from RAG systems to semantic search engines. However, adopting a vector database can be a significant investment in infrastructure costs and operational overhead. Choosing the appropriate type and instance size for a vector database is critical to both performance and cost. + +Systematic benchmarking of vector databases enables an organization to make data-driven decisions based on actual workloads. Stakeholders can perform realistic cost modeling at current and projected scales. Without proper benchmarking, your team may encounter: + +- Performance degrading unexpectedly once you reach production scale +- Over-provisioning out of uncertainty +- Discovering performance issues only after committing to a platform. + +This guide helps you understand the basics of vector database benchmarking, specifically walking through an example benchmark result from testing an Akamai Managed PostgreSQL instance with VectorDBBench. + +## Understanding VectorDatabase Benchmarks + +In traditional databases, you're looking for exact matches. Either a record exists, or it doesn't. Vector databases are not like this. Vector databases are very different, as querying them is about finding approximate nearest neighbors. This fundamental difference means that every query involves a tradeoff between how fast you want results and how accurate those results need to be. How you choose and tune your vector database is driven by the quest to balance speed, accuracy, and resource costs simultaneously. + +### Key Performance Metrics + +When benchmarking vector databases, track metrics across the following three categories. Each metric answers a specific question about your database's capabilities. + +#### Query performance + + - **Query latency (p50, p95, p99)**: Percentiles show typical (p50), common (p95), and worst-case (p99) response times + - **Throughput (QPS)**: Measures how many searches per second your system can handle under load + - **Recall@K**: Measures accuracy by showing what percentage of the K truly nearest neighbors were actually found + +#### Resource cost metrics + + - **Memory usage**: Shows how much RAM your index needs, directly impacting instance sizing + - **Storage requirements**: Includes both original vectors and index overhead + - **CPU utilization**: Varies dramatically by index type and algorithm + - **Network I/O**: Important for cloud-hosted solutions and geographically distributed queries + - **Queries per dollar (QP$)**: Cost-efficiency metric comparing performance across different instance tiers + +#### Operational metrics + + - **Insert/update performance**: Determines whether you can support real-time applications or need batch processing + - **Concurrent query performance**: Tests how well the system scales under a realistic multi-user load + - **Index build time**: Matters for initial setup and periodic index rebuilds + - **Scaling behavior**: Shows how performance changes as your data grows 10x or 100x + +### Tradeoffs between speed and recall + +The fundamental tension in vector search is this: you can make searches faster by sacrificing accuracy, or more accurate by accepting slower performance. This isn't a limitation—it's an inherent characteristic of approximate nearest neighbor algorithms. Most production systems target 95-99% recall, finding that sweet spot where results are accurate enough for their use case while maintaining acceptable query latency. Understanding where your application falls on this spectrum is essential for proper benchmarking. + +## Benchmarking`pgvector` in PostgreSQL with VectorDBBench + +`pgvector` is an open-source PostgreSQL extension that adds vector similarity search to your relational database. Instead of managing a separate vector database, you store embeddings alongside traditional data and query them with SQL. It supports two index types: + +1. HNSW for better query performance +2. IVFFlat for faster builds with lower memory + +`pgvector` is often adopted by teams already using PostgreSQL who want to add vector search without managing separate infrastructure. Teams may also adopt it if they have applications that require vector search alongside relational queries. + +[VectorDBBench](https://github.com/zilliztech/VectorDBBench) is an open-source benchmarking framework from Zilliz that provides standardized performance testing across multiple vector databases. It offers a CLI, a web-based UI, pre-configured test scenarios, support for various databases, and automatic collection of comprehensive metrics. + +Using VectorDBBench ensures consistent methodology and provides industry-standard metrics. This makes results easily shareable and comparable. + +## Running Benchmarks and Interpreting Results + +VectorDBBench supports benchmarking across vector databases, including Milvus, Pinecone, and `pgvector`. For a complete list of supported database clients, see the [VectorDBBench README](https://github.com/zilliztech/VectorDBBench). + +This guide focuses on `pgvector` in PostgreSQL to demonstrate a practical benchmarking example. The concepts and interpretation techniques apply broadly to other vector databases as well. + +For complete setup instructions on provisioning resources, installing dependencies, and running VectorDBBench with `pgvector` on Akamai Managed PostgreSQL, see "How to Set Up VectorDBBench to Benchmark Akamai Managed PostgreSQL." + +### Benchmark parameters for `pgvector` + +When running VectorDBBench with `pgvector`, you can configure parameters across several categories, including the following: + +#### Dataset configuration + + - `--case-type`: Predefined test cases (For example: `Performance1536D50K` for 50K vectors at 1536 dimensions) + - `--k`: Number of nearest neighbors to return (Default: 100) + - Custom dataset options: `--custom-dataset-name`, `--custom-dataset-size`, `--custom-dataset-dim` + +#### HNSW index parameters + + - `--m`: Number of connections per node in the HNSW graph. This affects index size and search quality. + - `--ef-construction`: Candidates considered during index building. A higher number yields better quality but a slower build. + - `--ef-search`: Candidates considered during search. A higher number yields better recall, but slower queries. + +#### Concurrency testing + + - `--maintenance-work-mem`: Memory allocated for index creation operations + - `--max-parallel-workers`: Number of parallel processes for index creation + +#### Test execution control + + - `--drop-old / --skip-drop-old`: Whether to drop existing data before testing + - `--load / --skip-load`: Whether to load data (useful for re-testing existing data) + - `--search-serial / --skip-search-serial`: Enable/disable serial search testing + - `--search-concurrent / --skip-search-concurrent`: Enable/disable concurrent search testing + - +#### Interpreting your results + +To better understand how to interpret the data, consider the following example of a real benchmark results JSON file. + +```json {title="Example JSON file with results from a VectorDBBench run"} +{ + "run_id": "32143a10abef4c9abc1b625e771ef581", + "task_label": "32143a10abef4c9abc1b625e771ef581", + "results": [ + { + "metrics": { + "max_load_count": 0, + "insert_duration": 10.1447, + "optimize_duration": 70.0666, + "load_duration": 80.2113, + "qps": 542.8612, + "serial_latency_p99": 0.0072, + "serial_latency_p95": 0.0053, + "recall": 0.9704, + "ndcg": 0.9758, + "conc_num_list": [ + 1, + 5, + 10, + 20, + 30, + 40, + 60, + 80 + ], + "conc_qps_list": [ + 217.7906, + 533.9531, + 534.8679, + 542.8612, + 523.398, + 519.0258, + 505.6169, + 502.7718 + ], + "conc_latency_p99_list": [ + 0.008155493999993262, + 0.021081581450016502, + 0.03925018654992982, + 0.07296883467007315, + 0.11555392853994134, + 0.1581347975700339, + 0.26107694360998934, + 0.4130624263500287 + ], + "conc_latency_p95_list": [ + 0.0060548709999466155, + 0.01499669475001042, + 0.028643750499952603, + 0.058125223350015166, + 0.09157506069996088, + 0.12197972049999066, + 0.18427210214998127, + 0.24925758274994791 + ], + "conc_latency_avg_list": [ + 0.004570292678859688, + 0.009346250124765887, + 0.01862953539550514, + 0.036622078273554445, + 0.05650672965547461, + 0.07569008427615372, + 0.1145797859643314, + 0.15085457710523364 + ], + "st_ideal_insert_duration": 0, + "st_search_stage_list": [], + "st_search_time_list": [], + "st_max_qps_list_list": [], + "st_recall_list": [], + "st_ndcg_list": [], + "st_serial_latency_p99_list": [], + "st_serial_latency_p95_list": [], + "st_conc_failed_rate_list": [], + "st_conc_num_list_list": [], + "st_conc_qps_list_list": [], + "st_conc_latency_p99_list_list": [], + "st_conc_latency_p95_list_list": [], + "st_conc_latency_avg_list_list": [] + }, + "task_config": { + "db": "PgVector", + "db_config": { + "db_label": "my_postgresql_db", + "version": "", + "note": "", + "user_name": "**********", + "password": "**********", + "host": "a407200-akamai-prod-183144-default.g2a.akamaidb.net", + "port": 10033, + "db_name": "defaultdb", + "table_name": "vdbbench_table_test" + }, + "db_case_config": { + "metric_type": "COSINE", + "create_index_before_load": false, + "create_index_after_load": true, + "iterative_scan": "relaxed_order", + "m": 16, + "ef_construction": 128, + "ef_search": 128, + "index": "hnsw", + "maintenance_work_mem": null, + "max_parallel_workers": null, + "quantization_type": "vector", + "table_quantization_type": "vector", + "reranking": false, + "quantized_fetch_limit": null, + "reranking_metric": "COSINE" + }, + "case_config": { + "case_id": 50, + "custom_case": {}, + "k": 100, + "concurrency_search_config": { + "num_concurrency": [ + 1, + 5, + 10, + 20, + 30, + 40, + 60, + 80 + ], + "concurrency_duration": 30, + "concurrency_timeout": 3600 + } + }, + "stages": [ + "drop_old", + "load", + "search_serial", + "search_concurrent" + ] + }, + "label": ":)" + } + ], + "file_fmt": "result_{}_{}_{}.json", + "timestamp": 1766966400 +} +``` + +This example is from `pgvector` running on an Akamai Managed Database (`g6-standard-2`: 2 vCPUs, 4GB RAM) with 50,000 vectors at 1,536 dimensions using HNSW indexing. + +For this test set, VectorDBBench ran on a separate Linode instance (`g6-standard-8`: 8 vCPUs, 16GB RAM), which provided enough memory to run tests up to a concurrency level of 80. + +Note that the compute instance running VectorDBBench needs sufficient RAM to handle multiple concurrent test processes, but this is separate from the database instance being benchmarked. Higher-concurrency tests require more RAM on the benchmarking machine, not necessarily on the database server. + +The key metrics examined in this guide are: + + - **Latency percentiles** (p50, p95, p99) + - **QPS** (queries per second) + - **Recall@K** accuracy measurements + - **Resource usage** (memory, CPU, storage) + - **QP$** (queries per dollar) for cost efficiency + +#### Understanding latency percentiles + +Latency percentiles tell you what most users experience, not just averages. Imagine you run 100 queries and sort them by speed from fastest to slowest: + + - **p50**: The speed of the 50th query (the median). Half were faster, half were slower. + - **p95**: The speed of the 95th query. 95% of queries were this fast or faster, but 5% were slower. + - **p99**: The speed of the 99th query. 99% of queries were this fast or faster, but 1% were slower + +p99 matters more than averages because it shows your worst-case user experience. If p99 is 500ms, that means 1 out of every 100 users waits half a second or more. + +In the example results shown above, the serial (single-threaded) search shows p99 of 7.2ms and p95 of 5.3ms. This indicates that in the best-case scenario with no concurrent load, 99% of queries complete in under 7.2ms, which is excellent performance for real-time applications. However, these numbers change significantly under load with concurrency in play. + +#### Concurrency test results + +The example benchmark tested performance at eight different concurrency levels, revealing how the system behaves as multiple users query simultaneously. The throughput (QPS) by concurrency level was: + +| Number of Clients | QPS | +|-------------------|---------| +| 1 | 217.79 | +| 5 | 533.95 | +| 10 | 534.87 | +| 20 | 542.86 | +| 30 | 523.40 | +| 40 | 519.03 | +| 60 | 505.61 | +| 80 | 502.77 | + +The system reached peak throughput (542.86 QPS) at 20 concurrent clients, after which performance declined slightly at higher concurrency levels. Notice that throughput more than doubled, going from 1 to 5 clients, but barely improved from 5 to 20 clients. This indicates the two vCPU instance was approaching its limits early on. + +As concurrency increased, individual query latency grew substantially: + +| Number of Clients | p99 Latency | +|-------------------|-------------| +| 1 | 8.2ms | +| 5 | 21.1ms | +| 10 | 39.3ms | +| 20 | 73.0ms | +| 30 | 115.6ms | +| 40 | 158.1ms | +| 60 | 261.1ms | +| 80 | 413.1ms | + +At 80 concurrent clients, the p99 latency (413.1ms) was 50x worse than single-client performance 7.2ms). This dramatic increase shows the system under severe resource constraints, with queries spending significant time waiting for CPU availability. + +#### Understanding QPS and concurrency + +When you see high QPS but poor latency at high concurrency levels, here's what is happening: + +At low concurrency (1-5 clients), the database CPU often sits idle, waiting for network round-trips (sending queries and results) and client preparation time (processing results and preparing the next query). This is why single-client QPS is often much lower than peak QPS. + +At higher concurrency (10+ clients), multiple queries keep the CPU constantly busy. While one query is in network transit, others are being processed. CPU utilization reaches 100%, and throughput (QPS) reaches its maximum. + +But once the CPU is fully saturated, adding more concurrent clients doesn't increase throughput; it creates a queue. The QPS stays constant because the CPU can only process so many queries per second. However, latency increases as queries wait longer in the queue before being processed. + +#### Peak performance and finding the optimal operating point + +When you see benchmark results, you may notice that **peak performance** (or **peak throughput**) occurs at some concurrency level, but the **optimal operating point** (or **recommended concurrency level**) is at a lower concurrency level. Why is this? + +*Peak performance* is the maximum total throughput the system can achieve. At this point, you're squeezing every last query out of the CPU, but individual query latency has degraded significantly. + +The *recommended concurrency level* is the best balance of throughput AND acceptable latency. At this sweet spot, you might get 98-99% of peak throughput while achieving a much better user experience. + +For production systems, operate at this sweet spot rather than at peak. This gives you good throughput with low latency and headroom for occasional traffic spikes. + +#### Why CPU count matters + +Vector search is CPU-intensive work. Each query requires graph traversal through the HNSW index, distance calculations on high-dimensional vectors (1,536 dimensions means 1,536 multiplications per comparison), and result ranking and sorting. + +With only two vCPUs, this instance could process roughly 540 queries per second regardless of how many clients were waiting. More CPUs enable more queries to be processed in parallel. A 4 vCPU instance would likely deliver 2,000+ QPS (nearly 4x improvement), with better latency under the same concurrent load and the ability to handle more concurrent users. Index builds would be faster, and you would have better support for larger datasets. + +Running your database on a machine with more CPUs fundamentally changes the system's capacity and responsiveness. + +#### Search accuracy + +The benchmark measured two accuracy metrics that are critical for vector search: + +1. **Recall**: Of the 100 true nearest neighbors for each query, the system correctly found an average of 97.04. This means only about three vectors out of 100 were missed. For most production applications, 97% recall is excellent. Users get highly relevant results. +2. **NDCG** (Normalized Discounted Cumulative Gain): This measures ranking quality on a scale from 0.0 (worst) to 1.0 (perfect). A score of 0.9758 indicates that not only were the right neighbors found, but they appeared in nearly optimal order. The most similar vectors consistently ranked highest in the results. This is most important for applications like semantic search or recommendation systems, where ranking matters as much as the results themselves. + +#### Cost efficiency + +Calculate QP$ (queries per dollar) to compare cost-effectiveness. + +``` +QP$ = Peak QPS / Monthly instance cost + +HigherQP$ = better value +``` + +Consider these example benchmarks: + +| Instance Size | Monthly Cost | Peak QPS | QP$ | +|--------------------|--------------|----------|-------| +| 2 vCPU, 4GB RAM | $63 | 540 | 8.6 | +| 4 vCPU, 8GB RAM | $126 | 2000 | 15.9 | + +The larger instance costs twice as much but delivers 3.7x the performance (2000/540), making it 1.85x (15.9/8.6) more cost-efficient despite the higher price. + +### Next steps after benchmarking + +If the benchmarking results from your test setup meet your performance needs, then document this configuration for future reference. Plan for 2-3x growth headroom to avoid emergency scaling. Set up production system monitoring to catch performance degradation early. Establish your performance baselines so you can track changes over time. + +If the results do not meet your needs, then you can systematically explore other options: + + - Test larger instance sizes to see if more resources may solve the problem. + - Optimize index parameters. Tune the HNSW parameters (`m`, `ef_construction`, and `ef_search`) to see how they affect performance. + - Consider data structure changes, such as reducing vector dimensionality. + +### Conclusion + +Benchmarking vector databases, such as `pgvector` on Akamai Managed PostgreSQL, provides the concrete data you need to make informed decisions about performance, costs, and scalability. By understanding the key metrics and using tools such as VectorDBBench, you can obtain and interpret results in the context of your specific workload. With the numbers to guide your decisions, you can confidently deploy vector search capabilities that meet your application's needs. From 569c1156beb6fa9595e9af49a056abdbf71f9ee5 Mon Sep 17 00:00:00 2001 From: Dianna Hoober Date: Tue, 3 Mar 2026 16:14:54 -0800 Subject: [PATCH 02/10] Added author and contributor names --- docs/guides/databases/vector-db-benchmarking/index.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/guides/databases/vector-db-benchmarking/index.md b/docs/guides/databases/vector-db-benchmarking/index.md index d6335e3eb9a..3eb384f159b 100644 --- a/docs/guides/databases/vector-db-benchmarking/index.md +++ b/docs/guides/databases/vector-db-benchmarking/index.md @@ -2,9 +2,9 @@ slug: vector-db-benchmarking title: "Guide to Vector Database Benchmarking" description: "Adopting a vector database can be a significant investment in infrastructure costs and operational overhead" -author: ['Akamai'] -contributors: ['Akamai'] -published: false +author: Andy Stevens +contributors: Nathan Melehan +published: keywords: ['vector databases','benchmarking','approximate nearest neighbor','ANN','recall','latency','throughput','embeddings','similarity search'] license: "[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)" --- From 8c1304e33e5e0bc61e885e10e5f3a48b71e7732a Mon Sep 17 00:00:00 2001 From: Dianna Hoober Date: Tue, 3 Mar 2026 16:35:28 -0800 Subject: [PATCH 03/10] Added terms to dictionary --- ci/vale/dictionary.txt | 3 +++ docs/guides/databases/vector-db-benchmarking/index.md | 6 ++++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/ci/vale/dictionary.txt b/ci/vale/dictionary.txt index 67342db7119..e758d61b9cd 100644 --- a/ci/vale/dictionary.txt +++ b/ci/vale/dictionary.txt @@ -1168,6 +1168,7 @@ iterables iTerm itk ivanti +IVFFlat jamstack Janca java6 @@ -1932,6 +1933,7 @@ phusion pid pidfile PIDs +Pinecone pingable pinterest pipefs @@ -3005,6 +3007,7 @@ zgrep zhao zigbee zile +Zilliz zimbra zipkin zipkinhost diff --git a/docs/guides/databases/vector-db-benchmarking/index.md b/docs/guides/databases/vector-db-benchmarking/index.md index 3eb384f159b..a6f6143c564 100644 --- a/docs/guides/databases/vector-db-benchmarking/index.md +++ b/docs/guides/databases/vector-db-benchmarking/index.md @@ -2,8 +2,10 @@ slug: vector-db-benchmarking title: "Guide to Vector Database Benchmarking" description: "Adopting a vector database can be a significant investment in infrastructure costs and operational overhead" -author: Andy Stevens -contributors: Nathan Melehan +author: + - Akamai +contributors: + - Akamai published: keywords: ['vector databases','benchmarking','approximate nearest neighbor','ANN','recall','latency','throughput','embeddings','similarity search'] license: "[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)" From bf3fd8af0585f487b45fbf180ec3d45e27839bb7 Mon Sep 17 00:00:00 2001 From: Dianna Hoober Date: Tue, 3 Mar 2026 16:41:31 -0800 Subject: [PATCH 04/10] Remarked out publish date --- docs/guides/databases/vector-db-benchmarking/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/databases/vector-db-benchmarking/index.md b/docs/guides/databases/vector-db-benchmarking/index.md index a6f6143c564..47cf93acad3 100644 --- a/docs/guides/databases/vector-db-benchmarking/index.md +++ b/docs/guides/databases/vector-db-benchmarking/index.md @@ -6,7 +6,7 @@ author: - Akamai contributors: - Akamai -published: +# published: keywords: ['vector databases','benchmarking','approximate nearest neighbor','ANN','recall','latency','throughput','embeddings','similarity search'] license: "[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)" --- From d8a5df2ac8c692a2fc112f80d80499a7de0b426c Mon Sep 17 00:00:00 2001 From: Dianna Hoober Date: Wed, 4 Mar 2026 08:36:28 -0800 Subject: [PATCH 05/10] Metadata authors per linting --- docs/guides/databases/vector-db-benchmarking/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/databases/vector-db-benchmarking/index.md b/docs/guides/databases/vector-db-benchmarking/index.md index 47cf93acad3..f78bb842c6a 100644 --- a/docs/guides/databases/vector-db-benchmarking/index.md +++ b/docs/guides/databases/vector-db-benchmarking/index.md @@ -2,7 +2,7 @@ slug: vector-db-benchmarking title: "Guide to Vector Database Benchmarking" description: "Adopting a vector database can be a significant investment in infrastructure costs and operational overhead" -author: +authors: - Akamai contributors: - Akamai From 1e0a891a10b4afb1be652df02c2519746da22694 Mon Sep 17 00:00:00 2001 From: Dianna Hoober Date: Wed, 4 Mar 2026 08:48:57 -0800 Subject: [PATCH 06/10] Metadata requires publication date --- docs/guides/databases/vector-db-benchmarking/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/databases/vector-db-benchmarking/index.md b/docs/guides/databases/vector-db-benchmarking/index.md index f78bb842c6a..ead05590d97 100644 --- a/docs/guides/databases/vector-db-benchmarking/index.md +++ b/docs/guides/databases/vector-db-benchmarking/index.md @@ -6,7 +6,7 @@ authors: - Akamai contributors: - Akamai -# published: +published: 2026-03-15 keywords: ['vector databases','benchmarking','approximate nearest neighbor','ANN','recall','latency','throughput','embeddings','similarity search'] license: "[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)" --- From 0f4afa9eab76016276d2fff876a10b8ee8406ca3 Mon Sep 17 00:00:00 2001 From: Dianna Hoober Date: Wed, 4 Mar 2026 09:00:45 -0800 Subject: [PATCH 07/10] Metadata requires a current publication date --- docs/guides/databases/vector-db-benchmarking/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/databases/vector-db-benchmarking/index.md b/docs/guides/databases/vector-db-benchmarking/index.md index ead05590d97..18770572761 100644 --- a/docs/guides/databases/vector-db-benchmarking/index.md +++ b/docs/guides/databases/vector-db-benchmarking/index.md @@ -6,7 +6,7 @@ authors: - Akamai contributors: - Akamai -published: 2026-03-15 +published: 2026-03-04 keywords: ['vector databases','benchmarking','approximate nearest neighbor','ANN','recall','latency','throughput','embeddings','similarity search'] license: "[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)" --- From 9c30d964903a2376ec7ddd6e8723d7c89de2b974 Mon Sep 17 00:00:00 2001 From: Dianna Hoober Date: Wed, 4 Mar 2026 09:20:14 -0800 Subject: [PATCH 08/10] Fixed centered text issues --- docs/guides/databases/vector-db-benchmarking/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/guides/databases/vector-db-benchmarking/index.md b/docs/guides/databases/vector-db-benchmarking/index.md index 18770572761..98859c83f07 100644 --- a/docs/guides/databases/vector-db-benchmarking/index.md +++ b/docs/guides/databases/vector-db-benchmarking/index.md @@ -102,7 +102,7 @@ When running VectorDBBench with `pgvector`, you can configure parameters across - `--load / --skip-load`: Whether to load data (useful for re-testing existing data) - `--search-serial / --skip-search-serial`: Enable/disable serial search testing - `--search-concurrent / --skip-search-concurrent`: Enable/disable concurrent search testing - - + #### Interpreting your results To better understand how to interpret the data, consider the following example of a real benchmark results JSON file. @@ -351,7 +351,7 @@ Calculate QP$ (queries per dollar) to compare cost-effectiveness. ``` QP$ = Peak QPS / Monthly instance cost - +. HigherQP$ = better value ``` From 44d469af62c66de6897995ab4ac9f92ce5b9977b Mon Sep 17 00:00:00 2001 From: Dianna Hoober Date: Wed, 4 Mar 2026 11:38:51 -0800 Subject: [PATCH 09/10] Added png file to illustrate a point --- .../databases/vector-db-benchmarking/QPS2.png | Bin 0 -> 20270 bytes .../databases/vector-db-benchmarking/index.md | 6 +----- 2 files changed, 1 insertion(+), 5 deletions(-) create mode 100644 docs/guides/databases/vector-db-benchmarking/QPS2.png diff --git a/docs/guides/databases/vector-db-benchmarking/QPS2.png b/docs/guides/databases/vector-db-benchmarking/QPS2.png new file mode 100644 index 0000000000000000000000000000000000000000..15f3f53f18fcdc3af285659500fbdb237c2f8737 GIT binary patch literal 20270 zcmeGEbyS&#k7u3784GCMPSX7=p;*@@H9QYOTs#lynFB2-mT z(8I!dgmZuW;pyZ1cdle7hWn2P-g?S%Sk)8Hcke5Y9A!0Sv9M}E_}B0LxfA!jim5ji z7A5OHuLr7nEQeTFT0d14WDNqW_B;JUa*fkaojxjM(Lc{# zKkmT8S~LD}KQRAY4)x!1U}3cg2*_V2Yv4Y3@L=n-*XMsp6TXM@kHp4E&|BRfo3%=s zd02VSG0CXLk9#2YSChyKiFbXWjN!!b>_3QJAiAtP%(*=}8R>UE`c;}=(qY2=K4{XB zPyh}X*^+}35|b9fmI5qlu_Ib5vF8al39zt!Fg|?G_#Zj$`^x0Lud2r$rDK;*d7FvR z#Z5{eLm_ASV>b_T((lN4t%`OZExrLc5ylk%Ao3={Y#jdF1S1TT(6aG@L9DN zwW-g1TdHdi4owH|p|^6Ca7$H%F+6kVH69`1R0kJkHs4nMdqz^tIjg&P&rX<1h&?VY zQynd)_8mszr|-wNFgohzh1Dw$#s`(@F7YX=t`ywuYP{SLOH-OZ&sIJ*h1wMMa&Y#J zy_}z(n18lQ#O1ngL=+K$%^@scffi};`3%z=YFhoZ_8qzgdkD(Di-?R<8aqe+ip`}P z04ZaITWnd4F>e}_tt572ZnDn-^xNGb$dA6@^FzmZbNFA} z23l3~)t;W0w2dFgC}}Ip*hRl|%;EL9GRpL3=~lR(Is-2$*Iy!Ocj{$?eQIw)%zQyH zkDo*XB*Y!ISYt;?nal(ST=xPxc^*qzKzDG+fQpwHKvNGaL zbv9Qse1iBW^J&A)A2`YF$HTOqx&6K@S~ZQ>=@2W2&(IaGuG?v)UR{q;Yxm`v%kGd1 zHcd3NSo4>XN1l`G-t$)RaawY{m;mLH^f_&7TvvvA@gWdAm&DT{`)^NVV&}xdd;ds$5 zof-BElaN5|8sqxr55F766>+N%K9JN?dN>y*26vf6SxP{l+S!Gd1=2<9L1&4b1{PLD zmoLoOI25g-r=(XK=NhcF9O_20UOS;M2R&qqp??#Q!A~9wB_z`Dsvz1OKQQ9&sN!=mif^3MSYQEs$C%*@~7pepOyJ3>w^Lnd@f%a)} z-A5FVdMBeiB;(TChV6(@Zk!9d$kE+fr_kfU!6s@?=k={|Q@h`dMoPIQpaGAkg1~DY zzod(1h+84TczpIcdNy8Yn~}cDm1DADwqSGnBz`;?aJLPjB@D8X9{rI?zq$#r9|ZAP z$@@cK#&uBOc7)_ls`#EX{ej99_IbvTJlK`&T`)%%ki6n~@a}dfYe3~G z6;8|~es)5M&Twmx=H0yGLCIUHl#hcQ^JcFT$%%nUJVn8IlCCp3uU9sPpY=4OBPLAG zYN~Tfw3@1F?6UH2cJg+#HpMmV?APgHs`JdG5ky*fHG+$0(yp&kh6LGzR{D=y(U+@6 z6V)@6;uA|k2Miy;?r46*69yXd0ydBStoj|Ho7F z)&^`G4gaImSQm8vY;hW~**Mq{k!5`@j@@^zMXJ0*szo^7P?xShIa3;4q2E6-bPPK^ zK()GOK`}&mP>6?sS-Zp>HIIj$&GR%vi^=QiB75*kvj^$2Yt7|&dMEfm)Iu!`UDh5N z_)-9niR!dK1DvjZ$|M-cH}Yt0`mlD8q1P7kHgR#GVDUWbEKlj1HmJ9w9lUZ^4D8vo zI8%lB?7Wz2FfPH1CVUNk@NHCl}NlKuq{6Q*WZskX_@Si5kO7 zGOJ&8+Gq))?Az5A7b4a^MOWb7ijAg3>)74Ru!ImK>v_$t%*}5@yau!FQ(>q=7z)cB z;mA#NL8qBg6Y|$jO;da3I=*Iq?nElRfN=CP0CnwTZYExaXxa;P9j~U;)m@)on%_(Y zQn~N}riiVD!8ghtLUTOc4oE9sJO3T|)s;EL)$0X^adkRe>zh7$fV)c*(i-8icy=jr zK1J(#b7)G%Crjc5+`!Z(n5_NDohmF_bRqoLJ#0eEk;dE3pEH% zL}#zmCDxuoW+&aZzpKDX zN=iD$pjxc=+xVM~mJJ${pRjx7oQJM7?)Km{&f%ryU*|$N$LL}QAn&he5v`VI`Q+_kOj#TWB$mRY39+ z_`40CprelH7HD!WA|@Iz5(n znOyMF#Z3_AeI!Fx#QbVxB5Ptpp^DpWnburFXn!a0<$#$ZOJBCv%XF%P;0Z_5HQZt& z38H05#sQdilkIS*@=y}PekT;9#x;I^P=R`0jPbN8ymW5O0tYF_a99mIAa|}hBT>f2 zy0ancj{es^JluK8eQ%VJZ-nS6D-Zl`?_Pvn=1CWAlM(_|KF!($Bo3gns8$P7ww1__ zh#_Hb;C1CeLF+u^`fHw5b29v_X`U_sI+=h29!=Nm#ofEhrATm?JLa9HKO;|3jr2P} zzxi3tWP_6nTa|^)24w&y@odw~hsGnC?lj1GsBDTY^K7Y8k&n_wi0`0bSlYj%FR6-fZLeR^wvfCC-~{*vM+<~>Rc!L$+)iKl`1_tr zhArOuOaFzmh4HZV(TwmUne2c30bh$kcLJM`ecba5U@7wCjM?&w8XV}av zQfn@iMQ8~zB9H|Qc->c_yCJE8ud=UX%*%GW@0OO%YY`dnMOHSWAFH95c`%f* zSg2K|1oNQrDpjR6&>xlWLWC3d?!{0wqbEwD?6Ua`+yL5RbPG)}=6h{+-9FEguT!D8 z9M+-$_b1AMO2*Cpcy7Buw%B$Bpo~5&dy)vMTS7^|XvI9;1Q)eyiE!)Cs`+Wl@A8Zp zzSk0I+>z5uSt;>#+YW^30;8d&0a{+?S?-Y2zU7Kn(sN@nTDJ;ZOYS^oInRfrLejCj zE^ruvoa3`nv=*s3?iY=Hip>E~+R=^j$nVqKLBQQA!(#sKY#(2%Xc+kT=J)o#PvuNQ zHr$RcTQFBTbd`48{2|dUi%S!~O5Rn~(Gg5bGar3A@JG(?EvX)v&2C(Kl*}qCIXQz7;42E9NwrQ`VN75cAMnn#xJmLBJ4;DBS$Xk(7~bIp5cg|RL?ur`iHT7 zg2Ug_l^HvK_GM=d?E9Bo5E#puasNoFH+{}8fk)+0Z9bVWH_x%DnHX2(vNCKkyS){bsl^ZzF~-%*D>A3!@9{bv z%drInw6pC$`PHiBLF%>*r|OPC-LZ^DeRryM#_%J~?`=e^`_o@It{2}+&2jpZFyL7< zGFR+as+s$=Gc|sp`h8_fPF|h)V(956a$9+94$>TteV`~JWT~+=PF&(>^_nl+v4ZY5 zDWP*xJYwJI*wXprYM0{ZGn@m3&E(VdTIED97aG$Qd9%xu~-sF_4kNN zdr7OOsV2;BS$=Dk7@_!Z_lYCZ^(uofpRx^5W+9_!R5NUKB{afn)!wS<_ACXO*CquC zcCHV->9qEkqS9DL&*g3e^L5C#mLZ?qC?5#k^zdk$HX=(fkURnNPN|SlGPCCCmWt~u zU;WkQb4Xz#m*)5OXMX0RK7rLdkomTv5R-UQs(`Bbu1BHq)?YGyg@-kpQsMB89?VJ) zWU)auYd5U=@b*XFdIy70QJuqGZ>iLHw!e}O5L~9!l?y@?B8?V7c|2VAiK#ABh0yfwCY&#+YEwfv~5H-1r>Jt*7)m%1^drHf43wsTc`ZcH6dN$)plp zs&C(fJmIxe<4jwF;sQ=yM)TjIpR7B8yWaz8;DLT+cs7FAxhrZg@vtAXB3CRgsxhqx+9zlwV1S#o2p=Z`ViNoAlN~ zAe((sS~3m)n%o)zmK#7C;+>L}c+`6tl^-MEXd_?E1@9Ar*_mb?Gr5E_Z4Hg$G<(pg2_5e_h_Q&G5KeE(!Z`{;VT(`Vc~3Xz2v~ zSqjs&cTe#^a|p0JnCRQG7+8;>5d%m{s?~J)e}WdX=$YAYOZmZQ(~mUNVnI-6G3Sut ziiIE}7woio!1Ny+oJC4sBXWxm)k)I6tKwk6P`q)7zRGSHx3wgjS_}8lk{^>uCwD2@ z0jX!U(FFUukaw! zUh32ns!*q-&{g`tZU4Sijniek{=y>5*5`ToFX~I$x;d<1?NnEAuWxog`ZJUJ1PphL zMVKvu%!auv6~iVB9(;6uC%0EVy|^fQ^H9FQ9;VtB=uB!~!ZMjUM$w4XYQ}RMVAG;a zW;#dKs!?L0T-NO5Wq*w;-Tcg1*;V)Z&IL`{dE*l7+})zOS~^bhmYm$CEPK#PQ$8D< zU6U!1Rw1JF@ujTKv0G!(>EeLjjoaJ|!3Z*tSzI6iuXJY;aDZ<4_PX(eR4nyvFzLGj zmuX&E+Y$@15S<9zm2TimR@On+tj&O>;FeC-$y^I|)Y7 zv<2zlu0gK8yqw%CHl|N?7gwSE+cPo%g{-o;)R!MsTlaIhX6=%XfZ>?3*+vpBXnt!c=iKzJ&yu!#vK>F;NvC~^68gIcCko}*pz@bf4xg^7Aolrr8zsma; z{p@TyA#Q*~0zy6R^Ud06?*`y7_viJHo|)%B2Le#cQ@m~I43xjgQ>-<}kU!VIj)7~E ze_Xh13XM<=ZCJRAfx_O4uAIWwT+XgB<|1+ZM~`sg!0~^w7k{tqa^@-;O=MRu65zgX zL_5(C5`rd^zjBKEw0%8e;3@bFW)lzY`siUMt{-W=+~|;pYcog{FRWEbQfO1e!O3DG zltEaPn@U1g0{(eUl%Z>A8Smlf+tilqu$|4576NWk&G+3zE6N8P$Ys0y>X!)$hZhvQ zo6R4c5)oB-C@)X3g_}>%o|tPQ=+w_%rjg7;%$lEmZzjg!X@b)8if#tgoafi!K3gYZ zQc(L+XRO{GP94`iA8+RO2^OU%gLOKBhEmuBirrh;iQ*JgU8-~WaJTyM6XH85X*bF|_ z+%^baxh3B&zkJqeGU1zo<;(6C96NWYb40Tkx&;`8({x%Lpf7T{44HYrx~D>n^d)c8 zwWoq8HO99v8b!Y94tg4PkH&|_^MA6Y`Q*2JCd#QSrdcSP=|3hJbr0hKhx9n>mLw{u z2NtwiHXd`e5-KuRn5j6DdZk?qnA*0RDaPW2o8ZMdG=uD$a}b6N{pYJ4CWC6;!6Lak ze(CvM^`#DFbB2K5g$vITw+V2!*!OvADHNbtP$<#oq^G5Su8ho@zrWdgN$a>vTg%WA zILu8LK}pX{Nn8W~ye^hT*s#6;JMFp>K7KVv8Pec?_FHt>+M6F`3BpJH~+c?Vy)dj8kJX-f>8iR*Cu&lp+pU6pHY^UDt z$p3k{O`={J$AhE6O)~X(bhUq)S^1&cv6ND4l5LS>67B4rm7Uf=%#w8MpqG^y)A-n^ z&i)4%Id1orz?{KQT=$)$RXAL^a z4`GIPN7|>0snGyR*P7BWG{e3GFdf*wA1-aXQ;_0w?ZyKW^yTEzyEVjjys*iPO-nj4 zXM^o|<9UyFyf~`7_~z*Z7qc9=u4~nA45=fmauegEJu1GSPOpD&HiC{GNL?3bMd3S# zWASDrB&RoNB|fQ5*@>=S(%Ru|nDoeV^?xqeZ_kZH6>#22)6y^)sz*l&qBTND9qQ7t z_4vGFaZFmxdQ@pjELr_j^W8urH^r1RWarVNzqdLPF`5IL9v;ml^`(D)4X2uL2zUe} z5kAG8_U3{K=@=GayvqG=&6dI>;bpd>cdHMyYtzgE@{)Iaf%w{kheEw5;Z z^2!a|vz%@v+Jc>ra>~lnbjK5%g0Qg-^j{u5P#EHxAyl3RzZeL@Yi96DckHG*&I7PPQC0{Iw4T?5!p`X2}X1beOtKw0lDiC16!&#B0 zGQXNaXfpT$Re{vp+-v2wh#qzkJE5`Y1}{1UIj-WCd%BdEN&mTUa&j)ZLK;bx?`8j% zv&APQCe)pp;hy!0j;7t8w`+{vz+YKdF0J3|+Ej|FBR>gPZIFcMECrB19>a_RevcMz+B& zgqkKzQrAR5G^*ADY43QWajdAmX%Bj$sh@+L}qv}R>` zO*MA9_pYDP(_|j`{tY>98)O_(_s^^Iv{aB(%%}lk0S2i_uaXJmj)!!=Z>vj6KMYlH zuyId4p6VeL|KT0^Vo~dz-^af`?-r|5cJR`I)48Us;?rdu)!4$gHw_!$$0oNmxvn|S zK7Ds`s7`7QIMA+H&XVIMdQ|&@B%Zd%pExcS=Kj)q_`ufj)YGPfU%zkz?X9R^1NNG4 zzD7}}^GtpFd>WTfdxa!gR%$HiG!I?bc~2MhDJpaS51ZHupE5&9j0xRP$H3qe+|cat zu0*nRF3M+OGtw&^dtyVjlKU7sRLzsX6FpYpQ8D$*&+$ndoRpHj`XIhyhuz zXocovV5Ba81Tld0o%g?9QaqznfY-xA5EQ(Qt*NMovOGA{Qx`CDQ_=@%G5k#RB|GEk z#iO4SaH7WI`W+|s&b5tU;b!lvQup!XTck|=8a0;uCdgF3SQ3VUHPc8XS(e^T_czvUdn`A1D^ds5XqVZ0{IYJLy`7#Dy|bV%6o^m>7I= z7I8M?fY5jMzHZJi;Zvd~)95KYNcaeQC4zYYQ z9Oq>b^ZGL=u@I}P6b%vrHr0s)yrAb0s{tMr6>=|iRDomxToK?A7me;0)jtRoa z4Vh=>rsRKyfZQ?K(_>|(<;muN<~KJw=4Ok|gX?6CWF!`*a=j6a`~7-9Ow1om%KLL0 z@7q0DMWyY>2pxA{aEUbglq1b=0&1{lE03y}<>Y?{m?kPo%9sYqhPI$v{}u|RSyEap z)u|Kh^9G|L+R68bggDRXH{L85C5iJt3Y)@*=nGwp;6dzyBN< z6Cw|6?G5@ryaMhneJYRI-``(vh#{hYl+@2^>WQ0mfBUj6kT9)BcEMm=I_1A;)kLFx)sTq5Xg4U0B`O$EGYF07f_M667Kyo%mL*2fXCfpKL?fb+ol> zT;6O;azM<=-_y+=1;}_cDQ$l!TPA$-2g!}o=Wp&oKa7T1)MBlDCh~@;n>YBL>zX@) zuKXCc1mgPDHHuFhJI-2%qWXkM9!dBD|6&)B`3B% z&QsV%VBQdMWJ^{Tm0A~F#j{2$gfTkM=o5Nrfp0baVo30l4!^v;X?{UT{9%8l(qIJE zHKrGCpPxK{vu}Nw@MOUg%(PjnMahitM>H|>RH%*o9G-=uP`j)NvE{2FZ_j3 zwTJ@;r9a8TXBkixfwekI?mRhKVs3w_vUT_U3keVGNB&Ufi}nVDuiV^E(}?3AT?Li&=>(AJkrsm$(7JE}*uc3({x$<$e-_@fbp83-2z40?vf z8S^4{ldOf;rj~{Bv4C*jAD&2<=fN+DA)MEp}TEEKYcQ-e}VISJE%K6c=PRP|P;? zUgFHPb<0x^SUd6|6dUW1@gF+pzsm*3KirX}-1B?x=E1XXq^?cT~L)x!Z*KsI}ZZ~afObD&RQDznk!@Ye+ihA3mobN?3xrM*<< ztl`FTGtzYVF=ar;nnkv8C*k4?zJHbZo1XP`@M`Dk(4^<8l!A3oND;E$dm2Fh=b#jI z>r#Of)Ehf^4q9#;xmXjV4U1R^Y)Q3R^wU9akn_fBNWX_Ojjy(?By9gwR^QF=Jea+v zlKv16y}Zlc2;mi^^CtbO7+9bz99Cwq8f-K{D0(68&eJ1NQ9(QwIMhZm9~ z-l-nOa6OPPbu_&R518F&*C*hT0XH zs5-8@Uw}B&Wvh3aV?3FuH#PdZ-a2OcIdhr?i`21p4N?Ux;n$XjvMsiu zsofWW2I#t!h%eMsiu@beFrRa3{2Ew&&S%5)!x|Z&BAmlH?gUpP+Fdd{a&Tgr3q?P5 zO~oj50~y$~1Ymm2+vA1OP|+_XV6A~}QA89CS=Z?DN@cui7a$_PJ~0u3OM8@?Z_dE|z z@Ys2!$+M?6IOlQg#;(BRL|uM|K~Ah6GC%G~zW+kC))@&t-<-aDfX=(z4+eK|s#z9j* z*Ei?#$7WQIZgo^T?bKyu$AqFD0mb4;^*xbleECq^Jh@#N6u3$3c8A0v z*60Ps0vqw5t^!4L=NXQe(96a1>3xf*b_fA}9;$m^sZd&b>EbP!_na(7&nQquV^Z692xF(;Howb8_@&nCvI*(7iK zbz>|vxW&rm4^N9gyjXjR|7tz}#;nZk!38S!n1-U1of8(gQQaClq5`k+Y_aR3EMDy<`}~=wlgaN-0=tv~SAT6P z-EMX%z3hT}+}hnC_!^wH))S zXWPt55Vsi_hSnsq*=pZSy_iW@-a@{l(Q50ipY^Yd#B;wHY0Byiqp#KSBm3rOd~NZ9Fd~W0m(Lp&1WC02A3X6@)R$kq$r>ALXvTt z^IswygclT_pWE;W%~fKB%V_@9 z!2i!?g%$Sjo`VbL+BBd~7z78z7j9;)y3L0;r%fFq9a%Aelszigq)(dAuupTR4%$d7mM2@%rJ{* z>9GEimVH&o$h@5Prx%!OFhH{q-X`Jgx&foyv9RK+Z^_>u{^$gR?uH3x^MS@#`Lf&9&wuQ3kB!n$o}aZ~!&3lrXr_0I_@EtzJcjzhMAE;Llh_O6LAh|OH)*;cVD za9_@r;{_trvckI1N&InTcM19(QHO=Zqub&m=eY>%vfjto2;esb{RJbh(~ey*L}>ZZ zyA*eUntZJGccPkkeID<77!~2%8pYcW!vb<1hrV_C1iUW9%?bRg(Fl!d-90(FClIWf zUQ<-tMg1exb?Zm#U;oj>|AsK{g|ms^ST98V5nWZ-Gv7zMb|`T@h=Hw6UvkOsT{*_Z zCYj*Bs)i~4t^^&%9AWO{ZMJ4pNrA0l7NV=bxWCr!%$@DI9LIHEIhF?MS{Ni0|7jr1 z<^P|h=8a_|x`nU(6p*GC?!KHSe-^W$yNYIpvxC(8c^)4eABA33r9FPQHCm(cP5 zubKZbHvji{`~QcXc0u0jN_;=&^AG>`+<Yx0RD-bS&Z8z6BUD1@C#xAZ5YzcliK3an6nBoVXcTCf>gG65I zz_I+hWqwlqi0e9KgEMA$_q+bZH+rM;su9ra(p6Hg4DT&7RbWO+TxcV4vaKmP9~d1d zEu&&qW!myLD8IR7hmv$DQg?aO&0`JlngtH>;5{;VN&Hr<)5?2nb~6vtuEcaNMnz9~ z*uN^j=J{XK$Kyrbi=rrYt1mR06jryhtw_x`Dg|qI>%Y~vu=@lhZ%ngtM09kPYU{VEh8VPr zM&P#m{!H6B^RJxm-8H=aj}yZ6SWrCG3N)D>!$54i1XK?iT34 zj7>%ltTBC-6pVBwm;4o-3?Re&W4AjAWUzVHM3TqVPN}dnuxHFdQ|80DOCc#8HH`DA z%&UuhzPB`+6FEBVdlEc9&M#T#7rQMlos9=}5RA_isQ#D(x;Is_pt({vsVRBIm;WRR z(2|qOT284`bm;;Ay*3w#o89B=}WDVmj#Y$7sZS9c+_OWUxV>g?@=<%?X0dHvDSAq3^c;_m$x;mQvhbMbcVLjD5NF96R}lG zK?X?9?=V8+uiosXK{Df(obB_>Z+0#OG|RbhZx?)Goh3M%eK<~qf8(-1jn*(nI;HI}o1irLz3HAqTd3`_lExhPd zW4JsR{RHEOPhDf}U5lNgeOMpn1_)lH7;oQagv|=+Dt)cpxEPHQ zaRO)3YS(l%TBA|VMXPp;=!_?8$u{F5In))qU{=LcYUdG>lLw*RMfWaf)By*S2czY3 z>nY=#0Xa~yh#$;-y)kE*1+p>Xw>EI$*<)-01Ue#&SIz9%jwd>?TNXcAe-A| zG+#Q*5I(D{?VWZWI`m6f!>(1(wBxMH24~$bgnG4!mmwU%uP{)Y{e+hWQ=NO--Q;lq z++5tSAxL~0WWN^-ma=j|7W2sfF`w>fNKyI&8uA1b7Dwp~=Hm9;(iNd8EwDfFD}W=T z33BsKW>QQqQ2yUV>c{FoF(#}ftelb`!A9`5o42^3+ee9DkZ(G+Bo(Alx>F|L$Wl|r z`%WBeABwRKL;S`2-rSPI^fQqdlVCwkb6sud%`xM0xs>#%kmLDxiRW&JFg19Q{mvza z!vrz0kUcFe1JpeInRAzu8D;3G8Q$0kY!U!q`E-`|YCc4(?I^7K4thTVj8%c9Gz3ox ziplKeLu6V$#e1plps^3Y2)bATO1uB7|O;quPP^vT>a@81Oy ziHuiaQtgKDf_2iX>3ftn4?A(YrwqP*lDjj$<6p(^+rEQjJ*jfEf?t1`ZvVZT zkqE-{nhyN2J-W`?qJK?#%x(^eJKP_X0g6P$Q41~3@$cP*2?az{mHb$-S@gY62|-y% zX~S>Wxys5q)zPk06fSKZss-8)N#8ckc5*-QJdiH9-1uy!$!v1#(fFbH?F2wG_?`q> z5IJ+|YAY~=6WyABqL<;HN(K{ohgF)3%A;M~ua?X)yMF`8>x*)0NP7b``ua0nAG?i3 zm(-(o84%W5`pfeO?QDN4*RGaACHT{HjCls)czgt&8XW)lYCP1c*+{09MpZiThOS+M zo|##0_x2PY0LgaVcp2L`Y8PZiu`Q7PNJLu8$Rc0a!?g0Ug_>MKA==Bk!WomXB3--- zj)IptMRT}%r!4GlR1thW7Ti5c^9C?J+x6q2LEq34taSyhYr=_Huif5t zl-vi?aUVXEKRIrX5rX8oRl45u5Ajh0OS^ViSumoEoFdE|&rC>8VklD%uZVl6J;WK4 zR4vUpRO=1iXWs3OpTn;Hbg!q<1poVjmHNZ;dUG{QUSLIcN~cB0wXy>n>j#RwG`X_c zeVV4Z4w{9W7=qq&`FI4!-;mw`0SUC(5>5d=2}!1eh(goW)Eqm&drQlEcPpvV$GkV? z)t=Cvd+^-@RT!O4lo-MIoOX;@0Nr}AJwk9d-4tmMRGsI?z)@6=KW>l;QCxr7Lot3X z7E=BhjA)y&dAl8fy&P$A5VmF)Wfhh>sxv%KK$NKC;UI7yyPUcQDrSBys#fK8-fORN z;}Qd~dImLAhIh39T{b&w=rh41!YlmSU?*-Alr;KFb(YZGAfzc8sR5CbwJBLUn$+c$ zU;RPS*@dP~Gn$=RhZ|Cm)AYDgtqQA~2Kn0c6>uzB=uTf@k&<6FJWdnU!rGS*_@*I*hpVk#x z=$0Ob+J`QybMPlG8KqGS@3YzO?*xtUjp%ou0lHT6Y#qz$+d&hSP8(-`3aoB18(D$n zomyKv0Wt15(G?0;13%r_ce}_P%bE((guOAUW1luz4ZGwwsGp0QPTc5~9s3s4Ce}@p z8Qz8f5h*{MtN0GR@1Y$X;`yQuXtI{ERsMbWt4f=qxu^Y!_R*>f2dgx=4q9BMK|n+> z{px&P+j_bG#$5Z9d$qpxE@jiUA+(v1vw)nNd>6Nt6XQJWwP<|Ls&LDEyoYGYj9B8l zrZ`J+T6a_XE(1I;O9=|;3nXgmt|ij+Qa6&qk|CNDjeb7*I{C`5?BtAh)$74(;N0}S z{V)0Ru{g1*sC!qROXwdk!uhPhO`1=M$74mzL~hXZ@z+anj^M|jNK)6(`oJA8mrYPQ`mg|&8ys68GQFdEt086aZ0`L>` z{c7BEdoP+9W61XV)nq|&hV$=k&4(86;a{0PZ`X}9VgklS9ZPm-oj6B38VV9H7r#{& z?`b{Xld~yj25)buW5s%snZ&P+VRSB~^+4|TkO=@@ikOnlOD5W_ri*cjm_UX<(6{O2 zwIf;VM$c?s_MS_OhjMKOi^ynKGj0nfCTt%cwU(-}4h$0I@lEJ6-DzI&W(<}jv?7~4 z%GX+tFS1OvF`-$j*H4*xDbzM&r66nhwWmfMNcTR)f66=MzUx*XdBXJiDe-jjceD<>+YH0OC0rsoC0N{6w&Y0Jf8=~#L&pI7Url`i0wD+CiR7ADh8Fv1jjn4Ab5bIqVi_8$sqKG=dq zjkjy*L<*tvjNe}AE%ALQEq1!KAyL$nCJ8ObhztZ-m*eomh-lcAiA;USC}9G;uUx5M zCpgpDfAQ1trJHTknfi zk;)b>(c^QVyAydjr`LSB3?1;tv@EU>ywwy<2ZLJKSYPhXBpK4@L*roBxL}DuMBFBz{K4M16`nzwo^teUvg%kgcPYMl}_06w+Fmk*2a^y;Os1Wpm zZ>qQX-lvbx+Uv;BrmBxCda-=ZzjtttJcXpg+#u(ORu&6n-j{ebOs}Vi=loh$@4SpX zcHZZ#F8qs&j|UI^PV;O0bw=5?9BBf9f2$28l@dY@>UxBm?=7_MrUShr+$k_{XD6G3 zZDa|h4+8ItwKn(^44e!%?}suXS`sKH1oy=qsO;|5HAHl*`QG;V?{ZDla(_y_w{%;H zb1Izm2v;w!DBD73EeOjS@6#?HUvPo*6mh4LQ!|n^gf!hghUwWi^9YC z#dU+M#`6%BkG=bsoGF-SEvH_@Q>GG6y~;40EgODsYn}mq;K-(^s69UR}jF5zPyZKDOo(Gq37_ z_5HmV4mQ@cauC3Y$B9aX?6KRM54nHI$SO8}tP~TarS`LG#HV7a=9_TO4$Ciai|JF^ zzdCD<68i4^zUiLw&J~9E4o+QlP~GuKP-;%8{`KVnv;92<=)4&C?Gq7&@$}b)K#S^t z#g-3A>Dq2weW$Y3fmMhENnb1ROXBO<&yAL3Cz z0S>73EurK>D;|HSbluHQTbj^=Y}C2ep2?fZeV7o)E-?G90!_;F=XkeH&~Ld|j5`GM}2k8y?`s$2&6bM>hYwaVygej<`Mj z^yy0s!rDSF=_r#b{(ASAZf;uE}U ztqO~=Wo52YtqMz;MiX3<9d5WBp$+b<94=d}au!T|v;E^#%A*ry9kK1Ys2^WO{0=Cx z_{JLVCy2D<|88Ra*PlH6AXES((YkR-rW%d&P1^4EGI?`nYT`R0f0J$<{eEeLLo`)4 z{L)O+dZkddcWX7ogtcgZQ$;laUR2#G5AwXR5xlwB-VYiBrxy$k7<%tpZi&~pv~E+= z)6f3ANKhB+D4)%chaE{B+nBOY7d(g8>z$Ago}T7al;6P5csiaN)Ta<#A#6cZ zJMcyR_@0g#G^oya4ry_GCgqhO#zF#qzGzKmr%8lxSPs2O9iJaQYSdmSQnoO7EreV26Y#Gd#;~-a={*ajbijH47=#@JpI*#DrV4I9kq9? zOX&VT0tpND^uo!pBKP8>Z*W+TkU$d@POTsKDmg+l=wzZExf#iNWTnK@xSXbaU7?)oa{g%Q^lUZA(5vf&+c!6KfSYqlzp>F#rw~<30C~+a=u(B%wJBtc)^oVA8lEcMvm<+BSe32frr)BCGm*ecY zdfy@23Cb}&CruA&u)(2WJ$|}rhaNi)l~8(ef*ysr>3YZ*bRh-H)<(|A0|A@jx5tW-T7;kiPN={czycf-Zr z0002+;o;dD00000000000R9MK3IG7W4Zy_|IDer@r`)RAW?ER7tD5nMF#c?8L?&ix zXlkdC(J?w)a$e7u?aWe=<5jDshic-2>GauZ{$~KSclGF>*DNfgK=Jo=8a_VnJU`>j z-N)6>|KVGgs*m}-H!Y@{78TO^#f7wL@l0AUE04k>BaDVqw2Xs8y0d^F+w%!C`8s_F zQ(4`pX-$2b#xam@pm1sQx~Q$ln&Wu_)j z9G?@Y5&ykZR#~U_lP%p=%{Tu3axXzNs}kv5kFiVr$0vidfe;cHsFB-cJ3aAVmTeXn z%*dwE;SmZ74WlE);Z=3JLPi$nY?gR+u_UEQai?6t%S2b-0vL5~4L; z!9wl>ZC$>e|NntF4X->eJvmOb+RJ9=)5^Ki)hA#2MUIv8rqSvJ(`n6u0-BMNuG)FA zW|-7N#^GWLRGx37lJoUi$5rzR^fr;vu~buAuUqHIwt3=zFI!d4EXd{d52U`{K8lVE zqm$*eY6&1QD%P!B%<_<@?Km|pPUC7wEP}+?C<+e?HKN8hmn{orn>yLrQMT=r*aQn_ z=ILhv+?D~7vb;MDy!5xk3lPo!p5rDihFEzlD$3V$bQcA&4v~Tzai+okA%1p*mhZNe zb5!QnuUbO0w0wCN%g3^t6c?qj0c1x9xrNA+ATc}Uah(-g7Kt?=k^J?Lyj*Y%Xj%E5 zk=#DDjjg)fVl|IFXZa)BElQb-Se|u@O_q%2mQLLs^M!2(sj0o2$NEfq?A}{x-ppw% z2T6KOX{Bt_*`wtWUw>(kFm8LbsE~2v31PXd(KeK=4kcZ^a0Ug-E(h|t)>EvK#%M2( zAL$1{;w~(nGo9)inyII!S9f)&sA=Rj8>7|p0dv78_k@a zrTOPyGkfAoRPOrL4$b%W4QMJm7DyC&;Ul-N&~#*Ul<#Bq`tI%<)O1R60*#W2y|7~(d6?sc=jdsQ8O&oTpp!!&BOoI!T~_VCaU`sBT9HJ_cCtm$O;upf`d zTA-+gcOpfq`+B!!9TSU?tHcv*e8cx1Jws31{!L4W(NrujWcL6;qU;NQ@xWS&i8Y@O zak~l6ZQe_VPn-ScGg9I;m6#bFJ^h+ryHIwrAi8bUoQoDAwoPBzc9`DSeN^+}b1no2 z2GY`^e9doIK3h}1(6j^d<_-qp0Swuy4mO}h;a85xsZB%;E@H2*(8W5%(>L3nZN ze$7|bwop@>nPa+o%^Z`PE^~|-H;>%9RMVLFSbF{4gPPy;p6MfryCD`Ef}3~5O6Dt% zZ`6FUEz6lR)%5f$J2Y>Hs#;_8^$%-4G}Or5$M0T6>2|)jmN5bV0O0+YZ%Y6G00000 z0002^9k3Jt0Dv2Siz!f5V|?<u^Kb1r zp{eZm?zHZK9&_KGzJ7|g8ur^)&e2pNkU#bEcFhOc=VzoO(va1*$v)>od3C){Yc0Of z?;bd<`E;vIFPc7u#EL;EKWF^otLj>5jotI#>{Tz?z5uGJZ`Hi>x9;xkR}G{f@ezdl ztQ4QWe@R8HrqzvYw9aaa#m`u{+86@A3+L}F&n&4j<$qzzK22*H+UUu9Ox;&6o298} zoSjDUK(Xlu*%w}X^aW=XFp;0QC(xsQ^}_DsCN{&aBc+sKwa;^Am^N5GC-0&~#2>l# z)Ad!q?3I7k>|I|eu`w8trR-Fr`E@ZN!ox#-RvxlLf*||SA2@B=vdqS%5FhcJG^62O zGAmD0iKOrP-L$o;EC}wL4iKk6!D>!+6;rwb$5d$D@5M$aUkNC$LpcJWFE zz~%S*ec-E;doD|&cfgK8{qLo?#Q_SDKC{=l@=LvJJ;#q@U!5*#Gk15nhq>N6aB-|^ z+X>_Q$Za=j7fJMfwvC)?g* z0Ptsy)4rdrs-quS_W>7lPqu_}n)tfrc1@GxW9ZWltTXw9SQ;C7zvlJN1z+$%n{GdJ znqJ>&g3^h*g|K|?bh>x_Ld`oj4cQiPviP=Vv)a_NwM~?pX3VH#q9ZiDn)?9&U?~6q d0M`$q{~s-H%>w*@DjWa+002ovPDHLkV1g@kWRd^? literal 0 HcmV?d00001 diff --git a/docs/guides/databases/vector-db-benchmarking/index.md b/docs/guides/databases/vector-db-benchmarking/index.md index 98859c83f07..98f85df595e 100644 --- a/docs/guides/databases/vector-db-benchmarking/index.md +++ b/docs/guides/databases/vector-db-benchmarking/index.md @@ -349,11 +349,7 @@ The benchmark measured two accuracy metrics that are critical for vector search: Calculate QP$ (queries per dollar) to compare cost-effectiveness. -``` -QP$ = Peak QPS / Monthly instance cost -. -HigherQP$ = better value -``` +![QP$ formula](QPS2.png) Consider these example benchmarks: From 0f90b745573f0d20fc982b92ebcb04e6fa726667 Mon Sep 17 00:00:00 2001 From: Dianna Hoober Date: Wed, 4 Mar 2026 11:51:07 -0800 Subject: [PATCH 10/10] Fixed spacing --- docs/guides/databases/vector-db-benchmarking/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/databases/vector-db-benchmarking/index.md b/docs/guides/databases/vector-db-benchmarking/index.md index 98f85df595e..125b0c53e06 100644 --- a/docs/guides/databases/vector-db-benchmarking/index.md +++ b/docs/guides/databases/vector-db-benchmarking/index.md @@ -54,7 +54,7 @@ When benchmarking vector databases, track metrics across the following three cat The fundamental tension in vector search is this: you can make searches faster by sacrificing accuracy, or more accurate by accepting slower performance. This isn't a limitation—it's an inherent characteristic of approximate nearest neighbor algorithms. Most production systems target 95-99% recall, finding that sweet spot where results are accurate enough for their use case while maintaining acceptable query latency. Understanding where your application falls on this spectrum is essential for proper benchmarking. -## Benchmarking`pgvector` in PostgreSQL with VectorDBBench +## Benchmarking `pgvector` in PostgreSQL with VectorDBBench `pgvector` is an open-source PostgreSQL extension that adds vector similarity search to your relational database. Instead of managing a separate vector database, you store embeddings alongside traditional data and query them with SQL. It supports two index types: