feat: add OSS OpenSearch and semantic FTS support

[jamesgao-jpg]

Summary

This PR adds OSS OpenSearch and the backend support needed for both unfiltered and filtered full-text search benchmarking. It also publishes validated permuted filtered-concurrency results for Elasticsearch, OSS OpenSearch, Zilliz Cloud, and Turbopuffer, with a dedicated Filtered QPS frontend view. Filtered runs are kept separate from the standard semantic recall, NDCG, and MRR views.

PR #794 introduced the preceding full-text search benchmark foundation used by this work. This PR supersedes #814 with a reduced publication scope. PR #818 introduces shared CLI run-context notes; its commit is incorporated here so FTS results preserve deployment and constraint context.

Changes

• add the OSS OpenSearch backend client, configuration, CLI registration, and focused tests
• retain unfiltered semantic FTS metrics and controls
• restore filtered FTS backend cases, dataset preparation, runner orchestration, metrics, and CLI --fts-filter-rate control
• distribute FTS filter_id values with a deterministic bijection so exact-selectivity filters are scattered across corpus insertion order rather than selecting a contiguous suffix
• expose --fts-filter-id-distribution [permuted|sequential], defaulting to permuted, and propagate the selected mode through case construction, dataset preparation, qrel filtering, and result metadata
• restore filtered FTS request translation for Milvus, Elastic Cloud, OSS OpenSearch, TurboPuffer, and Vespa
• add a Filtered QPS frontend tab with dataset, backend, and ID-distribution filters
• show filtered concurrency QPS by backend, filter rate, distribution, and concurrency without presenting zero-valued serial recall fields as measured recall
• publish 40 validated permuted result JSONs: ten each for Elasticsearch, OSS OpenSearch, Zilliz Cloud, and Turbopuffer
• cover HotpotQA Large and MS MARCO Large at filter rates 0.50, 0.75, 0.90, 0.95, and 0.99, with concurrency 60 and 80
• remove the legacy sequential Zilliz filtered artifact and the published Vespa FTS result while preserving both backend implementations
• report OSS OpenSearch bulk item failures with the actual successful-document count instead of treating partial writes as full success
• add shared --note and --note-file CLI options while preserving Pinecone compatibility
• preserve current upstream backend and compatibility changes by incorporating the latest main

Result Provenance

The Elasticsearch, OSS OpenSearch, and Zilliz Cloud filtered results come from the approved 2026-07-16 total-series benchmark executed with VDBBench commit e24110b (feat(fts): make filter id distribution configurable) on a clean fts_v2_backend tree. The Turbopuffer results come from the completed 2026-08-02 run executed with commit 07bcefd (feat: publish permuted filtered FTS results) on the same clean branch.

The first 30 results were copied byte-for-byte from the selected total-series evidence bundle. The ten Turbopuffer results preserve the measured result content with only db_config.api_key cleared before publication. All 40 results were validated for:

• custom_case.filter_id_distribution = "permuted"
• additional_parameters.fts_filter.filter_id_distribution = "affine_permutation_v1"
• valid affine multiplier and offset values forming a bijection for the dataset cardinality
• concurrency levels [60, 80] with two QPS measurements
• serial search disabled, so recall, NDCG, and MRR are not presented for these filtered runs
• embedded backend run context with published credentials empty

The benchmark used 30 seconds per concurrency level, k=100, IDs-only payloads, insert batch size 1,000, and load concurrency 4. OpenSearch and Elasticsearch ran as isolated single-node Docker deployments; Zilliz Cloud used Task 15844. Turbopuffer used the managed service in aws-us-west-2 with the same filtered dataset, rate, and concurrency matrix.

Turbopuffer Run Caveat

The Turbopuffer run observed service-side HTTP 429 responses at several concurrency points, including 63 responses for HotpotQA Large at 50% matched documents. The published QPS values count successful request throughput under that service limit; they should not be interpreted as an error-free saturation curve.

Deferred Scope

• publication of the sequential comparison result set in the frontend
• filtered semantic-recall presentation for a separately designed run that enables serial search
• session and documentation-only migration files

Compatibility

FTS collections loaded before commit bf85cf7 contain sequential filter_id values. Reload those collections before running filtered FTS search so stored IDs and semantic qrel filtering use the same mapping. Changing --fts-filter-id-distribution between permuted and sequential also requires reloading the collection.

Verification

• make lint
• pre-push CI baseline and focused frontend/metric tests
• 80 focused filtered-FTS backend, case, CLI, dataset, runner, frontend, OpenSearch, and TurboPuffer tests
• Streamlit AppTest smoke: six tabs, no exceptions, four filtered backends, one explicit Permuted distribution, and a 40-row filtered concurrency table for the selected dataset family
• filtered chart transformation: eight traces covering four backends at concurrency 60 and 80
• 40 result files validated for metadata, affine-bijection, matrix coverage, and empty published credentials; the ten Turbopuffer files are semantically identical to the source after clearing api_key
• DCO verification for every outgoing commit
• VDBBench pre-push gate and impact-map validation
• exact production restoration recorded in commit 659f739, followed by isolated current-lint compatibility commit a352b73
• deterministic exact-selectivity FTS filter distribution added in commit bf85cf7
• explicit sequential/permuted CLI and case switch added in commit e24110b

A new Turbopuffer provider run was completed for this update; the other three backend result sets continue to use the previously validated total-series evidence.

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jamesgao-jpg
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment