feat: add hybrid fusion strategy(dbsf) support for search queries (#24)

Author: srimon12

README.md

@@ -5,7 +5,7 @@
 [![PyPI version](https://img.shields.io/pypi/v/qql-cli?color=blue&label=PyPI)](https://pypi.org/project/qql-cli/)
 [![Python 3.12+](https://img.shields.io/pypi/pyversions/qql-cli)](https://pypi.org/project/qql-cli/)
 [![MIT License](https://img.shields.io/badge/license-MIT-green)](LICENSE)
-[![Tests](https://img.shields.io/badge/tests-375%20passing-brightgreen)](tests/)
+[![Tests](https://img.shields.io/badge/tests-405%20passing-brightgreen)](tests/)
 
 Write `INSERT`, `SEARCH`, `SCROLL`, `RECOMMEND`, `DELETE`, and `CREATE COLLECTION` statements instead of Python SDK calls. Supports hybrid dense+sparse vector search, cross-encoder reranking, quantization (scalar, turbo, binary, product), SQL-style `WHERE` filters, script execution, and collection dump/restore.
 
@@ -48,7 +48,7 @@ Your query string
   Qdrant instance
 ```
 
-When you run `INSERT`, the `text` field is automatically converted into a dense vector using [Fastembed](https://github.com/qdrant/fastembed). In **hybrid mode** (`USING HYBRID`), a sparse BM25 vector is also generated alongside the dense vector, and searches use Qdrant's Reciprocal Rank Fusion (RRF) to merge the results of both retrieval methods.
+When you run `INSERT`, the `text` field is automatically converted into a dense vector using [Fastembed](https://github.com/qdrant/fastembed). In **hybrid mode** (`USING HYBRID`), a sparse BM25 vector is also generated alongside the dense vector, and searches use Qdrant's Reciprocal Rank Fusion (RRF) by default to merge the results of both retrieval methods. You can switch hybrid search to DBSF with `FUSION 'dbsf'`.
 
 ---
 
@@ -102,6 +102,7 @@ INSERT BULK INTO COLLECTION articles VALUES [{'text': '...'}, {'text': '...'}]
 SEARCH articles SIMILAR TO 'query' LIMIT 10
 SEARCH articles SIMILAR TO 'query' LIMIT 10 WHERE year >= 2020
 SEARCH articles SIMILAR TO 'query' LIMIT 10 USING HYBRID
+SEARCH articles SIMILAR TO 'query' LIMIT 10 USING HYBRID FUSION 'dbsf'
 SEARCH articles SIMILAR TO 'query' LIMIT 10 USING HYBRID RERANK
 
 -- Scroll
@@ -142,7 +143,7 @@ Tests do not require a running Qdrant instance — the Qdrant client is mocked.
 pytest tests/ -v
 ```
 
-Expected: **375 tests passing**.
+Expected: **405 tests passing**.
 
 ---
 

docs/getting-started.md

@@ -24,7 +24,7 @@ Your query string
   Qdrant instance
 ```
 
-When you run `INSERT`, the `text` field is automatically converted into a dense vector using [Fastembed](https://github.com/qdrant/fastembed). In **hybrid mode** (`USING HYBRID`), a sparse BM25 vector is also generated alongside the dense vector, and searches use Qdrant's Reciprocal Rank Fusion (RRF) to merge the results of both retrieval methods.
+When you run `INSERT`, the `text` field is automatically converted into a dense vector using [Fastembed](https://github.com/qdrant/fastembed). In **hybrid mode** (`USING HYBRID`), a sparse BM25 vector is also generated alongside the dense vector, and searches use Qdrant's Reciprocal Rank Fusion (RRF) by default to merge the results of both retrieval methods. You can override that with `FUSION 'dbsf'` on hybrid searches.
 
 ---
 

docs/index.html

@@ -114,7 +114,7 @@ <h1>QQL</h1>
     <a href="https://pypi.org/project/qql-cli/"><img src="https://img.shields.io/pypi/v/qql-cli?color=blue&label=PyPI" alt="PyPI version" /></a>
     <a href="https://pypi.org/project/qql-cli/"><img src="https://img.shields.io/pypi/pyversions/qql-cli" alt="Python versions" /></a>
     <a href="https://github.com/pavanjava/qql/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-green" alt="MIT License" /></a>
-    <a href="https://github.com/pavanjava/qql/actions"><img src="https://img.shields.io/badge/tests-375%20passing-brightgreen" alt="375 tests" /></a>
+    <a href="https://github.com/pavanjava/qql/actions"><img src="https://img.shields.io/badge/tests-405%20passing-brightgreen" alt="405 tests" /></a>
   </div>
 
   <pre><span class="cmt"># Install</span>

docs/reference.md

@@ -36,6 +36,9 @@ SEARCH docs SIMILAR TO 'hello' LIMIT 5 USING MODEL 'BAAI/bge-small-en-v1.5'
 -- Hybrid with custom dense model
 SEARCH docs SIMILAR TO 'hello' LIMIT 5 USING HYBRID DENSE MODEL 'BAAI/bge-base-en-v1.5'
 
+-- Hybrid with explicit fusion strategy
+SEARCH docs SIMILAR TO 'hello' LIMIT 5 USING HYBRID FUSION 'dbsf'
+
 -- Hybrid with both custom
 SEARCH docs SIMILAR TO 'hello' LIMIT 5
   USING HYBRID DENSE MODEL 'BAAI/bge-base-en-v1.5' SPARSE MODEL 'prithivida/Splade_PP_en_v1'
@@ -159,7 +162,7 @@ Tests do not require a running Qdrant instance — the Qdrant client is mocked.
 pytest tests/ -v
 ```
 
-Expected output: **375 tests passing**.
+Expected output: **405 tests passing**.
 
 ---
 

docs/search.md

@@ -14,7 +14,7 @@ SEARCH <collection_name> SIMILAR TO '<query_text>' LIMIT <n>
 SEARCH <collection_name> SIMILAR TO '<query_text>' LIMIT <n> USING MODEL '<model_name>'
 SEARCH <collection_name> SIMILAR TO '<query_text>' LIMIT <n> [USING MODEL '<model>'] WHERE <filter>
 SEARCH <collection_name> SIMILAR TO '<query_text>' LIMIT <n> USING HYBRID
-SEARCH <collection_name> SIMILAR TO '<query_text>' LIMIT <n> USING HYBRID [DENSE MODEL '<model>'] [SPARSE MODEL '<model>'] [WHERE <filter>]
+SEARCH <collection_name> SIMILAR TO '<query_text>' LIMIT <n> USING HYBRID [FUSION 'rrf|dbsf'] [DENSE MODEL '<model>'] [SPARSE MODEL '<model>'] [WHERE <filter>]
 SEARCH <collection_name> SIMILAR TO '<query_text>' LIMIT <n> USING SPARSE [MODEL '<sparse_model>']
 SEARCH <collection_name> SIMILAR TO '<query_text>' LIMIT <n> EXACT
 SEARCH <collection_name> SIMILAR TO '<query_text>' LIMIT <n> [USING ...] [WHERE <filter>] [RERANK] WITH { hnsw_ef: <n>, exact: true|false, acorn: true|false }
@@ -33,7 +33,7 @@ Search only papers published after 2020:
 SEARCH articles SIMILAR TO 'deep learning' LIMIT 10 WHERE year > 2020
 ```
 
-Hybrid search (combines dense semantic + sparse BM25 keyword retrieval via RRF):
+Hybrid search (combines dense semantic + sparse BM25 keyword retrieval via RRF by default):
 ```sql
 SEARCH articles SIMILAR TO 'attention mechanism' LIMIT 10 USING HYBRID
 ```
@@ -126,13 +126,13 @@ SCROLL FROM articles AFTER 'cursor-id' LIMIT 50
 
 ## Hybrid Search (USING HYBRID)
 
-Hybrid search combines **dense semantic vectors** and **sparse BM25 keyword vectors** in a single query and merges the results with Qdrant's **Reciprocal Rank Fusion (RRF)** algorithm. This typically outperforms either method alone.
+Hybrid search combines **dense semantic vectors** and **sparse BM25 keyword vectors** in a single query. By default QQL merges the two result sets with Qdrant's **Reciprocal Rank Fusion (RRF)** algorithm, and you can optionally switch to **DBSF** with a `FUSION` clause.
 
 ### How it works internally
 
 1. Both a dense vector (`TextEmbedding`) and a sparse BM25 vector (`SparseTextEmbedding`) are generated from your query text.
 2. Qdrant fetches the top candidates from each index independently (`prefetch limit = LIMIT × 4`).
-3. The two result lists are merged using RRF — a rank-based fusion that does not require score normalization.
+3. The two result lists are merged using the selected fusion strategy (`RRF` by default, or `DBSF` when requested).
 4. The final top-N results are returned.
 
 ### Step 1: Create a hybrid collection
@@ -165,6 +165,9 @@ SEARCH articles SIMILAR TO 'transformer architecture' LIMIT 10 USING HYBRID
 -- Hybrid search with a WHERE filter
 SEARCH articles SIMILAR TO 'attention' LIMIT 10 USING HYBRID WHERE year >= 2017
 
+-- Hybrid with DBSF fusion
+SEARCH articles SIMILAR TO 'hybrid retrieval' LIMIT 10 USING HYBRID FUSION 'dbsf'
+
 -- Hybrid with custom dense model
 SEARCH articles SIMILAR TO 'embeddings' LIMIT 5
   USING HYBRID DENSE MODEL 'BAAI/bge-base-en-v1.5'
@@ -180,6 +183,7 @@ SEARCH articles SIMILAR TO 'sparse retrieval' LIMIT 5
 |---|---|
 | Dense model | configured default (`sentence-transformers/all-MiniLM-L6-v2`) |
 | Sparse model | `Qdrant/bm25` |
+| Fusion | `rrf` |
 
 ### Dense vs. hybrid — when to use which
 

src/qql/ast_nodes.py

@@ -195,6 +195,7 @@ class SearchStmt:
     limit: int
     model: str | None               # dense model; None → use config default
     hybrid: bool = False            # if True, use prefetch+RRF hybrid search
+    fusion: str | None = None       # hybrid fusion strategy; None → default rrf
     sparse_only: bool = False       # if True, query only the sparse vector (no dense)
     sparse_model: str | None = None # sparse model for hybrid/sparse-only; None → SparseEmbedder.DEFAULT_MODEL
     query_filter: FilterExpr | None = None  # optional WHERE clause; default keeps existing tests valid

src/qql/cli.py

@@ -57,7 +57,7 @@
   [yellow]SEARCH[/yellow] <name> [yellow]SIMILAR TO[/yellow] '<text>' [yellow]LIMIT[/yellow] <n>
       Semantic search by vector similarity.
       Optional: [yellow]USING MODEL[/yellow] '<model>'
-      Optional: [yellow]USING HYBRID[/yellow] [DENSE MODEL '<model>'] [SPARSE MODEL '<model>']
+      Optional: [yellow]USING HYBRID[/yellow] [FUSION 'rrf|dbsf'] [DENSE MODEL '<model>'] [SPARSE MODEL '<model>']
       Optional: [yellow]USING SPARSE[/yellow] [MODEL '<model>']   sparse-vector-only search
       Optional: [yellow]WHERE[/yellow] <filter>   (e.g. WHERE year > 2020 AND status = 'ok')
       Optional: [yellow]RERANK[/yellow] [MODEL '<model>']   rerank results with a cross-encoder

src/qql/executor.py

@@ -464,7 +464,7 @@ def _execute_search(self, node: SearchStmt) -> ExecutionResult:
         # enough material to reorder; only `node.limit` results are returned.
         fetch_limit = node.limit * _RERANK_FETCH_MULTIPLIER if node.rerank else node.limit
 
-        # ── Hybrid SEARCH: prefetch dense+sparse, fuse with RRF ───────────
+        # ── Hybrid SEARCH: prefetch dense+sparse, fuse with the requested strategy ──
         if node.hybrid:
             dense_model = node.model or self._config.default_model
             sparse_model_name = node.sparse_model or SparseEmbedder.DEFAULT_MODEL
@@ -495,7 +495,7 @@ def _execute_search(self, node: SearchStmt) -> ExecutionResult:
                             params=search_params,
                         ),
                     ],
-                    query=FusionQuery(fusion=Fusion.RRF),
+                    query=FusionQuery(fusion=self._resolve_hybrid_fusion(node.fusion)),
                     limit=fetch_limit,
                     query_filter=qdrant_filter,
                 )
@@ -598,6 +598,15 @@ def _execute_search(self, node: SearchStmt) -> ExecutionResult:
             data=results,
         )
 
+    def _resolve_hybrid_fusion(self, fusion: str | None) -> Fusion:
+        if fusion is None or fusion == "rrf":
+            return Fusion.RRF
+        if fusion == "dbsf":
+            return Fusion.DBSF
+        raise QQLRuntimeError(
+            f"Unsupported hybrid fusion '{fusion}'; expected 'rrf' or 'dbsf'"
+        )
+
     def _execute_recommend(self, node: RecommendStmt) -> ExecutionResult:
         if not self._client.collection_exists(node.collection):
             raise QQLRuntimeError(f"Collection '{node.collection}' does not exist")

src/qql/lexer.py

@@ -14,6 +14,7 @@ class TokenKind(Enum):
     USING = auto()
     MODEL = auto()
     HYBRID = auto()
+    FUSION = auto()
     DENSE = auto()
     SPARSE = auto()
     RERANK = auto()
@@ -104,6 +105,7 @@ class TokenKind(Enum):
     "USING": TokenKind.USING,
     "MODEL": TokenKind.MODEL,
     "HYBRID": TokenKind.HYBRID,
+    "FUSION": TokenKind.FUSION,
     "DENSE": TokenKind.DENSE,
     "SPARSE": TokenKind.SPARSE,
     "RERANK": TokenKind.RERANK,

src/qql/parser.py

@@ -44,6 +44,8 @@
     TokenKind.LTE:        "<=",
 }
 
+_HYBRID_FUSION_VALUES = {"rrf", "dbsf"}
+
 
 class Parser:
     def __init__(self, tokens: list[Token]) -> None:
@@ -333,16 +335,26 @@ def _parse_search(self) -> SearchStmt:
 
         model: str | None = None
         hybrid: bool = False
+        fusion: str | None = None
         sparse_only: bool = False
         sparse_model: str | None = None
         if self._peek().kind == TokenKind.USING:
             self._advance()  # consume USING
             if self._peek().kind == TokenKind.HYBRID:
                 self._advance()  # consume HYBRID
                 hybrid = True
-                # Optional DENSE MODEL and/or SPARSE MODEL sub-clauses, any order
-                while self._peek().kind in (TokenKind.DENSE, TokenKind.SPARSE):
+                # Optional FUSION / DENSE MODEL / SPARSE MODEL sub-clauses, any order.
+                while self._peek().kind in (TokenKind.FUSION, TokenKind.DENSE, TokenKind.SPARSE):
                     sub = self._advance()
+                    if sub.kind == TokenKind.FUSION:
+                        value_tok = self._expect(TokenKind.STRING)
+                        fusion = value_tok.value.lower()
+                        if fusion not in _HYBRID_FUSION_VALUES:
+                            raise QQLSyntaxError(
+                                f"Unsupported hybrid fusion '{value_tok.value}'; expected 'rrf' or 'dbsf'",
+                                value_tok.pos,
+                            )
+                        continue
                     self._expect(TokenKind.MODEL)
                     m = self._expect(TokenKind.STRING).value
                     if sub.kind == TokenKind.DENSE:
@@ -397,6 +409,7 @@ def _parse_search(self) -> SearchStmt:
             limit=limit,
             model=model,
             hybrid=hybrid,
+            fusion=fusion,
             sparse_only=sparse_only,
             sparse_model=sparse_model,
             query_filter=query_filter,