[BUG] Hybrid search ORDER BY RANK RRF throws IndexOutOfBoundsException — coalescedAndSortedResults Mono subscribed twice without .cache()

[MarioPadilla]

Describe the bug

Cross-partition hybrid search queries (ORDER BY RANK RRF(VectorDistance(...), FullTextScore(...))) intermittently throw IndexOutOfBoundsException at HybridSearchDocumentQueryExecutionContext.lambda$computeRRFScores$3, e.g.:

java.lang.IndexOutOfBoundsException: Index 363 out of bounds for length 363
    at java.util.ArrayList.get(ArrayList.java:428)
    at com.azure.cosmos.implementation.query.HybridSearchDocumentQueryExecutionContext.lambda$computeRRFScores$3(HybridSearchDocumentQueryExecutionContext.java:343)

This is distinct from #46267 (already fixed via CosmosFullTextScoreScope) — it still reproduces with CosmosFullTextScoreScope.LOCAL set, and is unrelated to full-text statistics scope.

Root cause

In hybridSearch(...), coalescedAndSortedResults (a Mono<List<HybridSearchQueryResult<Document>>>) is built once but subscribed to twice independently, without .cache():

Mono<List<HybridSearchQueryResult<Document>>> coalescedAndSortedResults = coalesceAndSortResults(componentQueryResults);
Mono<List<List<ScoreTuple>>> componentScoresList = retrieveComponentScores(coalescedAndSortedResults, componentWeights); // subscription #1
Mono<List<List<Integer>>> ranks = computeRanks(componentScoresList);
return computeRRFScores(ranks, coalescedAndSortedResults, componentWeights) // subscription #2, via zipWith

Since the upstream source merges results across many partitions over real network I/O, each independent subscription re-executes the whole cross-partition merge and can non-deterministically produce a different-sized result list. computeRRFScores then indexes into a ranks list sized from the other subscription, causing the exception. Confirmed still present on main (checked against HybridSearchDocumentQueryExecutionContext.java on the default branch as of 2026-07-15).

Suggested fix

Add .cache() to coalescedAndSortedResults so both consumers observe the same materialized result list:

Mono<List<HybridSearchQueryResult<Document>>> coalescedAndSortedResults = coalesceAndSortResults(componentQueryResults).cache();

To Reproduce

Cross-partition hybrid search with k large enough (empirically ~≥90-100 against our container) to increase the window for the two subscriptions' network timing to diverge. Not reliably reproducible with a small local dataset — occurs intermittently in production against a multi-partition container under normal query load.

Environment

• azure-cosmos 4.80.0
• Java 25
• Spring Boot 4.0.6

[github-actions]

Thank you for your feedback. Tagging and routing to the team member(s) best able to assist.

//cc: @kushagraThapar @pjohari-ms @TheovanKraay

[github-actions]

🎯 Agentic Issue Triage

Summary: Hybrid search queries using ORDER BY RANK RRF(VectorDistance(...), FullTextScore(...)) intermittently throw IndexOutOfBoundsException because coalescedAndSortedResults — a cold Mono — is subscribed twice independently (without .cache()), causing the two subscriptions to produce differently-sized result lists in cross-partition scenarios.

📋 Issue Details

Package: com.azure:azure-cosmos:4.80.0
Affected API: HybridSearchDocumentQueryExecutionContext.hybridSearch(...) / computeRRFScores (line ~343)
Scenarios:

• Cross-partition hybrid search with ORDER BY RANK RRF(VectorDistance(...), FullTextScore(...)) against a multi-partition container
• Occurs intermittently in production when k is large enough (~≥90-100 results) that timing between two subscriptions can diverge

Root ask: Fix the race condition by adding .cache() to the coalescedAndSortedResults Mono so both downstream consumers observe the same materialized result list.

🔎 Debugging / Reproduction Notes

The reporter has identified the root cause precisely: coalescedAndSortedResults is a cold Mono built from a cross-partition network merge and is subscribed twice — once in retrieveComponentScores(...) and once in computeRRFScores(...) via zipWith. Each independent subscription re-executes the merge and can non-deterministically return a different number of documents, causing an index-out-of-bounds when one list size is used to index into data sized from the other subscription.

The suggested fix is straightforward:

Mono<List<QueryInfo.SortOrder>> coalescedAndSortedResults =
    coalesceAndSortResults(componentQueryResults).cache();

Suggested investigation steps:

1. Review HybridSearchDocumentQueryExecutionContext.hybridSearch(...) on main to confirm the double-subscription pattern
2. Assess whether .cache() is the right operator (vs. materializing eagerly via collectList() and piping) and whether any memory/lifecycle concerns apply
3. Add a regression test that exercises cross-partition hybrid search with a large result window to prevent recurrence
4. Verify the fix does not interact with issue [BUG] Cannot perform queries involving ORDER BY RANK FullTextScore using the Azure Cosmos DB Java SDK #46267 (already fixed via CosmosFullTextScoreScope)

🏷️ Label Confidence

• Category: Client — The bug is in SDK implementation code (HybridSearchDocumentQueryExecutionContext), not in the REST API or service
• Service: Cosmos — Explicitly references com.azure:azure-cosmos and the /sdk/cosmos/ module
• Confidence: High — Package is explicitly named, category is unambiguous, and the pattern matches quality reference issues (e.g., [Query] Java createAndInitializeObjectMapper can I override the default ObjectMapper? #36246, [QUERY] When will Azure Cosmos DB OLTP Spark 4 connector be ready? #46290) which carry Client + Cosmos

👥 Owner Routing

• Matched CODEOWNERS entry: # ServiceLabel: %Cosmos (line 295) — service label Cosmos matched directly
• AzureSdkOwners: none listed
• ServiceOwners: @kushagraThapar @pjohari-ms @TheovanKraay
• Routing action: Added Service Attention and needs-team-attention labels; @mentioned ServiceOwners above
• Scan notes: Scanned bottom-to-top; no more-specific multi-label entry for %Cosmos %Client exists, so the %Cosmos entry (line 295) was the first match

Generated by Agentic Triage for #49823 · 29.8 AIC · ⌖ 6.63 AIC · ⊞ 11K · ◷

[MarioPadilla]

Additional evidence that strengthens this report, and raises its severity beyond just an availability bug.

Correlating logs at the exact request that throws the exception, I found com.azure.cosmos.implementation.query.Fetcher logging this WARN repeatedly, immediately before the IndexOutOfBoundsException:

WARN c.a.c.implementation.query.Fetcher - Azure Cosmos DB BackEnd Service returned more than requested 10 items, Context: n/a

(fired 11 times in the ~2 seconds before the exception, for a request with k=10)

Looking at Fetcher.java's updateState(...):

if (top.get() != -1) {
    top.accumulateAndGet(response.getResults().size(), (left, right) -> left - right);
    if (top.get() < 0) {
        // this shouldn't happen
        // this means backend retrieved more items than requested
        logger.warn("Azure Cosmos DB BackEnd Service returned more than requested {} items, Context: {}", ...);
        top.set(0);
    }
    maxItemCount.accumulateAndGet(top.get(), Math::min);
}

top here is seeded from the query's own TOP/LIMIT clause (in our case literally our request's k=10, from OFFSET 0 LIMIT 10). The comment // this shouldn't happen indicates the SDK assumes a single partition's fetcher should never receive more rows than the overall query's requested limit — but for hybrid RRF queries specifically, this assumption appears to be violated routinely (this fires on ~28% of all log lines sampled, i.e. on nearly every hybrid search), which is exactly the kind of per-partition non-determinism that would cause the two independent subscriptions of coalescedAndSortedResults (described above) to observe differently-sized/differently-ordered result sets.

This also implies a more severe, silent variant of the bug: if the two subscriptions happen to produce lists of the same size but different contents/order (very plausible given this per-partition non-determinism), computeRRFScores would zip a ranks matrix computed from subscription #1 against results from subscription #2 with no exception thrown at all — silently mispairing scores with documents. That would explain intermittent relevance/ranking regressions (a previously top-ranked, clearly relevant document disappearing from results between otherwise-identical requests) that never surface as an error in application logs, which is harder to detect than the crash case and likely under-reported as a result.

Wanted to flag this in case it helps prioritize / scope the fix — happy to share more log excerpts if useful (redacted; our data model doesn't allow raw log sharing but I can provide the same shape with synthetic values).