Skip to content

feat(supabase): add async support to SupabaseGroongaDocumentStore and SupabaseGroongaBM25Retriever#3380

Merged
davidsbatista merged 10 commits into
deepset-ai:mainfrom
Aftabbs:feat/supabase-groonga-async-support
Jun 4, 2026
Merged

feat(supabase): add async support to SupabaseGroongaDocumentStore and SupabaseGroongaBM25Retriever#3380
davidsbatista merged 10 commits into
deepset-ai:mainfrom
Aftabbs:feat/supabase-groonga-async-support

Conversation

@Aftabbs
Copy link
Copy Markdown
Contributor

@Aftabbs Aftabbs commented Jun 1, 2026

Summary

  • Adds warm_up_async() and a full set of *_async() methods to SupabaseGroongaDocumentStore using supabase-py's AsyncClient and acreate_client
  • Fixes SupabaseGroongaBM25Retriever.run_async(), which was a stub that delegated to the synchronous run() instead of using real async I/O
  • Adds 14 new unit tests for the async path across both classes

Closes #3379

Root Cause

supabase-py ships two parallel clients: create_client (sync) and acreate_client (async). The original implementation only wired up the sync client. run_async() on the retriever simply called self.run(), blocking the event loop on every invocation — making AsyncPipeline usage effectively synchronous.

Changes

File What changed
groonga_document_store.py Added _async_client field; warm_up_async(), _setup_table_async(), and async counterparts for all public methods (count_documents_async, filter_documents_async, write_documents_async, delete_documents_async, delete_all_documents_async, delete_by_filter_async, update_by_filter_async, _groonga_retrieval_async)
groonga_bm25_retriever.py Fixed run_async() to call _groonga_retrieval_async(); removed stale docstring note saying async is unsupported
test_groonga_document_store.py Added TestDocumentStoreAsync class with 12 async unit tests using AsyncMock for the async client
test_groonga_retriever.py Updated test_run_async to verify _groonga_retrieval_async is actually awaited (not the sync path)

Testing

  • Existing sync tests pass (no sync code was changed)
  • New async tests: TestDocumentStoreAsync — 12 tests covering warm_up, count, filter, write (OVERWRITE/SKIP/FAIL policies), delete, delete_all, retrieval, and uninitialized-client guard
  • Retriever test: test_run_async_uses_async_retrieval — asserts _groonga_retrieval_async is awaited
  • ruff check passes with no errors

Impact

Users running SupabaseGroongaBM25Retriever inside an AsyncPipeline now get true non-blocking I/O instead of a blocking call that defeats the purpose of async execution.

… SupabaseGroongaBM25Retriever

Implements the full async interface for the Groonga document store using
supabase-py's AsyncClient and acreate_client. SupabaseGroongaBM25Retriever.run_async()
previously delegated to the synchronous run() path instead of using real async I/O.

Closes deepset-ai#3379
@Aftabbs Aftabbs requested a review from a team as a code owner June 1, 2026 18:11
@Aftabbs Aftabbs requested review from sjrl and removed request for a team June 1, 2026 18:11
@github-actions github-actions Bot added integration:supabase type:documentation Improvements or additions to documentation labels Jun 1, 2026
@davidsbatista davidsbatista requested review from davidsbatista and removed request for davidsbatista June 2, 2026 06:58
@davidsbatista
Copy link
Copy Markdown
Contributor

@Aftabbs thanks for your contribution - can you please make the CI green so that I can start the review

@sjrl sjrl requested review from davidsbatista and removed request for sjrl June 2, 2026 06:59
@davidsbatista
Copy link
Copy Markdown
Contributor

Regarding the tests, since we are testing against a local Docker instance no need to mock up; please convert them to integration tests

- Fix ruff format violation in test_groonga_document_store.py: collapse
  3-line MagicMock call to 1 line to match 120-char line length limit
- Add async integration tests to test_groonga_integration.py:
  TestGroongaDocumentStoreAsyncIntegration covers warm_up_async(),
  write_documents_async(), count_documents_async(), filter_documents_async(),
  delete_all_documents_async(), delete_documents_async(), and
  _groonga_retrieval_async() against a live Docker stack
  TestGroongaBM25RetrieverAsyncIntegration covers run_async() with real data
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 2, 2026

Coverage report (supabase)

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  integrations/supabase/src/haystack_integrations/components/retrievers/supabase
  groonga_bm25_retriever.py
  integrations/supabase/src/haystack_integrations/document_stores/supabase
  groonga_document_store.py 136-137, 140, 168-169, 211-212, 222-229, 461, 476-477, 479-481, 486-515, 527-528, 537-541, 570, 580-593, 601-602, 608-609, 623, 633-634, 682-694, 731, 743, 761
Project Total  

This report was generated by python-coverage-comment-action

@davidsbatista
Copy link
Copy Markdown
Contributor

Hey @Aftabbs, thanks for the fix to run_async()

The new warm_up_async() method is inconsistent with how every other integration in this repo handles async initialization.

Looking at others, none of them expose a warm_up_async(). Instead, they use lazy initialization: each async method calls a private _initialize_async_client() at the top before doing any work, creating the client on first use. For example, from the Qdrant store:

  async def _initialize_async_client(self) -> None:
      if self._async_client is None:
          self._async_client = qdrant_client.AsyncQdrantClient(...)
          await self._set_up_collection_async()

  async def count_documents_async(self) -> int:
      await self._initialize_async_client()

Copy link
Copy Markdown
Contributor

@davidsbatista davidsbatista left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have handled the remaining issues. Thanks for your contribution

@davidsbatista davidsbatista merged commit 8b35e9d into deepset-ai:main Jun 4, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integration:supabase type:documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add async support to SupabaseGroongaDocumentStore and SupabaseGroongaBM25Retriever

2 participants