feat(arangodb): add ArangoDocumentStore and ArangoEmbeddingRetriever by SyedShahmeerAli12 · Pull Request #3340 · deepset-ai/haystack-core-integrations

SyedShahmeerAli12 · 2026-05-21T14:45:41Z

Related Issues

partially addresses ArangoDB Integration #2896

Proposed Changes:

Adds a new arangodb-haystack integration with:

ArangoDocumentStore full Haystack DocumentStore backed by ArangoDB, supporting write_documents, filter_documents, delete_documents, count_documents, and vector similarity retrieval via AQL COSINE_SIMILARITY (requires ArangoDB 3.12+)
ArangoEmbeddingRetriever @component wrapper that calls _embedding_retrieval on the document store
filters.py translates Haystack metadata filter dicts to AQL FILTER expressions
Auth via python-arango client; password stored as a Haystack Secret
Serialization/deserialization with to_dict / from_dict

How did you test it?

26 unit tests (fully mocked) covering init, serialization round-trip, write/filter/delete/count, duplicate policies, and embedding retrieval
1 integration test verified against a live ArangoDB instance running locally via Docker (docker run -p 8529:8529 -e ARANGO_ROOT_PASSWORD=testpw arangodb:latest): writes 2 documents with embeddings, retrieves by cosine similarity, deletes one, asserts count

Notes for the reviewer

AQL vector search uses COSINE_SIMILARITY(doc.embedding, @query_vec) requires ArangoDB 3.12+; documented in the docstring
python-arango>=8.0.0 added as the only new dependency
Integration test is gated on ARANGO_HOST and ARANGO_PASSWORD env vars

Checklist

I have read the contributors guidelines and the code of conduct
I have updated the related issue with new insights and changes
I added unit tests and updated the docstrings
I've used one of the conventional commit types for my PR title: feat:

socket-security · 2026-05-21T14:47:03Z

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	python-arango@8.3.3

View full report

SyedShahmeerAli12 · 2026-05-21T14:48:58Z

@bogdankostic All 26 unit tests pass and the integration test was verified locally against ArangoDB via Docker.

bogdankostic

Thank you for the PR @SyedShahmeerAli12!

I left a couple of inline comments. Apart from those, this integration could be further improved by adding async counterpart to write_documents etc.m python-arango ships an async client. Moreover, we could add an AranfoBM25Retriever to support text search.

bogdankostic · 2026-05-26T12:25:19Z

+        *,
+        host: str = "http://localhost:8529",
+        database: str = "haystack",
+        username: str = "root",


Let's make username also a Secret that can be set via an environment variable.

bogdankostic · 2026-05-26T12:32:52Z

+                    written += 1
+                    continue
+            else:
+                col.insert(arango_doc)


Could we make use here of col.import_bulk or col.insert_many which should be optimized for writing many document at once?

bogdankostic · 2026-05-26T12:34:36Z

+        col = cast(StandardCollection, self._col)
+        for doc_id in document_ids:
+            if col.has(doc_id):
+                col.delete(doc_id)


Can we make use here of col.delete_many?

bogdankostic · 2026-05-26T12:45:20Z

+        FOR doc IN {self.collection_name}
+            FILTER doc.embedding != null
+            {filter_clause}
+            LET score = COSINE_SIMILARITY(doc.embedding, @query_vec)


Should we use the approximate functions mentioned here? They are probably optimized for vector sector. Also, it would be nice to support dot product distance as well.

bogdankostic · 2026-05-26T12:53:59Z

+    reason="Export ARANGO_HOST and ARANGO_PASSWORD to run integration tests.",
+)
+@pytest.mark.integration
+class TestArangoDocumentStoreIntegration:


Let's use our DocumentStoreTestMixins for the integration tests

bogdankostic · 2026-05-26T12:54:49Z

+readme = "README.md"
+requires-python = ">=3.10"
+license = "Apache-2.0"
+keywords = []


Let's add some relevant keywords.

bogdankostic · 2026-05-26T12:55:14Z

+all = 'pytest {args:tests}'
+unit-cov-retry = 'pytest --cov=haystack_integrations --reruns 3 --reruns-delay 30 -x -m "not integration" {args:tests}'
+integration-cov-append-retry = 'pytest --cov=haystack_integrations --cov-append --reruns 3 --reruns-delay 30 -x -m "integration" {args:tests}'
+types = "mypy -p haystack_integrations.document_stores.arangodb {args}"


Let's add the retriever here.

bogdankostic

Thank you for addressing the review comments @SyedShahmeerAli12! I just added two more minor comments.

Apart from that, the integration tests are currently skipped in the CI. It would be nice to have a docker compose file and run it in the CI, similar to how we do in the Opensearch integration.

bogdankostic · 2026-06-01T14:00:44Z

+class TestArangoDocumentStoreIntegration(
+    CountDocumentsTest,
+    DeleteDocumentsTest,
+    FilterDocumentsTest,
+    WriteDocumentsTest,


You can use DocumentStoreBaseTests here which includes all of these mixins.

SyedShahmeerAli12 · 2026-06-01T15:09:13Z

Switched to APPROX_NEAR_COSINE, APPROX_NEAR_INNER_PRODUCT, APPROX_NEAR_L2 as suggested - Replaced individual mixins with DocumentStoreBaseTests

CI / docker:

Added docker-compose.yml with ArangoDB 3.12
Updated workflow to Linux-only, starts container before tests, passes ARANGO_HOST and
ARANGO_PASSWORD

bogdankostic

Thanks @SyedShahmeerAli12, this PR is almost good to go, I just have some last minor cosmetic comments to further improve this PR.

SyedShahmeerAli12 · 2026-06-03T13:43:29Z

Addressed all comments updated the retriever docstring to reflect all three similarity functions, removed conftest.py since the fixture is now in the test class, added docker compose instructions to the README, and cleaned up the unreachable op_map block in filters.py.

feat(arangodb): add ArangoDocumentStore and ArangoEmbeddingRetriever

6c93c2f

SyedShahmeerAli12 requested a review from a team as a code owner May 21, 2026 14:45

SyedShahmeerAli12 requested review from bogdankostic and removed request for a team May 21, 2026 14:45

github-actions Bot added topic:CI type:documentation Improvements or additions to documentation labels May 21, 2026

Merge branch 'main' into feat/arangodb-document-store

9f0359b

bogdankostic added the integration:arangodb label May 26, 2026

bogdankostic requested changes May 26, 2026

View reviewed changes

SyedShahmeerAli12 added 4 commits May 28, 2026 16:02

address review comments

56d3dac

fix lint

1b6b697

add py.typed marker for retrievers package

070f508

fix mypy errors

2624f1e

SyedShahmeerAli12 requested a review from bogdankostic May 28, 2026 11:17

Merge branch 'main' into feat/arangodb-document-store

73ab6b4

bogdankostic requested changes Jun 1, 2026

View reviewed changes

fix

334e6fd

bogdankostic requested changes Jun 2, 2026

View reviewed changes

fix

5d80d57

SyedShahmeerAli12 requested a review from bogdankostic June 3, 2026 11:03

Merge branch 'main' into feat/arangodb-document-store

34323f5

Conversation

SyedShahmeerAli12 commented May 21, 2026 • edited by bogdankostic Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related Issues

Proposed Changes:

How did you test it?

Notes for the reviewer

Checklist

Uh oh!

socket-security Bot commented May 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SyedShahmeerAli12 commented May 21, 2026

Uh oh!

bogdankostic left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bogdankostic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SyedShahmeerAli12 commented Jun 1, 2026

Uh oh!

bogdankostic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SyedShahmeerAli12 commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

SyedShahmeerAli12 commented May 21, 2026 •

edited by bogdankostic

Loading

socket-security Bot commented May 21, 2026 •

edited

Loading

SyedShahmeerAli12 commented Jun 3, 2026 •

edited

Loading