Skip to content

fix: use flatten=False in Document.__eq__ to avoid metadata key collisions - #12172

Closed
rekha0suthar wants to merge 1 commit into
deepset-ai:mainfrom
rekha0suthar:fix/document-eq-flatten
Closed

fix: use flatten=False in Document.__eq__ to avoid metadata key collisions#12172
rekha0suthar wants to merge 1 commit into
deepset-ai:mainfrom
rekha0suthar:fix/document-eq-flatten

Conversation

@rekha0suthar

Copy link
Copy Markdown

Summary

Document.__eq__ called to_dict() with the default flatten=True, which merges the document's meta dictionary into the top-level representation. When a document's metadata contains a key that shares a name with a real Document field (such as "id" or "score"), the meta value silently overwrites the field value before the comparison is made. This causes two Documents with different metadata to compare as equal.

Example that was broken before this fix:

doc1 = Document(content="text", meta={"id": "meta-id-1"})
doc2 = Document(content="text", meta={"id": "meta-id-2"})
assert doc1 != doc2  # was incorrectly passing as equal

Fix

Changed line 97 of haystack/dataclasses/document.py from:

return self.to_dict() == other.to_dict()

to:

return self.to_dict(flatten=False) == other.to_dict(flatten=False)

flatten=False keeps the meta dictionary nested under its own key, so collisions between meta keys and top-level field names cannot affect the comparison result.

Tests

Added test_equality_with_colliding_meta_keys to test/dataclasses/test_document.py to prevent regressions on this exact case.

Fixes #11969

…sions

Document.__eq__ called to_dict() with the default flatten=True, which
merges metadata into the top-level dictionary. When a Document's metadata
contains a key that collides with a real Document field (e.g. 'id' or
'score'), the meta value overwrites the field value before comparison,
causing two Documents with different metadata to compare as equal.

Switching to to_dict(flatten=False) compares the structured representation
directly, so meta is kept separate from top-level fields and collisions
can no longer produce false positives.

Added a regression test covering the case where two Documents share the
same content but differ only in a meta key named 'id'.

Fixes deepset-ai#11969
@rekha0suthar
rekha0suthar requested a review from a team as a code owner July 28, 2026 09:07
@rekha0suthar
rekha0suthar requested review from anakin87 and removed request for a team July 28, 2026 09:07
@vercel

vercel Bot commented Jul 28, 2026

Copy link
Copy Markdown

@rekha-suthar is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@github-actions

Copy link
Copy Markdown
Contributor

Hi @rekha0suthar, thanks for your interest in contributing to Haystack! 🙏

⚠️ Issue #11969 is already being addressed by open pull request(s) #11982. Before opening a PR for an issue, please check whether a PR is already linked to it, and consider contributing to the existing PR instead. We may close duplicate PRs to keep the review queue manageable.

This is an automated message to help us keep the review queue healthy.

@rekha0suthar

Copy link
Copy Markdown
Author

Closing as a duplicate of #11982 which is already addressing this issue. Thanks for the heads up!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: Document.__eq__ ignores colliding metadata keys during equality comparison

3 participants