fix: detect numeric type changes (int/float) when ignore_order=True#580
Open
frankgoldfish wants to merge 1 commit intoqlustered:masterfrom
Open
fix: detect numeric type changes (int/float) when ignore_order=True#580frankgoldfish wants to merge 1 commit intoqlustered:masterfrom
frankgoldfish wants to merge 1 commit intoqlustered:masterfrom
Conversation
Fixes qlustered#485 When ignore_order=True, DeepDiff([{'a': 1}], [{'a': 1.0}]) incorrectly returned {} instead of reporting a type_change for the int→float difference. Root cause ---------- Python's numeric equality semantics (hash(1) == hash(1.0) and 1 == 1.0) caused int 1 and float 1.0 to map to the same slot in the shared DeepHash cache dictionary. As a result, {'a': 1} and {'a': 1.0} received the same deephash, landed in the hash *intersection*, and were silently treated as identical items — so the type difference was never reported. Fix --- After the existing hash-intersection logic in _diff_iterable_with_deephash, add a post-pass that re-examines every pair of items whose hashes are equal (i.e. in the intersection) when ignore_numeric_type_changes is False. A lightweight type-strict equality helper _items_are_type_equal() checks whether the pair is truly identical (same type tree and values). If not, a full _diff() is run on the pair, which surfaces any nested type_changes. This approach: - Touches only the ignore_order code path (no change to deephash.py) - Adds zero overhead when all intersection items are genuinely equal - Fully respects ignore_numeric_type_changes=True (no post-pass in that case) - Passes all pre-existing tests (23 failures are pre-existing, not introduced) Example ------- Before fix: DeepDiff([{'a': 1}], [{'a': 1.0}], ignore_order=True) # {} ← wrong After fix: DeepDiff([{'a': 1}], [{'a': 1.0}], ignore_order=True) # {'type_changes': {"root[0]['a']": {old_type: int, new_type: float, ...}}}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #485
Root Cause
Python's numeric equality semantics (
hash(1) == hash(1.0)and1 == 1.0) causedint 1andfloat 1.0to map to the same slot in the shared DeepHash cache dictionary. Consequently,{'a': 1}and{'a': 1.0}received identical deephashes and landed in the hash intersection — items DeepDiff considers equal — so the type difference was never reported.Fix
Added a post-pass in
_diff_iterable_with_deephashthat re-examines every intersection pair whenignore_numeric_type_changes=False. A new helper_items_are_type_equal()performs a type-strict equality check (same type tree AND same values). If the pair is not truly type-equal, a full_diff()is run on it, surfacing any nestedtype_changes.Key properties of the fix:
ignore_ordercode path — zero change todeephash.pyignore_numeric_type_changes=True(post-pass is skipped)Tests
Added
TestIgnoreOrderNumericTypeChangeintests/test_ignore_order.pywith 6 targeted cases covering: the core regression, theignore_numeric_type_changes=Truesuppression, value-only changes, reorder no-op, mixed lists, and theignore_order=Falsepath.