Skip to content
Open

8.7.0 #579

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 8.6.1
current_version = 8.7.0
commit = True
tag = True
tag_name = {new_version}
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ A clear and concise description of what you expected to happen.
**OS, DeepDiff version and Python version (please complete the following information):**
- OS: [e.g. Ubuntu]
- Version [e.g. 20LTS]
- Python Version [e.g. 3.9.12]
- Python Version [e.g. 3.10.12]
- DeepDiff Version [e.g. 5.8.0]

**Additional context**
Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ jobs:
build:
runs-on: ubuntu-latest
env:
DEFAULT_PYTHON: '3.12'
DEFAULT_PYTHON: '3.14'
strategy:
matrix:
python-version: ['3.9','3.10','3.11','3.12','3.13']
python-version: ['3.10','3.11','3.12','3.13','3.14']
architecture: ['x64']

steps:
Expand All @@ -29,18 +29,18 @@ jobs:
run: pip install nox==2025.5.1

- name: Lint with flake8
if: ${{ matrix.python-version == '3.12' }}
if: ${{ matrix.python-version == '3.14' }}
run: |
nox -s flake8 -- deepdiff --count --select=E9,F63,F7,F82 --show-source --statistics
nox -s flake8 -- deepdiff --count --exit-zero --max-complexity=26 --max-line-length=250 --statistics

- name: Test with pytest (no coverage)
if: ${{ matrix.python-version != '3.12' }}
if: ${{ matrix.python-version != '3.14' }}
run: |
nox -s pytest-${{ matrix.python-version }} -- --benchmark-disable tests/

- name: Test with pytest (+ coverage)
if: ${{ matrix.python-version == '3.12' }}
if: ${{ matrix.python-version == '3.14' }}
run: |
nox -s pytest-${{ matrix.python-version }} -- \
--benchmark-disable \
Expand All @@ -49,7 +49,7 @@ jobs:
tests/ --runslow

- name: Upload coverage
if: ${{ matrix.python-version == '3.12' }}
if: ${{ matrix.python-version == '3.14' }}
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# DeepDiff Change log

- v8-7-0
- Dropping support for Python 3.9
- Support for python 3.14
- `to_dict()` and `to_json()` now accept a `verbose_level` parameter and always return a usable text-view dict. When the original view is `'tree'`, they default to `verbose_level=2` for full detail. The old `view_override` parameter is removed.

- v8-6-1
- Patched security vulnerability in the Delta class which was vulnerable to class pollution via its constructor, and when combined with a gadget available in DeltaDiff itself, it could lead to Denial of Service and Remote Code Execution (via insecure Pickle deserialization).

Expand Down
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,6 @@ authors:
given-names: "Sep"
orcid: "https://orcid.org/0009-0009-5828-4345"
title: "DeepDiff"
version: 8.6.1
version: 8.7.0
date-released: 2024
url: "https://github.com/seperman/deepdiff"
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ nox

## Development Notes

- **Python Support**: 3.9+ and PyPy3
- **Python Support**: 3.10+ and PyPy3
- **Main Branch**: `master` (PRs typically go to `dev` branch)
- **Build System**: Modern `pyproject.toml` with `flit_core`
- **Dependencies**: Core dependency is `orderly-set>=5.4.1,<6`
Expand Down
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# DeepDiff v 8.6.1
# DeepDiff v 8.7.0

![Downloads](https://img.shields.io/pypi/dm/deepdiff.svg?style=flat)
![Python Versions](https://img.shields.io/pypi/pyversions/deepdiff.svg?style=flat)
Expand All @@ -15,14 +15,19 @@
- [Extract](https://zepworks.com/deepdiff/current/extract.html): Extract an item from a nested Python object using its path.
- [commandline](https://zepworks.com/deepdiff/current/commandline.html): Use DeepDiff from commandline.

Tested on Python 3.9+ and PyPy3.
Tested on Python 3.10+ and PyPy3.

- **[Documentation](https://zepworks.com/deepdiff/8.6.1/)**
- **[Documentation](https://zepworks.com/deepdiff/8.7.0/)**

## What is new?

Please check the [ChangeLog](CHANGELOG.md) file for the detailed information.

DeepDiff 8-7-0
- Dropping support for Python 3.9
- Support for python 3.14
- `to_dict()` and `to_json()` now accept a `verbose_level` parameter and always return a usable text-view dict. When the original view is `'tree'`, they default to `verbose_level=2` for full detail. The old `view_override` parameter is removed.

DeepDiff 8-6-1
- Patched security vulnerability in the Delta class which was vulnerable to class pollution via its constructor, and when combined with a gadget available in DeltaDiff itself, it could lead to Denial of Service and Remote Code Execution (via insecure Pickle deserialization).

Expand Down
2 changes: 1 addition & 1 deletion deepdiff/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""This module offers the DeepDiff, DeepSearch, grep, Delta and DeepHash classes."""
# flake8: noqa
__version__ = '8.6.1'
__version__ = '8.7.0'
import logging

if __name__ == '__main__':
Expand Down
4 changes: 1 addition & 3 deletions deepdiff/deephash.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import logging
import datetime
import uuid
from typing import Union, Optional, Any, List, TYPE_CHECKING, Dict, Tuple, Set, Callable, Iterator, Generator, TypeVar, Protocol
from typing import Union, Optional, Any, List, TYPE_CHECKING, Dict, Tuple, Set, Callable, Generator
from collections.abc import Iterable, MutableMapping
from collections import defaultdict
from hashlib import sha1, sha256
Expand All @@ -20,8 +20,6 @@

if TYPE_CHECKING:
from pytz.tzinfo import BaseTzInfo
import pandas as pd
import polars as pl
import numpy as np

# Type aliases for better readability
Expand Down
5 changes: 3 additions & 2 deletions deepdiff/diff.py
Original file line number Diff line number Diff line change
Expand Up @@ -1810,7 +1810,7 @@ def _diff(self, level, parents_ids=frozenset(), _original_type=None, local_tree=
else:
self._diff_obj(level, parents_ids)

def _get_view_results(self, view):
def _get_view_results(self, view, verbose_level=None):
"""
Get the results based on the view
"""
Expand All @@ -1820,7 +1820,8 @@ def _get_view_results(self, view):
if view == TREE_VIEW:
pass
elif view == TEXT_VIEW:
result = TextResult(tree_results=self.tree, verbose_level=self.verbose_level)
effective_verbose_level = verbose_level if verbose_level is not None else self.verbose_level
result = TextResult(tree_results=self.tree, verbose_level=effective_verbose_level)
result.remove_empty_keys()
elif view == DELTA_VIEW:
result = self._to_delta_dict(report_repetition_required=False)
Expand Down
2 changes: 1 addition & 1 deletion deepdiff/distance.py
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ def numpy_apply_log_keep_sign(array, offset=MATH_LOG_OFFSET):
return signed_log_values


def logarithmic_similarity(a: NumberType, b: NumberType, threshold: float=0.1) -> float:
def logarithmic_similarity(a: NumberType, b: NumberType, threshold: float=0.1) -> bool:
"""
A threshold of 0.1 translates to about 10.5% difference.
A threshold of 0.5 translates to about 65% difference.
Expand Down
71 changes: 57 additions & 14 deletions deepdiff/serialization.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
strings,
get_type,
TEXT_VIEW,
TREE_VIEW,
np_float32,
np_float64,
np_int32,
Expand Down Expand Up @@ -171,7 +172,7 @@ def from_json_pickle(cls, value):
except ImportError: # pragma: no cover. Json pickle is getting deprecated.
logger.error('jsonpickle library needs to be installed in order to run from_json_pickle') # pragma: no cover. Json pickle is getting deprecated.

def to_json(self, default_mapping: Optional[dict]=None, force_use_builtin_json=False, **kwargs):
def to_json(self, default_mapping: Optional[dict]=None, force_use_builtin_json=False, verbose_level: Optional[int]=None, **kwargs):
"""
Dump json of the text view.
**Parameters**
Expand All @@ -186,6 +187,8 @@ def to_json(self, default_mapping: Optional[dict]=None, force_use_builtin_json=F
When True, we use Python's builtin Json library for serialization,
even if Orjson is installed.

verbose_level: int, default=None
Override the verbose_level for the serialized output. See to_dict() for details.

kwargs: Any other kwargs you pass will be passed on to Python's json.dumps()

Expand All @@ -208,27 +211,35 @@ def to_json(self, default_mapping: Optional[dict]=None, force_use_builtin_json=F
>>> ddiff.to_json(default_mapping=default_mapping)
'{"type_changes": {"root": {"old_type": "A", "new_type": "B", "old_value": "obj A", "new_value": "obj B"}}}'
"""
dic = self.to_dict(view_override=TEXT_VIEW)
dic = self.to_dict(verbose_level=verbose_level)
return json_dumps(
dic,
default_mapping=default_mapping,
force_use_builtin_json=force_use_builtin_json,
**kwargs,
)

def to_dict(self, view_override: Optional[str]=None) -> dict:
def to_dict(self, verbose_level: Optional[int]=None) -> dict:
"""
convert the result to a python dictionary. You can override the view type by passing view_override.
Convert the result to a python dictionary.

**Parameters**

view_override: view type, default=None,
override the view that was used to generate the diff when converting to the dictionary.
The options are the text or tree.
verbose_level: int, default=None
Override the verbose_level for the serialized output.
When None, the behavior depends on the original view:
- If the original view is 'text', the verbose_level from DeepDiff initialization is used.
- If the original view is 'tree', verbose_level=2 is used to provide the most detailed output.
Valid values are 0, 1, or 2.
"""

view = view_override if view_override else self.view # type: ignore
return dict(self._get_view_results(view)) # type: ignore
if verbose_level is not None and verbose_level not in {0, 1, 2}:
raise ValueError('verbose_level should be 0, 1, or 2.')
if verbose_level is None:
if self.view == TREE_VIEW: # type: ignore
verbose_level = 2
else:
verbose_level = self.verbose_level # type: ignore
return dict(self._get_view_results(TEXT_VIEW, verbose_level=verbose_level)) # type: ignore

def _to_delta_dict(
self,
Expand Down Expand Up @@ -736,6 +747,28 @@ def json_dumps(
...


_INT64_MAX = 9223372036854775807
_INT64_MIN = -9223372036854775808


def _convert_oversized_ints(obj):
"""Recursively convert integers exceeding 64-bit range to strings.
orjson cannot serialize integers outside the signed 64-bit range."""
if isinstance(obj, bool):
return obj
if isinstance(obj, int) and (obj > _INT64_MAX or obj < _INT64_MIN):
return str(obj)
if isinstance(obj, dict):
return {k: _convert_oversized_ints(v) for k, v in obj.items()}
if isinstance(obj, (list, tuple)):
converted = [_convert_oversized_ints(v) for v in obj]
if hasattr(obj, '_fields'):
# NamedTuple: reconstruct using keyword arguments
return type(obj)(**dict(zip(obj._fields, converted)))
return type(obj)(converted)
return obj


def json_dumps(
item: Any,
default_mapping:Optional[dict]=None,
Expand Down Expand Up @@ -763,10 +796,20 @@ def json_dumps(
"orjson does not accept the sort_keys parameter. "
"If you need to pass sort_keys, set force_use_builtin_json=True "
"to use Python's built-in json library instead of orjson.")
result = orjson.dumps(
item,
default=json_convertor_default(default_mapping=default_mapping),
**kwargs)
try:
result = orjson.dumps(
item,
default=json_convertor_default(default_mapping=default_mapping),
**kwargs)
except TypeError as e:
if 'Integer exceeds 64-bit range' in str(e):
item = _convert_oversized_ints(item)
result = orjson.dumps(
item,
default=json_convertor_default(default_mapping=default_mapping),
**kwargs)
else:
raise
if return_bytes:
return result
return result.decode(encoding='utf-8')
Expand Down
5 changes: 5 additions & 0 deletions docs/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ Changelog

DeepDiff Changelog

- v8-7-0
- Dropping support for Python 3.9
- Support for python 3.14
- ``to_dict()`` and ``to_json()`` now accept a ``verbose_level`` parameter and always return a usable text-view dict. When the original view is ``'tree'``, they default to ``verbose_level=2`` for full detail. The old ``view_override`` parameter is removed.

- v8-6-1
- Patched security vulnerability in the Delta class which was vulnerable to class pollution via its constructor, and when combined with a gadget available in DeltaDiff itself, it could lead to Denial of Service and Remote Code Execution (via insecure Pickle deserialization).

Expand Down
4 changes: 2 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,9 @@
# built documents.
#
# The short X.Y version.
version = '8.6.1'
version = '8.7.0'
# The full version, including alpha/beta/rc tags.
release = '8.6.1'
release = '8.7.0'

load_dotenv(override=True)
DOC_VERSION = os.environ.get('DOC_VERSION', version)
Expand Down
10 changes: 9 additions & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
contain the root `toctree` directive.


DeepDiff 8.6.1 documentation!
DeepDiff 8.7.0 documentation!
=============================

*******
Expand All @@ -31,6 +31,14 @@ The DeepDiff library includes the following modules:
What Is New
***********

DeepDiff 8-7-0
--------------

- Dropping support for Python 3.9
- Support for python 3.14
- ``to_dict()`` and ``to_json()`` now accept a ``verbose_level`` parameter and always return a usable text-view dict. When the original view is ``'tree'``, they default to ``verbose_level=2`` for full detail. The old ``view_override`` parameter is removed.


DeepDiff 8-6-1
--------------

Expand Down
26 changes: 24 additions & 2 deletions docs/serialization.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,18 @@ To Dict
-------

In order to convert the DeepDiff object into a normal Python dictionary, use the to_dict() method.
The result is always a text-view dictionary regardless of the original view used to create the DeepDiff object.

**Parameters**

verbose_level: int, default=None
Override the verbose_level for the serialized output.
When None, the behavior depends on the original view:

- If the original view is 'text', the verbose_level from DeepDiff initialization is used.
- If the original view is 'tree', verbose_level=2 is used to provide the most detailed output.

Valid values are 0, 1, or 2.

Example:
>>> t1 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 2, 3]}}
Expand All @@ -20,15 +32,22 @@ Example:
{'type_changes': {"root[4]['b']": {'old_type': <class 'list'>, 'new_type': <class 'str'>, 'old_value': [1, 2, 3], 'new_value': 'world\n\n\nEnd'}}}


Note that you can override the :ref:`view_label` that was originally used to generate the diff here.
When the original view is 'tree', to_dict() defaults to verbose_level=2 for the most detailed output:

Example:
>>> t1 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 2, 3]}}
>>> t2 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": "world\n\n\nEnd"}}
>>> ddiff = DeepDiff(t1, t2, view='tree')
>>> ddiff.to_dict(view_override='text')
>>> ddiff.to_dict()
{'type_changes': {"root[4]['b']": {'old_type': <class 'list'>, 'new_type': <class 'str'>, 'old_value': [1, 2, 3], 'new_value': 'world\n\n\nEnd'}}}

You can also override the verbose_level:

Example:
>>> ddiff = DeepDiff(t1, t2, view='tree')
>>> ddiff.to_dict(verbose_level=0)
{'type_changes': {"root[4]['b']": {'old_type': <class 'list'>, 'new_type': <class 'str'>}}}

.. _to_json_label:

To Json
Expand All @@ -46,6 +65,9 @@ by default DeepDiff converts certain data types. For example Decimals into float
If you have a certain object type that the json serializer can not serialize it, please pass the appropriate type
conversion through this dictionary.

verbose_level: int, default=None
Override the verbose_level for the serialized output. Same behavior as to_dict().

kwargs: Any other kwargs you pass will be passed on to Python's json.dumps()


Expand Down
Loading
Loading