Skip to content

synthesizer.to_pandas() returns KeyError: 'evolutions' #2413

@GeisaFaustino

Description

@GeisaFaustino

Describe the bug
It is not possible to convert the synthetically generated goldens into a DataFrame as illustrated in the documentation.
synthesizer.to_pandas() returns KeyError: 'evolutions'.

To Reproduce

from deepeval.synthesizer import Synthesizer

print(f"We are using the model: {aoai_model.get_model_name()}")
synthesizer = Synthesizer(model=aoai_model)

conversational_goldens = synthesizer.generate_conversational_goldens_from_contexts(
    # Provide a list of context for synthetic data generation
    contexts=[
        ["The Earth revolves around the Sun.", "Planets are celestial bodies."],
        ["Water freezes at 0 degrees Celsius.", "The chemical formula for water is H2O."],
    ]
)

print(f"Generated {len(conversational_goldens)} conversational goldens.")

df = synthesizer.to_pandas()
df.head()

Output:

We are using the model: gpt-4o (Azure)

Generated 4 conversational goldens.

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[5], [line 1](vscode-notebook-cell:?execution_count=5&line=1)
----> [1](vscode-notebook-cell:?execution_count=5&line=1) df = synthesizer.to_pandas()
      2 df.head()

File /opt/anaconda3/envs/deep-eval-378/lib/python3.11/site-packages/deepeval/synthesizer/synthesizer.py:1438, in Synthesizer.to_pandas(self)
   1435 df = pd.DataFrame(data)
   1437 # Optional: Fill NaN evolutions for better clarity
-> [1438](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/deep-eval-378/lib/python3.11/site-packages/deepeval/synthesizer/synthesizer.py:1438) df["evolutions"] = df["evolutions"].apply(
   1439     lambda x: x if x is not None else "None"
   1440 )
   1442 return df

File /opt/anaconda3/envs/deep-eval-378/lib/python3.11/site-packages/pandas/core/frame.py:4113, in DataFrame.__getitem__(self, key)
   4111 if self.columns.nlevels > 1:
   4112     return self._getitem_multilevel(key)
-> [4113](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/deep-eval-378/lib/python3.11/site-packages/pandas/core/frame.py:4113) indexer = self.columns.get_loc(key)
   4114 if is_integer(indexer):
   4115     indexer = [indexer]

File /opt/anaconda3/envs/deep-eval-378/lib/python3.11/site-packages/pandas/core/indexes/range.py:417, in RangeIndex.get_loc(self, key)
    415         raise KeyError(key) from err
    416 if isinstance(key, Hashable):
--> [417](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/deep-eval-378/lib/python3.11/site-packages/pandas/core/indexes/range.py:417)     raise KeyError(key)
    418 self._check_indexing_error(key)
    419 raise KeyError(key)

KeyError: 'evolutions'

Expected behavior
A dataframe as illustrated here.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions