[FLINK-40186][python] Introduce the PyFlink DataFrame API#28797
[FLINK-40186][python] Introduce the PyFlink DataFrame API#28797auroflow wants to merge 4 commits into
Conversation
Generated-by: OpenAI Codex (GPT-5.6 Sol)
Generated-by: OpenAI Codex (GPT-5.6 Sol)
Generated-by: OpenAI Codex (GPT-5.6 Sol)
Generated-by: OpenAI Codex (GPT-5.6 Sol)
dianfu
changed the title
1b7cd38 to
9684ffc
Compare
|
|
||
| self.assertEqual(result.collect(), [Row(1, "Alice", 31)]) | ||
|
|
||
| def test_from_dict_respects_schema_order_and_subset(self): |
There was a problem hiding this comment.
It has introduced about 30 it cases. Please check if we could merge them. Besides, it's also worth to check if it's possible to convert part of them into unit tests.
| ) | ||
| field_rows.append(tuple(field_values[name] for name in columns)) | ||
| rows = field_rows | ||
| elif isinstance(first_record, Sequence): |
There was a problem hiding this comment.
The isinstance(record, Sequence) check also accepts str, bytes, and similar scalar values. For example: from_records(["ab", "cd"], schema=["x", "y"]) produces rows ("a", "b") and ("c", "d") which is not expected.
|
|
||
| .. versionadded:: 2.4.0 | ||
| """ | ||
| if not isinstance(data, Mapping): |
There was a problem hiding this comment.
The column validation also accepts strings because they implement Sequence:
from_dict({"name": "Bob"}) would create three rows containing "B", "o", and "b".
|
|
||
|
|
||
| @PublicEvolving() | ||
| def set_table_environment(t_env: Optional["StreamTableEnvironment"]) -> None: |
There was a problem hiding this comment.
validate the type of the given t_env
| for name in columns: | ||
| if name not in field_values: | ||
| raise ValueError( | ||
| "schema field %r is not present in records" % name |
There was a problem hiding this comment.
Since validation is performed per record, the message should identify the failing index consistently.
3 participants
What is the purpose of the change
This pull request introduces the foundation for a Pythonic DataFrame API in PyFlink.
Brief change log
pyflink.dataframepackage and its public evolving interfaces.Verifying this change
This change added tests and can be verified as follows:
Does this pull request potentially affect one of the following parts:
@Public(Evolving): yesDocumentation
Was generative AI tooling used to co-author this PR?
Generated-by: OpenAI Codex (GPT-5.6 Sol)