Skip to content

[python] Support pypaimon-rust 0.4 #8856

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
shyjsarah wants to merge 2 commits into
base: master
Choose a base branch
from
+7 −6
Open

[python] Support pypaimon-rust 0.4 #8856

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8ce988f
[python] Support pypaimon-rust 0.4
Jul 27, 2026
a15c0c4
[python] Align DataFusion ABI for pypaimon-rust 0.4
Jul 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions .github/workflows/paimon-python-checks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ env:
JDK_VERSION: 8
MAVEN_OPTS: -Dmaven.wagon.httpconnectionManager.ttlSeconds=30 -Dmaven.wagon.http.retryHandler.requestSentEnabled=true
LUMINA_DATA_VERSION: 0.1.0
PYPAIMON_RUST_REV: 7c568274a4c5df6bd00e8d60edea21395412202b
PYPAIMON_RUST_REV: 6b6782923a59eeba766cbd45dde6989705b282bc


concurrency:
Expand Down Expand Up @@ -131,11 +131,12 @@ jobs:
else
python -m pip install --upgrade pip
pip install torch --index-url https://download.pytorch.org/whl/cpu
python -m pip install pyroaring readerwriterlock==1.0.9 fsspec==2024.3.1 cachetools==5.3.3 ossfs==2023.12.0 ray==2.54.0 fastavro==1.11.1 'isal>=1.8,<2' pyarrow==16.0.0 zstandard==0.24.0 polars==1.32.0 duckdb==1.3.2 numpy==1.24.3 pandas==2.0.3 pylance==0.39.0 cramjam flake8==4.0.1 pytest~=7.0 py4j==0.10.9.9 requests parameterized==0.9.0 'daft>=0.7.6' 'datafusion>=52' datasketches
python -m pip install pyroaring readerwriterlock==1.0.9 fsspec==2024.3.1 cachetools==5.3.3 ossfs==2023.12.0 ray==2.54.0 fastavro==1.11.1 'isal>=1.8,<2' pyarrow==16.0.0 zstandard==0.24.0 polars==1.32.0 duckdb==1.3.2 numpy==1.24.3 pandas==2.0.3 pylance==0.39.0 cramjam flake8==4.0.1 pytest~=7.0 py4j==0.10.9.9 requests parameterized==0.9.0 'daft>=0.7.6' 'datafusion>=54,<55' datasketches
if [[ "${{ matrix.python-version }}" == "3.11" ]]; then
# Exercise the split-planning API in one lane until the compatible 0.3 wheel is published.
# Exercise the 0.4 API in one lane until its wheel is published.
python -m pip install "git+https://github.com/apache/paimon-rust.git@${PYPAIMON_RUST_REV}#subdirectory=bindings/python"
python -c "from pypaimon_rust.datafusion import PaimonCatalog; assert hasattr(PaimonCatalog, 'get_table')"
python -m pip install "./paimon-python[sql]"
python -c "import tempfile; from datafusion import SessionContext; from pypaimon_rust.datafusion import PaimonCatalog; warehouse = tempfile.TemporaryDirectory(); ctx = SessionContext(); ctx.register_catalog_provider('paimon', PaimonCatalog({'warehouse': warehouse.name}))"
else
python -m pip install pypaimon-rust==0.2.0
fi
Expand Down
4 changes: 2 additions & 2 deletions paimon-python/setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,8 +187,8 @@ def read_requirements():
'paimon-ftindex==0.1.0; python_version>="3.8"',
],
'sql': [
'pypaimon-rust>=0.3.0,<0.4.0; python_version>="3.10"',
'datafusion>=52; python_version>="3.10"',
'pypaimon-rust>=0.3.0; python_version>="3.10"',
'datafusion>=54,<55; python_version>="3.10"',
],
'hdfs': [
'hdfs-native>=0.13,<1; python_version >= "3.10" and platform_system != "Windows"',
Expand Down
Loading