[python] Fix incomplete Python dev requirements

[aisk]

Closes #8082

Purpose

Unify the dependencies for CI and local development. See the original issue for more info.

Tests

[JingsongLi]

Please rebase master.

[aisk]

Please rebase master.

Done.

[JingsongLi]

This looks like it will break the existing Python 3.6/3.7 jobs.

paimon-python/dev/requirements.txt now pins pyroaring==1.0.0 for python_version < "3.8", but the workflow still runs Python 3.6.15 and installs paimon-python/dev/requirements-dev.txt. I checked pip resolution for binary wheels, and pyroaring==1.0.0 is not available for cp36/cp37; for Python 3.6 pip only sees the old 0.3.x wheels, and for Python 3.7 it only sees up to 0.4.x. So this requirement can fail dependency installation, or force a source build in an environment that previously used wheels.

Could we either keep a Python 3.6/3.7-compatible dependency path with a fallback for the BitMap64 usage, or explicitly raise the supported Python/CI minimum to 3.8+ in the same PR?

[aisk]

Hi, thank you for your review.

I checked that although PyRoarning don't have a wheel package for Python3.6, but it's actually build-able from source and can be used under Python3.6, since it don't have a python-requires filed on it's setup.py. The CI have build it successfully:

Building wheels for collected packages: pyroaring
  Building wheel for pyroaring (setup.py): started
  Building wheel for pyroaring (setup.py): finished with status 'done'
  Created wheel for pyroaring: filename=pyroaring-1.0.0-cp36-cp36m-linux_x86_64.whl size=386841 sha256=29c17f1e9f3224de50f20299149e1eec2b2c51ef20d80a9d92c514b3a8da1283
  Stored in directory: /tmp/pip-ephem-wheel-cache-zowmzlj2/wheels/a9/8e/dd/ced219123113d222e5dbe62e7a7ac640fb1e88ce2ce63af9c5
Successfully built pyroaring

I think it's ok to have it on Python3.6. But it's not officially support, and will introduce some installation issues for users like if they don't have GCC installed. So I think it's better to drop support for Python3.6 and 3.7, because they have sunset for a long time.

What do you think?