Convert setup.py to pyproject.toml

[pjonsson]

The same files are included before/after
when doing "python3 -m build" on
my machine.

[pjonsson]

I can't see the log from continuous-integration/jenkins/pr-head, so not sure why that is red.

[soxofaan]

from the error logs:

+ setuptools==75.3.4
 + setuptools-lint==0.6.0
 + toml==0.10.2
 + typing-extensions==4.13.2
 + wrapt==1.13.3
+ '[' -f requirements.txt ']'
+ '[' -f setup.py ']'
+ '[' -f setup.cfg ']'
+ uv pip install '.[tests]'
Using Python 3.8.8 environment at: venv38
  × Failed to build `openeo @
  │ [file:///var/lib/jenkins/workspace/enEO_openeo-python-client_PR-908`](file:///var/lib/jenkins/workspace/enEO_openeo-python-client_PR-908%60)
  ├─▶ The build backend returned an error
  ╰─▶ Call to `setuptools.build_meta.build_wheel` failed (exit status: 1)

      [stdout]
      configuration error: `project.license` must be valid exactly by one
      definition (2 matches found):

          - keys:
              'file': {type: string}
            required: ['file']
          - keys:
              'text': {type: string}
            required: ['text']

      DESCRIPTION:
          `Project license <https://peps.python.org/pep-0621/#license>`_.

      GIVEN VALUE:
          "Apache-2.0"

as far as I understand the license field should not be a string but a mapping with field "text" containing that string.
This is probably because the build chain used here is stuck at outdated conventions

[soxofaan]

probably related to

• Drop support for Python 3.8 #717

[pjonsson]

@soxofaan ok, this revision of the PR works in a Python 3.8 container for me, and I've rebased to latest master.

[pjonsson]

Still red in the CI despite building locally, so I'm guessing the CI logs that I can't see are trying to run uv run pytest after installing the tests dependencies.

Accepted any version of sphinx-autodoc-typehints for Python versions that have reached end of life, and skipped openeo_pg_parser_networkx/openeo_processes_dask[implementations] on Python 3.8: https://github.com/Open-EO/openeo-python-client/compare/15d8ce848ba2ec05ead03de7db0a5398b6d0ce4f..c18db737723efc375efd4d75916f1ec87a470eff and after that I can run the tests in the Python 3.8 container locally as well (and they pass).

[pjonsson]

Apparently still red, but I can't guess what the CI does next, so I need to see some more logs.

[soxofaan]

yes I was afraid this would be bit of a rabbit whole and wrestling with CI contexts built on outdated assumptions

the failure on 8900d43 is an actual unit test failure:

tests.internal.test_warnings.test_user_deprecation_warning (from pytest)

assert 'This is a UserDeprecationWarning (stacklevel 1)' in 'Traceback (most recent call last):\n  File "myscript.py", line 1, in <module>\n    from openeo.internal.warnings impo... from openeo.rest.connection import Connection, connect, session\nModuleNotFoundError: No module named \'openeo.rest\''

Stacktrace

pytester = <Pytester PosixPath('/var/lib/jenkins/workspace/enEO_openeo-python-client_PR-908/pytest-tmp/pytest-of-jenkins/pytest-0/test_user_deprecation_warning0')>

    def test_user_deprecation_warning(pytester):
        pytester.makepyfile(myscript="""
            from openeo.internal.warnings import test_warnings
            test_warnings()
            test_warnings(2)
        """)
    
        result = pytester.runpython("myscript.py")
        stderr = "\n".join(result.errlines)
>       assert "This is a UserDeprecationWarning (stacklevel 1)" in stderr
E       assert 'This is a UserDeprecationWarning (stacklevel 1)' in 'Traceback (most recent call last):\n  File "myscript.py", line 1, in <module>\n    from openeo.internal.warnings impo... from openeo.rest.connection import Connection, connect, session\nModuleNotFoundError: No module named \'openeo.rest\''

/var/lib/jenkins/workspace/enEO_openeo-python-client_PR-908/tests/internal/test_warnings.py:17: AssertionError

[soxofaan]

On a more general note: is there any particular reason you work on this? Are the current setup.py based wheel releases at e.g. https://pypi.org/project/openeo/ not sufficient in some sense? Do you want to make custom builds or something like that?

But to be clear: I definitively also want to move to pyproject.toml, but was planning to do that after we can drop the python 3.8 support (because that would eliminate some annoying intricacies in our internal CI setup)

[pjonsson]

result = pytester.runpython("myscript.py")
stderr = "\n".join(result.errlines)
> assert "This is a UserDeprecationWarning (stacklevel 1)" in stderr
E assert 'This is a UserDeprecationWarning (stacklevel 1)' in 'Traceback (most recent call last):\n File "myscript.py", line 1, in <module>\n from openeo.internal.warnings impo... from openeo.rest.connection import Connection, connect, session\nModuleNotFoundError: No module named \'openeo.rest\''

Since it can't find openeo.rest I'm guessing the CI doesn't install the package before starting the tests, so PYTHONPATH is probably faulty.

The test seems to still pass in the Python 3.8 container when I run with uv:

openeo-python-client $ uv run --python 3.8 pytest -vvv tests/internal/test_warnings.py::test_user_deprecation_warning
============================= test session starts ==============================
platform linux -- Python 3.8.20, pytest-8.3.5, pluggy-1.5.0 -- /files/openeo-python-client/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /files/openeo-python-client
configfile: pytest.ini
plugins: time-machine-2.15.0, requests-mock-1.12.1
collected 1 item

tests/internal/test_warnings.py::test_user_deprecation_warning PASSED    [100%]

============================== 1 passed in 0.20s ===============================

so I think the test/package itself is fine.

On a more general note: is there any particular reason you work on this?

Well, the immediate thing I wanted is a py.typed in this package (just opened #921), but my Python journey started with 3.10 so I don't know anything about Python package building predating that. I figured it would be easier to convert this package to pyproject.toml now, and since that is presumably where things will be moving in the long-term it wouldn't be wasted effort to make the conversion now.

Switching to pyproject.toml would also make it easier to use uv on this package, and that would make my existing PyCharm configuration work as-is more or less.

I think it's the first time I've seen a private CI used on GitHub and I assume you have good reasons for why things are set up the way they are, but the black-box turning red does raise the bar for third-party contributions (and seems to give the maintainers extra work) compared to regular GitHub actions.