PEP 803: add maintainer comments; correct tag in table

peps/pep-0803.rst

@@ -1,6 +1,6 @@
 PEP: 803
 Title: "abi3t": Stable ABI for Free-Threaded Builds
-Author: Petr Viktorin <encukou@gmail.com>
+Author: Petr Viktorin <encukou@gmail.com>, Nathan Goldbaum <nathan.goldbaum@gmail.com>
 Discussions-To: https://discuss.python.org/t/106181
 Status: Draft
 Type: Standards Track
@@ -81,6 +81,172 @@ This PEP proposes additional API limitations, as required to be compatible
 with both GIL-enabled and free-threaded builds of CPython.
 
 
+Ecosystem maintainers want decreased maintenance burden
+-------------------------------------------------------
+
+A major advantage of the limited API and stable ABI wheels is that new
+Python versions are supported on the day of release. Without stable ABI
+wheels, maintainers are left with a choice between closely following CPython
+releases and producing wheels during CPython beta periods or dealing with
+inevitable user requests for support for new CPython versions.
+
+
+Cryptography
+^^^^^^^^^^^^
+
+The cryptography project shipped 48 wheel files with their `most recent release
+<https://pypi.org/project/cryptography/46.0.5/#files>`_. Cryptography is
+somewhat unusual in that they ship 14 wheels each for both the ``cp38`` and
+``cp311`` stable ABIs to enable optimizations available in newer limited API
+versions. They also ship 14 additional ``cp314t`` wheels and 6 wheels for
+pypy. If there is no free-threaded stable ABI, then with Python 3.15,
+cryptography will be using roughly the same amount of space on PyPI to support
+two versions of the free-threaded build as *all* non-EOL versions of the
+GIL-enabled build.
+
+Cryptography maintainer Alex Gaynor `expressed a desire
+<https://discuss.python.org/t/making-pyobject-opaque-in-the-limited-api/77206/2>`_
+on Discourse for a free-threaded stable ABI:
+
+    Just to state this explicitly from the PyCA maintainers perspective, as
+    long as we have O(1) builds, that’s ok. What we can’t/won’t do is O(n)
+    where we need new builds for every Python release.
+
+When one of the PEP authors asked Alex in the ``#pyca`` Libera IRC channel for
+his current opinion, he said:
+
+    One other thing I'll note that's *really* valuable about ``abi3`` is that it
+    means our old wheels keep working for new Python versions. If we have
+    per-Python release wheels, we have to do a bunch of work at various points
+    in the python release cycle (including potentially backport releases to add
+    new wheels, if we're not otherwise planning a release at that time).
+
+    As maintainers, we *really* like to structure our work to avoid being "on
+    the clock" like that.
+
+
+moocore
+^^^^^^^
+
+The `moocore project ships <https://pypi.org/project/moocore/0.2.0/#files>`_
+seven ``abi3`` wheels. When the topic of adding support for
+the free-threaded build `came up on the moocore issue tracker
+<https://github.com/multi-objective/moocore/issues/37>`_, maintainer Manuel
+López-Ibáñez let the person reporting the issue know that:
+
+    I don't want to build for 3.14 free-threading unless you really need it.
+
+Later, after discovering the tracking issue for supporting the limited API on the
+free-threaded build, he commented:
+
+    By the way, python/cpython#111506 is about extending the stable ABI to
+    support free-threaded Python. If they do that, then the builds of moocore
+    will work in both classical and free-threaded Python versions, without
+    needing to build new wheels for each Python version.
+
+    [...]
+
+    I will revisit this again once Python 3.15 is released. Hopefully the ABI
+    will be stable (or even better, free-threading will be the default).
+
+Pydantic
+^^^^^^^^
+
+Pydantic maintainer David Hewitt `observed
+<https://github.com/python/peps/pull/4831#issuecomment-3972183807>`_:
+
+    Pydantic distributes wheels for a native core built using Rust & PyO3. The
+    latest release of `pydantic-core distributed
+    <https://pypi.org/project/pydantic_core/2.42.0/#files>`_ 112 wheels and
+    this number is set to grow as more environments are to be added
+    (e.g. Android, iOS, wasm). Pydantic has historically not distributed using
+    the stable ABI because the feature set was too immature. Much Pydantic
+    functionality interacts with Python objects via the C API in hot loops so
+    performance is key concern. As the stable ABI matures it will be ideal for
+    Pydantic to switch tier 2 platforms to the stable ABI (and perhaps
+    eventually tier 1 platforms too), which will significantly reduce the
+    number of wheels to build, test, and distribute.
+
+    I would like to highlight that if free-threading does not adopt a stable
+    ABI, all the benefits above will be lost when free-threading becomes the
+    default and only build (which seems the expected long-term plan).
+
+
+SciPy
+^^^^^
+
+The SciPy project uploaded 60 version-specific wheel files for its `last release
+<https://pypi.org/project/scipy/1.17.1/#files>`_ to support four different
+CPython versions. They do not upload wheels for PyPy.
+
+When asked about this proposal, SciPy steering council chair `Ralf Gommers said
+<https://github.com/python/peps/pull/4831#issuecomment-3968143777>`_:
+
+    SciPy and a number of other projects in the scientific Python ecosystem
+    are quite interested in starting to use the Stable ABI, in particular to
+    reduce the maintenance load of `providing more wheels
+    <https://pypackaging-native.github.io/meta-topics/user_expectations_wheels/>`_.
+    With recent CPython, Cython and NumPy releases, this now seems
+    possible. The performance costs seem acceptable and small, although we'll
+    only really build confidence in that assessment after having made the
+    switch.
+
+By providing a new free-threaded stable ABI in Python 3.15, SciPy will not have
+to consider the lack of a stable ABI on the free-threaded build as the project
+considers switching to stable ABI wheels.
+
+
+Bindings generators
+-------------------
+
+Both moocore and cryptography use bindings generators to interface with the C
+API. Cryptography uses PyO3 and CFFI while moocore uses only CFFI. Both CFFI and
+PyO3 already handle all the details of abstracting over the C API to enable
+different build configurations and there is no need to laboriously port
+extension types to use APIs that are only available on one build or another.
+
+Using bindings generators will enable these projects to quickly adopt the new
+stable ABI. Initial testing using the experimental ``_Py_OPAQUE_PYOBJECT`` flag
+defined in CPython's ``main`` branch, indicates that PyO3, CFFI, and Cython will
+all work with PEP 803 using packaging tools that have been patched to account
+for an ``abi3.abi3t`` tag.
+
+PyO3 maintainer `David Hewitt said
+<https://github.com/python/peps/pull/4831#issuecomment-3972183807>`_ in
+support of this proposal:
+
+    PyO3 greatly benefits from having a stable ABI - one of the biggest
+    challenges for the framework is the need to abstract over a wide range of
+    Python / OS / CPU / environment combinations. We also offer the
+    possibility to build with the stable ABI for each of these environments
+    (targeting a given minimum Python version's stable ABI). The goal is
+    always that all functionality PyO3 offers works the same on all these
+    combinations (sometimes with a Python version floor to access certain
+    features). We currently support Python 3.7+. All functionality added to
+    the stable ABI is a very welcome promise that PyO3 will not need to
+    introduce further conditional code to support a given feature. In the long
+    run this makes it possible for PyO3 to simplify code paths as support for
+    older Python versions is dropped, helping to keep maintenance burden under
+    control.
+
+When asked to comment about this proposal, Cython
+maintainer `David Woods said
+<https://github.com/cython/cython/issues/7399#issuecomment-3947605797>`_:
+
+    Cython doesn't have huge problems with the number of wheels we distribute
+    because ultimately it works fine as pure-Python. We do distribute wheels for
+    a few of the smaller platforms as Stable ABI wheels but that's more
+    "dogfooding" than because we actually need to. So I'm adding this in
+    anticipation that other people will find it useful rather than because I
+    will.
+
+    I do remain slightly concerned that the performance trade-offs for this will
+    turn out to be too much for many Cython users (it's possible that the
+    trade-off may be different for other binding tools). That's not a huge
+    disaster since we're not getting rid of the regular compilation mode so
+    people are free to pick their own personal trade-offs.
+
+
 Rationale
 =========
 
@@ -500,10 +666,10 @@ built with a given interpreter and ``Py_LIMITED_API`` macro:
 | ``cp315-abi3``        | 3.15+ (GIL) | 3.15               | ---                 | continued |
 +-----------------------+-------------+--------------------+---------------------+-----------+
 | ``cp315-abi3t``       | 3.15+       | ---                | 3.15                | new       |
-+                       +-------------+--------------------+---------------------+           +
-|                       | 3.15+ (FT)  | 3.15               | ---                 |           |
 +-----------------------+-------------+--------------------+---------------------+-----------+
-| ``cp315-abi3.abi3t``  | 3.15+       | 3.15               | 3.15                | new       |
+| ``cp315-abi3.abi3t``  | 3.15+ (FT)  | 3.15               | ---                 | new       |
++                       +-------------+--------------------+---------------------+           +
+|                       | 3.15+       | 3.15               | 3.15                |           |
 +-----------------------+-------------+--------------------+---------------------+-----------+