Skip to content

Use standard extension configuration for CPython 3.12+ - #1180

Open
jjhelmus wants to merge 18 commits into
mainfrom
jjh/standard_mod_config_312
Open

Use standard extension configuration for CPython 3.12+#1180
jjhelmus wants to merge 18 commits into
mainfrom
jjh/standard_mod_config_312

Conversation

@jjhelmus

@jjhelmus jjhelmus commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Switch CPython 3.12+ builds to the upstream Setup.stdlib extension-building mechanism while preserving the existing static/shared linkage behavior.

  • Build pkgconf on macOS to provide Tcl/Tk configuration to CPython.
  • Generate a minimal Setup.local for shared and disabled overrides on CPython 3.12+.
  • Restore CPython’s post-build extension checks and remove obsolete patches/workarounds.
  • Preserve SQLite extension loading and macOS symbol hiding.
  • Explicitly disable the clinic test and xxlimited_3_13 extensions. c.f. Enable _testclinic, _testclinic_limited and xxlimited_3_13 #1185

With this change, extension-modules.yml is still used for the following in CPython 3.12:

  • Disabling modules
  • Specifying static vs shared linkage
  • Validating the extension modules specified in the CPython source, Setup.stdlib.in
  • Constructing PYTHON.json

Constructing PYTHON.json for CPython 3.12+ from other sources would be a good follow up.

Makes various patches unnecessary:

Configuration can now appropriately detect SIMD support in the toolchain and adjusts libhacl flags accordingly.

This does not add shared extension suggested in #227. This could be done with a small change as a follow up.

@jjhelmus
jjhelmus force-pushed the jjh/standard_mod_config_312 branch from 2bc109e to 059aed0 Compare July 21, 2026 15:03
Comment thread src/validation.rs Outdated
Configure pkgconf to report the temporary directories of the prefix that
is used on macOS during CPython builds.
Build standard-library extensions through CPython's configure and
Setup.stdlib mechanism for Python 3.12 and newer.
Default to statically linking these modules into libpython/python with a
few overrides speficied in a Setup.local file created from
extension-modules.yaml.

This allows CPython's extension checks to run.

Python 3.11 and 3.10 contrinue to use the existing process to configure
and build extension modules.
Configuration correctly detect the SIMD support in the toolchain and
applies the appropiate flags when building libhacl.
@jjhelmus

jjhelmus commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

With MODULE_BUILDTYPE=static set CPython 3.12+ still builds a handful of extensions modules as shared, specifically:

  • _ctypes_test
  • _testimportmultiple
  • _testmultiphase
  • _testsinglephase
  • xxlimited
  • xxlimited_35
  • xxlimited_3_13

Using the custom configuration the last three of these are disabled and not built. The first four are build as shared modules except for static builds where they are statically linked. Because of the behavior of Setup.stdlib.in theses three need to be included in Setup.local to restore the static linkage. This should be revisited as disabling these in the static builds may be more prudent.

The existing behavior for 3.10 and 3.11 is to include a shared module, #1222 which is retained.
No shared modules are included in 3.12+ which is replicated here and validated.

…stdlib.in

A handful of modules are marked as always building shared which causes
these to be built as shared modules in the static builds.
@jjhelmus
jjhelmus force-pushed the jjh/standard_mod_config_312 branch from 5bcf925 to a77eb2c Compare August 18, 2026 18:32
@jjhelmus
jjhelmus requested a review from EliteTK August 18, 2026 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arch:all Select all architectures platform:darwin Specific to the macOS platform platform:linux Specific to the Linux platform python:3.10 python:3.11 python:3.12 python:3.14

Projects

None yet

2 participants