Sync upstream tag 20260814 - #21
Draft
Schamper wants to merge 62 commits into
Draft
Conversation
As reported in astral-sh#956, executable stacks cause thread creation to fail under MemoryDenyWriteExecute=true. In kernel 6.3+ (which the GitHub Actions runners should have) this is implemented using prctl(PR_SET_MDWE). In older versions systemd uses a more complicated seccomp filter.
…h#1075) Currently, pull requests run all targets by default and labels can be used to select a subset of the matrix. However, CI is quite expensive and it's very rare to need to test the whole matrix, so the default is a bit backwards. In this change, we update pull requests to run on a subset of targets by default: Python 3.14 on macOS, Linux, and Windows with the most popular architecture. We include both glibc and musl variants of Linux. We include armv7 as an arbitrary cross-compile case — I'd be happy to take suggestions on an alternative there. This breaks our labeling concept a bit, as our labels currently do pure subsetting of the matrix. The labels will continue to subset, but with some nuances: - Applying a label will generally subset the default selection, e.g., `platform:linux` will only run the Linux subset of the defaults described above. - Labels that would null the default subset, e.g., `python:3.12` or `build:debug`, will instead change the default set to target that variant. - There are new `platform:all`, `python:all`, `arch:all`, `libc:all`, and `build:all` labels that can be used to expand the targets.
Statically link `python` to `libpython` on macOS. A dynamic `libpython` is still built and included in the distribution for embedded use. Remove patches that disable linking the libraries in statically linked Python stdlib modules to python as these are now needed. closes astral-sh#636
…sh#1104) Remove the Tcl/Tk translation files (.msg) and the Tk demos from the distribution artifacts. The former would only be needed in edge cases and the later is never be needed. closes astral-sh#1095
This patch addresses a segmentation fault issue on `LoongArch64` platforms caused by incorrect ELF segment alignment during the build process. - fix: astral-sh#1106 Signed-off-by: 吴小白 <296015668@qq.com>
* Update CPython 3.15 to 3.15.0b1
* Update CPython 3.14.5rc1 -> 3.14.5
Use the same LLVM toolchain that is used for native x86-64 builds to cross compile for riscv64 using a sysroot.
Following the behavior of CPython which dropped uplink support in 3.12, build OpenSSL with the `no-uplink` option starting with CPython 3.12. This allows a SSLContext to be created even when a SSLKEYLOGFILE environment variable is set on Windows. closes astral-sh#640
Include vm_sockets.h which is missing from the kernel UAPI headers in version 3.16 of the kernel. This file is missing due to a typo in the Kbuild file. The file is needed for CPython to make socket.AF_VSOCK available. closes astral-sh#1050
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
3.15.0b1 -> 3.15.0b2. Remove patch introduced in astral-sh#1123 as a different fix was merged upstream in python/cpython#149799 This makes astral-sh#1130 unnecessary
* CPython 3.13.13 -> 3.13.14 * CPython 3.14.5 -> 3.14.6
…3.11 (astral-sh#1147) This fixes the inability to load DER correctly with OpenSSL 3.5.7.
A maximum python version was not added when this file was added in astral-sh#1146. This will likely need to be removed or reverted when the next beta of 3.15 is released.
…-sh#1116)" (astral-sh#1150) This reverts commit 96a589a. Having a split LLVM / older GNU ld toolchain causes issues with relocation symbols and truncations.
…1152) Safely remove PYTHON* environment variables when running the distibution tests. Since the env dictionary is mutated within the loop a copy must be made.
libedit is used for command-line editing, readline is not used.
* zlib 1.3.1 -> 1.3.2
Disable the vectorized crc32 algorithm on s390x which requires z13. The current s390x target is z10.
This xz release includes a fix for CVE-2026-34743
* libX11 1.6.12 -> 1.8.13 * libXau 1.0.11 -> 1.0.12 * xorgproto 2024.1 -> 2025.1
…ts (astral-sh#1163) Use modern Linux kernel UAPI headers when building CPython for glibc Linux targets. Adds a `linux-uapi` package to the project that contains the unpacked `linux-libc-dev` package from a modern Debian release, currently `trixie-backports`. This provides Linux UAPI headers from a modern kernel release. Building CPython against these headers makes newer syscall definitions and macros available at build time, enabling CPython functionality such as `os.pidfd_open`. This does not increase the minimum required glibc or runtime kernel version. APIs compiled using these definitions may still fail at runtime. For example, if an API uses a direct syscall and the running kernel does not implement it, the kernel returns `ENOSYS`, which Python exposes as an `OSError`. Consequently, functions such as `os.pidfd_open` will exist when running on an older kernel that does not implement the corresponding syscall. Applications should handle `OSError` with `errno.ENOSYS` when runtime kernel support is uncertain. This should be included as a quirk of `python-build-standalone` and documented as such. The trade-off here is worth introducing this quirk. Related to astral-sh#193 Related to astral-sh/uv#11811, astral-sh/uv#9374
…#1166) Build CPython and dependencies with a set of hardening flags that take into account performance. * Enable stack protection and immediate binding for all linux builds. * Enable source fortification for optimized glibc targets. * Enable stack-clash protection on aarch64/x86-64 linux targets. * Validate these hardening properties in distributions using the `validate-distribution` command. Control-flow and branch protection flags are not included. On x86-64 `-fcf-protection` requires modern kernels, glibc and hardware (CET). Additionally, native extensions can fail to load when they are built without CET (which most are not) if the interpreter includes these protections. On aarch64, `-mbranch-protection` is of limited use unless all objects include the necessary markers. This includes the CRT and compiler runtimes which come from the base image which pre-dates these features. The `-fstack-protector` flag is used rather than `-fstack-protector-strong` which would instrument additional functions. The later was found to have a negative effect on performance (~2-3%). Debian currently uses this flag when building CPython in the 3.13 release for Trixie. Executable are not validate or required to be position independent (PIE). Testing found that requiring this has significant negative effects on performance (~5%). Debian builds CPython without PIE in the 3.13 release for Trixie. closes astral-sh#837
…ces (astral-sh#1174) Similar validation is done in the Debian `hardening-check` utility. closes astral-sh#1172
Add scripts/update_downloads.py, which finds updates to the entries in
pythonbuild/downloads.py. The optional --write argument write any
updates, including calculated metadata to the file.
Run using:
uv run scripts/update_downloads.py
Always build CPython with `os.memfd_create` but remove the Python function from the namespace at runtime when glibc does not provide the an implementation. This is accomplished using a weak `memfd_create` symbol which is evaluated at runtime. This requires the modern kernel UAPI headers from astral-sh#1163 to define the `MFD_*` constants.
…1178) Configure BOLT behavior using the `BOLT_COMMON_FLAGS` and `BOLT_APPLY_FLAGS` environment variables. The existing BOLT behavior is preserved. Previously patches were used to modify the default values defined in configure.ac This approach allows complete control over the BOLT behavior and makes the configuration easier to test, debug and update in the future. Would make the existing patches unnecessary. closes astral-sh#877 closes astral-sh#878 closes astral-sh#879
* CPython 3.15.0b3 -> 3.15.0b4
* expat 2.8.1 -> expat 2.8.2
Enable `os.getrandom` when building CPython against the modern Linux UAPI headers. Older glibc versions do not define `SYS_getrandom` which is used for the `os.getrandom` implementation. Patch CPython to use `__NR_getrandom` which is defined in the UAPI headers. This make `os.getrandom` available in all builds. This does not increase the minimum required glibc or runtime kernel version. If the running kernel does not support `getrandom`, the syscall returns `ENOSYS`, which Python exposes as `OSError`. Related to astral-sh#193.
Use the pattern from astral-sh#1164 to make `os.copy_file_range` available at runtime when the host glibc version provides the function.
Optimized x86-64 GNU/Linux builds can perform invalid stack writes in the HACL SHA-2 and BLAKE2 compressors because BOLT's allocation combiner removes their required stack allocation while retaining frame-pointer-relative accesses below the red zone. Extend `-skip-funcs` for the affected compressor symbols so BOLT leaves these functions intact while continuing to optimize the rest of the interpreter. See astral-sh#1186
Update to astral-sh#1056 to use `docker buildx` for reliable handeling of the `platform` argument. --------- Co-authored-by: Gregory Szorc <gregory.szorc@gmail.com>
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
* CPython 3.15.0b4 -> 3.15.0rc1
Update to the CPython provided Tcl/Tk 9.0.4 for 3.15+ on Windows.
Update: * Tcl 9.0.3 -> 9.0.4 * Tk 9.0.3 -> 9.0.4 Update build scripts to account for the bundled versions of itcl and thread. Include a fix for the Thread package's zipfs logic.
Switch macOS `pythonbuild` crate and CPython distribution builds from Depot to the existing `namespace-profile-macos-15` runner profile through `ci-runners.yaml`. Preserve the existing shared matrix and runner configuration for other platforms.
Use a sparse checkout of only the files needed during validations to speed up the checkout of the macOS SDKs repository. A full check out typically takes between 2-3 minutes although occasionally the checkout can take much longer.
* CPython 3.14.6 -> 3.14.7 On Windows, CPython 3.14 is built against Tcl/Tk 9.0.4.
* CPython 3.13.14 -> 3.13.15 Include backports from 3.15 in the `_testlimitedcapi` module.
Comes with one bug fix for using keyring from a non-activated virtual environment when installing build dependencies Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
Select the musl-static toolchain and tarball for static builds. Prior to this change `musl-static-<ver>-<target>.tar` was not declared as a dependency despite being a possible output of `build-musl.sh`.
Enable static builds for the `aarch64-unknown-linux-musl` target. --------- Co-authored-by: Tomasz Kramkowski <tomasz@kramkow.ski>
…1216) glibc only grew the copy_file_range() and memfd_create() wrappers in 2.27, and CPython compiles os.copy_file_range() and os.memfd_create() out when the libc it is built against lacks them. We worked around that by forcing the configure checks on and weak linking the wrappers, which kept the functions out of the os module whenever the runtime glibc was older than 2.27, even on kernels implementing the syscalls. Backport python/cpython#155520 instead, which calls the wrappers when they exist and issues the raw syscalls when they don't, so both functions work on any sufficiently new kernel regardless of the glibc in use. The UAPI header overlay already provides the __NR_ constants and the MFD_ flags for all glibc targets. A single patch covers 3.10 through 3.15, so the weak linking patches and the 3.10 specific configure patch go away. The distribution tests now assert that both functions are always present on Linux GNU targets instead of tying their availability to the runtime libc. Signed-off-by: Daan De Meyer <daan@amutable.com>
when packaging a `uv python` installed python into a .msix app for the microsoft app store, i hit some `signtool` failures with error `0x800700C1`. these are caused by `AppxSIP` going thru every binary in your package and validating them in some way. `llvm-strip` removes the signature from a bunch of DLLs that come with these python buidls, but it keeps the certificate table entry in the PE optional header , which makes the entry point past the end of the file. this is *exactly* the bug from astral-sh#855 that astral-sh#856 solved. i'm extending it to more DLLs and adding a comment explaining it. a more permanent fix might be detecting signatures on files and skipping if we see them? let me know if you'd rather that, i could wireup detecting signatures and/or zipfs to get rid of the hardcoded list that'll probably regress as versions change what DLLs are shipped.
* expat 2.8.2 -> 2.8.3
* CPython 3.10.20 -> 3.10.21 * CPython 3.11.15 -> 3.11.16 * CPython 3.12.13 -> 3.12.14 Remove DER parsing patch that was merged upstream in these releases.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Warning
Merge conflicts detected when merging upstream tag 20260814 into main.
The conflict markers have been committed so they are visible in the PR diff.
Conflicting files
Resolution