Skip to content

gh-153400: Add syscall fallbacks for copy_file_range/memfd_create - #155520

Merged
vstinner merged 1 commit into
python:mainfrom
daandemeyer:push-rwtvzrusrnyp
Aug 13, 2026
Merged

gh-153400: Add syscall fallbacks for copy_file_range/memfd_create#155520
vstinner merged 1 commit into
python:mainfrom
daandemeyer:push-rwtvzrusrnyp

Conversation

@daandemeyer

@daandemeyer daandemeyer commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

glibc only grew copy_file_range() and memfd_create() in 2.27, and we compile the os functions out when the libc we build against doesn't have them. That loses them for good in a redistributable built against an older glibc, such as the python-build-standalone builds targeting glibc 2.17, even when the kernel it runs on implements the syscalls.

Keep calling the libc wrappers when they are available, so we don't lose their symbol versioning and _FORTIFY_SOURCE checks, and issue the syscall directly when they aren't. If the syscall number is missing as well, the functions are still left out. pidfd_open() and pidfd_getfd() already use raw syscalls, so nothing changes for them.

Include <sys/syscall.h> whenever it exists rather than only when the getrandom() syscall was detected, since _NR* is now needed for more than getrandom().

@read-the-docs-community

read-the-docs-community Bot commented Aug 10, 2026

Copy link
Copy Markdown

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The overall change LGTM.

Comment thread Modules/posixmodule.c
Comment thread Modules/posixmodule.c
@daandemeyer

Copy link
Copy Markdown
Contributor Author

@vstinner Addressed comments, no other changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@jjhelmus

Copy link
Copy Markdown
Contributor

This also closes #153463 and makes #153464 unnecessary

@vstinner

Copy link
Copy Markdown
Member

I merged my PR gh-155518 which replace syscall() calls with glibc function calls. Oh, it created some conflicts in generated files. Would you mind to update your branch (git merge main) and resolve merge conflicts (make clinic)?

glibc only grew copy_file_range() and memfd_create() in 2.27, and we compile
the os functions out when the libc we build against doesn't have them. That
loses them for good in a redistributable built against an older glibc, such
as the python-build-standalone builds targeting glibc 2.17, even when the
kernel it runs on implements the syscalls.

Keep calling the libc wrappers when they are available, so we don't lose
their symbol versioning and _FORTIFY_SOURCE checks, and issue the syscall
directly when they aren't. If the syscall number is missing as well, the
functions are still left out. pidfd_open() and pidfd_getfd() already use raw
syscalls, so nothing changes for them.

Include <sys/syscall.h> whenever it exists rather than only when the
getrandom() syscall was detected, since __NR_* is now needed for more than
getrandom().

Signed-off-by: Daan De Meyer <daan@amutable.com>
@daandemeyer

Copy link
Copy Markdown
Contributor Author

@vstinner Done

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

In general, I would prefer to only support an os function if it has a glibc (wrapper) function. But well, in practice, glibc takes time to add a new function. And getting the right {Python, glibc and kernel} combo can be complicated. So I'm fine with adding syscall() implementation for copy_file_range() and memfd_create().

@vstinner
vstinner enabled auto-merge (squash) August 13, 2026 12:12
@vstinner
vstinner merged commit 6e079e5 into python:main Aug 13, 2026
54 checks passed
zanieb pushed a commit to astral-sh/python-build-standalone that referenced this pull request Aug 13, 2026
glibc only grew the copy_file_range() and memfd_create() wrappers in
2.27, and CPython compiles os.copy_file_range() and os.memfd_create()
out when the libc it is built against lacks them. We worked around that
by forcing the configure checks on and weak linking the wrappers, which
kept the functions out of the os module whenever the runtime glibc was
older than 2.27, even on kernels implementing the syscalls.

Backport python/cpython#155520 instead, which
calls the wrappers when they exist and issues the raw syscalls when they
don't, so both functions work on any sufficiently new kernel regardless
of the glibc in use. The UAPI header overlay already provides the __NR_
constants and the MFD_ flags for all glibc targets. A single patch
covers 3.10 through 3.15, so the weak linking patches and the 3.10
specific configure patch go away.

The distribution tests now assert that both functions are always present
on Linux GNU targets instead of tying their availability to the runtime
libc.

Signed-off-by: Daan De Meyer <daan@amutable.com>
mbeijen pushed a commit to mbeijen/cpython that referenced this pull request Aug 14, 2026
…te (python#155520)

glibc only grew copy_file_range() and memfd_create() in 2.27, and we compile
the os functions out when the libc we build against doesn't have them. That
loses them for good in a redistributable built against an older glibc, such
as the python-build-standalone builds targeting glibc 2.17, even when the
kernel it runs on implements the syscalls.

Keep calling the libc wrappers when they are available, so we don't lose
their symbol versioning and _FORTIFY_SOURCE checks, and issue the syscall
directly when they aren't. If the syscall number is missing as well, the
functions are still left out. pidfd_open() and pidfd_getfd() already use raw
syscalls, so nothing changes for them.

Include <sys/syscall.h> whenever it exists rather than only when the
getrandom() syscall was detected, since __NR_* is now needed for more than
getrandom().

Signed-off-by: Daan De Meyer <daan@amutable.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants