gh-153400: Add syscall fallbacks for copy_file_range/memfd_create - #155520
Conversation
10bc047 to
fe87313
Compare
Documentation build overview
25 files changed ·
|
fe87313 to
0dc17c3
Compare
|
@vstinner Addressed comments, no other changes |
|
I merged my PR gh-155518 which replace syscall() calls with glibc function calls. Oh, it created some conflicts in generated files. Would you mind to update your branch ( |
glibc only grew copy_file_range() and memfd_create() in 2.27, and we compile the os functions out when the libc we build against doesn't have them. That loses them for good in a redistributable built against an older glibc, such as the python-build-standalone builds targeting glibc 2.17, even when the kernel it runs on implements the syscalls. Keep calling the libc wrappers when they are available, so we don't lose their symbol versioning and _FORTIFY_SOURCE checks, and issue the syscall directly when they aren't. If the syscall number is missing as well, the functions are still left out. pidfd_open() and pidfd_getfd() already use raw syscalls, so nothing changes for them. Include <sys/syscall.h> whenever it exists rather than only when the getrandom() syscall was detected, since __NR_* is now needed for more than getrandom(). Signed-off-by: Daan De Meyer <daan@amutable.com>
0dc17c3 to
96a7a84
Compare
|
@vstinner Done |
vstinner
left a comment
There was a problem hiding this comment.
LGTM.
In general, I would prefer to only support an os function if it has a glibc (wrapper) function. But well, in practice, glibc takes time to add a new function. And getting the right {Python, glibc and kernel} combo can be complicated. So I'm fine with adding syscall() implementation for copy_file_range() and memfd_create().
glibc only grew the copy_file_range() and memfd_create() wrappers in 2.27, and CPython compiles os.copy_file_range() and os.memfd_create() out when the libc it is built against lacks them. We worked around that by forcing the configure checks on and weak linking the wrappers, which kept the functions out of the os module whenever the runtime glibc was older than 2.27, even on kernels implementing the syscalls. Backport python/cpython#155520 instead, which calls the wrappers when they exist and issues the raw syscalls when they don't, so both functions work on any sufficiently new kernel regardless of the glibc in use. The UAPI header overlay already provides the __NR_ constants and the MFD_ flags for all glibc targets. A single patch covers 3.10 through 3.15, so the weak linking patches and the 3.10 specific configure patch go away. The distribution tests now assert that both functions are always present on Linux GNU targets instead of tying their availability to the runtime libc. Signed-off-by: Daan De Meyer <daan@amutable.com>
…te (python#155520) glibc only grew copy_file_range() and memfd_create() in 2.27, and we compile the os functions out when the libc we build against doesn't have them. That loses them for good in a redistributable built against an older glibc, such as the python-build-standalone builds targeting glibc 2.17, even when the kernel it runs on implements the syscalls. Keep calling the libc wrappers when they are available, so we don't lose their symbol versioning and _FORTIFY_SOURCE checks, and issue the syscall directly when they aren't. If the syscall number is missing as well, the functions are still left out. pidfd_open() and pidfd_getfd() already use raw syscalls, so nothing changes for them. Include <sys/syscall.h> whenever it exists rather than only when the getrandom() syscall was detected, since __NR_* is now needed for more than getrandom(). Signed-off-by: Daan De Meyer <daan@amutable.com>
glibc only grew copy_file_range() and memfd_create() in 2.27, and we compile the os functions out when the libc we build against doesn't have them. That loses them for good in a redistributable built against an older glibc, such as the python-build-standalone builds targeting glibc 2.17, even when the kernel it runs on implements the syscalls.
Keep calling the libc wrappers when they are available, so we don't lose their symbol versioning and _FORTIFY_SOURCE checks, and issue the syscall directly when they aren't. If the syscall number is missing as well, the functions are still left out. pidfd_open() and pidfd_getfd() already use raw syscalls, so nothing changes for them.
Include <sys/syscall.h> whenever it exists rather than only when the getrandom() syscall was detected, since _NR* is now needed for more than getrandom().