Skip to content

iOS: build the _multiprocessing extension (importable, not spawnable)#27

Merged
FeodorFitsner merged 3 commits into
mainfrom
ios-enable-multiprocessing
Jul 1, 2026
Merged

iOS: build the _multiprocessing extension (importable, not spawnable)#27
FeodorFitsner merged 3 commits into
mainfrom
ios-enable-multiprocessing

Conversation

@ndonkoHenri

Copy link
Copy Markdown
Contributor

Build the _multiprocessing C-extension for iOS. CPython marks it n/a on iOS (alongside _posixsubprocess and _posixshmem) because process spawning can't work in the app sandbox. But the extension itself builds fine on Darwin — macOS ships it — and import multiprocessing[.connection/.synchronize] is perfectly usable. Today, any package that imports multiprocessing at module load crashes on iOS with ModuleNotFoundError: No module named '_multiprocessing' — e.g. scikit-learn's sklearn.callback._transport, which does an unconditional from multiprocessing.connection import ....

This makes multiprocessing importable, not runnable. Process().start(), Pool, etc. still fail — that's the inherent iOS sandbox limit, and _posixsubprocess/_posixshmem stay disabled. The goal is only to stop the import-time crash, which is what actually bites in practice.

How

Two edits in darwin/build_ios.py, both after the patch and before Apple build:

  1. Flip the module on in the generated configure: py_cv_module__multiprocessing=n/a=yes. Version-agnostic — it hits both the vendored ios_patches/<3.14 configure and upstream 3.14+'s PY_STDLIB_MOD_SET_NA; a no-op if the string isn't present.
  2. Add ac_cv_func_sem_timedwait=no and ac_cv_func_sem_clockwait=no to the existing ios-config.site (next to the pipe2/dup3 overrides). Darwin has neither; the SDK may still declare them, so this forces _multiprocessing/semaphore.c onto its no-timeout fallback instead of referencing symbols that won't link.

Validation (3.12.13)

  • Build: produces _multiprocessing.xcframework with both ios-arm64 and ios-arm64_x86_64-simulator slices (a proper _multiprocessing.framework each) — compiles cleanly through Darwin's missing sem_timedwait.
  • Runtime (iOS simulator): injected the built framework into a Flet app and ran a probe.
    • Before: _multiprocessing MISSING, 1/5 — only the lazy import multiprocessing survives; .connection, .synchronize, SemLock all fail.
    • After: _multiprocessing PRESENT, 5/5 — including a real SemLock acquire / release / trywait round-trip (so sem_open works and the no-sem_timedwait fallback is fine), plus the .connection / .synchronize imports packages actually need.
image

Useful for scikit-learn on iOS (flet-dev/mobile-forge#92) without a per-recipe workaround, and fixes the same import crash for every other package that merely imports multiprocessing.

CPython marks _multiprocessing n/a on iOS (with _posixsubprocess/_posixshmem) because
process spawning can't work in the sandbox. But the module builds on Darwin (macOS ships
it): SemLock via sem_open, socket-based Connection — so importing multiprocessing.
connection/synchronize is fine; only Process().start() fails. Packages that import
multiprocessing at load without spawning (e.g. scikit-learn's sklearn.callback._transport)
currently crash on iOS with ModuleNotFoundError.

Flip py_cv_module__multiprocessing to yes in configure (version-agnostic — hits the
vendored <3.14 patch and upstream 3.14+ SET_NA), and add ac_cv_func_sem_timedwait/
sem_clockwait=no to CONFIG_SITE (Darwin lacks both) so semaphore.c takes its no-timeout
fallback. _posixsubprocess/_posixshmem stay disabled.

[skip ci]
@ndonkoHenri ndonkoHenri requested a review from FeodorFitsner July 1, 2026 15:16
@FeodorFitsner FeodorFitsner merged commit 5cc961a into main Jul 1, 2026
19 of 29 checks passed
@FeodorFitsner FeodorFitsner deleted the ios-enable-multiprocessing branch July 1, 2026 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants