Skip to content

fix(darwin): framework-ize ctypes .dylib libs so they load on the iOS simulator#223

Merged
FeodorFitsner merged 1 commit into
mainfrom
fix/darwin-ctypes-dylib-xcframework
Jul 1, 2026
Merged

fix(darwin): framework-ize ctypes .dylib libs so they load on the iOS simulator#223
FeodorFitsner merged 1 commit into
mainfrom
fix/darwin-ctypes-dylib-xcframework

Conversation

@ndonkoHenri

Copy link
Copy Markdown
Collaborator

Problem

A Python package that ships plain .dylib shared libraries loaded via ctypes (rather than a CPython .so extension) fails to dlopen on the iOS simulator:

dlopen(.../llama_cpp/lib/libllama.dylib): mach-o file (...), but
incompatible platform (have 'iOS', need 'iOS-simulator')

C-extensions (numpy, stdlib) work fine on the simulator; only .dylib-shipping packages fail.

Root cause

The darwin site-packages sync framework-izes only *.so files (xcframework_utils.sh: dylib_ext=so). Each .so becomes a device+simulator xcframework + a .fwork pointer, and Xcode extracts the correct per-slice binary — that's why C-extensions load on both device and simulator. Plain *.dylib files are skipped entirely and copied straight from the device base (sync_site_packages.sh copies archs[0] = iphoneos.arm64), so a simulator app ends up with a device-platform dylib.

The simulator/device intent never reaches this layer: flet build ipa and flet build ios-simulator both call serious_python … --platform iOS; they only diverge later at flutter build ios --simulator. So the fix has to make the .dylib path platform-correct on its own.

Fix (2 files, serious_python_darwin/darwin/)

  • sync_site_packages.sh: iterate both so and dylib in the conversion loop, passing the extension to the helper.
  • xcframework_utils.sh: take the extension as a parameter and use it for the simulator-slice globs, so .dylib libs get the same device+fat-simulator xcframework + .fwork as .so.
  • Preserve the install-name for .dylib (gate the install_name_tool -id rewrite on ext=so). This is the key to multi-lib ctypes packages: e.g. libllama has LC_LOAD_DYLIB @rpath/libggml.dylib. Keeping the install-name lets a wrapper that preloads its dependency libs with RTLD_GLOBAL satisfy those loads via dyld's already-loaded-image match. Rewriting the id to the framework path (the .so behavior) would break that.

Non-regression / blast radius

  • The .so path is byte-for-byte unchanged (ext=so keeps the id-rewrite) — verified numpy/stdlib frameworks are identical.
  • The change only activates on .dylib. The flet-lib* ctypes recipes all ship a single unversioned lib*.so on purpose, so nothing shipping today produces a .dylib — this is purely additive.
  • Guards for future .dylib recipes: -type f skips SONAME symlink chains, and conversion is skipped when a lib has no simulator slice (avoids an empty lipo), leaving it as-is.

Testing

  • llama-cpp-python on the iOS simulator (arm64): import + native ctypes calls + real GGUF inference all pass (previously the platform-mismatch dlopen failure). Verified against a normally-built app on both the published 1.0.1 darwin scripts and this branch's scripts.
  • .so non-regression: numpy frameworks unchanged (id still rewritten to the framework path).
  • Device slice validated statically from the intermediate xcframework (ios-arm64: platform 2, install-names preserved, inter-lib deps intact). Device runtime / code-signing not verified on hardware — the .dylib→framework path uses the same Xcode embed/sign as the existing .so framework path.

Proposed CHANGELOG change

* darwin: also package ctypes `.dylib` shared libs as per-slice xcframeworks on
  iOS (previously only `.so` C-extensions), so they load on the simulator;
  preserve their install-name so multi-lib packages resolve their siblings.

… simulator

The darwin site-packages sync only converted Python C-extension *.so files
into device+simulator xcframeworks (dylib_ext=so); plain *.dylib shared libs
(the kind a ctypes wrapper dlopens — e.g. llama-cpp-python's libllama +
libggml*) were skipped and shipped straight from the archs[0]=iphoneos DEVICE
slice. On the simulator that dylib then fails to dlopen:

    mach-o file (...libllama.dylib), but incompatible platform
    (have 'iOS', need 'iOS-simulator')

C-extensions pass because their .so is framework-ized into a fat (device +
simulator) xcframework and Xcode extracts the right per-slice binary; .dylib
libs never got that treatment.

Teach create_xcframework_from_dylibs to take the file extension as a parameter
and iterate both `so` and `dylib` in sync_site_packages.sh, so .dylib libs also
become a device+fat-simulator xcframework + a .fwork pointer, exactly like .so.

For .dylib, PRESERVE the original install-name (gate the `install_name_tool -id`
rewrite on ext=so). Multi-lib ctypes packages — libllama depends on
@rpath/libggml*.dylib — then resolve their siblings: the wrapper preloads the
dependency libs with RTLD_GLOBAL and dyld matches them by their preserved
install-name. The .so path is untouched (ext=so keeps the -id rewrite), so
C-extensions (numpy, stdlib) are byte-for-byte identical.

Guards: `-type f` skips SONAME symlink chains; conversion is skipped when a lib
has no simulator slice (avoids an empty `lipo`), leaving it as-is.

Tested with llama-cpp-python on the iOS simulator: import + native ctypes calls
+ real GGUF inference all pass (previously failed with the platform mismatch).
numpy (.so) unchanged; the device xcframework slice was validated statically
(platform 2, install-names preserved, inter-lib deps intact).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the Darwin site-packages packaging pipeline so ctypes-loaded .dylib libraries (not just CPython .so extensions) are converted into per-slice xcframework-backed frameworks for iOS, allowing them to load correctly on the iOS simulator.

Changes:

  • Extend site-packages conversion to process both *.so and *.dylib artifacts during iOS staging.
  • Parameterize the xcframework helper by extension and skip install_name_tool -id rewriting for .dylib to preserve install-names.
  • Add a guard to leave device-only artifacts unconverted when no simulator slice exists.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
src/serious_python_darwin/darwin/xcframework_utils.sh Adds extension-parameter support, simulator-slice existence guard, and conditional install-name rewriting for .so only.
src/serious_python_darwin/darwin/sync_site_packages.sh Expands the conversion loop to framework-ize both .so and .dylib files into xcframeworks during iOS packaging.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/serious_python_darwin/darwin/xcframework_utils.sh
@FeodorFitsner FeodorFitsner merged commit c8ce599 into main Jul 1, 2026
51 of 54 checks passed
@FeodorFitsner FeodorFitsner deleted the fix/darwin-ctypes-dylib-xcframework branch July 1, 2026 16:37
FeodorFitsner added a commit that referenced this pull request Jul 2, 2026
Re-pin python-build to release 20260701 and bump all packages to 4.2.1.

- python-build 20260701 only rebuilds the iOS runtime (now builds the
  `_multiprocessing` extension, importable-not-spawnable); the manifest is
  otherwise identical to 20260630, so all Python/Pyodide/dart_bridge/ABI
  versions are unchanged and every non-iOS runtime is byte-for-byte the same.
  Regenerated python_versions.dart + the four python_versions.properties via
  `gen_version_tables --release-date 20260701`.
- serious_python_darwin 4.2.1 ships the ctypes `.dylib` iOS-simulator fix
  (#223): plain `.dylib` shared libs are now framework-ized like `.so`
  C-extensions (with install-name preserved for multi-lib packages), so
  `.dylib`-shipping packages (e.g. llama-cpp-python) load on the simulator.
- Bump version in all six pubspec.yaml, the Android build.gradle.kts, and the
  Darwin podspec; add CHANGELOG entries to every package.
- Add a `prepare-release` Claude Code skill documenting this process.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants