Fix Python 3.13/3.14 on-device crashes: mimalloc seccomp open() + pruned _pyrepl#29
Merged
Merged
Conversation
Two distinct on-device runtime bugs that only surface on a genuine 3.13/3.14 mobile run (3.12 unaffected), found via mobile-forge genuine multi-Python mobile tests (fork run 27439192170): 1. _pyrepl was pruned from the iOS + Android stdlib excludes as a "dev-only / interactive-REPL" module. But CPython 3.14 pdb.py imports _pyrepl at module load, so any on-device code importing pdb (incl. pytest) dies with "ModuleNotFoundError: No module named _pyrepl". 3.13 pdb does not import it, which is why 3.13 iOS was unaffected. Un-prune _pyrepl in darwin/python-darwin-stdlib.exclude + android/python-android-dart.exclude. 2. mimalloc (bundled in CPython since 3.13) issues a bare open(2) syscall from its load-time constructor (mi_process_load -> unix_detect_overcommit reading /proc/sys/vm/overcommit_memory) where SYS_open is defined (x86_64). Android bionic seccomp forbids open(2) -> SIGSYS at dlopen of libpython, before the interpreter starts. arm64 is unaffected (no SYS_open; mimalloc falls back to libc open() -> openat). Patch mi_prim_open to use SYS_openat (android/patches/mimalloc_seccomp_openat.patch, gated >=3.13 in build.sh).
ndonkoHenri
added a commit
to flet-dev/mobile-forge
that referenced
this pull request
Jul 14, 2026
…er/packaging fixes, deploy-via-dispatch (#106) Flet 0.86 changed the mobile packaging model — site-packages ship as a compressed `sitepackages.zip` (zipimport); native `.so` are relocated to `jniLibs/<abi>/` with `.soref` markers on Android; and on iOS each extension is framework-ized and **linked into the app at launch**. That broke a cluster of recipes *on-device* even though their wheels build cleanly. This branch fixes the regressions across three layers — **forge core**, the **recipes**, and the **CI / recipe-tester** — and adds a **deploy-via-dispatch** path so fixed wheels can be published to pypi.flet.dev without a push to `main`. ## forge core — `src/forge/build.py` (`fix_wheel`), `schema/meta-schema.yaml` Three generic, recipe-agnostic packaging fixes: - **iOS `MH_BUNDLE` → `MH_DYLIB`** (`06f4f71`). CMake/scikit-build extensions link as `MH_BUNDLE` (they ignore CPython's `LDSHARED=-dynamiclib`); serious_python framework-izes and **links** them, and `ld` rejects a bundle (`Unsupported mach-o filetype (only MH_OBJECT and MH_DYLIB can be linked)`). `fix_wheel` injects an `LC_ID_DYLIB`, flips the filetype, and re-signs ad-hoc. - **Android foreign-arch `.so` drop** (`385430f`). A plain-setuptools sdist that builds `build_ext --inplace` into forge's reused source tree leaks a prior ABI slice's arch-tagged `.so` into the next slice's wheel (pymongo shipped the arm64 `_cbson`/`_cmessage` **byte-for-byte** in the x86_64 wheel → serious_python strips the arch off the SOABI tag → two arches collide on one `<name>.soref` → Gradle `duplicate entry`). `fix_wheel` now drops any `.so` whose ABI-tag triplet ≠ the target's. - **Android symbol-versioned `PyInit` ABI-tag** (`385430f`). forge ABI-tags a bare `NAME.so` exporting `PyInit_<name>` so serious_python writes its `.soref`. onnxruntime's pybind module exports the init symbol **version-scripted** (`PyInit_..._state@@VERS_1.0`), which the exact-match test missed → the `.so` shipped bare on both arches → on-device `ModuleNotFoundError`. The match now tolerates a trailing `@version` suffix. Plus a new `extract_packages` meta field (schema + recipe-tester wiring): Android-only list of packages to ship **extracted to disk** rather than inside `sitepackages.zip`, for "path-hungry" packages that read bundled data through a real `__file__` path (`NotADirectoryError` inside a zip). ## Recipe fixes **Flet 0.86 loader / packaging:** - **llama-cpp-python** — ctypes loader loads bundled libs by soname, falls through to the `jniLibs` bare-soname on Android, accepts `sys.platform == "android"` (PEP 738, py3.13+), and resolves iOS `.fwork` framework markers. - **opencv-python** — load native `cv2` under the top-level name (Android) + resolve the iOS framework-ized native. - **python-magic** — ship `magic.mgc` **in-wheel** and load it from memory (`magic_load_buffers`); Flet 0.86's `copyOpt` drops the flet-lib `opt/` data file. - **pycryptodome / pycryptodomex** — resolve the native module via `importlib`. - **selectolax** — drop the vestigial `modest` namespace import. - **matplotlib** — fix an iOS **launch SIGSEGV**: ft2font's pybind11 enum helper (`_enums.h` `P11X_DECLARE_ENUM`) acquired the Python GIL in a C++ **static initializer**, which dyld runs at app launch **before `Py_Initialize()`** under serious_python's link-at-launch iOS model. Deferred the GIL/pybind work to `bind_enums()` (runs inside `PYBIND11_MODULE`, at import). - **jq** — gate the static-link syntax per platform (GNU `ld` `-l:NAME.a` vs Apple `ld64 -lX`); `flet-libjq` keeps its static archives (`libjq.a`/`libonig.a`). - **scikit-learn** — drop the obsolete iOS multiprocessing-guard patch (python-build ships `_posixshmem` now). **Build-number bumps (republish with the fixes above):** pymongo→2, onnxruntime→2, matplotlib→2, llama-cpp-python→2, opencv-python→2, ncnn→2, faiss-cpu→2, pyarrow→2, pyobjus→2, opaque→2, pycryptodome→2, pycryptodomex→2, python-magic→2, scikit-learn→2, jq→2, coolprop→11, pysodium→11, selectolax→11, flet-libjq→11. **Minor (no wheel change):** astropy/spacy/thinc gain `extract_packages` (data via `__file__`); flet-lib{crc32c,freetype,jpeg,png,pyjni,yaml,geos} version jinja-templating. ## CI / recipe-tester (`.github/workflows/*`, `tests/recipe-tester/*`, `setup.sh`) - **`publish` input** — publish to pypi.flet.dev via `workflow_dispatch` (`publish: true`), in addition to the automatic push-to-main path. Lets a deploy dispatch publish with a pinned `python_build_run_id` + `serious_python_ref`. - **`serious_python_ref`** + **`python_build_run_id`** inputs — validate recipes against an unreleased serious_python branch / a specific python-build run before release; the recipe-tester template appends the sp git-override when set. - Android on-device app is built for **x86_64 only** (the emulator's arch) — faster and sidesteps `excluded_arches` pip-resolution failures (e.g. pyarrow/onnxruntime dropping armeabi-v7a). - `stage_recipe.sh` derives `[tool.flet.android] target_arch` from the built wheels and wires `extract_packages` + `test.requires`. - `setup.sh` pins python-build release **`20260714`** (includes `dc76612`'s `_pyrepl` prune + mimalloc seccomp `open()` fix — required for genuine 3.13/3.14 on-device tests). ## Validation - **matplotlib** — full all-python matrix green (iOS + android × py3.12/3.13/3.14). - On-device verified across the cluster: pymongo, onnxruntime, pyarrow (android), polars, shapely, pyzmq, selectolax, xxhash, llama-cpp-python, jq, opaque, ncnn (+opencv-python), python-magic, … - The earlier 3.13/3.14 android "seccomp `SIGSYS`" timeouts and py3.14 iOS `_pyrepl` failures were **flet/python-build infra bugs**, fixed by python-build `dc76612` (flet-dev/python-build#29) — not recipe bugs. - On-device flow uses serious_python `main`. ## Consumer notes The following packages now build **and run on-device** under Flet 0.86 (Android + iOS), published to `pypi.flet.dev`. Add them under `[tool.flet.dependencies]` (or your app's deps) as usual. - **matplotlib** — Agg backend works; `savefig(..., format="png")` verified on device. (iOS previously crashed at launch — fixed.) - **pymongo** — driver installs cleanly across all arches (no more x86_64/arm64 `.soref` collision). - **onnxruntime** — `InferenceSession` loads and runs (the pybind module now imports on device). - **opencv-python** — `import cv2` works on Android and iOS; heavy `.so`, expect a larger app. - **python-magic** — works without a system `magic` DB: the compiled `magic.mgc` ships in the wheel and is loaded from memory; no `[tool.flet]` data config needed. - **llama-cpp-python** — GGUF models load via the ctypes bindings on Android and iOS. - **ncnn** — pulls **opencv-python** as a runtime dep (both must resolve on your target). - **jq / opaque** — depend on `flet-libjq` / `pysodium` respectively (published alongside). - **pyarrow, faiss-cpu, scikit-learn, pyobjus, pycryptodome(x), selectolax, coolprop** — import + core functionality verified on device. Path-hungry packages (astropy, spacy, thinc, matplotlib) set `extract_packages` in their recipe, so their bundled data resolves on Android automatically — no app-side config required.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two runtime bugs make Python 3.13/3.14 apps crash on mobile — before any app code runs. Both are
flet buildmobile-test blockers on 3.13/3.14 (3.12 is unaffected).Bug 1 — Android: mimalloc's raw
open(2)is seccomp-killed (x86_64)CPython bundles mimalloc starting in 3.13. Its load-time constructor (
_mi_process_init→mi_process_init, an.init_arrayentry) reads/proc/sys/vm/overcommit_memoryviami_prim_open, which on platforms whereSYS_openis defined (x86_64/x86/arm32) issues the bareopen(2)syscall directly (to avoid recursion while the allocator is initializing). Android's bionic seccomp policy forbidsopen(2)— onlyopenat(2)is allowed — so the kernel raisesSIGSYS(SYS_SECCOMP) atdlopen()oflibpython, before the interpreter starts. Every native import (indeed the whole app) dies.SYS_open; mimalloc's#elsepath uses libcopen(), which bionic routes throughopenat), so real arm64 devices are unaffected — but the CI/emulator target is x86_64, where it always crashes.Fix:
android/patches/mimalloc_seccomp_openat.patchrewrites the onemi_prim_opencall:SYS_openat(AT_FDCWD, …)is semantically identical and is the syscall Android permits. Applied gated>= 3.13inandroid/build.sh(validated to apply to 3.13.14 and 3.14.6 sources).Bug 2 — iOS:
_pyreplis pruned but 3.14pdbimports it_pyreplwas excluded from the mobile stdlib as "interactive-REPL / dev-only". But CPython 3.14'spdb.pyimports_pyreplat module load, so anything importingpdb(e.g. pytest's debugging plugin) dies withModuleNotFoundError: No module named '_pyrepl'. 3.13'spdbdoesn't import it, which is why 3.13 iOS was unaffected.Fix: un-prune
_pyreplinandroid/python-android-dart.excludeanddarwin/python-darwin-stdlib.exclude.Validation
Built green here (run 29187831914, all versions × platforms). Verified the fixed binaries directly: the x86_64
libpython3.14.so'smi_prim_opensite now emitssyscall(257=openat, AT_FDCWD)instead ofsyscall(2=open), and the stdlib bundle restoresstdlib/_pyrepl/. Then ran it end-to-end on-device via mobile-forge (bundling this run's runtime): all six legs green — android + iOS × 3.12/3.13/3.14 — where android 3.13/3.14 and iOS 3.14 previously crashed.