Skip to content

feat(boxed): island classes cross the module boundary, 86% of yt-dlp top levels native - #795

Draft
espetro wants to merge 27 commits into
work/planesplitfrom
work/imptab
Draft

espetro wants to merge 27 commits into
work/planesplitfrom
work/imptab

Conversation

@espetro

@espetro espetro commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Island classes now cross the module boundary, stacked on #793 (plane split). The premise for this work was that stdlib imports outside the closed table gate the module top levels. Measured on yt-dlp, that was wrong: a top-level import uuid binds an island handle and does not fence. The dominant gate was the cross-module "not exported" fence (the #641 cascade), and its biggest cause was island classes missing from the export table (ExtractorError and friends).

Changes:

  • a4544e8f the export table carries island classes with their owner module, and the importing side binds them.
  • b082f453 an island class binds to its defining module, not the importing one (package re-exports).
  • 97ae42e3 an island seed drops the import of a sibling that lowered native, so Stage 2 seeds do not re-execute it.

Per-unit plane map on yt-dlp 2026.10.26 (1,043 units):

  • module top levels native: 17 (1.6%) to 896 (85.9%)
  • module top levels island: 1,026 to 147
  • island-class exports: 0 to 1,013
  • PS2002 fences in the catalog's failed set: 2,133 to 1

Honest result: the executed native fraction on yt-dlp is still 0%. The split does not engage because _program_plan refuses three real module-level import cycles (yt_dlp.utils.traversal, yt_dlp.networking.common, yt_dlp.extractor.common). That is the next wall. Planned native functions are 2 of 457, so even past the cycles the native lane needs the PS1001 annotation gap (421 fences) addressed through infer pins or yt-dlp-side typing.

Tests: tests/test_plane_split.py 14 passed (6 new). Boxed ledger (mise run boxed:cost, window d3902d6b..97ae42e3): code 5.6%, churn 20.3% (the two earlier windows read 0.0%), bugs 0.0%, tax 0.0%, drop_flag not raised (it needs two consecutive windows above 20%). The churn is the island-floor build-out; re-measure after the cycle work.

Agent added 6 commits October 30, 2026 16:10
One island class in a widely imported module dragged every consumer's
top level island-side (Decision A): the export table carried only
lowered classes, so the cross-module import fenced at module level with
'not exported by this package'.

The export table now carries island classes with the island flag, the
way it carries island functions (Refs #641). A consumer binds the class
across the boundary as an island-floor class: instantiation and
class-as-value go through the namespaced seam
(ps_island_mod_def_handle), a native subclass and an except clause
naming the class fence their own definition instead of the module
level.

In split mode a module's own island floor also rides the namespaced
seam: the single-namespace seed does not exist, so the calls and
handles name the owning module. The per-module seeds run in the order
CPython finishes the module bodies, so a seed that imports a sibling
finds that sibling's namespace in sys.modules.
…orting one

A package re-exports a class (`from ._utils import *`), so the module
the import names is not the module that holds the class object. The
cross-module island binding used the importing module's name, so the
handle and the call looked in a namespace that does not exist. The
export entry carries the owner, and the binding uses it.

The island class instantiation also returns the class's own handle
type, so an annotated destination agrees with the call site.
… native

A Stage 2 seed re-executed every top-level import, so a fenced entry
module whose import target lowered native died at startup with
ModuleNotFoundError: the split seeds one namespace per fenced module and
the native sibling has none. The seed now keeps an import only when its
target rides the island too (or is not a program module at all). The
import is not load-bearing: an island definition that used the native
sibling's definition already refused at the island-to-compiled seam.
The module-owns-its-floor test now exercises the seam with a real
native-to-island call in the entry.
…ntime

The bundle compile of a program now stages the runtime pack's CPython
headers (--embed-headers), the runtime lock file and the pack's own stdlib
zip, so the compiled binary can compile island programs at run time with no
Python install, no runtime pack cache and no network. A bundle dependency
root can be declared extern (--bundle-exclude), and a missing header
directory names the remedy.
tests/test_selfbin.py (marked slow) builds the self-compiled pyscriptc
through the cli and runs a hello world compile with no Python on PATH, and
mise run selfbin wraps the same build. The runtime-pack reference documents
the two-file bundle layout and the external tools the binary still needs.
…emedy

The -m pyrefly fallback in a self-compiled binary launches the compiler
itself, so the failure surfaced as an argparse usage error. The failure now
names PYREFLY_BIN, pip install pyrefly or --skip-check, and the doubled
Install one of: in PS0001 is deduplicated.
@espetro espetro added this to the v0.8.0 milestone Oct 30, 2026
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Deploying pyscriptc with  Cloudflare Pages  Cloudflare Pages

Latest commit: b69eaa1
Status: ✅  Deploy successful!
Preview URL: https://2e367470.pyscriptc.pages.dev
Branch Preview URL: https://work-imptab.pyscriptc.pages.dev

View logs

Agent added 6 commits November 2, 2026 11:47
…o the self-compiled bundle

The self-compiled bundle keeps its own carried_modules helper (a pack
whose stdlib travels as a zip carries the zip's names), so the merge
keeps it and adds the runtime set on top: the check answers the runtime
the bundle links, not the compiler host's build.
The --bundle-exclude and --embed-headers flags landed without a docs
regeneration, so the generated-docs seam reported the CLI reference as
out of date.
Agent added 2 commits November 2, 2026 14:00
…compiled bundle

The runtime-linked stdlib manifest check adds 7 self-host fences in
stdlibzip.py, so the bundle branch's census pin (3278) is stale on the
merged tree. The merged tree measures 3285 fences and the same package
sum; the pin moves with a comment that names both deltas.
…eed import drop

The runtime-linked stdlib manifest check adds 7 self-host fences in
stdlibzip.py, so the island-seed branch's census pin (3284) is stale on
the merged tree. The merged tree measures 3291 fences and the same
package sum; the pin moves with a comment that names both deltas.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

1 participant