Skip to content

feat(boxed): route import cycles island-side so the plane split engages on yt-dlp - #803

Draft
espetro wants to merge 6 commits into
work/imptabfrom
work/cycles
Draft

espetro wants to merge 6 commits into
work/imptabfrom
work/cycles

Conversation

@espetro

@espetro espetro commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Routes module-level import cycles island-side so the plane split engages on yt-dlp. With this branch the compiled yt-dlp binary runs and its output is byte-identical to CPython on --version, --help, --list-extractors (1755 lines) and a real --simulate download.

Three bugs were fixed: a cycle group's seed dropped the import of a group member (NameError: __version__), the island group did not carry the package chain (ImportError on a relative import), and the earlier "0% native" reading was not a cycle problem at all. On the plain entry shape the plane split now reports 5 IR functions, 63 classes, an 86-module island group and 941 per-module seeds; the entry top level is native, but only 3 of 458 functions are native.

The remaining wall is the stdlib import table: every yt-dlp module imports stdlib modules outside the closed table (binascii, codecs, collections.abc, email.header, html.parser, locale, mimetypes, netrc, secrets, shlex, ...), and one such fence routes that module's whole top level island-side. The split cannot lift that.

Gates: tests/test_plane_split.py 20 passed, island/plane/boxed selection 149 passed with 5 failures that also fail on the base branch, boxed:cost drop_flag false.

Stacked on #795.

🤖 Generated with Claude Code

Agent added 6 commits October 30, 2026 22:17
Refs #641, the plane split's cycle rule. _program_plan records the SCC
members instead of refusing them (under the boxed tier; without the tier
the refusal stays, there is no island floor to carry the cycle). The
group closes under the members' imports and parents, rides one bootstrap
under CPython (a finder over the group's sources plus the import order),
and its modules are not seeded individually (their imports need the
finder; a second seed would run the bodies twice). The split engages
around the group: the entry stays out of it (its main guard would never
open under a real-name import) and the package-shaped rule relaxes for
the group's own modules (the bootstrap reproduces the package semantics
the per-module seed cannot).
A three-module cycle with a native module outside the SCC: the members
ride the island group, the clean module lowers native, the output is
CPython's, and the binary carries one group bootstrap with no per-module
seed for the members.
…ses island-side

The refinement round on top of the cycle rule:

- An island-group member's module-level fences drop with its top level:
  the group's bootstrap execs the real body under CPython, so a lambda
  the checker refuses or an empty container literal there is CPython's
  own, exactly as under the whole-program island.

- The split's module rule generalizes: under the plane split any
  module-level fence routes that module's top level to its own island
  floor, not just the import fences. Without a plane split the fence
  list reaches the CLI as before.

- A group member leaves the init-owner map: its body runs in the group
  bootstrap, so a native init call would reach an island namespace with
  no __init__.

- A group member's own namedtuple named like a stdlib type stays
  island-side: the merged class table is program-wide, and emitting the
  module's Time next to the stdlib datetime Time refused the whole
  compile with PS2002 (yt-dlp's utils/_utils.py declares exactly that
  namedtuple).
A fenced module's island seed drops an import of a sibling whose
exports carry no island flag: the native lane ran the sibling's body
and the seed has nothing to resolve against. An island group member
breaks that rule: its namespace lives in `sys.modules` (the group
bootstrap execs the group's bodies under CPython before the per-module
seeds run) even when its definitions are constants. Dropping the
import loses a binding an island class body reads at definition time
(`PROVIDER_VERSION = __version__` in yt_dlp, NameError at startup).

The lowerer carries the group's member names; the seed keeps an import
whose target is a group member.
The group bootstrap installs every group member in `sys.modules`
before the per-module seeds run, so a seed's import of a group member
resolves against that namespace even when the member's exports carry no
island flag (its definitions are constants, so they lower native).
Dropping the import loses a binding the seed's island class body reads
at definition time: the binary died with a NameError before the entry
ran (yt-dlp's `PROVIDER_VERSION = __version__`).

The case: `alpha` and `beta` form the cycle, `version` holds only the
constant `VERSION`, and `provider`'s island class body reads it.
A group's module's relative import resolves through its package's
namespace. When the package is not in the group, its body runs from
disk and re-enters the submodule the group is still executing, so the
import fails with "cannot import name '_EJS_WIKI_URL' from
'yt_dlp.extractor.youtube.jsc._builtin.ejs' (unknown location)".

Every package unit joins the group when the cycle put one there: the
group's bootstrap owns the whole chain, so the relative imports resolve
against the group's own namespaces. Without a group the package-shaped
program still runs whole island-side, as before.
@espetro espetro added this to the v0.8.0 milestone Nov 2, 2026
@espetro espetro added this to pyscriptc Nov 2, 2026
@github-project-automation github-project-automation Bot moved this to Todo in pyscriptc Nov 2, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

1 participant