Skip to content

feat(pydantic): pydantic programs compile on the default lane - #805

Draft
espetro wants to merge 9 commits into
work/ecoreflectfrom
work/pydantic
Draft

espetro wants to merge 9 commits into
work/ecoreflectfrom
work/pydantic

Conversation

@espetro

@espetro espetro commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Makes pydantic programs compile and run on the default lane with no island flags, and records a 40-snippet corpus and a baseline.

Native lane share on the corpus: 0/40 before, 40/40 after. Island lane stays 40/40. The compatibility is out of the box; the model operations still execute in the embedded interpreter. Native validation is not claimed.

What changed

  • pydantic and pydantic_core route to the island lane when the pydantic-core distribution is installed (explicit [imports] entries still win).
  • A program importing an island-defaulted module gets the definition tier automatically, so class User(BaseModel) bodies no longer fence PS2002. Programs without such an import keep the refusal.
  • The island's site-packages directories travel in the IR and are seeded before the island seed.
  • Host-mode link no longer takes -lpython from a relocated interpreter's stale LIBDIR.
  • tests/test_sitedeps.py called a removed check_pure; it now calls check_staging.
  • Corpus and harness in benchmarks/pydantic/, design note in .agents/notes/2026-11-02-pydantic-support.md.

Divergence and gap

pydantic-core is a PyO3 crate whose schema dict embeds Python callables and class objects, and the shipped .so has 200+ undefined Py* symbols. A libpython-free seam is not possible; the island is the PyO3 boundary. Compile-time schema derivation for plain models (option B in the note) is the follow-up that would move field access and scalar validation native.

Known unrelated failure

test_embed_acceptance.py::test_the_bundle_stays_inside_the_size_gate fails identically at its base (bundle over the 60 MB gate).

🤖 Generated with Claude Code

Agent added 9 commits November 2, 2026 08:19
sysconfig answers LIBDIR with the build machine's prefix for a relocated
interpreter (a uv-managed toolchain, an installed binary), so the island
link pointed at a directory that does not exist and every island compile
failed. Fall back to the running executable's own prefix.
…the first

A multi-word distribution name (pydantic-core) lost its second word, so
the native-extension RECORD of one distribution was attributed to
another.
The bundle staging refused every native-backed distribution (PS4302) even
when the extension matches this ABI: pydantic-core (the Rust wheel behind
pydantic) could never travel in a bundle. Stage same-ABI, same-platform
extensions instead of refusing them, and keep refusing a mismatched one.
Also fix two closure bugs the staging exposed: a distribution whose name
equals its import root had its requirements walk skipped, and the copy
root recorded the requirement string instead of the import root
(pydantic-core vs pydantic_core).
pydantic's Python layer cannot lower, but the island lane is the working
boundary to pydantic-core: with pydantic installed, an import of it now
routes to the embedded interpreter by default instead of fencing PS2002
and requiring the user to know about [imports]. The default only fires
when the pydantic-core distribution is installed and native-backed; the
per-import policy still wins.

Alongside it: the research note for the pydantic seam and the 40-snippet
corpus under benchmarks/pydantic/ with its harness and baseline (native
0/40, island 40/40 under --fallback=cpython --embed-python=host).
…erpreter

A relocated interpreter answers LIBDIR with the build machine's prefix, so
host_has_shared_libpython() said no and auto picked the bundle. The run
cache then held a directory where it expected a binary, and the run
failed with a permission error instead of a compile error. Check the
running executable's own prefix too, the way the island link does.
A program that imports pydantic compiled to a refusal (PS2002) unless the
user knew about --fallback=cpython or an [imports] table. The island lane is
the working boundary to pydantic-core, so the default lane now takes it: a
program importing an island-defaulted module gets the definition tier too,
so `class User(BaseModel)` bodies ride the lane instead of fencing, and the
binary seeds the island dependencies' site-packages into the embedded
interpreter's sys.path (Py_InitializeEx starts with a bare one, so the
installed wheel was not importable).

The promotion is scoped to programs that import such a module: a program
that imports none keeps the no-flag refusal contract, and the explicit
[imports] table still wins.
check_pure is gone: the check that refuses a native dependency before
staging is check_staging, which also verifies the extension's ABI tag
against the pack. The tests still called the old name, so two of them
failed with an AttributeError. The fake extension also carried a real ABI
tag, which made the refusal a version question rather than the ABI
question the check answers.
…nest

The baseline recorded the default lane at 0/40 (every snippet fenced
PS2002) and the island lane at 40/40. With the seam in place the default
lane scores 40/40, so BASELINE.json is rewritten; the harness drops two
unused locals and the zip() gets its strict flag, and the corpus keeps its
typing spellings (Generic[T], Union[...]) through a per-file ignore, since
the modern spellings would change what the snippets measure.
…al share

Divergence 48 names the installed-default import route and the definition
tier it buys, 49 the site-packages seed and what it does not cover (the
compile-time directories, the program's own imports only). The guide
documents that a pydantic program runs out of the box and what that costs
(the model layer runs under the embedded interpreter; the values that cross
follow the handle rules). The note records the final numbers: default lane
0/40 -> 40/40, island lane 40/40 -> 40/40, and the caveat that this is
compatibility, not a native path for the model layer.
@espetro espetro added this to the Future milestone Nov 2, 2026
@espetro espetro added this to pyscriptc Nov 2, 2026
@github-project-automation github-project-automation Bot moved this to Todo in pyscriptc Nov 2, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

1 participant