Skip to content

notes(assess): yt-dlp bottleneck profile is not compilable Python - #807

Draft
espetro wants to merge 1 commit into
masterfrom
work/ytbottle-note
Draft

espetro wants to merge 1 commit into
masterfrom
work/ytbottle-note

Conversation

@espetro

@espetro espetro commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Records the assessment of moving yt-dlp's bottlenecks into pyscriptc-compiled helpers that stock CPython yt-dlp imports, versus compiling the whole app. The note only; no code.

Verdict: not worth it on current evidence.

  • 93.8% of a real offline yt-dlp run is not yt-dlp's own Python. The largest block (36 to 40% of a 1.4 to 1.7 s CLI run) is CPython's re compiling 1704 _VALID_URL patterns.
  • The one real yt-dlp hotspot, orderedSet, is an O(n^2) dedup: 36.3 ms for 1754 items, 0.4 ms with a set. A one-line upstream fix beats a native rewrite.
  • No importable-extension path exists: --emit binding makes a ctypes loader, not a PyInit_* module, and its ABI carries only int, float, bool and str. A hand-built PyInit_* prototype of parse_duration ran 1.4x to 2.4x slower than stock (a 1.49 us ABI floor per call plus a regex engine slower than CPython's C re).
  • The native lane itself is fast on a typed orderedSet (24.8x), but that program cannot be called from CPython.
  • Every candidate fences PS1001 because stock yt-dlp is unannotated, so this is an annotation wall, not a language wall.

Cut from the ytbottle session's branch (53 commits, much of it duplicating the ytnative chain) so the note lands cleanly from master.

🤖 Generated with Claude Code

@espetro espetro added this to the Future milestone Nov 2, 2026
@espetro espetro added this to pyscriptc Nov 2, 2026
@github-project-automation github-project-automation Bot moved this to Todo in pyscriptc Nov 2, 2026
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying pyscriptc with  Cloudflare Pages  Cloudflare Pages

Latest commit: 3da95d5
Status: ✅  Deploy successful!
Preview URL: https://474b3beb.pyscriptc.pages.dev
Branch Preview URL: https://work-ytbottle-note.pyscriptc.pages.dev

View logs

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

1 participant