Skip to content

Let media play in the Linux lightweight engine (Canvas 33) - #60

Merged
shannah merged 1 commit into
masterfrom
claude/admiring-fermat-vwq7dm
Sep 30, 2026
Merged

shannah merged 1 commit into
masterfrom
claude/admiring-fermat-vwq7dm

Conversation

@shannah

@shannah shannah commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator

In the Linux lightweight engine (the default mode on Linux), HTML media never starts. No <video> or <audio> fires loadstart, networkState stays NETWORK_NO_SOURCE, and no error fires, whatever the source (data:, http:, blob:). The same page plays in WebKitGTK's own MiniBrowser on the same machine.

Why

This is from WebKitGTK 2.52.6's source:

  • A page starts with setCanStartMedia(false) and only turns it on once its view is IsInWindow (WebPage::updateIsInWindow).
  • widgetIsOnscreenToplevelWindow (UIProcess/gtk/GtkUtilities.cpp) returns false for a GtkOffscreenWindow, and that is what gtk_off_create_engine used.
  • So every media element in the lightweight engine waits forever.

The change (Canvas 33)

  • The new window. gtk_off_create_engine now creates a GTK_WINDOW_POPUP instead of gtk_offscreen_window_new(). Before showing it:
    • gtk_window_move(-32000, -32000)
    • no focus (accept_focus and focus_on_map off)
    • skip_taskbar and skip_pager hints
    • not decorated
  • Why it stays hidden. A popup is override-redirect on X11, the only backend the pump allows. So it is mapped, which is what WebKit needs, but it is off every monitor, undecorated, unfocusable and out of the taskbar.
  • Unchanged: pixel capture (gtk_widget_draw into our own surface, which never used the offscreen surface), input synthesis through gtk_main_do_event, the synthetic focus event, gtk_window_resize, destruction and popup adoption (Canvas 19). Only comments that named the window a GtkOffscreenWindow were corrected.
  • Custom schemes as a media src stay out of reach on Linux (D4). WebKitGTK's GStreamer player only accepts blob, data, file, http and https sources. Setting WEBKIT_GST_ALLOWED_URI_PROTOCOLS doesn't help either, because its source element only claims http, https and blob. Pages should fetch the scheme URL into a blob: and play that, which the smoke test checks.
  • SPDD: story 9, an analysis, Canvas 33, and a pointer from Canvas 7's approach.

Testing

New run-linux-media-smoketest.sh runs demos/WebViewMediaSmokeTest. It prints one line per check and exits 0 or 1, so it can run unattended under Xvfb:

Check Before this change After
A a <video> with a data:video/webm source reaches loadedmetadata 16x16 FAIL (nothing loaded) PASS
B a <video> playing a blob: made by fetch('mediatest://app/clip.webm') reaches loadedmetadata 16x16 FAIL PASS
P the component, painted into an image, shows the page's red background PASS PASS

Both columns ran under Xvfb with WebKitGTK 2.52.6 on Ubuntu 24.04. The same fix, reproduced first in a small PyGObject WebKit2 4.1 view, also played an H.264 MP4 at 1280×720.

Not checked:

  • Typing, clicking and scrolling. Those code paths are unchanged, but the smoke test doesn't drive them, so run run-linux-demo.sh by hand.
  • A real desktop session or compositor (Xvfb only).
  • The framework app with this build. That needs a release and a dependency bump in agentic-app-framework.

Found while

This came up while adding inline video playback to chat file cards in webliteca/agentic-app-framework#406. There, video cards stay plain on Linux until this lands.

Not in this PR: Windows

Canvas 32 deliberately allows fetch and XHR to a custom scheme only from pages on that same scheme (SetAllowedOrigins(["<scheme>://*"]), AC3). An app whose page is file:// therefore probably can't fetch its scheme media on Windows. Widening that is a security decision, and it can't be tested here, so it's left for a separate change.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VG4PCm6BfvNxs9mKKa7Df8


Generated by Claude Code

HTML media never started in the lightweight engine: no loadstart, no
error, from any source. WebKitGTK only lets a page start media once its
view is "in a window" (WebPage::updateIsInWindow), and it never counts a
view inside a GtkOffscreenWindow as one (widgetIsOnscreenToplevelWindow).

The engine now hosts its WebKitWebView in a GTK_WINDOW_POPUP placed at
(-32000, -32000): override-redirect on X11 (the only backend the pump
allows), so it is mapped but off every monitor, undecorated, unfocusable
and out of the taskbar. Pixel capture (gtk_widget_draw into our own
surface), input synthesis, sizing and popup adoption are unchanged.

WebViewMediaSmokeTest (run-linux-media-smoketest.sh) exits 0/1: a data:
WebM and a blob: made by fetch()ing a custom-scheme WebM both reach
loadedmetadata, and the page still paints. Under Xvfb it fails A and B
before this change and passes all three after it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VG4PCm6BfvNxs9mKKa7Df8
@shannah
shannah marked this pull request as ready for review September 30, 2026 14:35
@shannah
shannah merged commit 4fca56c into master Sep 30, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants