feat(cli): restore cursor overlay through page.evaluate - #2869
Conversation
|
…-cli-cursor-overlay
…-cli-cursor-overlay
There was a problem hiding this comment.
All reported issues were addressed across 4 files
Architecture diagram
sequenceDiagram
participant User as Browse CLI User
participant CLI as Browse CLI
participant Runtime as Runtime Command Handlers
participant Manager as Page Manager
participant Page as Active Browser Page
participant DOM as Page DOM Overlay
participant Browser as Browser Input and Renderer
Note over CLI,DOM: CLI-owned cursor visualization (no Stagehand core cursor API)
User->>CLI: browse cursor
CLI->>Runtime: Dispatch cursor command
Runtime->>Manager: Resolve activePage()
Manager-->>Runtime: Active page
Runtime->>Page: evaluate(CURSOR_OVERLAY_SCRIPT)
Page->>DOM: Ensure fixed overlay exists
alt Overlay already installed
DOM-->>Page: Reuse __browse_cursor_overlay__
else Overlay not installed
DOM->>DOM: Create fixed SVG cursor element
DOM->>DOM: Set aria-hidden, z-index, pointer-events none
DOM->>DOM: Append overlay to document root
DOM->>DOM: Install capture-phase mousemove listener
end
Page-->>Runtime: Evaluation complete
Runtime-->>CLI: Return enabled=true
CLI-->>User: Cursor overlay enabled
User->>CLI: browse mouse hover x y
CLI->>Browser: Move mouse to viewport coordinates
Browser->>DOM: Dispatch mousemove event
DOM->>DOM: Clamp clientX and clientY to non-negative values
DOM->>DOM: Update fixed overlay left and top
User->>CLI: browse screenshot --path cursor.png
CLI->>Page: Capture current page
Page->>Browser: Render DOM and overlay
Browser-->>CLI: Screenshot including visible cursor
CLI-->>User: Save cursor.png
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
…rbase/stagehand into agent/browse-v4-cli-cursor-overlay
There was a problem hiding this comment.
All reported issues were addressed across 5 files (changes from recent commits).
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Summary
Restore Browse's visible cursor as a CLI-owned DOM overlay, without adding a cursor API to core Stagehand V4.
cursor-overlay.tsfile.page.evaluate(CURSOR_OVERLAY_SCRIPT)and for future navigations throughpage.addInitScript(...).Stack (#2872)
--return-xpath; supported V3 parity/release checkpointWhy this is separate
The cursor is a self-contained optional visual feature with different review concerns from the combined V4 runtime/command migration: injected DOM/CSS, idempotency, event handling, and screenshot behavior. Keeping it additive on #2834 lets this feature be reviewed or reverted without disturbing browser lifecycle or commands.
E2E Test Matrix
The actual built CLI on final head
79d30de4bran these flows against synthetic pages in real Browserbase browsers:The point
(380, 270)was first asserted to be inside the iframe. The top-document DOM assertion then returned:{ "exists": true, "left": "380px", "top": "270px", "pointerEvents": "none" }browse cursor; navigate the same page; inspect the page-side registrationaddInitScriptpersisted the overlay behavior into a new documentbrowse mouse hover 380 270; inspect the top documentleft: 380px,top: 270px,pointer-events: nonebrowse screenshot --path cursor.pngmouse dragreturneddragged: trueand the target navigation was observedbrowse stoppnpm --filter browse lintpnpm --filter browse testThe screenshot visibly shows the single top-level cursor over iframe content after navigation:
The fixtures are synthetic and deterministic; no LLM path is involved.