Skip to content

fix(ios): book every surface on the lit panel, not the first window - #4122

Merged
thymikee merged 29 commits into
mainfrom
fix/foldable-springboard-surface
Nov 3, 2026
Merged

thymikee merged 29 commits into
mainfrom
fix/foldable-springboard-surface

Conversation

@thymikee

@thymikee thymikee commented Oct 31, 2026 •

Copy link
Copy Markdown
Member

Summary

Foldable surface commands name the lit panel and book a window by app bundle and request UUID. Capture, interaction, selectors and text probes reuse proven-lit roots. Typed refusals prevent outer-app fallback.

Required window-read exceptions now return APP_SCREEN_UNRESOLVED; genuinely absent windows retain APP_SCREEN_WINDOW_UNRESOLVED and are not asked for frames.

70 files in the full PR; this round changes two (+18/-1). No linked issue supplied.

Validation

Head ec80eecb1e0251d51d93aa68071c88577f1ea7ba; base c8ad8fb1257787ce0973cf74271cb9a4fafb51d9.

  • Baseline 4d6f523 plus the new native test fails five assertions across unreadable-only, unreadable-before-lit and unreadable-before-dark lists. At head, six selected native tests pass, including absent-window and ordinary booking controls.
  • iOS unit-runner build, format and XCTest-selection pass. Independent package/design review found no fixture leaks or justified smaller design.
  • pnpm check:affected --run --head ec80eecb passed; receipt .tmp/validation/run-RD6UYM/receipt.json, exactCommit=true, checkoutChanged=false; 11 passed, 10 skipped, 1 deduplicated. Vitest: 18,215 passed, 1 skipped, 0 failed.
  • CI Size: +24.1 kB unpacked/+7.2 kB compressed, all runtime content. CLI help measured 70.6 ms versus base 40.0 ms; attribution remains unproven. Prior head measured 43.3 ms with unchanged JavaScript sources.
  • All 19 hosted checks passed at 4d6f523; earlier live evidence retains its attribution. The exception regression uses native fault injection on iOS 27.1 Duo. No natural exception is claimed.

Hosted checks at this head: 17 succeeded, one optional matrix lane skipped, none failed. Human review approval remains outstanding.

@vercel

vercel Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
agent-device-website Ready Ready Preview Nov 3, 2026 1:39pm UTC

Request Review

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.71 MB 5.73 MB +24.1 kB
Package (unpacked) 5.71 MB 5.73 MB +24.1 kB
Package (download) 1.74 MB 1.75 MB +7.2 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 18.6 ms 25.0 ms +6.4 ms
CLI --help 40.0 ms 70.6 ms +30.6 ms

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 36 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

@apex-sentinel-staging

apex-sentinel-staging Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Apex Sentinel

Verdict: approve

Since the last review: 12 new · 3 still open · 11 no longer raised

15 finding(s): 7 minor, 8 nit. Verdict: approve.

🟡 Minor (7)

Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:100 elementSearchRoots throws unresolvedScreen for every empty lit window list, so the ADR's "no window at all" case answers with the wrong code
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:203 🆕 Ambiguous raw selector matches can no longer refuse before the coordinate booking, undoing the read the new test asserts
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift:82 🆕 Selector tap can be rejected as off-screen: the tap-point gate and the Maestro fallback gate use different window frames
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SnapshotAcquisition.swift:339 🆕 collapsedTabCandidates returns nil when a later query cannot start, discarding candidates the first queries already proved
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.m:205 A window whose read raises is booked as absent, so a walk that did not answer can report APP_SCREEN_OFF_LIT_PANEL
packages/apple-runner/src/runner-command-traits.ts:100 🆕 KeyboardDismiss is listed as a surface command, but no runner keyboardDismiss handler resolves a screen, so it pays a panel-table read per attempt
packages/platform-apple/src/core/runner-client.ts:27 🆕 Press-series chunks carry a lit panel only through one call-site convention; the re-exported runApplePressSeries still bypasses the platform wrapper
🔵 Nit (8)
Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Lifecycle.swift:55 🆕 Recording pumps frames after the request ends, where requestLitPanel is nil, so clip frames silently fall back to the first window
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerAppScreenCapture.swift:126 🆕 Off-lit-panel capture skips the system-surface route even for an app with no window on the named panel
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:96 🆕 elementSearchRoots treats any held XCTest issue during the window walk as a refusal, so an unrelated AX issue fails an element read with APP_SCREEN_UNRESOLVED
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift:42 executePrepared's screenshot exemption from lit-panel booking is unexplained
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swift:299 🆕 Lit-panel snapshots drop every window whose element snapshot names no display, silently shrinking the capture the single-panel route produced
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SnapshotExecution.swift:169 🆕 litPanelBackendPinRefusal maps an unparsable backend pin to "customActions"
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextInputProbe.swift:180 🆕 textInputQueries walks the whole window list four times per probe
packages/platform-apple/src/foldable/runner-panel.ts:13 🆕 Panel-probe refusal codes are duplicated as string literals instead of the exported typed constants

Reply under an inline comment with @apex-sentinel-staging note ... or @apex-sentinel-staging pivot ... to re-check that finding.
Comment @apex-sentinel-staging review on this conversation to run a full review.

Earlier reviews

Review of 06843017 · 03.11.2026 11:40 UTC

Apex Sentinel

Verdict: request_changes

Since the last review: 11 new · 3 still open · 13 no longer raised

14 finding(s): 1 major, 7 minor, 6 nit. Verdict: request_changes.

Required fixes

  • A lit-panel booking is never released when the next command starts, so the booking guard can be bypassed

🟠 Major (1)

Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:68 🆕 A lit-panel booking is never released when the next command starts, so the booking guard can be bypassed

🟡 Minor (7)

Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.m:112 🆕 Lit-panel read failure on the SpringBoard/alert host reports APP_SCREEN_WINDOW_UNRESOLVED instead of the panel reason
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:96 🆕 elementSearchRoots refuses an empty lit-panel window list that elementSearchRoots' own doc treats as valid
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift:33 🆕 Lit-panel booking is skipped only for screenshot; other capture-routing commands may pay a booking refusal before resolving their own surface
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntry.swift:589 boundTextEntryInputQuery picks the first matching root query by side-effecting .count, losing the fail-closed .element semantics
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextInputProbe.swift:96 probeTextInputs' empty-tree check still reads the whole app tree, not the lit-panel roots it searches
packages/platform-apple/src/foldable/orientation.ts:81 🆕 A device whose fold facts or panel count are unknown is routed as single-panel, so it silently keeps the first-window route
packages/platform-apple/src/foldable/orientation.ts:89 🆕 readFoldablePanelTable can cache a never-read panel count as single-panel for 5s when a short request is cut off mid-read
🔵 Nit (6)
Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:200 Selector candidate hasTappableFrame is now only computed for the non-hittable fallback path
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandDispatch.swift:434 🆕 A lit-panel recording keeps a captured panel value that dies with the request's Command
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:169 🆕 findElement(app:text:) loses the app-wide fail-open and can report the wrong first match across lit roots
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swift:327 🆕 hard-coded display-ID sentinel in the new windowsOnLitDisplay helper
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SynthesizedInteraction.swift:281 🆕 Touch visualization frame still uses app.frame's origin on a lit-panel request, while the tap anchors at the booked window
packages/apple-runner/src/runner-command-traits.ts:100 🆕 keyboardDismiss scoped as a surface command for a capture reason it does not serve
Review of d6aab77a · 03.11.2026 10:53 UTC

Apex Sentinel

Verdict: approve

Since the last review: 13 new · 3 still open · 11 no longer raised

16 finding(s): 9 minor, 7 nit. Verdict: approve.

🟡 Minor (9)

Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextInputProbe.swift:96 🆕 The probe's treeReadable check still reads the whole app, not the lit-panel roots it resolved
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:20 RunnerLitPanelState is read and written from several threads without synchronisation
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Alert.swift:190 🆕 Unsettled: XCTest issue containment inside withBoundedInteractionIdleTimeoutIfSupported for a throwing activateElement
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:270 🆕 resolveRunnerWindow can hand callers CGRect.null, whose origin is +infinity, where the predecessor returned app.frame
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:270 🆕 resolveRunnerWindow's .null fallback for a lit-panel request does not prevent the app-frame fallback it claims to forbid
packages/platform-apple/src/core/runner-client.ts:27 🆕 Surface commands sent through runApplePressSeries bypass the new lit-panel naming
packages/platform-apple/src/core/runner-client.ts:27 🆕 A retried or resent surface command reuses a lit panel read before the retry
packages/platform-apple/src/foldable/runner-panel.ts:60 🆕 The probe's time is charged to the request budget only when a panel is named
packages/platform-apple/src/foldable/runner-panel.ts:64 The probe-budget refusal does not disclose dispatched even though the disclosure fixture says it never dispatched
🔵 Nit (7)
Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:104 Lit-panel element search re-walks the whole (uncapped) window list on every read
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:200 🆕 hasTappableFrame is now skipped for candidates that are hittable or outside the expected point
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:74 🆕 bookRequestLitPanelSurface refuses an unreadable app bundle identity as APP_SCREEN_UNRESOLVED for every surface command
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntry.swift:589 🆕 boundTextEntryInputQuery picks its query by count, not by resolution
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:97 🆕 elementQueries' doc describes only element reads and omits text-input probes
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swift:484 🆕 publishLitPanelRoot is applied in two different shapes for the same invariant
packages/apple-runner/src/runner-command-traits.ts:100 🆕 ADR 0025's amendment table and the TS traits table agree, but nothing links them to the Swift booking rules
Review of 096d77d2 · 03.11.2026 10:09 UTC

Apex Sentinel

Verdict: request_changes

Since the last review: 8 new · 6 still open · 10 no longer raised

14 finding(s): 1 major, 5 minor, 8 nit. Verdict: request_changes.

Required fixes

  • Lit-panel bookings are reused across apps because === compares the XCUIAutomationObject proxy, not the app

🟠 Major (1)

Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:44 🆕 Lit-panel bookings are reused across apps because === compares the XCUIAutomationObject proxy, not the app

🟡 Minor (5)

Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:20 A lit-panel booking survives the next command when the request leaves main idle work behind
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:233 hasTappableFrame turns a booking refusal into a thrown error, changing what a duplicate-match refusal reports
packages/platform-apple/src/foldable/orientation.ts:76 🆕 readFoldablePanelTable back-off can strand a genuinely multi-panel device on the historic route after a first failed read
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:64 🆕 elementSearchRoots throws away the distinction it just logged
packages/platform-apple/src/foldable/runner-panel.ts:63 panel-table-unreadable can be reported for a panel table that answered, and the code reuses APP_SCREEN_UNRESOLVED
🔵 Nit (8)
Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Navigation.swift:107 🆕 backInApp ignores the booked lit panel: the back control is searched app-wide, so a dark-panel duplicate can be tapped
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:63 🆕 elementQueries re-walks the whole window subtree for every read a lit-panel request performs
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Lifecycle.swift:54 captureRunnerFrame now reads the main-only requestLitPanel from off the main thread
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Lifecycle.swift:52 🆕 captureRunnerFrame keeps reading the live requestLitPanel while the recording pump was pinned to its own panel
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swift:296 windowsOnLitDisplay's generic parameter buys nothing and the nil-display rule is duplicated
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.m:240 🆕 Lit-panel gestureViewport reports a null frame when no window books on the lit panel
packages/platform-apple/src/core/runner-client.ts:27 🆕 The lit-panel probe reads devicectl on the path of every tap on any device the fold fact admits
packages/platform-apple/src/foldable/runner-panel.ts:57 A surface request whose panel read came back empty still sends the full timeoutMs
Review of 34154226 · 03.11.2026 09:28 UTC

Apex Sentinel

Verdict: approve

Since the last review: 11 new · 5 still open · 14 no longer raised

16 finding(s): 9 minor, 7 nit. Verdict: approve.

🟡 Minor (9)

Where Finding
packages/platform-apple/src/foldable/runner-panel.ts:52 🆕 Panel probe consumes the request timeout twice; the runner gets less budget than options still promises
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:267 A lit-panel request whose window cannot be booked yields a .null window frame into TapPointPolicy instead of a typed refusal
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntry.swift:590 🆕 boundTextEntryInputQuery drops the app-wide fallback and picks roots by query count, so a lit-panel request can lose an input the old code resolved
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:62 elementSearchRoots caches the lit windows for the whole request, so a window that vanishes mid-request stays a search root
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift:34 🆕 backSystem and appSwitcher are refused with APP_SCREEN_OFF_LIT_PANEL for a gesture that never touches the app's windows
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:236 Lit-panel tap-point gate fails open when no window is booked, unlike the route it replaced
packages/platform-apple/src/foldable/orientation.ts:33 🆕 A single failing 5s panel read routes a foldable's surface commands through the first-window path for 5s
packages/platform-apple/src/foldable/runner-panel.ts:54 🆕 Panel-table probe silently overwrites the request's own timeoutMs and can refuse the request outright
packages/platform-apple/src/foldable/runner-panel.ts:55 🆕 A request with no explicit timeoutMs gets no budget accounting for the panel read
🔵 Nit (7)
Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:20 🆕 Lit-panel write is queued asynchronously, so a command can read the previous command's panel
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.m:313 Element search walks every window uncapped, discarding the #​2995 issue cap the booking walk exists for
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:17 Lit-panel state lives on RunnerTests but is written by a queue hop and read from other threads' queued work without a documented single owner
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift:429 🆕 The .type no-response fallback now shares a Result with the lit-panel refusal
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swift:296 🆕 windowsOnLitDisplay and snapshotDisplayID are only ever exercised with their real arguments
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.h:43 🆕 AGENT_DEVICE_MAX_LIT_PANEL_WINDOWS is a bare define with no literal in ADR 0025
packages/platform-apple/src/foldable/orientation.ts:81 🆕 Lit-panel gate depends on device name matching, so an unnamed Duo silently keeps the buggy first-window route
Review of 19b038ae · 03.11.2026 08:38 UTC

Apex Sentinel

Verdict: request_changes

Since the last review: 16 new · 3 still open · 11 no longer raised

19 finding(s): 2 major, 10 minor, 7 nit. Verdict: request_changes.

Required fixes

  • Coordinate taps/long-presses/drags on a lit-panel request anchor at the booked window's real origin while offsetting by the panel-box origin
  • RunnerCopyLitPanelWindows ignores the AGENT_DEVICE_MAX_LIT_PANEL_WINDOWS cap the header documents, so every lit-panel element read walks all ~40 windows

🟠 Major (2)

Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:642 🆕 Coordinate taps/long-presses/drags on a lit-panel request anchor at the booked window's real origin while offsetting by the panel-box origin
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.m:319 🆕 RunnerCopyLitPanelWindows ignores the AGENT_DEVICE_MAX_LIT_PANEL_WINDOWS cap the header documents, so every lit-panel element read walks all ~40 windows

🟡 Minor (10)

Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:637 A lit-panel tap that fails to book its window computes coordinates from CGRect.null origin
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:79 elementSearchRoots treats an empty lit-window list as a valid search root set, so a foldable can report ELEMENT_NOT_FOUND instead of the typed refusal
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:80 🆕 Lit-panel element-search roots are cached for the whole request, so a window opened by a mutation is never searched
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:239 hasTappableFrame can hand TapPointPolicy a null window frame on a lit-panel request
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntry.swift:555 🆕 A lit-panel refusal during text-entry confirmation is reported as a text-typing failure, not the typed panel code
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.m:260 🆕 A lit-panel request whose window list holds only vanished windows books nothing and is answered APP_SCREEN_WINDOW_UNRESOLVED instead of a panel-typed code
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.m:306 🆕 RunnerCopyLitPanelWindows walks the whole window list, ignoring the cap the header says exists for this walk
packages/platform-apple/src/core/display-inventory.ts:418 🆕 appleLitPanel checks activeDisplay but then picks the lit panel from power
packages/platform-apple/src/core/runner-client.ts:27 🆕 runAppleRunnerCommand now has two entry points with different behaviour: the new wrapper in runner-client.ts (whi…
packages/platform-apple/src/foldable/orientation.ts:33 🆕 A first multi-panel request that finds no lit panel permanently disables lit-panel routing for that device
🔵 Nit (7)
Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:43 🆕 RunnerResolveLitPanelWindow discards the window frame it resolved and recomputes a panel box
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandDispatch.swift:436 🆕 Reading requestLitPanel for the recording clip breaks the state's main-thread-only rule
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift:36 🆕 Lit-panel booking is skipped for .screenshot only via a command comparison, so the list of surface commands is duplicated in the runner
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntry.swift:594 🆕 `return (queries.count == 1 ?
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextInputProbe.swift:180 🆕 Duplicated input-type list in textInputQueries
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.m:255 🆕 Unplaced-window rule is stricter than the ADR text (refuses a full-panel window found before an unplaced one)
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.m:294 🆕 RunnerResolveLitPanelWindow's log line prints "application screen resolved" for a lit-panel booking
Review of 9f5d9469 · 03.11.2026 01:51 UTC

Apex Sentinel

Verdict: request_changes

Since the last review: 13 new · 1 still open · 12 no longer raised

14 finding(s): 1 major, 5 minor, 8 nit. Verdict: request_changes.

Required fixes

  • litPanelScreenFrame picks the panel's orientation from the window's aspect, so a partial window books an off-panel reference box

🟠 Major (1)

Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:63 🆕 litPanelScreenFrame picks the panel's orientation from the window's aspect, so a partial window books an off-panel reference box

🟡 Minor (5)

Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Lifecycle.swift:54 🆕 Recording pump reads requestLitPanel from shared state instead of the panel its recordStart booked
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerAppScreenCapture.swift:110 🆕 Off-lit-panel capture refusals get the wrong hint text
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:73 elementSearchRoots treats a read that answers with no lit windows as a successful empty search
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:236 🆕 hasTappableFrame names the booked window but passes onScreenWindowFrame
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SnapshotExecution.swift:172 🆕 New runner error code is a bare string literal in the refusal
🔵 Nit (8)
Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:73 🆕 Element-search issues on a lit-panel read are held and thrown away, not re-recorded
packages/platform-apple/src/foldable/orientation.ts:33 🆕 A lit-panel request re-reads the panel table on every request; the 5s back-off only ever applies to a device whose panel count is still unknown
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntry.swift:592 🆕 boundTextEntryInputQuery reads XCUIElementQuery.count outside any issue containment
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:68 🆕 One over-long line in litPanelScreenFrame breaks the file's wrapping
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:90 🆕 elementQueries runs one full-subtree query per lit window for every read
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:642 🆕 A lit-panel tap anchors at an infinite offset when the booked window belongs to another app instance instead of refusing
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swift:209 🆕 Snapshot root is published after the walk in one route and before it in the other
packages/platform-apple/src/foldable/orientation.ts:89 🆕 A cancelled panel-table read for a known foldable is reported as panel-table-unreadable instead of a cancellation
Review of 83add297 · 02.11.2026 22:33 UTC

Apex Sentinel

Verdict: approve

Since the last review: 10 new · 3 still open · 11 no longer raised

13 finding(s): 5 minor, 8 nit. Verdict: approve.

🟡 Minor (5)

Where Finding
packages/platform-apple/src/core/runner-client.ts:32 🆕 runAppleRunnerCommand recomputes the request timeoutMs, so a read-only surface command is resent with the short remaining budget
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift:429 🆕 typeCommand swallows a thrown lit-panel refusal into a generic 'type produced no response' error
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:65 elementSearchRoots returns empty array, not a refusal, when zero windows match the lit display
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swift:299 🆕 The lit-panel snapshot has no fallback when a window snapshot reports no displayID
packages/apple-runner/src/runner-command-traits.ts:121 🆕 home is excluded from RUNNER_COMMAND_RESOLVES_SURFACE although the ADR names a home-screen gesture as the bug being fixed
🔵 Nit (8)
Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:634 Interaction coordinate: lit-panel branch falls through to the shared path on iOS, duplicating the resolved-window logic
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.m:328 RunnerCopyLitPanelWindows walks the whole window list, ignoring the AGENT_DEVICE_MAX_LIT_PANEL_WINDOWS cap it exists to bound
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SnapshotExecution.swift:169 🆕 litPanelBackendPinRefusal hardcodes the tree backend's rawValue instead of comparing to the enum
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:29 🆕 setRequestLitPanel's main-queue ordering guarantee is void when the command starts on the main thread
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:48 🆕 Mid-request app switch silently replaces the booked lit-panel surface
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift:25 🆕 executePrepared books the lit-panel surface for every command except screenshot (RunnerTests+CommandExecution.swi…
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swift:331 🆕 The panel filter's display read is a second, looser implementation of the bridge's display-ID lookup
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextInputProbe.swift:22 🆕 Text-input queries are built outside the probe's issue scope, so containment rests on XCUITest query laziness
Review of 72bc4411 · 02.11.2026 21:55 UTC

Apex Sentinel

Verdict: request_changes

Since the last review: 9 new · 5 still open · 6 no longer raised

14 finding(s): 1 major, 8 minor, 5 nit. Verdict: request_changes.

Required fixes

  • The lit-panel window walk reads every window's frame outside the text-input probe's issue containment

🟠 Major (1)

Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:60 The lit-panel window walk reads every window's frame outside the text-input probe's issue containment

🟡 Minor (8)

Where Finding
packages/platform-apple/src/core/runner-client.ts:27 🆕 Every surface command on a known foldable now pays a fresh devicectl/guest panel probe, up to 5s of its own timeoutMs
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:232 🆕 hasTappableFrame now fails open on a lit-panel request whose window cannot be booked
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntry.swift:536 lit-panel refusals swallowed by try? fall back to app-wide reads that may be on the dark panel
packages/platform-apple/src/foldable/orientation.ts:66 readFoldablePanelTable keeps a permanent single-panel verdict from a failed first read on a device whose panel count was never read
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swift:206 🆕 Collapsed-tab fallback nodes are still gathered from the whole application tree on a lit-panel snapshot
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.m:302 🆕 RunnerCopyLitPanelWindows ignores the AGENT_DEVICE_MAX_LIT_PANEL_WINDOWS cap its own constant documents
packages/platform-apple/src/core/runner-client.ts:22 litPanel is added only by the platform-apple runner-client wrapper, not at the apple-runner contract entry
packages/platform-apple/src/foldable/runner-panel.ts:33 🆕 A cancelled panel-table read is reported as panel-table-unreadable
🔵 Nit (5)
Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:60 elementSearchRoots caches the lit-panel window list for the whole request, so a mid-request fold answers from the dark panel
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:22 🆕 New RunnerLitPanelFact fields are decoded but never validated against the units the runner compares them in
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:629 🆕 interactionCoordinate's #if os(iOS) block was reopened, so the lit-panel branch silently changes the macOS path structure
packages/apple-runner/src/runner-command-traits.ts:95 🆕 RUNNER_COMMAND_RESOLVES_SURFACE is positioned between the traits table and the accessors that read it
packages/platform-apple/src/__tests__/interactor-foldable-runner-requests.test.ts:33 🆕 The new test file's leading comment is a single five-line sentence
Review of 0c2a02ed · 02.11.2026 19:44 UTC

Apex Sentinel

Verdict: request_changes

Since the last review: 6 new · 5 still open · 103 no longer raised

11 finding(s): 1 major, 6 minor, 4 nit. Verdict: request_changes.

Required fixes

  • Lit-panel capture treats an unreadable window list as APP_SCREEN_WINDOW_UNRESOLVED, which falls back to SpringBoard — the dark-panel capture the ADR says must fail

🟠 Major (1)

Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.m:247 Lit-panel capture treats an unreadable window list as APP_SCREEN_WINDOW_UNRESOLVED, which falls back to SpringBoard — the dark-panel capture the ADR says must fail

🟡 Minor (6)

Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntry.swift:536 🆕 probeTextInputs / resolveTextEntryInput now throw away a lit-panel refusal, reading it as no input
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift:30 Lit-panel booking now gates system-surface commands (backSystem, appSwitcher, keyboardDismiss) on the session app having a lit-panel window
packages/platform-apple/src/foldable/orientation.ts:81 The panel-table back-off caches a transient failure as "single-panel" and silences the lit panel for 5s
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:60 🆕 Lit-panel window list is cached for the whole request, so a window that appears mid-request is invisible to later reads
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SnapshotExecution.swift:172 🆕 A stale private-ax backend pin now aborts every Apple settle/selector capture on a foldable
packages/platform-apple/src/core/runner-client.ts:27 host-bound runAppleRunnerCommand callers bypass lit-panel booking entirely
🔵 Nit (4)
Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.m:318 RunnerCopyLitPanelWindows walks every window with no cap, the hazard AGENT_DEVICE_MAX_LIT_PANEL_WINDOWS exists to bound
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:60 🆕 A selector/point read of an app whose windows are all off the lit panel reports "no match" instead of APP_SCREEN_OFF_LIT_PANEL
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SnapshotExecution.swift:172 🆕 litPanelBackendPinRefusal message can claim a pin to the same tree backend it requires
packages/platform-apple/src/core/runner-client.ts:42 🆕 Exhausted panel-read budget is turned into a 1 ms runner timeout instead of a typed refusal
Review of 04bd1109 · time not recorded

Apex Sentinel

Verdict: request_changes

Since the last review: 14 new · 1 still open · 93 no longer raised

15 finding(s): 1 major, 7 minor, 7 nit. Verdict: request_changes. reached max rounds (1).

Required fixes

  • A stale litPanel on the session app's command hard-refuses taps for a single-panel (or non-foldable) target served in place

🟠 Major (1)

Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift:16 🆕 A stale litPanel on the session app's command hard-refuses taps for a single-panel (or non-foldable) target served in place

🟡 Minor (7)

Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SnapshotCapturePlan.swift:165 🆕 A raw or backend-pinned snapshot on a foldable is not narrowed to the tree tier, yet still filters every window by displayID — so it can always refuse with APP_SCREEN_UNRESOLVED
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift:261 🆕 Lit-panel route silently falls back to app.frame when the booking fails, the fallback the amendment says it removes
packages/platform-apple/src/core/runner-client.ts:32 The panel probe is added in front of the runner timeout instead of taken from it, so a surface command can spend 2x timeoutMs
packages/platform-apple/src/foldable/orientation.ts:32 🆕 Panel-table back-off is shared with the rotation route, so one failed probe also disables guest-HID routing for 5s
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:32 🆕 Panel booking is keyed to no app, so a later read for a different process reuses the first app's window
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandDispatch.swift:431 🆕 A fold during a recording makes every pump frame refuse on the lit panel, and the pump drops them silently: recordStop reports success on an empty clip
packages/platform-apple/src/foldable/orientation.ts:69 🆕 A lit-panel read that fails on a caller deadline backs off every surface command for 5s, even while rotations keep succeeding
🔵 Nit (7)
Where Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swift:214 🆕 Snapshot sets the root rect to the panel's native box while its windows keep turned frames
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:32 🆕 Booked lit-panel XCUIElement is reused across the whole request; a vanished window later records an XCTest issue (#​2995)
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Models.swift:222 🆕 New RunnerLitPanelFact decodes non-numeric JSON fields as 0, silently taking the single-panel route
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swift:207 🆕 Unused let on the lit-panel snapshot path
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SnapshotCapturePlan.swift:165 🆕 plan shadows its own parameter in the new lit-panel override
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.h:38 🆕 RunnerLitPanel and the panel-fact struct duplicate the same three fields with two spellings
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.m:191 🆕 RunnerLitPanelWindowScreen logs on every call with no mute and reads two KVC keys per window

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 19 new findings on 37db43f. The verdict and the full list are in the summary comment.

@thymikee
thymikee force-pushed the fix/foldable-springboard-surface branch from 37db43f to 563e4e8 Compare October 31, 2026 14:33

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 9 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/adr/0025-foldable-apple-panels.md Resolved

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 19 new findings on 563e4e8. The verdict and the full list are in the summary comment.

@thymikee

Copy link
Copy Markdown
Member Author

At 563e4e8 the lit-panel fix is not ready: two device-facing paths changed at this head have no live run, and several earlier problems still stand. CI is green (20 checks, none failing), and there are no conflicts. Before merge, the Duo evidence below must be posted and the open P1 threads fixed.

The graft was rewritten (it no longer throws, it copies seen, and it adds a graft-failed reason), and readText joined the panel-carrying set. The PR evidence prints republishedRoot=yes droppedSurfaces=4 but no GRAFTED or SKIPPED marker, so it does not show that the graft ever fires on a Duo. No Duo run shows a coordinate read booked on the lit panel either. If the graft never fires, about 200 production lines and a dozen open threads guard dead code. If it does fire, its timing against the tree_processing watchdog is unmeasured. readText could still read the dark panel (https://github.com/callstack/agent-device/blob/563e4e8/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Snapshot.swift#L352). Please run two unfolded Duo runs at this head. First, snapshot -i on the home screen, and paste the runner.log lines AGENT_DEVICE_RUNNER_PANEL_SURFACE_TREE_GRAFTED (or _SKIPPED reason=…) and AGENT_DEVICE_RUNNER_PANEL_SURFACE_TREE, plus the published node count. If GRAFTED never appears in either pose, drop the graft. Second, get text by coordinate on a home-screen or Settings label, and paste AGENT_DEVICE_RUNNER_PANEL_SURFACE lit=3 … selection=0|1 for that request and the returned text.

The snapshot-tree rewrite (root republish, off-panel surface drop, SpringBoard window graft) is a second concept on top of the surface-window fix. It spans RunnerPanelSurfaceTree.swift (158 lines), the graft and walk extraction in Snapshot.swift (about 200), the SnapshotCapturePlan hook, and 237 test lines. Fifteen open threads sit on it, including the P1 below on the main-thread read, which still holds at this head (https://github.com/callstack/agent-device/blob/563e4e8/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerPanelSurfaceTree.swift#L1). Please land the resolver and viewport fix first. Move the republish and graft to a follow-up, gated on a Duo run that shows the corrected viewport alone still leaves the lit panel's icons absent.

Not blocking, and you can take or leave these: (1) widthPt and heightPt have one runner reader, RunnerPanelFrameIsLitPanelSurface (RunnerPanelSurface.m:5-26), and the selection already falls back to the largest window on the named display, so the size half of the wire rarely decides anything and adds transposition, a 0.5 tolerance and the pointScale division in appleLitPanel; sending litPanel as { displayId } only, and amending ADR 0025 to match, would remove that. (2) recordStart is in RUNNER_SURFACE_RESOLVING_COMMANDS (runner-client.ts:51), but startAppleRecording sends every simulator to simctl recordVideo, so on the production route a recordStart never carries a panel; remove it from the set, with the recording clause at runner-client.ts:47-50 and the ADR consequence line. (3) appleLitPanel (display-inventory.ts:425) re-derives the 'exactly one lit panel decided' rule from inventory.displays, and so accepts the incomplete-table branch that buildInventory never decides; have buildInventory expose litDisplay, set only when decided, and read that. (4) RUNNER_SURFACE_RESOLVING_COMMANDS (runner-client.ts:32) is a hand list that must mirror which Swift handlers reach RunnerResolveSurfaceWindow, and nothing ties the two together, so the next handler that calls onScreenWindowFrame takes the first-window route on a foldable; sending litPanel on every runner command for a device cached as multi-panel, or declaring a resolvesSurface trait on the runner command descriptor and asserting the host set against it, would close that. (5) The AppleLitPanel doc in interactor-types.ts:117 and ADR 0025 (~line 720) say an absent panel means a single-panel device, while runner-contract.ts:158-163 correctly adds 'or the table decided nothing'; use the runner-contract wording in all three.

Could the PR be smaller? Send litPanel: { displayId } only, taken from a litDisplay that buildInventory sets only when decided, and let the runner pick the largest measurable window on that display or refuse. That drops widthPt/heightPt, transposition, both tolerances, the pointScale division and RunnerPanelFrameIsLitPanelSurface. For a device cached as multi-panel, send the panel on every runner command, or derive the set from a runner descriptor trait, and delete the hand list with its dead recordStart entry. Pass the panel as an argument from the handler into resolveRunnerWindowElement instead of the lock-guarded slot, so abandoned work cannot read a later command's panel. Move the republish and graft (about 350 production lines plus 237 test lines) to a follow-up gated on the first Duo run above. Together this plausibly cuts 700-900 production lines and closes most open threads, including the P1s on the graft, the cap and the slot. To get there, ADR 0025 needs to change first: drop the native-point-box clause, or justify it with a Duo window larger than the panel, and add litDisplay to AppleDisplayInventory.

These open threads still apply and need a fix at their owners: the abort on the devicectl path that parks the device for 5s (#4122 (comment)), the incomplete-table lit panel (#4122 (comment)), the single-slot panel read (#4122 (comment)), the main-thread graft watchdog (#4122 (comment)), the off-panel refusal gap (#4122 (comment)) and the cap of 8 (#4122 (comment)). The P2s at #4122 (comment) (synthesized dispatch display), #4122 (comment) (runbook lines removed), #4122 (comment) (absent panel wording), #4122 (comment) (probe before litPanel check), #4122 (comment) (requestId cancellation) and #4122 (comment) (selection=3 overreach) also hold, as do about 25 lower-priority threads and nits. Please resolve these threads, which no longer apply: #4122 (comment) (readText is now in the set and has a fixture), #4122 (comment) (the no-panel walk matches main's loop), #4122 (comment) (the graft is non-throwing now), #4122 (comment) (readText is in the set), #4122 (comment) (reportedCount is written before the cap), #4122 (comment) (sliceLeft is taken after the resolve), #4122 (comment) (seen is copied, rollback discards it), #4122 (comment) (redundant branch removed), #4122 (comment) (factCount matches the ADR), #4122 (comment) (absent litPanel takes the historic branch), #4122 (comment) (the probe runs once before the retry loop), #4122 (comment) (same sliceLeft ordering), #4122 (comment) (the catch rolls back every graft node), #4122 (comment) (each attribute is read once), #4122 (comment) (home uses pressHomeButton) and #4122 (comment) (all producers use the wrapper).

I could not run a Duo simulator and saw only the PR-body log excerpts, not the raw runner.log. I did not run the Swift unit tests or pnpm check:affected; the green status is CI's and the author's. The redundancy claim in note (1) assumes no window on the lit display is larger than the panel, which I did not find in the fixtures or the evidence but cannot prove for every SpringBoard state. I did not read the interactions.test.ts mock wiring or the header whitespace threads.

@thymikee
thymikee force-pushed the fix/foldable-springboard-surface branch from 563e4e8 to 32e56a4 Compare October 31, 2026 15:44

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 18 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/agents/device-foldable.md Resolved
Comment thread docs/adr/0025-foldable-apple-panels.md Resolved

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 15 new findings and 1 moved to a new line on 32e56a4. The verdict and the full list are in the summary comment.

@thymikee
thymikee force-pushed the fix/foldable-springboard-surface branch from 32e56a4 to 5f2be24 Compare October 31, 2026 16:38

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 8 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/adr/0025-foldable-apple-panels.md Resolved

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 16 new findings on 5f2be24. The verdict and the full list are in the summary comment.

@thymikee

Copy link
Copy Markdown
Member Author

I reviewed 5f2be24. The earlier findings from the 563e4e8 review are mostly fixed, but two defects remain, and the required Smoke Tests check is red.

The macOS phase of Smoke Tests (run 41220653930) fails in the runner unit suite: 390 tests, 2 assertion failures, both in testSurfaceWindowWalkRefusesAWindowOnTheLitPanelWhenTheListWasCutShort (:478 and :482). That test exercises the new guard, so this failure comes from the diff. The iOS phase passed, with flaky retries on 01-settings.ad and gesture-pan-duration.ad. I could not run a Duo simulator, and I did not run the Swift tests locally.

The new guard at RunnerXCTestEventBridge.m:348 can never run. It sits inside failure != RunnerApplicationScreenFailureNone, and litDisplayWindow maps to None (:215-217). A cut-short list still books the largest window on the lit display, here the 226x155.3 corner strip. This is the wrong-box booking the PR set out to fix, and ADR 0025:735 and the PR body describe a refusal the code does not make. The rule: the resolver books a non-exact window, or refuses, only from a list it read whole. A result that the cap left undecided must never close the system-surface fallback.

The truncation downgrade at :340 also stops offLitPanel from reaching the host. It rewrites offLitPanel to the unresolved-screen failure when sawUnplaceableWindow || listTruncated is set. selectObservedScreenCapture moves on to SpringBoard only for .unresolvedWindow or .offLitPanel (RunnerAppScreenCapture.swift:79). reportedCount is written before the cap (:165-167), and a Duo's SpringBoard names 39-43 windows against a cap of 8. So listTruncated is always YES on the SpringBoard fallback, and APP_SCREEN_OFF_LIT_PANEL can never reach the host on the device this PR targets. A capture that the system surface could serve fails with "did not report the display it is shown on". The same happens to screenshot and the keyboard and recording samples when an app has one unplaceable window or more than 8. The rule: a walk that found nothing on the lit panel must still move on to the system surface, and only the final answer may weaken its claim. selectObservedScreenCapture lists the move-on cases. Please add a unit test with more than 8 windows, all off the lit panel, that asserts the selectObservedScreenCapture result and not only the walk result.

The evidence the earlier review asked for is still missing (RunnerTests+Snapshot.swift:355). The PR body has no AGENT_DEVICE_RUNNER_PANEL_SURFACE_TREE_GRAFTED or _TREE_SKIPPED reason=... line, no published node count, and no coordinate readText run on the lit panel. Without them it is unknown whether the graft ever fires on a Duo, how its main-thread read fares against the tree_processing watchdog, or whether readText books the lit panel. After the two fixes above, on an unfolded iPhone Duo, please run snapshot -i on the home screen in both poses and paste the runner.log lines AGENT_DEVICE_RUNNER_PANEL_SURFACE_TREE ... and _TREE_GRAFTED frame=(...) nodes=<n> (or _TREE_SKIPPED reason=...) with the published node count. If GRAFTED never appears, please delete the graft. Then run get text by coordinate on a home-screen or Settings label and paste the AGENT_DEVICE_RUNNER_PANEL_SURFACE lit=3(669.0x951.0) ... selection=0|1 line for that request and the returned text. I have not checked the raw runner.log behind the "Device evidence at 5f2be24" claim.

Could a smaller design remove all of this? The design question in the earlier review got no reply, and this delta adds more patches around the same concepts: a second truncation downgrade (dead, as above), a bypass flag on the retry memo, an optional panel parameter that falls back to the slot, and an observedFrame write that no caller reads. (1) Uncap the panel walk: read windows in order, stop at the first exact panel-box match, and only when there is none, finish the list and take the largest window on the lit display or refuse offLitPanel. The ADR says the lit panel's full-screen window sits inside the first eight in every observed pose, so real poses cost the same. This deletes AGENT_DEVICE_MAX_PANEL_SURFACE_WINDOWS, listTruncated and both downgrades, and it fixes both defects. (2) Move panelTableRetriesAt into resolveRunnerRequestLitPanel and return isAppleFoldableSimulator to main's shape, which deletes askPanelTableAgain. (3) Read command.litPanel once in executeAccepted and pass a non-optional RunnerLitPanel down, which deletes the slot, its lock, the seven reads and the ?? requestSurfacePanel fallback. (4) Have buildInventory expose litDisplay, set only when decided. (5) Move the republish and the graft to a follow-up gated on the evidence. ADR 0025 would need to change first: drop the walk-cap clause at :735, replace the rejected "threading one optional panel" alternative with per-command argument passing, and add litDisplay to AppleDisplayInventory. What do you think?

Not blocking, and fine to take or leave: the delta joined let home = Self.duoHomeScreenWindow() onto the signature line of testSurfaceWindowWalkSkipsAWindowThatWillNotNameItsDisplay (RunnerTests+XCTestEventBridgeTests.swift:434), so it can go back on its own line.

Open threads: these still apply and should stay open, P1 #4122 (comment) (one panel slot) and #4122 (comment) (main-thread graft), P2 #4122 (comment) (missing xcrun), #4122 (comment) (guard cannot run) and #4122 (comment) (no-panel wording). About 40 lower-priority threads also still apply. These do not apply at 5f2be24 and can be resolved. Fixed in code: #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment). Answered or benign: the graft does not throw and rollback drops its keys #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment); readText, screenshot, home and the wrapper are already covered #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment); the count and else-if comments no longer match the code #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment); replay, probe and abort paths are correct #4122 (comment) #4122 (comment) #4122 (comment) #4122 (comment); the no-panel walk stops at the first measurable window #4122 (comment) #4122 (comment) #4122 (comment).

@thymikee
thymikee force-pushed the fix/foldable-springboard-surface branch from 5f2be24 to 1fbcbfa Compare October 31, 2026 18:01

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 15 new findings and 1 moved to a new line on 1fbcbfa. The verdict and the full list are in the summary comment.

@thymikee

Copy link
Copy Markdown
Member Author

This is a follow-up on the earlier review (#4122 (comment)), now at 1fbcbfa. The two failing unit assertions are fixed, and a truncated or unplaceable lit-panel window now refuses instead of booking the strip. I checked this by reading the fixture and resolver, not by running the Swift runner suite. Smoke Tests is still running. Its macOS phase runs the resolver this change touches, so its result counts for this PR. There are no conflicts.

One defect remains. The new refusal at https://github.com/callstack/agent-device/blob/1fbcbfa/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerXCTestEventBridge.m#L341 still turns an off-lit-panel walk into unresolvedScreen when a window is unplaceable or the list is truncated. A SpringBoard on the Duo names 39-43 windows against the cap of 8, so the fallback is always truncated. selectObservedScreenCapture only moves on for unresolvedWindow or offLitPanel, so an app-side downgrade skips the system surface, and a SpringBoard-side downgrade ends in APP_SCREEN_UNRESOLVED. On the Duo, APP_SCREEN_OFF_LIT_PANEL can then never reach the host from the fallback. A capture, screenshot, keyboard or recording sample that SpringBoard could serve fails with "did not report the display it is shown on" and a misleading retry hint. The rule: a walk that found nothing on the lit panel must still let the caller ask the system surface, and only the final answer may weaken its claim. The smallest fix is to uncap the panel walk (stop at the first exact panel-box match, otherwise read the whole list). That removes listTruncated and AGENT_DEVICE_MAX_PANEL_SURFACE_WINDOWS. Keep returning offLitPanel from the walk, and apply the unplaceable downgrade only to the fallback result in selectObservedScreenCapture. Please add a unit test with more than 8 windows, all off the lit panel, that asserts the selectObservedScreenCapture result. Invert testSurfaceWindowWalkReportsADisplayRefusalWhenAnUnreadableWindowRemains to match.

The PR body still has no live output for the graft, the tree republish or readText by coordinate (https://github.com/callstack/agent-device/blob/1fbcbfa/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests%2BSnapshot.swift#L355). After the fix above, please run snapshot -i on the home screen of an unfolded iPhone Duo in both poses. Paste the runner.log lines AGENT_DEVICE_RUNNER_PANEL_SURFACE_TREE republishedRoot=.. droppedSurfaces=.. nodes=a->b and _TREE_GRAFTED frame=(...) nodes=<n> (or _TREE_SKIPPED reason=...), plus the published node count with icons listed. Delete the graft if GRAFTED never appears. Then run get text by coordinate on a home-screen or Settings label. Paste the AGENT_DEVICE_RUNNER_PANEL_SURFACE lit=3(669.0x951.0) ... selection=0|1 line and the returned text. This shows whether the main-thread graft read fits the tree_processing watchdog, whether the turned republish drops the app tree (#4122 (comment)), and whether readText books the lit panel.

Not blocking: these can be taken or left. The new unplaceable-window refusal has no walk case that combines ScreenlessWindowFixture with innerPanelStripWindow (RunnerTests+XCTestEventBridgeTests.swift:467), so one case with [ScreenlessWindowFixture(), innerPanelStripWindow()] expecting .unresolvedScreen and a nil window would cover it.

Open threads from other reviewers. These still apply: the one-slot overwrite in command dispatch (#4122 (comment)), the main-thread graft read (#4122 (comment)), the ADR 0025 wording (#4122 (comment)), and the missing xcrun in device-foldable.md (#4122 (comment)). 41 lower-priority threads also still apply, and eight new ones from this commit are not traced yet. These do not apply and can be resolved: 4431955384 (graft does not throw out of the tier), 4431955390 (readText is already in the surface-resolving list), 4431955397 and 4432325679 (reported= is the pre-cap count, so listTruncated is live), 4431955428 (the else-if is gone), 4431955430 (placed counts only display-named windows), 4431955435 (replay path is correct), 4431955437 (probe path is correct), 4431955445, 4432076445, 4432076450 and 4432325666 (rotation route fixed, retry memo no longer gates rotation), 4432076435 and 4432325683 (fixed: observedFrame is chosen.frame), 4432309422 (fixed: guard runs before the failure mapping), 4432325668 and 4432465773 (a truncated or unplaceable litDisplayWindow now refuses), 4432325690 and 4432485202 (no-panel walk stops at the first measurable window), 4432485215 (abort path is correct), 4431955402, 4432076412, 4432076421, 4432076426, 4432076431 and 4432325670 (benign), 4431955408 (rollback drops its keys), and 4432076437, 4432076439, 4432076443, 4432325687, 4432485192 and 4432485206 (already covered). All of these are under #4122 (comment).

Before merge, please fix the off-lit-panel downgrade with its test, post the Duo snapshot and readText output, and get Smoke Tests green.

@thymikee
thymikee marked this pull request as draft October 31, 2026 18:35
@thymikee
thymikee marked this pull request as ready for review October 31, 2026 19:40

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 28 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/adr/0025-foldable-apple-panels.md Resolved
@thymikee
thymikee marked this pull request as draft October 31, 2026 19:48

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 10 new findings and 1 moved to a new line on bf68d70. The verdict and the full list are in the summary comment.

@thymikee
thymikee marked this pull request as ready for review October 31, 2026 20:47

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 34 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/adr/0025-foldable-apple-panels.md Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/adr/0025-foldable-apple-panels.md Resolved

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 14 new findings and 1 moved to a new line on 04bd110. The verdict and the full list are in the summary comment.

@thymikee

Copy link
Copy Markdown
Member Author

Most of the earlier findings (#4122 (comment)) are fixed: readText now books the lit window, the old graft and resolver code is gone, and the refusal rules for unplaced windows and truncated lists hold at 04bd110. Three defects remain, and the Duo run is still missing. CI shows only the Apex Sentinel status as not passing. That status belongs to its own inline comments and does not build or run a route, so it looks unrelated. The other 18 checks pass, including Smoke Tests, but its iOS phase uses a single-panel simulator and only covers the no-panel routes. There are no conflicts. I did not run a device, the Swift runner suites, pnpm check:affected or the TS tests. The 951x669 booked frame comes from ADR 0025 and RunnerFrameFillsLitPanel, not from a run.

readTextAt still ignores the booking (RunnerTests+Interaction.swift#L359). It gathers candidates from textInputCandidatesAt(app:) and app.descendants(matching: .any), filtered only by frame.contains(point). On a Duo home screen the app tree holds windows from both panels, and both start at (0,0), so a dark-panel element can win. get text by coordinate may then succeed with the wrong label. Every element read in a lit-panel request should come only from windows proven on the lit display, the same rule the snapshot applies with windowsOnLitDisplay. Could one owner provide those windows (app.windows filtered by RunnerResolveWindowDisplayID == litPanel.displayID), with readTextAt, textInputCandidatesAt and the selector path (findElement/queryElement) all searching under it when a panel is named?

The host still fails open when the panel read is unresolved (orientation.ts#L84). After a devicectl failure, or a read cut short by the 5 s clamp, readFoldablePanelTable returns undefined and sets the 5 s back-off. resolveRunnerRequestLitPanel then returns undefined, and runner-client.ts#L41-L44 sends the command without litPanel. For the next 5 s every surface command takes the first-window route to the outer status-bar window, with no error, even though multiPanelByDevice already says the device is multi-panel. A surface request for a known multi-panel device should either name the lit panel or refuse with a typed AppError (for example the APP_SCREEN_UNRESOLVED family with a "panel table unreadable" hint). Keep the historic route only for a device whose panel count was never resolved. Please add a runner-panel.test.ts case: a resolved multi-panel read, then a failed read, then a surface command that refuses.

The live evidence the earlier review asked for is still absent for the paths in RunnerAppScreenCapture.swift#L72. It is unknown whether a Duo home-screen screenshot books a full-panel SpringBoard window within the first 8 of about 40, or refuses UnresolvedScreen. It is also unknown whether get text returns lit-panel text, and whether the published root matches the viewport. On an iPhone Duo at the fix head, please run both unfolded (lit=3, 669x951) and folded (lit=1, 466x678). (a) Run screenshot on the home screen and with Settings in front. Paste the AGENT_DEVICE_RUNNER_LIT_PANEL_WINDOW and AGENT_DEVICE_RUNNER_SCREEN_CAPTURE lines, with display equal to lit, and confirm the PNG shows the lit panel. (b) On the unfolded home screen, run get text by coordinate on an icon label inside x<466, y<678, then on a Settings row. Paste the LIT_PANEL_WINDOW line and the returned text, which must equal the visible label. (c) After the root-rect fix, run a regular snapshot (not -i) on the unfolded home screen. nodes[0].rect must equal the booked frame (0,0,951,669), and one right-column icon with x>669 must rank on-screen in find. Snapshot.swift#L220 still publishes root.rect as the native 669x951 box while the booked window is 951x669 turned, and I did not confirm what presentation does to it. (d) For one tap, paste SYNTHESIZED_DISPATCH reference= equal to that frame.

Not blocking, and fine to take or leave: the snapshot booking in RunnerTests+SnapshotExecution.swift (line 102) is a second main-thread hop right after command_preparation, so it could book inside that block and drop the extra hop; the plan narrowing at RunnerTests+SnapshotCapturePlan.swift (line 166) silently drops a preferredBackend pin, and --raw keeps both panels' windows, so either refuse or document that, and apply the display filter in the raw path or say in the ADR that raw is unfiltered; and runner-client.ts#L36 spends up to 5 s on the panel read and then sends the command with the full timeoutMs, so it should send the remaining budget.

One P1 thread still applies, on the shared panel slot being overwritten: #4122 (comment). 14 lower-priority threads still apply. The earlier threads below no longer apply and can be resolved.

Fixed at this head: #4122 (comment), #discussion_r4431929847, #discussion_r4431929864, #discussion_r4431929869, #discussion_r4431929873, #discussion_r4431929878, #discussion_r4431929881, #discussion_r4431929892, #discussion_r4431929899, #discussion_r4431929909, #discussion_r4432062431, #discussion_r4432309418, #discussion_r4432309422, #discussion_r4432309424, #discussion_r4432309434, #discussion_r4432309440, #discussion_r4432465773 (same PR; the number is the discussion id).

Graft code deleted: #discussion_r4431929859, #discussion_r4431929888, #discussion_r4431955384, #discussion_r4431955402, #discussion_r4431955408, #discussion_r4431955423, #discussion_r4432062434, #discussion_r4432076412, #discussion_r4432076421, #discussion_r4432076426, #discussion_r4432325670, #discussion_r4432325675, #discussion_r4432325696, #discussion_r4432325699, #discussion_r4432485194, #discussion_r4432747462, #discussion_r4432747466.

Old resolver, surface or tree code deleted: #discussion_r4431955410, #discussion_r4431955413, #discussion_r4431955420, #discussion_r4431955424, #discussion_r4431955428, #discussion_r4431955430, #discussion_r4432076415, #discussion_r4432076431, #discussion_r4432076435, #discussion_r4432076457, #discussion_r4432076464, #discussion_r4432325677, #discussion_r4432325683, #discussion_r4432325692, #discussion_r4432325705, #discussion_r4432325711, #discussion_r4432325717, #discussion_r4432325719, #discussion_r4432485199, #discussion_r4432485217, #discussion_r4432747436, #discussion_r4432747450, #discussion_r4432747453, #discussion_r4432747458, #discussion_r4432747460.

Back-off is not read on those paths, or abort handling is correct: #discussion_r4431955392, #discussion_r4431955445, #discussion_r4431955449, #discussion_r4432076445, #discussion_r4432076450, #discussion_r4432325666, #discussion_r4432485192, #discussion_r4432485208, #discussion_r4432485211, #discussion_r4432485215, #discussion_r4432747448, #discussion_r4433338782.

No-panel route unchanged from main, or the booking refuses before acting: #discussion_r4431929885, #discussion_r4431929905, #discussion_r4431955390, #discussion_r4431955397, #discussion_r4431955435, #discussion_r4431955437, #discussion_r4432076417, #discussion_r4432076437, #discussion_r4432076439, #discussion_r4432076460, #discussion_r4432325668, #discussion_r4432325679, #discussion_r4432325687, #discussion_r4432325690, #discussion_r4432485188, #discussion_r4432485197, #discussion_r4432485201, #discussion_r4432485202, #discussion_r4432485204, #discussion_r4432485206, #discussion_r4432485220, #discussion_r4432485223, #discussion_r4432747440, #discussion_r4432747449, #discussion_r4433338770, #discussion_r4433338775, #discussion_r4433338811, #discussion_r4433338818.

@thymikee
thymikee force-pushed the fix/foldable-springboard-surface branch from 04bd110 to dd5236e Compare November 1, 2026 18:46
thymikee and others added 15 commits November 3, 2026 13:17
…comments

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…earch

The bound-input identifier query and the in-app back control searched the
whole application, so on an unfolded Duo a same-identifier input or back
button on the dark panel could answer. Both now search the element
search's roots, and its refusal propagates: text-entry resolution throws
instead of answering nil, and boundTextEntryInputIsGone answers "cannot
prove gone". One owner returns the request's remaining budget, so the
panel read and the runner timeout share one clock. The unreachable
hasTappableFrame guard is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…trip

On the unfolded Duo, an app's window is 871x669 beside the inner panel's
status strip, and the booking used that window frame as the synthesized
dispatch reference. The orientation flip then mapped points against 871
instead of the 951-point screen, so taps landed 80 points off, and a
fill's focus tap missed. The booked frame is now the panel's box turned
like the window, at its origin, which is unchanged for full-panel
windows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… reads

The synthesized commit wait's observe closure and the replacement clear's
read swallowed a lit-panel refusal with `try?`, so the wait timed out or
the clear reported TEXT_INPUT_CLEAR_NOT_OBSERVED. Both now throw, and the
refusal answers the request. The clear read also keeps main's answer for
an input that is simply gone.

The bridge exports one lit-panel resolver that returns the window, frame
and screen, and lit-panel comments are trimmed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The booked box was inferred from the window: its origin and whether it
was wider than tall. A window that the inner panel's status strip pushes
to a non-zero origin would then book a box 80 points past the screen,
and every tap, reference and snapshot root would land off. The box is
now the panel's native portrait box laid out in the app's interface
orientation, at the screen origin, through the existing
soleScreenReferenceFrame. The XCUICoordinate anchor keeps the window's
own origin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nslation

The previous commit booked the screen box, which broke the placed-window
translation: XCTest reports a window at (0,80) from its own top-left, and
the snapshot shifts those frames into screen space only when the viewport
is that window's frame. The booking is the window frame again, as for a
placed iPad window (#3899), and on a lit-panel request the synthesized
turn is about the lit panel's screen in the interface orientation instead
of the window's shape. That replaces the aspect guess and the window
origin inference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e walk cap

The rotation-basis case now books the launched app's window under a lit
panel taller than it and checks the synthesized reference and turn box,
so a wiring regression fails it. The one-caller screen-frame helper is
inlined. The window cap's doc names the booking walk it bounds; the
element-search copy stays uncapped so every lit window is searched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… cap

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee

thymikee commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

This branch now conflicts with main. Please rebase onto main and resolve the conflicts; the review at 0684301 (collapsed-tab candidates and the overlapping dark-window case) still applies to the rebased head.

@thymikee
thymikee force-pushed the fix/foldable-springboard-surface branch from 0684301 to 4d6f523 Compare November 3, 2026 12:41
@thymikee

thymikee commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

Rebased onto main c8ad8fb1257787ce0973cf74271cb9a4fafb51d9. The range-diff preserves the existing fixes and integrates the just-merged #4157: named foldable panels supply the rotation box; unnamed single-display requests retain measured XCTest sizing, command-local caching, and typed refusals. Throwing call sites and their native tests were updated. Original worktrees were left untouched.

Fixed the remaining collapsed-tab read. collapsedTabCandidates uses the same proven-lit element-query owner as selectors, text, and Back, with the snapshot capture target's pinned display ID. Root walking needs only the display ID; the native bridge no longer takes unused panel geometry. A missing/failed root walk throws APP_SCREEN_UNRESOLVED through recursive acquisition. Each scoped query retains the existing processing-budget admission check. There is no application-wide fallback for a scoped capture.

Regression proof at rebased baseline 294c7c6acbc0764661b91bde8c85f0dfbf07bfde plus the new fixture: four assertions failed—an overlapping dark control entered candidates and the serialized capture, clearing request state lost the pinned scope, and a failed window walk did not throw. The fixed regression passes while retaining both lit controls and the unscoped application-wide behavior.

Native evidence on iPhone Duo 6D50D839-E2B0-4A85-9552-D5373AC3C760, iOS 27.1: 33 selected tests passed; the new live test initially expected labels hidden by its accessible tab-bar fixture. That test now uses the existing overlapping sibling buttons, and its targeted rerun passes with both controls in the proven-lit query and serialized capture. The cross-display failure is pinned by the mocked regression; the live fixture proves the real scoped XCTest query, not a live dark-window overlap. The live fixture closes its app in teardown. No manual device session or daemon was opened.

Fresh iOS unit-runner, iOS production, and macOS production builds passed. pnpm format and pnpm check:xctest-selection passed. Independent read-only review found no regressions in the rebase, fix, or design; the final test-only fixture adjustment was self-reviewed. The full PR remains large; this round extends the existing query owner without adding a lifecycle owner or framework.

pnpm check:affected --run --head HEAD: passed at 4d6f5231c3ee6d4c6eb646bfe6eeadc10386747c, receipt run-tJhhFZ; 11 passed, 10 explicitly skipped, one deduplicated; exactCommit=true, checkoutChanged=false.

One lease-protected push. Earlier live evidence keeps its original commit attribution. Fresh hosted coverage/native/integration checks and reviews remain required. No merge or labels.

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 4 new findings and 1 moved to a new line on 4d6f523. The verdict and the full list are in the summary comment.

@thymikee

thymikee commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

Current-review follow-up at ec80eecb1e0251d51d93aa68071c88577f1ea7ba, on top of reviewed 4d6f523.

Confirmed correctness fix: RunnerReadLitPanelCandidate treated a caught required-read exception as UnresolvedWindow, the same as a genuinely absent/unmeasurable window. The booking loop skipped it, so a later full lit window could be booked from an incomplete walk; with no later candidate, capture could take the system-surface fallback. Exceptions now return UnresolvedScreen. Genuine absence still skips the frame read and retains its existing code.

Regression proof on iPhone Duo (fixtures), iOS 27.1, 8F0B51BD-1B55-4B5E-A579-92959A027C2F: baseline 4d6f523 plus the new native fixture test fails five assertions. It feeds an unreadable frame alone, before a full lit window, and before a dark window. The fixed test passes, as do five existing controls: absent windows are not asked for frames, dark windows are excluded, ordinary lit booking works, and unreadable lists/screen identities refuse. Six native tests passed, zero failures. The iOS unit runner built successfully; pnpm format and pnpm check:xctest-selection passed. Independent read-only review found no issues. Two files, +18/-1.

The test exercises the real Objective-C exception catcher and native booking owner with fault-injection window objects on the simulator. No natural live window-read exception is claimed. No manual app session or daemon was opened; the test device was shut down and the temporary checkout removed. Logs and results were retained in the parent checkout's .tmp/pr-4122-window-read/.

Other current findings were checked against this head:

  • onScreenWindowFrame calls throwing resolveRunnerWindow, whose lit-panel branch requires the booked surface. The selector gate does not fall back to app.frame there; prepared commands book before selector resolution. The ambiguity concern describes an unprepared helper shape, not a new production dispatch route.
  • The production press-series caller passes the lit-panel wrapper for every chunk, covered by the existing multi-chunk test. The lower-level helper's callback remains its package boundary.
  • dismissKeyboard runs screenshot stability sampling inside the runner, through captureRunnerFrame, so its request needs a panel. These are not separate host screenshot requests.
  • The recording pump closes over clipLitPanel and calls captureRunnerFrameResult(..., litPanel: clipLitPanel). It does not read ambient request state after completion.
  • A partial collapsed-tab query set remains deadline-exhausted rather than being published as a complete recovery. Scope/deadline guarantees were not weakened.
  • The ADR explicitly distinguishes booking from element search: an empty proven-lit search refuses unresolvedScreen; app-off-panel capture is refused rather than swapped for SpringBoard. Unknown-display snapshot windows are deliberately excluded. Recorded issues inside a required root walk remain a refusal.
  • Screenshot's own-surface exemption is documented immediately at the gate. Backend pins are retained strings in the option projection, and the refusal chooses customActions only when the command has no explicit pin.
  • Per-read root freshness and existing wire-code literals remain unchanged; the remaining suggestions are local optimization/style requests rather than reproduced correctness defects.

No replies to Apex threads were posted. Exact-head affected-gate evidence follows below. Prior live evidence stays attributed to its original SHA.

Final exact-head validation: AGENT_DEVICE_VITEST_MAX_WORKERS=2 pnpm check:affected --run --head ec80eecb1e0251d51d93aa68071c88577f1ea7ba passed (exit 0). Receipt: .tmp/validation/run-RD6UYM/receipt.json; command log: .tmp/pr-4122-window-read/check-affected-publication.log. Receipt fields: status=passed, exactCommit=true, checkoutChanged=false, initial/final HEAD ec80eecb1e0251d51d93aa68071c88577f1ea7ba, initial/final clean status.

22 selected checks: 11 passed, 10 skipped, 1 deduplicated. Passed: format, lint, typecheck, layering, fallow, native-dupes, build, vitest-related, unit, xctest-selection, packaged-runner-swift. Related Vitest: 1,059 passed, 0 failed, 0 skipped. Unit Vitest: 17,156 passed, 0 failed, 1 skipped. No failed modules or unhandled errors. The provider-integration check was deduplicated into related tests. Coverage, native iOS/macOS, web smoke, Linux evidence and replay lanes retain their receipt skip reasons (GitHub authority or parked manual workflows); they are not local passes.

Publication uses the validated commit with an explicit lease against 4d6f5231c3ee6d4c6eb646bfe6eeadc10386747c. Hosted checks and reviews at the new head remain for CI and the parent reviewer to assess. No merge-ready claim is made.

@thymikee

thymikee commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

The code looks good at 4d6f523. The collapsed-tab read now goes through the proven-lit query owner, so the earlier query thread is fixed, and the newer inline threads on this head either no longer apply or describe behavior the ADR requires. All 19 checks pass and there are no conflicts.

The published Duo run covers the scoped query and acquisition at this head. The overlapping dark-window case is covered by a native fixture that reaches the changed production functions, but it was not reproduced live or through the daemon.

This review covers 4d6f523. The newer head ec80eec adds a refusal for an unread lit-panel window frame with a unit test, and the next pass will look at it.

@thymikee

thymikee commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

Package-growth assessment at ec80eecb1e0251d51d93aa68071c88577f1ea7ba:

  • CI Size run reports +24,097 B unpacked and approximately +7.2 kB compressed. Its uploaded base/head npmPack.entries account for the entire increase: native runner source +20,695 B, JavaScript +3,402 B. Neither package contains UnitTests, tests, or test/spec source files.
  • Independent read-only smaller-design review found no issues. Host panel admission, native display/window resolution, and request-scoped booking have distinct owners. Capture and interaction reuse the booking; selectors and text searches reuse proven-lit roots. The remaining native call-site growth propagates typed refusals. No new dependency/framework or justified smaller design was found.
  • The built runner client imports the panel reader dynamically. CLI bootstrap/help static import closures exclude runner-client, runner-panel, orientation and display-inventory; no new eager probe path was found.
  • Startup measurements remain explicitly attributed to CI: this run's help median is 70.64 ms versus base 40.00 ms. The prior 4d6f523 run measured 43.29 ms versus base 40.05 ms. The intervening patch changes only two native files; JavaScript sources and packaged JavaScript paths/sizes are unchanged. These observations do not establish the cause of the timing increase; no host-contention claim is made.

Vercel deployment is ready. No source changes were needed for these reports. Hosted checks/reviews remain pending; this is not a merge-ready claim.

@thymikee

thymikee commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

Quality and review-thread pass completed at ec80eecb1e0251d51d93aa68071c88577f1ea7ba.

Independent read-only review of the change since 4d6f523 found no issues. The two-file patch correctly separates a caught required window-read exception (UnresolvedScreen) from a genuinely absent/unmeasurable window (UnresolvedWindow), so booking cannot skip an unreadable candidate and select a later window or silently switch to another process. The fixture regression reaches the real native reader and booking loop; its previously posted red/green evidence remains attributed to that exact head.

All six current inline findings now have code-linked replies and are resolved:

  • Selector taps book before lookup; candidate checks reuse the request/bundle booking.
  • onScreenWindowFrame requires the same booked surface on a named-panel request; app.frame fallback is only on the unnamed route.
  • An incomplete collapsed-tab query set stays deadline-exhausted/truncated; returning an ordinary success array would hide the incomplete read.
  • Keyboard dismissal samples screenshots inside this runner request, so it needs the panel field.
  • App-off-panel capture refusal and exclusion of unknown-display snapshot windows are explicit ADR 0025 policies.

No source edit, push, or test rerun was warranted for these dispositions. GitHub reports MERGEABLE / CLEAN at this head: 17 checks successful, one cache-matrix check skipped, none failed or pending. The earlier exact-head affected receipt run-RD6UYM and native evidence remain unchanged. No new live-device proof is claimed.

The implementation and current review concerns have no identified blocker in this pass. Final human approval and merge remain the maintainer's decision; no merge or labels were performed.

@thymikee

thymikee commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

The code looks good at ec80eec. The new commit refuses an unread lit-panel window frame through the same booking owner, and the collapsed-tab query fix from the earlier review is still in place. All checks pass and there are no conflicts.

The inline threads on this head either no longer apply or describe behavior the ADR requires, so they can be resolved. The live evidence covers the Duo query, booking and daemon delivery, and the new refusal is covered by your native fault-injection run; a live cross-display overlap was not reproduced.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Nov 3, 2026
@thymikee
thymikee added this pull request to stack #4197 November 3, 2026 17:54
@thymikee
thymikee merged commit 0d5e5d6 into main Nov 3, 2026
18 checks passed
@thymikee
thymikee deleted the fix/foldable-springboard-surface branch November 3, 2026 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant