Skip to content
Merged
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
id: 01M6ERHPXM4ZVJSQWDFV4GWZBF
anchor: '@file'
created: 2026-11-02T18:32:34Z
norm: '1'
sig: 27abcba5e975f16e
body_hash: 26774f711766dcfe
raw_hash: 0518e62bb9b3ac20
lines: 1-516
---

Chromium exposes AXPress only on elements with their own click listener; pages that delegate clicks at the root (jsaction, frameworks) expose nothing, so a Chromium click with no pressable candidate must fall back to a routed pointer click, which Chromium accepts in the background
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
id: 01M6ERHPXTY4WPFS8YAQED9FTP
anchor: '@file'
created: 2026-11-02T18:32:34Z
norm: '1'
sig: 1574ff353918c008
body_hash: 41da1377acf8f7eb
raw_hash: e4a74ca0b8f98d06
lines: 1-130
---

a background app's context menu closes when other input arrives, so items must be pressed in the call that opened it; AXShowMenu blocks while the menu is open, so a menu seen open is a finished delivery
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ anchor: '@file'
created: 2026-10-20T12:22:04Z
norm: '1'
sig: 27abcba5e975f16e
body_hash: 672dfcdec2f063af
raw_hash: dbf52ab1720e2cc0
vouched: 2026-11-02T16:12:35Z
body_hash: 6a3d6b95132b9fb4
raw_hash: b9fc531ce6abe093
vouched: 2026-11-02T18:32:47Z
lines: 1-582
---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ anchor: '@file'
created: 2026-10-20T03:00:42Z
norm: '1'
sig: 27abcba5e975f16e
body_hash: 88fb3f87934b3c36
raw_hash: 7fdf393a2afc7183
vouched: 2026-11-02T16:12:35Z
body_hash: 99f766697f5cf49a
raw_hash: 2e2cf668fd7553e6
vouched: 2026-11-03T02:45:55Z
lines: 1-360
---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ anchor: '@file'
created: 2026-10-20T05:17:02Z
norm: '1'
sig: 27abcba5e975f16e
body_hash: 88fb3f87934b3c36
raw_hash: 7fdf393a2afc7183
vouched: 2026-11-02T16:12:35Z
body_hash: 99f766697f5cf49a
raw_hash: 2e2cf668fd7553e6
vouched: 2026-11-03T02:45:55Z
lines: 1-360
---

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
id: 01M6FH0PW0B2MSDCKFQ89Z7W8A
anchor: '@file'
created: 2026-11-03T01:40:11Z
norm: '1'
sig: 27abcba5e975f16e
body_hash: 99f766697f5cf49a
raw_hash: 2e2cf668fd7553e6
vouched: 2026-11-03T02:45:55Z
lines: 1-560
---

a routed left click into a Chromium window that isn't key in its app is dropped (right clicks still arrive); AXFocusedWindow can't tell, only the app's own AXFrontmost after prepareBackgroundWindow confirms it
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
id: 01M6FH414M25XSGQ0GX40CHS7N
anchor: '@file'
created: 2026-11-03T01:42:00Z
norm: '1'
sig: 27abcba5e975f16e
body_hash: 99f766697f5cf49a
raw_hash: 2e2cf668fd7553e6
vouched: 2026-11-03T02:45:55Z
lines: 1-560
supersedes: 01M6ERHPXM4ZVJSQWDFV4GWZBF
---

Chromium exposes AXPress only on elements with their own click listener; pages that delegate clicks at the root (jsaction, frameworks) expose nothing, so a Chromium click with no pressable candidate falls back to a routed pointer click, after making the window key in its app (see 01M6FH0PW0)
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@ anchor: '@file'
created: 2026-11-02T16:12:36Z
norm: '1'
sig: 78473a106add9cfc
body_hash: 7f06e23d5bcdb8a5
raw_hash: 038b784f6508c201
body_hash: 20a6956cc14ca9d2
raw_hash: 0fb3a60c49e77ca1
vouched: 2026-11-03T02:45:55Z
lines: 1-980
supersedes: 01M5D682W73BEYBRAPTSK672XW
---
Expand Down
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ Notable changes to Desktop Tools, newest first. Each release's GitHub release no
### Added

- Chrome and Electron web accessibility that Desktop Tools turned on is turned back off once no read of that app has relied on it for 10 minutes, so a heavy page doesn't stay slow until the app quits. `web_accessibility: false` on `desktop_inspect`, `desktop_read` or `desktop_wait` turns it off at once and reports `restored`. Web accessibility another assistive app turned on is never turned off. ([#231](https://github.com/githubnext/desktop-tools/issues/231))
- `desktop_click` with `kind: "right"` takes `menu_path`, the exact titles of a context-menu item, and opens the menu and presses that item in one call. A background app closes its context menu as soon as other input arrives, so a menu opened by one call was usually gone before the next could press an item. A right click without `menu_path` lists the menu it opened in `context_menu.items`. A wrong path closes the menu again, choosing nothing, and lists the titles found. ([#227](https://github.com/githubnext/desktop-tools/issues/227))
- `desktop_click` takes `foreground: true` for a plain single, double or triple click with the real pointer, as modifier clicks already worked: the app is brought forward, clicked, and the pointer and focus are given back, with an `unknown` result. A background single click is an Accessibility press or selection, which apps that check `NSApp.currentEvent` for user input ignore; the description now says so, and a background click that selects a row says so in its `note`. ([#225](https://github.com/githubnext/desktop-tools/issues/225))

### Fixed

Expand All @@ -16,6 +18,7 @@ Notable changes to Desktop Tools, newest first. Each release's GitHub release no
- Keys from `desktop_key` reach a SwiftUI `List` in the background. A window of an app that isn't active is made key first, without bringing the app forward (`key_window`: `made_key`, or `already_key`), since a `List`, `onDeleteCommand` and web views take some keys only in the key window. When only the window itself has keyboard focus, as after a background click selects a row, which selects it without focusing its list, the window's one list or table with a selected row is focused first (`list_focused`). Delete and Return were reported `completed` but never arrived. A window that can't be made key, or has no focused view, now gives an unknown result (`KEY_WINDOW_UNAVAILABLE`, `KEY_NO_FOCUSED_VIEW`) instead. In Chrome and other Chromium apps, the window is made key by a click on the blank part of the tab strip, since they have no standard title bar. A second preparation soon after the first no longer clicks the title bar again, which macOS took as a double-click that zoomed the window. ([#226](https://github.com/githubnext/desktop-tools/issues/226))
- `desktop_insert` into a web terminal's input, such as xterm.js in Google Cloud Shell, types the text instead of pasting it, and `route: "paste"` there is refused with `PASTE_UNVERIFIABLE_RECEIVER`. xterm clears its input after every keystroke or paste, so a paste could never be confirmed, stayed on the clipboard, and the shell could run it later. Typed text into a terminal is marked `web_terminal` and unverified, with a hint to check the screen with `ocr` and press Return with `desktop_key` rather than type again. ([#229](https://github.com/githubnext/desktop-tools/issues/229))
- A WebKit app's page, such as a WKWebView web app that isn't frontmost, shows its fields on the first `desktop_inspect`: WebKit builds the page's Accessibility tree only when first asked, so the first read found an empty group, and the read is now repeated, for up to 2 seconds, until the web area appears. Background `desktop_key` Command-V pastes into the page's focused field: the app names no focused element for a web view, so the page's focused field, which reports focus itself, is used instead, and the window is made key first. ([#5](https://github.com/githubnext/desktop-tools/issues/5))
- A background click in Chrome and other Chromium apps no longer refuses clickable page content with "No pressable accessibility element", such as a Google Cloud console link or a row's ⋮ button, or a Google Calendar settings row clicked by element ID. Chromium marks only elements with their own click listener as pressable, and pages that listen once at their root mark nothing, so where nothing at the point is pressable the click is now sent as a pointer click routed to the window, which the page handles as a real click, with a `note` saying so. Chrome drops such a click in a window that isn't key in its app, as in a Chrome that was opened in the background and never brought forward, so the window is made key first, without bringing Chrome forward, as background keys do. When that can't be done the click is refused with nothing sent, and when the window can't be confirmed key afterwards the result is `unknown`. ([#230](https://github.com/githubnext/desktop-tools/issues/230))

## [0.19.0] - 2026-11-02

Expand Down
4 changes: 3 additions & 1 deletion dist/tools.js
Original file line number Diff line number Diff line change
Expand Up @@ -613,7 +613,7 @@ export function desktopTools({ place, app }) {
{
name: "desktop_click",
op: "click",
description: `Click an element or point from a fresh desktop_inspect snapshot: kind single (default), double, right, middle or triple. Background, without moving the pointer; completed means delivered to that window, and a single click on a text field also reports whether it got focus. With modifiers (command, option, shift) it is foreground: the host activates the app and clicks with the real pointer and keys, then restores them, and the result is unknown, so inspect to check. A Control-click is kind right. Stale or unsupported targets are refused before input. ${expectNote}`,
description: `Click an element or point from a fresh desktop_inspect snapshot: kind single (default), double, right, middle or triple. Background, without moving the pointer; completed means delivered to that window, and a single click on a text field also reports whether it got focus. A background single click is an Accessibility press or selection, not a mouse event (in Chromium apps, a pointer click routed to the window where nothing is pressable), so an app that acts only on real user input may ignore it. With foreground true, or modifiers (command, option, shift), it is foreground: the host activates the app and clicks with the real pointer and keys, then restores them, and the result is unknown, so inspect to check. A Control-click is kind right. A right click lists the context_menu it opened; a background app's menu closes when other input arrives, so pass menu_path (exact titles) to choose an item in the same call. Stale or unsupported targets are refused before input. ${expectNote}`,
parameters: {
type: "object",
required: ["snapshot"],
Expand All @@ -629,6 +629,8 @@ export function desktopTools({ place, app }) {
maxItems: 3,
uniqueItems: true,
},
...foreground,
menu_path: menuPath,
expect,
},
},
Expand Down
Loading
Loading