Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 37 additions & 9 deletions Sources/BlurtEngine/FocusCapture/FocusCapture+Editability.swift
Original file line number Diff line number Diff line change
Expand Up @@ -101,16 +101,44 @@
isBrowserBundleID(app?.bundleIdentifier)
}

/// Whether `app` is AX-opaque — an Electron editor or a web browser — where a
/// focused text field can expose no editable AX signal at all. These are the
/// one case the injector still pastes into on no signal: dropping the user's
/// words into a copy-only fallback there would be the worse mistake. The
/// accepted trade-off is a rare ⌘V beep when such an app truly has nothing
/// editable focused.
/// Bundle-identifier prefixes of native editors that draw their own text view
/// and expose no text element to Accessibility at all: with the caret in a
/// buffer, the focused element is the bare `AXWindow` — no value, no selection
/// range — or the system-wide query fails outright (`kAXErrorCannotComplete`).
/// Measured on Sublime Text 4 (build 4215), Sublime Merge (build 2132, the
/// commit-message box) and Zed 1.23.2; ⌘V lands in all three. Prefix-matched
/// so channel variants classify with their stable siblings.
private static let customDrawnEditorBundleIDPrefixes: [String] = [
"com.sublimetext.", // Sublime Text 3 + 4
"com.sublimemerge",
"dev.zed.Zed", // Zed + Preview/Nightly
]

/// Pure decision behind `isCustomDrawnEditorApp`, split out for the same reason
/// as `isBrowserBundleID`.
static func isCustomDrawnEditorBundleID(_ bundleID: String?) -> Bool {
guard let bundleID else { return false }
return customDrawnEditorBundleIDPrefixes.contains { bundleID.hasPrefix($0) }
}

/// Whether `app` is a known custom-drawn editor (Sublime Text, Sublime Merge,
/// Zed). Neither a browser nor Electron, yet just as invisible to Accessibility
/// — so without this it would read as "nothing editable focused" and every
/// dictation would land on the clipboard instead of in the buffer.
static func isCustomDrawnEditorApp(_ app: NSRunningApplication?) -> Bool {
isCustomDrawnEditorBundleID(app?.bundleIdentifier)
}

/// Whether `app` is AX-opaque — an Electron editor, a web browser, or a
/// custom-drawn editor — where a focused text field can expose no editable AX
/// signal at all. These are the one case the injector still pastes into on no
/// signal: dropping the user's words into a copy-only fallback there would be
/// the worse mistake. The accepted trade-off is a rare ⌘V beep when such an
/// app truly has nothing editable focused.
static func isAXOpaqueApp(_ app: NSRunningApplication?) -> Bool {
// Browser first: it's a string prefix check, whereas isElectronApp probes
// the disk (FileManager.fileExists) — skip that I/O for the common case.
isBrowserApp(app) || isElectronApp(app)
// String prefix checks first, whereas isElectronApp probes the disk
// (FileManager.fileExists) — skip that I/O for the common case.
isBrowserApp(app) || isCustomDrawnEditorApp(app) || isElectronApp(app)
}

/// Whether the system-wide focused element can accept pasted text right now.
Expand Down
2 changes: 1 addition & 1 deletion Sources/BlurtEngine/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ func insert(_ text: String, after priorText: String?, windowTitle: String?) asyn

`CapturedFocus` is a pid and a display name rather than an `NSRunningApplication`, so the seam is the same on every platform the engine builds for: `KeyInjector` resolves the process handle from the pid on macOS, and a host with no notion of a target app — an iPhone keyboard inserts into the field it is already attached to — receives `nil` and ignores it.

`KeyInjector.insert` **always** pastes: it saves the current pasteboard, writes the transcript, activates the captured target app, posts a synthesized ⌘V, waits for the target to read the clipboard (`pasteSettleDuration`, default 400 ms, tunable in the initializer), then restores the prior pasteboard contents. That restore never destroys what it can't put back: an unreadable pasteboard snapshots as `nil` (not as empty) and is skipped, and the replacement items are built before `clearContents()`, with the plain-string flavor as a floor when promised representations can't be materialized. There is no keystroke-by-keystroke typing path and no length threshold. If the target app is gone or nothing editable is focused it leaves the text on the clipboard and throws `.targetAppLost` / `.noEditableTarget` — which the session turns into the quiet `.noTarget` outcome. AX-opaque targets — Electron editors and web browsers, per `FocusCapture.isAXOpaqueApp` — are exempt from the editability gate and are pasted into even with no editable AX signal: web content is routinely invisible to Accessibility (Chromium builds its tree lazily; `contenteditable` composers expose no editable role), so "no signal" there usually means "AX can't see the field," not "no field," and the accepted trade-off is a rare beep over dropping the user's words to copy-only. `priorText` (the text before the caret, captured at press time) drives `InsertionSeparator.withLeadingSeparator`, which joins consecutive dictations with a space so they don't run together — a platform-free type, because a host that inserts text itself faces the same decision. When `priorText` is unreadable (an Accessibility-opaque editor, or a browser tab like Google Docs whose canvas-rendered body exposes no AX text), `InsertionSeparator.basis` falls back to what was last pasted — but only when both the target app **and** `windowTitle` match the last successful insert, so the fallback tracks "the same window," not just "the same process" (a browser hosts many unrelated tabs/documents under one PID).
`KeyInjector.insert` **always** pastes: it saves the current pasteboard, writes the transcript, activates the captured target app, posts a synthesized ⌘V, waits for the target to read the clipboard (`pasteSettleDuration`, default 400 ms, tunable in the initializer), then restores the prior pasteboard contents. That restore never destroys what it can't put back: an unreadable pasteboard snapshots as `nil` (not as empty) and is skipped, and the replacement items are built before `clearContents()`, with the plain-string flavor as a floor when promised representations can't be materialized. There is no keystroke-by-keystroke typing path and no length threshold. If the target app is gone or nothing editable is focused it leaves the text on the clipboard and throws `.targetAppLost` / `.noEditableTarget` — which the session turns into the quiet `.noTarget` outcome. AX-opaque targets — Electron editors, web browsers and custom-drawn editors like Sublime Text and Zed, per `FocusCapture.isAXOpaqueApp` — are exempt from the editability gate and are pasted into even with no editable AX signal: web content is routinely invisible to Accessibility (Chromium builds its tree lazily; `contenteditable` composers expose no editable role), so "no signal" there usually means "AX can't see the field," not "no field," and the accepted trade-off is a rare beep over dropping the user's words to copy-only. `priorText` (the text before the caret, captured at press time) drives `InsertionSeparator.withLeadingSeparator`, which joins consecutive dictations with a space so they don't run together — a platform-free type, because a host that inserts text itself faces the same decision. When `priorText` is unreadable (an Accessibility-opaque editor, or a browser tab like Google Docs whose canvas-rendered body exposes no AX text), `InsertionSeparator.basis` falls back to what was last pasted — but only when both the target app **and** `windowTitle` match the last successful insert, so the fallback tracks "the same window," not just "the same process" (a browser hosts many unrelated tabs/documents under one PID).

The session calls `setTarget` at press time with the app that was frontmost when recording started — so the paste lands where the user was, even if focus moved during transcription.

Expand Down
39 changes: 39 additions & 0 deletions Tests/BlurtEngineTests/BrowserBundleIDTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,43 @@
}
}

/// The custom-drawn-editor arm of the AX-opaque exemption: native editors like
/// Sublime Text and Zed that render their own text view and expose no AX text
/// element, so they paste on no signal just as browsers and Electron apps do.
@Suite("FocusCapture.isCustomDrawnEditorBundleID")
struct CustomDrawnEditorBundleIDTests {
@Test(
"known custom-drawn editors classify, channel variants included",
arguments: [
"com.sublimetext.4",
"com.sublimetext.3",
"com.sublimemerge",
"dev.zed.Zed",
"dev.zed.Zed-Preview",
"dev.zed.Zed-Nightly",
])
func knownEditors(bundleID: String) {
#expect(FocusCapture.isCustomDrawnEditorBundleID(bundleID))
}

@Test(
"other apps keep the copy-don't-beep fallback",
arguments: [
"com.apple.TextEdit",
"com.apple.finder",
"com.sublimetext", // the prefix carries the trailing dot
"com.microsoft.VSCode",
])
func others(bundleID: String) {
#expect(!FocusCapture.isCustomDrawnEditorBundleID(bundleID))
}

@Test("a nil bundle ID is not a custom-drawn editor")
func nilBundleID() {
#expect(!FocusCapture.isCustomDrawnEditorBundleID(nil))
}
}

/// The other half of the AX-opaque exemption: Electron detection, and the
/// `isAXOpaqueApp` disjunction the injector actually calls.
///
Expand Down Expand Up @@ -150,6 +187,7 @@
let current = NSRunningApplication.current
#expect(!FocusCapture.isBrowserApp(current))
#expect(!FocusCapture.isElectronApp(current))
#expect(!FocusCapture.isCustomDrawnEditorApp(current))
#expect(!FocusCapture.isAXOpaqueApp(current))
}

Expand All @@ -160,6 +198,7 @@
// target would be attempted anyway.
#expect(!FocusCapture.isBrowserApp(nil))
#expect(!FocusCapture.isElectronApp(nil))
#expect(!FocusCapture.isCustomDrawnEditorApp(nil))
#expect(!FocusCapture.isAXOpaqueApp(nil))
}
}
Expand Down
Loading