diff --git a/.claude/agents/cleanup-reviewer.md b/.claude/agents/cleanup-reviewer.md index 3ff11c76..ca5a0440 100644 --- a/.claude/agents/cleanup-reviewer.md +++ b/.claude/agents/cleanup-reviewer.md @@ -71,6 +71,26 @@ design. Note also that `check.sh` already runs `periphery scan --strict` with `retain_public: false`, so plainly-unreferenced symbols are caught — spend your effort on what periphery cannot see. +On the iPhone (`App/BlurtiOS/`), equally deliberate: + +- **The two losing mic concepts are on purpose.** `VoiceRibs`, `VoiceStreak`, + `VoiceElementKind`'s `.ribs`/`.streak`, `SharedStore.voiceElementKind` and the + `-BlurtGalleryVoice` switch are marked TEMPORARY: three concepts are built side + by side until one is picked (`DESIGN.md` › The voice element), then the losers go + together. Don't propose deleting one piecemeal. +- **The gallery and the probe screens are entry points**, reached by launch + arguments (`-BlurtGallery…`, `-BlurtProbeField`) from `scripts/design-capture.sh`, + `ios-keyboard-shot.sh`, `ios-keyboard-flows.sh` and `apple-geometry.sh` — not + from any view. Likewise the `BlurtiOSProbe` XCUITest helpers. +- **`BlurtiOSCore` uses `package` access, not `public`,** and its explicit + `package init`s exist because a memberwise initializer is only ever internal. + Seams with one production use (`KeyboardModel.transport`, `.urlOpener`, + `.proxyOverride`, `SharedStore.override`) are there for the tests. +- **`SimulatorAudioSource` is the simulator's microphone**, not a duplicate of + `WindowedAudioSource`: `AVCaptureSession` carries no audio in the simulator. +- The iPhone periphery scan is CI's and `ios-check.sh`'s, with `--retain-public`, + so it reports the iPhone code's own dead declarations — `package` ones included. + ## Method Verify every claim by reading the real code; never report a suspicion. For an diff --git a/.claude/agents/macos-hig-reviewer.md b/.claude/agents/macos-hig-reviewer.md index f3c22b71..1514ca49 100644 --- a/.claude/agents/macos-hig-reviewer.md +++ b/.claude/agents/macos-hig-reviewer.md @@ -1,6 +1,6 @@ --- name: macos-hig-reviewer -description: Reviews SwiftUI/AppKit UI changes against macOS Human Interface Guidelines — modality, alerts vs inline validation, grouped-form structure, controls, and native idioms. Use after editing views in App/Blurt (wizard, settings, overlay). +description: Reviews SwiftUI/AppKit UI changes against macOS Human Interface Guidelines — modality, alerts vs inline validation, grouped-form structure, controls, and native idioms. Use after editing views in App/Blurt (wizard, settings, overlay). Mac only — not for App/BlurtiOS, whose views follow App/BlurtiOS/DESIGN.md. tools: Read, Grep, Glob --- diff --git a/.claude/agents/swift6-concurrency-reviewer.md b/.claude/agents/swift6-concurrency-reviewer.md index 4a2c3f15..5c5808fb 100644 --- a/.claude/agents/swift6-concurrency-reviewer.md +++ b/.claude/agents/swift6-concurrency-reviewer.md @@ -1,13 +1,17 @@ --- name: swift6-concurrency-reviewer -description: Reviews Swift changes for Swift 6 strict-concurrency correctness (actor isolation, Sendable, @MainActor, data races) and Blurt's documented audio/pipeline invariants. Use after editing engine or app code that touches actors, async, the mic capture path, or the dictation pipeline. +description: Reviews Swift changes for Swift 6 strict-concurrency correctness (actor isolation, Sendable, @MainActor, data races) and Blurt's documented audio/pipeline invariants. Use after editing engine, Mac app or iPhone (App/BlurtiOS) code that touches actors, async, the mic capture path, the dictation pipeline, or the keyboard's hand-off with the app. tools: Read, Grep, Glob, Bash --- -You are a Swift 6 strict-concurrency reviewer for Blurt, a macOS dictation -app. The engine (`Sources/BlurtEngine/`) is a `swift-tools-version:6.2` -package with **no external dependencies**; the app shell (`App/Blurt/`) is -AppKit/SwiftUI. +You are a Swift 6 strict-concurrency reviewer for Blurt, a dictation app for +macOS and iPhone. The engine (`Sources/BlurtEngine/`) is a `swift-tools-version:6.2` +package with **no external dependencies**; the Mac shell (`App/Blurt/`) is +AppKit/SwiftUI. The iPhone app (`App/BlurtiOS/`) is two processes over one App +Group — the app (listens and transcribes) and a keyboard extension (inserts the +words) — with their shared logic in the `BlurtiOSCore` static framework. Every +iPhone target defaults to the main actor (`SWIFT_DEFAULT_ACTOR_ISOLATION`), so +anything that runs elsewhere is spelled `nonisolated`. ## What to review @@ -51,6 +55,29 @@ Look at the changes (default to the working diff via `git diff` and `BlurtUITests` XCUITest bundle is the exception), and must never touch the real Keychain (`APIKeyStore`) — use an isolated service. +### On the iPhone (`App/BlurtiOS/`) + +- **The keyboard never hears anything:** no `AVFoundation`/`AVFAudio` in + `BlurtKeyboard/`, `Shared/` or `BlurtiOSCore/` (the keyboard links the core). + Capture lives only in the app (`WindowedAudioSource`, `SimulatorAudioSource`). +- **Payloads that cross processes or the capture path are `nonisolated`** — + `SharedStore`, `PhaseSnapshot`, `KeyboardCommand`, `DictationResult` and the + rest in `SharedState.swift`. Flag one that drifts back to the main-actor default. +- **System callbacks arrive off the main actor:** a `DarwinObserver` handler and + `requestSupplementaryLexicon`'s callback (an XPC queue) are `@Sendable` and hop + with `Task { @MainActor in … }`; a main-actor closure there traps at runtime. +- **The keyboard never cancels words behind the user's back:** a `.cancel` over a + recording or a transcription comes only from the × key or the term field opened + over a highlighted word. Ordering bugs between the keyboard's picture (the last + phase it read) and the app's real state are where this breaks — flag a cancel + sent on a stale `.recording`/`.processing` snapshot. +- **An automatic window close ends the session before the feed:** the release has + to find the audio, so `ListeningWindow.onClosingOnItsOwn` is awaited before + `source.close()`. +- **`BlurtiOSTests` run serially** — they swap the global `SharedStore.override`; + an async test suspended mid-await must not see another suite's. Flag a change + that re-enables parallel testing for that bundle. + ## How to report Verify before asserting: read the surrounding code, and if useful run diff --git a/.claude/settings.json b/.claude/settings.json index ad55d8c9..ac54a6ac 100644 --- a/.claude/settings.json +++ b/.claude/settings.json @@ -22,6 +22,10 @@ "Bash(scripts/check.sh --portable)", "Bash(./scripts/check.sh --portable)", "Bash(scripts/dev-build.sh)", + "Bash(scripts/ios-sim.sh*)", + "Bash(scripts/ios-test.sh*)", + "Bash(scripts/ios-check.sh*)", + "Bash(scripts/ios-typecheck.sh*)", "Bash(prettier *)", "Bash(actionlint)", "Bash(actionlint *)", diff --git a/.claude/skills/check/SKILL.md b/.claude/skills/check/SKILL.md index 26ce09b7..6a8b40da 100644 --- a/.claude/skills/check/SKILL.md +++ b/.claude/skills/check/SKILL.md @@ -123,3 +123,16 @@ hides every lint finding behind it — expect to fix a batch, not a queue. missing tool. A local run that ends `ok (UI suite + leak scan NOT run …)` is green for everything it covered — just don't claim the integration suite passed. That answer comes from `check.yml` on the PR. + +## The iPhone app + +`check.sh` covers `App/BlurtiOS` with the dependency guard, the invariants and the linters, and — +off CI, when an iPhone simulator on this Xcode's iOS runtime exists — `scripts/ios-check.sh`: the +tests with `BlurtiOSCore`'s coverage gate, `swiftlint analyze`, the probe build and periphery over +the iPhone project. Without that runtime it is skipped with a note (the closing `ok` line says so) +and CI's `ios-build` job, which runs the same script, is the authority; `check.sh` then runs +`scripts/ios-typecheck.sh` instead, which typechecks the iPhone targets against the Mac Catalyst +frameworks — the stand-in for a Mac with only the Command Line Tools. Locally, `scripts/ios-test.sh` runs `BlurtiOSTests` on a +simulator when Xcode is installed and enforces `BlurtiOSCore`'s line-coverage floor (`MIN_IOS_COVERAGE`, 88%; +`BLURT_IOS_SANITIZER=thread` or `address` runs the TSan / ASan pass CI's `ios-sanitizers` job +runs), and `scripts/ios-sim.sh` builds and launches the app there. diff --git a/.claude/skills/project-guardrails/SKILL.md b/.claude/skills/project-guardrails/SKILL.md index 6c78bbd3..cd1fd6c4 100644 --- a/.claude/skills/project-guardrails/SKILL.md +++ b/.claude/skills/project-guardrails/SKILL.md @@ -37,7 +37,7 @@ genuinely correct, and reaching for it means it's time to stop and ask. a second time inside the capture lock and held a duplicate of the whole utterance (~3.7 MB at the cap) until release. Don't reintroduce a buffered copy "just in case" — there is no retry-from-buffer path, by design. -- **Don't pre-open the mic to make presses feel faster.** `MicCapture.warmUp()` +- **(macOS shell) Don't pre-open the mic to make presses feel faster.** `MicCapture.warmUp()` is stateless on purpose — build a session, drop it — and there is no warm or prepared recorder to reuse. Measured on hardware: building a session (or the retired `prepareToRecord()`) leaves the device closed and costs ~15 ms, while @@ -129,7 +129,7 @@ only one of stt_prompt or prompt; they are the same field`, before the audio is default does not describe the behaviour. Note the plural is the documented spelling, but the singular `language_code` is accepted too (a 200, not an unknown-key 400), so both are live and `KeytermsWireTests` pins both absent. -- **Injection is always a clipboard paste** (save → write → ⌘V → settle → +- **(macOS shell) Injection is always a clipboard paste** (save → write → ⌘V → settle → restore), degrading to "left it on the clipboard" when the target is lost. No keystroke-by-keystroke typing path, no length threshold. @@ -179,3 +179,27 @@ only one of stt_prompt or prompt; they are the same field`, before the audio is hardened runtime and a **secure timestamp** (`--options runtime --timestamp`), or notarization rejects the build. `release-build.sh` re-signs frameworks for this — don't remove it. + +## iPhone (App/BlurtiOS) + +- **The keyboard never hears anything.** iOS lets no keyboard use the microphone; the app + listens and transcribes, the keyboard is a remote control over the App Group that inserts + the words. No `AVFoundation` in `BlurtKeyboard/`, `Shared/` or `BlurtiOSCore/` (the keyboard + links it) (mechanized). +- The app's microphone is a _listening window_, opened once while the app is in front and + kept open, not a per-press capture — the two rows marked "(macOS shell)" above do not apply. +- Insertion is the keyboard's `textDocumentProxy`; the clipboard is only the fallback when no + keyboard is there to take the words, and the phase then says "Copied", never "Pasted". +- **Logic lives in `BlurtiOSCore`, UI in the targets** — the Mac's engine/shell split. The + core has no SwiftUI, no views and no audio; its API is `package`, not `public` (periphery + still sees dead code); it is static, linked by both targets. It alone is coverage-gated at + 88%. A UI hook the model needs is a shell-side extension, never SwiftUI in the core. +- **The keyboard never cancels words behind the user's back.** A cancel over a recording or a + transcription comes only from the × key or the term field opened over a highlighted word — + not from leaving the screen, a swipe across the orb, or the plain +. The fuzz test enforces it. +- **No private Settings URLs** (`App-Prefs:`): `openSettingsURLString` only, and the copy names + the whole path (Apps › Blurt › Keyboards). +- **No bundled brand fonts** — removed over their licence; `BlurtType`'s roles use system faces. +- **`BlurtiOSTests` run serially** (scheme + `-parallel-testing-enabled NO`): they swap a global + `SharedStore.override`, which `.serialized` alone doesn't protect across suites. +- The full list, with every number, is `App/BlurtiOS/DESIGN.md`. diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index eb523615..36e1fe78 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -213,6 +213,96 @@ jobs: if-no-files-found: error retention-days: 14 + # The engine is a two-platform package, and `check` only ever compiles it for + # macOS: `swift test` builds for the host, and every job above runs on one. + # A macOS-only symbol that escapes its `#if os(macOS)` fence therefore passes + # `check` and breaks the iOS app, which is why this job exists and why `gate` + # treats it as required. It builds the library alone, for the simulator SDK — + # `xcodebuild` rather than `swift build`, because cross-compiling needs the + # SDK/destination handling xcodebuild owns, and it resolves the package + # straight from the checkout, no project needed. `generic/platform=iOS + # Simulator` builds against the runner's default (latest installed) simulator + # SDK without naming a device or pinning an iOS version. The scheme is the + # package's own `BlurtEngine` product scheme, so only the library builds; the + # test target stays a macOS (`swift test`) concern. + ios-build: + needs: changes + if: needs.changes.outputs.code == 'true' + runs-on: macos-26 + # The simulator build plus the unit tests take ~12 minutes on macos-26 and + # the periphery scan rebuilds after them; 15 cut the scan off mid-run. + timeout-minutes: 30 + + steps: + - name: Checkout blurt + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + path: blurt + persist-credentials: false + + - name: Build BlurtEngine for iOS + working-directory: blurt + run: | + xcodebuild -scheme BlurtEngine \ + -destination 'generic/platform=iOS Simulator' \ + -derivedDataPath "$RUNNER_TEMP/BlurtEngine-iOS" \ + build + + - name: Install XcodeGen + # Just the one formula, not `brew bundle --file=Brewfile`: the iPhone + # app's project is the only thing this job generates, and installing the + # other eleven tools would cost more than the build it precedes. xcodegen + # is in the Brewfile too, so this is the version check.sh uses. + run: brew install xcodegen + + - name: Install the iPhone gates' tools + # swiftlint for `swiftlint analyze`, periphery for the unused-code scan, + # xcbeautify for the build output; all in the Brewfile too, so these are + # the versions check.sh uses. The project itself is generated, not + # committed (people open App/Blurt's; this one is regenerated a second + # before every build by ios-test.sh, so it can never go stale). + run: brew install swiftlint periphery xcbeautify + + - name: Build the iPhone app and its keyboard, and run its quality gates + working-directory: blurt + # scripts/ios-check.sh — the iPhone's check.sh: BlurtiOSTests (warnings + # as errors, BlurtiOSCore's coverage gate) on whichever iPhone simulator + # the runner has (scripts/ios-lib.sh picks), then `swiftlint analyze` + # over that from-scratch build's log, then periphery. Ad hoc signing: the + # simulator accepts it and, unlike no signing, it embeds the App Group + # entitlement. + run: BLURT_DERIVED_DATA="$RUNNER_TEMP/BlurtiOS" scripts/ios-check.sh + + # The iPhone suite under each sanitizer, as check.sh runs the engine's. A job + # of its own, one runner per sanitizer, rather than more steps in ios-build: + # each is a full instrumented rebuild, and ios-build is already near its + # timeout. The coverage gate runs in ios-build's plain pass only. + ios-sanitizers: + needs: changes + if: needs.changes.outputs.code == 'true' + runs-on: macos-26 + timeout-minutes: 30 + strategy: + fail-fast: false + matrix: + sanitizer: [thread, address] + + steps: + - name: Checkout blurt + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + path: blurt + persist-credentials: false + + - name: Install XcodeGen and xcbeautify + run: brew install xcodegen xcbeautify + + - name: Run the iPhone unit tests under the ${{ matrix.sanitizer }} sanitizer + working-directory: blurt + env: + BLURT_IOS_SANITIZER: ${{ matrix.sanitizer }} + run: BLURT_DERIVED_DATA="$RUNNER_TEMP/BlurtiOS" scripts/ios-test.sh + # Always-runs summary job, so a *skipped* `check` can't be mistaken for a pass. # GitHub reports a skipped required check to branch protection as successful, so # anything that makes `check` skip — the `changes` filter erroring, a `needs` @@ -223,30 +313,40 @@ jobs: # NOTE: this only protects the repo once branch protection requires `gate` # instead of (or as well as) `check` — that's a repo settings change. gate: - needs: [changes, check] + needs: [changes, check, ios-build, ios-sanitizers] if: always() runs-on: ubuntu-latest steps: - - name: Assert the macOS gate ran or was intentionally skipped + - name: Assert the macOS, iOS and iOS-sanitizer gates ran or were intentionally skipped env: CHECK_RESULT: ${{ needs.check.result }} + IOS_RESULT: ${{ needs.ios-build.result }} + IOS_SANITIZERS_RESULT: ${{ needs.ios-sanitizers.result }} CHANGES_RESULT: ${{ needs.changes.result }} CODE_CHANGED: ${{ needs.changes.outputs.code }} run: | - echo "changes=$CHANGES_RESULT code=$CODE_CHANGED check=$CHECK_RESULT" + echo "changes=$CHANGES_RESULT code=$CODE_CHANGED check=$CHECK_RESULT ios-build=$IOS_RESULT ios-sanitizers=$IOS_SANITIZERS_RESULT" if [ "$CHANGES_RESULT" != "success" ]; then echo "error: the changes filter did not succeed, so the gate cannot be trusted" >&2 exit 1 fi - case "$CHECK_RESULT" in - success) echo "macOS gate passed" ;; - skipped) - if [ "$CODE_CHANGED" = "false" ]; then - echo "docs-only change; macOS gate intentionally skipped" - else - echo "error: check was skipped but code changed" >&2 - exit 1 - fi - ;; - *) echo "error: macOS gate result was '$CHECK_RESULT'" >&2; exit 1 ;; - esac + # Same rule for both platform gates: green, or skipped only because the + # change was docs-only. Written once so the iOS gate can't drift into + # a looser reading than the macOS one. + assert_gate() { + case "$2" in + success) echo "$1 gate passed" ;; + skipped) + if [ "$CODE_CHANGED" = "false" ]; then + echo "docs-only change; $1 gate intentionally skipped" + else + echo "error: $1 gate was skipped but code changed" >&2 + exit 1 + fi + ;; + *) echo "error: $1 gate result was '$2'" >&2; exit 1 ;; + esac + } + assert_gate macOS "$CHECK_RESULT" + assert_gate iOS "$IOS_RESULT" + assert_gate "iOS sanitizers" "$IOS_SANITIZERS_RESULT" diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index 0d5e9c43..1cb9fa60 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -95,6 +95,15 @@ jobs: -derivedDataPath "$DERIVED" \ -resolvePackageDependencies + # The iPhone app's project is generated, not committed (see check.yml's + # ios-build), so generate it before tracing starts. It has no remote + # packages to resolve — only the local engine. + - name: Generate the iPhone app's project (untraced) + working-directory: App/BlurtiOS + run: | + brew install xcodegen + xcodegen generate --quiet + - name: Initialize CodeQL uses: github/codeql-action/init@1c5b675653bb5c22dbe9b12b556ec555138e09fd # v4 with: @@ -122,6 +131,27 @@ jobs: CODE_SIGNING_ALLOWED=NO \ build + # The iPhone app, its keyboard and BlurtiOSCore, traced in the same + # database: CodeQL analyses whatever the job compiles between init and + # analyze, so the Mac build alone would leave App/BlurtiOS unscanned. + # Simulator, signing off — nothing here needs a device or a team. One + # architecture: CodeQL extracts every compile, and a second slice would + # compile and extract every file again for nothing new. + - name: Build Swift (iPhone) + working-directory: App/BlurtiOS + run: | + xcodebuild \ + -project BlurtiOS.xcodeproj \ + -scheme BlurtiOS \ + -configuration Debug \ + -destination 'generic/platform=iOS Simulator' \ + -derivedDataPath "$DERIVED-ios" \ + ARCHS=arm64 ONLY_ACTIVE_ARCH=YES \ + CODE_SIGN_IDENTITY="-" \ + CODE_SIGNING_REQUIRED=NO \ + CODE_SIGNING_ALLOWED=NO \ + build + - name: Perform CodeQL Analysis uses: github/codeql-action/analyze@1c5b675653bb5c22dbe9b12b556ec555138e09fd # v4 with: diff --git a/.gitignore b/.gitignore index fa6e3894..e2f3481f 100644 --- a/.gitignore +++ b/.gitignore @@ -36,3 +36,11 @@ __pycache__/ # working notes rather than repo content — quote a number into the PR instead. results*.json *.log + +# The iPhone app's project, Info.plists and entitlements are generated by +# `xcodegen generate` from App/BlurtiOS/project.yml (CI does it right before it +# builds; so does a local checkout). Nothing opens a stale copy, so none of it +# is committed — unlike App/Blurt, whose project people open in Xcode. +App/BlurtiOS/BlurtiOS.xcodeproj/ +App/BlurtiOS/*/Info.plist +App/BlurtiOS/*/*.entitlements diff --git a/.periphery.yml b/.periphery.yml index 4f9465c9..63f572bd 100644 --- a/.periphery.yml +++ b/.periphery.yml @@ -7,6 +7,15 @@ schemes: # redundant-public and fails --strict. Keep engine internals `internal`; tests # see them through `@testable import`. retain_public: false +# The iPhone app (App/BlurtiOS, its own generated project) consumes engine API +# this scan can't see — FocusedFieldContext, InsertionSeparator, +# DictationSession.HostFocusCapture — so those carry `// periphery:ignore` to +# keep their `public` off the redundant-public list. Periphery ≥ 3.5 reports an +# ignore on any referenced declaration as superfluous, even when it is there to +# suppress that result, so the superfluous-ignore check is off. Turn it back on +# once the scan covers both apps (a workspace with both projects, or a second +# scan of App/BlurtiOS in the ios-build job). +superfluous_ignore_comments: false # DictationLog.Entry's and .ErrorEntry's properties exist solely to be encoded by # JSONEncoder; Periphery can't see Codable's reflective key emission. retain_codable_properties: true diff --git a/.swiftlint.yml b/.swiftlint.yml index 7d9848e0..ed3b3834 100644 --- a/.swiftlint.yml +++ b/.swiftlint.yml @@ -13,6 +13,7 @@ included: - Tests - App/Blurt/Blurt - App/Blurt/BlurtUITests + - App/BlurtiOS excluded: - .build diff --git a/AGENTS.md b/AGENTS.md index 3048aab2..1f036d24 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -33,10 +33,11 @@ Four reflexes before you touch anything: ### The two layers -| Layer | What it is | -| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `Sources/BlurtEngine/` | Swift package (`swift-tools-version:6.2`, `platforms: [.macOS(.v15)]`) owning the pipeline. Pure logic behind protocol seams, no AppKit-shell deps, **no external SPM deps** — Foundation/Security/AVFoundation/CoreAudio plus toolchain modules like Synchronization, with AppKit types only at the seams. | -| `App/Blurt/` | AppKit/SwiftUI shell (Xcode project generated by XcodeGen) that wires the engine to an overlay window, the main window, a Settings scene, a menu bar item, and the trigger key. Its packages are the local `BlurtEngine` and Sparkle (both declared in `App/Blurt/project.yml`). | +| Layer | What it is | +| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `Sources/BlurtEngine/` | Swift package (`swift-tools-version:6.2`, `platforms: [.macOS(.v15), .iOS(.v18)]`) owning the pipeline. Pure logic behind protocol seams, no AppKit-shell deps, **no external SPM deps** — Foundation/Security/AVFoundation/CoreAudio plus toolchain modules like Synchronization, with AppKit types only at the seams. The macOS-only files (Accessibility reads, clipboard paste, CoreAudio route reads, TCC and code-signing helpers) are fenced behind `#if os(macOS)`; `check.yml`'s `ios-build` job builds the library for the iOS simulator so a symbol that escapes its fence fails the PR. | +| `App/Blurt/` | AppKit/SwiftUI shell (Xcode project generated by XcodeGen) that wires the engine to an overlay window, the main window, a Settings scene, a menu bar item, and the trigger key. Its only package is the local `BlurtEngine` (declared in `App/Blurt/project.yml`). | +| `App/BlurtiOS/` | The iPhone app (SwiftUI) and the keyboard it embeds, generated by XcodeGen from its own `project.yml` and built by `check.yml`'s `ios-build` job. The app owns the engine's pipeline over an always-on capture; the keyboard is a remote control that inserts the words. Their logic is `BlurtiOSCore`, a framework apart from the UI the way the engine is apart from the Mac shell. `App/BlurtiOS/README.md` is its guide. | The engine's dependency-free rule is a **rule**. The app carries one remote package — Sparkle, the in-place updater (see [Updates](#updates)) — allowlisted by URL in `check.sh`'s dependency guard, and @@ -54,8 +55,8 @@ Sources/BlurtEngine/ the engine (dependency-free Swift package) README.md the engine's developer guide (quick start, seams, error table) Audio/ MicCapture (+meter) over CaptureSessionRecorder (the AVCaptureSession backend), MicLiveness (mic bring-up gate), - AudioRoute (+Monitor) — the CoreAudio output route, AudioTransport - — what a transport type means, + AudioRoute (+Monitor) — the CoreAudio output route (macOS only), + AudioTransport — what a transport type means, AudioInputDevices + MicDeviceStore (microphone selection), SoundPack/Catalog/Store (the voice *descriptors*; the voices themselves are app-side) @@ -64,10 +65,16 @@ Sources/BlurtEngine/ the engine (dependency-free Swift package) Config/ Keychain-backed API key, key terms, developer mode, style profiles, DefaultsKey + PersistedSettings (every defaults key, and the reset sweep over them) - FocusCapture/ Accessibility reads of the frontmost app / focused field + FocusCapture/ FocusContext (CapturedFocus + FocusedFieldContext, every platform); + the Accessibility reads of the frontmost app / focused field + (macOS only); +Pure, the decisions that need no AX call Hotkey/ TriggerKey(+Store), DictationKeyGate, DictationKeyRouter - Injection/ KeyInjector (clipboard paste), SystemClipboard - Permissions/ PermissionsChecker (mic + Accessibility) + Injection/ InsertionSeparator (the spacing rule, every platform); KeyInjector + (clipboard paste) + SystemClipboard (macOS only); on iOS the + keyboard inserts through its textDocumentProxy (KeyboardRelayInjector) + Permissions/ PermissionStatus + SetupReadiness (every platform); + PermissionsChecker (mic + Accessibility), PermissionsReset, + SigningIdentity (macOS only) Pipeline/ DictationSession (actor) + phases, UI projections, geometry, log STT/ AssemblyAITranscriber, Conversation/TranscriptionContext, KeytermsBoost, CleanupInstruction, SyncSTTLimits @@ -79,9 +86,30 @@ App/Blurt/ Resources/Sounds/ audio they name) Shared/ UITestIdentifiers.swift — compiled into BOTH app and UI-test targets BlurtUITests/ XCUITest bundle (see Tests) +App/BlurtiOS/ the iPhone app and its keyboard (README.md there is the guide): + project.yml XcodeGen source of truth — the project is generated, NOT committed + BlurtiOSCore/ the iPhone code's logic apart from its UI, as BlurtEngine is apart + from the Mac shell: SharedState (the App Group contract), + KeyboardModel, VoiceState, KeyboardInteraction, TermPack, + KeyboardRelayInjector. A static framework both targets link; no + SwiftUI, no audio; `package` access; coverage-gated (ios-test.sh) + BlurtiOS/Sources/ the app: DictationCoordinator (composition root), ListeningWindow + + WindowedAudioSource (the always-on capture, since iOS can't open + the mic from the background), HomeView and the other screens + BlurtKeyboard/Sources/ the keyboard's views: a remote control that inserts the words; three + layouts (slim bar / panel / full keyboard) over BlurtiOSCore's model + Shared/ the UI both targets draw with (DesignTokens, type, brand), compiled + into BOTH Tests/BlurtEngineTests/ Swift Testing suites; Stubs/ holds the seam doubles scripts/ check.sh, check-portability.sh, check-invariants.sh, - bootstrap.sh, dev-build.sh, uitest.sh, leaks.sh, release*.sh + bootstrap.sh, dev-build.sh, uitest.sh, leaks.sh, release*.sh, + ios-typecheck.sh (the iPhone targets without an iOS SDK), + ios-sim.sh (build + run the iPhone app in the simulator), + ios-test.sh (the iPhone unit tests, BlurtiOSTests, on a simulator, + with BlurtiOSCore's line-coverage gate; BLURT_IOS_SANITIZER=thread|address + runs them under a sanitizer instead), + ios-check.sh (the iPhone's check.sh: ios-test, then swiftlint analyze + and periphery over App/BlurtiOS — CI's ios-build and a local check.sh) hand-run maintainer tools — no automated caller, invoked by a human, so "nothing references it" here does NOT mean dead code: screenshot.swift + beautify.swift (window imagery, capture @@ -93,7 +121,10 @@ evals/ offline decision support — the repo's only Python dictation-prompt/ DSPy harness for tuning the dictation API's cleanup instruction; check.sh lints (ruff), format-checks (ruff format), and tests it ruff.toml ruff config for the above — the repo's only Python config -.github/workflows/ check.yml (the gate, macos-26), +.github/workflows/ check.yml (the gate, macos-26; its ios-build job builds the engine for the + iOS simulator, then the iPhone app with its keyboard, and runs + BlurtiOSTests on a simulator with its coverage gate; ios-sanitizers + runs them again under TSan and ASan) release-bump.yml (version bump) + release.yml (sign + publish), codeql.yml .claude/ Claude Code hooks, skills, subagents (see CLAUDE.md) @@ -178,11 +209,29 @@ the two checks that genuinely need the build products stay behind them (step 12) 12. **`swiftlint analyze`** (unused imports — reads the compiler log the app build captured) and **`periphery scan --strict`** (unused declarations — runs its own xcodebuild + index). The only two checks that can't move earlier. +13. **`scripts/ios-check.sh`** — the same gates for `App/BlurtiOS`: `BlurtiOSTests` with + `BlurtiOSCore`'s coverage gate, then `swiftlint analyze` over that build's log and periphery + over the iPhone project. Locally only, and only when an iPhone simulator on this Xcode's iOS + runtime exists (`xcodebuild -downloadPlatform iOS`); otherwise skipped with a note in the + closing `ok` line. On CI the `ios-build` job runs it instead. CI (`.github/workflows/check.yml`) installs all of these via Homebrew on `macos-26` and runs the same -script, so a clean local `check.sh` matches CI by construction. Two sibling jobs exist purely to -shorten the loop for whoever is editing Swift without a toolchain, and **neither is a required -check** — both can only go red where `check` would too: +script, so a clean local `check.sh` matches CI by construction. One more job **is** required, because +`check.sh` cannot cover it: **`ios-build`** compiles the engine for the iOS simulator +(`xcodebuild -scheme BlurtEngine -destination 'generic/platform=iOS Simulator'`), which is the only +place a macOS-only symbol that escaped its `#if os(macOS)` fence gets caught — `swift test` builds +for the host, and every other job runs on a Mac. It then runs **`scripts/ios-check.sh`**, which holds +the iPhone code to the bar the engine and the Mac shell meet in `check.sh`: `BlurtiOSTests` with +warnings as errors and a **line-coverage gate** on `BlurtiOSCore`, the iPhone code's logic +(≥ `MIN_IOS_COVERAGE` in `scripts/ios-test.sh`, the engine's 88%; raise it as coverage grows), then +**`swiftlint analyze`** over that from-scratch build's log and **periphery** over the iPhone +project; `check.sh`'s swift-format, `swiftlint lint` and invariants already read every Swift file +there. In the sibling +**`ios-sanitizers`** job, one runner per sanitizer — **ThreadSanitizer** and **AddressSanitizer** +passes run the same tests. `codeql.yml`'s Swift analysis builds the iPhone app as well as the Mac +one, so both are in its database. `gate` fails unless all of them are green (or skipped for a docs-only change). Two sibling jobs exist purely to shorten the loop for whoever is +editing Swift without a toolchain, and **neither is a required check** — both can only go red where +`check` would too: - **`compile`** — `swift build --build-tests -Xswiftc -warnings-as-errors` and nothing else. A test target that doesn't build is the most common way a PR here goes red, and this reports it in ~2 @@ -344,28 +393,34 @@ a rule that outlives its row keeps firing and keeps citing this table while enfo project has already reversed. Reworded the row? Update the anchor. Reversed the decision? Delete the rule along with the row. -| Don't | Because | -| --------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Add an external SPM dependency to the engine | Dependency-free by rule (biggest supply-chain risk); a `check.sh` guard fails on `.package(` in `Package.swift` or a `url:`/`github:` package in `project.yml`. Extend `BlurtEngine` instead. | -| Use `AVAudioEngine` / `installTap` for capture | A long-lived engine bound its input graph to one device and went stale on a mic↔built-in switch — `-10868` (`kAudioUnitErr_FormatNotSupported`) or all-zero buffers. `MicCapture` builds a fresh `AVCaptureSession` recorder per capture (owner-directed move from `AVAudioRecorder`, 2026-08-25). | -| Pre-open the mic to shave bring-up latency (a warm/prepared recorder) | Measured against `kAudioDevicePropertyDeviceIsRunningSomewhere`: neither building an `AVCaptureSession` nor the retired `AVAudioRecorder.prepareToRecord()` opens the device, so neither pre-pays the 180–600 ms route activation `record()` costs. A warm-recorder lifecycle bought ~15 ms and cost a device-identity check, a pin check, an expiry and a bring-up flag. `MicCapture.warmUp()` is stateless: it absorbs the process's first-touch cost and holds nothing. | -| Add streaming STT | The dictation API returns the full (already rewritten) text in one response; the overlay shows "Transcribing…" then the full text. The one WebSocket allowed is the experimental read-aloud feature's streaming **TTS**, confined to `Sources/BlurtEngine/TTS/AssemblyAISpeechSynthesizer.swift` (the invariant gate exempts only that file). | -| Call the dictation API in a way its docs don't describe | **The reference is the source of truth.** Every path, header, multipart part, `config` key, response key and error key Blurt sends or reads is one it documents. The route has been renamed repeatedly and still answers to names the docs have dropped — `prompt`, `keyterms`, `word_boost`, `language_code`, `llm: null` — so a 200 is not evidence a field is supported, only that it has not been removed yet; treat anything absent from the docs as deprecated. Undocumented behaviour carries no compatibility promise, so a feature resting on it can stop working silently: that is why `llm: null` (the only rewrite off switch there is) and the `message` error field were dropped on 2026-09-11, along with `warmUp()`'s GET at the bare host root in favour of the documented `GET /warm`. Running tighter than a documented cap is fine — `STTPrompt.characterCap` does. Measuring is still how a claim gets settled, and the docs have been wrong twice (see `DictationWireTypes` on unknown fields, `STTPrompt` on the 6000 cap), but a measurement licenses _distrusting_ a documented field, never _sending_ an undocumented one. | -| Add a client-side LLM cleanup pass | Cleanup is the dictation API's server-side rewrite, requested by `config.llm_instruction` on the same `/v1/transcribe/live` call. No LLM Gateway client, no `StylerProtocol`, no styling stage, no second request — transcription steering belongs in `STTPrompt`. | -| Add local models or model downloads | Transcription is a remote AssemblyAI call: no on-device ASR/LLM, no model cache, no download UI. | -| Pin transcription to English, or set a language at all | Hurt non-English transcription; language is left to the model's own detection. **No `config.language_codes`** either — note the plural, which is the documented spelling, though the undocumented singular `language_code` is accepted too (a 200, not the unknown-key 400 a bogus name earns), so both are live and both are pinned absent. Detection was measured to work with neither it nor a prompt set (es/fr/de/ja clips each transcribed in their own language against the live endpoint, rewrite included), and re-measured 2026-09-11: a Spanish clip came back as Spanish with no field, with `language_codes: ["es"]`, **and** with `language_codes: ["en"]` — asking for English did not force English, so the reference's `["en"]` default does not describe this route and omission is not a bet on a default. Setting one would only take working detection away. `KeytermsWireTests` asserts the absence of both spellings. | -| Send `config.prompt` alongside `config.stt_prompt` | They are the same field under two names — the docs give `stt_prompt` as the field and note it is "Also accepted as `prompt`" — and a request carrying both is rejected before the audio is read (`provide only one of stt_prompt or prompt; they are the same field`). So this is never a compatible addition: it is a 400 on _every_ dictation. `stt_prompt` is the documented spelling and the only one to send; `prompt` is the legacy one, and the route's own inner validator still answers under it (see `STTPrompt.characterCap`), which is a reason to treat it as deprecated rather than interchangeable. | -| Bring back `config.conversation_context` | Replaced by `config.stt_prompt` (`STTPrompt`), which carries the same two signals as one string. Both fields still work and can ride the same request, so re-adding the turn list sends the prior text twice rather than failing. `prompt` is `stt_prompt`'s own alias — a request carrying both is a 400 (`they are the same field`) — and a custom prompt replaces the service's managed default, which is the trade this swap accepted. | -| Widen the request's context past history + the prior chunk | `STTPrompt.text` reads exactly two fields of `TranscriptionContext` — `recentTranscripts` and `priorText`. The app name, window title, field label and selected text are captured for the paste path and the developer-mode log and stay on the machine; the app/window/field hints and the `Selected text:` block were removed, not gated. Don't add one back, and don't route that context onto the request by another path. The user's key terms are the exception that proves the rule: they _are_ sent, as the request's own `keyterms_prompt` list (`KeytermsBoost`) — never folded back into the prompt. | -| Add a "remove filler words (um, uh, like)" clause | Not in the STT model's trained instruction set — a no-op, deliberately dropped; disfluency removal is the server-side LLM rewrite's job. | -| Add a keystroke-typing paste path or a length threshold | Injection is **always** clipboard paste (save → write → ⌘V → settle → restore), with the copied-to-clipboard degradation when the target is lost. | -| Add `LSUIElement` or a menu-bar-**only** mode | Blurt is a Dock app first. The `MenuBarExtra` status item is convenience layered on the Dock icon; the notch can hide a status item, so nothing may depend on it. A menu-bar-only variant was reverted twice. | -| Add a `KeyboardShortcuts` package or a key+modifier chord | The trigger is a single lone modifier, home-grown (`CGEventTap` + `DictationKeyGate`), and swallows nothing. | -| Hand-roll an updater beside Sparkle, or start it in a dev build | Updates go through Sparkle (`UpdaterModel`, app-side; EdDSA-signed appcast published per release). The download-only GitHub-API checker it replaced and the earlier `mxcl/AppUpdater` are gone — don't add a second update path, and keep update code out of the engine. Only the shipping bundle id starts the updater: a dev build must never replace itself with the release. | -| Hand-edit `Blurt.xcodeproj/project.pbxproj` | Generated from `project.yml`; `check.sh`'s drift check fails on any manual edit (a Claude PreToolUse hook also blocks it). | -| Redirect the post-build install away from `/Applications` | TCC won't register apps in DerivedData/`/tmp`, so permission toggles never appear. | -| Touch the real Keychain in tests | `APIKeyStore` is the production item — a test that writes it triggers Keychain prompts and corrupts the real item's ACL. Use an isolated service (see `KeychainStoreTests`) or `InMemoryAPIKeyStore`. | -| Add backwards-compat shims for removed types | Deleted types stay deleted — no deprecated re-exports. | +| Don't | Because | +| ------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Add an external SPM dependency to the engine | Dependency-free by rule (biggest supply-chain risk); a `check.sh` guard fails on `.package(` in `Package.swift` or a `url:`/`github:` package in `project.yml`. Extend `BlurtEngine` instead. | +| Use `AVAudioEngine` / `installTap` for capture | A long-lived engine bound its input graph to one device and went stale on a mic↔built-in switch — `-10868` (`kAudioUnitErr_FormatNotSupported`) or all-zero buffers. `MicCapture` builds a fresh `AVCaptureSession` recorder per capture (owner-directed move from `AVAudioRecorder`, 2026-08-25). | +| Pre-open the mic to shave bring-up latency (a warm/prepared recorder) | Measured against `kAudioDevicePropertyDeviceIsRunningSomewhere`: neither building an `AVCaptureSession` nor the retired `AVAudioRecorder.prepareToRecord()` opens the device, so neither pre-pays the 180–600 ms route activation `record()` costs. A warm-recorder lifecycle bought ~15 ms and cost a device-identity check, a pin check, an expiry and a bring-up flag. `MicCapture.warmUp()` is stateless: it absorbs the process's first-touch cost and holds nothing. | +| Add streaming STT | The dictation API returns the full (already rewritten) text in one response; the overlay shows "Transcribing…" then the full text. The one WebSocket allowed is the experimental read-aloud feature's streaming **TTS**, confined to `Sources/BlurtEngine/TTS/AssemblyAISpeechSynthesizer.swift` (the invariant gate exempts only that file). | +| Call the dictation API in a way its docs don't describe | **The reference is the source of truth.** Every path, header, multipart part, `config` key, response key and error key Blurt sends or reads is one it documents. The route has been renamed repeatedly and still answers to names the docs have dropped — `prompt`, `keyterms`, `word_boost`, `language_code`, `llm: null` — so a 200 is not evidence a field is supported, only that it has not been removed yet; treat anything absent from the docs as deprecated. Undocumented behaviour carries no compatibility promise, so a feature resting on it can stop working silently: that is why `llm: null` (the only rewrite off switch there is) and the `message` error field were dropped on 2026-09-11, along with `warmUp()`'s GET at the bare host root in favour of the documented `GET /warm`. Running tighter than a documented cap is fine — `STTPrompt.characterCap` does. Measuring is still how a claim gets settled, and the docs have been wrong twice (see `DictationWireTypes` on unknown fields, `STTPrompt` on the 6000 cap), but a measurement licenses _distrusting_ a documented field, never _sending_ an undocumented one. | +| Add a client-side LLM cleanup pass | Cleanup is the dictation API's server-side rewrite, requested by `config.llm_instruction` on the same `/v1/transcribe/live` call. No LLM Gateway client, no `StylerProtocol`, no styling stage, no second request — transcription steering belongs in `STTPrompt`. | +| Add local models or model downloads | Transcription is a remote AssemblyAI call: no on-device ASR/LLM, no model cache, no download UI. | +| Pin transcription to English, or set a language at all | Hurt non-English transcription; language is left to the model's own detection. **No `config.language_codes`** either — note the plural, which is the documented spelling, though the undocumented singular `language_code` is accepted too (a 200, not the unknown-key 400 a bogus name earns), so both are live and both are pinned absent. Detection was measured to work with neither it nor a prompt set (es/fr/de/ja clips each transcribed in their own language against the live endpoint, rewrite included), and re-measured 2026-09-11: a Spanish clip came back as Spanish with no field, with `language_codes: ["es"]`, **and** with `language_codes: ["en"]` — asking for English did not force English, so the reference's `["en"]` default does not describe this route and omission is not a bet on a default. Setting one would only take working detection away. `KeytermsWireTests` asserts the absence of both spellings. | +| Send `config.prompt` alongside `config.stt_prompt` | They are the same field under two names — the docs give `stt_prompt` as the field and note it is "Also accepted as `prompt`" — and a request carrying both is rejected before the audio is read (`provide only one of stt_prompt or prompt; they are the same field`). So this is never a compatible addition: it is a 400 on _every_ dictation. `stt_prompt` is the documented spelling and the only one to send; `prompt` is the legacy one, and the route's own inner validator still answers under it (see `STTPrompt.characterCap`), which is a reason to treat it as deprecated rather than interchangeable. | +| Bring back `config.conversation_context` | Replaced by `config.stt_prompt` (`STTPrompt`), which carries the same two signals as one string. Both fields still work and can ride the same request, so re-adding the turn list sends the prior text twice rather than failing. `prompt` is `stt_prompt`'s own alias — a request carrying both is a 400 (`they are the same field`) — and a custom prompt replaces the service's managed default, which is the trade this swap accepted. | +| Widen the request's context past history + the prior chunk | `STTPrompt.text` reads exactly two fields of `TranscriptionContext` — `recentTranscripts` and `priorText`. The app name, window title, field label and selected text are captured for the paste path and the developer-mode log and stay on the machine; the app/window/field hints and the `Selected text:` block were removed, not gated. Don't add one back, and don't route that context onto the request by another path. The user's key terms are the exception that proves the rule: they _are_ sent, as the request's own `keyterms_prompt` list (`KeytermsBoost`) — never folded back into the prompt. | +| Add a "remove filler words (um, uh, like)" clause | Not in the STT model's trained instruction set — a no-op, deliberately dropped; disfluency removal is the server-side LLM rewrite's job. | +| Add a keystroke-typing paste path or a length threshold | Injection is **always** clipboard paste (save → write → ⌘V → settle → restore), with the copied-to-clipboard degradation when the target is lost. | +| Add `LSUIElement` or a menu-bar-**only** mode | Blurt is a Dock app first. The `MenuBarExtra` status item is convenience layered on the Dock icon; the notch can hide a status item, so nothing may depend on it. A menu-bar-only variant was reverted twice. | +| Add a `KeyboardShortcuts` package or a key+modifier chord | The trigger is a single lone modifier, home-grown (`CGEventTap` + `DictationKeyGate`), and swallows nothing. | +| Hand-roll an updater beside Sparkle, or start it in a dev build | Updates go through Sparkle (`UpdaterModel`, app-side; EdDSA-signed appcast published per release). The download-only GitHub-API checker it replaced and the earlier `mxcl/AppUpdater` are gone — don't add a second update path, and keep update code out of the engine. Only the shipping bundle id starts the updater: a dev build must never replace itself with the release. | +| Hand-edit `Blurt.xcodeproj/project.pbxproj` | Generated from `project.yml`; `check.sh`'s drift check fails on any manual edit (a Claude PreToolUse hook also blocks it). | +| Redirect the post-build install away from `/Applications` | TCC won't register apps in DerivedData/`/tmp`, so permission toggles never appear. | +| Touch the real Keychain in tests | `APIKeyStore` is the production item — a test that writes it triggers Keychain prompts and corrupts the real item's ACL. Use an isolated service (see `KeychainStoreTests`) or `InMemoryAPIKeyStore`. | +| Add backwards-compat shims for removed types | Deleted types stay deleted — no deprecated re-exports. | +| Give the keyboard extension a microphone or any audio type | iOS lets no keyboard use the microphone, and the design keeps every audio type out of it: the app listens and transcribes over its always-on capture, the keyboard is a remote control that inserts the words (`KeyboardRelayInjector`). An `AVFoundation` import in `App/BlurtiOS/BlurtKeyboard`, `Shared` or `BlurtiOSCore` (which the keyboard links) is that decision being reversed by accident. | +| Move the iPhone's logic back into its UI targets, or views into `BlurtiOSCore` | The iPhone code splits the way the Mac's does: `App/BlurtiOS/BlurtiOSCore` is the logic (the App Group contract, `KeyboardModel`, `VoiceState`, `KeyboardInteraction`, term packs, the relay injector) — no SwiftUI, no views, no audio, `package` access, a static framework both targets link — and it alone carries the 88% coverage gate (`scripts/ios-test.sh`), as the engine alone does on the Mac. Logic in a view file escapes that gate; SwiftUI in the core drags the views' colours and tokens into a module the keyboard's memory budget pays for. A UI hook the model needs becomes a shell-side extension (`KeyboardModel+Palette.swift`). | +| Deep-link into Settings › General › Keyboard (`App-Prefs:`, `prefs:`) | Private URLs: App Review rejects them (2.5.1) and they break between iOS releases. `UIApplication.openSettingsURLString` — Blurt's own page, whose Keyboards row iOS adds for any app with a keyboard — is the only Settings link; the setup step names the whole path (Apps › Blurt › Keyboards) because some builds (the iOS 26 simulator) open Settings at its top level instead. | +| Bundle the brand's fonts in the iPhone app | Modern Gothic Mono, UN 11ST and Oceanic Text were bundled and removed: their licence covers the web, not an app in a public MIT repo. `BlurtType`'s roles (`mono`, `heading`, `body`) map to the system's monospaced, serif and default faces; a face comes back only with written app-embedding rights, through the same roles. | + +**Platform scope.** Two rows above are the Mac shell's, not the iPhone's, and the iPhone app diverges from them on purpose: "no warm recorder" — on iOS the app keeps the microphone open for a listening window (`App/BlurtiOS/BlurtiOS/Sources/AudioSource.swift`), because iOS refuses to open it from the background and the keyboard, which is where a dictation starts, is in the background of whatever app the user is typing in; and "injection is always a clipboard paste" — on iOS the keyboard inserts through its `textDocumentProxy`, and the clipboard is only the degradation when no keyboard is there to take the words. Neither changes the engine: `MicCapture` and `KeyInjector` stay as they are and are macOS-only; the iOS app supplies its own `MicCaptureProtocol` and `InjectorProtocol` conformances. `App/BlurtiOS/DESIGN.md` is the iPhone's own list. Release-side invariants (hardened runtime and a secure timestamp on every nested mach-o and embedded framework, or notarization rejects the build; roll-forward-only for a bad release) live in @@ -735,8 +790,9 @@ those degrades to the plain-text floor rather than a byte-faithful round trip. It first activates the captured target app; if that app has terminated or won't activate, it leaves the transcript on the clipboard and throws `.targetAppLost`, which the pipeline degrades to the quiet "copied" notice (`.noTarget`) rather than a red failure. Consecutive dictations are joined with a -leading separator space via `withLeadingSeparator`; when the prior text is unreadable, the separator -basis falls back to what was last pasted — but only when the target app **and** window title both +leading separator space via `InsertionSeparator.withLeadingSeparator` (its own platform-free type, +because a host that inserts text itself faces the same decision); when the prior text is unreadable, +the separator basis falls back to what was last pasted — but only when the target app **and** window title both match the last successful insert, so the fallback tracks "the same window", not just "the same process" (a browser hosts many unrelated tabs under one PID). diff --git a/App/Blurt/Blurt/AppIcon.icon/Assets/blurt-wordmark.png b/App/Blurt/Blurt/AppIcon.icon/Assets/blurt-wordmark.png index e2879229..8bf85a45 100644 Binary files a/App/Blurt/Blurt/AppIcon.icon/Assets/blurt-wordmark.png and b/App/Blurt/Blurt/AppIcon.icon/Assets/blurt-wordmark.png differ diff --git a/App/Blurt/Blurt/UITestSupport.swift b/App/Blurt/Blurt/UITestSupport.swift index 72525050..22b3e4f4 100644 --- a/App/Blurt/Blurt/UITestSupport.swift +++ b/App/Blurt/Blurt/UITestSupport.swift @@ -108,7 +108,7 @@ /// target app). The harness window plays the role of "the app being pasted /// into", surfacing the recorded text for the test to assert on. nonisolated struct UITestInjector: InjectorProtocol { - func setTargetApp(_ app: NSRunningApplication?) async {} + func setTarget(_ focus: CapturedFocus?) async {} func insert(_ text: String, after priorText: String?, windowTitle: String?) async throws { await MainActor.run { UITestState.shared.recordPaste(text) } } diff --git a/App/BlurtiOS/BlurtKeyboard/PrivacyInfo.xcprivacy b/App/BlurtiOS/BlurtKeyboard/PrivacyInfo.xcprivacy new file mode 100644 index 00000000..3ae44d86 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/PrivacyInfo.xcprivacy @@ -0,0 +1,24 @@ + + + + + NSPrivacyTracking + + NSPrivacyTrackingDomains + + NSPrivacyAccessedAPITypes + + + NSPrivacyAccessedAPIType + NSPrivacyAccessedAPICategoryUserDefaults + NSPrivacyAccessedAPITypeReasons + + + 1C8F.1 + + CA92.1 + + + + + diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/AddTermKey.swift b/App/BlurtiOS/BlurtKeyboard/Sources/AddTermKey.swift new file mode 100644 index 00000000..2db8dc35 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/AddTermKey.swift @@ -0,0 +1,104 @@ +import BlurtiOSCore +import SwiftUI + +/// The small + beside the mic: a new key term. A bare glyph, quiet until it +/// is needed; a check for a moment after one was saved. +/// +/// Highlight a word in the text and the + is a chip instead — the field's +/// shape, holding the word — and one tap adds the word as it stands: + for a +/// word Blurt doesn't know yet, ✓ once it does. After the check the + comes +/// back, highlight or not; a tap on a ✓ chip puts it back at once. Holding +/// the chip opens the field pre-filled instead, to fix the spelling first +/// (`KeyboardModel`: `selectedTerm`, `addSelectedTerm`, `beginAddingTerm`). +/// The row it hangs in (`VoiceRow`) decides how wide the chip may grow. +struct AddTermKey: View { + var model: KeyboardModel + @Environment(\.keyboardPalette) private var palette + /// Lit while a finger is on the chip. A gesture state, so a touch the + /// system takes away (a call, the keyboard dismissed mid-press) unlights it + /// too; a plain state would stay lit. + @GestureState private var pressed = false + @State private var held = false + + var body: some View { + ZStack { + if let term = model.selectedTerm { + chip(term).transition(.opacity) + } else { + plain.transition(.opacity) + } + } + .animation(.easeInOut(duration: DesignTokens.Motion.stateFade), value: model.selectedTerm == nil) + .disabled(!model.hasFullAccess) + .opacity(model.hasFullAccess ? 1 : DesignTokens.Metrics.opacityDisabled) + } + + private var saved: Bool { model.termSavedAt != nil } + + /// The bare +: a new term typed from nothing (or from the selection, seeded). + private var plain: some View { + GlyphKey(symbol: saved ? "checkmark" : "plus", tint: saved ? palette.signal : mutedTint, label: "Add a key term") { + model.beginAddingTerm() + } + } + + /// The chip: the highlighted word, ready to add with one tap. + private func chip(_ term: String) -> some View { + let known = model.selectedTermIsKnown + return HStack(spacing: DesignTokens.Metrics.addtermChipGap) { + // Green while there is something to do, and for the moment after it + // was done; quiet once the word is simply known. + glyph(known ? "checkmark" : "plus", tint: known && !saved ? mutedTint : palette.signal) + Text(term) + .font(.system(size: DesignTokens.Typography.sizeTerm, weight: DesignTokens.Typography.weightTerm)) + .foregroundStyle(palette.keyText) + .lineLimit(1) + .truncationMode(.tail) + } + .padding(.horizontal, DesignTokens.Metrics.addtermChipPad) + .frame(height: DesignTokens.Metrics.termHeight) + .background( + RoundedRectangle(cornerRadius: DesignTokens.Metrics.termRadius) + .fill(palette.field) + .strokeBorder(palette.fieldBorder, lineWidth: DesignTokens.Metrics.termBorder) + ) + .brightness(pressed ? DesignTokens.Metrics.opacityPressBrighten : 0) + .animation(.easeOut(duration: DesignTokens.Motion.keyPress), value: pressed) + .contentShape(Rectangle()) + .accessibilityElement(children: .ignore) + .accessibilityLabel(known ? "“\(term)” is a key term" : "Add “\(term)” as a key term") + .accessibilityHint(known ? "Tap to put the + back" : "Hold to edit it first") + .accessibilityAddTraits(.isButton) + .accessibilityIdentifier("blurt-add-selected") + // A hold opens the field with the word in it. The system's long press: + // it fails on its own when the finger travels or the touch is taken + // away, so no timer of ours can open the field after the finger is gone. + .simultaneousGesture( + LongPressGesture(minimumDuration: DesignTokens.Motion.chipHold, maximumDistance: KeyboardInteraction.tapTravel) + .onEnded { _ in + held = true + model.beginAddingTerm() + } + ) + // A tap adds; a touch that travelled (the panel's carousel), or one the + // hold already answered, does nothing. + .simultaneousGesture( + DragGesture(minimumDistance: 0) + .updating($pressed) { _, state, _ in state = true } + .onEnded { value in + defer { held = false } + guard !held, KeyboardInteraction.isTap(value.translation) else { return } + model.addSelectedTerm() + } + ) + } + + private var mutedTint: Color { palette.keyText.opacity(DesignTokens.Metrics.opacityLegendMuted) } + + private func glyph(_ symbol: String, tint: Color) -> some View { + Image(systemName: symbol) + .font(.system(size: DesignTokens.Typography.sizeGlyph, weight: DesignTokens.Typography.weightGlyph)) + .foregroundStyle(tint) + .contentTransition(.symbolEffect(.replace)) + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/FullKeyboardView.swift b/App/BlurtiOS/BlurtKeyboard/Sources/FullKeyboardView.swift new file mode 100644 index 00000000..66b7b935 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/FullKeyboardView.swift @@ -0,0 +1,141 @@ +import BlurtiOSCore +import SwiftUI + +/// A complete keyboard laid out as the iPhone's own — ten letter keys across +/// at one width, the middle row centred, shift and delete flanking the bottom +/// letters, then 123 · globe · space · return, at the system keyboard's +/// measured spacing (`KeyGeometry`) — with the voice bar where the suggestion +/// bar would be. Letters pop up while pressed, sentences capitalise +/// themselves, a double space ends one. 123 opens the numbers and symbols, +/// #+= the rest of them, as the system keyboard's pages go. +/// No autocorrect or suggestions yet. iOS swaps in its own keyboard for +/// password fields, so those never reach here. +struct FullKeyboardView: View { + var model: KeyboardModel + + private static let symbols = ["1234567890", "-/:;()$&@\"", ".,?!'"] + private static let more = ["[]{}#%^*+=", "_\\|~<>€£¥•", ".,?!'"] + + var body: some View { + GeometryReader { geo in + // The root has already taken the side margins off; the rest is the row. + let geometry = KeyGeometry(rowWidth: geo.size.width) + let gap = geometry.gap + VStack(spacing: 0) { + // The voice row sits where the system's suggestion bar does, and the + // first key row starts right under it, as the iPhone's does. + VoiceBar(model: model) + VStack(spacing: KeyboardPalette.rowGap) { + ForEach(Array(rows.enumerated()), id: \.offset) { index, row in + let third = index == 2 // literal-ok: the third row + // The third row's keys fill what shift and delete leave: seven + // letters at the letter width, or the five punctuation keys, wider, + // as the system keyboard has them on its symbol pages. + let keyWidth = third && model.symbolsPage ? geometry.punctuationWidth : geometry.letterWidth + HStack(spacing: third ? geometry.sideGap : gap) { + if third { modifierKey(width: geometry.sideWidth) } + HStack(spacing: gap) { + ForEach(Array(row), id: \.self) { character in + LetterKey(label: label(for: character), width: keyWidth) { + model.type(label(for: character)) + } + } + } + if third { + KeyCap(systemImage: "delete.left", dark: true, width: geometry.sideWidth) { model.deleteBackward() } + } + } + .frame(maxWidth: .infinity) + } + // The stock bottom row; voice lives only in the bar above. + HStack(spacing: gap) { + KeyCap(title: model.symbolsPage ? "ABC" : "123", dark: true, width: geometry.abcWidth) { + model.toggleSymbols() + } + if model.needsGlobe { + KeyCap(systemImage: "globe", dark: true, width: geometry.abcWidth) { model.globe() } + } + KeyCap(title: "space", flexible: true) { model.space() } + KeyCap(title: model.returnLabel ?? "return", dark: true, width: geometry.returnWidth) { model.newline() } + } + } + } + } + } + + private var rows: [String] { + guard model.symbolsPage else { return model.letterRows } + return model.morePage ? Self.more : Self.symbols + } + + private func label(for character: Character) -> String { + let text = String(character) + return model.shifted && !model.symbolsPage ? text.uppercased() : text + } + + /// Shift on the letters; on the symbol pages the key that swaps between + /// the two of them, labelled with where it goes, as the iPhone's is. + @ViewBuilder private func modifierKey(width: CGFloat) -> some View { + if model.symbolsPage { + KeyCap(title: model.morePage ? "123" : "#+=", dark: true, width: width) { model.toggleMore() } + } else { + KeyCap(systemImage: model.shifted ? "shift.fill" : "shift", dark: true, width: width) { model.toggleShift() } + } + } +} + +/// A letter key with the system keyboard's 24 pt legend and its pop-up: a +/// larger copy of the letter above the key while the finger is down, so the +/// finger doesn't hide what it's pressing. Typed on release, as iOS does. +private struct LetterKey: View { + let label: String + let width: CGFloat + let action: () -> Void + /// A gesture state, so a touch the system takes away unlights the key and + /// drops its pop-up; a plain state would leave both up. + @GestureState private var pressed = false + @Environment(\.keyboardPalette) private var palette + @Environment(\.keyboardInContainer) private var inContainer + + var body: some View { + Text(label) + .font(.system(size: DesignTokens.Typography.sizeLetter, weight: DesignTokens.Typography.weightLetter)) + .foregroundStyle(palette.keyText) + .frame(width: width, height: KeyCap.height) + .keyCap(palette.keyFill(modifier: false, inContainer: inContainer)) + .overlay(alignment: .top) { + if pressed { popup } + } + .zIndex(pressed ? 1 : 0) + .contentShape(Rectangle()) + .accessibilityLabel(label) + .accessibilityAddTraits(.isButton) + .simultaneousGesture( + DragGesture(minimumDistance: 0) + .updating($pressed) { _, state, _ in state = true } + .onEnded { value in + // A touch that swiped away (the panel's carousel) was not a tap; + // one that ended on the key was, wherever the thumb rolled. + guard + KeyboardInteraction.isTap( + value.translation, endedAt: value.location, in: CGSize(width: width, height: KeyCap.height)) + else { return } + action() + } + ) + } + + private var popup: some View { + Text(label) + .font(.system(size: DesignTokens.Typography.sizePopup, weight: DesignTokens.Typography.weightPopup)) + .foregroundStyle(palette.keyText) + .frame(width: width + DesignTokens.Metrics.popupExtraWidth, height: DesignTokens.Metrics.popupHeight) + .background(palette.popupFill, in: RoundedRectangle(cornerRadius: DesignTokens.Metrics.popupRadius)) + .shadow( + color: .black.opacity(DesignTokens.Metrics.opacityPopupShadow), radius: DesignTokens.Metrics.popupShadowRadius, + y: DesignTokens.Metrics.popupShadowY + ) + .offset(y: -DesignTokens.Metrics.popupOffset) + .allowsHitTesting(false) + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/Grain.swift b/App/BlurtiOS/BlurtKeyboard/Sources/Grain.swift new file mode 100644 index 00000000..0981d8c4 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/Grain.swift @@ -0,0 +1,28 @@ +import SwiftUI + +/// Film grain: a scatter of faint dots drawn from a seed, so a frame is +/// reproducible — the brand's texture, on the keyboard's surface +/// (`SurfaceFinish`), still, one seed. Drawn synchronously: a canvas that +/// renders asynchronously never presents inside a keyboard extension. +struct BlurtiOSGrain: View { + let seed: Int + + var body: some View { + Canvas { context, size in + var state = UInt64(truncatingIfNeeded: seed &* 6_364_136_223_846_793_005 &+ 1) + func next() -> CGFloat { + state = state &* 6_364_136_223_846_793_005 &+ 1_442_695_040_888_963_407 + return CGFloat(state >> 40) / CGFloat(1 << 24) + } + let count = Int(size.width * size.height / 26) + for _ in 0.. 0.5 ? .white.opacity(0.18) : .black.opacity(0.22))) + } + } + .accessibilityHidden(true) + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/KeyCap.swift b/App/BlurtiOS/BlurtKeyboard/Sources/KeyCap.swift new file mode 100644 index 00000000..2360179f --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/KeyCap.swift @@ -0,0 +1,128 @@ +import BlurtiOSCore +import SwiftUI + +/// One ordinary key: a glyph (SF Symbols, as the system keyboard's) or a +/// word (the brand's mono eyebrow) on a cap in the current face; `dark` for +/// the modifier keys, a step darker as on the system keyboard; `bare` for a +/// glyph with no cap at all. `flexible` keys (the space bar) take the width +/// they're given, `width` fixes one — the layouts pass `KeyGeometry`'s, so +/// a key is the same size on every page — and a key with neither is at least +/// the iPhone's 123 key wide, its legend padded inside that. `height` fixes +/// the cap's height where a row is not the standard 43 (the slim bar). +struct KeyCap: View { + var title: String? + var systemImage: String? + var tint: Color? + var flexible = false + var dark = false + var bare = false + var width: CGFloat? + var height: CGFloat? + var action: () -> Void + @Environment(\.keyboardPalette) private var palette + @Environment(\.keyboardInContainer) private var inContainer + + static let height = DesignTokens.Metrics.keyHeight + + init( + title: String? = nil, systemImage: String? = nil, tint: Color? = nil, flexible: Bool = false, + dark: Bool = false, bare: Bool = false, width: CGFloat? = nil, height: CGFloat? = nil, + action: @escaping () -> Void + ) { + self.title = title + self.systemImage = systemImage + self.tint = tint + self.flexible = flexible + self.dark = dark + self.bare = bare + self.width = width + self.height = height + self.action = action + } + + var body: some View { + Group { + if let systemImage { + Image(systemName: systemImage) + .font(.system(size: DesignTokens.Typography.sizeLegend, weight: DesignTokens.Typography.weightLegend)) + .foregroundStyle(tint ?? palette.keyText) + } else { + // A word on a key — 123, ABC, #+=, space, the return label — is the + // brand's eyebrow: the system mono, uppercase, tracked, a step + // quieter than a letter. Tracking adds its space after the last + // letter too; the same again before the first keeps the word centred. + Text((title ?? "").uppercased()) + .font(BlurtType.mono(DesignTokens.Typography.sizeLabel, weight: DesignTokens.Typography.weightLabel)) + .tracking(DesignTokens.Typography.trackingEyebrow) + .padding(.leading, DesignTokens.Typography.trackingEyebrow) + .foregroundStyle(tint ?? palette.keyTextSecondary) + } + } + .opacity(DesignTokens.Metrics.opacityLegend) + // The padding sits inside the minimum width, so a glyph key is exactly + // the minimum and only a long word widens a key past it. + .padding(.horizontal, flexible || width != nil ? 0 : DesignTokens.Metrics.keyPad) + .frame(maxWidth: flexible ? .infinity : nil) + .frame(width: width, height: height) + .frame(minWidth: width == nil ? DesignTokens.Metrics.keyMinWidth : nil, minHeight: height ?? Self.height) + .keyCap(bare ? .clear : palette.keyFill(modifier: dark, inContainer: inContainer)) + .keyPress(action) + .accessibilityLabel(title ?? systemImage ?? "") + } +} + +extension View { + /// The cap under a key's legend: the palette's fill at the corner radius and + /// nothing else — flat. No drop, no edge, no gloss. + func keyCap(_ fill: Color) -> some View { + background(fill, in: RoundedRectangle(cornerRadius: KeyboardPalette.keyRadius)) + } + + /// A key's touch: lights while the finger is down, acts on release — and + /// only if the finger ended on the key, so a swipe across a key (the + /// panel's carousel) never types while a thumb that rolls still does. The + /// system keyboard's own rule (`KeyboardInteraction.isTap`). + func keyPress(_ action: @escaping () -> Void) -> some View { + modifier(KeyPress(action: action)) + } +} + +struct KeyPress: ViewModifier { + let action: () -> Void + /// A gesture state: a touch the system takes away (a swipe from the screen's + /// bottom edge, a call, the keyboard dismissed) unlights the key too, where + /// a plain state would leave it lit. + @GestureState private var pressed = false + @State private var size = CGSize.zero + + func body(content: Content) -> some View { + content + .brightness(pressed ? DesignTokens.Metrics.opacityPressBrighten : 0) + .animation(.easeOut(duration: DesignTokens.Motion.keyPress), value: pressed) + .contentShape(Rectangle()) + .onGeometryChange(for: CGSize.self) { + $0.size + } action: { + size = $0 + } + .accessibilityAddTraits(.isButton) + .simultaneousGesture( + DragGesture(minimumDistance: 0) + .updating($pressed) { _, state, _ in state = true } + .onEnded { value in + guard KeyboardInteraction.isTap(value.translation, endedAt: value.location, in: size) else { return } + action() + } + ) + } +} + +/// A key lightens while the finger is on it, the way the system's change +/// shade, instead of the default button dimming. +struct KeyPressStyle: ButtonStyle { + func makeBody(configuration: Configuration) -> some View { + configuration.label + .brightness(configuration.isPressed ? DesignTokens.Metrics.opacityPressBrighten : 0) + .animation(.easeOut(duration: DesignTokens.Motion.keyPress), value: configuration.isPressed) + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/KeyGeometry.swift b/App/BlurtiOS/BlurtKeyboard/Sources/KeyGeometry.swift new file mode 100644 index 00000000..b14dd565 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/KeyGeometry.swift @@ -0,0 +1,41 @@ +import CoreFoundation + +/// The full keyboard's widths, the iPhone's own: ten letter caps across at +/// one width, the side keys taking what seven letters and their gaps leave, +/// the bottom row's 123 and globe at the width the iPhone gives them and +/// return two of those wide. The numbers were read off the system keyboard +/// on the iPhone 18 Pro (`Design/apple-geometry.json`, scripts/apple-geometry.sh) +/// and live in the tokens; this is the arithmetic between them, unit-tested, +/// so the views carry none of it. +nonisolated struct KeyGeometry: Equatable { + private typealias Metrics = DesignTokens.Metrics + + /// The width inside the side margins — what a row of keys has. + let rowWidth: CGFloat + + init(rowWidth: CGFloat) { self.rowWidth = rowWidth } + + /// From the keyboard's whole width, margins included. + init(width: CGFloat) { self.init(rowWidth: width - 2 * Metrics.marginSide) } + + var gap: CGFloat { Metrics.keyGap } + /// Ten keys and nine gaps across the row. + var letterWidth: CGFloat { (rowWidth - 9 * gap) / 10 } + /// Shift and delete stand a little further from the letters than letters + /// stand from each other. + var sideGap: CGFloat { Metrics.keySideGap } + var sideWidth: CGFloat { (rowWidth - 7 * letterWidth - 6 * gap - 2 * sideGap) / 2 } + /// On the symbol pages the third row holds five punctuation keys between + /// shift's and delete's places, filling the same span seven letters do. + var punctuationWidth: CGFloat { (rowWidth - 2 * sideWidth - 2 * sideGap - 4 * gap) / 5 } + /// The widths measured at the reference width scale with the keyboard. + private var scale: CGFloat { (rowWidth + 2 * Metrics.marginSide) / Metrics.keyReferenceWidth } + /// 123 (or ABC), and the globe when there is one. + var abcWidth: CGFloat { Metrics.keyAbcWidth402 * scale } + /// Return: two 123 keys and a gap. + var returnWidth: CGFloat { 2 * abcWidth + gap } + /// Space takes what the bottom row leaves. + func spaceWidth(globe: Bool) -> CGFloat { + rowWidth - abcWidth - (globe ? abcWidth + gap : 0) - returnWidth - 2 * gap + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/KeyboardModel+Palette.swift b/App/BlurtiOS/BlurtKeyboard/Sources/KeyboardModel+Palette.swift new file mode 100644 index 00000000..f4b9e71e --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/KeyboardModel+Palette.swift @@ -0,0 +1,13 @@ +import BlurtiOSCore + +extension KeyboardModel { + /// The palette for the face the model draws (`KeyboardModel.face`). + var palette: KeyboardPalette { .resolve(face.themeID, dark: face.dark) } +} + +extension ThemeFace { + /// The face a palette is: what a preview hands the model to draw it. + init(_ palette: KeyboardPalette) { + self.init(themeID: palette.id, dark: palette.face == .dark) + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/KeyboardPalette.swift b/App/BlurtiOS/BlurtKeyboard/Sources/KeyboardPalette.swift new file mode 100644 index 00000000..a55e4f66 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/KeyboardPalette.swift @@ -0,0 +1,147 @@ +import SwiftUI + +/// The colours the keys draw with: the brand's two faces. The keyboard follows +/// the field it is typing into — ink with the brand's film grain when the +/// host app is dark, warm paper when it is light — so Blurt sits on any app +/// as one keyboard, on the brand and off Apple's grey. The spacing never +/// changes (the iPhone keyboard's own — 6 pt between keys, 11 between rows, +/// 3 at the edges), so a face changes how the keyboard looks and never how it +/// types. The keys are flat: one colour, no drop, no edge, no gloss. +/// +/// One theme for now, with two faces. The curated themes come back later as +/// value sets through the same shape: `all` is the picker's list, `resolve` +/// the keyboard's lookup, and neither needs to change for a third entry. +struct KeyboardPalette: Equatable, Identifiable { + enum Face: String { + case light + case dark + } + + let id: String + let face: Face + let name: String + /// One line for the picker. + let vibe: String + let surface: Color + let key: Color + let keyDark: Color + let keyText: Color + /// The word labels (123, ABC, return, space) in mono, and the + at rest: + /// a step quieter than a letter. + let keyTextSecondary: Color + /// The voice on this surface: the wave, the caret, the saved check. The + /// brand green that reads on it — the wordmark's on paper, the lifted one + /// on ink. + let signal: Color + let popupFill: Color + /// The key-term field and its hairline. + let field: Color + let fieldBorder: Color + /// The error word, the cancel ×, the Full Access note: orange on both faces. + let notice: Color + /// How much film grain lies over the surface: the brand's texture, matte. + let grain: Double + /// The keys when the host wraps the keyboard in its own material (iOS 26's + /// Liquid Glass apps: Messages, Notes, Safari) and the surface goes clear: + /// colours that read on the system's light or dark material. + let containerKey: Color + let containerKeyModifier: Color + /// The host's material under a clear surface, as the gallery and the home + /// screen stand in for it (the system's own keyboard colour, measured). + let material: Color + + static let rowGap = DesignTokens.Metrics.rowGap + static let margin = DesignTokens.Metrics.marginSide + static let keyRadius = DesignTokens.Metrics.keyRadius + + private typealias Themes = DesignTokens.Themes + + /// The brand theme's id, and what any unknown id resolves to. + static let brandID = "blurt" + private static let brandName = "Blurt" + private static let brandVibe = "Ink or paper, with the app you're in" + + /// The light face: paper `#ECEBE5`, white keys, `#DAD7CB` modifiers, ink + /// legends, the wordmark green. + static let brandLight = KeyboardPalette( + id: brandID, face: .light, name: brandName, vibe: brandVibe, + surface: Themes.lightSurface, key: Themes.lightKey, keyDark: Themes.lightKeyModifier, + keyText: Themes.lightLegend, keyTextSecondary: Themes.lightLegendSecondary, signal: Themes.lightSignal, + popupFill: Themes.lightPopup, field: Themes.lightField, fieldBorder: Themes.lightFieldBorder, + notice: Themes.lightNotice, grain: DesignTokens.Metrics.opacityGrainLight, + containerKey: Themes.lightContainerKey, containerKeyModifier: Themes.lightContainerKeyModifier, + material: Themes.lightMaterial) + + /// The dark face: ink `#1D1B16`, `#33302A` keys, `#26231E` modifiers, warm + /// white legends, the lifted green. + static let brandDark = KeyboardPalette( + id: brandID, face: .dark, name: brandName, vibe: brandVibe, + surface: Themes.darkSurface, key: Themes.darkKey, keyDark: Themes.darkKeyModifier, + keyText: Themes.darkLegend, keyTextSecondary: Themes.darkLegendSecondary, signal: Themes.darkSignal, + popupFill: Themes.darkPopup, field: Themes.darkField, fieldBorder: Themes.darkFieldBorder, + notice: Themes.darkNotice, grain: DesignTokens.Metrics.opacityGrainDark, + containerKey: Themes.darkContainerKey, containerKeyModifier: Themes.darkContainerKeyModifier, + material: Themes.darkMaterial) + + /// The picker's list, one entry per theme (a theme's two faces share an id). + static let all: [KeyboardPalette] = [brandLight] + + /// The theme with this id in the given appearance, or the brand's for an id + /// that no longer exists (the old `system` default, a retired theme). + static func resolve(_ id: String, dark: Bool) -> KeyboardPalette { + // One theme so far: a known id and a retired one land in the same place. + guard all.contains(where: { $0.id == id }) else { return dark ? brandDark : brandLight } + return dark ? brandDark : brandLight + } + + /// What stands under the keyboard where the host's material would be + /// (the gallery, a preview): the face's surface when it is painted, else + /// the material. + var standIn: Color { KeyboardRootView.paintsSurface ? surface : material } + + /// A key's fill: the face's, or the container's when the host's material + /// is the surface. + func keyFill(modifier: Bool, inContainer: Bool) -> Color { + if inContainer { return modifier ? containerKeyModifier : containerKey } + return modifier ? keyDark : key + } + + static func == (lhs: KeyboardPalette, rhs: KeyboardPalette) -> Bool { lhs.id == rhs.id && lhs.face == rhs.face } +} + +extension Color { + /// `Color(hex: 0x1D1B16)`. `nonisolated` so the generated `DesignTokens` + /// statics, which are nonisolated too, can call it. + nonisolated init(hex: UInt32) { + self.init( + red: Double((hex >> 16) & 0xFF) / 255, green: Double((hex >> 8) & 0xFF) / 255, blue: Double(hex & 0xFF) / 255) + } +} + +private struct KeyboardInContainerKey: EnvironmentKey { + static let defaultValue = false +} + +extension EnvironmentValues { + /// Whether the host wraps the keyboard in its own material (iOS 26's + /// Liquid Glass apps inset a third-party keyboard inside a rounded + /// container nothing can paint over): the surface goes clear and the keys + /// take the palette's container colours, so there is no band of a + /// different grey above the keyboard. + var keyboardInContainer: Bool { + get { self[KeyboardInContainerKey.self] } + set { self[KeyboardInContainerKey.self] = newValue } + } +} + +private struct KeyboardPaletteKey: EnvironmentKey { + static let defaultValue = KeyboardPalette.brandLight +} + +extension EnvironmentValues { + /// The palette the keys draw with, set once at the root from the model. + var keyboardPalette: KeyboardPalette { + get { self[KeyboardPaletteKey.self] } + set { self[KeyboardPaletteKey.self] = newValue } + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/KeyboardRootView.swift b/App/BlurtiOS/BlurtKeyboard/Sources/KeyboardRootView.swift new file mode 100644 index 00000000..111b6815 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/KeyboardRootView.swift @@ -0,0 +1,65 @@ +import BlurtiOSCore +import SwiftUI + +/// Picks the layout the user chose. All three share the voice bar and the +/// mic control, `KeyCap`, and the same model underneath; they differ in how +/// much keyboard surrounds the mic. The surface is the face's, fixed whatever +/// the host app's appearance — the keyboard floats over whichever app the +/// user is typing in, like the Mac pill — at the iPhone keyboard's own +/// spacing. +struct KeyboardRootView: View { + var model: KeyboardModel + + /// The keyboard's top margin (and the slim bar's bottom): the arithmetic in + /// `KeyboardLayout.height` is built on it and the palette's row gap. + static let verticalMargin = DesignTokens.Metrics.marginVertical + /// Under a bottom row of keys, the iPhone's own margin. + static let bottomMarginKeys = DesignTokens.Metrics.marginBottomKeys + /// Whether the face's surface is painted under the keys (`surface/paint`). + static let paintsSurface = DesignTokens.Metrics.surfacePaint > 0 + + var body: some View { + Group { + switch model.layout { + case .panel: PanelCarousel(model: model, reduceMotion: reduceMotion) + case .full: FullKeyboardView(model: model) + case .slimBar: + // The keys come up for a key term; the bar is back when it closes. + if model.termDraft == nil { SlimBarView(model: model) } else { FullKeyboardView(model: model) } + } + } + .padding(.horizontal, KeyboardPalette.margin) + .padding(.top, Self.verticalMargin) + .padding(.bottom, model.effectiveLayout == .slimBar ? Self.verticalMargin : Self.bottomMarginKeys) + // Anchored to the bottom: if the host ever gives the keyboard more + // height than its layout, the extra is surface above the keys, never + // a hole under them. + .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .bottom) + // iOS 26 wraps a third-party keyboard in the host's own rounded material, + // inset above and below, and nothing the keyboard draws reaches the + // inset: a painted surface shows a band of the host's grey above it. So + // by default the surface is clear — the host's material is the surface, + // seamlessly — and the keys take the colours that read on it; the face + // is painted, grain on top, only when `surface/paint` says so. + .background { + if Self.paintsSurface { SurfaceFinish.Ground(surface: model.palette.surface) } + } + .overlay { + if Self.paintsSurface { SurfaceFinish.Grain() } + } + .clipped() + .environment(\.keyboardPalette, model.palette) + .environment(\.keyboardInContainer, !Self.paintsSurface) + .environment(\.voiceElementKind, model.voiceKind) + // The carousel's swipe is not a SwiftUI gesture here. The surface is + // clear, and the host hands the keyboard only the touches that land on + // painted pixels: `KeyboardViewController` paints a floor the eye can't + // see and recognises the swipe on the input view itself, which sees + // every touch the keyboard gets, the panel's empty space included. + // The slide itself is `PanelCarousel`'s. + } + + @Environment(\.accessibilityReduceMotion) private var systemReduceMotion + @Environment(\.keyboardMotionHeld) private var motionHeld + private var reduceMotion: Bool { systemReduceMotion || motionHeld } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/KeyboardViewController.swift b/App/BlurtiOS/BlurtKeyboard/Sources/KeyboardViewController.swift new file mode 100644 index 00000000..f3b2345c --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/KeyboardViewController.swift @@ -0,0 +1,161 @@ +import BlurtiOSCore +import SwiftUI +import UIKit + +/// The keyboard's entry point — what iOS instantiates when the user picks Blurt +/// from the globe key. Thin on purpose: it hosts the SwiftUI keyboard, sizes it +/// to the layout the user chose, and forwards the lifecycle to `KeyboardModel`, +/// which does the talking to the app. It never hears anything: iOS lets no +/// keyboard use the microphone, so the app listens and this inserts. +final class KeyboardViewController: UIInputViewController, UIGestureRecognizerDelegate { + private let model = KeyboardModel() + + /// The input view says it wants clicks, so `UIDevice.playInputClick()` on + /// each key press plays the system's keyboard click — and only if the + /// user has keyboard clicks on in Settings. Their setting, not ours. + private final class ClickingInputView: UIInputView, UIInputViewAudioFeedback { + var enableInputClicksWhenVisible: Bool { true } + + } + + private var heightConstraint: NSLayoutConstraint? + private var hasAppeared = false + private let floor = UIView() + + /// The least the floor can be painted and still be the keyboard's to the + /// host — never zero. In the host material's own colour, so the 1 % adds + /// nothing the eye can see: 1 % black over the light material was a + /// keyboard a shade darker than the border around it. + private static let floorAlpha: CGFloat = 0.01 + + override func loadView() { + view = ClickingInputView(frame: .zero, inputViewStyle: .keyboard) + } + + override func viewDidLoad() { + super.viewDidLoad() + model.attach(to: self) + // The floor. The host hands the keyboard only the touches that land on + // pixels it drew: where the keyboard is fully transparent — and the + // surface is clear, the host's own material showing through — a touch + // never reaches this process at all (its hit test is not even asked), + // and goes to the host's keyboard chrome instead. So the panel's empty + // space, where a finger swipes the carousel, must be painted, if only + // just: a floor the eye cannot see but the compositor can (`paintFloor`). + floor.translatesAutoresizingMaskIntoConstraints = false + view.addSubview(floor) + let host = UIHostingController(rootView: KeyboardRootView(model: model)) + addChild(host) + host.view.translatesAutoresizingMaskIntoConstraints = false + host.view.backgroundColor = .clear + view.addSubview(host.view) + NSLayoutConstraint.activate([ + floor.leadingAnchor.constraint(equalTo: view.leadingAnchor), + floor.trailingAnchor.constraint(equalTo: view.trailingAnchor), + floor.topAnchor.constraint(equalTo: view.topAnchor), + floor.bottomAnchor.constraint(equalTo: view.bottomAnchor), + host.view.leadingAnchor.constraint(equalTo: view.leadingAnchor), + host.view.trailingAnchor.constraint(equalTo: view.trailingAnchor), + host.view.topAnchor.constraint(equalTo: view.topAnchor), + host.view.bottomAnchor.constraint(equalTo: view.bottomAnchor), + ]) + host.didMove(toParent: self) + // The keyboard's height is ours to declare, at just under required + // priority, and only once the view has appeared (from + // updateViewConstraints): activated from viewDidLoad it fights the host's + // first layout and the keyboard flashes at the wrong size. What the host + // draws *around* the keyboard — iOS 26's rounded container, inset above + // and below — is the host's, and nothing here changes it. + let height = view.heightAnchor.constraint(equalToConstant: model.effectiveLayout.height) + height.priority = .required - 1 + heightConstraint = height + // The panel's carousel changes the keyboard's height when it flips. + model.onLayoutChange = { [weak self] in self?.updateHeight(animated: true) } + // The carousel's swipe, on the input view itself: it sees every touch the + // keyboard gets — a key, the mic, the floor — where a SwiftUI gesture on + // the hosted root only sees what SwiftUI hit-tests. It runs beside + // SwiftUI's own gestures and cancels none of them: the keys and the mic + // keep their touch and apply their travel rules (`KeyboardInteraction`), + // so a swipe across a key never types and one across the mic never + // dictates. + let swipe = UIPanGestureRecognizer(target: self, action: #selector(swiped)) + swipe.cancelsTouchesInView = false + swipe.delaysTouchesBegan = false + swipe.delaysTouchesEnded = false + swipe.delegate = self + view.addGestureRecognizer(swipe) + } + + @objc private func swiped(_ swipe: UIPanGestureRecognizer) { + guard swipe.state == .ended else { return } + let travel = swipe.translation(in: view) + guard let direction = KeyboardInteraction.flip(CGSize(width: travel.x, height: travel.y)) else { return } + model.flipPanel(towardsLeading: direction == .towardsLeading) + } + + func gestureRecognizer( + _ gestureRecognizer: UIGestureRecognizer, shouldRecognizeSimultaneouslyWith other: UIGestureRecognizer + ) -> Bool { + true + } + + override func viewWillAppear(_ animated: Bool) { + super.viewWillAppear(animated) + model.appeared() + paintFloor() + hasAppeared = true + view.setNeedsUpdateConstraints() + } + + /// The host's traits are resolved by now where they may not have been in + /// `viewWillAppear`: the face is read again here, so a dark host is dark + /// from the first frame. Never from a trait-change registration — that + /// callback runs inside the host's layout pass, and a model change made + /// there left the mic element's drawing uncommitted in Messages (nothing + /// drew, every time). + override func viewIsAppearing(_ animated: Bool) { + super.viewIsAppearing(animated) + model.contextChanged() + paintFloor() + } + + /// The floor in the face's material at `floorAlpha`: the host's own + /// keyboard colour, measured, so nothing shows through the paint but the + /// touches reach the keyboard. + private func paintFloor() { + floor.backgroundColor = UIColor(model.palette.material).withAlphaComponent(Self.floorAlpha) + } + + override func updateViewConstraints() { + super.updateViewConstraints() + guard hasAppeared, let heightConstraint else { return } + heightConstraint.constant = model.effectiveLayout.height + if !heightConstraint.isActive { heightConstraint.isActive = true } + } + + private func updateHeight(animated: Bool) { + heightConstraint?.constant = model.effectiveLayout.height + view.setNeedsUpdateConstraints() + guard animated else { return } + UIView.animate(withDuration: DesignTokens.Motion.heightChange) { self.view.superview?.layoutIfNeeded() } + } + + override func viewWillDisappear(_ animated: Bool) { + super.viewWillDisappear(animated) + model.disappeared() + } + + override func textDidChange(_ textInput: (any UITextInput)?) { + super.textDidChange(textInput) + model.contextChanged() + // The field's appearance can change with the field. + paintFloor() + } + + /// The highlight moved without the text changing — a word selected, or the + /// selection cleared: the + follows it (`KeyboardModel.selectedTerm`). + override func selectionDidChange(_ textInput: (any UITextInput)?) { + super.selectionDidChange(textInput) + model.contextChanged() + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/MicControl.swift b/App/BlurtiOS/BlurtKeyboard/Sources/MicControl.swift new file mode 100644 index 00000000..d57bd875 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/MicControl.swift @@ -0,0 +1,113 @@ +import BlurtiOSCore +import SwiftUI + +/// The mic key. Finger down starts, finger up decides tap (latched) or hold +/// (push-to-talk) — `KeyboardModel` runs the engine's gate, +/// `MicPressSequencer` the press's steps, `VoiceState` what shows, and the +/// voice element draws it (`VoiceElementView`): dimmed when Blurt isn't +/// ready (the tap opens Blurt), working while the mic comes up and the +/// words are made, the meter while you talk, a glint when the words land, +/// orange when something failed. Only the two notices that need saying carry +/// a glyph — a clipboard when the words went there, an exclamation mark on a +/// failure. Each state also has its haptic. +/// +/// Nothing snaps or springs: every change on the key fades over `state-fade`. +/// Only the press itself answers at once. +struct MicControl: View { + var model: KeyboardModel + let slot: VoiceSlot + /// The key's width: as wide as the element shows (`VoiceRowLayout`). + var width: CGFloat? + /// The key's touch target is at least this tall — the row's height — so + /// the whole row under the element is the key, not just the element's box. + var hitHeight: CGFloat? + /// Down while the finger is on the key. A gesture state, so a touch the + /// system takes away lets the key back up; a plain state would hold it down. + @GestureState private var pressed = false + @State private var press = MicPressSequencer() + @State private var pressTask: Task? + @Environment(\.accessibilityReduceMotion) private var systemReduceMotion + @Environment(\.keyboardMotionHeld) private var motionHeld + @Environment(\.keyboardPalette) private var palette + private var reduceMotion: Bool { systemReduceMotion || motionHeld } + + var body: some View { + let state = model.voiceState + let box = slot.box + let hit = CGSize(width: width ?? box.width, height: max(box.height, hitHeight ?? 0)) + ZStack { + VoiceElementView( + inputs: VoiceElementInputs( + state: state, landedAt: model.resultLandedAt, animated: !reduceMotion, palette: palette, slot: slot)) + if let glyph = state.glyph { + Image(systemName: Self.symbol(glyph)) + .font( + .system( + size: box.height * DesignTokens.Typography.ratioVoiceGlyph, + weight: DesignTokens.Typography.weightVoiceGlyph) + ) + .foregroundStyle(palette.keyText) + .transition(.opacity) + } + } + .frame(width: hit.width, height: hit.height) + .animation(.easeInOut(duration: DesignTokens.Motion.stateFade), value: state.phase) + .scaleEffect(pressed ? DesignTokens.Metrics.voicePressScale : 1) + .animation(.easeOut(duration: DesignTokens.Motion.press), value: pressed) + .contentShape(Rectangle()) + .accessibilityLabel(state.accessibilityLabel) + .accessibilityValue(state.accessibilityValue) + .accessibilityAddTraits(.isButton) + .accessibilityIdentifier("blurt-mic") + .onDisappear { + pressTask?.cancel() + pressTask = nil + press = MicPressSequencer() + } + .simultaneousGesture( + DragGesture(minimumDistance: 0) + .updating($pressed) { _, state, _ in state = true } + .onChanged { value in + if press.began() { + // The press waits a beat, so a swipe that starts on the key (the + // panel's carousel) never starts a dictation it must then cancel. + pressTask = Task { [model] in + try? await Task.sleep(for: KeyboardInteraction.micPressDelay) + // A touch the system took away in the meantime (the gesture + // state is down again) presses nothing. + guard !Task.isCancelled, pressed else { + press = MicPressSequencer() + return + } + if press.delayFired() { model.micDown() } + } + } + if press.moved(travelled: KeyboardInteraction.micTravelled(value.translation)) { + pressTask?.cancel() + pressTask = nil + } + } + .onEnded { value in + pressTask?.cancel() + pressTask = nil + switch press.ended(travelled: KeyboardInteraction.micTravelled(value.translation)) { + case .tap(let needsDown): + // A tap quicker than the delay is still a tap. + if needsDown { model.micDown() } + model.micUp() + case .cancelSentPress: + model.undoPress() + case .nothing: + break + } + } + ) + } + + private static func symbol(_ glyph: VoiceState.Glyph) -> String { + switch glyph { + case .clipboard: "doc.on.clipboard" + case .exclamation: "exclamationmark" + } + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/MotionHeld.swift b/App/BlurtiOS/BlurtKeyboard/Sources/MotionHeld.swift new file mode 100644 index 00000000..d0275dc0 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/MotionHeld.swift @@ -0,0 +1,18 @@ +import SwiftUI + +/// Whether the keyboard's continuous motion — the ring's sweep, the meter's +/// idle wave, the orb's fluid and grain, the caret's blink — is held at time +/// zero. Reduce Motion holds it for the user; the gallery's +/// `-BlurtGalleryStill` holds it for a capture, so the same state is the same +/// pixels every time (DESIGN.md › Reproducing it). The system value is read-only, so +/// this is the switch the views combine it with. +private struct KeyboardMotionHeldKey: EnvironmentKey { + static let defaultValue = false +} + +extension EnvironmentValues { + var keyboardMotionHeld: Bool { + get { self[KeyboardMotionHeldKey.self] } + set { self[KeyboardMotionHeldKey.self] = newValue } + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/PanelCarousel.swift b/App/BlurtiOS/BlurtKeyboard/Sources/PanelCarousel.swift new file mode 100644 index 00000000..a3c18d51 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/PanelCarousel.swift @@ -0,0 +1,142 @@ +import BlurtiOSCore +import SwiftUI + +/// The panel's two pages — the mic panel and the full keyboard — as a strip +/// the finger slides, one page wide. The page that is up sits in the middle; +/// when a flip comes, the other page is set down a whole page away on the +/// side the finger moved towards, and the strip slides one page: the first +/// leaves one way as the second arrives the other. Side by side, a page +/// apart, the two can never cross, whichever way the flips come — left then +/// right, or left twice. A flip back before the slide has settled retreats +/// rather than starting a second slide over the first. `PanelCarouselState` +/// holds the rules (tested); this view only animates what it says. +struct PanelCarousel: View { + var model: KeyboardModel + let reduceMotion: Bool + @State private var carousel: PanelCarouselState + /// How far the strip has slid, in pages: 0 at rest, −1 or 1 at the end of + /// a slide. The only animated value. + @State private var slide: CGFloat = 0 + @State private var onScreen = false + + init(model: KeyboardModel, reduceMotion: Bool) { + self.model = model + self.reduceMotion = reduceMotion + _carousel = State(initialValue: PanelCarouselState(keysUp: model.panelShowsKeys)) + } + + var body: some View { + GeometryReader { geo in + let width = geo.size.width + ZStack(alignment: .bottom) { + // The keys come up for a key term whichever page is up, in place. + page(keys: carousel.keysUp || model.termDraft != nil, width: width) + .offset(x: slide * width) + if let edge = carousel.incomingEdge, model.termDraft == nil { + page(keys: !carousel.keysUp, width: width) + .offset(x: (slide + (edge == .trailing ? 1 : -1)) * width) + } + } + .frame(width: width, height: geo.size.height, alignment: .bottom) + } + .onAppear { onScreen = true } + .onDisappear { onScreen = false } + .onChange(of: model.panelShowsKeys) { _, keysUp in flip(to: keysUp) } + } + + /// One page at its own height, so neither is stretched while the keyboard + /// is resizing under the slide. + @ViewBuilder private func page(keys: Bool, width: CGFloat) -> some View { + let height = + (keys ? KeyboardLayout.full : .panel).height - KeyboardRootView.verticalMargin + - KeyboardRootView.bottomMarginKeys + if keys { + FullKeyboardView(model: model).id("keys").frame(width: width, height: height, alignment: .bottom) + } else { + PanelView(model: model).id("panel").frame(width: width, height: height, alignment: .bottom) + } + } + + private func flip(to keysUp: Bool) { + let motion = carousel.flip( + to: keysUp, towardsLeading: model.flipTowardsLeading, animated: !reduceMotion && onScreen) + guard let motion else { + slide = 0 + return + } + withAnimation(.easeInOut(duration: DesignTokens.Motion.flip)) { + slide = motion.slide + } completion: { + // Only the slide that was last asked for settles the strip; an older + // one was overtaken and its ending means nothing. + guard carousel.settle(motion.generation) else { return } + var settled = Transaction() + settled.disablesAnimations = true + withTransaction(settled) { slide = 0 } + } + } +} + +/// The carousel's rules, out of the view: which page is at rest in the +/// middle, which the model wants, where the other page waits, and which slide +/// is the current one — so a flip back mid-slide retreats, a retreat cut +/// short keeps its side, and a slide that was overtaken cannot settle the +/// strip late. +nonisolated struct PanelCarouselState: Equatable { + enum Edge: Equatable { + case leading + case trailing + } + + /// What the view animates: the strip's slide in pages, and the generation + /// that may settle it when the slide ends. + struct Motion: Equatable { + let slide: CGFloat + let generation: Int + } + + /// The page in the middle at rest (true: the keys). + private(set) var keysUp: Bool + /// The page the model wants — the same as `keysUp` unless a slide is in flight. + private(set) var target: Bool + /// The side the other page waits on while a slide is in flight. + private(set) var incomingEdge: Edge? + private(set) var generation = 0 + + var inFlight: Bool { incomingEdge != nil } + + init(keysUp: Bool) { + self.keysUp = keysUp + target = keysUp + } + + /// The model flipped to `page`. Nil: settled at once, nothing to animate. + /// Otherwise what `slide` animates to: a page away towards the finger, or + /// back to 0 when this flip undoes one still in flight. + mutating func flip(to page: Bool, towardsLeading: Bool, animated: Bool) -> Motion? { + target = page + generation += 1 + guard animated else { + keysUp = page + incomingEdge = nil + return nil + } + guard page != keysUp else { + // Flipped back before the slide settled: the incoming page retreats. + return inFlight ? Motion(slide: 0, generation: generation) : nil + } + // A retreat cut short keeps its side; otherwise the finger's. + let edge = incomingEdge ?? (towardsLeading ? Edge.trailing : .leading) + incomingEdge = edge + return Motion(slide: edge == .trailing ? -1 : 1, generation: generation) + } + + /// A slide ended. True, and the strip is settled on the target page, only + /// for the generation last asked for. + mutating func settle(_ generation: Int) -> Bool { + guard generation == self.generation else { return false } + keysUp = target + incomingEdge = nil + return true + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/PanelView.swift b/App/BlurtiOS/BlurtKeyboard/Sources/PanelView.swift new file mode 100644 index 00000000..da574873 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/PanelView.swift @@ -0,0 +1,46 @@ +import BlurtiOSCore +import SwiftUI + +/// The voice element, big, dead centre — or at the edge Settings chose — with +/// the + beside it and cancel × on its other side while something is in +/// flight; one row of keys at the iPhone's widths. 216 pt. +struct PanelView: View { + var model: KeyboardModel + @Environment(\.keyboardPalette) private var palette + @Environment(\.layoutDirection) private var direction + + var body: some View { + GeometryReader { geo in + let geometry = KeyGeometry(rowWidth: geo.size.width) + let alignment = model.micAlignment.alignment(in: direction) + let atEdge = alignment != .center + VStack(spacing: DesignTokens.Metrics.panelSpacing) { + // The row takes all the height above the keys. The spacing under it + // is the margin above the page, so the mic is centred between the + // keyboard's top edge and the keys. + VoiceRow(model: model, slot: .panel, height: VoiceSlot.panel.box.height, cancels: !atEdge) + .frame(maxHeight: .infinity) + // The stock bottom row at the full keyboard's widths, so nothing + // changes size when the carousel flips to the keys and back. + HStack(spacing: geometry.gap) { + if model.needsGlobe { + KeyCap(systemImage: "globe", dark: true, width: geometry.abcWidth) { model.globe() } + } + KeyCap(title: "space", flexible: true) { model.space() } + KeyCap(systemImage: "delete.left", dark: true, width: geometry.abcWidth) { model.deleteBackward() } + KeyCap(title: model.returnLabel ?? "return", dark: true, width: geometry.returnWidth) { model.newline() } + } + } + .overlay(alignment: alignment == .trailing ? .topLeading : .topTrailing) { + // With the mic at an edge, cancel goes to the far corner, clear of + // the mic and the + on its inner side. + if atEdge, model.voiceState.canCancel { + GlyphKey(symbol: "xmark", tint: palette.notice, label: "Cancel dictation") { model.cancel() } + .padding(DesignTokens.Metrics.addtermInset) + .transition(.opacity) + } + } + .animation(.easeInOut(duration: DesignTokens.Motion.stateFade), value: model.voiceState.canCancel) + } + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/SlimBarView.swift b/App/BlurtiOS/BlurtKeyboard/Sources/SlimBarView.swift new file mode 100644 index 00000000..10197fcb --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/SlimBarView.swift @@ -0,0 +1,29 @@ +import BlurtiOSCore +import SwiftUI + +/// One row: the globe (when the phone doesn't draw its own), the voice bar, +/// delete and return, at the full keyboard's key widths and gap. 60 pt. +struct SlimBarView: View { + var model: KeyboardModel + + var body: some View { + GeometryReader { geo in + let geometry = KeyGeometry(rowWidth: geo.size.width) + // The keys are as tall as the row, so nothing sits half a point off. + let height = VoiceBar.height + HStack(spacing: geometry.gap) { + if model.needsGlobe { + KeyCap(systemImage: "globe", dark: true, width: geometry.abcWidth, height: height) { model.globe() } + } + VoiceBar(model: model) + KeyCap(systemImage: "delete.left", dark: true, width: geometry.abcWidth, height: height) { + model.deleteBackward() + } + KeyCap(title: model.returnLabel ?? "return", dark: true, width: geometry.returnWidth, height: height) { + model.newline() + } + } + } + .frame(height: VoiceBar.height) + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/SurfaceFinish.swift b/App/BlurtiOS/BlurtKeyboard/Sources/SurfaceFinish.swift new file mode 100644 index 00000000..d16a91f8 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/SurfaceFinish.swift @@ -0,0 +1,40 @@ +import SwiftUI + +/// The surface's finish. Under the keys: the face's flat colour and, when +/// the tokens ask for one, a soft vignette toward the edges. Over everything: +/// the brand's film grain, still (one seed, no shimmer — a keyboard has no +/// energy to spare), heavier on ink than on paper, so both faces read matte +/// rather than painted. Neither takes a touch. +enum SurfaceFinish { + struct Ground: View { + let surface: Color + + var body: some View { + GeometryReader { geo in + ZStack { + surface + if DesignTokens.Metrics.opacitySurfaceVignette > 0 { + Rectangle().fill( + RadialGradient( + colors: [.clear, .black.opacity(DesignTokens.Metrics.opacitySurfaceVignette)], center: .center, + startRadius: geo.size.width * DesignTokens.Metrics.surfaceVignetteStart, + endRadius: geo.size.width * DesignTokens.Metrics.surfaceVignetteEnd)) + } + } + } + .accessibilityHidden(true) + } + } + + struct Grain: View { + @Environment(\.keyboardPalette) private var palette + + var body: some View { + BlurtiOSGrain(seed: 7) + .opacity(palette.grain) + .blendMode(.overlay) + .allowsHitTesting(false) + .accessibilityHidden(true) + } + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/TermField.swift b/App/BlurtiOS/BlurtKeyboard/Sources/TermField.swift new file mode 100644 index 00000000..9e02ec4d --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/TermField.swift @@ -0,0 +1,61 @@ +import BlurtiOSCore +import SwiftUI + +/// The voice bar as a field — the brand's input, a rectangle with a hairline, +/// never a capsule: × to leave, what has been typed with a caret, ✓ to save. +/// Return saves too. × and ✓ are the same glyph keys as the +. +struct TermField: View { + var model: KeyboardModel + @Environment(\.keyboardPalette) private var palette + + var body: some View { + HStack(spacing: DesignTokens.Metrics.termGap) { + GlyphKey(symbol: "xmark", tint: palette.keyText.opacity(DesignTokens.Metrics.opacityTermCancel), label: "Cancel") + { + model.cancelAddingTerm() + } + HStack(spacing: 0) { + if let draft = model.termDraft, !draft.isEmpty { + Text(draft).foregroundStyle(palette.keyText) + } else { + Text("New key term").foregroundStyle(palette.keyText.opacity(DesignTokens.Metrics.opacityPlaceholder)) + } + Caret() + Spacer(minLength: 0) + } + .font(.system(size: DesignTokens.Typography.sizeTerm, weight: DesignTokens.Typography.weightTerm)) + .lineLimit(1) + .padding(.horizontal, DesignTokens.Metrics.termPad) + .frame(maxWidth: .infinity) + .frame(height: DesignTokens.Metrics.termHeight) + .background( + RoundedRectangle(cornerRadius: DesignTokens.Metrics.termRadius) + .fill(palette.field) + .strokeBorder(palette.fieldBorder, lineWidth: DesignTokens.Metrics.termBorder) + ) + .accessibilityElement(children: .ignore) + .accessibilityLabel("Key term: \(model.termDraft ?? "")") + GlyphKey(symbol: "checkmark", tint: palette.signal, label: "Save the key term") { model.saveTerm() } + .disabled(model.termDraft?.trimmingCharacters(in: .whitespaces).isEmpty ?? true) + } + .padding(.horizontal, DesignTokens.Metrics.termInset) + } +} + +/// A blinking caret, as a text field's; held on while motion is held. +private struct Caret: View { + @Environment(\.keyboardPalette) private var palette + @Environment(\.keyboardMotionHeld) private var motionHeld + + var body: some View { + TimelineView(.periodic(from: .now, by: DesignTokens.Motion.caret)) { timeline in + let on = motionHeld || Int(timeline.date.timeIntervalSinceReferenceDate / DesignTokens.Motion.caret) % 2 == 0 + RoundedRectangle(cornerRadius: DesignTokens.Metrics.caretRadius) + .fill(palette.signal) + .frame(width: DesignTokens.Metrics.caretWidth, height: DesignTokens.Metrics.caretHeight) + .opacity(on ? 1 : 0) + .padding(.leading, DesignTokens.Metrics.caretLead) + } + .accessibilityHidden(true) + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/VoiceBar.swift b/App/BlurtiOS/BlurtKeyboard/Sources/VoiceBar.swift new file mode 100644 index 00000000..215ebc44 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/VoiceBar.swift @@ -0,0 +1,47 @@ +import BlurtiOSCore +import SwiftUI + +/// The keyboard's voice row, and the only place voice lives in the bar and +/// the full keyboard: the mic key — tap to start and stop, hold to talk — +/// with the small + beside it for the one thing people want the instant +/// Blurt mishears a word: add it as a key term. The row is `VoiceRow`; this +/// swaps it for the key-term field the keys type into, or for the one note +/// the element can't carry (no Full Access). +struct VoiceBar: View { + var model: KeyboardModel + @Environment(\.keyboardPalette) private var palette + + static let height = DesignTokens.Metrics.voicebarHeight + + var body: some View { + ZStack { + if model.termDraft != nil { + TermField(model: model) + .transition(.opacity) + } else if !model.hasFullAccess { + // Without Full Access nothing here can work, and the user has to be + // told where to go. The line is the key: a tap opens Blurt, whose + // checklist says the rest. + Button { + model.openApp() + } label: { + Text("Allow Full Access in Settings") + .font(.footnote) + .foregroundStyle(palette.notice) + .lineLimit(1) + .minimumScaleFactor(0.8) // literal-ok: the note may shrink a little on a narrow phone + .frame(maxWidth: .infinity, maxHeight: .infinity) + } + .buttonStyle(KeyPressStyle()) + .accessibilityLabel("Start Blurt") + .transition(.opacity) + } else { + VoiceRow(model: model, slot: .bar, height: Self.height) + .transition(.opacity) + } + } + .frame(maxWidth: .infinity) + .frame(height: Self.height) + .animation(.easeInOut(duration: DesignTokens.Motion.termSwap), value: model.termDraft == nil) + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/VoiceElement.swift b/App/BlurtiOS/BlurtKeyboard/Sources/VoiceElement.swift new file mode 100644 index 00000000..9512cb81 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/VoiceElement.swift @@ -0,0 +1,103 @@ +import BlurtiOSCore +import SwiftUI + +/// Where the element sits, for the sizes that differ. +nonisolated enum VoiceSlot: String, CaseIterable, Sendable { + case bar + case panel + case home + + /// The box the element is given (`voice/*` tokens). + var box: CGSize { + let metrics = DesignTokens.Metrics.self + return switch self { + case .bar: CGSize(width: metrics.voiceBarWidth, height: metrics.voiceBarHeight) + case .panel: CGSize(width: metrics.voicePanelWidth, height: metrics.voicePanelHeight) + case .home: CGSize(width: metrics.voiceHomeWidth, height: metrics.voiceHomeHeight) + } + } +} + +extension VoiceElementKind { + /// How wide the element shows in a slot — what the + sits beside and the + /// pair is centred on. The slot's box is the most it takes (the ribs awake, + /// the streak); at rest the grille and the ribs are narrower, and a pair + /// centred on the box would put the + off at the edge. + func visibleWidth(slot: VoiceSlot, recording: Bool) -> CGFloat { + switch self { + case .grille: VoiceGrille.box(slot).width + case .ribs: recording ? slot.box.width : DesignTokens.Metrics.ribsRestWidth + case .streak: slot.box.width + } + } +} + +/// Every candidate draws with a synchronous `Canvas`: one that renders +/// asynchronously never presents inside a keyboard extension, and the +/// keyboard is where these live. +/// +/// Everything a candidate draws from — nothing else reaches it: the state, +/// the level inside it, the moment the words landed, whether motion is +/// allowed, the face's colours, and where it sits. +struct VoiceElementInputs { + let state: VoiceState + let landedAt: Date? + let animated: Bool + let palette: KeyboardPalette + let slot: VoiceSlot +} + +/// The seam: draws whichever kind the environment names. +struct VoiceElementView: View { + let inputs: VoiceElementInputs + @Environment(\.voiceElementKind) private var kind + + var body: some View { + switch kind { + case .grille: VoiceGrille(inputs: inputs) + case .ribs: VoiceRibs(inputs: inputs) + case .streak: VoiceStreak(inputs: inputs) + } + } +} + +private struct VoiceElementKindKey: EnvironmentKey { + static let defaultValue = VoiceElementKind.shipped +} + +extension EnvironmentValues { + var voiceElementKind: VoiceElementKind { + get { self[VoiceElementKindKey.self] } + set { self[VoiceElementKindKey.self] = newValue } + } +} + +/// The clocks the candidates share, so their motion cannot drift apart. +nonisolated enum VoiceClock { + /// The landing glint's progress, 0 → 1 over `motion.landing` after the + /// words landed; nil before and after. + static func landing(landedAt: Date?, now: Date) -> Double? { + guard let landedAt else { return nil } + let elapsed = now.timeIntervalSince(landedAt) + guard elapsed >= 0, elapsed < DesignTokens.Motion.landing else { return nil } + return elapsed / DesignTokens.Motion.landing + } + + /// Where the sheen's band is, 0 → 1 across the element and back to 0, on + /// the rest period or the working one. + static func sheen(time: TimeInterval, working: Bool) -> Double { + let period = working ? DesignTokens.Motion.sheenWorking : DesignTokens.Motion.sheen + return (time.truncatingRemainder(dividingBy: period)) / period + } + + /// A small deterministic number in 0…1 for a facet and a moment — the + /// glints are random to the eye and the same in every capture. + static func hash(_ index: Int, _ tick: Int) -> Double { + var state = UInt64( + truncatingIfNeeded: index &* 6_364_136_223_846_793_005 &+ tick &* 1_442_695_040_888_963_407 &+ 17) + state ^= state >> 29 + state = state &* 0x9E37_79B9_7F4A_7C15 + state ^= state >> 32 + return Double(state >> 40) / Double(1 << 24) + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/VoiceGrille.swift b/App/BlurtiOS/BlurtKeyboard/Sources/VoiceGrille.swift new file mode 100644 index 00000000..27e00751 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/VoiceGrille.swift @@ -0,0 +1,228 @@ +import BlurtEngine +import BlurtiOSCore +import SwiftUI + +/// Candidate A — the grille. A dot-matrix mic head: a lattice of small dots +/// in a rectangle with the brand's corners, reading as a mic grille, a +/// rhinestone mesh and the brand's halftone grain at once. It never changes +/// shape; only its dots light. At rest a slow sheen crosses it, the chrome +/// catching light. While recording the lattice is the meter: columns light +/// bottom-up with the level in stepped green (the brand's dithered gradient, +/// an LED VU), and on the peaks a few dots flash white — the facets +/// glistening. While the mic comes up or the words are made, one lit dot +/// runs the perimeter on the ring's cadence. When the words land a thin +/// white cross glint sweeps across it once; a failure turns it orange. +/// +/// Drawn as plain SwiftUI paths — never a `Canvas`. A Canvas has its own +/// render surface, and inside a keyboard extension that surface has failed +/// to present three separate times (asynchronous first; then in Messages, +/// whose host swaps scenes as the keyboard appears): the key was there, the +/// dots were not. Paths go through the ordinary view renderer with nothing +/// to lose. The frame is a handful of layers, one per colour, a few hundred +/// dots each. +struct VoiceGrille: View { + let inputs: VoiceElementInputs + + private typealias Metrics = DesignTokens.Metrics + + var body: some View { + let box = Self.box(inputs.slot) + TimelineView(.animation(minimumInterval: keyboardAnimationInterval, paused: !inputs.animated)) { timeline in + let now = inputs.animated ? timeline.date : Date(timeIntervalSinceReferenceDate: 0) + let layers = Self.layers(size: box, inputs: inputs, now: now) + ZStack { + ForEach(layers) { layer in + if layer.stroke > 0 { + layer.path.stroke(layer.colour.opacity(layer.opacity), lineWidth: layer.stroke) + } else { + layer.path.fill(layer.colour.opacity(layer.opacity)) + } + } + } + .frame(width: box.width, height: box.height) + } + .frame(width: box.width, height: box.height) + .opacity(inputs.state.isReady ? 1 : Metrics.opacityOff) + .animation(.easeInOut(duration: DesignTokens.Motion.stateFade), value: inputs.state.isReady) + .accessibilityHidden(true) + } + + static func box(_ slot: VoiceSlot) -> CGSize { + switch slot { + case .bar: CGSize(width: Metrics.grilleBarWidth, height: Metrics.grilleBarHeight) + case .panel: CGSize(width: Metrics.grillePanelWidth, height: Metrics.grillePanelHeight) + case .home: CGSize(width: Metrics.grilleHomeWidth, height: Metrics.grilleHomeHeight) + } + } + + /// One colour's worth of the frame: the dots (or the glint's lines) in it. + struct Layer: Identifiable { + let id: Int + var path = Path() + let colour: Color + let opacity: Double + /// A stroke width for the glint's lines; 0 fills. + var stroke: CGFloat = 0 + } + + /// The sheen's lift on a dot is quantised to this many steps, so the frame + /// is a few layers rather than one per dot; the lift is faint (0.22 at + /// most), and the steps are not seen. + private static let sheenSteps = 4 + + // swiftlint:disable:next function_body_length + static func layers(size: CGSize, inputs: VoiceElementInputs, now: Date) -> [Layer] { + let state = inputs.state + let palette = inputs.palette + let time = now.timeIntervalSinceReferenceDate + let pitch = Metrics.grillePitch + let dot = Metrics.grilleDot + let lattice = Self.lattice(size, pitch: pitch) + let columns = lattice.columns + let rows = lattice.rows + let origin = lattice.origin + // The lattice is the meter: the engine's envelope and idle wave give each + // column a height, quantised to whole dots from the bottom. + let meter = MeterBarRow(count: columns, availableHeight: size.height) + let level = state.isRecording ? state.level : 0 + let sheenPhase = VoiceClock.sheen(time: time, working: state.isWorking && !state.isRecording) + let sheenBand = size.width * Metrics.grilleSheenWidth + let sheenCentre = sheenPhase * (size.width + size.height + sheenBand) - sheenBand + let runner = + state.isWorking && !state.isRecording ? Self.runnerIndex(time: time, columns: columns, rows: rows) : nil + let (glintTick, glintPhase) = Self.glintClock(time) + let (restColour, restOpacity) = Self.rest(state: state, palette: palette) + // No sheen while Blurt isn't ready: the grille sits still as well as dim. + let sheen = inputs.animated && state.isReady && !state.isRecording ? (centre: sheenCentre, band: sheenBand) : nil + // The layers: rest, the sheen's steps over rest, lit (and the runner), the facets' flashes. + var rest = Layer(id: 0, colour: restColour, opacity: restOpacity) + var sheened = (1...sheenSteps).map { step in + Layer( + id: step, colour: restColour, + opacity: min(1, restOpacity + Metrics.opacitySheen * Double(step) / Double(sheenSteps))) + } + var lit = Layer(id: sheenSteps + 1, colour: palette.signal, opacity: 1) + var flash = Layer(id: sheenSteps + 2, colour: DesignTokens.Brand.white, opacity: 1 - glintPhase) + for column in 0.. 0.35 && VoiceClock.hash(column, glintTick) < Double(level) * 0.5 + for row in 0.. 0, fromBottom <= litCount { + // The top of a loud column glistens: a facet catches the light. + if fromBottom == litCount, glints { + flash.path.addEllipse(in: rect) + } else { + lit.path.addEllipse(in: rect) + } + } else if let runner, runner == Self.perimeterIndex(column: column, row: row, columns: columns, rows: rows) { + lit.path.addEllipse(in: rect) + } else if let sheen, abs(x + y - sheen.centre) < sheen.band / 2 { + // The sheen: a diagonal band of light passing over the lattice, + // half the band each side of its centre, in steps. + let lift = 1 - abs(x + y - sheen.centre) / (sheen.band / 2) + let step = max(1, min(sheenSteps, Int((lift * Double(sheenSteps)).rounded(.up)))) + sheened[step - 1].path.addEllipse(in: rect) + } else { + rest.path.addEllipse(in: rect) + } + } + } + var layers = [rest] + sheened + [lit, flash] + if let progress = VoiceClock.landing(landedAt: inputs.landedAt, now: now) { + layers += Self.glint(size: size, progress: progress, firstID: layers.count) + } + return layers.filter { !$0.path.isEmpty } + } + + /// How many dots fit, and where the first one sits so the lattice is centred. + private struct Lattice { + let columns: Int + let rows: Int + let origin: CGPoint + } + + private static func lattice(_ size: CGSize, pitch: CGFloat) -> Lattice { + let columns = max(1, Int(size.width / pitch)) + let rows = max(1, Int(size.height / pitch)) + return Lattice( + columns: columns, rows: rows, + origin: CGPoint( + x: (size.width - CGFloat(columns - 1) * pitch) / 2, y: (size.height - CGFloat(rows - 1) * pitch) / 2)) + } + + /// Which flash we are in, and how far through it. + private static func glintClock(_ time: TimeInterval) -> (Int, Double) { + let life = DesignTokens.Motion.glintLife + return (Int(time / life), time.truncatingRemainder(dividingBy: life) / life) + } + + /// The lattice's colour when nothing is lit: quiet at rest, green for a + /// landing, orange for a failure. + private static func rest(state: VoiceState, palette: KeyboardPalette) -> (Color, Double) { + switch state.phase { + case .error: (palette.notice, 1) + case .pasted, .copied: (palette.signal, 1) + case .idle, .connecting, .recording, .processing: (palette.keyText, Metrics.opacityGrilleRest) + } + } + + /// The perimeter dot the runner is on while something is happening. + private static func runnerIndex(time: TimeInterval, columns: Int, rows: Int) -> Int { + let perimeter = max(1, 2 * columns + 2 * rows - 4) + let phase = VoiceClock.sheen(time: time, working: true) + return Int(phase * Double(perimeter)) % perimeter + } + + /// A dot's place on the perimeter, clockwise from the top-left, or nil inside. + private static func perimeterIndex(column: Int, row: Int, columns: Int, rows: Int) -> Int? { + if row == 0 { return column } + if column == columns - 1 { return columns - 1 + row } + if row == rows - 1 { return columns - 1 + rows - 1 + (columns - 1 - column) } + if column == 0 { return 2 * (columns - 1) + rows - 1 + (rows - 1 - row) } + return nil + } + + /// The cross glint: a thin white four-point flare that sweeps left to right + /// as the words land, its vertical arm taller than the grille, with a + /// faint prismatic fringe — the photograph's lens flare, once. + private static func glint(size: CGSize, progress: Double, firstID: Int) -> [Layer] { + let alpha = Metrics.opacityGlint * (1 - progress) + let x = size.width * progress + let arm = size.height * Metrics.grilleGlintArm + let width = Metrics.grilleGlintWidth + let midY = size.height / 2 + func line(_ id: Int, _ from: CGPoint, _ to: CGPoint, colour: Color, opacity: Double) -> Layer { + var layer = Layer(id: id, colour: colour, opacity: opacity, stroke: width) + layer.path.move(to: from) + layer.path.addLine(to: to) + return layer + } + let fringe = alpha * 0.4 // literal-ok: the fringe is a fraction of the flare + let core = Metrics.grilleDot * 2 // literal-ok: the flare's centre is a double dot + var centre = Layer(id: firstID + 4, colour: DesignTokens.Brand.white, opacity: alpha) + let centreRect = CGRect(x: x - core / 2, y: midY - core / 2, width: core, height: core) // literal-ok: centred + centre.path.addEllipse(in: centreRect) + return [ + line( + firstID, CGPoint(x: x + width, y: midY - arm / 2), CGPoint(x: x + width, y: midY + arm / 2), + colour: DesignTokens.Brand.violetIris, opacity: fringe), + line( + firstID + 1, CGPoint(x: x - width, y: midY - arm / 2), CGPoint(x: x - width, y: midY + arm / 2), + colour: DesignTokens.Brand.green400, opacity: fringe), + line( + firstID + 2, CGPoint(x: x, y: midY - arm / 2), CGPoint(x: x, y: midY + arm / 2), + colour: DesignTokens.Brand.white, opacity: alpha), + // The horizontal arm sits back. + line( + firstID + 3, CGPoint(x: 0, y: midY), CGPoint(x: size.width, y: midY), colour: DesignTokens.Brand.white, + opacity: alpha * 0.7), + centre, + ] + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/VoiceRibs.swift b/App/BlurtiOS/BlurtKeyboard/Sources/VoiceRibs.swift new file mode 100644 index 00000000..43192bd5 --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/VoiceRibs.swift @@ -0,0 +1,106 @@ +import BlurtiOSCore +import SwiftUI + +/// Candidate B — the ribs. The mic at rest is a short block of vertical ribs: +/// the thin wave's own bars, still, under a symmetric envelope — the art-deco +/// grille of a Shure 55, the ribbon of an RCA 77. While recording the ribs +/// are the wave: they widen to the box and follow the voice. So there is no +/// circle anywhere, ever, and the element the recording shows is the one +/// that was there all along. A sheen passes over the ribs at rest; a white +/// glint runs them when the words land; a failure turns them orange. +struct VoiceRibs: View { + let inputs: VoiceElementInputs + + private typealias Metrics = DesignTokens.Metrics + + var body: some View { + let state = inputs.state + let box = inputs.slot.box + let awake = state.isRecording + let waveHeight: CGFloat = + switch inputs.slot { + case .bar: Metrics.waveBarHeight + case .panel: Metrics.wavePanelHeight + case .home: Metrics.waveHomeHeight + } + ZStack { + if awake { + WaveformMeter( + level: state.level, animated: inputs.animated, color: inputs.palette.signal, + barWidth: WaveformMeter.slimBar, barSpacing: WaveformMeter.slimGap + ) + .frame(width: box.width, height: waveHeight) + .transition(.opacity) + } else { + TimelineView(.animation(minimumInterval: keyboardAnimationInterval, paused: !inputs.animated)) { timeline in + let now = inputs.animated ? timeline.date : Date(timeIntervalSinceReferenceDate: 0) + Canvas { context, size in + Self.ribs(&context, size: size, inputs: inputs, now: now) + } + } + .frame(width: Metrics.ribsRestWidth, height: Metrics.ribsRestHeight) + .transition(.opacity) + } + } + .frame(width: box.width, height: box.height) + .animation(.easeInOut(duration: DesignTokens.Motion.waveFade), value: awake) + .opacity(state.isReady ? 1 : Metrics.opacityOff) + .animation(.easeInOut(duration: DesignTokens.Motion.stateFade), value: state.isReady) + .overlay { + TimelineView(.animation(minimumInterval: keyboardAnimationInterval, paused: !inputs.animated)) { timeline in + Canvas { context, size in + if let progress = VoiceClock.landing(landedAt: inputs.landedAt, now: timeline.date) { + Self.glint(&context, size: size, progress: progress) + } + } + } + .allowsHitTesting(false) + } + .accessibilityHidden(true) + } + + /// The ribs asleep: bars at the wave's pitch under a cosine envelope, in + /// the face's quiet legend colour (orange for a failure, green for a + /// landing), with the sheen passing over them. + private static func ribs(_ context: inout GraphicsContext, size: CGSize, inputs: VoiceElementInputs, now: Date) { + let state = inputs.state + let pitch = WaveformMeter.slimBar + WaveformMeter.slimGap + let count = max(1, Int((size.width + WaveformMeter.slimGap) / pitch)) + let inset = (size.width - CGFloat(count) * pitch + WaveformMeter.slimGap) / 2 // literal-ok: centred + let time = now.timeIntervalSinceReferenceDate + let sheen = VoiceClock.sheen(time: time, working: state.isWorking) + let colour: Color + switch state.phase { + case .error: colour = inputs.palette.notice + case .pasted, .copied: colour = inputs.palette.signal + default: colour = inputs.palette.keyTextSecondary + } + for index in 0.. Placement { + let mic = min(rowWidth, max(minMicWidth, shown)) + switch side { + case .leading: + let addX = mic + clearance + let room = max(0, rowWidth - addX) + return Placement( + micX: 0, micWidth: mic, addX: addX, addWidth: chip ? room : min(glyphHit, room), addGrowsTrailing: true, + cancelX: nil, paired: false) + case .trailing: + let micX = rowWidth - mic + let room = max(0, micX - clearance) + let width = chip ? room : min(glyphHit, room) + return Placement( + micX: micX, micWidth: mic, addX: room - width, addWidth: width, addGrowsTrailing: false, cancelX: nil, + paired: false) + case .center: + let micX = (rowWidth - mic) / 2 // literal-ok: centred + let addX = micX + mic + clearance + let room = max(0, rowWidth - addX) + if chip, room < chipMinWidth { + // Not enough beside a centred mic: the pair centres, the chip taking + // what the mic leaves. + return Placement( + micX: 0, micWidth: mic, addX: 0, addWidth: max(0, rowWidth - mic - clearance), addGrowsTrailing: true, + cancelX: nil, paired: true) + } + // The + never leaves the row: beside a mic wider than the row allows + // it moves in over the mic's edge, and so does its mirror. + let plusX = max(0, min(addX, rowWidth - glyphHit)) + let cancelX = max(0, min(micX - clearance - glyphHit, rowWidth - glyphHit)) + return Placement( + micX: micX, micWidth: mic, addX: chip ? addX : plusX, addWidth: chip ? room : glyphHit, addGrowsTrailing: true, + cancelX: cancelX, paired: false) + } + } +} + +/// The row itself: the mic key where `VoiceRowLayout` puts it, the add key +/// hanging off it and — where `cancels` — the cancel × mirroring the add key +/// while a dictation is in flight. Fills the width it is given; `height` is +/// the row's, and the mic key's touch target is that tall. +struct VoiceRow: View { + var model: KeyboardModel + let slot: VoiceSlot + let height: CGFloat + var cancels = false + @Environment(\.keyboardPalette) private var palette + @Environment(\.voiceElementKind) private var kind + @Environment(\.layoutDirection) private var direction + + var body: some View { + GeometryReader { geo in + let recording = model.voiceState.isRecording + let chip = model.selectedTerm != nil + let layout = VoiceRowLayout(rowWidth: geo.size.width, side: Self.side(model.micAlignment, in: direction)) + let placed = layout.place( + micWidth: kind.visibleWidth(slot: slot, recording: recording), minMicWidth: DesignTokens.Metrics.keyMinWidth, + chip: chip) + let cancel = cancels && model.voiceState.canCancel + ZStack(alignment: .leading) { + if placed.paired { + // The chip at its own width, capped — a frame would stretch to the + // cap and the pair would sit off centre. + HStack(spacing: layout.clearance) { + if cancel { cancelKey.transition(.opacity) } + MicControl(model: model, slot: slot, width: placed.micWidth, hitHeight: height) + CappedWidth(cap: placed.addWidth) { AddTermKey(model: model) } + } + .frame(maxWidth: .infinity) + } else { + MicControl(model: model, slot: slot, width: placed.micWidth, hitHeight: height) + .padding(.leading, placed.micX) + AddTermKey(model: model) + .frame(width: placed.addWidth, alignment: placed.addGrowsTrailing ? .leading : .trailing) + .padding(.leading, placed.addX) + if cancel, let cancelX = placed.cancelX { + cancelKey.padding(.leading, cancelX).transition(.opacity) + } + } + } + .frame(width: geo.size.width, height: geo.size.height, alignment: .leading) + .animation(.easeInOut(duration: DesignTokens.Motion.waveFade), value: recording) + .animation(.easeInOut(duration: DesignTokens.Motion.stateFade), value: chip) + .animation(.easeInOut(duration: DesignTokens.Motion.stateFade), value: cancel) + } + .frame(height: height) + } + + private var cancelKey: some View { + GlyphKey(symbol: "xmark", tint: palette.notice, label: "Cancel dictation") { model.cancel() } + } + + /// The row's side for the alignment Settings chose, as SwiftUI counts sides. + static func side(_ alignment: MicAlignment, in direction: LayoutDirection) -> VoiceRowLayout.Side { + switch alignment.alignment(in: direction) { + case .leading: .leading + case .trailing: .trailing + default: .center + } + } +} + +/// A bare glyph as a key — the +, × and ✓: an SF Symbol at `size/glyph` in a +/// `glyph/hit` square (the HIG's 44 pt), no cap, lighting while pressed. One +/// shape wherever a glyph is a key, so the row's furniture matches. +struct GlyphKey: View { + let symbol: String + let tint: Color + let label: String + let action: () -> Void + + var body: some View { + Button(action: action) { + Image(systemName: symbol) + .font(.system(size: DesignTokens.Typography.sizeGlyph, weight: DesignTokens.Typography.weightGlyph)) + .foregroundStyle(tint) + .contentTransition(.symbolEffect(.replace)) + .frame(width: DesignTokens.Metrics.glyphHit, height: DesignTokens.Metrics.glyphHit) + .contentShape(Rectangle()) + } + .buttonStyle(KeyPressStyle()) + .accessibilityLabel(label) + } +} + +/// One view at its own width, but never wider than `cap`: what a chip needs +/// when it is centred with the mic as a pair. (`frame(maxWidth:)` is greedy — +/// it takes the cap whatever the content wants.) +struct CappedWidth: Layout { + let cap: CGFloat + + func sizeThatFits(proposal: ProposedViewSize, subviews: Subviews, cache: inout ()) -> CGSize { + guard let subview = subviews.first else { return .zero } + let ideal = subview.sizeThatFits(.unspecified) + let width = min(ideal.width, cap, proposal.width ?? .infinity) + return subview.sizeThatFits(ProposedViewSize(width: width, height: proposal.height)) + } + + func placeSubviews(in bounds: CGRect, proposal: ProposedViewSize, subviews: Subviews, cache: inout ()) { + subviews.first?.place(at: bounds.origin, anchor: .topLeading, proposal: ProposedViewSize(bounds.size)) + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/VoiceStreak.swift b/App/BlurtiOS/BlurtKeyboard/Sources/VoiceStreak.swift new file mode 100644 index 00000000..cd755bed --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/VoiceStreak.swift @@ -0,0 +1,116 @@ +import BlurtiOSCore +import SwiftUI + +/// Candidate C — the streak. The flare as the meter: at rest a hairline the +/// width of the box with one bright point at its centre (the photograph's +/// star). While recording the point blooms into a horizontal light streak +/// whose reach and brightness follow the level, with a prismatic fringe at +/// its ends. While something is happening a short streak sweeps the line. +/// When the words land the vertical arm of the cross flashes through once. +/// Nothing else. (A line that grows can read as a loading bar — the risk +/// this candidate carries, on purpose, so it is seen and not argued.) +struct VoiceStreak: View { + let inputs: VoiceElementInputs + + private typealias Metrics = DesignTokens.Metrics + + var body: some View { + let box = inputs.slot.box + TimelineView(.animation(minimumInterval: keyboardAnimationInterval, paused: !inputs.animated)) { timeline in + let now = inputs.animated ? timeline.date : Date(timeIntervalSinceReferenceDate: 0) + Canvas { context, size in + Self.draw(&context, size: size, inputs: inputs, now: now) + } + } + .frame(width: box.width, height: box.height) + .opacity(inputs.state.isReady ? 1 : Metrics.opacityOff) + .animation(.easeInOut(duration: DesignTokens.Motion.stateFade), value: inputs.state.isReady) + .accessibilityHidden(true) + } + + private static func draw(_ context: inout GraphicsContext, size: CGSize, inputs: VoiceElementInputs, now: Date) { + let state = inputs.state + let palette = inputs.palette + let midY = size.height / 2 + let lineColour: Color = + switch state.phase { + case .error: palette.notice + case .pasted, .copied: palette.signal + case .idle, .connecting, .recording, .processing: palette.keyTextSecondary + } + // The hairline. + let line = Metrics.streakLine + context.fill( + Path(CGRect(x: 0, y: midY - line / 2, width: size.width, height: line)), // literal-ok: centred on the line + with: .color(lineColour.opacity(Metrics.opacityHairline))) + if state.isRecording { + streak(&context, size: size, level: state.level, palette: palette) + } else if state.isWorking, inputs.animated { + sweep(&context, size: size, time: now.timeIntervalSinceReferenceDate, palette: palette) + } + // The point. + let point = Metrics.streakPoint + let half = point / 2 + let dot = CGRect(x: size.width / 2 - half, y: midY - half, width: point, height: point) // literal-ok: centred + context.fill(Path(ellipseIn: dot), with: .color(state.isRecording ? DesignTokens.Brand.white : palette.keyText)) + if let progress = VoiceClock.landing(landedAt: inputs.landedAt, now: now) { + flash(&context, size: size, progress: progress) + } + } + + /// The streak: from the centre, both ways, as far as the voice reaches, + /// white at the heart, the signal along it, a violet fringe at the tips. + private static func streak(_ context: inout GraphicsContext, size: CGSize, level: Float, palette: KeyboardPalette) { + let midY = size.height / 2 + let reach = size.width * CGFloat(pow(Double(level), 0.7)) // literal-ok: the ear's curve + let thick = Metrics.streakHeight + let x = (size.width - reach) / 2 + let rect = CGRect(x: x, y: midY - thick / 2, width: reach, height: thick) // literal-ok: centred + context.fill( + Path(roundedRect: rect, cornerRadius: thick / 2), // literal-ok: capsule ends + with: .linearGradient( + Gradient(stops: [ + .init(color: DesignTokens.Brand.violetPeriwinkle.opacity(0), location: 0), + .init(color: palette.signal, location: 0.35), // literal-ok: where the fringe gives way to the signal + .init(color: DesignTokens.Brand.white, location: 0.5), // literal-ok: the centre + .init(color: palette.signal, location: 0.65), // literal-ok: mirrored + .init(color: DesignTokens.Brand.violetPeriwinkle.opacity(0), location: 1), + ]), + startPoint: CGPoint(x: rect.minX, y: midY), endPoint: CGPoint(x: rect.maxX, y: midY))) + } + + /// A short streak sweeping the line back and forth while the mic comes up + /// or the words are made. + private static func sweep( + _ context: inout GraphicsContext, size: CGSize, time: TimeInterval, palette: KeyboardPalette + ) { + let midY = size.height / 2 + let phase = VoiceClock.sheen(time: time, working: true) + let position = phase < 0.5 ? phase * 2 : 2 - phase * 2 // literal-ok: there and back + let length = size.width * 0.25 // literal-ok: a quarter of the line + let x = (size.width - length) * position + let thick = Metrics.streakHeight + let rect = CGRect(x: x, y: midY - thick / 2, width: length, height: thick) // literal-ok: centred + context.fill( + Path(roundedRect: rect, cornerRadius: thick / 2), // literal-ok: capsule ends + with: .linearGradient( + Gradient(colors: [palette.signal.opacity(0), palette.signal, palette.signal.opacity(0)]), + startPoint: CGPoint(x: rect.minX, y: midY), endPoint: CGPoint(x: rect.maxX, y: midY))) + } + + /// The landing: the cross's vertical arm flashes through the point, and + /// the line lights with it. + private static func flash(_ context: inout GraphicsContext, size: CGSize, progress: Double) { + let midY = size.height / 2 + let alpha = Metrics.opacityGlint * (1 - progress) + let arm = size.height * Metrics.streakArm + var path = Path() + path.move(to: CGPoint(x: size.width / 2, y: midY - arm / 2)) + path.addLine(to: CGPoint(x: size.width / 2, y: midY + arm / 2)) + context.stroke(path, with: .color(DesignTokens.Brand.white.opacity(alpha)), lineWidth: Metrics.grilleGlintWidth) + let line = Metrics.streakLine + context.fill( + Path(CGRect(x: 0, y: midY - line / 2, width: size.width, height: line)), // literal-ok: centred on the line + with: .color(DesignTokens.Brand.white.opacity(alpha))) + } +} diff --git a/App/BlurtiOS/BlurtKeyboard/Sources/WaveformMeter.swift b/App/BlurtiOS/BlurtKeyboard/Sources/WaveformMeter.swift new file mode 100644 index 00000000..0d5fa14b --- /dev/null +++ b/App/BlurtiOS/BlurtKeyboard/Sources/WaveformMeter.swift @@ -0,0 +1,51 @@ +import BlurtEngine +import BlurtiOSCore +import SwiftUI + +/// The Mac pill's live meter: a row of bars that fills the width it is given +/// and tracks the current level, with the engine's envelope and idle wave +/// (`MeterBarGeometry`, unit-tested there). The keyboard hears the level from +/// the app at ~12 Hz; the wave keeps the row alive between ticks. The pitch +/// is the Mac's unless the caller sets its own: the keyboard's wave is thin — +/// `slimBar` wide, `slimGap` apart — and dense, a fine signal rather than a +/// bank of bars. +struct WaveformMeter: View { + let level: Float + let animated: Bool + let color: Color + var barWidth: CGFloat = MeterBarGeometry.barWidth + var barSpacing: CGFloat = MeterBarGeometry.barSpacing + + /// The keyboard's pitch: 2 pt bars, 2 pt apart. + static let slimBar = DesignTokens.Metrics.waveBar + static let slimGap = DesignTokens.Metrics.waveGap + + var body: some View { + GeometryReader { geo in + let count = Int((geo.size.width + barSpacing) / (barWidth + barSpacing)) + let layout = MeterBarRow(count: count, availableHeight: geo.size.height) + Group { + if animated { + TimelineView(.animation(minimumInterval: keyboardAnimationInterval)) { timeline in + bars(layout: layout, time: timeline.date.timeIntervalSinceReferenceDate) + } + } else { + bars(layout: layout, time: 0) + } + } + .frame(width: geo.size.width, height: geo.size.height) + } + .accessibilityHidden(true) + } + + private func bars(layout: MeterBarRow, time: TimeInterval) -> some View { + HStack(spacing: barSpacing) { + ForEach(0.. + + + + NSPrivacyTracking + + NSPrivacyTrackingDomains + + NSPrivacyAccessedAPITypes + + + NSPrivacyAccessedAPIType + NSPrivacyAccessedAPICategoryUserDefaults + NSPrivacyAccessedAPITypeReasons + + + 1C8F.1 + + CA92.1 + + + + + diff --git a/App/BlurtiOS/BlurtiOS/Sources/AppChrome.swift b/App/BlurtiOS/BlurtiOS/Sources/AppChrome.swift new file mode 100644 index 00000000..2dabcfda --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/AppChrome.swift @@ -0,0 +1,134 @@ +import SwiftUI +import UIKit + +/// The app's chrome, on the brand: the appearance-adaptive colours from the +/// asset catalog (generated from the tokens — page, text, muted, the card, +/// the one green button), the card, the eyebrow, the button, the wordmark. +/// The keyboard never uses these: it has its own faces. +extension BlurtBrand { + /// `green` in light, the lifted green in dark — the catalog's `AccentColor`, + /// also the app-level accent (`project.yml`). + nonisolated static let accent = Color.accentColor + /// The page: the design system's warm off-white in light, ink in dark. + nonisolated static let page = Color("Page") + nonisolated static let text = Color("Text") + /// Eyebrows, captions, secondary lines. + nonisolated static let muted = Color("Muted") + /// The card: warm tint on paper, a step up from ink in dark, with a hairline. + nonisolated static let cardFill = Color("CardFill") + nonisolated static let cardBorder = Color("CardBorder") + /// The one green button per row, and its text. + nonisolated static let cta = Color("CTA") + nonisolated static let ctaText = Color("CTAText") +} + +extension View { + /// The brand's card: the warm fill, 12 pt corners, a 1 pt hairline, no shadow. + func card(radius: CGFloat = DesignTokens.Metrics.cardRadius) -> some View { + background(BlurtBrand.cardFill, in: RoundedRectangle(cornerRadius: radius)) + .overlay( + RoundedRectangle(cornerRadius: radius).strokeBorder( + BlurtBrand.cardBorder, lineWidth: DesignTokens.Metrics.cardBorder)) + } + + /// The page under a screen: the ground colour with the brand's film grain over it. + func page() -> some View { + background { + ZStack { + BlurtBrand.page + BlurtiOSGrain(seed: 7) + .opacity(DesignTokens.Metrics.opacityGrainDark) + .blendMode(.overlay) + .allowsHitTesting(false) + } + .ignoresSafeArea() + } + } +} + +extension View { + /// A form or list on the brand: the page under it, the card as each row's + /// ground, the body face for its text, the accent for its controls. + func brandForm() -> some View { + scrollContentBackground(.hidden) + .page() + .listRowBackground(BlurtBrand.cardFill) + .font(BlurtType.body(DesignTokens.Typography.sizeBody)) + .foregroundStyle(BlurtBrand.text) + .tint(BlurtBrand.accent) + } +} + +/// An eyebrow: the system mono, uppercase, tracked, muted — the brand's +/// section label, above every group on a screen. +struct Eyebrow: View { + let text: String + var color = BlurtBrand.muted + + init(_ text: String, color: Color = BlurtBrand.muted) { + self.text = text + self.color = color + } + + var body: some View { + Text(text.uppercased()) + .font(BlurtType.mono(DesignTokens.Typography.sizeEyebrow, weight: .medium)) + .tracking(DesignTokens.Typography.trackingEyebrow) + .foregroundStyle(color) + } +} + +/// The brand's button: 40 pt, 4 pt corners — rectangular, never a pill — +/// its label in uppercase mono; a colour change while pressed, no lift. +/// `primary` is the one green button per row; `secondary` a hairline. +struct BrandButtonStyle: ButtonStyle { + enum Role { + case primary + case secondary + } + + var role = Role.primary + + func makeBody(configuration: Configuration) -> some View { + configuration.label + .font(BlurtType.mono(DesignTokens.Typography.sizeCta, weight: .medium)) + .tracking(DesignTokens.Typography.trackingCta) + .textCase(.uppercase) + .foregroundStyle(role == .primary ? BlurtBrand.ctaText : BlurtBrand.text) + .padding(.horizontal, DesignTokens.Metrics.appButtonPad) + .frame(maxWidth: .infinity, minHeight: DesignTokens.Metrics.appButtonHeight) + .background( + role == .primary ? BlurtBrand.cta : Color.clear, + in: RoundedRectangle(cornerRadius: DesignTokens.Metrics.radiusButton) + ) + .overlay( + RoundedRectangle(cornerRadius: DesignTokens.Metrics.radiusButton) + .strokeBorder( + role == .secondary ? BlurtBrand.cardBorder : Color.clear, lineWidth: DesignTokens.Metrics.cardBorder) + ) + .opacity(configuration.isPressed ? DesignTokens.Metrics.opacityPressed : 1) + .animation(.easeInOut(duration: DesignTokens.Motion.colour), value: configuration.isPressed) + } +} + +/// The lowercase wordmark from the Mac's ready screen, tinted with the accent +/// (`blurt-ready-logo.png`, shared from `App/Blurt/Blurt/Branding`). +struct Wordmark: View { + private static let height = DesignTokens.Metrics.wordmarkHeight + + var body: some View { + if let image = UIImage(named: "blurt-ready-logo") { + Image(uiImage: image) + .renderingMode(.template) + .resizable() + .scaledToFit() + .frame(height: Self.height) + .foregroundStyle(BlurtBrand.accent) + .accessibilityLabel("Blurt") + } else { + Text("blurt") + .font(BlurtType.heading(DesignTokens.Typography.sizeTitle)) + .foregroundStyle(BlurtBrand.accent) + } + } +} diff --git a/App/BlurtiOS/BlurtiOS/Sources/AudioSource.swift b/App/BlurtiOS/BlurtiOS/Sources/AudioSource.swift new file mode 100644 index 00000000..cd697647 --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/AudioSource.swift @@ -0,0 +1,228 @@ +import AVFoundation +import BlurtEngine +import CoreMedia +import Foundation + +/// The microphone behind an iOS dictation: an audio-only capture session that +/// stays open for the whole listening window and hands each utterance its own +/// feed. +/// +/// This is deliberately not the Mac's fresh-session-per-press `MicCapture`. +/// iOS refuses to open the microphone from the background, and during a +/// dictation the app *is* in the background — the user is in the app they are +/// typing in, with the Blurt keyboard up. So the app opens the microphone once, +/// while it is in front (the `blurt://start` hop the keyboard makes), keeps it +/// open for the window, and a press simply starts forwarding the frames that +/// are already flowing. What that buys beyond legality is the fastest start +/// the engine has ever had: no device open, no liveness wait, `.recording` +/// within a frame of the press. What it costs is the orange microphone +/// indicator for the length of the window, which is the trade every iPhone +/// dictation keyboard makes. +/// +/// The output's native format is converted to the 16 kHz mono 16-bit the +/// dictation API wants by `PCMConverter` (`audioSettings`, which does that on +/// the Mac, is not in the iOS SDK). `@unchecked Sendable` by confinement, the +/// same way the Mac recorder is: the feed and its tally live behind +/// `UtteranceFeed`'s lock, the converter is touched only on the serial delivery +/// queue, and the session is configured, started and stopped only on the +/// serial control queue. +nonisolated final class WindowedAudioSource: NSObject, ListeningSource, @unchecked Sendable { + private let feed = UtteranceFeed() + private let session = AVCaptureSession() + private let output = AVCaptureAudioDataOutput() + /// Serial, so the converter below needs no lock. + private let deliveryQueue = DispatchQueue(label: "dev.alex.blurt.ios.capture") + /// Where the session is started and stopped — both block for a while on a + /// phone — so the main actor never waits on the capture stack. + private let controlQueue = DispatchQueue(label: "dev.alex.blurt.ios.capture.control") + private nonisolated(unsafe) var converter: PCMConverter? + + var levels: AsyncStream { feed.levels } + + override init() { + super.init() + output.setSampleBufferDelegate(self, queue: deliveryQueue) + } + + /// Running and not taken away: iOS keeps a session "running" through an + /// interruption (a phone call, Siri), while no frames arrive. + var isOpen: Bool { session.isRunning && !session.isInterrupted } + + /// Opens the microphone. Only works from the foreground; the audio session + /// must already be active (`ListeningWindow` does both, in that order). + func open() async throws { + try await withCheckedThrowingContinuation { (continuation: CheckedContinuation) in + controlQueue.async { + do { + try self.openOnControlQueue() + continuation.resume() + } catch { + continuation.resume(throwing: error) + } + } + } + } + + private func openOnControlQueue() throws { + // A session left "running" through an interruption resumes on its own + // terms; stopping it first makes the reopen deterministic. + if session.isRunning, session.isInterrupted { session.stopRunning() } + guard !session.isRunning else { return } + try configure() + session.startRunning() + guard session.isRunning else { throw ListeningSourceFailure.noInputDevice } + } + + /// Adds the input and output once. `commitConfiguration` runs on every + /// path out, including a throw, so a failed attempt never leaves the + /// session mid-configuration for the next one. + private func configure() throws { + session.beginConfiguration() + defer { session.commitConfiguration() } + session.automaticallyConfiguresApplicationAudioSession = false + // A session refused its input or output would still start running — and + // hear nothing, which is worse than failing: the window would open, the + // keyboard would record, and the upload would get no frames. Refused is + // no microphone, said where the user can see it. + if session.inputs.isEmpty { + guard let device = AVCaptureDevice.default(for: .audio) else { throw ListeningSourceFailure.noInputDevice } + let input = try AVCaptureDeviceInput(device: device) + guard session.canAddInput(input) else { throw ListeningSourceFailure.noInputDevice } + session.addInput(input) + } + if session.outputs.isEmpty { + guard session.canAddOutput(output) else { throw ListeningSourceFailure.noInputDevice } + session.addOutput(output) + } + } + + /// Releases the microphone and ends any utterance in flight. + func close() async { + await withCheckedContinuation { (continuation: CheckedContinuation) in + controlQueue.async { + if self.session.isRunning { self.session.stopRunning() } + self.feed.end() + continuation.resume() + } + } + } + + // MARK: MicCaptureProtocol + + func start() throws -> AsyncStream { + // An interrupted session is "running" and delivering nothing: a feed from + // it would claim `.recording` over a dead mic, which the engine's timing + // contract forbids. + guard session.isRunning, !session.isInterrupted else { + throw BlurtError.audioCaptureFailed(underlying: ListeningSourceFailure.windowClosed) + } + return feed.begin() + } + + func stop() -> Int { feed.end() } + + func cancelCapture() { feed.end() } +} + +// The conformance is spelled `nonisolated`: with the target defaulting to the +// main actor, an extension's conformance is inferred main-actor even on a +// `nonisolated` class, and the output calls it on the delivery queue. +extension WindowedAudioSource: nonisolated AVCaptureAudioDataOutputSampleBufferDelegate { + func captureOutput( + _ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection + ) { + if converter == nil { converter = PCMConverter() } + guard let converter, let chunk = converter.convert(sampleBuffer) else { return } + feed.deliver(chunk) + } +} + +/// Converts whatever format the microphone delivers into the dictation API's +/// 16 kHz mono 16-bit little-endian PCM. One instance per delivery queue; it +/// rebuilds its converter if the input format changes underneath it (a route +/// change to AirPods, say). +nonisolated final class PCMConverter { + /// One buffer per `convert` call: the first pull gets it, every later pull + /// is told there is no more for now. The converter's input block is + /// `@Sendable` and escaping in type but runs synchronously on this thread + /// before `convert` returns, so an unchecked box is sound — and unlike a + /// `Mutex`, it can be captured. + private final class Handoff: @unchecked Sendable { + private var buffer: AVAudioPCMBuffer? + + init(_ buffer: AVAudioPCMBuffer) { self.buffer = buffer } + + func take() -> AVAudioPCMBuffer? { + defer { buffer = nil } + return buffer + } + } + + private let outputFormat: AVAudioFormat + private var inputFormat: AVAudioFormat? + private var converter: AVAudioConverter? + + init?() { + guard + let format = AVAudioFormat( + commonFormat: .pcmFormatInt16, sampleRate: Double(SyncSTTLimits.sampleRate), channels: 1, + interleaved: true) + else { return nil } + outputFormat = format + } + + func convert(_ sampleBuffer: CMSampleBuffer) -> Data? { + guard let description = CMSampleBufferGetFormatDescription(sampleBuffer), + let streamDescription = CMAudioFormatDescriptionGetStreamBasicDescription(description) + else { return nil } + let frames = AVAudioFrameCount(CMSampleBufferGetNumSamples(sampleBuffer)) + guard frames > 0, let input = inputBuffer(matching: streamDescription, frames: frames) else { return nil } + input.frameLength = frames + let copied = CMSampleBufferCopyPCMDataIntoAudioBufferList( + sampleBuffer, at: 0, frameCount: Int32(frames), into: input.mutableAudioBufferList) + guard copied == noErr, let converter, let inputFormat else { return nil } + let ratio = outputFormat.sampleRate / inputFormat.sampleRate + let capacity = AVAudioFrameCount((Double(frames) * ratio).rounded(.up)) + 64 + guard let outputBuffer = AVAudioPCMBuffer(pcmFormat: outputFormat, frameCapacity: capacity) else { + return nil + } + let handoff = Handoff(input) + var conversionError: NSError? + let status = converter.convert(to: outputBuffer, error: &conversionError) { _, inputStatus in + guard let buffer = handoff.take() else { + inputStatus.pointee = .noDataNow + return nil + } + inputStatus.pointee = .haveData + return buffer + } + guard status != .error, conversionError == nil, outputBuffer.frameLength > 0, + let channels = outputBuffer.int16ChannelData + else { return nil } + return Data(bytes: channels[0], count: Int(outputBuffer.frameLength) * MemoryLayout.size) + } + + /// A buffer in the input's own format, rebuilding the converter when the + /// format is new or changed. + private func inputBuffer( + matching streamDescription: UnsafePointer, frames: AVAudioFrameCount + ) -> AVAudioPCMBuffer? { + if inputFormat == nil || !Self.matches(inputFormat, streamDescription.pointee) { + guard let format = AVAudioFormat(streamDescription: streamDescription) else { return nil } + inputFormat = format + converter = AVAudioConverter(from: format, to: outputFormat) + } + guard let inputFormat else { return nil } + return AVAudioPCMBuffer(pcmFormat: inputFormat, frameCapacity: frames) + } + + private static func matches(_ format: AVAudioFormat?, _ description: AudioStreamBasicDescription) -> Bool { + guard let format else { return false } + let current = format.streamDescription.pointee + return current.mSampleRate == description.mSampleRate + && current.mChannelsPerFrame == description.mChannelsPerFrame + && current.mFormatID == description.mFormatID + && current.mFormatFlags == description.mFormatFlags + && current.mBitsPerChannel == description.mBitsPerChannel + } +} diff --git a/App/BlurtiOS/BlurtiOS/Sources/BlurtiOSApp.swift b/App/BlurtiOS/BlurtiOS/Sources/BlurtiOSApp.swift new file mode 100644 index 00000000..e8dd7066 --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/BlurtiOSApp.swift @@ -0,0 +1,77 @@ +import BlurtEngine +import BlurtiOSCore +import SwiftUI + +@main +struct BlurtiOSApp: App { + @State private var coordinator: DictationCoordinator + + /// The composition root, and the first thing that runs: the engine's identity + /// has to be set before any engine type is touched (`HostIdentity.configure` + /// documents why), and the coordinator below touches the Keychain in its + /// initializer. The identity is the iPhone app's own — its Keychain item is + /// separate from the Mac app's by construction, since Keychains are per + /// device, but naming it separately keeps the two builds' data apart the day + /// they share an iCloud Keychain. The defaults prefix stays `Blurt` so the + /// engine's stores read and write the same keys they do on the Mac. + init() { + HostIdentity.configure( + HostIdentity( + productName: "Blurt", + subsystem: "dev.alex.blurt.ios", + keychainService: "blurt-ios", + defaultsPrefix: "Blurt", + logDirectoryName: "Blurt")) + // The unit tests are hosted by this app: they get a coordinator that + // touches neither the Keychain nor the App Group (the Mac's UITestSupport + // does the same for its harness), and `start()` below never runs. + let coordinator = + Self.isTestHost + ? DictationCoordinator(apiKey: APIKeyModel(keyStore: InMemoryAPIKeyStore())) : DictationCoordinator() + _coordinator = State(initialValue: coordinator) + } + + private static var isTestHost: Bool { + ProcessInfo.processInfo.environment["XCTestConfigurationFilePath"] != nil + } + + var body: some Scene { + WindowGroup { + #if DEBUG + let rows = KeyboardGalleryView.rows(from: CommandLine.arguments) + if let probe = KeyboardProbeView.parse(CommandLine.arguments) { + // The keyboard over this field is only live if the app runs as it + // does under the home screen: the coordinator up, listening on request. + probe.task { await start() } + } else if let voice = KeyboardGalleryView.VoiceStillView.parse(CommandLine.arguments) { + voice + } else if CommandLine.arguments.contains("-BlurtSettings") { + // The settings sheet as the root, for a screenshot without a tap. + SettingsView(coordinator: coordinator) + } else if rows.isEmpty { + home + } else { + KeyboardGalleryView(rows: rows, options: .parse(CommandLine.arguments)) + } + #else + home + #endif + } + } + + private var home: some View { + HomeView(coordinator: coordinator) + .onOpenURL { coordinator.handle($0) } + .task { await start() } + } + + private func start() async { + guard !Self.isTestHost else { return } + coordinator.start() + #if DEBUG + // `-BlurtStartListening` opens the mic at launch, so the listening + // state can be screenshotted without a tap (see scripts/ios-sim.sh). + if CommandLine.arguments.contains("-BlurtStartListening") { await coordinator.startListening() } + #endif + } +} diff --git a/App/BlurtiOS/BlurtiOS/Sources/DictationCoordinator.swift b/App/BlurtiOS/BlurtiOS/Sources/DictationCoordinator.swift new file mode 100644 index 00000000..738df78f --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/DictationCoordinator.swift @@ -0,0 +1,292 @@ +import AVFoundation +import BlurtEngine +import BlurtiOSCore +import Foundation +import Observation +import Synchronization + +/// The one place the engine is composed for the iPhone app — the role +/// `AppCoordinator` plays in the Mac shell. It owns the listening window (the +/// microphone), the session, and the two channels to the keyboard: commands +/// coming in over the App Group, phase and results going back out. +/// +/// Key terms live in the App Group (`SharedStore.keyTerms`) rather than the +/// engine's `KeyTermsStore` (which reads the process's own defaults), so the +/// keyboard can add one on the spot; `keyTermsProvider` is how they reach the +/// request. +@MainActor +@Observable +final class DictationCoordinator { + let window: ListeningWindow + let apiKey: APIKeyModel + private(set) var phase: PipelinePhase = .idle + private(set) var recent = RecentDictations() + private(set) var level: Float = 0 + private(set) var microphoneDenied = false + /// How many contact names the keyboard last read off the phone — the key + /// terms that come for free, so the home screen can say so. + private(set) var lexiconNameCount = 0 + /// A shared key-term list the app was asked to open, until the user has + /// answered it (`ImportTermsView`). + var pendingTermPack: TermPack? + /// A file or link that claimed to be a key-term list and wasn't. + var termPackUnreadable = false + /// The window can't open without a key: the last "Start listening" said so. + var needsKey = false + + @ObservationIgnored private let session: DictationSession + @ObservationIgnored private let recents: AsyncStream + @ObservationIgnored private var observers: [DarwinObserver] = [] + @ObservationIgnored private var tasks: [Task] = [] + @ObservationIgnored private var lastCommandID: UUID? + @ObservationIgnored private var lastLevelPublish = ContinuousClock.now + + /// What the keyboard saw around the cursor on the press that started the + /// dictation in flight, read by the engine's context seam — and cleared once + /// used, so nothing from another app's field primes a later request. + nonisolated private static let pressContext = Mutex(.empty) + /// The keyboard process that pressed, for the words to go back to it. + nonisolated private static let pressKeyboard = Mutex(nil) + /// Contact names, parsed once per lexicon refresh rather than on the press. + nonisolated private static let lexiconNames = Mutex<[String]>([]) + + init(apiKey: APIKeyModel = APIKeyModel()) { + let window = ListeningWindow() + self.window = window + self.apiKey = apiKey + let (recents, continuation) = AsyncStream.makeStream( + of: RecentDictations.self, bufferingPolicy: .bufferingNewest(1)) + self.recents = recents + session = DictationSession( + mic: window.source, + transcriber: AssemblyAITranscriber(), + injector: KeyboardRelayInjector(presser: { Self.pressKeyboard.withLock { $0 } }), + keyTermsProvider: { Self.keyTerms() }, + readinessCheck: apiKey.readinessCheck(), + onTranscriptDelivered: { _, ring in continuation.yield(ring) }, + // The keyboard is the only thing that can see the field: the press + // command carries the text before the cursor, and that is what primes + // the request — the same signal the Mac reads through Accessibility. + hostFocusCapture: DictationSession.HostFocusCapture( + frontmost: { nil }, field: { Self.fieldContext() })) + window.onClosingOnItsOwn = { [weak self] in await self?.windowClosingOnItsOwn() } + } + + func start() { + // `.task` on the root view runs this once; a scene reconnect must not + // start a second set of observers publishing every phase twice. + guard tasks.isEmpty else { return } + observers = [ + DarwinObserver(name: BlurtShared.Signal.command) { [weak self] in + Task { @MainActor in self?.handleCommand() } + }, + DarwinObserver(name: BlurtShared.Signal.lexicon) { [weak self] in + Task { @MainActor in self?.reloadLexicon() } + }, + ] + reloadLexicon() + tasks = [observePhases(), observeLevels(), observeRecents()] + // Whatever the last run left in the App Group is over. + publish(.idle) + } + + // MARK: - The listening window + + /// Opens the microphone for the keyboard. Foreground only — this is what + /// `blurt://start` arrives to do. Not without a key: an open window whose + /// every press fails quietly would look like a keyboard that ignores taps. + func startListening() async { + apiKey.refreshStatus() + needsKey = !apiKey.hasAPIKey + guard !needsKey else { return } + guard await AVAudioApplication.requestRecordPermission() else { + microphoneDenied = true + return + } + microphoneDenied = false + await window.open() + } + + /// Closes the window. A dictation in flight is cancelled first, upload and + /// all — closing the feed alone would end the utterance and let the request + /// complete, transcribing (and billing) audio nobody asked for. + func stopListening() async { + if !phase.isTerminal { await session.cancel() } + await window.close() + publish(.idle) + } + + /// The window is closing without the user — its time ran out, or a call or + /// Siri took the microphone. Unlike `stopListening`, what was already said + /// still counts, so this releases rather than cancels — and leaves the + /// deciding to the session. Its commands run in order and a release checks + /// the session's own phase on its turn: a recording is released and its + /// words land (on the clipboard when no keyboard is up); a mic still coming + /// up finishes, then releases a capture too short to transcribe; anything + /// later is untouched. Not this coordinator's `phase`, a copy that trails + /// the session and could still say `.connecting` over a live recording. + /// Awaited before the feed closes, so the release still finds the audio. + private func windowClosingOnItsOwn() async { + await session.release() + } + + func handle(_ url: URL) { + if TermPack.looksLikePack(url) { + if let pack = TermPack.from(url) { + pendingTermPack = pack + } else { + termPackUnreadable = true + } + return + } + guard url.scheme == BlurtShared.urlScheme, url.host() == BlurtShared.startHost else { return } + Task { await startListening() } + } + + /// A dictation started from the app itself, with no keyboard to land in — + /// the words go to the clipboard. The way to try Blurt before the keyboard is + /// set up, and the way to test the pipeline without one. No field, so no + /// prior text primes it. + func toggleDictation() { + Self.pressContext.withLock { $0 = .empty } + Self.pressKeyboard.withLock { $0 = nil } + session.submit(phase.isCapturing ? .release : .press) + } + + // MARK: - Commands from the keyboard + + private func handleCommand() { + guard let command = SharedStore.read(KeyboardCommand.self, forKey: BlurtShared.Key.command), + command.id != lastCommandID + else { return } + lastCommandID = command.id + // The command is consumed: nothing re-reads it, and the text it carried + // from the user's field doesn't sit in the group container. + SharedStore.remove(forKey: BlurtShared.Key.command) + // A notification held while the app was suspended delivers a press from + // minutes ago; the user has long since moved on. + guard command.isFresh() else { return } + switch command.kind { + case .press: + Self.pressContext.withLock { + $0 = FocusedFieldContext( + priorText: command.priorText, selectedText: command.selectedText, windowTitle: nil, fieldLabel: nil) + } + Self.pressKeyboard.withLock { $0 = command.keyboard } + session.submit(.press) + case .release: session.submit(.release) + case .cancel: session.submit(.cancel) + } + } + + private func reloadLexicon() { + let entries = SharedStore.read([LexiconEntry].self, forKey: BlurtShared.Key.lexicon) ?? [] + let names = entries.filter(\.isName).map(\.documentText) + Self.lexiconNames.withLock { $0 = names } + lexiconNameCount = names.count + } + + /// The user's own key terms (from the App Group, where the keyboard adds + /// them on the spot), then the contact names the keyboard read off the + /// phone, deduplicated case-insensitively. `KeytermsBoost` fits the list + /// to the request's caps (100 terms, 2048 bytes), first entries first, which + /// is why the typed terms lead. + nonisolated static func keyTerms() -> [String] { + // Not through the comma-separated form: a contact called "Smith, John" + // is one name, and the engine takes an array. + var seen = Set() + var terms: [String] = [] + for term in SharedStore.keyTerms + lexiconNames.withLock({ $0 }) { + let trimmed = term.trimmingCharacters(in: .whitespacesAndNewlines) + guard !trimmed.isEmpty, seen.insert(trimmed.lowercased()).inserted else { continue } + terms.append(trimmed) + } + return terms + } + + /// What the keyboard saw around the cursor on the press in flight, once. + nonisolated static func fieldContext() -> FocusedFieldContext { + pressContext.withLock { context in + defer { context = .empty } + return context + } + } + + // MARK: - Phase, level and results out to the keyboard + + private func observePhases() -> Task { + let session = session + return Task { [weak self] in + for await phase in await session.phaseStream() { + guard let self else { return } + self.render(phase) + } + } + } + + private func render(_ phase: PipelinePhase) { + self.phase = phase + if phase == .recording { window.extend() } + // The Mac shows a setup blocker as calm idle beside a settings button; + // the keyboard has no such button, so it hears an error and the orb says so. + if phase.setupBlocker != nil { + publish(.error(message: "Blurt needs an API key. Open Blurt to add one.")) + return + } + publish(phase.overlayState) + } + + private func observeLevels() -> Task { + let levels = window.source.levels + return Task { [weak self] in + for await value in levels { + guard let self else { return } + // Only while recording: the feed reports levels only for an open + // utterance, and the home screen's hero shouldn't redraw for a + // window sitting idle in the background. + guard self.phase == .recording else { continue } + self.level = value + // The keyboard redraws its meter from these; ~12 Hz is plenty for a pill + // and keeps the shared-defaults writes off the capture path's back. + guard self.lastLevelPublish.duration(to: ContinuousClock.now) > .milliseconds(80) else { continue } + self.lastLevelPublish = ContinuousClock.now + self.publish(.recording) + } + } + } + + private func observeRecents() -> Task { + let recents = recents + return Task { [weak self] in + for await ring in recents { + guard let self else { return } + self.recent = ring + } + } + } + + private func publish(_ state: OverlayUIState) { + let snapshot = PhaseSnapshot( + state: Self.snapshotState(state), message: Self.message(state), level: Double(level), at: Date(), + command: lastCommandID) + SharedStore.write(snapshot, forKey: BlurtShared.Key.phase) + SharedStore.post(BlurtShared.Signal.phase) + } + + private static func snapshotState(_ state: OverlayUIState) -> PhaseSnapshot.State { + switch state { + case .idle: .idle + case .connecting: .connecting + case .recording: .recording + case .processing: .processing + case .pasted: .pasted + case .noTarget: .copied + case .error: .error + } + } + + private static func message(_ state: OverlayUIState) -> String? { + guard case .error(let message) = state else { return nil } + return message + } +} diff --git a/App/BlurtiOS/BlurtiOS/Sources/HomeHero.swift b/App/BlurtiOS/BlurtiOS/Sources/HomeHero.swift new file mode 100644 index 00000000..2d01121f --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/HomeHero.swift @@ -0,0 +1,72 @@ +import BlurtiOSCore +import SwiftUI + +/// The hero: the status as an eyebrow, a serif headline and a line of body +/// text, the voice element at hero size, and the one green button that opens +/// the mic (or the hairline one that closes it). +struct HomeHero: View { + var coordinator: DictationCoordinator + let status: HomeStatus + /// When the last dictation's words landed — the hero's glint. + let landedAt: Date? + @Environment(\.accessibilityReduceMotion) private var reduceMotion + @Environment(\.colorScheme) private var colorScheme + + private var state: VoiceState { + VoiceState(overlay: coordinator.phase.overlayState, windowOpen: coordinator.window.isOpen, level: coordinator.level) + } + + var body: some View { + VStack(alignment: .leading, spacing: DesignTokens.Metrics.appHeroGap) { + Eyebrow("Status") + Text(status.title) + .font(BlurtType.heading(DesignTokens.Typography.sizeTitle)) + .foregroundStyle(BlurtBrand.text) + Text(status.subtitle) + .font(BlurtType.body(DesignTokens.Typography.sizeBody)) + .foregroundStyle(BlurtBrand.muted) + VoiceElementView( + inputs: VoiceElementInputs( + state: state, landedAt: landedAt, animated: !reduceMotion, + palette: colorScheme == .dark ? .brandDark : .brandLight, slot: .home) + ) + .frame(width: VoiceSlot.home.box.width, height: VoiceSlot.home.box.height) + .frame(maxWidth: .infinity) + if coordinator.window.isOpen { + Button("Stop listening") { + Task { await coordinator.stopListening() } + } + .buttonStyle(BrandButtonStyle(role: .secondary)) + } else { + Button("Start listening") { + Task { await coordinator.startListening() } + } + .buttonStyle(BrandButtonStyle(role: .primary)) + .disabled(!coordinator.apiKey.hasAPIKey) + } + Button(coordinator.phase.isCapturing ? "Stop and transcribe" : "Dictate here, to the clipboard") { + coordinator.toggleDictation() + } + .font(BlurtType.body(DesignTokens.Typography.sizeCaption)) + .foregroundStyle(BlurtBrand.muted) + .frame(maxWidth: .infinity) + .disabled(!coordinator.window.isOpen) + if let error = coordinator.window.lastError { + note(error) + } + if coordinator.microphoneDenied { + note("Blurt needs the microphone. Allow it in Settings.") + } + if coordinator.needsKey { + note("Add your API key first (Settings → Account).") + } + } + .padding(DesignTokens.Metrics.appHeroPad) + .frame(maxWidth: .infinity, alignment: .leading) + .card(radius: DesignTokens.Metrics.radiusHero) + } + + private func note(_ text: String) -> some View { + Text(text).font(BlurtType.body(DesignTokens.Typography.sizeCaption)).foregroundStyle(BlurtBrand.errorOrange) + } +} diff --git a/App/BlurtiOS/BlurtiOS/Sources/HomeView.swift b/App/BlurtiOS/BlurtiOS/Sources/HomeView.swift new file mode 100644 index 00000000..19b3a438 --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/HomeView.swift @@ -0,0 +1,98 @@ +import AVFoundation +import BlurtEngine +import BlurtiOSCore +import SwiftUI + +/// The app's one screen: the Blurt landing page on a phone. The wordmark and +/// the gear across the top; the status as an eyebrow, a serif headline and a +/// line of body text over the voice element; the one green button; then the +/// styles and the recent dictations under their eyebrows. Setup sits on top +/// while anything is missing; everything adjustable lives behind the gear. +struct HomeView: View { + var coordinator: DictationCoordinator + @Environment(\.scenePhase) private var scenePhase + @State private var microphoneGranted = AVAudioApplication.shared.recordPermission == .granted + @State private var keyboardSeen = SharedStore.keyboardEverSeen + @State private var showsKeyEntry = false + @State private var showsSettings = false + /// When the last dictation's words landed — the hero's glint. + @State private var landedAt: Date? + /// TEMPORARY: the mic concept from Settings, so the hero shows the one + /// the keyboard will. Goes with the losers. + @AppStorage(BlurtShared.Key.voiceElement, store: SharedStore.defaults) private var voiceRaw = + VoiceElementKind.shipped.rawValue + + private var status: HomeStatus { + HomeStatus( + overlay: coordinator.phase.overlayState, windowOpen: coordinator.window.isOpen, until: coordinator.window.until, + hasKey: coordinator.apiKey.hasAPIKey, micGranted: microphoneGranted, keyboardSeen: keyboardSeen) + } + + var body: some View { + NavigationStack { + ScrollView { + VStack(alignment: .leading, spacing: DesignTokens.Metrics.appSectionGap) { + header + if !status.isSetUp { + SetupCard( + hasKey: coordinator.apiKey.hasAPIKey, microphoneGranted: microphoneGranted, keyboardSeen: keyboardSeen, + enterKey: { showsKeyEntry = true }, refresh: refreshStatus) + } + HomeHero(coordinator: coordinator, status: status, landedAt: landedAt) + StyleChips() + RecentSection(coordinator: coordinator) + Eyebrow("Powered by AssemblyAI") + .frame(maxWidth: .infinity) + } + .padding(.horizontal, DesignTokens.Metrics.appPagePad) + .padding(.bottom, DesignTokens.Metrics.appSectionGap) + } + .page() + .environment(\.voiceElementKind, VoiceElementKind(rawValue: voiceRaw) ?? .shipped) + .toolbar(.hidden, for: .navigationBar) + .sheet(isPresented: $showsKeyEntry) { KeyEntryView(apiKey: coordinator.apiKey) } + .sheet(isPresented: $showsSettings) { SettingsView(coordinator: coordinator) } + .sheet(item: Binding(get: { coordinator.pendingTermPack }, set: { coordinator.pendingTermPack = $0 })) { + ImportTermsView(pack: $0) + } + .alert( + "Couldn't read that list", + isPresented: Binding(get: { coordinator.termPackUnreadable }, set: { coordinator.termPackUnreadable = $0 }) + ) { + Button("OK") {} + } message: { + Text("It isn't a Blurt key-term list, or it's too big.") + } + .onChange(of: scenePhase) { _, phase in + if phase == .active { refreshStatus() } + } + .onChange(of: coordinator.phase) { _, _ in + if status.landed { landedAt = Date() } + } + } + .tint(BlurtBrand.accent) + } + + /// The wordmark, and the gear that opens Settings. + private var header: some View { + HStack { + Wordmark() + Spacer() + Button { + showsSettings = true + } label: { + Image(systemName: "gearshape") + .font(.system(size: DesignTokens.Metrics.appIcon, weight: DesignTokens.Typography.weightGlyph)) + .foregroundStyle(BlurtBrand.muted) + } + .accessibilityLabel("Settings") + } + .frame(height: DesignTokens.Metrics.appHeaderHeight) + } + + private func refreshStatus() { + microphoneGranted = AVAudioApplication.shared.recordPermission == .granted + keyboardSeen = SharedStore.keyboardEverSeen + coordinator.apiKey.refreshStatus() + } +} diff --git a/App/BlurtiOS/BlurtiOS/Sources/ImportTermsView.swift b/App/BlurtiOS/BlurtiOS/Sources/ImportTermsView.swift new file mode 100644 index 00000000..d88de63d --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/ImportTermsView.swift @@ -0,0 +1,53 @@ +import BlurtiOSCore +import SwiftUI + +/// "Add these key terms?" — what a shared list opens into. Every term is +/// ticked; untick any, and Add merges the rest into Blurt's key terms +/// without duplicates. +struct ImportTermsView: View { + let pack: TermPack + @Environment(\.dismiss) private var dismiss + @State private var excluded: Set = [] + + private var chosen: [String] { pack.terms.filter { !excluded.contains($0) } } + + var body: some View { + NavigationStack { + List { + Section { + ForEach(pack.terms, id: \.self) { term in + Toggle(term, isOn: binding(for: term)) + } + } header: { + Eyebrow(pack.from.map { "From \($0)" } ?? "Shared with you") + } footer: { + Text( + "They'll ride along with every dictation so Blurt spells them right. Terms you already have are skipped; " + + "the first \(TermPack.termCap) terms ride each request, so keep the list to what matters.") + } + } + .brandForm() + .navigationTitle(pack.name) + .navigationBarTitleDisplayMode(.inline) + .toolbar { + ToolbarItem(placement: .cancellationAction) { Button("Not now") { dismiss() } } + ToolbarItem(placement: .confirmationAction) { + Button("Add \(chosen.count)") { + for term in chosen { SharedStore.addKeyTerm(term) } + dismiss() + } + .disabled(chosen.isEmpty) + } + } + } + .tint(BlurtBrand.accent) + } + + private func binding(for term: String) -> Binding { + Binding( + get: { !excluded.contains(term) }, + set: { on in + if on { excluded.remove(term) } else { excluded.insert(term) } + }) + } +} diff --git a/App/BlurtiOS/BlurtiOS/Sources/KeyEntryView.swift b/App/BlurtiOS/BlurtiOS/Sources/KeyEntryView.swift new file mode 100644 index 00000000..823c6afd --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/KeyEntryView.swift @@ -0,0 +1,55 @@ +import BlurtEngine +import BlurtiOSCore +import SwiftUI + +/// Paste an AssemblyAI API key, verified against the API before it is saved +/// (`APIKeySubmission` owns that rule). Debug builds only: the shipping app +/// signs in with AssemblyAI and never shows a key. It exists so the pipeline can +/// be tested on a phone before that service is in place. +struct KeyEntryView: View { + var apiKey: APIKeyModel + @Environment(\.dismiss) private var dismiss + @State private var key = "" + @State private var inlineError: String? + @State private var isSubmitting = false + + var body: some View { + NavigationStack { + Form { + Section { + SecureField("Paste your key", text: $key) + .autocorrectionDisabled() + .textInputAutocapitalization(.never) + if let inlineError { + Text(inlineError).font(BlurtType.body(DesignTokens.Typography.sizeCaption)).foregroundStyle( + BlurtBrand.errorOrange) + } + } header: { + Eyebrow("API key") + } footer: { + Text("Keys live at assemblyai.com/dashboard/api-keys. Stored in the Keychain on this phone.") + } + } + .brandForm() + .navigationTitle("API key") + .toolbar { + ToolbarItem(placement: .cancellationAction) { Button("Cancel") { dismiss() } } + ToolbarItem(placement: .confirmationAction) { + Button(isSubmitting ? "Checking…" : "Connect") { Task { await submit() } } + .disabled(key.trimmingCharacters(in: .whitespaces).isEmpty || isSubmitting) + } + } + } + } + + private func submit() async { + isSubmitting = true + defer { isSubmitting = false } + let outcome = await apiKey.submit(key) + switch outcome.failureReport { + case .none: dismiss() + case .some(.inline(let message)): inlineError = message + case .some(.alert(let title, let message)): inlineError = "\(title) \(message)" + } + } +} diff --git a/App/BlurtiOS/BlurtiOS/Sources/KeyboardGalleryView.swift b/App/BlurtiOS/BlurtiOS/Sources/KeyboardGalleryView.swift new file mode 100644 index 00000000..8d757d63 --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/KeyboardGalleryView.swift @@ -0,0 +1,261 @@ +import BlurtiOSCore + +#if DEBUG + import SwiftUI + + /// Every keyboard layout in every state, rendered inside the app: the design + /// reference, and the way a screenshot of the keyboard is taken without + /// tapping through Notes. Debug builds only, reached by launch argument: + /// + /// -BlurtGallery [,…] [face] [-BlurtGalleryStill] [-BlurtGalleryBare] [-BlurtGalleryVoice a|b|c] [-BlurtGalleryAlign left|center|right] + /// + /// The last word is a face, `light` or `dark` (the older theme words still + /// land: `system` and `paper` are light, `system-dark` and `ink` dark); the + /// light face when left out. Two switches make a capture reproducible for the design loop + /// (DESIGN.md › Reproducing it): `-BlurtGalleryStill` holds every motion at time + /// zero — the ring, the meter's wave, the orb's fluid and grain — so two + /// captures of one state are the same pixels; `-BlurtGalleryBare` draws the + /// first row alone, no caption, inside a 2 pt `#FF00FF` registration border + /// that `scripts/design-diff.swift crop` cuts to. `-BlurtGalleryVoice a|b|c` + /// picks which mic concept draws (`VoiceElementKind`); `-BlurtGalleryAlign + /// left|center|right` puts the mic at a side (`MicAlignment`, as Settings → + /// Mic side does); `-BlurtVoice + /// [light|dark]` renders that element by itself. + /// where a state is `off` (no Full Access), `start` (app not listening), + /// `idle`, `connecting`, `recording`, `processing`, `pasted`, `copied`, + /// `error`, `landed` (the drop, timed for a screenshot) or `live` (a whole + /// dictation walked on a clock, over and over, for watching or recording + /// the motion), plus the keyboard's own `keys` (the panel flipped to its + /// keyboard page), `symbols`, `term` (the key-term field mid-typing), + /// `send` (a return key with a label), `saved` (the + just after a term + /// was saved), `selected` (a word highlighted in the text: the + is the + /// chip holding it) and `known` (the same, for a word already in the key + /// terms). `scripts/ios-sim.sh` passes `BLURT_LAUNCH_ARGS` through, so + /// + /// BLURT_LAUNCH_ARGS="-BlurtGallery panel idle,recording,processing" \ + /// scripts/ios-sim.sh --screenshot panel.png + /// + /// is the whole loop. The keyboard's sources are compiled into the app (see + /// project.yml: the theme picker and the home screen draw the real keyboard + /// and orb too); the keyboard itself never runs here. + struct KeyboardGalleryView: View { + struct Row: Identifiable { + let id = UUID() + let caption: String + let model: KeyboardModel + } + + /// The capture switches, read from the launch arguments. + struct Options: Equatable { + /// Hold every motion at time zero, as Reduce Motion does, so a capture + /// is the same pixels every time. + var still = false + /// One row, no caption, a registration border round the keyboard. + var bare = false + static func parse(_ arguments: [String]) -> Options { + Options(still: arguments.contains("-BlurtGalleryStill"), bare: arguments.contains("-BlurtGalleryBare")) + } + } + + let rows: [Row] + var options = Options() + + /// The registration colour and width `scripts/design-diff.swift crop` looks + /// for. Drawn outside the keyboard's frame, so what is inside the border + /// is exactly the keyboard's pixels — above and below only, so the + /// keyboard keeps the phone's full width (the crop reads a missing side + /// border as "the edge of the screen"). + static let registration = Color(red: 1, green: 0, blue: 1) + static let registrationWidth: CGFloat = 2 + + var body: some View { + if options.bare, let row = rows.first { + VStack { + Spacer(minLength: 0) + keyboard(row) + .padding(.vertical, Self.registrationWidth) + .background(Self.registration) + Spacer(minLength: 0) + } + .frame(maxWidth: .infinity, maxHeight: .infinity) + .background(Color(uiColor: .systemBackground)) + .ignoresSafeArea() + } else { + ScrollView { + VStack(alignment: .leading, spacing: 16) { + ForEach(rows) { row in + VStack(alignment: .leading, spacing: 4) { + Text(row.caption).font(.caption.weight(.medium)).foregroundStyle(.secondary).padding(.horizontal) + keyboard(row) + } + } + } + .padding(.vertical, 12) + } + .background(Color(uiColor: .systemBackground)) + } + } + + private func keyboard(_ row: Row) -> some View { + KeyboardRootView(model: row.model) + .frame(height: row.model.effectiveLayout.height) + // The host's material, under a clear surface, as the app stands in for it. + .background(row.model.palette.standIn) + .clipped() + .environment(\.keyboardMotionHeld, options.still) + } + + /// The rows named by the launch arguments; empty when they don't ask for + /// the gallery at all. + static func rows(from arguments: [String]) -> [Row] { + guard let flag = arguments.firstIndex(of: "-BlurtGallery"), arguments.count > flag + 2, + let layout = KeyboardLayout(rawValue: arguments[flag + 1]) + else { return [] } + // The theme is the next word unless it is a switch. + let theme = arguments.count > flag + 3 && !arguments[flag + 3].hasPrefix("-") ? arguments[flag + 3] : "light" + let voice = VoiceElementKind.parse(arguments) ?? .shipped + let align = MicAlignment.parse(arguments) ?? .center + return arguments[flag + 2].split(separator: ",").map { name in + Row( + caption: "\(layout.rawValue) · \(name) · \(theme) · \(voice.rawValue) · \(align.rawValue)", + model: model(layout: layout, state: String(name), theme: theme, voice: voice, align: align)) + } + } + + /// The words that ask for the dark face; anything else is the light one. + static let darkFaces: Set = ["dark", "system-dark", "ink"] + + /// The pipeline's phases by gallery name; `landed` is pasted, the moment + /// the words went in. + private static let phases: [String: PhaseSnapshot.State] = [ + "connecting": .connecting, "recording": .recording, "processing": .processing, "pasted": .pasted, + "copied": .copied, "error": .error, "landed": .pasted, + ] + + private static func model( + layout: KeyboardLayout, state: String, theme: String, voice: VoiceElementKind, align: MicAlignment + ) -> KeyboardModel { + let model = KeyboardModel() + model.layout = layout + model.voiceKindOverride = voice + model.micAlignmentOverride = align + model.faceOverride = ThemeFace(themeID: KeyboardPalette.brandID, dark: Self.darkFaces.contains(theme)) + // `keys` shows the panel's carousel flipped to its keyboard page; + // `term` the voice bar as the key-term field, mid-typing. + model.panelShowsKeys = state == "keys" || state == "symbols" + if state == "term" { model.termDraft = "Rizz" } + // `symbols`: the keys on their symbols page; `send`: a field whose + // return key says so; `saved`: the + a moment after a term was saved. + model.symbolsPage = state == "symbols" + if state == "send" { model.returnLabel = "send" } + if state == "saved" { model.termSavedAt = Date() } + // `selected`: a word is highlighted in the text, so the + is the chip + // holding it; `known`: the same word, already one of the key terms. + if state == "selected" || state == "known" { model.selectedTerm = "Rizz" } + model.selectedTermIsKnown = state == "known" + // `landed`: the words go in 2.75 s after launch, so a screenshot taken + // 3 s in (scripts/ios-sim.sh) catches the glint mid-sweep. + if state == "landed" { model.resultLandedAt = Date().addingTimeInterval(2.75) } + if state == "live" { walk(model) } + model.hasFullAccess = state != "off" + model.isListening = state != "off" && state != "start" + let phase = Self.phases[state] ?? .idle + model.snapshot = PhaseSnapshot( + state: phase, message: phase == .error ? "The microphone didn't start." : nil, + level: phase == .recording ? 0.62 : 0, at: Date()) + return model + } + + /// `live`: the key walks a whole dictation, again and again — idle for + /// 1.5 s, connecting for 1 s, 4.5 s of recording with a made-up voice at + /// the app's 12 Hz, processing for 1.5 s, the words landing, the pasted + /// notice, idle again — so the fades can be watched, or recorded with + /// `xcrun simctl io booted recordVideo`. + private static func walk(_ model: KeyboardModel) { + Task { @MainActor in + @MainActor func set(_ state: PhaseSnapshot.State, level: Double = 0) { + model.snapshot = PhaseSnapshot(state: state, message: nil, level: level, at: Date()) + } + while !Task.isCancelled { + set(.idle) + try? await Task.sleep(for: .seconds(1.5)) + set(.connecting) + try? await Task.sleep(for: .seconds(1)) + let start = Date() + while Date().timeIntervalSince(start) < 4.5 { + let t = Date().timeIntervalSince(start) + let voice = 0.15 + 0.55 * abs(sin(t * 2.6)) * (0.55 + 0.45 * sin(t * 7.1)) + set(.recording, level: min(1, max(0, voice))) + try? await Task.sleep(for: .milliseconds(80)) + } + set(.processing) + try? await Task.sleep(for: .seconds(1.5)) + model.resultLandedAt = Date() + set(.pasted) + try? await Task.sleep(for: .seconds(1.2)) + set(.idle) + try? await Task.sleep(for: .seconds(2)) + } + } + } + + /// `-BlurtVoice [light|dark]`: one voice + /// element by itself, in its slot's box, on the face's surface, inside the + /// registration border — for the review sheet. States are + /// the pipeline's plus `off` (not ready) and `landed` (the glint, timed + /// for a 3 s screenshot). + struct VoiceStillView: View { + let kind: VoiceElementKind + let slot: VoiceSlot + let state: VoiceState + let landedAt: Date? + let dark: Bool + + var body: some View { + let palette: KeyboardPalette = dark ? .brandDark : .brandLight + VStack { + Spacer(minLength: 0) + VoiceElementView( + inputs: VoiceElementInputs(state: state, landedAt: landedAt, animated: true, palette: palette, slot: slot) + ) + .frame(width: slot.box.width, height: slot.box.height) + .padding(KeyboardGalleryView.registrationWidth) + .background(KeyboardGalleryView.registration) + .environment(\.voiceElementKind, kind) + Spacer(minLength: 0) + } + .frame(maxWidth: .infinity, maxHeight: .infinity) + .background(palette.standIn) + .ignoresSafeArea() + } + + static func parse(_ arguments: [String]) -> VoiceStillView? { + guard let flag = arguments.firstIndex(of: "-BlurtVoice"), arguments.count > flag + 3, + let kind = VoiceElementKind(rawValue: arguments[flag + 1]), + let slot = VoiceSlot(rawValue: arguments[flag + 2]) + else { return nil } + let dark = arguments.count > flag + 4 && darkFaces.contains(arguments[flag + 4]) + guard let (state, landedAt) = Self.state(named: arguments[flag + 3]) else { return nil } + return VoiceStillView(kind: kind, slot: slot, state: state, landedAt: landedAt, dark: dark) + } + + /// The states by name: the pipeline's, `off` (not ready) and `landed` + /// (pasted, with the glint timed for the screenshot). + private static func state(named name: String) -> (VoiceState, Date?)? { + let states: [String: VoiceState] = [ + "off": VoiceState(phase: .idle, isReady: false), + "idle": VoiceState(phase: .idle, isReady: true), + "connecting": VoiceState(phase: .connecting, isReady: true), + "recording": VoiceState(phase: .recording, isReady: true, level: 0.62), + "processing": VoiceState(phase: .processing, isReady: true), + "pasted": VoiceState(phase: .pasted, isReady: true), + "copied": VoiceState(phase: .copied, isReady: true), + "error": VoiceState(phase: .error, isReady: true, message: "The microphone didn't start."), + "landed": VoiceState(phase: .pasted, isReady: true), + ] + guard let state = states[name] else { return nil } + return (state, name == "landed" ? Date().addingTimeInterval(2.75) : nil) + } + } + } +#endif diff --git a/App/BlurtiOS/BlurtiOS/Sources/KeyboardProbeView.swift b/App/BlurtiOS/BlurtiOS/Sources/KeyboardProbeView.swift new file mode 100644 index 00000000..280ba2b3 --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/KeyboardProbeView.swift @@ -0,0 +1,99 @@ +import BlurtiOSCore + +#if DEBUG + import SwiftUI + import UIKit + + /// `-BlurtProbeField light|dark`: a bare text field that takes the system + /// keyboard on launch, so the `BlurtiOSProbe` UI test can read that + /// keyboard's geometry off this simulator — the numbers + /// `Design/apple-geometry.json` records and the `key/*` tokens are pinned + /// to (`scripts/apple-geometry.sh`, DESIGN.md › Apple's geometry). Debug + /// builds only; the keyboard itself never runs here. + struct KeyboardProbeView: View { + let dark: Bool + /// `-BlurtProbeText `: the field starts with this in it, so a flow + /// can highlight a word in any layout, letter keys or not. + var text = "" + + /// `-BlurtProbeField light|dark [a|b|c] [slimBar|panel|full] [left|center|right]`: + /// the words after the face set the mic concept, the layout and the mic's + /// side in the App Group, so the Blurt keyboard — which reads them on every + /// appearance — comes up that way for a capture of the real extension + /// (`scripts/ios-keyboard-shot.sh`, `scripts/ios-keyboard-flows.sh`). + static func parse(_ arguments: [String]) -> KeyboardProbeView? { + guard let flag = arguments.firstIndex(of: "-BlurtProbeField") else { return nil } + let face = arguments.count > flag + 1 ? arguments[flag + 1] : "light" + guard face == "light" || face == "dark" else { return nil } + // The mic's side is the middle unless a word says otherwise: the App + // Group outlives a run, and an earlier `left` must not leak into this one. + SharedStore.micAlignment = .center + for word in arguments.dropFirst(flag + 2).prefix(while: { !$0.hasPrefix("-") }) { + if let kind = VoiceElementKind(rawValue: word) { SharedStore.voiceElementKind = kind } + if let layout = KeyboardLayout(rawValue: word) { SharedStore.layout = layout } + if let side = MicAlignment(rawValue: word) { SharedStore.micAlignment = side } + } + // `-BlurtProbeResetTerms`: an empty key-term list, so a flow that adds a + // word finds it new on every run (the App Group outlives the test). + if arguments.contains("-BlurtProbeResetTerms") { SharedStore.keyTerms = [] } + var text = "" + if let flag = arguments.firstIndex(of: "-BlurtProbeText"), arguments.count > flag + 1 { + text = arguments[flag + 1] + } + return KeyboardProbeView(dark: face == "dark", text: text) + } + + var body: some View { + VStack { + ProbeField(dark: dark, text: text) + .frame(height: DesignTokens.Metrics.keyMinWidth) + .padding() + // Puts the keyboard away without leaving the field, so a flow can + // bring it back into the same host and see what a re-appearance + // in a living extension keeps or loses. + Button("Dismiss keyboard") { + UIApplication.shared.sendAction(#selector(UIResponder.resignFirstResponder), to: nil, from: nil, for: nil) + } + .accessibilityIdentifier("probe-dismiss") + .foregroundStyle(dark ? Color.white : Color.black) + // What Settings does while a keyboard is up elsewhere: the layout + // changes in the App Group, and the keyboard reads it when it next + // appears. One button per layout, so a flow can switch and re-show. + HStack { + ForEach(KeyboardLayout.allCases) { layout in + Button(layout.rawValue) { SharedStore.layout = layout } + .accessibilityIdentifier("probe-layout-\(layout.rawValue)") + } + } + .foregroundStyle(dark ? Color.white : Color.black) + Spacer(minLength: 0) + } + .frame(maxWidth: .infinity, maxHeight: .infinity) + .background(dark ? Color.black : Color.white) + } + } + + /// A UIKit field, since `keyboardAppearance` and `autocorrectionType` are + /// what decide which keyboard iOS shows: the appearance picks the face, and + /// autocorrection brings the predictive bar up, which is the band the + /// keyboard's voice row stands in for. + private struct ProbeField: UIViewRepresentable { + let dark: Bool + let text: String + + func makeUIView(context: Context) -> UITextField { + let field = UITextField() + field.accessibilityIdentifier = "probe-field" + field.text = text + field.keyboardAppearance = dark ? .dark : .light + field.autocorrectionType = .yes + field.autocapitalizationType = .none + field.borderStyle = .roundedRect + field.placeholder = "probe" + DispatchQueue.main.async { field.becomeFirstResponder() } + return field + } + + func updateUIView(_ uiView: UITextField, context: Context) {} + } +#endif diff --git a/App/BlurtiOS/BlurtiOS/Sources/ListeningSource.swift b/App/BlurtiOS/BlurtiOS/Sources/ListeningSource.swift new file mode 100644 index 00000000..6c61333b --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/ListeningSource.swift @@ -0,0 +1,111 @@ +import BlurtEngine +import Foundation +import Synchronization + +/// What the listening window needs from a microphone: opened once for the +/// window, closed with it, and feeding the engine per utterance in between. +/// `WindowedAudioSource` is the phone's; `SimulatorAudioSource` stands in +/// where the simulator has no capture session. +nonisolated protocol ListeningSource: MicCaptureProtocol { + var isOpen: Bool { get } + /// The 0…1 meter the orb and the home screen render, one value per chunk + /// of an open utterance. + var levels: AsyncStream { get } + /// Opens the microphone. Async because opening a capture session blocks for + /// hundreds of milliseconds on a phone and must not do so on the main actor. + func open() async throws + func close() async +} + +/// Why a microphone couldn't open or start. +nonisolated enum ListeningSourceFailure: Error, LocalizedError { + /// A press arrived with no listening window open — the keyboard's job is to + /// open the app first, so this is a plumbing fault, not a user error. + case windowClosed + case noInputDevice + + var errorDescription: String? { + switch self { + case .windowClosed: "Blurt isn't listening. Open Blurt to start." + case .noInputDevice: "No microphone is available." + } + } +} + +/// The per-utterance feed both microphones push into: the engine gets one +/// stream per press, the meter a level per chunk, and the byte tally is what +/// `stop()` reports so the engine can drop a press too short to transcribe. +nonisolated final class UtteranceFeed: Sendable { + private struct State { + var sink: AsyncStream.Continuation? + var bytes = 0 + } + + private let state = Mutex(State()) + private let levelsContinuation: AsyncStream.Continuation + let levels: AsyncStream + + init() { + let (stream, continuation) = AsyncStream.makeStream(bufferingPolicy: .bufferingNewest(1)) + levels = stream + levelsContinuation = continuation + } + + /// Starts an utterance: chunks delivered from now on go to the returned + /// stream. Any utterance still open is finished first. + func begin() -> AsyncStream { + let (stream, continuation) = AsyncStream.makeStream(bufferingPolicy: .unbounded) + state.withLock { state in + state.sink?.finish() + state.sink = continuation + state.bytes = 0 + } + return stream + } + + /// Ends the utterance and reports how many bytes it carried. + @discardableResult + func end() -> Int { + state.withLock { state in + let bytes = state.bytes + state.sink?.finish() + state.sink = nil + state.bytes = 0 + return bytes + } + } + + /// A chunk of 16 kHz mono 16-bit PCM from the microphone, on whatever + /// thread the microphone delivers on. + func deliver(_ chunk: Data) { + guard !chunk.isEmpty else { return } + // Nothing to report between utterances: a window sits open for minutes, + // and a level a dozen times a second would only keep views redrawing. + let delivered = state.withLock { state -> Bool in + guard let sink = state.sink else { return false } + state.bytes += chunk.count + sink.yield(chunk) + return true + } + if delivered { levelsContinuation.yield(Self.level(of: chunk)) } + } + + /// dBFS of a chunk of 16-bit PCM, mapped to 0…1 with the Mac meter's -50 dB + /// floor so room ambient reads as empty bars. + static func level(of pcm: Data) -> Float { + let count = pcm.count / MemoryLayout.size + guard count > 0 else { return 0 } + let energy = pcm.withUnsafeBytes { raw -> Double in + raw.bindMemory(to: Int16.self).reduce(0) { sum, sample in + let value = Double(sample) + return sum + value * value + } + } + let rms = (energy / Double(count)).squareRoot() / Double(Int16.max) + guard rms > 0 else { return 0 } + let floor = -50.0 + let db = 20 * log10(rms) + guard db > floor else { return 0 } + return Float(min(1, (db - floor) / -floor)) + } +} diff --git a/App/BlurtiOS/BlurtiOS/Sources/ListeningWindow.swift b/App/BlurtiOS/BlurtiOS/Sources/ListeningWindow.swift new file mode 100644 index 00000000..9ac7ec81 --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/ListeningWindow.swift @@ -0,0 +1,130 @@ +import AVFoundation +import BlurtiOSCore +import Foundation +import Observation + +/// The stretch of time the app keeps the microphone open so the keyboard can +/// start a dictation without bringing the app forward — Wispr calls the same +/// thing a "Flow Session". +/// +/// Opening needs the app in front (iOS refuses to start capture from the +/// background), which is why the keyboard's mic key opens `blurt://start` when +/// no window is open. Once open, `UIBackgroundModes: audio` plus the active +/// audio session keep it alive after the user swipes back. Every dictation +/// extends the window; it closes on its own after `SharedStore.windowMinutes` +/// of silence (0 means never), when the user closes it in the app, or when a +/// phone call, Siri or another app takes the microphone: the window is over +/// then, and saying so beats a mic that looks open and hears nothing. +@MainActor +@Observable +final class ListeningWindow { + #if targetEnvironment(simulator) + // The simulator's capture session carries no audio; see SimulatorAudioSource. + let source: any ListeningSource = SimulatorAudioSource() + #else + let source: any ListeningSource = WindowedAudioSource() + #endif + private(set) var until: Date? + private(set) var lastError: String? + @ObservationIgnored private var expiry: Task? + @ObservationIgnored private var heartbeat: Task? + @ObservationIgnored private var interruptions: Task? + /// Run when the window closes on its own — its time ran out, or iOS took + /// the microphone — and awaited *before* the source closes: a dictation + /// riding on the feed has to be ended by whoever owns the session while + /// the feed still holds its audio, not left recording into a closed one. + @ObservationIgnored var onClosingOnItsOwn: (() async -> Void)? + + var isOpen: Bool { source.isOpen && (until.map { $0 > Date() } ?? false) } + + /// Opens the microphone for `SharedStore.windowMinutes`. Foreground only. + func open() async { + do { + let audioSession = AVAudioSession.sharedInstance() + // `.mixWithOthers` so the window doesn't silence whatever the user is + // playing for the whole time it is open; `.defaultToSpeaker` so that + // audio doesn't drop to the earpiece while the microphone is ours; HFP + // so AirPods can be the input. + try audioSession.setCategory( + .playAndRecord, mode: .default, options: [.allowBluetoothHFP, .mixWithOthers, .defaultToSpeaker]) + try audioSession.setActive(true) + try await source.open() + lastError = nil + extend() + startHeartbeat() + watchInterruptions() + } catch { + lastError = error.localizedDescription + await close() + } + } + + /// Pushes the window's end out again — called on open and after every + /// dictation, so a window only closes on real silence. + func extend() { + let minutes = SharedStore.windowMinutes + let end = minutes > 0 ? Date().addingTimeInterval(TimeInterval(minutes * 60)) : Date.distantFuture + until = end + SharedStore.listeningUntil = end + expiry?.cancel() + guard end != .distantFuture else { return } + expiry = Task { [weak self] in + try? await Task.sleep(for: .seconds(end.timeIntervalSinceNow)) + guard !Task.isCancelled else { return } + await self?.closeOnItsOwn() + } + } + + /// Tells the keyboard the app is alive and the microphone really is open — + /// `SharedStore.isListening` needs both. Stops reporting the moment capture + /// is interrupted, so the keyboard's next tap reopens the app instead of + /// sending a press into silence. + private func startHeartbeat() { + heartbeat?.cancel() + heartbeat = Task { [weak self] in + while !Task.isCancelled { + guard let self else { return } + SharedStore.appSeenAt = source.isOpen ? Date() : nil + try? await Task.sleep(for: .seconds(SharedStore.appHeartbeatInterval)) + } + } + } + + /// A phone call, Siri, an alarm: iOS takes the microphone and tells us. The + /// window closes rather than pretending; the user reopens it in Blurt. + private func watchInterruptions() { + interruptions?.cancel() + interruptions = Task { [weak self] in + let began = NotificationCenter.default.notifications(named: AVAudioSession.interruptionNotification) + .filter { notification in + let raw = notification.userInfo?[AVAudioSessionInterruptionTypeKey] as? UInt + return raw.flatMap(AVAudioSession.InterruptionType.init(rawValue:)) == .began + } + for await _ in began { + guard !Task.isCancelled else { return } + await self?.closeOnItsOwn() + return + } + } + } + + /// The window ending without the user asking: the owner hears first. + private func closeOnItsOwn() async { + await onClosingOnItsOwn?() + await close() + } + + func close() async { + expiry?.cancel() + expiry = nil + heartbeat?.cancel() + heartbeat = nil + interruptions?.cancel() + interruptions = nil + await source.close() + until = nil + SharedStore.listeningUntil = nil + SharedStore.appSeenAt = nil + try? AVAudioSession.sharedInstance().setActive(false, options: .notifyOthersOnDeactivation) + } +} diff --git a/App/BlurtiOS/BlurtiOS/Sources/RecentSection.swift b/App/BlurtiOS/BlurtiOS/Sources/RecentSection.swift new file mode 100644 index 00000000..5fe2d426 --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/RecentSection.swift @@ -0,0 +1,45 @@ +import SwiftUI +import UIKit + +/// The recent dictations as cards under their eyebrow: the words, the style +/// as a small mono tag, the time, a copy button. +struct RecentSection: View { + var coordinator: DictationCoordinator + + var body: some View { + VStack(alignment: .leading, spacing: DesignTokens.Metrics.appStackGap) { + Eyebrow("Recent") + if coordinator.recent.displayed.isEmpty { + Text("Your recent blurts will appear here.") + .font(BlurtType.body(DesignTokens.Typography.sizeBody)).foregroundStyle(BlurtBrand.muted) + .frame(maxWidth: .infinity, alignment: .leading) + .padding(DesignTokens.Metrics.appCardPad) + .card() + } + ForEach(coordinator.recent.displayed) { entry in + VStack(alignment: .leading, spacing: DesignTokens.Metrics.appLineGap) { + Text(entry.text) + .font(BlurtType.body(DesignTokens.Typography.sizeBody)).foregroundStyle(BlurtBrand.text) + .lineLimit(3) // literal-ok: three lines of a dictation + HStack(spacing: DesignTokens.Metrics.appChipGap) { + if let style = entry.style { Eyebrow(style, color: BlurtBrand.accent) } + Text(entry.relativeLabel(now: Date())) + .font(BlurtType.body(DesignTokens.Typography.sizeCaption)).foregroundStyle(BlurtBrand.muted) + Spacer() + Button { + UIPasteboard.general.string = entry.text + } label: { + Image(systemName: "doc.on.doc") + .font(.system(size: DesignTokens.Typography.sizeGlyph, weight: DesignTokens.Typography.weightGlyph)) + .foregroundStyle(BlurtBrand.muted) + } + .buttonStyle(.borderless) + .accessibilityLabel("Copy") + } + } + .padding(DesignTokens.Metrics.appCardPad) + .card() + } + } + } +} diff --git a/App/BlurtiOS/BlurtiOS/Sources/SettingsView.swift b/App/BlurtiOS/BlurtiOS/Sources/SettingsView.swift new file mode 100644 index 00000000..d37bc0d7 --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/SettingsView.swift @@ -0,0 +1,158 @@ +import BlurtEngine +import BlurtiOSCore +import SwiftUI + +/// Everything adjustable, grouped the way the Mac's Settings are: the keyboard +/// (layout, theme, hands-free), listening, transcription, the account, and +/// about — a form on the brand's page, its sections under mono eyebrows. +/// Presented from the home screen's gear. +struct SettingsView: View { + var coordinator: DictationCoordinator + @Environment(\.dismiss) private var dismiss + @State private var layout = SharedStore.layout + @State private var micAlignment = SharedStore.micAlignment + @State private var autoDictate = SharedStore.autoDictate + @State private var windowMinutes = SharedStore.windowMinutes + @AppStorage(BlurtShared.Key.theme, store: SharedStore.defaults) private var themeID = "system" + /// TEMPORARY: which mic concept to try. Goes with the losers. + @AppStorage(BlurtShared.Key.voiceElement, store: SharedStore.defaults) private var voiceRaw = + VoiceElementKind.shipped.rawValue + @State private var showsKeyEntry = false + @AppStorage(SharedStore.keyTermsKey, store: SharedStore.defaults) private var keyTerms = "" + @AppStorage(EnhancedTranscriptsStore.defaultsKey) private var enhancedTranscripts = + EnhancedTranscriptsStore.defaultValue + + var body: some View { + NavigationStack { + Form { + keyboardSection + listeningSection + transcriptionSection + accountSection + aboutSection + } + .brandForm() + .navigationTitle("Settings") + .navigationBarTitleDisplayMode(.inline) + .toolbar { + ToolbarItem(placement: .confirmationAction) { Button("Done") { dismiss() } } + } + .sheet(isPresented: $showsKeyEntry) { KeyEntryView(apiKey: coordinator.apiKey) } + } + .tint(BlurtBrand.accent) + } + + private var keyboardSection: some View { + Section { + Picker("Layout", selection: $layout) { + ForEach(KeyboardLayout.allCases) { Text($0.title).tag($0) } + } + .pickerStyle(.segmented) + .onChange(of: layout) { _, value in SharedStore.layout = value } + Text(layout.summary).font(BlurtType.body(DesignTokens.Typography.sizeCaption)).foregroundStyle(BlurtBrand.muted) + // The mic's side, for one hand — the same shape as the layout above: three + // segments and a line saying what the choice does. + Picker("Mic alignment", selection: $micAlignment) { + ForEach(MicAlignment.allCases) { Text($0.title).tag($0) } + } + .pickerStyle(.segmented) + .onChange(of: micAlignment) { _, value in SharedStore.micAlignment = value } + Text(micAlignment.summary).font(BlurtType.body(DesignTokens.Typography.sizeCaption)) + .foregroundStyle(BlurtBrand.muted) + NavigationLink { + ThemePickerView() + } label: { + LabeledContent("Theme", value: KeyboardPalette.resolve(themeID, dark: false).name) + } + Toggle("Hands-free", isOn: $autoDictate) + .onChange(of: autoDictate) { _, value in SharedStore.autoDictate = value } + // TEMPORARY: the three mic concepts, to try each in use. Goes with the losers. + Picker("Mic", selection: $voiceRaw) { + Text("Grille").tag(VoiceElementKind.grille.rawValue) + Text("Ribs").tag(VoiceElementKind.ribs.rawValue) + Text("Streak").tag(VoiceElementKind.streak.rawValue) + } + .pickerStyle(.segmented) + } header: { + Eyebrow("Keyboard") + } footer: { + Text( + "Hands-free starts dictating the moment the Blurt keyboard comes up in a text field; tap the mic to stop. " + + "Mic is a temporary switch between the three concepts while one is chosen. " + + "The keyboard picks up changes the next time it appears.") + } + } + + private var listeningSection: some View { + Section { + Picker("Keep listening for", selection: $windowMinutes) { + Text("5 minutes").tag(5) + Text("15 minutes").tag(15) + Text("1 hour").tag(60) + Text("Until I stop it").tag(0) + } + .onChange(of: windowMinutes) { _, value in SharedStore.windowMinutes = value } + } header: { + Eyebrow("Listening") + } footer: { + Text("How long the mic stays open with nobody dictating, so the keyboard can start without opening Blurt.") + } + } + + private var transcriptionSection: some View { + Section { + Toggle("Enhanced transcripts", isOn: $enhancedTranscripts) + NavigationLink("Output styles") { StylesView() } + TextField("Key terms, comma-separated", text: $keyTerms, axis: .vertical) + .autocorrectionDisabled() + LabeledContent("Key terms", value: "\(KeyTermList.parse(keyTerms).count)") + if !KeyTermList.parse(keyTerms).isEmpty { + ShareLink( + item: termPack, subject: Text("Blurt key terms"), message: Text(termPack.plainText), + preview: SharePreview(termPack.name, icon: Image(systemName: "text.badge.plus")) + ) { + Label("Share key terms…", systemImage: "square.and.arrow.up") + } + } + } header: { + Eyebrow("Transcription") + } footer: { + Text( + "Enhanced transcripts clean up punctuation and wording. Key terms are names and jargon to spell right — " + + "add one from the keyboard with the + beside the orb, share the list with a group chat so everyone's " + + "dictation gets the names right; \(coordinator.lexiconNameCount) contact names come along automatically.") + } + } + + private var accountSection: some View { + Section { + LabeledContent("Sign in with AssemblyAI", value: "Coming soon") + #if DEBUG + Button(coordinator.apiKey.hasAPIKey ? "Replace the API key" : "Use an API key") { showsKeyEntry = true } + #endif + } header: { + Eyebrow("Account") + } + } + + private var aboutSection: some View { + Section { + LabeledContent("Version", value: Self.version) + Link("Blurt on GitHub", destination: URL(string: "https://github.com/AssemblyAI/blurt") ?? URL(filePath: "/")) + Text("Powered by AssemblyAI").foregroundStyle(BlurtBrand.muted) + } header: { + Eyebrow("About") + } + } + + /// The whole list, for a friend: a `.blurtterms` file plus the words as text. + private var termPack: TermPack { + TermPack(name: "Key terms", from: nil, terms: KeyTermList.parse(keyTerms)) + } + + private static var version: String { + let short = Bundle.main.object(forInfoDictionaryKey: "CFBundleShortVersionString") as? String ?? "?" + let build = Bundle.main.object(forInfoDictionaryKey: "CFBundleVersion") as? String ?? "?" + return "\(short) (\(build))" + } +} diff --git a/App/BlurtiOS/BlurtiOS/Sources/SetupCard.swift b/App/BlurtiOS/BlurtiOS/Sources/SetupCard.swift new file mode 100644 index 00000000..4f0f5977 --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/SetupCard.swift @@ -0,0 +1,92 @@ +import AVFoundation +import SwiftUI +import UIKit + +/// The setup checklist, on top of the home screen while anything is missing: +/// three numbered steps — sign in (a key, in debug builds), the microphone, +/// the keyboard — each with its control until it is done. +struct SetupCard: View { + let hasKey: Bool + let microphoneGranted: Bool + let keyboardSeen: Bool + /// Debug builds take an API key in place of the sign-in that doesn't exist yet. + let enterKey: () -> Void + /// Re-read the permissions after one was asked for. + let refresh: () -> Void + + var body: some View { + VStack(alignment: .leading, spacing: DesignTokens.Metrics.appStackGap) { + Eyebrow("Set up Blurt") + SetupRow(number: "01", done: hasKey, title: "Sign in with AssemblyAI") { + VStack(alignment: .leading, spacing: DesignTokens.Metrics.appLineGap) { + Text("Coming soon — sign-in needs the AssemblyAI login service.") + .font(BlurtType.body(DesignTokens.Typography.sizeCaption)).foregroundStyle(BlurtBrand.muted) + #if DEBUG + Button("Use an API key instead") { enterKey() }.buttonStyle(BrandButtonStyle(role: .secondary)) + #endif + } + } + SetupRow(number: "02", done: microphoneGranted, title: "Allow the microphone") { + if AVAudioApplication.shared.recordPermission == .denied { + // iOS asks once; after a refusal only Settings can change it. + Button("Allow in Settings") { Self.openAppSettings() }.buttonStyle(BrandButtonStyle(role: .primary)) + } else { + Button("Allow") { + Task { + _ = await AVAudioApplication.requestRecordPermission() + refresh() + } + } + .buttonStyle(BrandButtonStyle(role: .primary)) + } + } + SetupRow(number: "03", done: keyboardSeen, title: "Add the Blurt keyboard") { + VStack(alignment: .leading, spacing: DesignTokens.Metrics.appLineGap) { + // The button's link (`openSettingsURLString`) is the only Settings + // link Apple allows — the General › Keyboard deep links are private + // API and fail review. It is meant to land on Blurt's own page, where + // iOS puts a Keyboards row for any app with a keyboard, but not every + // build honours it (the iOS 26 simulator opens Settings' top level), + // so the steps name the whole path, which holds from either screen. + Text( + "In Settings, go to Apps › Blurt › Keyboards, turn on Blurt, then Allow Full Access. " + + "Full Access is what lets the keyboard send your words to Blurt." + ) + .font(BlurtType.body(DesignTokens.Typography.sizeCaption)).foregroundStyle(BlurtBrand.muted) + Button("Open Settings") { Self.openAppSettings() }.buttonStyle(BrandButtonStyle(role: .secondary)) + } + } + } + .padding(DesignTokens.Metrics.appCardPad) + .frame(maxWidth: .infinity, alignment: .leading) + .card() + } + + static func openAppSettings() { + guard let url = URL(string: UIApplication.openSettingsURLString) else { return } + UIApplication.shared.open(url) + } +} + +/// One step: its number in mono, the step, and its control while it is still +/// to do; a check when done. +private struct SetupRow: View { + let number: String + let done: Bool + let title: String + @ViewBuilder let action: () -> Action + + var body: some View { + HStack(alignment: .top, spacing: DesignTokens.Metrics.appStackGap) { + Eyebrow(number, color: done ? BlurtBrand.accent : BlurtBrand.muted) + .frame(width: DesignTokens.Metrics.appSetupNumberWidth, alignment: .leading) + VStack(alignment: .leading, spacing: DesignTokens.Metrics.appLineGap) { + HStack { + Text(title).font(BlurtType.body(DesignTokens.Typography.sizeBody)).foregroundStyle(BlurtBrand.text) + if done { Image(systemName: "checkmark").foregroundStyle(BlurtBrand.accent) } + } + if !done { action() } + } + } + } +} diff --git a/App/BlurtiOS/BlurtiOS/Sources/SimulatorAudioSource.swift b/App/BlurtiOS/BlurtiOS/Sources/SimulatorAudioSource.swift new file mode 100644 index 00000000..20c72f03 --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/SimulatorAudioSource.swift @@ -0,0 +1,87 @@ +#if targetEnvironment(simulator) + import AudioToolbox + import BlurtEngine + import Foundation + + /// The simulator's microphone. `AVCaptureSession` carries no audio there + /// (FigCaptureSessionSimulator fails with -12782 the moment it starts), so + /// this feeds the same stream from an Audio Queue that asks for the dictation + /// format outright — 16 kHz mono 16-bit — and lets Core Audio convert from + /// the Mac's microphone. Never built for a phone; `WindowedAudioSource` is + /// the real thing and this only has to be good enough to exercise the loop. + nonisolated final class SimulatorAudioSource: ListeningSource, @unchecked Sendable { + /// 100 ms per buffer, three in flight. + private static let bufferBytes: UInt32 = 3200 + private static let bufferCount = 3 + + private let feed = UtteranceFeed() + private nonisolated(unsafe) var queue: AudioQueueRef? + private(set) nonisolated(unsafe) var isOpen = false + + var levels: AsyncStream { feed.levels } + + deinit { closeNow() } + + func open() async throws { + guard queue == nil else { return } + var format = AudioStreamBasicDescription( + mSampleRate: Double(SyncSTTLimits.sampleRate), mFormatID: kAudioFormatLinearPCM, + mFormatFlags: kLinearPCMFormatFlagIsSignedInteger | kLinearPCMFormatFlagIsPacked, + mBytesPerPacket: 2, mFramesPerPacket: 1, mBytesPerFrame: 2, mChannelsPerFrame: 1, mBitsPerChannel: 16, + mReserved: 0) + var created: AudioQueueRef? + let unmanagedSelf = Unmanaged.passUnretained(self).toOpaque() + guard AudioQueueNewInput(&format, Self.deliver, unmanagedSelf, nil, nil, 0, &created) == noErr, + let created + else { throw ListeningSourceFailure.noInputDevice } + for _ in 0.. AsyncStream { + guard isOpen else { + throw BlurtError.audioCaptureFailed(underlying: ListeningSourceFailure.windowClosed) + } + return feed.begin() + } + + func stop() -> Int { feed.end() } + + func cancelCapture() { feed.end() } + + /// Core Audio's callback: a C function pointer, so it captures nothing and + /// finds its way back to the source through the user-data pointer. + private static let deliver: AudioQueueInputCallback = { userData, queue, buffer, _, _, _ in + guard let userData else { return } + let source = Unmanaged.fromOpaque(userData).takeUnretainedValue() + let count = Int(buffer.pointee.mAudioDataByteSize) + if count > 0 { + source.feed.deliver(Data(bytes: buffer.pointee.mAudioData, count: count)) + } + _ = AudioQueueEnqueueBuffer(queue, buffer, 0, nil) + } + } +#endif diff --git a/App/BlurtiOS/BlurtiOS/Sources/StyleChips.swift b/App/BlurtiOS/BlurtiOS/Sources/StyleChips.swift new file mode 100644 index 00000000..689a3409 --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/StyleChips.swift @@ -0,0 +1,66 @@ +import BlurtEngine +import SwiftUI + +/// The output styles as chips under their eyebrow: Default and each profile, +/// the active one filled with the accent, rectangular like everything on the +/// brand; Edit opens the styles editor. +struct StyleChips: View { + @AppStorage(StyleProfileStore.defaultsKey) private var profilesRaw = "" + @AppStorage(StyleProfileStore.activeDefaultsKey) private var activeRaw = "" + private let styles = StyleProfileStore() + + private var profiles: [StyleProfile] { styles.profiles(decoding: profilesRaw) } + private var activeStyle: StyleProfile? { StyleProfileStore.active(in: profiles, id: activeRaw) } + + var body: some View { + VStack(alignment: .leading, spacing: DesignTokens.Metrics.appStackGap) { + Eyebrow("Style") + ScrollView(.horizontal, showsIndicators: false) { + HStack(spacing: DesignTokens.Metrics.appChipGap) { + StyleChip(name: StyleProfileStore.defaultStyleName, selected: activeStyle == nil) { styles.activateDefault() } + ForEach(profiles) { profile in + StyleChip(name: profile.name, selected: activeStyle?.id == profile.id) { styles.activate(profile) } + } + NavigationLink { + StylesView() + } label: { + Label("Edit", systemImage: "slider.horizontal.3") + .font(BlurtType.body(DesignTokens.Typography.sizeCaption, bold: true)) + .foregroundStyle(BlurtBrand.muted) + .padding(.horizontal, DesignTokens.Metrics.appChipPadX) + .padding(.vertical, DesignTokens.Metrics.appChipPadY) + .overlay( + RoundedRectangle(cornerRadius: DesignTokens.Metrics.radiusButton) + .strokeBorder(BlurtBrand.cardBorder, lineWidth: DesignTokens.Metrics.cardBorder)) + } + } + } + } + } +} + +/// One style, as a chip: filled with the accent while active. +private struct StyleChip: View { + let name: String + let selected: Bool + let activate: () -> Void + + var body: some View { + Button(action: activate) { + Text(name) + .font(BlurtType.body(DesignTokens.Typography.sizeCaption, bold: true)) + .padding(.horizontal, DesignTokens.Metrics.appChipPadX) + .padding(.vertical, DesignTokens.Metrics.appChipPadY) + .foregroundStyle(selected ? BlurtBrand.ctaText : BlurtBrand.text) + .background( + selected ? BlurtBrand.cta : BlurtBrand.cardFill, + in: RoundedRectangle(cornerRadius: DesignTokens.Metrics.radiusButton) + ) + .overlay( + RoundedRectangle(cornerRadius: DesignTokens.Metrics.radiusButton) + .strokeBorder(selected ? Color.clear : BlurtBrand.cardBorder, lineWidth: DesignTokens.Metrics.cardBorder)) + } + .buttonStyle(.plain) + .accessibilityAddTraits(selected ? .isSelected : []) + } +} diff --git a/App/BlurtiOS/BlurtiOS/Sources/StylesView.swift b/App/BlurtiOS/BlurtiOS/Sources/StylesView.swift new file mode 100644 index 00000000..07a18965 --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/StylesView.swift @@ -0,0 +1,151 @@ +import BlurtEngine +import SwiftUI + +/// Output styles: Default (the engine's base cleanup) plus up to four named +/// profiles whose instructions ride the cleanup rewrite. The store owns the +/// JSON and the active-style rule (`StyleProfileStore`); this view binds to the +/// raw defaults slots only to redraw when they change, and writes through the +/// store, the same discipline as the Mac's style row. +struct StylesView: View { + @AppStorage(StyleProfileStore.defaultsKey) private var profilesRaw = "" + @AppStorage(StyleProfileStore.activeDefaultsKey) private var activeRaw = "" + @State private var editing: StyleProfile? + @State private var adding = false + private let store = StyleProfileStore() + + private var profiles: [StyleProfile] { store.profiles(decoding: profilesRaw) } + private var active: StyleProfile? { StyleProfileStore.active(in: profiles, id: activeRaw) } + + var body: some View { + List { + Section { + StyleRow( + name: StyleProfileStore.defaultStyleName, detail: "Filler out, punctuation fixed, your words kept.", + isActive: active == nil + ) { store.activateDefault() } + ForEach(profiles) { profile in + StyleRow(name: profile.name, detail: profile.instructions, isActive: active?.id == profile.id) { + store.activate(profile) + } + .swipeActions { + Button("Edit") { editing = profile }.tint(BlurtBrand.green) + } + } + } header: { + Eyebrow("Styles") + } footer: { + Text("Tap a style to use it. Swipe a style to edit it. Up to \(StyleProfileStore.profileLimit).") + } + Section { + Button("Add style") { adding = true } + .disabled(profiles.count >= StyleProfileStore.profileLimit) + } + } + .brandForm() + .navigationTitle("Output styles") + .sheet(isPresented: $adding) { StyleEditor(profile: nil, store: store) } + .sheet(item: $editing) { StyleEditor(profile: $0, store: store) } + } +} + +private struct StyleRow: View { + let name: String + let detail: String + let isActive: Bool + let activate: () -> Void + + var body: some View { + Button(action: activate) { + HStack { + VStack(alignment: .leading, spacing: DesignTokens.Metrics.appLineGap) { + Text(name).foregroundStyle(BlurtBrand.text) + Text(detail).font(BlurtType.body(DesignTokens.Typography.sizeCaption)).foregroundStyle(BlurtBrand.muted) + .lineLimit(2) // literal-ok: two lines of instructions + } + Spacer() + if isActive { Image(systemName: "checkmark").foregroundStyle(BlurtBrand.green) } + } + } + } +} + +/// Name plus instructions, with the same caps the Mac's editor enforces: 24 +/// characters of name, and instructions trimmed to the bytes the API's +/// instruction cap leaves once the base cleanup text has taken its share. +private struct StyleEditor: View { + let profile: StyleProfile? + let store: StyleProfileStore + @Environment(\.dismiss) private var dismiss + @State private var name: String + @State private var instructions: String + + init(profile: StyleProfile?, store: StyleProfileStore) { + self.profile = profile + self.store = store + _name = State(initialValue: profile?.name ?? "") + _instructions = State(initialValue: profile?.instructions ?? "") + } + + private var canSave: Bool { + !name.trimmingCharacters(in: .whitespaces).isEmpty + && !instructions.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty + } + + var body: some View { + NavigationStack { + Form { + Section { + TextField("e.g. Casual", text: $name) + .onChange(of: name) { _, value in + if value.count > StyleProfileStore.nameLimit { name = String(value.prefix(StyleProfileStore.nameLimit)) } + } + } header: { + Eyebrow("Name") + } + Section { + TextField( + "e.g. add fitting emojis sparingly, or always write in lowercase", text: $instructions, axis: .vertical + ) + .lineLimit(3...8) // literal-ok: the editor's rows + .onChange(of: instructions) { _, value in + let trimmed = value.prefix(maxUTF8Bytes: StyleProfileStore.characterLimit) + if trimmed != value { instructions = trimmed } + } + } header: { + Eyebrow("Instructions") + } footer: { + Text("\(instructions.utf8.count)/\(StyleProfileStore.characterLimit)") + } + if profile != nil { + Section { + Button("Delete style", role: .destructive) { + store.profiles = store.profiles.filter { $0.id != profile?.id } + dismiss() + } + } + } + } + .brandForm() + .navigationTitle(profile == nil ? "New style" : "Edit style") + .toolbar { + ToolbarItem(placement: .cancellationAction) { Button("Cancel") { dismiss() } } + ToolbarItem(placement: .confirmationAction) { Button("Save") { save() }.disabled(!canSave) } + } + } + } + + private func save() { + var list = store.profiles + if let profile, let index = list.firstIndex(where: { $0.id == profile.id }) { + list[index].name = name + list[index].instructions = instructions + store.profiles = list + } else { + let created = StyleProfile(name: name, instructions: instructions) + list.append(created) + store.profiles = list + store.activate(created) + } + dismiss() + } +} diff --git a/App/BlurtiOS/BlurtiOS/Sources/ThemePickerView.swift b/App/BlurtiOS/BlurtiOS/Sources/ThemePickerView.swift new file mode 100644 index 00000000..de227131 --- /dev/null +++ b/App/BlurtiOS/BlurtiOS/Sources/ThemePickerView.swift @@ -0,0 +1,85 @@ +import BlurtiOSCore +import SwiftUI + +/// Pick a keyboard theme from live previews: a card per theme, each the real +/// keyboard drawn small in both of its faces — the light one the keyboard +/// shows over a light app, the dark one over a dark app — the chosen theme +/// ringed in the accent. One theme for now; the curated ones return through +/// `KeyboardPalette.all`. The keyboard picks the choice up the next time it +/// comes up. +struct ThemePickerView: View { + @State private var chosen = KeyboardPalette.resolve(SharedStore.themeID, dark: false).id + + var body: some View { + ScrollView { + VStack(spacing: DesignTokens.Metrics.appStackGap) { + ForEach(KeyboardPalette.all) { palette in + ThemeCard(palette: palette, chosen: palette.id == chosen) { + chosen = palette.id + SharedStore.themeID = palette.id + } + } + } + .padding(DesignTokens.Metrics.appPagePad) + } + .page() + .navigationTitle("Theme") + .navigationBarTitleDisplayMode(.inline) + } +} + +private struct ThemeCard: View { + let palette: KeyboardPalette + let chosen: Bool + let pick: () -> Void + + /// The preview keyboard is the full layout, drawn at the reference width + /// and scaled down to fit the card. + private static let previewWidth = DesignTokens.Metrics.pickerPreviewWidth + private static let scale = DesignTokens.Metrics.pickerScale + + var body: some View { + Button(action: pick) { + VStack(alignment: .leading, spacing: DesignTokens.Metrics.appStackGap) { + HStack(spacing: DesignTokens.Metrics.appPreviewGap) { + preview(KeyboardPalette.resolve(palette.id, dark: false)) + preview(KeyboardPalette.resolve(palette.id, dark: true)) + } + VStack(alignment: .leading, spacing: DesignTokens.Metrics.appLineGap) { + Text(palette.name).font(BlurtType.body(DesignTokens.Typography.sizeBody, bold: true)).foregroundStyle( + BlurtBrand.text) + Text(palette.vibe).font(BlurtType.body(DesignTokens.Typography.sizeCaption)).foregroundStyle(BlurtBrand.muted) + } + } + .padding(DesignTokens.Metrics.appCardPad) + .frame(maxWidth: .infinity, alignment: .leading) + .card() + .overlay( + RoundedRectangle(cornerRadius: DesignTokens.Metrics.cardRadius) + .strokeBorder(chosen ? BlurtBrand.accent : Color.clear, lineWidth: DesignTokens.Metrics.cardBorder)) + } + .buttonStyle(.plain) + .accessibilityLabel(palette.name) + .accessibilityAddTraits(chosen ? .isSelected : []) + } + + private func preview(_ face: KeyboardPalette) -> some View { + KeyboardRootView(model: Self.model(for: face)) + .frame(width: Self.previewWidth, height: KeyboardLayout.full.height) + .background(face.standIn) + .scaleEffect(Self.scale, anchor: .topLeading) + .frame(width: Self.previewWidth * Self.scale, height: KeyboardLayout.full.height * Self.scale) + .clipShape(RoundedRectangle(cornerRadius: DesignTokens.Metrics.pickerRadius)) + .allowsHitTesting(false) + } + + private static func model(for palette: KeyboardPalette) -> KeyboardModel { + let model = KeyboardModel() + model.layout = .full + model.hasFullAccess = true + model.isListening = true + model.needsGlobe = true + model.faceOverride = ThemeFace(palette) + return model + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/APIKeyModel.swift b/App/BlurtiOS/BlurtiOSCore/APIKeyModel.swift new file mode 100644 index 00000000..0cb42974 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/APIKeyModel.swift @@ -0,0 +1,81 @@ +import BlurtEngine +import Observation + +/// A copy of the Mac shell's `APIKeyModel` (App/Blurt/Blurt/APIKeyModel.swift): +/// nothing in it is platform-specific, and the two apps are separate targets +/// that cannot share a file without a third package. Keep the two in step. +/// +/// The app shell's API-key surface, extracted from `AppCoordinator` so the +/// coordinator stays focused on wiring the pipeline to the UI. Owns the storage +/// seam (`APIKeyGateway`), the validate-then-save flow (`APIKeySubmission`), and +/// the observable `hasAPIKey` flag the wizard/Settings react to. +/// +/// `@Observable` so views that only care about the key (the API-key step, the +/// wizard's readiness gate) observe *this* directly instead of reaching through +/// the coordinator. The validate-and-never-persist-an-unverified-key rules +/// themselves live in the engine's `APIKeySubmission`, where `swift test` covers +/// them; this type just forwards and mirrors the result into `hasAPIKey`. +@Observable +package final class APIKeyModel { + /// Storage for the API key. Production hits the Keychain via `APIKeyStore`; + /// UI tests inject an in-memory store so the real key is never touched. + @ObservationIgnored private let keyStore: any APIKeyGateway + /// The validate-then-save flow over `keyStore` (engine-owned + unit-tested). + /// Carries the injected validator: AssemblyAI's real network check in + /// production, an offline stub under UI testing — both through the one submit + /// path, so there's no test-only branch here. + @ObservationIgnored private let submission: APIKeySubmission + + /// Whether an AssemblyAI API key is currently saved. Drives the wizard (which + /// gates dictation on having a key) and the Settings UI. + package private(set) var hasAPIKey: Bool + + /// `validateKey` defaults to the engine's real AssemblyAI check; UI tests + /// inject an offline validator so the settings flow needs no network. + /// + /// The `hasKey` read below also loads the Keychain memo (`APIKeyStore.current`), + /// so every later readiness check on the press→recording latency path is + /// served from memory instead of paying a cold `SecItemCopyMatching`. + package init( + keyStore: any APIKeyGateway = ProductionAPIKeyStore(), + validateKey: @escaping @Sendable (String) async -> APIKeyValidator.Result = { + await APIKeyValidator().validate($0) + } + ) { + self.keyStore = keyStore + self.submission = APIKeySubmission(keyStore: keyStore, validate: validateKey) + self.hasAPIKey = keyStore.hasKey + } + + /// A `@Sendable` snapshot of the readiness gate for `DictationSession`: a press + /// with no key saved fails fast as `.failed(.apiKeyMissing)` before any capture. + /// Captures the (Sendable) store, not this main-actor model, so it can cross + /// into the session's `@Sendable` closure. + package func readinessCheck() -> @Sendable () -> BlurtError? { + let keyStore = keyStore + return { keyStore.hasKey ? nil : .apiKeyMissing } + } + + /// Re-reads the Keychain and updates `hasAPIKey`. The flag is otherwise only + /// set at `init` and after `submit`, so it can drift out of sync with what's + /// actually stored — e.g. an early-launch Keychain read fails (the item's ACL + /// needs re-approval after a re-sign) before a later read succeeds. Calling this + /// when the API-key UI appears keeps the readiness gate honest: a key that's + /// already present flips `hasAPIKey` true, so the wizard advances to the ready + /// screen instead of stranding the user on a setup step whose only control (a + /// *changed*-key "Update") is disabled. + package func refreshStatus() { + let hasKey = keyStore.hasKey + if hasKey != hasAPIKey { hasAPIKey = hasKey } + } + + /// Verifies `key` against AssemblyAI and saves it only when AssemblyAI + /// actively accepts it (`.valid`) — the engine's `APIKeySubmission` owns + /// (and unit-tests) that never-persist-an-unverified-key rule. Mirrors the + /// outcome into `hasAPIKey` so the wizard/Settings UI reacts. + package func submit(_ key: String) async -> APIKeySubmission.Outcome { + let outcome = await submission.submit(key) + refreshStatus() + return outcome + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/DarwinObserver.swift b/App/BlurtiOS/BlurtiOSCore/DarwinObserver.swift new file mode 100644 index 00000000..99c19f04 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/DarwinObserver.swift @@ -0,0 +1,30 @@ +import CoreFoundation + +/// A Darwin-notification subscription that lives as long as this object does. +/// +/// The handler runs on whichever thread the system delivers on and is +/// `@Sendable` for that reason; hop to the main actor inside it when the work +/// is UI. A class rather than a token so `deinit` can deregister — a listener +/// left behind outlives its owner, since the system holds the callback. +package nonisolated final class DarwinObserver: Sendable { + private let name: String + private let handler: @Sendable () -> Void + + package init(name: String, handler: @escaping @Sendable () -> Void) { + self.name = name + self.handler = handler + let observer = Unmanaged.passUnretained(self).toOpaque() + CFNotificationCenterAddObserver( + CFNotificationCenterGetDarwinNotifyCenter(), observer, + { _, observer, _, _, _ in + guard let observer else { return } + Unmanaged.fromOpaque(observer).takeUnretainedValue().handler() + }, name as CFString, nil, .deliverImmediately) + } + + deinit { + CFNotificationCenterRemoveObserver( + CFNotificationCenterGetDarwinNotifyCenter(), Unmanaged.passUnretained(self).toOpaque(), + CFNotificationName(name as CFString), nil) + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/HomeStatus.swift b/App/BlurtiOS/BlurtiOSCore/HomeStatus.swift new file mode 100644 index 00000000..8205a17c --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/HomeStatus.swift @@ -0,0 +1,44 @@ +import BlurtEngine +import Foundation + +/// What the home screen says about the app's state, in words: the headline, +/// the line under it, whether setup is complete, whether the last dictation +/// just landed. Pure, so the wording is tested and the view only lays it out. +package nonisolated struct HomeStatus: Equatable { + package let title: String + package let subtitle: String + /// A key, the microphone and the keyboard, all in place. + package let isSetUp: Bool + /// The words just went in (or to the clipboard): the hero's moment. + package let landed: Bool + + package init( + overlay: OverlayUIState, windowOpen: Bool, until: Date?, hasKey: Bool, micGranted: Bool, keyboardSeen: Bool + ) { + title = Self.title(overlay, windowOpen: windowOpen) + if !windowOpen { + subtitle = "Open the mic so the Blurt keyboard can dictate anywhere you type." + } else if let until, until != .distantFuture { + subtitle = "Until \(until.formatted(date: .omitted, time: .shortened)) · tap the mic on the Blurt keyboard." + } else { + subtitle = "Until you stop it · tap the mic on the Blurt keyboard." + } + isSetUp = hasKey && micGranted && keyboardSeen + switch overlay { + case .pasted, .noTarget: landed = true + default: landed = false + } + } + + private static func title(_ overlay: OverlayUIState, windowOpen: Bool) -> String { + switch overlay { + case .connecting: "Connecting…" + case .recording: "Listening…" + case .processing: "Transcribing…" + case .pasted: "Pasted" + case .noTarget: "Copied" + case .error(let message): message + case .idle: windowOpen ? "Ready to dictate" : "Not listening" + } + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/KeyTermList.swift b/App/BlurtiOS/BlurtiOSCore/KeyTermList.swift new file mode 100644 index 00000000..cc212d1a --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/KeyTermList.swift @@ -0,0 +1,21 @@ +import Foundation + +/// The comma-separated key-term list, as the engine's `KeyTermsStore` reads +/// it: split, trimmed, emptied of blanks, deduplicated case-insensitively in +/// first-seen order. +package nonisolated enum KeyTermList { + package static func parse(_ raw: String) -> [String] { + var seen = Set() + var terms: [String] = [] + for piece in raw.split(separator: ",") { + let term = piece.trimmingCharacters(in: .whitespacesAndNewlines) + guard !term.isEmpty, seen.insert(term.lowercased()).inserted else { continue } + terms.append(term) + } + return terms + } + + package static func join(_ terms: [String]) -> String { + terms.joined(separator: ", ") + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/KeyboardInteraction.swift b/App/BlurtiOS/BlurtiOSCore/KeyboardInteraction.swift new file mode 100644 index 00000000..e2e8eaa7 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/KeyboardInteraction.swift @@ -0,0 +1,108 @@ +import CoreGraphics + +/// The keyboard's touch rules, out of the views so they can be tested and +/// never drift between layouts (DESIGN.md › The two processes, and every +/// number): a key acts on release and only if the finger didn't travel; the +/// mic key waits a beat before it presses, so a swipe that starts on it never +/// starts a dictation it must then cancel; a horizontal swipe flips the +/// panel's carousel. +package nonisolated enum KeyboardInteraction { + /// The most a touch may travel and still be a tap on a key. + package static let tapTravel: CGFloat = 12 + /// A touch on the mic that travelled this far was a swipe, not a press. + package static let micTravel: CGFloat = 24 + /// A swipe flips the panel once it has gone this far sideways… + package static let swipeDistance: CGFloat = 48 + /// …and more sideways than up or down, by this factor. + package static let swipeRatio: CGFloat = 1.5 + /// How long a touch must stay before it counts as a press rather than the + /// start of a swipe — well under a tap's own duration for a hold. + package static let micPressDelay: Duration = .milliseconds(90) + + package enum FlipDirection: Equatable { + /// The finger moved left: the incoming page arrives from the trailing edge. + case towardsLeading + case towardsTrailing + } + + /// Which way the panel flips for this drag, or nil if the drag was not a swipe. + package static func flip(_ translation: CGSize) -> FlipDirection? { + let dx = translation.width + guard abs(dx) > swipeDistance, abs(dx) > abs(translation.height) * swipeRatio else { return nil } + return dx < 0 ? .towardsLeading : .towardsTrailing + } + + package static func isTap(_ translation: CGSize) -> Bool { + abs(translation.width) < tapTravel && abs(translation.height) < tapTravel + } + + /// How far past a key's edge a finger may end and still have tapped it. + package static let tapSlop: CGFloat = 8 + + /// A key's tap: the finger barely moved, or it ended on the key (within + /// `tapSlop` of its edges) wherever it went in between — a thumb that rolls + /// still types, as on the system keyboard, while a swipe that left the key + /// (the panel's carousel) does not. `location` is in the key's own space. + package static func isTap(_ translation: CGSize, endedAt location: CGPoint, in size: CGSize) -> Bool { + if isTap(translation) { return true } + guard size.width > 0, size.height > 0 else { return false } + return CGRect(origin: .zero, size: size).insetBy(dx: -tapSlop, dy: -tapSlop).contains(location) + } + + package static func micTravelled(_ translation: CGSize) -> Bool { + abs(translation.width) > micTravel || abs(translation.height) > micTravel + } +} + +/// The mic key's press, step by step: finger down arms a timer; the timer +/// firing sends the press; finger up decides. A tap quicker than the delay is +/// still a tap (press and release together); a swipe cancels the timer, or +/// undoes a press that already went out; nothing else happens. The view owns +/// the timer and the model calls; this owns the decisions. +package nonisolated struct MicPressSequencer: Equatable { + package enum Outcome: Equatable { + /// Release the key: send the press first if the timer never fired. + case tap(needsDown: Bool) + /// A swipe after the press went out: undo it. + case cancelSentPress + /// A swipe before the press went out: nothing to undo. + case nothing + } + + package private(set) var pressed = false + package private(set) var timerArmed = false + package private(set) var pressSent = false + + package init() {} + + /// The finger touched (or moved, again). True the first time: arm the timer. + package mutating func began() -> Bool { + guard !pressed else { return false } + pressed = true + timerArmed = true + return true + } + + /// The finger moved. True when the timer must be cancelled: it travelled + /// before the press went out. + package mutating func moved(travelled: Bool) -> Bool { + guard travelled, timerArmed, !pressSent else { return false } + timerArmed = false + return true + } + + /// The timer fired. True when the press goes out now. + package mutating func delayFired() -> Bool { + guard pressed, timerArmed else { return false } + timerArmed = false + pressSent = true + return true + } + + /// The finger lifted. + package mutating func ended(travelled: Bool) -> Outcome { + defer { self = MicPressSequencer() } + if travelled { return pressSent ? .cancelSentPress : .nothing } + return .tap(needsDown: !pressSent) + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/KeyboardModel+Field.swift b/App/BlurtiOS/BlurtiOSCore/KeyboardModel+Field.swift new file mode 100644 index 00000000..3b8740b9 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/KeyboardModel+Field.swift @@ -0,0 +1,36 @@ +import UIKit + +// MARK: - What the host field says: the return key, the symbols page, the face + +extension KeyboardModel { + /// What the field asked for, the way the system keyboard honours it: the + /// return key's own word, and the symbols page first for a number field. + package func readField() { + returnLabel = proxy?.returnKeyType.flatMap(Self.returnLabel) + } + + package func readAppearance() { + switch proxy?.keyboardAppearance { + case .dark: isDark = true + case .light: isDark = false + default: isDark = controller?.traitCollection.userInterfaceStyle == .dark + } + } + + /// Number, decimal and phone fields open on the symbols page, as the system + /// keyboard would show a number pad. + package static func wantsSymbols(_ type: UIKeyboardType?) -> Bool { + switch type { + case .numberPad, .decimalPad, .phonePad, .numbersAndPunctuation, .asciiCapableNumberPad: true + default: false + } + } + + /// iOS's own words for the return key, by the type the field asked for. + package static let returnLabels: [UIReturnKeyType: String] = [ + .go: "go", .google: "search", .yahoo: "search", .search: "search", .join: "join", .next: "next", + .route: "route", .send: "send", .done: "done", .emergencyCall: "call", .continue: "continue", + ] + + package static func returnLabel(_ type: UIReturnKeyType) -> String? { returnLabels[type] } +} diff --git a/App/BlurtiOS/BlurtiOSCore/KeyboardModel+Lexicon.swift b/App/BlurtiOS/BlurtiOSCore/KeyboardModel+Lexicon.swift new file mode 100644 index 00000000..0620b0b5 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/KeyboardModel+Lexicon.swift @@ -0,0 +1,37 @@ +import Foundation +import UIKit + +// MARK: - The phone's word list, handed to the app + +extension KeyboardModel { + /// The phone's own word list — contact names and text replacements — for + /// the app to spell names right with no setup. Read here because only the + /// keyboard is offered it; the app reads it back out of the App Group. + func requestLexicon() { + let refreshed = SharedStore.lexiconRefreshedAt ?? .distantPast + guard Date().timeIntervalSince(refreshed) > Self.lexiconRefreshInterval else { return } + // UIKit calls back on an XPC queue, not the main thread (a main-actor + // closure here traps), while the lexicon's entries are main-actor in + // Swift's eyes. So: take the callback anywhere, hand the immutable lexicon + // over, and read it on the main actor. + controller?.requestSupplementaryLexicon { @Sendable lexicon in + let handoff = LexiconHandoff(lexicon) + Task { @MainActor in + Self.store(handoff.lexicon.entries.map { LexiconEntry(userInput: $0.userInput, documentText: $0.documentText) }) + } + } + } + + fileprivate nonisolated struct LexiconHandoff: @unchecked Sendable { + let lexicon: UILexicon + init(_ lexicon: UILexicon) { self.lexicon = lexicon } + } + + /// Hands the word list to the app: written to the App Group, stamped, and + /// signalled, so the next dictation spells the names right. + static func store(_ entries: [LexiconEntry]) { + SharedStore.write(entries, forKey: BlurtShared.Key.lexicon) + SharedStore.lexiconRefreshedAt = Date() + SharedStore.post(BlurtShared.Signal.lexicon) + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/KeyboardModel+Mic.swift b/App/BlurtiOS/BlurtiOSCore/KeyboardModel+Mic.swift new file mode 100644 index 00000000..d905573e --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/KeyboardModel+Mic.swift @@ -0,0 +1,121 @@ +import BlurtEngine +import Foundation +import UIKit + +// MARK: - The mic key: the engine's gate, driven by the finger + +extension KeyboardModel { + /// Whether a tap on the orb can do anything but open the app. + package var isReady: Bool { hasFullAccess && isListening } + + /// What the voice control shows, from the phase the app published. + package var voiceState: VoiceState { VoiceState(snapshot: snapshot, isReady: isReady) } + + /// Finger down on the orb. Ignored, with a buzz, while the last dictation + /// is still being transcribed: a press then would be dropped by the engine + /// and the user would talk into nothing. + package func micDown() { + guard isReady else { return } + switch snapshot.state { + case .connecting, .processing: + if hasFullAccess { UINotificationFeedbackGenerator().notificationOccurred(.warning) } + return + case .idle, .recording, .pasted, .copied, .error: + let action = gate.modifierDown(at: elapsed) + if action == .start { pressSentOver = snapshot.state } + perform(action) + } + } + + /// Finger up. When Blurt isn't ready — no Full Access, or the app isn't + /// listening — this is the tap that opens the app, whose checklist says what + /// is missing (opening the app needs no Full Access; everything else does). + /// The gate comes up either way, so a press the finger made while the app + /// was still there is never left armed. + package func micUp() { + perform(gate.modifierUp(at: elapsed)) + if !isReady { openApp() } + } + + package func cancel() { + gate.reset() + send(.cancel) + } + + package var elapsed: Duration { clockStart.duration(to: ContinuousClock.now) } + + package func perform(_ action: DictationKeyGate.Action) { + switch action { + case .start: send(.press) + case .stop: + // The engine drops a release before it is recording (the mic still + // coming up): hold it for the first recording phase instead. + if snapshot.state == .connecting { + releasePending = true + } else { + send(.release) + } + case .cancel: + releasePending = false + send(.cancel) + case .none: break + } + } + + package func send(_ kind: KeyboardCommand.Kind) { + let command = KeyboardCommand( + id: UUID(), kind: kind, priorText: proxy?.documentContextBeforeInput, + selectedText: proxy?.selectedText, sentAt: Date(), keyboard: instanceID) + transport(command) + unansweredPressCommand = kind == .press ? command : nil + // A press the app never answers would leave the gate latched over + // nothing. Either notification can be missed: first catch up with a phase + // the app may have published unheard; if the picture is still as it was, + // the press itself was missed — put it back (the app takes a command out + // of the store as it reads it) and signal again. The same id, so an app + // that did read it ignores the copy. + commandRetry?.cancel() + guard kind == .press else { return } + let stateAtSend = snapshot.state + commandRetry = Task { [weak self] in + try? await Task.sleep(for: Self.commandRetryDelay) + guard !Task.isCancelled, let self else { return } + phaseChanged() + guard snapshot.state == stateAtSend else { return } + transport(command) + } + } + + /// Lets go of a press the app can no longer answer — it drops one older + /// than `BlurtShared.commandFreshnessWindow` unread, and an app that was + /// killed answers nothing — so its latch can't outlive it. True when it did. + @discardableResult + package func expireUnansweredPress() -> Bool { + guard let press = unansweredPressCommand, !press.isFresh() else { return false } + unansweredPressCommand = nil + return true + } + + /// The gate's half of the same rule: a latch held only for an expired + /// press goes with it, once nothing is in flight. + func letGoOfExpiredPress() { + if expireUnansweredPress(), isSettled { gate.reset() } + } + + /// A swipe that began on the orb (the panel's carousel) after its press + /// went out: undo exactly what that press did. A fresh press is cancelled; + /// a press over a recording a tap had latched only re-latches it, so + /// flipping to the keys mid-dictation doesn't throw the words away; a press + /// the orb ignored (busy, or Blurt not ready) undoes nothing. + /// + /// Nor does a fresh press sent over a dictation the app was still running: + /// the picture can say recording for a moment after this keyboard released + /// it (leaving the screen, opening the term field), and the engine drops a + /// press then, so it started nothing — a cancel would land on the words + /// being transcribed. + package func undoPress() { + let action = gate.otherKeyDown() + if action == .cancel, pressSentOver == .recording || pressSentOver == .processing { return } + perform(action) + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/KeyboardModel+Phase.swift b/App/BlurtiOS/BlurtiOSCore/KeyboardModel+Phase.swift new file mode 100644 index 00000000..5a6ccd51 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/KeyboardModel+Phase.swift @@ -0,0 +1,114 @@ +import BlurtEngine +import Foundation +import UIKit + +// MARK: - What comes back from the app: phase, results, haptics + +extension KeyboardModel { + package func phaseChanged() { + guard let current = SharedStore.read(PhaseSnapshot.self, forKey: BlurtShared.Key.phase) else { return } + apply(current.isStale ? .idle : current) + } + + package func apply(_ current: PhaseSnapshot, haptics wantsHaptics: Bool = true) { + // A notice the keyboard already let go of (its dwell ran out) is not + // replayed by a second read of the same snapshot. + if let dismissedNoticeAt, current.at == dismissedNoticeAt { return } + let previous = snapshot.state + // A dictation in flight belongs to an app that is there to run it: with + // the app gone (killed, the mic taken), the picture is idle, not a mic + // nobody can stop. + var current = current + isListening = SharedStore.isListening + if !isListening, current.noticeDwellSeconds == nil, current.state != .idle { current = .idle } + snapshot = current + if wantsHaptics, current.state != previous { haptics(from: previous, to: current.state) } + // A release the finger made while the mic was still coming up goes out + // now that it is recording; anything else in the meantime clears it. + if releasePending { + releasePending = false + if current.state == .recording { + send(.release) + gate.reset() + return + } + } + syncGate(with: current) + // A notice is over after its dwell: the Mac pill fades out, this orb goes + // back to its resting circle. Only the picture settles — a gate latched by + // a tap the app hasn't answered yet is left alone. + noticeDwell?.cancel() + dismissedNoticeAt = nil + if let seconds = current.noticeDwellSeconds { + let at = current.at + noticeDwell = Task { [weak self] in + try? await Task.sleep(for: .seconds(seconds)) + guard !Task.isCancelled, let self else { return } + dismissedNoticeAt = at + snapshot = .idle + } + } + } + + /// The gate follows the app, both ways — see the comments inside. + private func syncGate(with current: PhaseSnapshot) { + // The gate follows the app, both ways. A dictation that ended without a + // finger event (auto-release, an error) would leave it latched and swallow + // the next tap — the same sync the Mac shell does on every terminal phase. + // But only a phase that answers our press may settle it: the previous + // dictation's notice can land a moment after a new press, and resetting + // then would make the next tap a second press instead of the stop. + // And a dictation in flight that this keyboard didn't start (hands-free + // in another app, or a previous keyboard process) must latch it, so the + // next tap is the stop rather than a press the engine drops. + // A press too old for the app to take will never be answered at all. + letGoOfExpiredPress() + if !isSettled || current.command == unansweredPress { unansweredPressCommand = nil } + if isSettled, !gate.isIdle { + if unansweredPress == nil { gate.reset() } + } else if !isSettled, gate.isIdle, current.state != .processing { + // Only a recording (or one coming up) can be stopped by a tap; while + // the words are being transcribed there is nothing for the gate to hold. + _ = gate.modifierDown(at: elapsed) + _ = gate.modifierUp(at: elapsed) + } + } + + package func resultArrived() { + guard let result = SharedStore.read(DictationResult.self, forKey: BlurtShared.Key.result), + result.id != lastResultID, let proxy, + Date().timeIntervalSince(result.deliveredAt) < Self.resultFreshnessWindow, + result.recipient == nil || result.recipient == instanceID + else { return } + lastResultID = result.id + // Taken out of the store once inserted, so no other keyboard process — + // each host app runs its own — can insert the same words again; the app + // watches for this removal to know the words landed. + SharedStore.remove(forKey: BlurtShared.Key.result) + // Joined against the live text before the cursor, not the press-time + // snapshot: the user may have typed since. + proxy.insertText(InsertionSeparator.withLeadingSeparator(result.text, after: proxy.documentContextBeforeInput)) + resultLandedAt = Date() + // Not typing, so not the term field's to claim. + if termDraft != nil { termHostBaseline = proxy.documentContextBeforeInput ?? "" } + // The words replaced whatever was highlighted: the chip for that word + // goes now, not on the host's next notification. + readSelection() + } + + package func haptics(from previous: PhaseSnapshot.State, to state: PhaseSnapshot.State) { + guard hasFullAccess else { return } + switch state { + case .recording: + UIImpactFeedbackGenerator(style: .medium).impactOccurred() + case .processing where previous == .recording: + UIImpactFeedbackGenerator(style: .light).impactOccurred() + case .pasted, .copied: + UINotificationFeedbackGenerator().notificationOccurred(.success) + case .error: + UINotificationFeedbackGenerator().notificationOccurred(.error) + case .idle, .connecting, .processing: + break + } + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/KeyboardModel+Typing.swift b/App/BlurtiOS/BlurtiOSCore/KeyboardModel+Typing.swift new file mode 100644 index 00000000..9050773a --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/KeyboardModel+Typing.swift @@ -0,0 +1,295 @@ +import BlurtEngine +import Foundation +import UIKit + +// MARK: - Typing and the app + +extension KeyboardModel { + package func type(_ text: String) { + UIDevice.current.playInputClick() + if termDraft != nil { + termDraft?.append(text) + updateShift() + return + } + proxy?.insertText(text) + if shifted, !symbolsPage { shifted = false } + typedIntoHost() + } + + /// The keyboard's own edit to the host's text replaces whatever was + /// highlighted: the picture follows at once, not on the host's next + /// notification. + private func typedIntoHost() { + readSelection() + } + + package func deleteBackward() { + UIDevice.current.playInputClick() + if termDraft != nil { + _ = termDraft?.popLast() + updateShift() + return + } + proxy?.deleteBackward() + typedIntoHost() + } + + package func newline() { + UIDevice.current.playInputClick() + if termDraft != nil { + saveTerm() + return + } + proxy?.insertText("\n") + typedIntoHost() + } + + // MARK: Quick-add key term + + /// The most characters a highlighted word may have and still be offered as + /// a key term: a name or a phrase, not a sentence. + package static let termLengthCap = 48 + + /// A selection as a key term: trimmed, on one line, short enough to be a + /// name or a phrase. Nil when it is none of those, or nothing is selected. + package static func termCandidate(from selected: String?) -> String? { + guard let trimmed = selected?.trimmingCharacters(in: .whitespacesAndNewlines), !trimmed.isEmpty, + trimmed.count <= termLengthCap, !trimmed.contains(where: \.isNewline) + else { return nil } + return trimmed + } + + /// What is highlighted in the host's text, as a key-term candidate, and + /// whether Blurt has it already. The list lives in the App Group, so only + /// with Full Access — without it there is nothing the chip could do. A word + /// the chip is done with (`dismissedSelection`) shows no chip while the + /// same highlight stands. + package func readSelection() { + guard hasFullAccess, let term = Self.termCandidate(from: proxy?.selectedText) else { + selectedTerm = nil + selectedTermIsKnown = false + dismissedSelection = nil + return + } + if let dismissed = dismissedSelection { + guard dismissed.caseInsensitiveCompare(term) != .orderedSame else { + selectedTerm = nil + selectedTermIsKnown = false + return + } + dismissedSelection = nil + } + selectedTerm = term + selectedTermIsKnown = SharedStore.keyTerms.contains { $0.caseInsensitiveCompare(term) == .orderedSame } + } + + /// The chip: the highlighted word goes into Blurt's key terms as it stands + /// — one tap, no typing, the text untouched. The chip shows its check for + /// the notice's moment and then gives way to the + (`dismissSelection`), + /// highlight or not: the host may keep the word highlighted for as long as + /// the user leaves it, and a chip that stayed with it read as stuck. A tap + /// on a chip for a word Blurt already has puts the + back at once. + package func addSelectedTerm() { + guard let term = selectedTerm else { return } + guard !selectedTermIsKnown else { + dismissSelection() + return + } + SharedStore.addKeyTerm(term) + selectedTermIsKnown = true + noteTermSaved(thenDismissChip: true) + } + + /// The chip is done with the highlighted word: the + comes back and stays + /// back while that highlight stands (`KeyboardModel.dismissedSelection`). + package func dismissSelection() { + dismissedSelection = selectedTerm + selectedTerm = nil + selectedTermIsKnown = false + } + + /// The voice bar becomes a field and the keys type into it. If the user + /// had selected a word — the one Blurt got wrong — it is the starting + /// point, and saving also replaces it in the text with what they typed. + /// The field takes the mic key's row, so a dictation in flight is closed + /// first — nothing runs on with no key to stop it. From the plain + a + /// recording is released and the words still land; from a highlighted + /// word it is cancelled: the words would land *over* the word being fixed + /// (a result replaces the selection), and the user's intent is the word. + /// Words already being transcribed follow the same rule: from the plain + + /// they still land (the mic is off; there is nothing to stop), from a + /// highlighted word they are cancelled. + package func beginAddingTerm() { + let seed = Self.termCandidate(from: proxy?.selectedText) ?? "" + switch snapshot.state { + case .recording: perform(seed.isEmpty ? .stop : .cancel) + case .connecting: perform(.cancel) + case .processing: + // The gate may still be latched from the tap that started it (an + // auto-release has no finger event to close it): that is no reason to + // cancel. + if !seed.isEmpty { perform(.cancel) } + case .idle, .pasted, .copied, .error: + // A press the app hasn't answered yet would start a dictation under + // the field: it is taken back. + if unansweredPress != nil || !gate.isIdle { perform(.cancel) } + } + gate.reset() + termDraftFromSelection = seed.isEmpty ? nil : seed + termHostBaseline = proxy?.documentContextBeforeInput ?? "" + termHostBaselineAfter = proxy?.documentContextAfterInput ?? "" + termDraft = seed + updateShift() + onLayoutChange?() + } + + package func cancelAddingTerm() { + termDraft = nil + termDraftFromSelection = nil + termHostBaseline = nil + termHostBaselineAfter = nil + // The highlight may still stand: the chip comes back as it is now. + readSelection() + updateShift() + onLayoutChange?() + } + + /// Saves the term to Blurt's key terms — read on the very next dictation — + /// and, when it began as a selection, puts it into the text in place of the + /// misheard word. + package func saveTerm() { + guard let draft = termDraft?.trimmingCharacters(in: .whitespacesAndNewlines), !draft.isEmpty else { + cancelAddingTerm() + return + } + SharedStore.addKeyTerm(draft) + // Out of the mode before touching the text, so the field's own change + // notification can't read the replacement as typing to claim. + let original = termDraftFromSelection + let baseline = termHostBaseline + termDraft = nil + termDraftFromSelection = nil + termHostBaseline = nil + termHostBaselineAfter = nil + // Only while the highlight the field opened over still stands where it + // was: the host may report a selection a moment after the user moved on, + // and inserting then would put the word in twice. + if let original, original != draft, let selected = proxy?.selectedText, + selected.trimmingCharacters(in: .whitespacesAndNewlines) == original, + (proxy?.documentContextBeforeInput ?? "") == (baseline ?? "") + { + // The selection may have carried the spaces around the word; keep them. + let lead = selected.prefix { $0.isWhitespace } + let trail = selected.reversed().prefix { $0.isWhitespace }.reversed() + proxy?.insertText(String(lead) + draft + String(trail)) + } + // The word is saved; whatever is still highlighted, the + is back with + // its check, not a chip for the word just dealt with. + readSelection() + dismissSelection() + noteTermSaved(thenDismissChip: false) + updateShift() + onLayoutChange?() + } + + /// A success haptic, and the + (or the chip) shows a check for a moment; + /// after it, the chip gives way to the + when asked — only if it is still + /// the chip for the word that was saved: a word highlighted meanwhile + /// keeps its own chip. + private func noteTermSaved(thenDismissChip: Bool) { + if hasFullAccess { UINotificationFeedbackGenerator().notificationOccurred(.success) } + termSavedAt = Date() + termNotice?.cancel() + let saved = selectedTerm + termNotice = Task { [weak self] in + try? await Task.sleep(for: .seconds(Self.termNoticeDwell)) + guard !Task.isCancelled, let self else { return } + termSavedAt = nil + if thenDismissChip, let saved, let current = selectedTerm, + current.caseInsensitiveCompare(saved) == .orderedSame + { + dismissSelection() + } + } + } + + /// How long the + (or the chip) shows its check after a term was saved. + package static let termNoticeDwell: TimeInterval = 1.2 + + /// A space — or, tapped twice quickly after a word, the system keyboard's + /// "." shortcut: the first space becomes a full stop and a space. + package func space() { + UIDevice.current.playInputClick() + if termDraft != nil { + if termDraft?.isEmpty == false, termDraft?.hasSuffix(" ") == false { termDraft?.append(" ") } + updateShift() + return + } + let now = ContinuousClock.now + let before = proxy?.documentContextBeforeInput ?? "" + if let last = lastSpaceAt, now - last < .milliseconds(450), before.hasSuffix(" "), !before.hasSuffix(" "), + let previous = before.dropLast().last, previous.isLetter || previous.isNumber + { + proxy?.deleteBackward() + proxy?.insertText(". ") + lastSpaceAt = nil + } else { + proxy?.insertText(" ") + lastSpaceAt = now + } + typedIntoHost() + updateShift() + } + + package func toggleShift() { shifted.toggle() } + + /// 123 / ABC: the symbols come up on their first page, or the letters return. + package func toggleSymbols() { + symbolsPage.toggle() + morePage = false + } + + /// #+= / 123: between the two symbol pages; nothing on the letters. + package func toggleMore() { + guard symbolsPage else { return } + morePage.toggle() + } + + package func globe() { controller?.advanceToNextInputMode() } + + /// Brings the app forward to open the microphone — the one thing a keyboard + /// cannot do for itself. iOS gives extensions no `UIApplication.shared`, so + /// this walks the responder chain to the application object and asks it, the + /// way every keyboard that opens its app does. With `open(_:options:)`: since + /// iOS 18 the old `openURL:` only logs "BUG IN CLIENT OF UIKIT" and opens + /// nothing. + package func openApp() { + guard let controller, + let url = URL(string: "\(BlurtShared.urlScheme)://\(BlurtShared.startHost)") + else { return } + var responder: UIResponder? = controller + while let current = responder { + if let application = current as? UIApplication { + urlOpener(application, url) + return + } + responder = current.next + } + } +} + +/// The three letter rows, by the language the user types in most. +package nonisolated enum LetterLayout { + package static let qwerty = ["qwertyuiop", "asdfghjkl", "zxcvbnm"] + package static let azerty = ["azertyuiop", "qsdfghjklm", "wxcvbn"] + package static let qwertz = ["qwertzuiop", "asdfghjkl", "yxcvbnm"] + + /// From the phone's own language order — no setting to make. + package static func forPreferredLanguages(_ languages: [String] = Locale.preferredLanguages) -> [String] { + guard let first = languages.first?.lowercased() else { return qwerty } + if first.hasPrefix("fr") { return azerty } + if ["de", "cs", "sk", "hu"].contains(where: { first.hasPrefix($0) }) { return qwertz } + return qwerty + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/KeyboardModel.swift b/App/BlurtiOS/BlurtiOSCore/KeyboardModel.swift new file mode 100644 index 00000000..690e9819 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/KeyboardModel.swift @@ -0,0 +1,374 @@ +import BlurtEngine +import Foundation +import Observation +import UIKit + +/// The keyboard's state and its side of the conversation with the app: it +/// sends press / release / cancel with the text around the cursor, shows the +/// phase the app publishes, and inserts the words that come back. +/// +/// The mic key's tap-versus-hold rule is the engine's own `DictationKeyGate`, +/// driven by the finger instead of a modifier key: finger down is the key going +/// down, finger up the key coming up, so a tap latches and a hold is +/// push-to-talk exactly as on the Mac. +@MainActor +@Observable +package final class KeyboardModel { + /// How old a result may be and still be inserted. Darwin notifications are + /// held for a suspended process and delivered when it resumes, so the + /// keyboard inside the app the user *left* hears about a dictation done + /// elsewhere minutes later; anything older than this is not its to insert. + package static let resultFreshnessWindow: TimeInterval = 10 + /// The phone's word list is re-read at most this often. Contacts change by + /// the day, and reading thousands of them on every appearance is what the + /// keyboard's memory budget cannot afford. + package static let lexiconRefreshInterval: TimeInterval = 60 * 60 + + package var layout: KeyboardLayout = .panel + /// The panel is a two-page carousel — the mic panel and the full keyboard — + /// flipped by a swipe in either direction, as often as you like. Which page + /// is up; back to the mic each time the keyboard appears. + package var panelShowsKeys = false + /// The direction of the last flip, so the pages slide the way the finger went. + package var flipTowardsLeading = false + package var snapshot = PhaseSnapshot.idle + /// When this keyboard last put the words into the field — the orb's drop. + package var resultLandedAt: Date? + /// The key term being typed on the keys, while the voice bar is a field; + /// nil otherwise. See `beginAddingTerm`. + package var termDraft: String? + /// When the last term was saved, so the bar can show a check for a moment. + package var termSavedAt: Date? + /// The word highlighted in the host's text, when it could be a key term + /// (`termCandidate`) and the key terms are in reach (Full Access): the + + /// grows into a chip holding it, and one tap adds it. Read off the proxy + /// on every context change (`readSelection`); the gallery sets it. + package var selectedTerm: String? + /// Whether `selectedTerm` is already one of Blurt's key terms: the chip + /// shows a check and adds nothing. + package var selectedTermIsKnown = false + /// The field's own return key, as iOS labels it: "send", "search", "go"… + package var returnLabel: String? + /// The letter rows for the user's first keyboard language (AZERTY for + /// French, QWERTZ for German and its neighbours, QWERTY otherwise). + package var letterRows = LetterLayout.qwerty + package var isListening = false + package var hasFullAccess = false + package var needsGlobe = true + package var shifted = true + package var symbolsPage = false + /// The symbols' second page (#+=), only while `symbolsPage`. + package var morePage = false + + // Internal, not private: the model's typing, mic and phase halves live in + // KeyboardModel+Typing/+Mic/+Phase.swift. + @ObservationIgnored package weak var controller: UIInputViewController? + @ObservationIgnored private var observers: [DarwinObserver] = [] + @ObservationIgnored package var gate = DictationKeyGate() + @ObservationIgnored package let clockStart = ContinuousClock.now + @ObservationIgnored package var lastResultID: UUID? + @ObservationIgnored private var heartbeat: Task? + @ObservationIgnored package var lastSpaceAt: ContinuousClock.Instant? + @ObservationIgnored package var noticeDwell: Task? + /// This keyboard process, for results addressed to it (`DictationResult.recipient`). + @ObservationIgnored package let instanceID = UUID().uuidString + /// How `openApp` asks the application object, once the responder chain + /// reaches it: a test's capture in place of really opening the app. + @ObservationIgnored var urlOpener: (UIApplication, URL) -> Void = { application, url in + application.open(url, options: [:], completionHandler: nil) + } + /// Where commands go: the App Group, or a test's capture. + @ObservationIgnored package var transport: (KeyboardCommand) -> Void = { command in + SharedStore.write(command, forKey: BlurtShared.Key.command) + SharedStore.post(BlurtShared.Signal.command) + } + /// A press that got no phase back is re-signalled once (a Darwin + /// notification can be missed), after this long. + package static let commandRetryDelay: Duration = .milliseconds(600) + @ObservationIgnored package var commandRetry: Task? + @ObservationIgnored package var termDraftFromSelection: String? + /// A highlighted word the chip is done with — added, or its ✓ tapped — + /// while the host still shows it highlighted: the + is back, and stays + /// back until the highlight moves to another word or goes. + @ObservationIgnored package var dismissedSelection: String? + /// The text before and after the cursor when the term field opened, so + /// typing that reaches the host field anyway (a hardware keyboard: an + /// iPad's, a Bluetooth one, the simulator's Mac) can be pulled into the + /// term instead — and a cursor move, which changes both sides, cannot be + /// mistaken for it. + @ObservationIgnored package var termHostBaseline: String? + @ObservationIgnored package var termHostBaselineAfter: String? + @ObservationIgnored package var termNotice: Task? + /// A release made while the mic was still coming up (the engine drops + /// one before it records): held, and sent on the first recording phase. + @ObservationIgnored package var releasePending = false + /// The press the app has not answered yet. A settled phase that does not + /// answer it (the previous dictation's notice landing late) leaves the + /// gate alone; anything in flight, or a phase carrying its id, answers it. + /// Kept whole, so its id and its age can't disagree: one too old for the + /// app to take is never answered (`expireUnansweredPress`). + @ObservationIgnored package var unansweredPressCommand: KeyboardCommand? + package var unansweredPress: UUID? { unansweredPressCommand?.id } + /// What the picture showed when the orb last sent a press — whether that + /// press could have started a dictation at all (see `undoPress`). + @ObservationIgnored package var pressSentOver: PhaseSnapshot.State? + /// The `at` of a notice the keyboard already let go of: a re-read of the + /// same snapshot cannot replay it. + @ObservationIgnored package var dismissedNoticeAt: Date? + + /// The host's text field. Tests hand in a fake in place of a controller. + package var proxy: (any UITextDocumentProxy)? { proxyOverride ?? controller?.textDocumentProxy } + @ObservationIgnored package var proxyOverride: (any UITextDocumentProxy)? + + /// The chosen theme, by id. Its colours are the shell's: `palette` + /// (KeyboardModel+Palette.swift) resolves this, in the host's appearance. + package var themeID = SharedStore.themeID + /// For a preview: this theme face, whatever Settings and the host say. + package var faceOverride: ThemeFace? + /// The face to draw: the preview's, else the chosen theme in the host's + /// appearance — as `voiceKind` and `micAlignment` resolve theirs. + package var face: ThemeFace { faceOverride ?? ThemeFace(themeID: themeID, dark: isDark) } + /// TEMPORARY: which mic concept draws, from Settings — or, for a preview + /// or a gallery row, whatever it is told. Goes with the losers. + package var voiceKind: VoiceElementKind { voiceKindOverride ?? storedVoiceKind } + package var storedVoiceKind = SharedStore.voiceElementKind + package var voiceKindOverride: VoiceElementKind? + /// Where the mic key sits across the row, from Settings (one hand: at the + /// thumb's edge) — or, for a preview or a gallery row, whatever it is told. + package var micAlignment: MicAlignment { micAlignmentOverride ?? storedMicAlignment } + package var storedMicAlignment = SharedStore.micAlignment + package var micAlignmentOverride: MicAlignment? + /// Whether the app being typed in wants a dark keyboard: what its field + /// asks for, else the app's own light or dark appearance. + package var isDark = false + + /// What is actually on screen: the panel's carousel may be showing its + /// keys, and typing a key term needs them whatever the layout. + package var effectiveLayout: KeyboardLayout { + if termDraft != nil { return .full } + return layout == .panel && panelShowsKeys ? .full : layout + } + + /// Told when `effectiveLayout` changes, so the host can resize the keyboard. + @ObservationIgnored package var onLayoutChange: (() -> Void)? + + /// A horizontal swipe on the panel: flip to the other page, sliding the way + /// the finger went. + package func flipPanel(towardsLeading: Bool) { + // Only the panel has two pages, and not while the keys are up for a term. + guard layout == .panel, termDraft == nil else { return } + flipTowardsLeading = towardsLeading + panelShowsKeys.toggle() + onLayoutChange?() + } + + /// Whether the phase leaves nothing in flight — the moments the gate has to + /// be reset, since a dictation can end with no finger event to close it. + package var isSettled: Bool { + switch snapshot.state { + case .idle, .pasted, .copied, .error: true + case .connecting, .recording, .processing: false + } + } + + package init() {} + + // MARK: - Lifecycle + + package func attach(to controller: UIInputViewController) { + self.controller = controller + observers = [ + DarwinObserver(name: BlurtShared.Signal.phase) { [weak self] in + Task { @MainActor in self?.phaseChanged() } + }, + DarwinObserver(name: BlurtShared.Signal.result) { [weak self] in + Task { @MainActor in self?.resultArrived() } + }, + ] + } + + package func appeared() { + // Without a controller (a test's model) the access stays as it was set. + hasFullAccess = controller?.hasFullAccess ?? hasFullAccess + needsGlobe = controller?.needsInputModeSwitchKey ?? true + letterRows = LetterLayout.forPreferredLanguages() + // Every appearance starts on the mic page with no term half-typed, and on + // the page the field asks for: symbols for a number, letters otherwise. + panelShowsKeys = false + termDraft = nil + termDraftFromSelection = nil + termHostBaseline = nil + termHostBaselineAfter = nil + dismissedSelection = nil + termNotice?.cancel() + termSavedAt = nil + releasePending = false + // `unansweredPress` is not reset here: an appearance can follow another + // with no disappearance between (iOS does that), and a press still out + // must keep the gate latched so hands-free does not press again over it. + // One too old for the app to take is over, though (`expireUnansweredPress`). + symbolsPage = Self.wantsSymbols(proxy?.keyboardType) + morePage = false + readAppearance() + readField() + readSelection() + updateShift() + // Without Full Access the App Group is out of reach: the keyboard still + // types, and says what it needs (see `VoiceBar`), but nothing below can + // run — and nothing from a previous appearance (a phase, a latched gate) + // may linger over a keyboard that can't hear the app. + guard hasFullAccess else { + snapshot = .idle + gate.reset() + noticeDwell?.cancel() + return + } + layout = SharedStore.layout + themeID = SharedStore.themeID + storedVoiceKind = SharedStore.voiceElementKind + storedMicAlignment = SharedStore.micAlignment + SharedStore.keyboardEverSeen = true + refresh(haptics: false) + // `refresh` may apply no phase at all (none stored, or a notice already + // let go of), so a press the app will never answer is let go of here too: + // otherwise its latch would skip hands-free on every appearance after. + letGoOfExpiredPress() + startHeartbeat() + requestLexicon() + // Words that landed while this keyboard was away, still fresh and meant + // for whoever is up, go in now rather than waiting for a signal nobody + // will send again. + resultArrived() + if SharedStore.autoDictate { autoStart() } + } + + /// Hands-free: a dictation begins the moment the keyboard is up, as if the + /// mic had been tapped — the same synthetic tap through the engine's gate, + /// so it latches and the next real tap stops it. Only when the app is + /// listening and nothing is in flight; otherwise the orb sits dimmed and the + /// first tap opens Blurt. + private func autoStart() { + guard isListening, isSettled, gate.isIdle else { return } + perform(gate.modifierDown(at: elapsed)) + perform(gate.modifierUp(at: elapsed)) + } + + /// The keyboard is leaving the screen. A dictation it started must not run + /// on without it: a latched recording is released (the words still land, on + /// the clipboard if no keyboard is there to take them), anything earlier is + /// cancelled. Words already being transcribed are left to finish — the mic + /// is off, and a cancel would throw them away. Presence ends, so a result + /// that finishes after this goes to the clipboard rather than to a keyboard + /// nobody can see. + package func disappeared() { + heartbeat?.cancel() + heartbeat = nil + commandRetry?.cancel() + if !gate.isIdle || !isSettled { + switch snapshot.state { + case .recording: send(.release) + case .processing: break + case .idle, .connecting, .pasted, .copied, .error: send(.cancel) + } + gate.reset() + } + SharedStore.keyboardSeenAt = nil + } + + deinit { + heartbeat?.cancel() + noticeDwell?.cancel() + termNotice?.cancel() + commandRetry?.cancel() + } + + /// The cursor moved or the text around it changed, including by our own + /// typing: re-read where the sentence stands and what the field wants. + package func contextChanged() { + readAppearance() + readField() + readSelection() + claimHostTypingForTerm() + updateShift() + } + + /// While the term field is open, characters that arrived in the host field + /// (a hardware keyboard types past the on-screen keys) belong to the term: + /// move them over and take them back out of the field. Only a short, + /// appended run right after where the cursor was; anything else is left. + /// Never when the field opened over a highlighted word: a tap that + /// collapses that highlight to its end grows the text before the cursor + /// by exactly the word, which is not typing, and taking it would delete + /// the user's word. + private func claimHostTypingForTerm() { + guard termDraft != nil, termDraftFromSelection == nil, let baseline = termHostBaseline, let proxy else { + return + } + let now = proxy.documentContextBeforeInput ?? "" + let delta = now.count - baseline.count + // Typing lengthens the text before the cursor by one character at a time + // and leaves the text after it alone; a cursor move changes both; a + // selection is neither; a block (a paste, the system's own dictation + // under the keyboard) is never typing and stays in the field. + guard delta == 1, proxy.selectedText == nil, + (proxy.documentContextAfterInput ?? "") == (termHostBaselineAfter ?? ""), + now.dropLast(delta).hasSuffix(baseline.suffix(24)) + else { return } + let typed = String(now.suffix(delta)) + proxy.deleteBackward() + termDraft?.append(typed) + termHostBaseline = proxy.documentContextBeforeInput ?? "" + } + + /// Auto-capitalisation, as the system keyboard does it: shift comes on at + /// the start of a sentence (or of every word, or always) according to what + /// the field asks for, and goes off after one letter. + package func updateShift() { + if let termDraft { + // A key term is usually a name, or two: capitalised to start and after + // a space, then as typed. + shifted = termDraft.isEmpty || termDraft.last == " " + return + } + guard let proxy else { return } + let before = proxy.documentContextBeforeInput ?? "" + switch proxy.autocapitalizationType ?? .sentences { + case .none: break + case .allCharacters: shifted = true + case .words: shifted = before.isEmpty || before.last?.isWhitespace == true + case .sentences: shifted = Self.isSentenceStart(before) + @unknown default: break + } + } + + package static func isSentenceStart(_ before: String) -> Bool { + let trimmed = before.reversed().drop { $0 == " " } + guard let last = trimmed.first else { return true } + return last == "\n" || ".?!".contains(last) + } + + /// Catches up with whatever the app last published. No haptics: nothing + /// just happened, the keyboard merely came up. A notice (pasted, copied, + /// error) was for the keyboard that was up when it happened, not this + /// appearance; a dictation in flight with the app gone is over. + private func refresh(haptics: Bool) { + isListening = SharedStore.isListening + guard let current = SharedStore.read(PhaseSnapshot.self, forKey: BlurtShared.Key.phase) else { return } + apply(current.isStale || current.noticeDwellSeconds != nil ? .idle : current, haptics: haptics) + } + + /// Tells the app the keyboard is on screen, so a finished dictation is + /// handed here rather than copied to the clipboard. + private func startHeartbeat() { + heartbeat?.cancel() + heartbeat = Task { [weak self] in + while !Task.isCancelled { + guard let self else { return } + SharedStore.keyboardSeenAt = Date() + SharedStore.keyboardInstance = instanceID + isListening = SharedStore.isListening + try? await Task.sleep(for: .seconds(SharedStore.keyboardHeartbeatInterval)) + } + } + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/KeyboardMotion.swift b/App/BlurtiOS/BlurtiOSCore/KeyboardMotion.swift new file mode 100644 index 00000000..f4b9a254 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/KeyboardMotion.swift @@ -0,0 +1,7 @@ +import BlurtEngine + +/// The redraw cap for the keyboard's continuous motion — the sheen, the +/// meter's idle wave, the glints. The Mac pill reads the same number from +/// the engine: the level feed moves at this cadence, so drawing faster only +/// burns the keyboard's tight energy and memory budget. +package let keyboardAnimationInterval = MicCapture.meterIntervalSeconds diff --git a/App/BlurtiOS/BlurtiOSCore/KeyboardRelayInjector.swift b/App/BlurtiOS/BlurtiOSCore/KeyboardRelayInjector.swift new file mode 100644 index 00000000..6e10b65a --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/KeyboardRelayInjector.swift @@ -0,0 +1,58 @@ +import BlurtEngine +import Foundation +import UIKit + +/// The engine's paste seam on iPhone. Nothing here can type into another app; +/// only the keyboard can, so "insert" means hand the words to the keyboard +/// and ping it — then make sure they arrived. The keyboard takes the result +/// out of the App Group once it has inserted it; if it hasn't within a few +/// seconds (it was dismissed, suspended with its app, or killed), or if no +/// keyboard is on screen at all, the words go to the clipboard instead and +/// the pipeline shows its quiet "Copied" — the degradation the Mac uses when +/// nothing editable is focused — rather than a "Pasted" nobody saw. +/// +/// The text goes over as returned: the keyboard applies +/// `InsertionSeparator.withLeadingSeparator` against the *live* text before the +/// cursor, which may have moved since the press. +package nonisolated struct KeyboardRelayInjector: InjectorProtocol { + /// How long the keyboard gets to insert before the clipboard takes over. + package static let deliveryTimeout: TimeInterval = 3 + package static let deliveryPoll: Duration = .milliseconds(250) + + /// The keyboard the dictation in flight was pressed in + /// (`KeyboardCommand.keyboard`), or nil for whichever keyboard is up. + package var presser: @Sendable () -> String? = { nil } + + package init(presser: @escaping @Sendable () -> String? = { nil }) { + self.presser = presser + } + + package func setTarget(_ focus: CapturedFocus?) async {} + + package func insert(_ text: String, after priorText: String?, windowTitle: String?) async throws { + // The words go to the keyboard that asked for them. If the user has + // moved on — another app's keyboard is the one on screen now — they go + // to the clipboard rather than into a field nobody dictated into. + let onScreen = SharedStore.keyboardInstance + let recipient = presser() ?? onScreen + guard SharedStore.isKeyboardPresent, recipient == onScreen else { try await copyOut(text) } + let result = DictationResult(id: UUID(), text: text, deliveredAt: Date(), recipient: recipient) + SharedStore.write(result, forKey: BlurtShared.Key.result) + SharedStore.post(BlurtShared.Signal.result) + let deadline = Date().addingTimeInterval(Self.deliveryTimeout) + while Date() < deadline { + try await Task.sleep(for: Self.deliveryPoll) + guard let pending = SharedStore.read(DictationResult.self, forKey: BlurtShared.Key.result), + pending.id == result.id + else { return } + } + SharedStore.remove(forKey: BlurtShared.Key.result) + try await copyOut(text) + } + + /// The words to the clipboard, and the quiet "Copied" outcome. + private func copyOut(_ text: String) async throws -> Never { + await MainActor.run { UIPasteboard.general.string = text } + throw BlurtError.noEditableTarget + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/MicAlignment.swift b/App/BlurtiOS/BlurtiOSCore/MicAlignment.swift new file mode 100644 index 00000000..41cb568f --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/MicAlignment.swift @@ -0,0 +1,36 @@ +/// Where the mic key sits across the keyboard, for one hand: at the left +/// edge, in the middle (the default), or at the right edge. Left and right +/// are the phone's own sides — the side the thumb is on — whatever the +/// language reads in (Settings → Keyboard, under the layout; the keyboard +/// reads it on appearance). +package nonisolated enum MicAlignment: String, CaseIterable, Codable, Sendable, Identifiable { + case left + case center + case right + + package var id: String { rawValue } + + package var title: String { + switch self { + case .left: "Left alignment" + case .center: "Default" + case .right: "Right alignment" + } + } + + package var summary: String { + switch self { + case .left: "The mic key at the left edge of the keyboard, under your thumb." + case .center: "The mic key in the middle of the keyboard." + case .right: "The mic key at the right edge of the keyboard, under your thumb." + } + } +} + +extension MicAlignment { + /// `-BlurtGalleryAlign left|center|right`, or nil when the arguments don't say. + package static func parse(_ arguments: [String]) -> MicAlignment? { + guard let flag = arguments.firstIndex(of: "-BlurtGalleryAlign"), arguments.count > flag + 1 else { return nil } + return MicAlignment(rawValue: arguments[flag + 1]) + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/SharedState.swift b/App/BlurtiOS/BlurtiOSCore/SharedState.swift new file mode 100644 index 00000000..18bff687 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/SharedState.swift @@ -0,0 +1,389 @@ +import Foundation + +// MARK: - The App Group contract + +/// What the Blurt app and the Blurt keyboard agree on. They are two processes: +/// the app listens and transcribes (iOS lets no keyboard use the microphone), +/// the keyboard drops the words into the field it is attached to. They talk +/// through the App Group's shared defaults and wake each other with Darwin +/// notifications — a system-wide "something changed" ping that carries no data, +/// so every payload lives in the defaults under one of the keys below. +package nonisolated enum BlurtShared { + /// The App Group both targets declare. A placeholder namespace inherited from + /// the Mac app; an org-owned id replaces it before the App Store, and the app + /// and the keyboard must change together. + package static let appGroup = "group.dev.alex.blurt" + /// The URL the keyboard opens to bring the app forward (`blurt://start`) — + /// the one moment iOS insists on before the microphone may open. + package static let urlScheme = "blurt" + package static let startHost = "start" + /// A press command older than this was held while the app was suspended: + /// the app drops it unanswered, and the keyboard stops waiting for an answer. + package static let commandFreshnessWindow: TimeInterval = 10 + + package nonisolated enum Key { + package static let layout = "keyboardLayout" + package static let autoDictate = "autoDictate" + package static let theme = "keyboardTheme" + /// TEMPORARY: which mic concept draws (a, b or c), while the three are + /// tried in use. Goes with the losers once one is picked. + package static let voiceElement = "voiceElement" + /// Where the mic key sits across the keyboard (`MicAlignment`). + package static let micAlignment = "micAlignment" + package static let listeningUntil = "listeningUntil" + package static let windowMinutes = "listeningWindowMinutes" + package static let phase = "phase" + package static let command = "command" + package static let result = "result" + package static let keyboardSeenAt = "keyboardSeenAt" + package static let keyboardInstance = "keyboardInstance" + package static let appSeenAt = "appSeenAt" + package static let keyboardEverSeen = "keyboardEverSeen" + package static let lexicon = "lexicon" + package static let lexiconRefreshedAt = "lexiconRefreshedAt" + } + + /// Darwin notification names. Reverse-DNS so they can't collide with another + /// app's on the same device. + package nonisolated enum Signal { + package static let command = "dev.alex.blurt.ios.command" + package static let phase = "dev.alex.blurt.ios.phase" + package static let result = "dev.alex.blurt.ios.result" + package static let lexicon = "dev.alex.blurt.ios.lexicon" + } +} + +/// Which keyboard the user chose. All three share the same plumbing underneath; +/// they differ only in how much keyboard sits around the mic. +package nonisolated enum KeyboardLayout: String, CaseIterable, Codable, Sendable, Identifiable { + /// A slim strip: the mic, delete, return and the globe. Typing letters means + /// switching back to the system keyboard. + case slimBar + /// A big orb with cancel beside it and a few keys — the shape Wispr and + /// Aqua ship — that swipes to the full keyboard. + case panel + /// A complete keyboard with the mic as the main key, so nobody has to switch + /// keyboards to fix a typo. + case full + + package var id: String { rawValue } + + package var title: String { + switch self { + case .slimBar: "Mic bar" + case .panel: "Mic panel" + case .full: "Full keyboard" + } + } + + package var summary: String { + switch self { + case .slimBar: "Just the mic and a few keys. Switch keyboards to type." + case .panel: "A big mic with status and cancel. Swipe sideways for the full keyboard, and back." + case .full: "Every letter key, with the orb above them." + } + } +} + +// MARK: - Payloads +// +// Every type below is `nonisolated`: both targets default their declarations to +// the main actor (`SWIFT_DEFAULT_ACTOR_ISOLATION`), but these values cross into +// the app's capture path and the engine's closures, which run anywhere. + +/// What the keyboard asks the app to do, with the context only the keyboard can +/// see: the text before the cursor primes the transcript exactly as the Mac's +/// Accessibility read does. +package nonisolated struct KeyboardCommand: Codable, Sendable { + package nonisolated enum Kind: String, Codable, Sendable { + case press + case release + case cancel + } + + package let id: UUID + package let kind: Kind + package let priorText: String? + package let selectedText: String? + package let sentAt: Date + /// The keyboard process that sent it (`KeyboardModel.instanceID`): the + /// words come back to the field the press was made in, not whichever is up. + package let keyboard: String? + + /// Young enough for the app to act on. A notification held while the app + /// was suspended can deliver one from minutes ago, which the app drops + /// unread — so the keyboard stops waiting for an answer to it, too. + package func isFresh(now: Date = Date()) -> Bool { + now.timeIntervalSince(sentAt) < BlurtShared.commandFreshnessWindow + } +} + +/// The words the app got back, for the keyboard to insert. The keyboard joins +/// them onto the live text before the cursor itself (`InsertionSeparator`), +/// since the user may have typed since the press. +package nonisolated struct DictationResult: Codable, Sendable { + package let id: UUID + package let text: String + package let deliveredAt: Date + /// The keyboard instance the words are for — the one whose heartbeat the + /// app last saw — so a second live keyboard in another app doesn't also + /// insert them. Nil means whoever is up. + package let recipient: String? +} + +/// What the keyboard shows while the app works: the pipeline's phase, flattened +/// to what the orb can show, plus the live microphone level. +package nonisolated struct PhaseSnapshot: Codable, Sendable, Equatable { + package nonisolated enum State: String, Codable, Sendable { + case idle + case connecting + case recording + case processing + case pasted + case copied + case error + } + + package let state: State + package let message: String? + package let level: Double + package let at: Date + /// The keyboard command the app last took when it published this — the + /// press this phase answers, or the release or cancel that ended it. The + /// keyboard settles its gate only for the press it is waiting on: a notice + /// from the *previous* dictation landing a moment after a new press must + /// not read as that press being over. + package let command: UUID? + + package init(state: State, message: String?, level: Double, at: Date, command: UUID? = nil) { + self.state = state + self.message = message + self.level = level + self.at = at + self.command = command + } + + package static let idle = PhaseSnapshot(state: .idle, message: nil, level: 0, at: .distantPast) + + /// How long a notice stays on the orb before it settles back to idle — the + /// Mac's 0.8 s / 1.6 s dwell, a little longer since a phone has no hover to + /// reveal more. Nil for the states that end on their own. + package var noticeDwellSeconds: Double? { + switch state { + case .pasted: 1.2 + case .copied, .error: 2.0 + case .idle, .connecting, .recording, .processing: nil + } + } + + /// Whether this is too old to show. A notice (pasted, copied, error) dwells + /// for a moment and is then over; an in-flight state older than the longest + /// possible dictation belongs to an app that was killed under it. The + /// keyboard reads the snapshot back on every appearance and on every signal, + /// including signals held while it was suspended, so it must decide for + /// itself. + package var isStale: Bool { isStale(now: Date()) } + + package func isStale(now: Date) -> Bool { + let age = now.timeIntervalSince(at) + switch state { + case .idle: return false + case .pasted, .copied, .error: return age > Self.noticeStaleAfter + case .connecting, .recording, .processing: return age > Self.inFlightStaleAfter + } + } + + /// The longest notice dwell (2 s) plus a second of slack. + package static let noticeStaleAfter: TimeInterval = 3 + /// The engine's 120 s cap on one dictation, plus ten seconds for the + /// transcript to come back. + package static let inFlightStaleAfter: TimeInterval = 130 +} + +/// One entry of the phone's own word list (`UILexicon`): contact names and the +/// user's text replacements. Names go to the request as key terms so they come +/// back spelled right; replacements are the phone's own text shortcuts. +package nonisolated struct LexiconEntry: Codable, Sendable { + package let userInput: String + package let documentText: String + + /// Contact names come through with both fields equal; a text replacement + /// has a shortcut on one side and its expansion on the other. + package var isName: Bool { userInput == documentText } +} + +// MARK: - Shared defaults + +/// Typed access to the App Group's defaults, usable from any thread — the +/// keyboard writes from the main actor, the app's capture path reads from +/// wherever the engine calls it. +package nonisolated enum SharedStore { + /// The App Group's defaults — what both processes read and write. Falls + /// back to the process's own only where the group is out of reach (a + /// keyboard without Full Access), so nothing crashes; nothing is shared then. + /// Tests point this at a throwaway suite (`override`) so they never touch + /// the real one. + package static var defaults: UserDefaults { override ?? shared } + nonisolated(unsafe) static var override: UserDefaults? + // `UserDefaults` is thread-safe by contract; the type just isn't marked Sendable. + nonisolated(unsafe) private static let shared = UserDefaults(suiteName: BlurtShared.appGroup) ?? .standard + + package static func write(_ value: Value, forKey key: String) { + guard let data = try? JSONEncoder().encode(value) else { return } + defaults.set(data, forKey: key) + } + + package static func read(_ type: Value.Type, forKey key: String) -> Value? { + guard let data = defaults.data(forKey: key) else { return nil } + return try? JSONDecoder().decode(type, from: data) + } + + package static func remove(forKey key: String) { + defaults.removeObject(forKey: key) + } + + package static var layout: KeyboardLayout { + get { KeyboardLayout(rawValue: defaults.string(forKey: BlurtShared.Key.layout) ?? "") ?? .panel } + set { defaults.set(newValue.rawValue, forKey: BlurtShared.Key.layout) } + } + + /// Which keyboard theme (`KeyboardPalette.id`); the iPhone's own until one + /// is picked. + package static var themeID: String { + get { defaults.string(forKey: BlurtShared.Key.theme) ?? "system" } + set { defaults.set(newValue, forKey: BlurtShared.Key.theme) } + } + + /// TEMPORARY: the mic concept to draw — `VoiceElementKind.rawValue`, the + /// shipped one until Settings says otherwise. Read by the keyboard on every + /// appearance and by the home screen live. Goes with the losers. + package static var voiceElementKind: VoiceElementKind { + get { VoiceElementKind(rawValue: defaults.string(forKey: BlurtShared.Key.voiceElement) ?? "") ?? .shipped } + set { defaults.set(newValue.rawValue, forKey: BlurtShared.Key.voiceElement) } + } + + /// Where the mic key sits across the keyboard; the middle until chosen. + package static var micAlignment: MicAlignment { + get { MicAlignment(rawValue: defaults.string(forKey: BlurtShared.Key.micAlignment) ?? "") ?? .center } + set { defaults.set(newValue.rawValue, forKey: BlurtShared.Key.micAlignment) } + } + + /// Hands-free: the keyboard starts a dictation the moment it appears in a + /// text field, so there is nothing to tap before talking. On by default. + package static var autoDictate: Bool { + get { + guard defaults.object(forKey: BlurtShared.Key.autoDictate) != nil else { return true } + return defaults.bool(forKey: BlurtShared.Key.autoDictate) + } + set { defaults.set(newValue, forKey: BlurtShared.Key.autoDictate) } + } + + /// How long a listening window stays open with nobody dictating, in minutes. + /// 0 means until the user closes it. + package static var windowMinutes: Int { + get { + guard defaults.object(forKey: BlurtShared.Key.windowMinutes) != nil else { return 15 } + return defaults.integer(forKey: BlurtShared.Key.windowMinutes) + } + set { defaults.set(newValue, forKey: BlurtShared.Key.windowMinutes) } + } + + package static var listeningUntil: Date? { + get { defaults.object(forKey: BlurtShared.Key.listeningUntil) as? Date } + set { defaults.set(newValue, forKey: BlurtShared.Key.listeningUntil) } + } + + // MARK: Presence — the contract between the two processes + + /// Each side writes a timestamp every few seconds while it is there (the app + /// while its microphone is open, the keyboard while it is on screen), and + /// the other side counts it present while that timestamp is younger than + /// the window. The window is more than twice the heartbeat, so one missed + /// beat — a busy main thread, a suspended process resuming — doesn't read + /// as absence, while a killed process reads as absent within seconds. + package static let appHeartbeatInterval: TimeInterval = 5 + package static let keyboardHeartbeatInterval: TimeInterval = 4 + package static let presenceWindow: TimeInterval = 15 + + /// Whether the app currently has the microphone open for the keyboard: the + /// window has not lapsed, and the app is still there to hear — iOS may have + /// killed it, or a phone call taken the microphone, with the window's end + /// still in the future. The keyboard then opens the app rather than sending + /// a press nobody would answer. + package static var isListening: Bool { isListening(now: Date()) } + + package static func isListening(now: Date) -> Bool { + guard (listeningUntil ?? .distantPast) > now else { return false } + return now.timeIntervalSince(appSeenAt ?? .distantPast) < presenceWindow + } + + /// Whether a keyboard is on screen to take the words (see `presenceWindow`). + package static var isKeyboardPresent: Bool { + Date().timeIntervalSince(keyboardSeenAt ?? .distantPast) < presenceWindow + } + + /// Which keyboard process is on screen — a fresh id per process, written + /// with its heartbeat — so a result can be addressed to it. + package static var keyboardInstance: String? { + get { defaults.string(forKey: BlurtShared.Key.keyboardInstance) } + set { defaults.set(newValue, forKey: BlurtShared.Key.keyboardInstance) } + } + + // MARK: Key terms + + /// The user's own key terms — names and jargon the request should spell + /// right — under the engine's own key (`KeyTermsStore.defaultsKey`, + /// `BlurtKeyTerms`, comma-separated) but in the App Group, so the keyboard + /// can add one on the spot and the app reads it on the very next press. + package static var keyTerms: [String] { + get { KeyTermList.parse(defaults.string(forKey: keyTermsKey) ?? "") } + set { defaults.set(KeyTermList.join(newValue), forKey: keyTermsKey) } + } + + /// `BlurtKeyTerms` — spelled out here rather than imported from the engine so + /// the keyboard, which links it too, and the app can't disagree. + package static let keyTermsKey = "BlurtKeyTerms" + + /// Adds a term unless it is already there (case-insensitively). Returns + /// whether the list changed. + @discardableResult + package static func addKeyTerm(_ term: String) -> Bool { + let trimmed = term.trimmingCharacters(in: .whitespacesAndNewlines) + guard !trimmed.isEmpty else { return false } + var terms = keyTerms + guard !terms.contains(where: { $0.caseInsensitiveCompare(trimmed) == .orderedSame }) else { return false } + terms.append(trimmed) + keyTerms = terms + return true + } + + package static var appSeenAt: Date? { + get { defaults.object(forKey: BlurtShared.Key.appSeenAt) as? Date } + set { defaults.set(newValue, forKey: BlurtShared.Key.appSeenAt) } + } + + package static var keyboardSeenAt: Date? { + get { defaults.object(forKey: BlurtShared.Key.keyboardSeenAt) as? Date } + set { defaults.set(newValue, forKey: BlurtShared.Key.keyboardSeenAt) } + } + + /// When the keyboard last copied the phone's word list into the App Group. + package static var lexiconRefreshedAt: Date? { + get { defaults.object(forKey: BlurtShared.Key.lexiconRefreshedAt) as? Date } + set { defaults.set(newValue, forKey: BlurtShared.Key.lexiconRefreshedAt) } + } + + /// Set by the keyboard the first time it runs with Full Access — the only + /// public-API way the app can tell that the keyboard was added and allowed. + package static var keyboardEverSeen: Bool { + get { defaults.bool(forKey: BlurtShared.Key.keyboardEverSeen) } + set { defaults.set(newValue, forKey: BlurtShared.Key.keyboardEverSeen) } + } + + /// Fires a Darwin notification. Carries nothing: the receiver reads the + /// payload back out of these defaults. + package static func post(_ signal: String) { + CFNotificationCenterPostNotification( + CFNotificationCenterGetDarwinNotifyCenter(), CFNotificationName(signal as CFString), nil, nil, true) + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/TermPack.swift b/App/BlurtiOS/BlurtiOSCore/TermPack.swift new file mode 100644 index 00000000..aa82b731 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/TermPack.swift @@ -0,0 +1,97 @@ +import CoreTransferable +import Foundation +import UniformTypeIdentifiers + +extension UTType { + /// A shared list of Blurt key terms: JSON, `.blurtterms`, declared in + /// `project.yml` so a tap on one in Messages opens Blurt. + package nonisolated static let blurtTerms = UTType(exportedAs: "dev.alex.blurt.terms", conformingTo: .json) +} + +/// A list of key terms on its way to a friend — a group chat's names and +/// slang, so everyone's dictation gets them right. Shared as a `.blurtterms` +/// file (the share sheet writes it), opened by a tap; also readable from a +/// `blurt://terms?add=a,b,c&name=…&from=…` link. +package nonisolated struct TermPack: Codable, Identifiable, Transferable { + package var name: String + package var from: String? + package var terms: [String] + + package init(name: String, from: String?, terms: [String]) { + self.name = name + self.from = from + self.terms = terms + } + + package var id: String { "\(name)|\(from ?? "")|\(terms.joined(separator: ","))" } + + /// The request's own cap on key terms (`keyterms_prompt`, 100), which is + /// also as many as a list is worth; a file over this size, or a term, name + /// or sender longer than a line, is refused. + package static let termCap = 100 + package static let fileCap = 64 * 1024 + package static let lengthCap = 80 + + package static var transferRepresentation: some TransferRepresentation { + FileRepresentation(exportedContentType: .blurtTerms) { pack in + let url = FileManager.default.temporaryDirectory + .appendingPathComponent(pack.fileStem, isDirectory: false) + .appendingPathExtension("blurtterms") + let encoder = JSONEncoder() + encoder.outputFormatting = [.prettyPrinted, .sortedKeys] + try encoder.encode(pack).write(to: url, options: .atomic) + return SentTransferredFile(url) + } + } + + /// The shared file's name: the list's name with anything a file name can't + /// carry taken out, or a plain one when nothing is left. + var fileStem: String { + let stem = name.components(separatedBy: CharacterSet.alphanumerics.union(.whitespaces).inverted).joined() + .trimmingCharacters(in: .whitespaces) + return stem.isEmpty ? "Blurt key terms" : stem + } + + /// The list as plain text, for the message beside the file — a friend + /// without Blurt still gets the words. + package var plainText: String { + let lead = from.map { "\($0)'s Blurt key terms" } ?? "Blurt key terms" + return "\(lead) (\(name)): \(KeyTermList.join(terms))" + } + + /// Whether a URL is meant to be a pack at all — a `.blurtterms` file or a + /// `blurt://terms` link — so a broken one can be reported rather than + /// ignored. + package static func looksLikePack(_ url: URL) -> Bool { + if url.isFileURL { return url.pathExtension.lowercased() == "blurtterms" } + return url.scheme == BlurtShared.urlScheme && url.host() == "terms" + } + + /// A pack from a file tapped in Messages or a `blurt://terms` link; nil for + /// anything else the app is asked to open, or a pack over the caps. A file + /// iOS handed over in the app's Inbox is deleted once read. + package static func from(_ url: URL) -> TermPack? { + guard looksLikePack(url) else { return nil } + if url.isFileURL { + defer { if url.path.contains("/Inbox/") { try? FileManager.default.removeItem(at: url) } } + guard let size = (try? url.resourceValues(forKeys: [.fileSizeKey]))?.fileSize, size <= fileCap, + let data = try? Data(contentsOf: url), let pack = try? JSONDecoder().decode(TermPack.self, from: data) + else { return nil } + return pack.capped() + } + guard let items = URLComponents(url: url, resolvingAgainstBaseURL: false)?.queryItems else { return nil } + let value = { (name: String) in items.first { $0.name == name }?.value } + return TermPack( + name: value("name") ?? "Shared key terms", from: value("from"), terms: KeyTermList.parse(value("add") ?? "") + ).capped() + } + + /// The pack within the caps, or nil when nothing usable is left. + private func capped() -> TermPack? { + let kept = KeyTermList.parse(KeyTermList.join(terms)).filter { $0.count <= Self.lengthCap } + let terms = Array(kept.prefix(Self.termCap)) + guard !terms.isEmpty else { return nil } + return TermPack( + name: String(name.prefix(Self.lengthCap)), from: from.map { String($0.prefix(Self.lengthCap)) }, terms: terms) + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/ThemeFace.swift b/App/BlurtiOS/BlurtiOSCore/ThemeFace.swift new file mode 100644 index 00000000..fdfe325a --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/ThemeFace.swift @@ -0,0 +1,12 @@ +/// One face of a keyboard theme — its id and whether it is the dark one — +/// with no colours attached: the keyboard model holds a choice, and the shell +/// turns it into a palette (`KeyboardPalette.resolve`). +package nonisolated struct ThemeFace: Equatable, Sendable { + package var themeID: String + package var dark: Bool + + package init(themeID: String, dark: Bool) { + self.themeID = themeID + self.dark = dark + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/VoiceElementKind.swift b/App/BlurtiOS/BlurtiOSCore/VoiceElementKind.swift new file mode 100644 index 00000000..dd134f3b --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/VoiceElementKind.swift @@ -0,0 +1,21 @@ +/// The mic control's face — the one part of the keyboard that says what is +/// happening, without a word. Three concepts are built side by side behind +/// this seam so they can be judged on sight from the same states, the same +/// level and the same landing moment; the one that ships stays, the others +/// go. Pick with `-BlurtGalleryVoice a|b|c` (DESIGN.md › The voice element). +package nonisolated enum VoiceElementKind: String, CaseIterable, Sendable { + /// A: the grille — a dot-matrix mic head that lights as a stepped meter. + case grille = "a" + /// B: the ribs — the thin wave asleep as a ribbed grille, awake as the wave. + case ribs = "b" + /// C: the streak — a hairline with a bright point that blooms into a light streak. + case streak = "c" + + package static let shipped = VoiceElementKind.grille + + /// `-BlurtGalleryVoice a|b|c`, or nil when the arguments don't say. + package static func parse(_ arguments: [String]) -> VoiceElementKind? { + guard let flag = arguments.firstIndex(of: "-BlurtGalleryVoice"), arguments.count > flag + 1 else { return nil } + return VoiceElementKind(rawValue: arguments[flag + 1]) + } +} diff --git a/App/BlurtiOS/BlurtiOSCore/VoiceState.swift b/App/BlurtiOS/BlurtiOSCore/VoiceState.swift new file mode 100644 index 00000000..7f8e50f7 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSCore/VoiceState.swift @@ -0,0 +1,145 @@ +import BlurtEngine + +/// What the voice control shows, from what the app published: one struct the +/// keyboard's key and the app's hero both draw from, so the two never tell a +/// different story. The keyboard builds it from the App Group's phase +/// snapshot, the app from the engine's overlay state; every visual decision +/// — the ring, a glyph, dimming, whether the meter is up — is a property here +/// and tested, not a branch in a view. +package nonisolated struct VoiceState: Equatable { + package enum Phase: Equatable { + case idle + case connecting + case recording + case processing + case pasted + case copied + case error + } + + package enum Ring: Equatable { + /// At rest, or while recording (the meter says it all). + case still + /// The mic coming up, the words being made. + case sweeping + /// A notice: green for the words landing, orange for a failure. + case solid(Tone) + } + + package enum Tone: Equatable { + case ok + case error + } + + package enum Glyph: Equatable { + case clipboard + case exclamation + } + + package let phase: Phase + /// Whether a press can do anything but open the app: Full Access and the + /// app listening (the keyboard), the mic window open (the app). + package let isReady: Bool + /// The voice level while recording, 0…1. + package let level: Float + /// The error's own words, for VoiceOver. + package let message: String? + + package init(phase: Phase, isReady: Bool, level: Float = 0, message: String? = nil) { + self.phase = phase + self.isReady = isReady + self.level = level + self.message = message + } + + /// The keyboard's: from the phase the app published. + package init(snapshot: PhaseSnapshot, isReady: Bool) { + let phase: Phase = + switch snapshot.state { + case .idle: .idle + case .connecting: .connecting + case .recording: .recording + case .processing: .processing + case .pasted: .pasted + case .copied: .copied + case .error: .error + } + self.init(phase: phase, isReady: isReady, level: Float(snapshot.level), message: snapshot.message) + } + + /// The app's: from the engine's overlay state. "No target" is the words + /// going to the clipboard — the keyboard's "copied". + package init(overlay: OverlayUIState, windowOpen: Bool, level: Float) { + switch overlay { + case .idle: self.init(phase: .idle, isReady: windowOpen) + case .connecting: self.init(phase: .connecting, isReady: windowOpen) + case .recording: self.init(phase: .recording, isReady: windowOpen, level: level) + case .processing: self.init(phase: .processing, isReady: windowOpen) + case .pasted: self.init(phase: .pasted, isReady: windowOpen) + case .noTarget: self.init(phase: .copied, isReady: windowOpen) + case .error(let message): self.init(phase: .error, isReady: windowOpen, message: message) + } + } + + /// The meter is up and is the key. + package var isRecording: Bool { isReady && phase == .recording } + + /// Something is in flight: the mic coming up, the voice, the words being made. + package var isWorking: Bool { + guard isReady else { return false } + switch phase { + case .connecting, .recording, .processing: return true + case .idle, .pasted, .copied, .error: return false + } + } + + package var isNotice: Bool { + switch phase { + case .pasted, .copied, .error: true + case .idle, .connecting, .recording, .processing: false + } + } + + /// Nothing in flight — the moments the cancel control has nothing to cancel. + package var isSettled: Bool { + switch phase { + case .idle, .pasted, .copied, .error: true + case .connecting, .recording, .processing: false + } + } + + /// Something is in flight and Blurt is there to stop it: no × beside a + /// dimmed mic whose app is gone. + package var canCancel: Bool { !isSettled && isReady } + + package var ring: Ring { + switch phase { + case .error: return .solid(.error) + case .pasted, .copied: return .solid(.ok) + case .connecting, .processing: return isReady ? .sweeping : .still + case .idle, .recording: return .still + } + } + + /// Only the two notices that need saying carry a glyph. + package var glyph: Glyph? { + guard isReady else { return nil } + switch phase { + case .copied: return .clipboard + case .error: return .exclamation + case .idle, .connecting, .recording, .processing, .pasted: return nil + } + } + + /// Blurt isn't ready: the control sits back, and a tap opens the app. + package var dimmed: Bool { !isReady } + + package var accessibilityLabel: String { + guard isReady else { return "Start Blurt" } + return isRecording ? "Stop dictation" : "Dictate" + } + + package var accessibilityValue: String { + phase == .error ? message ?? "Dictation failed." : "" + } +} diff --git a/App/BlurtiOS/BlurtiOSProbe/AppleKeyboardProbe.swift b/App/BlurtiOS/BlurtiOSProbe/AppleKeyboardProbe.swift new file mode 100644 index 00000000..309687fd --- /dev/null +++ b/App/BlurtiOS/BlurtiOSProbe/AppleKeyboardProbe.swift @@ -0,0 +1,62 @@ +import XCTest + +/// Reads the system keyboard's geometry on this simulator: every key's frame, +/// the keyboard's own, and whatever else sits in it (the predictive bar), on +/// both faces. `scripts/apple-geometry.sh` runs this and turns the attachment +/// into `Design/apple-geometry.json`, which the `key/*` tokens are pinned to. +/// XCTest, because XCUIAutomation needs it (the one exception AGENTS.md makes). +final class AppleKeyboardProbe: XCTestCase { + @MainActor + func testMeasure() throws { + for face in ["light", "dark"] { + let app = XCUIApplication() + app.launchArguments = ["-BlurtProbeField", face] + app.launch() + let keyboard = app.keyboards.firstMatch + XCTAssertTrue( + keyboard.waitForExistence(timeout: 15), + "no on-screen keyboard: turn off 'simulate hardware keyboard' in Device Hub / Simulator") + // Let the predictive bar and the key pop-in settle. + Thread.sleep(forTimeInterval: 1.5) + var out: [String: Any] = [ + "face": face, + "window": rect(app.windows.firstMatch.frame), + "keyboard": rect(keyboard.frame), + ] + var keys: [String: Any] = [:] + for key in keyboard.keys.allElementsBoundByIndex { + let name = key.identifier.isEmpty ? key.label : key.identifier + guard !name.isEmpty, keys[name] == nil else { continue } + keys[name] = rect(key.frame) + } + out["keys"] = keys + var others: [String: Any] = [:] + for element in keyboard.otherElements.allElementsBoundByIndex.prefix(24) { + let name = element.identifier.isEmpty ? element.label : element.identifier + guard !name.isEmpty, others[name] == nil else { continue } + others[name] = rect(element.frame) + } + for element in keyboard.buttons.allElementsBoundByIndex.prefix(24) { + let name = "button:" + (element.identifier.isEmpty ? element.label : element.identifier) + guard others[name] == nil else { continue } + others[name] = rect(element.frame) + } + out["others"] = others + let data = try JSONSerialization.data(withJSONObject: out, options: [.prettyPrinted, .sortedKeys]) + let text = try XCTUnwrap(String(bytes: data, encoding: .utf8)) + let json = XCTAttachment(string: text) + json.name = "apple-geometry-\(face).json" + json.lifetime = .keepAlways + add(json) + let shot = XCTAttachment(screenshot: app.screenshot()) + shot.name = "apple-keyboard-\(face).png" + shot.lifetime = .keepAlways + add(shot) + app.terminate() + } + } + + private func rect(_ r: CGRect) -> [String: Double] { + ["x": r.minX, "y": r.minY, "w": r.width, "h": r.height] + } +} diff --git a/App/BlurtiOS/BlurtiOSProbe/BlurtKeyboardFlows+Reappearance.swift b/App/BlurtiOS/BlurtiOSProbe/BlurtKeyboardFlows+Reappearance.swift new file mode 100644 index 00000000..52efef6a --- /dev/null +++ b/App/BlurtiOS/BlurtiOSProbe/BlurtKeyboardFlows+Reappearance.swift @@ -0,0 +1,145 @@ +import XCTest + +// The ways a keyboard comes back — the system globe away and back, put +// away and brought up in the same field, a layout changed in Settings +// meanwhile, a rough swipe on the panel — each ending with the mic key +// there and drawn. `BlurtKeyboardFlows.testFlows` runs these last. + +// MARK: - Switching keyboards + +extension BlurtKeyboardFlows { + /// The globe under the keyboard: away to the system keyboard and back to + /// Blurt. The mic key must be there again, drawn and full size — the one + /// thing a fresh appearance in a living extension process can lose. + func switchAwayAndBack() throws { + let globe = app.buttons["Next keyboard"].firstMatch + guard globe.exists else { + check(false, "no globe under the keyboard to switch with") + return + } + // Once from the page that is up, and — on the panel — once more from its + // keys page, the way a swipe leaves it: the panel must come back on its + // mic page, at its own height, with the mic drawn. + try switchAwayAndBack(globe: globe, from: "the mic page") + if micKey().exists, app.buttons["space"].exists, !keysUp { + flipToKeys() + if keysUp { try switchAwayAndBack(globe: globe, from: "the keys page") } + } + // And the other way a keyboard comes back: put away and brought up again + // in the same field, from the mic page and from the keys page. + try dismissAndReturn(from: "the mic page") + if micKey().exists, app.buttons["space"].exists, !keysUp { + flipToKeys() + if keysUp { try dismissAndReturn(from: "the keys page") } + } + // And with the layout changed in Settings meanwhile: to each of the + // other two, then back to this run's own, the keyboard re-shown after + // each change. + let others = ["slimBar", "panel", "full"].filter { $0 != layoutName } + [layoutName] + for other in others where app.buttons["probe-layout-\(other)"].exists { + app.buttons["probe-layout-\(other)"].tap() + try dismissAndReturn(from: "a layout change to \(other)") + } + if layoutName == "panel", micKey().exists, !keysUp { try roughSwipes() } + } + + /// The swipes a finger makes and a clean drag does not: one that rests on + /// the mic long enough to press it before it travels (a press goes out and + /// is cancelled), and one taken while a dictation is recording. After each + /// pair the panel must be back with its mic drawn. + private func roughSwipes() throws { + let mic = micKey() + let held = mic.coordinate(withNormalizedOffset: CGVector(dx: 0.5, dy: 0.5)) + held.press(forDuration: 0.35, thenDragTo: held.withOffset(CGVector(dx: -160, dy: 0))) + Thread.sleep(forTimeInterval: 1.2) + check(keysUp, "a slow swipe from the mic flips to the keys (\(micKey().label))") + swipeBackFromLetters() + var frame = micKey().exists ? micKey().frame : .zero + check(panelMicUp, "after a slow swipe from the mic and back, the mic is there (\(frame), \(micKey().label))") + shot("rough-slow") + guard micKey().exists, micKey().label != "Start Blurt" else { return } + // Hands-free may have it recording already; otherwise a tap starts one. + if micKey().label == "Dictate" { micKey().tap() } + check(micKey(labelled: "Stop dictation").waitForExistence(timeout: 4), "a tap starts a dictation") + let gap = micKey().coordinate(withNormalizedOffset: CGVector(dx: 0.5, dy: 1.15)) + gap.press(forDuration: 0.05, thenDragTo: gap.withOffset(CGVector(dx: -160, dy: 0))) + Thread.sleep(forTimeInterval: 1.2) + check(keysUp, "a swipe while recording flips to the keys") + swipeBackFromLetters() + frame = micKey().exists ? micKey().frame : .zero + check(panelMicUp, "back on the panel while recording, the mic is there (\(frame), \(micKey().label))") + shot("rough-recording") + if micKey(labelled: "Stop dictation").exists { micKey().tap() } + _ = micKey(labelled: "Dictate").waitForExistence(timeout: 12) + } + + private func swipeBackFromLetters() { + let g = letterKey("g").exists ? letterKey("g") : letterKey("G") + let on = g.coordinate(withNormalizedOffset: CGVector(dx: 0.5, dy: 0.5)) + on.press(forDuration: 0.05, thenDragTo: on.withOffset(CGVector(dx: 160, dy: 0))) + Thread.sleep(forTimeInterval: 1.2) + } + + /// A swipe on the panel's gap, to its keys page. + private func flipToKeys() { + let gap = micKey().coordinate(withNormalizedOffset: CGVector(dx: 0.5, dy: 1.15)) + gap.press(forDuration: 0.05, thenDragTo: gap.withOffset(CGVector(dx: -160, dy: 0))) + Thread.sleep(forTimeInterval: 1.2) + } + + private func dismissAndReturn(from page: String) throws { + let dismiss = app.buttons["probe-dismiss"] + guard dismiss.exists else { return } + let plus = app.buttons["Add a key term"] + dismiss.tap() + Thread.sleep(forTimeInterval: 1.5) + check(!plus.exists && !micKey().exists, "the keyboard goes away (from \(page))") + app.textFields.firstMatch.tap() + let back = plus.waitForExistence(timeout: 5) || micKey().waitForExistence(timeout: 2) + Thread.sleep(forTimeInterval: 1.5) + let mic = micKey() + let frame = mic.exists ? mic.frame : .zero + check( + back && frame.width > 40 && frame.height > 20, + "brought back from \(page) the mic key is there, full size (\(frame))") + checkDrawn(micKey(), "brought back from \(page)") + if frame == .zero { print("FLOW-TREE (mic missing after the return):\n\(app.debugDescription)") } + if page == "the keys page" { check(!keysUp, "the panel is back on its mic page after being put away") } + shot("returned-\(page.replacingOccurrences(of: " ", with: "-"))") + } + + private func switchAwayAndBack(globe: XCUIElement, from page: String) throws { + // XCUITest has no Keyboard element for a custom keyboard: the + stands in + // for where the keyboard begins — 8 pt under its top on the keys page + // (the voice bar), 58 on the panel (the mic's centre line, 80, less half + // its 44 pt hit). + let plus = app.buttons["Add a key term"] + let topBefore = plus.exists ? plus.frame.minY : 0 + globe.tap() + Thread.sleep(forTimeInterval: 2) + check(!plus.exists, "the globe switches away from Blurt (from \(page))") + shot("switched-away") + var hops = 0 + while !plus.waitForExistence(timeout: 2), hops < 4 { + app.buttons["Next keyboard"].firstMatch.tap() + hops += 1 + } + Thread.sleep(forTimeInterval: 1.5) + let mic = micKey() + let top = plus.exists ? plus.frame.minY : 0 + // One snapshot: an element that flickers would throw between two. + let frame = mic.exists ? mic.frame : .zero + check( + frame.width > 40 && frame.height > 20, + "back on Blurt from \(page) the mic key is there, full size (\(frame); top \(topBefore) → \(top))") + checkDrawn(mic, "back on Blurt from \(page)") + if frame == .zero { print("FLOW-TREE (mic missing after the switch):\n\(app.debugDescription)") } + if page == "the keys page" { + check(!keysUp, "the panel is back on its mic page after the switch") + // The keys page was up before, so the keyboard was 54 pt taller then + // (270 − 216), and the + sat 50 pt nearer its top (8 against 58). + check(abs(top - topBefore - 104) < 4, "the panel is back at its own height (top \(topBefore) → \(top))") + } + shot("switched-back") + } +} diff --git a/App/BlurtiOS/BlurtiOSProbe/BlurtKeyboardFlows.swift b/App/BlurtiOS/BlurtiOSProbe/BlurtKeyboardFlows.swift new file mode 100644 index 00000000..15d5365b --- /dev/null +++ b/App/BlurtiOS/BlurtiOSProbe/BlurtKeyboardFlows.swift @@ -0,0 +1,344 @@ +import XCTest + +/// The real keyboard, used: the extension in its own process over a field in +/// the app, driven through every layout's flows — the mic key present and +/// centred, the + beside it, the panel's carousel swiping to the keys and +/// back, the key-term field opening and closing, letters typing into the +/// field, the mic starting and stopping a dictation while the app listens. +/// One run per face × mic concept × layout (`TEST_RUNNER_BLURT_PROBE_ARGS`); +/// every step prints a FLOW line and attaches a screenshot, and +/// `scripts/ios-keyboard-flows.sh` runs the matrix and collects them. +@MainActor +final class BlurtKeyboardFlows: XCTestCase { + var app = XCUIApplication() + var shots = 0 + /// `left` or `right` when the run puts the mic at an edge (Settings › + /// alignment); nil for the middle. + var side: String? + /// The layout this run is of: `slimBar`, `panel` or `full`. + var layoutName = "panel" + + func testFlows() throws { + continueAfterFailure = true + let arguments = + ProcessInfo.processInfo.environment["BLURT_PROBE_ARGS"]?.split(separator: " ").map(String.init) ?? ["light"] + let layout = arguments.first { ["slimBar", "panel", "full"].contains($0) } ?? "panel" + layoutName = layout + side = arguments.first { ["left", "right"].contains($0) } + app = XCUIApplication() + // The app listens from launch, so the keyboard's mic key is live. + app.launchArguments = + ["-BlurtProbeField"] + arguments + ["-BlurtStartListening", "-BlurtProbeResetTerms", "-BlurtProbeText", "Rizz"] + app.launch() + try bringUpBlurt() + shot("up") + let mic = micKey() + check(mic.exists, "mic key present (\(mic.label))") + // "Stop dictation" is hands-free already recording: ready too. + check(mic.label != "Start Blurt", "mic key ready: the app is listening (\(mic.label))") + let plus = app.buttons["Add a key term"] + check(plus.exists, "the + present") + checkDrawn(mic, "on appearance") + if mic.exists, plus.exists { checkPlacement(layout: layout, mic: mic, plus: plus) } + switch layout { + case "panel": try panelFlows() + case "full": try fullFlows() + default: try slimFlows() + } + try selectionFlow(app.textFields.firstMatch) + try termFlow() + try dictationFlow() + try switchAwayAndBack() + } + + // MARK: Helpers + + /// Where the mic and the + sit, for this run's layout and side. + private func checkPlacement(layout: String, mic: XCUIElement, plus: XCUIElement) { + let width = app.windows.firstMatch.frame.width + let centre = mic.frame.midX + // Wherever the mic is, the + sits on its centre line. + check( + abs(plus.frame.midY - mic.frame.midY) < 4, + "the + at the mic's centre line (\(plus.frame.midY) vs \(mic.frame.midY))") + if let side { + // One hand: the mic at the chosen edge, the + on its inner side. The + // slim bar's row is bounded by the globe (when Blurt draws one; on a + // Face ID phone the system puts it under the keyboard) and delete + // keys, so there the mic is next to that key rather than the edge. + let left = side == "left" + let globe = app.buttons["globe"].firstMatch + let bound = + layout == "slimBar" + ? (left ? (globe.exists ? globe.frame.maxX : 0) : app.buttons["delete.left"].firstMatch.frame.minX) + : (left ? 0 : width) + let edge = left ? mic.frame.minX - bound : bound - mic.frame.maxX + let inner = left ? plus.frame.minX > mic.frame.maxX : plus.frame.maxX < mic.frame.minX + check(edge < 12, "mic at the \(side) edge (\(edge) pt in)") + check(inner, "the + on the mic's inner side") + } else if layout == "panel" { + check(abs(centre - width / 2) < 4, "mic centred in the panel (\(centre) of \(width))") + check(plus.frame.minX > mic.frame.maxX, "the + beside the mic, to its right") + } else { + // The element itself is centred in the voice bar's own width — the + // whole row in the full keyboard, the room between the globe (if any) + // and delete in the slim bar — with the + hanging beside it. + let globe = app.buttons["globe"].firstMatch + let from = layout == "slimBar" && globe.exists ? globe.frame.maxX : 0 + let to = layout == "slimBar" ? app.buttons["delete.left"].firstMatch.frame.minX : width + check(abs(centre - (from + to) / 2) < 8, "mic centred in its row (\(centre) in \(from)…\(to))") + check(plus.frame.minX > mic.frame.maxX, "the + beside the mic, to its right") + } + } + + private func bringUpBlurt() throws { + let blurt = app.buttons["Add a key term"] + let system = app.keyboards.firstMatch + let deadline = Date().addingTimeInterval(15) + while !blurt.exists, !system.exists, Date() < deadline { Thread.sleep(forTimeInterval: 0.5) } + var hops = 0 + while !blurt.waitForExistence(timeout: 2), hops < 4 { + let globe = app.buttons["Next keyboard"] + XCTAssertTrue(globe.waitForExistence(timeout: 5), "no globe key") + globe.tap() + hops += 1 + } + XCTAssertTrue(blurt.exists, "Blurt never came up") + Thread.sleep(forTimeInterval: 1.5) + } + + /// Blurt's mic key, by its identifier: the system's own bottom bar has a + /// "Dictate" button too, and a label query lands on it. + func micKey() -> XCUIElement { + app.buttons.matching(NSPredicate(format: "identifier == %@", "blurt-mic")).firstMatch + } + + func micKey(labelled label: String) -> XCUIElement { + app.buttons.matching(NSPredicate(format: "identifier == %@ AND label == %@", "blurt-mic", label)).firstMatch + } + + func letterKey(_ letter: String) -> XCUIElement { + app.buttons.matching(NSPredicate(format: "label == %@", letter)).firstMatch + } + + func check(_ condition: Bool, _ what: String) { + print("FLOW-\(condition ? "OK" : "FAIL"): \(what)") + XCTAssertTrue(condition, what) + } + + /// The mic element's pixels are there — a key with nothing drawn in it is + /// the failure the frame checks cannot see. + func checkDrawn(_ mic: XCUIElement, _ when: String) { + let ink = MicPixels.ink(app, mic: mic) ?? 0 + check(ink > MicPixels.drawnThreshold, "the mic is drawn \(when) (ink \(ink))") + } + + func shot(_ name: String) { + shots += 1 + let attachment = XCTAttachment(screenshot: app.screenshot()) + attachment.name = "flow-\(String(format: "%02d", shots))-\(name).png" + attachment.lifetime = .keepAlways + add(attachment) + } +} + +// MARK: - Flows + +extension BlurtKeyboardFlows { + // MARK: Flows + + private func panelFlows() throws { + // Where a finger swipes on the panel: the empty space under the element, + // the element itself, the space key. Each attempt says where it started + // and whether the carousel flipped; the page is put back between them. + let mic = { self.micKey().coordinate(withNormalizedOffset: CGVector(dx: 0.5, dy: 0.5)) } + let gap = { self.micKey().coordinate(withNormalizedOffset: CGVector(dx: 0.5, dy: 1.15)) } + let space = { self.app.buttons["space"].coordinate(withNormalizedOffset: CGVector(dx: 0.5, dy: 0.5)) } + let attempts = [ + Swipe(origin: "the gap under the mic, left", start: gap, dx: -160), + Swipe(origin: "the mic itself, left", start: mic, dx: -160), + Swipe(origin: "the space key, left", start: space, dx: -160), + Swipe(origin: "the gap under the mic, right", start: gap, dx: 160), + Swipe(origin: "the space key, right", start: space, dx: 160), + ] + for (index, swipe) in attempts.enumerated() { + let (origin, dx) = (swipe.origin, swipe.dx) + let from = swipe.start() + from.press(forDuration: 0.05, thenDragTo: from.withOffset(CGVector(dx: dx, dy: 0))) + Thread.sleep(forTimeInterval: 1.2) + let flipped = keysUp + check(flipped, "swipe from \(origin) flips the panel to the keys") + if index == 0 { shot("panel-keys") } + if flipped { + // Back to the panel from the letters, the swipe that is known to work. + let g = letterKey("g").exists ? letterKey("g") : letterKey("G") + let on = g.coordinate(withNormalizedOffset: CGVector(dx: 0.5, dy: 0.5)) + on.press(forDuration: 0.05, thenDragTo: on.withOffset(CGVector(dx: dx > 0 ? -160 : 160, dy: 0))) + Thread.sleep(forTimeInterval: 1.2) + check(!keysUp, "swipe from the letters (\(dx > 0 ? "left" : "right")) flips back to the panel") + if index == 0 { shot("panel-back") } + } + } + check(!keysUp && panelMicUp, "the panel is showing again, its mic full size") + checkDrawn(micKey(), "after the swipes back") + // Pairs from one fixed screen point (an element mid-slide is no anchor): + // a flip and a flip back, and two the same way — with no wait between, + // so the second lands mid-slide, and again after the slide has settled. + // Each pair lands on the panel, the cards never having crossed. + let anchor = gap().screenPoint + let fixed = app.coordinate(withNormalizedOffset: .zero).withOffset(CGVector(dx: anchor.x, dy: anchor.y)) + let pairs = [ + Pair(name: "left then right, mid-slide", first: -160, second: 160, pause: 0), + Pair(name: "left twice, mid-slide", first: -160, second: -160, pause: 0), + Pair(name: "left then right, settled", first: -160, second: 160, pause: 0.5), + Pair(name: "left twice, settled", first: -160, second: -160, pause: 0.5), + ] + for pair in pairs { + fixed.press(forDuration: 0.05, thenDragTo: fixed.withOffset(CGVector(dx: pair.first, dy: 0))) + if pair.pause > 0 { Thread.sleep(forTimeInterval: pair.pause) } + fixed.press(forDuration: 0.05, thenDragTo: fixed.withOffset(CGVector(dx: pair.second, dy: 0))) + Thread.sleep(forTimeInterval: 1.5) + check(!keysUp && panelMicUp, "two quick swipes, \(pair.name), land back on the panel") + shot("panel-pair-\(pair.name.replacingOccurrences(of: ", ", with: "-").replacingOccurrences(of: " ", with: "-"))") + } + } + + /// Two swipes from one point: how far each goes, and the wait between. + private struct Pair { + let name: String + let first: CGFloat + let second: CGFloat + let pause: TimeInterval + } + + /// Where a swipe starts and how far it goes. + private struct Swipe { + let origin: String + let start: () -> XCUICoordinate + let dx: CGFloat + } + + /// Whether the letter keys are on screen (either shift state). + var keysUp: Bool { letterKey("Q").exists || letterKey("q").exists } + + /// Whether the mic key on screen is the panel's: as tall as the panel's + /// element box (88), where the bar's is the row (44). The width says + /// nothing — the key is as wide as the element shows, whichever concept. + var panelMicUp: Bool { micKey().exists && micKey().frame.height > 60 } + + private func fullFlows() throws { + let q = letterKey("Q") + check(q.exists, "letter keys present") + q.tap() + letterKey("q").tap() + let field = app.textFields.firstMatch + check( + (field.value as? String)?.lowercased().contains("qq") == true, + "letters type into the field (\(field.value ?? "-"))") + check( + app.buttons["return"].exists || app.buttons["RETURN"].exists || app.buttons["Return"].exists, + "return key labelled") + shot("full-typed") + } + + /// Highlight the word in the field: the + becomes a chip holding it, one + /// tap adds it, and the chip then says Blurt has it. + private func selectionFlow(_ field: XCUIElement) throws { + // Double-tap the word itself — it sits at the field's leading edge — not + // the field's empty middle, and once more after a beat if the highlight + // didn't take the first time (a shot of the miss says why). + // Either face of the chip: the word to add, or — were the list not reset — + // the word Blurt already knows. + let chip = app.buttons.matching( + NSPredicate(format: "label BEGINSWITH %@ OR label ENDSWITH %@", "Add “", "” is a key term") + ).firstMatch + let mic = micKey() + for attempt in 1...2 where !chip.exists { + field.coordinate(withNormalizedOffset: CGVector(dx: 0.08, dy: 0.5)).doubleTap() + if chip.waitForExistence(timeout: 3) { break } + shot("\(layoutName)-select-miss-\(attempt)") + } + let label = chip.exists ? chip.label : "-" + check(chip.exists, "highlighting a word shows the chip (\(label))") + if chip.exists, mic.exists { + check(abs(chip.frame.midY - mic.frame.midY) < 4, "the chip at the mic's centre line") + check( + chip.frame.minX >= mic.frame.maxX - 1 || chip.frame.maxX <= mic.frame.minX + 1, + "the chip clear of the mic (\(chip.frame) vs \(mic.frame))") + } + shot("\(layoutName)-selected") + let plus = app.buttons["Add a key term"] + if label.hasPrefix("Add “") { + let word = label.dropFirst("Add “".count).prefix { $0 != "”" } + chip.tap() + let known = app.descendants(matching: .any) + .matching(NSPredicate(format: "label == %@", "“\(word)” is a key term")).firstMatch + // The check shows for 1.2 s; XCUITest's snapshot of another process + // can miss it, so the proof of the add is below: the word re-selected + // is a word Blurt knows. + if known.waitForExistence(timeout: 2) { shot("\(layoutName)-added") } + // The highlight still stands in the field; the chip gives way to the + + // on its own after the check, so the row is never stuck on the word. + check(plus.waitForExistence(timeout: 4), "after the check the + is back while the word stays highlighted") + check(mic.exists && mic.label != "Start Blurt", "the mic is still the mic after adding (\(mic.label))") + shot("\(layoutName)-after-add") + // Cleared and highlighted again: Blurt knows the word now, and a tap on + // that chip puts the + back at once. + field.coordinate(withNormalizedOffset: CGVector(dx: 0.95, dy: 0.5)).tap() + Thread.sleep(forTimeInterval: 0.5) + field.coordinate(withNormalizedOffset: CGVector(dx: 0.08, dy: 0.5)).doubleTap() + check(known.waitForExistence(timeout: 4), "one tap added “\(word)”: highlighted again, the chip knows it") + shot("\(layoutName)-known") + if known.exists { + known.tap() + check(plus.waitForExistence(timeout: 3), "a tap on the known chip puts the + back") + } + } else if chip.exists { + check(false, "the word was already a key term before this run (\(label)) — the list was not reset") + } + // Clear the highlight, cursor at the end, so typing appends rather than + // replaces; the + is there either way. + field.coordinate(withNormalizedOffset: CGVector(dx: 0.95, dy: 0.5)).tap() + check(plus.waitForExistence(timeout: 3), "the + present once the highlight is cleared") + Thread.sleep(forTimeInterval: 0.5) + } + + private func slimFlows() throws { + check(app.buttons["delete.left"].exists, "delete present on the slim bar") + shot("slim") + } + + private func termFlow() throws { + let plus = app.buttons["Add a key term"] + guard plus.exists else { return } + plus.tap() + let cancel = app.buttons["Cancel"] + check(cancel.waitForExistence(timeout: 3), "the + opens the key-term field") + check(letterKey("Q").exists, "the keys come up for the term") + letterKey("R").tap() + letterKey("i").tap() + shot("term") + let save = app.buttons["Save the key term"] + check(save.exists && save.isEnabled, "save enabled after typing") + cancel.tap() + check(micKey().waitForExistence(timeout: 3), "cancel returns to the mic") + } + + private func dictationFlow() throws { + let mic = micKey() + guard mic.exists else { return } + if mic.label == "Start Blurt" { + check(false, "mic not ready: the app is not listening (\(mic.label))") + return + } + mic.tap() + let stop = micKey(labelled: "Stop dictation") + check(stop.waitForExistence(timeout: 4), "a tap starts a dictation (recording)") + Thread.sleep(forTimeInterval: 1.5) + shot("recording") + if stop.exists { stop.tap() } + let back = micKey(labelled: "Dictate") + check(back.waitForExistence(timeout: 12), "a second tap stops it and the key settles") + shot("settled") + } +} diff --git a/App/BlurtiOS/BlurtiOSProbe/BlurtKeyboardShot.swift b/App/BlurtiOS/BlurtiOSProbe/BlurtKeyboardShot.swift new file mode 100644 index 00000000..830b0f9d --- /dev/null +++ b/App/BlurtiOS/BlurtiOSProbe/BlurtKeyboardShot.swift @@ -0,0 +1,41 @@ +import XCTest + +/// A screenshot of the real Blurt keyboard — the extension, in its own +/// process, over a text field — for the states the gallery cannot vouch +/// for. Launches the app on `-BlurtProbeField` (which also sets the face, +/// the mic concept and the layout), taps the system's globe until Blurt is +/// up, and attaches the screen. `scripts/ios-keyboard-shot.sh` runs it and +/// exports the attachment. Blurt must be enabled as a keyboard on the +/// simulator (Settings → General → Keyboard → Keyboards) once by hand. +final class BlurtKeyboardShot: XCTestCase { + @MainActor + func testShot() throws { + let arguments = + ProcessInfo.processInfo.environment["BLURT_PROBE_ARGS"]?.split(separator: " ").map(String.init) ?? [] + let app = XCUIApplication() + app.launchArguments = ["-BlurtProbeField"] + (arguments.isEmpty ? ["light"] : arguments) + app.launch() + // Blurt's own control: the + that adds a key term, which no system + // keyboard has. XCUITest sees the system keyboard as a keyboard and a + // custom one only through its controls, so wait for either; then the + // globe cycles the enabled keyboards until Blurt is up. + let blurt = app.buttons.matching(NSPredicate(format: "label BEGINSWITH %@", "Add a key term")).firstMatch + let system = app.keyboards.firstMatch + let deadline = Date().addingTimeInterval(15) + while !blurt.exists, !system.exists, Date() < deadline { Thread.sleep(forTimeInterval: 0.5) } + XCTAssertTrue(blurt.exists || system.exists, "no on-screen keyboard") + var hops = 0 + while !blurt.waitForExistence(timeout: 2), hops < 4 { + let globe = app.buttons["Next keyboard"] + XCTAssertTrue(globe.waitForExistence(timeout: 5), "no globe key: is more than one keyboard enabled?") + globe.tap() + hops += 1 + } + XCTAssertTrue(blurt.exists, "Blurt's keyboard never came up: enable it in Settings → General → Keyboard") + Thread.sleep(forTimeInterval: 2) + let shot = XCTAttachment(screenshot: app.screenshot()) + shot.name = "blurt-keyboard.png" + shot.lifetime = .keepAlways + add(shot) + } +} diff --git a/App/BlurtiOS/BlurtiOSProbe/MessagesCarouselProbe.swift b/App/BlurtiOS/BlurtiOSProbe/MessagesCarouselProbe.swift new file mode 100644 index 00000000..8c3a5728 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSProbe/MessagesCarouselProbe.swift @@ -0,0 +1,118 @@ +import XCTest + +/// The panel's carousel inside Messages — Neil's host — swiped every which +/// way from one fixed point, with a screenshot after each swipe and a +/// FLOW line saying which page is up. Run under `simctl io recordVideo` to +/// see the motion itself. Blurt must be enabled as a keyboard on the +/// simulator, and the App Group set to the panel (`-BlurtProbeField light a +/// panel` once). +@MainActor +final class MessagesCarouselProbe: XCTestCase { + private var shots = 0 + + func testSwipes() throws { + continueAfterFailure = true + let messages = XCUIApplication(bundleIdentifier: "com.apple.MobileSMS") + messages.launch() + let plus = try bringUpBlurt(in: messages) + Thread.sleep(forTimeInterval: 2) + let mic = messages.buttons.matching(NSPredicate(format: "identifier == %@", "blurt-mic")).firstMatch + XCTAssertTrue(mic.exists, "no mic key") + print("FLOW-INFO mic \(mic.frame) plus \(plus.frame)") + let inkUp = MicPixels.ink(messages, mic: mic) ?? 0 + print("FLOW-\(inkUp > MicPixels.drawnThreshold ? "OK" : "FAIL"): the mic is drawn on appearance (ink \(inkUp))") + shot(messages, "up") + // One fixed point: the gap under the panel's mic. + let anchor = mic.coordinate(withNormalizedOffset: CGVector(dx: 0.5, dy: 1.15)).screenPoint + let fixed = messages.coordinate(withNormalizedOffset: .zero).withOffset(CGVector(dx: anchor.x, dy: anchor.y)) + let sequence = [ + Swipe("left", -160, 1.2), Swipe("right", 160, 1.2), Swipe("left", -160, 1.2), Swipe("left", -160, 1.2), + Swipe("right", 160, 1.2), Swipe("right", 160, 1.2), Swipe("left", -160, 0), Swipe("right", 160, 1.2), + Swipe("left", -160, 0), Swipe("left", -160, 1.2), Swipe("right", 160, 0), Swipe("left", -160, 1.2), + ] + for (index, swipe) in sequence.enumerated() { + fixed.press(forDuration: 0.05, thenDragTo: fixed.withOffset(CGVector(dx: swipe.dx, dy: 0))) + guard swipe.pause > 0 else { continue } + Thread.sleep(forTimeInterval: swipe.pause) + let keys = messages.buttons.matching(NSPredicate(format: "label == %@ OR label == %@", "Q", "q")).firstMatch + // The panel's mic is element-wide when aligned to an edge, so its + // presence, not its width, says the panel is up. + let up = keys.exists ? "keys" : (mic.exists ? "panel" : "NEITHER") + let micFrame = mic.exists ? mic.frame : .zero + let plusFrame = plus.exists ? plus.frame : .zero + print("FLOW-INFO after \(index + 1) (\(swipe.name)): \(up) mic \(micFrame) plus \(plusFrame)") + if up == "panel" { + // The panel's mic key is the grille's box, the + beside it: anything + // else is the vanished-mic state Neil sees in Messages. + let beside = plusFrame.minX >= micFrame.maxX - 1 || plusFrame.maxX <= micFrame.minX + 1 + let sound = micFrame.width > 100 && micFrame.height > 60 && beside && abs(plusFrame.midY - micFrame.midY) < 4 + print("FLOW-\(sound ? "OK" : "FAIL"): panel back with the mic full size and the + beside it (\(index + 1))") + let ink = MicPixels.ink(messages, mic: mic) ?? 0 + print( + "FLOW-\(ink > MicPixels.drawnThreshold ? "OK" : "FAIL"): the mic is drawn after the return (\(index + 1), ink \(ink))" + ) + if !sound { + print( + "FLOW-TREE\n\(messages.debugDescription.split(separator: "\n").filter { $0.contains("Button") || $0.contains("Other") }.prefix(60).joined(separator: "\n"))" + ) + } + } + shot(messages, "\(index + 1)-\(swipe.name)") + } + } + + /// One swipe: how far, and how long to wait after it (0: the next comes at once). + private struct Swipe { + let name: String + let dx: CGFloat + let pause: TimeInterval + init(_ name: String, _ dx: CGFloat, _ pause: TimeInterval) { + self.name = name + self.dx = dx + self.pause = pause + } + } + + /// A conversation, its field, and Blurt on it: the globe's menu goes + /// straight there; hopping does when there is no menu. Returns the +. + private func bringUpBlurt(in messages: XCUIApplication) throws -> XCUIElement { + let cell = messages.cells.firstMatch + if cell.waitForExistence(timeout: 5) { + cell.tap() + } else { + messages.buttons["compose"].firstMatch.tap() + } + let field = messages.descendants(matching: .any).matching( + NSPredicate(format: "placeholderValue == %@ OR label == %@", "iMessage", "iMessage") + ).firstMatch + XCTAssertTrue(field.waitForExistence(timeout: 10), "no message field") + field.tap() + let plus = messages.buttons.matching(NSPredicate(format: "label BEGINSWITH %@", "Add a key term")).firstMatch + var hops = 0 + while !plus.waitForExistence(timeout: 2), hops < 12 { + let globe = messages.buttons["Next keyboard"] + XCTAssertTrue(globe.waitForExistence(timeout: 5), "no globe key") + if hops == 0 { + globe.press(forDuration: 1) + let item = messages.descendants(matching: .any).matching(NSPredicate(format: "label == %@", "Blurt")).firstMatch + if item.waitForExistence(timeout: 2) { + item.tap() + hops += 1 + continue + } + } + globe.tap() + hops += 1 + } + XCTAssertTrue(plus.exists, "Blurt never came up in Messages") + return plus + } + + private func shot(_ app: XCUIApplication, _ name: String) { + shots += 1 + let attachment = XCTAttachment(screenshot: app.screenshot()) + attachment.name = "flow-\(String(format: "%02d", shots))-\(name).png" + attachment.lifetime = .keepAlways + add(attachment) + } +} diff --git a/App/BlurtiOS/BlurtiOSProbe/MessagesKeyboardShot.swift b/App/BlurtiOS/BlurtiOSProbe/MessagesKeyboardShot.swift new file mode 100644 index 00000000..a75bdfbd --- /dev/null +++ b/App/BlurtiOS/BlurtiOSProbe/MessagesKeyboardShot.swift @@ -0,0 +1,62 @@ +import XCTest + +/// The real Blurt keyboard inside Messages — another host, another layout +/// history — repeated, since the system's keyboard container sometimes keeps +/// the previous keyboard's height. Attaches a screenshot per showing and +/// prints where Blurt's + landed, so the container's extra height (the +/// system's backdrop above our surface) can be measured. +final class MessagesKeyboardShot: XCTestCase { + @MainActor + func testShots() throws { + let messages = XCUIApplication(bundleIdentifier: "com.apple.MobileSMS") + messages.launch() + // A conversation if there is one, else a new message. + let cell = messages.cells.firstMatch + if cell.waitForExistence(timeout: 5) { + cell.tap() + } else { + messages.buttons["compose"].firstMatch.tap() + } + // The message field, whatever element Messages makes it this year. + let field = messages.descendants(matching: .any).matching( + NSPredicate(format: "placeholderValue == %@ OR label == %@", "iMessage", "iMessage") + ).firstMatch + if !field.waitForExistence(timeout: 10) { + let lines = messages.debugDescription.split(separator: "\n").filter { + $0.contains("Message") || $0.contains("TextView") || $0.contains("TextField") + } + XCTFail("no message field; candidates:\n" + lines.prefix(30).joined(separator: "\n")) + return + } + print("BLURT-FIELD \(field.elementType.rawValue) \(field.frame)") + let blurt = messages.buttons.matching(NSPredicate(format: "label BEGINSWITH %@", "Add a key term")).firstMatch + for round in 0..<3 { + field.tap() + // Round 1 comes to Blurt from the emoji keyboard, round 2 from the + // system keyboard: the container keeps the previous keyboard's height + // when it is taller, which is the band above the surface. + if round > 0 { + let globe = messages.buttons["Next keyboard"] + if blurt.exists, globe.waitForExistence(timeout: 3) { globe.tap() } + if round == 1 { Thread.sleep(forTimeInterval: 1.5) } + } + var hops = 0 + while !blurt.waitForExistence(timeout: 2), hops < 4 { + let globe = messages.buttons["Next keyboard"] + XCTAssertTrue(globe.waitForExistence(timeout: 5), "no globe key") + globe.tap() + hops += 1 + } + XCTAssertTrue(blurt.exists, "Blurt never came up in Messages") + Thread.sleep(forTimeInterval: 2.5) + print("BLURT-FRAMES round \(round): plus \(blurt.frame) space \(messages.buttons["space"].firstMatch.frame)") + let shot = XCTAttachment(screenshot: messages.screenshot()) + shot.name = "messages-\(round).png" + shot.lifetime = .keepAlways + add(shot) + // Dismiss and bring it back: swipe the conversation down to hide the keyboard. + messages.swipeDown() + Thread.sleep(forTimeInterval: 1.5) + } + } +} diff --git a/App/BlurtiOS/BlurtiOSProbe/MicPixels.swift b/App/BlurtiOS/BlurtiOSProbe/MicPixels.swift new file mode 100644 index 00000000..20a954ba --- /dev/null +++ b/App/BlurtiOS/BlurtiOSProbe/MicPixels.swift @@ -0,0 +1,67 @@ +import CoreGraphics +import UIKit +import XCTest + +/// Whether the mic element is actually *drawn* — not merely present in the +/// accessibility tree with a frame. The keyboard's key has been there with +/// nothing in it more than once (a drawing surface that never presented in +/// the extension), and only the pixels can say. Samples the screenshot inside +/// the mic key's frame and counts pixels that differ from the frame's own +/// corner, which is the host's material: the grille's dots, lit or at rest, +/// cover well over a tenth of the box; an empty key covers none. +/// Main-actor: everything it reads — screenshots, element frames — is +/// XCUIAutomation's, which is main-actor in Swift 6. +@MainActor +enum MicPixels { + /// The least share of differing pixels that counts as the element drawn. + static let drawnThreshold = 0.02 + + /// The share of pixels inside `frame` (points, screen coordinates) that are + /// not the material's colour. + static func inkFraction(in shot: XCUIScreenshot, frame: CGRect) -> Double { + guard let image = shot.image.cgImage, frame.width > 4, frame.height > 4 else { return 0 } + let scale = CGFloat(image.width) / UIScreen.main.bounds.width + let rect = CGRect( + x: frame.minX * scale, y: frame.minY * scale, width: frame.width * scale, height: frame.height * scale + ).integral.intersection(CGRect(x: 0, y: 0, width: image.width, height: image.height)) + guard let data = image.dataProvider?.data, let bytes = CFDataGetBytePtr(data), rect.width > 4, rect.height > 4 + else { return 0 } + let bytesPerPixel = image.bitsPerPixel / 8 + let bytesPerRow = image.bytesPerRow + struct RGB { + let r: Int + let g: Int + let b: Int + func distance(to other: RGB) -> Int { abs(r - other.r) + abs(g - other.g) + abs(b - other.b) } + } + func pixel(_ x: Int, _ y: Int) -> RGB { + let offset = y * bytesPerRow + x * bytesPerPixel + return RGB(r: Int(bytes[offset]), g: Int(bytes[offset + 1]), b: Int(bytes[offset + 2])) + } + // The material, read just inside the corner (the frame's edge may be antialiased). + let reference = pixel(Int(rect.minX) + 2, Int(rect.minY) + 2) + var differing = 0 + var total = 0 + var y = Int(rect.minY) + while y < Int(rect.maxY) { + var x = Int(rect.minX) + while x < Int(rect.maxX) { + if pixel(x, y).distance(to: reference) > 40 { differing += 1 } + total += 1 + x += 2 + } + y += 2 + } + return total == 0 ? 0 : Double(differing) / Double(total) + } + + /// The mic element's share of ink, or nil when the key is not there. + static func ink(_ app: XCUIApplication, mic: XCUIElement) -> Double? { + guard mic.exists else { return nil } + return inkFraction(in: app.screenshot(), frame: mic.frame) + } + + static func isDrawn(_ app: XCUIApplication, mic: XCUIElement) -> Bool { + (ink(app, mic: mic) ?? 0) > drawnThreshold + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/APIKeyModelTests.swift b/App/BlurtiOS/BlurtiOSTests/APIKeyModelTests.swift new file mode 100644 index 00000000..87405376 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/APIKeyModelTests.swift @@ -0,0 +1,36 @@ +import BlurtEngine +import Testing + +@testable import BlurtiOSCore + +/// The iPhone's API-key model over an in-memory store — never the real +/// Keychain — and a validator that answers without the network. +@Suite("API key model") +@MainActor +struct APIKeyModelTests { + @Test("a key AssemblyAI accepts is saved and flips the flag; a rejected one is not saved") + func submit() async { + let store = InMemoryAPIKeyStore() + let model = APIKeyModel(keyStore: store) { key in key == "good" ? .valid : .invalid } + #expect(!model.hasAPIKey) + #expect(await model.submit("bad") == .invalid) + #expect(!model.hasAPIKey) + #expect(store.current == nil) + #expect(await model.submit("good") == .valid) + #expect(model.hasAPIKey) + #expect(store.current == "good") + } + + @Test("the readiness gate reads the store as it is now; a refresh catches the flag up") + func readiness() { + let store = InMemoryAPIKeyStore() + let model = APIKeyModel(keyStore: store) { _ in .valid } + let check = model.readinessCheck() + #expect(check() == .apiKeyMissing) + store.save("key") + #expect(check() == nil) + #expect(!model.hasAPIKey) + model.refreshStatus() + #expect(model.hasAPIKey) + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/AppleGeometryTests.swift b/App/BlurtiOS/BlurtiOSTests/AppleGeometryTests.swift new file mode 100644 index 00000000..ce973c4a --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/AppleGeometryTests.swift @@ -0,0 +1,77 @@ +import Foundation +import Testing + +@testable import BlurtiOS + +/// The key tokens against what scripts/apple-geometry.sh read off the +/// iPhone's own keyboard (`Design/apple-geometry.json`): the design's numbers +/// are Apple's, measured, and a change to either side has to be a decision. +/// Tolerances are the measurement's: pixels come in thirds of a point, the +/// surface's edge is anti-aliased, the radius is a fit. +@Suite("Apple's keyboard geometry") +struct AppleGeometryTests { + private final class BundleToken {} + + private static func measured() throws -> [String: Double] { + let url = try #require(Bundle(for: BundleToken.self).url(forResource: "apple-geometry", withExtension: "json")) + let json = try #require(JSONSerialization.jsonObject(with: Data(contentsOf: url)) as? [String: Any]) + let cap = try #require(json["cap"] as? [String: Any]) + var numbers: [String: Double] = [:] + for (key, value) in cap { + if let number = value as? NSNumber { numbers[key] = number.doubleValue } + } + return numbers + } + + private static func value(_ numbers: [String: Double], _ key: String) throws -> Double { + try #require(numbers[key], "apple-geometry.json has no cap.\(key)") + } + + @Test("the keys are the iPhone's: width, height, gaps, margins, radius") + func keys() throws { + let cap = try Self.measured() + let metrics = DesignTokens.Metrics.self + #expect(abs(try Self.value(cap, "letter") - metrics.keyLetterWidth402) < 0.05) + #expect(try Self.value(cap, "height") == metrics.keyHeight) + #expect(try Self.value(cap, "gap") == metrics.keyGap) + #expect(try Self.value(cap, "rowGap") == metrics.rowGap) + #expect(abs(try Self.value(cap, "marginSide") - metrics.marginSide) < 0.2) + #expect(abs(try Self.value(cap, "radius") - metrics.keyRadius) < 0.5) + #expect(abs(try Self.value(cap, "side") - metrics.keySideWidth402) < 0.1) + #expect(abs(try Self.value(cap, "sideGap") - metrics.keySideGap) < 0.05) + #expect(abs(try Self.value(cap, "abc") - metrics.keyAbcWidth402) < 0.05) + #expect(abs(try Self.value(cap, "return") - metrics.keyReturnWidth402) < 0.5) + #expect(abs(try Self.value(cap, "space") - metrics.keySpaceWidth402) < 0.5) + } + + @Test("the rows are the iPhone's: the band above the keys, the margin under them, the whole height") + func rows() throws { + let cap = try Self.measured() + let metrics = DesignTokens.Metrics.self + #expect(abs(try Self.value(cap, "topBand") - (metrics.marginVertical + metrics.voicebarHeight)) < 1) + #expect(try Self.value(cap, "bottomMargin") == metrics.marginBottomKeys) + #expect(abs(try Self.value(cap, "keyboardHeight") - metrics.layoutFull) < 1) + } + + @Test("the geometry reproduces the measured row at the reference width") + func arithmetic() throws { + let cap = try Self.measured() + let width = DesignTokens.Metrics.keyReferenceWidth + let geometry = KeyGeometry(width: width) + #expect(abs(geometry.letterWidth - (try Self.value(cap, "letter"))) < 0.05) + #expect(abs(geometry.sideWidth - (try Self.value(cap, "side"))) < 0.1) + #expect(abs(geometry.returnWidth - (try Self.value(cap, "return"))) < 0.5) + // The rows add up to the width, margins included. + let margin = DesignTokens.Metrics.marginSide + #expect(abs(10 * geometry.letterWidth + 9 * geometry.gap + 2 * margin - width) < 0.001) + #expect( + abs( + 2 * geometry.sideWidth + 2 * geometry.sideGap + 7 * geometry.letterWidth + 6 * geometry.gap + 2 * margin - width + ) + < 0.001) + #expect( + abs( + 2 * geometry.abcWidth + geometry.spaceWidth(globe: true) + geometry.returnWidth + 3 * geometry.gap + 2 * margin + - width) < 0.001) + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/DesignTokensTests.swift b/App/BlurtiOS/BlurtiOSTests/DesignTokensTests.swift new file mode 100644 index 00000000..f64cbb8d --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/DesignTokensTests.swift @@ -0,0 +1,156 @@ +import Foundation +import SwiftUI +import Testing + +@testable import BlurtiOS +@testable import BlurtiOSCore + +/// The compiled tokens against the file they were generated from. check.sh's +/// `design-sync.sh --check` catches drift in the tree; this catches it in the +/// binary, so a hand edit to `DesignTokens.swift` that slipped past both the +/// header and the check still fails somewhere. +@Suite("Design tokens") +struct DesignTokensTests { + private final class BundleToken {} + + private static func tokensJSON() throws -> [String: Any] { + let url = try #require(Bundle(for: BundleToken.self).url(forResource: "tokens", withExtension: "json")) + return try #require(JSONSerialization.jsonObject(with: Data(contentsOf: url)) as? [String: Any]) + } + + /// A number the way scripts/design-tokens.swift prints it: `42`, `0.7`. + private static func format(_ number: Double) -> String { + number == number.rounded() ? String(Int(number)) : String(number) + } + + /// tokens.json flattened to `group.name: value`, aliases followed — the same + /// rendering the generator writes into `DesignTokens.manifest`. + private static func expectedManifest() throws -> [String: String] { + let json = try tokensJSON() + let groups = ["brand", "themes", "keyboard", "metrics", "type", "motion"] + var raw: [String: Any] = [:] + for group in groups { + let entries = try #require(json[group] as? [String: Any]) + for (name, entry) in entries { + raw["\(group).\(name)"] = (entry as? [String: Any])?["value"] ?? entry + } + } + func render(_ qualified: String, depth: Int = 0) throws -> String { + let value = try #require(raw[qualified], "\(qualified) is not a token") + try #require(depth < 8, "alias cycle at \(qualified)") + if let number = value as? NSNumber { return format(number.doubleValue) } + if let curve = value as? [NSNumber] { return curve.map { format($0.doubleValue) }.joined(separator: ",") } + let string = try #require(value as? String) + if string.hasPrefix("{"), string.hasSuffix("}") { + return try render(String(string.dropFirst().dropLast()), depth: depth + 1) + } + return string.hasPrefix("#") ? string.uppercased() : string + } + var manifest: [String: String] = [:] + for qualified in raw.keys { manifest[qualified] = try render(qualified) } + let gradients = try #require(json["gradients"] as? [String: Any]) + for (name, entry) in gradients { + let gradient = try #require(entry as? [String: Any]) + let stops = try #require(gradient["stops"] as? [[String: Any]]) + let rendered = try stops.map { stop -> String in + let color = try #require(stop["color"] as? String) + let location = try #require(stop["location"] as? NSNumber) + let hex = + color.hasPrefix("{") ? try render(String(color.dropFirst().dropLast())) : color.uppercased() + return "\(hex)@\(format(location.doubleValue))" + } + let start = try #require(gradient["start"] as? String) + let end = try #require(gradient["end"] as? String) + manifest["gradients.\(name)"] = "\(rendered.joined(separator: " ")) \(start)>\(end)" + } + return manifest + } + + @Test("every compiled token equals tokens.json (run scripts/design-sync.sh)") + func manifest() throws { + let expected = try Self.expectedManifest() + let compiled = DesignTokens.manifest + let missing = Set(expected.keys).subtracting(compiled.keys).sorted() + let extra = Set(compiled.keys).subtracting(expected.keys).sorted() + let changed = expected.keys.filter { compiled[$0] != nil && compiled[$0] != expected[$0] }.sorted() + #expect(missing.isEmpty, "tokens in tokens.json but not compiled: \(missing)") + #expect(extra.isEmpty, "tokens compiled but not in tokens.json: \(extra)") + #expect( + changed.isEmpty, + "tokens whose value differs: \(changed.map { "\($0): \(compiled[$0] ?? "?") vs \(expected[$0] ?? "?")" })") + } + + @Test("the layout heights are the rows at the token spacing, and the layout/* tokens agree") + func heights() { + let metrics = DesignTokens.Metrics.self + #expect(KeyboardLayout.slimBar.height == 2 * metrics.marginVertical + metrics.voicebarHeight) + #expect(KeyboardLayout.slimBar.height == metrics.layoutSlim) + #expect(KeyboardLayout.panel.height == metrics.layoutPanel) + #expect( + KeyboardLayout.full.height + == metrics.marginVertical + metrics.voicebarHeight + 4 * metrics.keyHeight + 3 * metrics.rowGap + + metrics.marginBottomKeys) + #expect(KeyboardLayout.full.height == metrics.layoutFull) + } + + @Test("the full layout at 402 pt matches the @402 tokens the design is drawn at") + func fullLayoutAt402() { + // The tokens are written to a thousandth of a point; the row's thirds + // round either way at the last digit. + let geometry = KeyGeometry(width: DesignTokens.Metrics.keyReferenceWidth) + #expect(abs(geometry.letterWidth - DesignTokens.Metrics.keyLetterWidth402) < 0.005) + #expect(abs(geometry.sideWidth - DesignTokens.Metrics.keySideWidth402) < 0.005) + #expect(abs(geometry.abcWidth - DesignTokens.Metrics.keyAbcWidth402) < 0.005) + #expect(abs(geometry.returnWidth - DesignTokens.Metrics.keyReturnWidth402) < 0.005) + #expect(abs(geometry.spaceWidth(globe: true) - DesignTokens.Metrics.keySpaceWidth402) < 0.005) + } + + /// Each face beside the ten theme tokens it must equal, in role order: + /// surface, key, modifier, legend, secondary legend, signal, pop-up, field, + /// field border, notice. + private static let paletteTokens: [(KeyboardPalette, [Color])] = { + typealias Themes = DesignTokens.Themes + return [ + ( + .brandLight, + [ + Themes.lightSurface, Themes.lightKey, Themes.lightKeyModifier, Themes.lightLegend, + Themes.lightLegendSecondary, Themes.lightSignal, Themes.lightPopup, Themes.lightField, + Themes.lightFieldBorder, Themes.lightNotice, Themes.lightContainerKey, Themes.lightContainerKeyModifier, + Themes.lightMaterial, + ] + ), + ( + .brandDark, + [ + Themes.darkSurface, Themes.darkKey, Themes.darkKeyModifier, Themes.darkLegend, Themes.darkLegendSecondary, + Themes.darkSignal, Themes.darkPopup, Themes.darkField, Themes.darkFieldBorder, Themes.darkNotice, + Themes.darkContainerKey, Themes.darkContainerKeyModifier, Themes.darkMaterial, + ] + ), + ] + }() + + @Test("every face is its theme's tokens, role by role") + func palettes() { + #expect(Self.paletteTokens.count == KeyboardPalette.all.count + 1) + for (palette, colors) in Self.paletteTokens { + let roles = [ + palette.surface, palette.key, palette.keyDark, palette.keyText, palette.keyTextSecondary, palette.signal, + palette.popupFill, palette.field, palette.fieldBorder, palette.notice, palette.containerKey, + palette.containerKeyModifier, palette.material, + ] + #expect(roles == colors, "\(palette.id) \(palette.face)") + } + } + + @Test("the brand's faces carry the brand: ink under the dark face, paper under the light, green as the signal") + func faces() { + #expect(DesignTokens.Themes.darkSurface == DesignTokens.Brand.ink) + #expect(DesignTokens.Themes.lightSurface == DesignTokens.Brand.paper200) + #expect(DesignTokens.Themes.darkSignal == DesignTokens.Brand.green400) + #expect(DesignTokens.Themes.lightSignal == DesignTokens.Brand.green700) + #expect(DesignTokens.Themes.lightNotice == DesignTokens.Brand.orange) + #expect(DesignTokens.Themes.darkNotice == DesignTokens.Brand.orange) + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/FeedAndLayoutTests.swift b/App/BlurtiOS/BlurtiOSTests/FeedAndLayoutTests.swift new file mode 100644 index 00000000..3d377dc7 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/FeedAndLayoutTests.swift @@ -0,0 +1,90 @@ +import Foundation +import Testing + +@testable import BlurtiOS +@testable import BlurtiOSCore + +@Suite("Utterance feed") +struct UtteranceFeedTests { + private func pcm(_ samples: [Int16]) -> Data { + samples.withUnsafeBufferPointer { Data(buffer: $0) } + } + + @Test("level: silence is 0, full scale is 1, room ambient under the floor is 0") + func level() { + #expect(UtteranceFeed.level(of: pcm([0, 0, 0, 0])) == 0) + #expect(UtteranceFeed.level(of: pcm([Int16.max, Int16.max, -Int16.max, -Int16.max])) == 1) + #expect(UtteranceFeed.level(of: pcm([3, -3, 2, -2])) == 0) + #expect(UtteranceFeed.level(of: Data()) == 0) + let mid = UtteranceFeed.level(of: pcm([8000, -8000, 8000, -8000])) + #expect(mid > 0 && mid < 1) + } + + @Test("chunks reach the utterance between begin and end, the tally counts them, and nothing outside") + func tally() async { + let feed = UtteranceFeed() + feed.deliver(pcm([1000, -1000])) + let stream = feed.begin() + feed.deliver(pcm([1000, -1000])) + feed.deliver(pcm([1000, -1000, 1000])) + let bytes = feed.end() + #expect(bytes == 10) + var received = 0 + for await chunk in stream { received += chunk.count } + #expect(received == 10) + feed.deliver(pcm([1])) + #expect(feed.end() == 0) + } +} + +@Suite("Layout arithmetic") +struct LayoutArithmeticTests { + @Test("the full layout's height is its rows at the theme spacing (DESIGN.md)") + func fullHeight() { + let expected = + KeyboardRootView.verticalMargin + VoiceBar.height + 4 * KeyCap.height + 3 * KeyboardPalette.rowGap + + KeyboardRootView.bottomMarginKeys + #expect(KeyboardLayout.full.height == expected) + } + + @Test("the gallery reads a layout, states and an optional theme from the launch arguments") + func galleryRows() { + #expect(KeyboardGalleryView.rows(from: ["app"]).isEmpty) + #expect(KeyboardGalleryView.rows(from: ["app", "-BlurtGallery", "panel"]).isEmpty) + #expect(KeyboardGalleryView.rows(from: ["app", "-BlurtGallery", "nope", "idle"]).isEmpty) + let rows = KeyboardGalleryView.rows(from: ["app", "-BlurtGallery", "panel", "idle,keys,term", "dark"]) + #expect(rows.count == 3) + #expect(rows[1].model.effectiveLayout == .full) + #expect(rows[2].model.termDraft == "Rizz") + #expect(rows[0].model.palette.face == .dark) + // The old theme words still land on a face, so a capture script never breaks. + let dark = KeyboardGalleryView.rows(from: ["app", "-BlurtGallery", "full", "idle", "system-dark"]) + #expect(dark[0].model.palette.keyText == DesignTokens.Themes.darkLegend) + let ink = KeyboardGalleryView.rows(from: ["app", "-BlurtGallery", "full", "idle", "ink"]) + #expect(ink[0].model.palette.face == .dark) + #expect( + KeyboardGalleryView.rows(from: ["app", "-BlurtGallery", "full", "idle", "paper"])[0].model.palette.face == .light) + } + + @Test("the capture switches ride along, and a switch where the theme would be is not a theme") + func galleryOptions() { + let args = ["app", "-BlurtGallery", "panel", "idle", "-BlurtGalleryStill", "-BlurtGalleryBare"] + #expect(KeyboardGalleryView.rows(from: args)[0].model.palette.face == .light) + #expect(KeyboardGalleryView.Options.parse(args) == .init(still: true, bare: true)) + #expect(KeyboardGalleryView.Options.parse(["app", "-BlurtGallery", "panel", "idle", "dark"]) == .init()) + #expect(KeyboardGalleryView.rows(from: args)[0].model.voiceKind == .shipped) + #expect(KeyboardGalleryView.rows(from: args + ["-BlurtGalleryVoice", "b"])[0].model.voiceKind == .ribs) + #expect(KeyboardGalleryView.rows(from: args + ["-BlurtGalleryVoice", "z"])[0].model.voiceKind == .shipped) + #expect(KeyboardGalleryView.VoiceStillView.parse(args) == nil) + let voice = KeyboardGalleryView.VoiceStillView.parse(["app", "-BlurtVoice", "a", "panel", "landed", "dark"]) + #expect(voice?.kind == .grille) + #expect(voice?.slot == .panel) + #expect(voice?.landedAt != nil) + #expect(voice?.dark == true) + #expect( + KeyboardGalleryView.VoiceStillView.parse(["app", "-BlurtVoice", "c", "bar", "recording"])?.state.level == 0.62) + #expect(KeyboardGalleryView.VoiceStillView.parse(["app", "-BlurtVoice", "a", "bar", "nope"]) == nil) + #expect(KeyboardGalleryView.VoiceStillView.parse(["app", "-BlurtVoice", "d", "bar", "idle"]) == nil) + #expect(KeyboardGalleryView.VoiceStillView.parse(["app", "-BlurtVoice", "a", "desk", "idle"]) == nil) + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/HomeStatusTests.swift b/App/BlurtiOS/BlurtiOSTests/HomeStatusTests.swift new file mode 100644 index 00000000..8792e23f --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/HomeStatusTests.swift @@ -0,0 +1,45 @@ +import BlurtEngine +import Foundation +import Testing + +@testable import BlurtiOSCore + +@Suite("Home status") +struct HomeStatusTests { + private func status( + _ overlay: OverlayUIState, open: Bool = true, until: Date? = nil, key: Bool = true, mic: Bool = true, + keyboard: Bool = true + ) -> HomeStatus { + HomeStatus(overlay: overlay, windowOpen: open, until: until, hasKey: key, micGranted: mic, keyboardSeen: keyboard) + } + + @Test("the headline follows the phase, and says whether the mic is open at rest") + func titles() { + #expect(status(.idle).title == "Ready to dictate") + #expect(status(.idle, open: false).title == "Not listening") + #expect(status(.connecting).title == "Connecting…") + #expect(status(.recording).title == "Listening…") + #expect(status(.processing).title == "Transcribing…") + #expect(status(.pasted).title == "Pasted") + #expect(status(.noTarget).title == "Copied") + #expect(status(.error(message: "No network.")).title == "No network.") + } + + @Test("the line under it says how long the mic stays open") + func subtitles() { + #expect(status(.idle, open: false).subtitle.hasPrefix("Open the mic")) + #expect(status(.idle, until: .distantFuture).subtitle.hasPrefix("Until you stop it")) + #expect(status(.idle, until: Date()).subtitle.hasPrefix("Until ")) + #expect(!status(.idle, until: Date()).subtitle.hasPrefix("Until you")) + } + + @Test("setup needs the key, the microphone and the keyboard; landed means pasted or copied") + func setupAndLanded() { + #expect(status(.idle).isSetUp) + #expect(!status(.idle, key: false).isSetUp) + #expect(!status(.idle, mic: false).isSetUp) + #expect(!status(.idle, keyboard: false).isSetUp) + #expect(status(.pasted).landed && status(.noTarget).landed) + #expect(!status(.idle).landed && !status(.error(message: "x")).landed) + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/KeyboardInteractionTests.swift b/App/BlurtiOS/BlurtiOSTests/KeyboardInteractionTests.swift new file mode 100644 index 00000000..cd3400bd --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/KeyboardInteractionTests.swift @@ -0,0 +1,98 @@ +import CoreGraphics +import Testing + +@testable import BlurtiOSCore + +@Suite("Keyboard interaction rules") +struct KeyboardInteractionTests { + @Test("a swipe flips the panel past 48 pt sideways and 1.5× the vertical, the way the finger went") + func flip() { + #expect(KeyboardInteraction.flip(CGSize(width: 47, height: 0)) == nil) + #expect(KeyboardInteraction.flip(CGSize(width: 49, height: 0)) == .towardsTrailing) + #expect(KeyboardInteraction.flip(CGSize(width: -49, height: 0)) == .towardsLeading) + #expect(KeyboardInteraction.flip(CGSize(width: 49, height: 40)) == nil) + #expect(KeyboardInteraction.flip(CGSize(width: 60, height: 39)) == .towardsTrailing) + } + + @Test("a key acts only under 12 pt of travel; the mic cancels past 24") + func travel() { + #expect(KeyboardInteraction.isTap(CGSize(width: 11.9, height: 11.9))) + #expect(!KeyboardInteraction.isTap(CGSize(width: 12, height: 0))) + #expect(!KeyboardInteraction.isTap(CGSize(width: 0, height: -12))) + #expect(!KeyboardInteraction.micTravelled(CGSize(width: 24, height: 24))) + #expect(KeyboardInteraction.micTravelled(CGSize(width: 24.1, height: 0))) + #expect(KeyboardInteraction.micTravelled(CGSize(width: 0, height: -24.1))) + } + + @Test("a touch that ends on the key is a tap wherever the thumb rolled; one that left it is not") + func endsOnKey() { + let key = CGSize(width: 33.5, height: 43) + // Barely moved: a tap wherever it ended. + #expect(KeyboardInteraction.isTap(CGSize(width: 5, height: 5), endedAt: CGPoint(x: -100, y: -100), in: key)) + // Rolled 20 pt but ended inside the key. + #expect(KeyboardInteraction.isTap(CGSize(width: 0, height: 20), endedAt: CGPoint(x: 16, y: 30), in: key)) + // Ended just past the edge, within the slop. + #expect( + KeyboardInteraction.isTap( + CGSize(width: 15, height: 0), endedAt: CGPoint(x: key.width + KeyboardInteraction.tapSlop - 0.1, y: 20), + in: key)) + // Left the key: a swipe. + #expect(!KeyboardInteraction.isTap(CGSize(width: 60, height: 0), endedAt: CGPoint(x: 80, y: 20), in: key)) + #expect(!KeyboardInteraction.isTap(CGSize(width: 0, height: -60), endedAt: CGPoint(x: 16, y: -40), in: key)) + // With no size known yet, only the travel rule applies. + #expect(!KeyboardInteraction.isTap(CGSize(width: 15, height: 0), endedAt: CGPoint(x: 16, y: 20), in: .zero)) + } + + // `#expect` captures its expression immutably, so each mutating step lands + // in a local first. + @Test("a quick tap presses and releases together") + func quickTap() { + var press = MicPressSequencer() + let armed = press.began() + #expect(armed) + let armedAgain = press.began() + #expect(!armedAgain) + let outcome = press.ended(travelled: false) + #expect(outcome == .tap(needsDown: true)) + #expect(press == MicPressSequencer()) + } + + @Test("a hold presses when the timer fires and releases on the lift") + func hold() { + var press = MicPressSequencer() + _ = press.began() + let fired = press.delayFired() + #expect(fired) + let firedAgain = press.delayFired() + #expect(!firedAgain) + let outcome = press.ended(travelled: false) + #expect(outcome == .tap(needsDown: false)) + } + + @Test("a swipe before the timer fires cancels it and does nothing") + func swipeBeforePress() { + var press = MicPressSequencer() + _ = press.began() + let stayed = press.moved(travelled: false) + #expect(!stayed) + let cancelled = press.moved(travelled: true) + #expect(cancelled) + let cancelledAgain = press.moved(travelled: true) + #expect(!cancelledAgain) + let fired = press.delayFired() + #expect(!fired) + let outcome = press.ended(travelled: true) + #expect(outcome == .nothing) + } + + @Test("a swipe after the press went out undoes it") + func swipeAfterPress() { + var press = MicPressSequencer() + _ = press.began() + _ = press.delayFired() + let cancelled = press.moved(travelled: true) + #expect(!cancelled) + let outcome = press.ended(travelled: true) + #expect(outcome == .cancelSentPress) + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/KeyboardLifecycleTests.swift b/App/BlurtiOS/BlurtiOSTests/KeyboardLifecycleTests.swift new file mode 100644 index 00000000..8b5ce1d8 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/KeyboardLifecycleTests.swift @@ -0,0 +1,176 @@ +import BlurtEngine +import Foundation +import Testing +import UIKit + +@testable import BlurtiOSCore + +/// The keyboard's side of being on screen, over time: the presence heartbeat, +/// the app's Darwin signals, the dwell that lets a notice go, the release held +/// while the mic comes up, the word list handed to the app, and the responder +/// walk that brings the app forward. +@Suite("Keyboard model: lifecycle and the app's signals", .serialized) +@MainActor +struct KeyboardLifecycleTests { + + /// Polls until `condition` holds or `timeout` passes; true when it held. + private func eventually(_ timeout: Duration = .seconds(3), _ condition: () -> Bool) async -> Bool { + let deadline = ContinuousClock.now + timeout + while ContinuousClock.now < deadline { + if condition() { return true } + try? await Task.sleep(for: .milliseconds(20)) + } + return condition() + } + + @Test("on screen, the keyboard tells the app it is there and which one it is; gone, it stops") + func heartbeat() async { + let suite = ScratchSuite() + defer { suite.tearDown() } + SharedStore.autoDictate = false + let model = KeyboardRig().model + model.appeared() + #expect(await eventually { SharedStore.keyboardSeenAt != nil }) + #expect(SharedStore.keyboardInstance == model.instanceID) + model.disappeared() + #expect(SharedStore.keyboardSeenAt == nil) + } + + @Test("attached to its controller, the keyboard follows the app's phase and result signals") + func darwinSignals() async { + let suite = ScratchSuite() + defer { suite.tearDown() } + let proxy = FakeProxy() + proxy.before = "Hi" + let model = KeyboardRig(proxy: proxy).model + let controller = UIInputViewController(nibName: nil, bundle: nil) + model.attach(to: controller) + SharedStore.write(snapshot(.recording), forKey: BlurtShared.Key.phase) + SharedStore.post(BlurtShared.Signal.phase) + #expect(await eventually { model.snapshot.state == .recording }) + SharedStore.write( + DictationResult(id: UUID(), text: "there", deliveredAt: Date(), recipient: nil), forKey: BlurtShared.Key.result) + SharedStore.post(BlurtShared.Signal.result) + #expect(await eventually { proxy.before == "Hi there" }) + } + + @Test("a notice lets go after its dwell, and a second read of the same snapshot does not bring it back") + func noticeDwell() async throws { + let suite = ScratchSuite() + defer { suite.tearDown() } + let model = KeyboardRig().model + let pasted = snapshot(.pasted) + let dwell = try #require(pasted.noticeDwellSeconds) + model.apply(pasted) + #expect(model.snapshot.state == .pasted) + #expect(await eventually(.seconds(dwell + 2)) { model.snapshot.state == .idle }) + model.apply(pasted) + #expect(model.snapshot.state == .idle) + // A new notice is not the one let go of. + model.apply(snapshot(.copied)) + #expect(model.snapshot.state == .copied) + } + + @Test("a release made while the mic is coming up is held for the first recording phase") + func releaseWhileConnecting() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let commands = rig.commands + model.micDown() + model.apply(snapshot(.connecting)) + // Push-to-talk let go before the mic was up: the engine would drop a + // release now, so it waits. + model.perform(.stop) + #expect(commands.sent.map(\.kind) == [.press]) + model.apply(snapshot(.recording)) + #expect(commands.sent.map(\.kind) == [.press, .release]) + #expect(model.gate.isIdle) + } + + @Test("a held release is dropped when the mic never comes up") + func heldReleaseDroppedOnFailure() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let commands = rig.commands + model.micDown() + model.apply(snapshot(.connecting)) + model.perform(.stop) + model.apply(snapshot(.error)) + model.apply(snapshot(.recording)) + #expect(commands.sent.map(\.kind) == [.press]) + } + + @Test("the saved check clears after its dwell and takes the chip for that word with it") + func termSavedDwell() async { + let suite = ScratchSuite() + defer { suite.tearDown() } + let proxy = FakeProxy() + proxy.selected = "Rizz" + let model = KeyboardRig(proxy: proxy).model + model.contextChanged() + #expect(model.selectedTerm == "Rizz") + model.addSelectedTerm() + #expect(model.termSavedAt != nil) + #expect(SharedStore.keyTerms == ["Rizz"]) + #expect(await eventually(.seconds(KeyboardModel.termNoticeDwell + 2)) { model.termSavedAt == nil }) + #expect(model.selectedTerm == nil) + #expect(model.dismissedSelection == "Rizz") + } + + @Test("the phone's word list reaches the app: written, stamped, and only re-read once it is old") + func lexicon() throws { + let suite = ScratchSuite() + defer { suite.tearDown() } + let entries = [ + LexiconEntry(userInput: "Neil Bisht", documentText: "Neil Bisht"), + LexiconEntry(userInput: "omw", documentText: "On my way!"), + ] + KeyboardModel.store(entries) + let stored = SharedStore.read([LexiconEntry].self, forKey: BlurtShared.Key.lexicon) + #expect(stored?.map(\.documentText) == ["Neil Bisht", "On my way!"]) + #expect(stored?.map(\.isName) == [true, false]) + let refreshed = try #require(SharedStore.lexiconRefreshedAt) + #expect(Date().timeIntervalSince(refreshed) < 5) + } + + @Test("opening the app walks the responder chain to the application and asks it for blurt://start") + func openApp() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let model = KeyboardModel() + // No controller: nothing to walk. + var opened: [(UIApplication, URL)] = [] + model.urlOpener = { opened.append(($0, $1)) } + model.openApp() + #expect(opened.isEmpty) + // A controller on screen: its chain ends at the application. + let controller = UIInputViewController(nibName: nil, bundle: nil) + let window = UIWindow(frame: CGRect(x: 0, y: 0, width: 10, height: 10)) + window.rootViewController = controller + window.isHidden = false + defer { window.isHidden = true } + model.controller = controller + // Not ready (no Full Access): the tap that would have dictated opens Blurt. + model.micDown() + model.micUp() + #expect(opened.count == 1) + #expect(opened.first?.0 === UIApplication.shared) + #expect(opened.first?.1.absoluteString == "\(BlurtShared.urlScheme)://\(BlurtShared.startHost)") + } + + @Test("the mic's side comes from Settings unless a preview says otherwise") + func micAlignment() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let model = KeyboardModel() + #expect(model.micAlignment == .center) + model.storedMicAlignment = .left + #expect(model.micAlignment == .left) + model.micAlignmentOverride = .right + #expect(model.micAlignment == .right) + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/KeyboardModelFuzzTests.swift b/App/BlurtiOS/BlurtiOSTests/KeyboardModelFuzzTests.swift new file mode 100644 index 00000000..815583b7 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/KeyboardModelFuzzTests.swift @@ -0,0 +1,314 @@ +import Foundation +import Testing + +@testable import BlurtiOSCore + +/// The keyboard model driven through thousands of random event sequences — +/// every finger, phase, host and lifecycle event the real keyboard sees, in +/// any order — with the invariants that must hold after every one of them +/// checked each time. Seeded, so a failure names the sequence that made it. +/// This is the "nothing is ever left stale" check: a state the model can be +/// driven into and not out of shows up here as a broken invariant. +@Suite("Keyboard model: random transition sequences", .serialized) +@MainActor +struct KeyboardModelFuzzTests { + /// One event the fuzz can send. `apply` phases come as the app would + /// publish them; the rest are the keyboard's own entry points. + enum Event: CaseIterable { + case micDown, micUp, cancel, swipeOffOrb + case phaseIdle, phaseConnecting, phaseRecording, phaseProcessing, phasePasted, phaseCopied, phaseError + case answerConnecting, answerError + case appNotListening, appListening + case beginTerm, cancelTerm, saveTerm, typeLetter, space, deleteBack, newline + case selectWord, selectOther, clearSelection, addSelected, dismissSelected + case flipPanel, toggleSymbols, toggleMore, toggleShift + case appeared, disappeared, appearedNoAccess, contextChanged + case resultFresh, resultStale, resultForOther + case hostTyped, hostMovedCursor + } + + /// A tiny deterministic generator (SplitMix64), so a run is repeatable. + struct Random { + var state: UInt64 + mutating func next() -> UInt64 { + state &+= 0x9E37_79B9_7F4A_7C15 + var z = state + z = (z ^ (z >> 30)) &* 0xBF58_476D_1CE4_E5B9 + z = (z ^ (z >> 27)) &* 0x94D0_49BB_1331_11EB + return z ^ (z >> 31) + } + mutating func pick(_ items: [T]) -> T { items[Int(next() % UInt64(items.count))] } + } + + /// Set by an event after which the command ledger starts over. + private final class Flag { + var value = false + } + private let epochFlag = Flag() + private var epoch: Bool { + get { epochFlag.value } + nonmutating set { epochFlag.value = newValue } + } + + @Test( + "ten thousand random events never leave the model in a stale or contradictory state", arguments: [1, 2, 3, 4, 5]) + func fuzz(seed: UInt64) { + let suite = ScratchSuite() + defer { suite.tearDown() } + var random = Random(state: seed &* 7919) + let proxy = FakeProxy() + let model = KeyboardModel() + model.proxyOverride = proxy + model.hasFullAccess = true + let commands = KeyboardRig.Commands() + model.transport = { commands.sent.append($0) } + KeyboardRig.listening(true) + model.appeared() + var trace: [Event] = [] + // The press the app has not taken yet: closed by our release or cancel, + // by any in-flight phase (the app is on it), or by a settled phase that + // names it. A second press while one is open is the protocol broken. + var pressOpen: UUID? + for step in 0..<2000 { + let event = random.pick(Event.allCases) + trace.append(event) + let before = commands.sent.count + // Read before the event: whether words were in flight, and whether a + // word is highlighted for the term field to open over. + let stateBefore = model.snapshot.state + let seededTerm = KeyboardModel.termCandidate(from: proxy.selectedText) != nil + drive(model, proxy, event, &random) + if epoch { + commands.sent.removeAll() + pressOpen = nil + } + for command in commands.sent.dropFirst(min(before, commands.sent.count)) { + switch command.kind { + case .press: + #expect( + pressOpen == nil || pressOpen == command.id, + "two presses in flight — seed \(seed) step \(step) after \(event); last 12: \(trace.suffix(12))") + pressOpen = command.id + case .release, .cancel: pressOpen = nil + } + // The user's words are never thrown away behind their back. A cancel + // over a recording or a transcription is only ever theirs to ask for: + // the × key, or the term field opened over a highlighted word (whose + // words would land on the word being fixed). Leaving the screen, a + // swipe across the orb, the plain + — none of them may drop speech. + if command.kind == .cancel, stateBefore == .recording || stateBefore == .processing { + let asked = event == .cancel || (event == .beginTerm && seededTerm) + #expect( + asked, + "words dropped: cancel in \(stateBefore) by \(event) — seed \(seed) step \(step); last 12: \(trace.suffix(12))" + ) + } + } + if Self.phaseEvents.contains(event), !model.isSettled || model.snapshot.command == pressOpen { + pressOpen = nil + } + check( + model, proxy, commands, after: event, + "seed \(seed) step \(step) after \(event); last 12: \(trace.suffix(12))") + } + } + + /// The events that apply a phase from the app. + static let phaseEvents: Set = [ + .phaseIdle, .phaseConnecting, .phaseRecording, .phaseProcessing, .phasePasted, .phaseCopied, .phaseError, + .answerConnecting, .answerError, .appNotListening, .appListening, + ] + + /// The events after which the term field's shift has just been re-read. + static let reevaluatesShift: Set = [ + .beginTerm, .cancelTerm, .saveTerm, .typeLetter, .space, .deleteBack, .newline, .contextChanged, .selectWord, + .selectOther, .clearSelection, .hostTyped, .hostMovedCursor, .appeared, .appearedNoAccess, + ] + + /// What must be true after every event, whatever came before. + private func check( + _ model: KeyboardModel, _ proxy: FakeProxy, _ commands: KeyboardRig.Commands, after event: Event, _ context: String + ) { + // The term field's bookkeeping lives and dies with the field. + let fieldOpen = model.termDraft != nil + #expect((model.termHostBaseline != nil) == fieldOpen, "term baseline vs field — \(context)") + #expect((model.termHostBaselineAfter != nil) == fieldOpen, "term baseline-after vs field — \(context)") + #expect(model.termDraftFromSelection == nil || fieldOpen, "seed outlives the field — \(context)") + #expect(!fieldOpen || model.effectiveLayout == .full, "keys must be up for the field — \(context)") + // Shift follows the term as typed; a tap on shift itself overrides it, + // as on the system keyboard, until the next thing that re-reads it. + if let draft = model.termDraft, Self.reevaluatesShift.contains(event) { + #expect(model.shifted == (draft.isEmpty || draft.last == " "), "term shift rule — \(context)") + } + // The chip and its dismissal never both stand. + #expect(model.dismissedSelection == nil || model.selectedTerm == nil, "chip and dismissal both up — \(context)") + if let term = model.selectedTerm { + let candidate = KeyboardModel.termCandidate(from: proxy.selectedText) + #expect(model.hasFullAccess, "chip without Full Access — \(context)") + #expect(candidate?.caseInsensitiveCompare(term) == .orderedSame, "chip for a word not highlighted — \(context)") + if model.selectedTermIsKnown { + #expect( + SharedStore.keyTerms.contains { $0.caseInsensitiveCompare(term) == .orderedSame }, + "known chip for an unknown word — \(context)") + } + } else { + #expect(!model.selectedTermIsKnown, "known without a chip — \(context)") + } + // Pages. + #expect(!model.morePage || model.symbolsPage, "#+= page without the symbols — \(context)") + // The voice picture never shows what the app can't be doing. + let state = model.voiceState + if !state.isReady { + #expect( + !state.isRecording && state.glyph == nil && !state.canCancel, "in-flight picture with no app — \(context)") + } + // A latched gate is either a press of ours the app hasn't settled, or a + // dictation in flight that the app published (started elsewhere, or a + // phase that arrived after our cancel): never a latch over nothing. + if !model.gate.isIdle { + let inFlight = !model.isSettled + #expect(commands.sent.last?.kind == .press || inFlight, "gate latched with no press out — \(context)") + } + #expect(!model.panelShowsKeys || model.layout == .panel, "keys page off the panel — \(context)") + #expect(commands.sent.allSatisfy { $0.keyboard == model.instanceID }, "command from another keyboard — \(context)") + // A result is never left in the store once it was inserted here. + if let pending = SharedStore.read(DictationResult.self, forKey: BlurtShared.Key.result) { + #expect(pending.id != model.lastResultID, "inserted result still in the store — \(context)") + } + } + + @Test("from any state, an appearance is a clean keyboard that can dictate") + func recovers() { + let suite = ScratchSuite() + defer { suite.tearDown() } + var random = Random(state: 42) + let proxy = FakeProxy() + let model = KeyboardModel() + model.proxyOverride = proxy + model.hasFullAccess = true + let commands = KeyboardRig.Commands() + model.transport = { commands.sent.append($0) } + KeyboardRig.listening(true) + // Hands-free off: a tap must be the press here, not the stop of one the + // appearance started. + SharedStore.autoDictate = false + model.appeared() + for round in 0..<200 { + for _ in 0..<25 { drive(model, proxy, random.pick(Event.allCases), &random) } + // The keyboard goes away and comes back, the app idle and listening. + model.disappeared() + SharedStore.write(snapshot(.idle), forKey: BlurtShared.Key.phase) + KeyboardRig.listening(true) + model.hasFullAccess = true + model.appeared() + #expect(model.termDraft == nil, "field open after an appearance (round \(round))") + #expect(!model.panelShowsKeys, "keys page after an appearance (round \(round))") + #expect(model.gate.isIdle, "gate latched after an appearance (round \(round))") + #expect(model.isSettled, "in flight after an appearance (round \(round))") + #expect(model.termSavedAt == nil, "saved check after an appearance (round \(round))") + // And a tap is a press. + let before = commands.sent.count + model.micDown() + model.micUp() + #expect( + commands.sent.count == before + 1 && commands.sent.last?.kind == .press, "tap is not a press (round \(round))") + model.apply(snapshot(.recording)) + model.micDown() + model.micUp() + #expect(commands.sent.last?.kind == .release, "second tap is not a release (round \(round))") + model.apply(snapshot(.pasted)) + model.apply(snapshot(.idle)) + } + } +} + +// The event → model call table, apart from the fuzz loop and its checks so +// neither crowds the other. +extension KeyboardModelFuzzTests { + // swiftlint:disable:next cyclomatic_complexity function_body_length + private func drive(_ model: KeyboardModel, _ proxy: FakeProxy, _ event: Event, _ random: inout Random) { + epoch = false + switch event { + case .micDown: model.micDown() + case .micUp: model.micUp() + case .cancel: model.cancel() + case .swipeOffOrb: + // A swipe that began on the orb and travelled after its press went out + // (MicControl's `.cancelSentPress`): the press, then its undo. + model.micDown() + model.undoPress() + case .phaseIdle: model.apply(snapshot(.idle)) + case .phaseConnecting: model.apply(snapshot(.connecting)) + case .phaseRecording: model.apply(snapshot(.recording)) + case .phaseProcessing: model.apply(snapshot(.processing)) + case .phasePasted: model.apply(snapshot(.pasted)) + case .phaseCopied: model.apply(snapshot(.copied)) + case .phaseError: model.apply(snapshot(.error)) + case .answerConnecting: + model.apply(PhaseSnapshot(state: .connecting, message: nil, level: 0, at: Date(), command: model.unansweredPress)) + case .answerError: + model.apply(PhaseSnapshot(state: .error, message: nil, level: 0, at: Date(), command: model.unansweredPress)) + case .appNotListening: + KeyboardRig.listening(false) + model.apply(model.snapshot) + case .appListening: + KeyboardRig.listening(true) + model.apply(model.snapshot) + case .beginTerm: model.beginAddingTerm() + case .cancelTerm: model.cancelAddingTerm() + case .saveTerm: model.saveTerm() + case .typeLetter: model.type(random.pick(["a", "B", "z"])) + case .space: model.space() + case .deleteBack: model.deleteBackward() + case .newline: model.newline() + case .selectWord: + proxy.selected = "Rizz" + model.contextChanged() + case .selectOther: + proxy.selected = random.pick(["Gyatt", "neil bisht", " Rizz "]) + model.contextChanged() + case .clearSelection: + proxy.selected = nil + model.contextChanged() + case .addSelected: model.addSelectedTerm() + case .dismissSelected: model.dismissSelection() + case .flipPanel: model.flipPanel(towardsLeading: random.next() % 2 == 0) + case .toggleSymbols: model.toggleSymbols() + case .toggleMore: model.toggleMore() + case .toggleShift: model.toggleShift() + case .appeared: + model.hasFullAccess = true + model.appeared() + case .appearedNoAccess: + // Without Full Access the keyboard can't reach the app: whatever was + // in flight is the app's to end; the command ledger starts over. + model.hasFullAccess = false + model.appeared() + model.hasFullAccess = true + epoch = true + case .disappeared: model.disappeared() + case .contextChanged: model.contextChanged() + case .resultFresh: + SharedStore.write( + DictationResult(id: UUID(), text: "words", deliveredAt: Date(), recipient: nil), forKey: BlurtShared.Key.result) + model.resultArrived() + case .resultStale: + SharedStore.write( + DictationResult(id: UUID(), text: "old", deliveredAt: Date().addingTimeInterval(-60), recipient: nil), + forKey: BlurtShared.Key.result) + model.resultArrived() + case .resultForOther: + SharedStore.write( + DictationResult(id: UUID(), text: "theirs", deliveredAt: Date(), recipient: "someone-else"), + forKey: BlurtShared.Key.result) + model.resultArrived() + case .hostTyped: + proxy.before += "x" + model.contextChanged() + case .hostMovedCursor: + proxy.before += "ab" + proxy.after = String(proxy.after.dropFirst(2)) + model.contextChanged() + } + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/KeyboardModelGateTests.swift b/App/BlurtiOS/BlurtiOSTests/KeyboardModelGateTests.swift new file mode 100644 index 00000000..9ce960ea --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/KeyboardModelGateTests.swift @@ -0,0 +1,241 @@ +import Foundation +import Testing + +@testable import BlurtiOSCore + +/// The mic key's gate against the app's phases: which phase settles which +/// press, the retry, the sender on every command, and the term field closing +/// a live dictation. +@Suite("Keyboard model: the gate and the app's answers", .serialized) +@MainActor +struct KeyboardModelGateTests { + + /// The press still out, aged past the window the app would take it in. + private func agePress(_ model: KeyboardModel) { + guard let press = model.unansweredPressCommand else { return } + model.unansweredPressCommand = KeyboardCommand( + id: press.id, kind: press.kind, priorText: press.priorText, selectedText: press.selectedText, + sentAt: Date().addingTimeInterval(-BlurtShared.commandFreshnessWindow - 1), keyboard: press.keyboard) + } + + @Test("a settled phase resets a latched gate — once it answers the press; an older notice does not") + func resetOnSettled() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let commands = rig.commands + model.micDown() + model.micUp() + #expect(!model.gate.isIdle) + // The previous dictation's words land a moment after the press: not ours. + model.apply(snapshot(.pasted)) + #expect(!model.gate.isIdle) + // The next tap is still the stop, not a second press. + model.micDown() + model.micUp() + #expect(commands.sent.map(\.kind) == [.press, .release]) + #expect(model.gate.isIdle) + // A phase that answers the press settles it. + model.micDown() + model.micUp() + let press = commands.sent.last?.id + model.apply(PhaseSnapshot(state: .error, message: "no mic", level: 0, at: Date(), command: press)) + #expect(model.gate.isIdle) + // And an in-flight phase answers it too, whatever id it carries. + model.micDown() + model.micUp() + model.apply(snapshot(.connecting)) + #expect(model.unansweredPress == nil) + model.apply(snapshot(.pasted)) + #expect(model.gate.isIdle) + } + + @Test("a command names the keyboard that sent it") + func commandCarriesSender() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + rig.model.micDown() + rig.model.micUp() + #expect(rig.commands.sent.map(\.keyboard) == [rig.model.instanceID]) + } + + @Test("a press the app never answers is sent again after the retry delay; one it answered is not") + func pressRetried() async throws { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let commands = rig.commands + model.micDown() + model.micUp() + try await Task.sleep(for: KeyboardModel.commandRetryDelay + .milliseconds(300)) + #expect(commands.sent.map(\.kind) == [.press, .press]) + #expect(commands.sent[0].id == commands.sent[1].id) + // Answered: the app published a phase the keyboard reads on the retry. + model.apply(PhaseSnapshot(state: .pasted, message: nil, level: 0, at: Date(), command: model.unansweredPress)) + model.micDown() + model.micUp() + SharedStore.write(snapshot(.recording), forKey: BlurtShared.Key.phase) + try await Task.sleep(for: KeyboardModel.commandRetryDelay + .milliseconds(300)) + #expect(commands.sent.map(\.kind) == [.press, .press, .press]) + #expect(model.snapshot.state == .recording) + } + + @Test( + "opening the key-term field over a live dictation closes it: a recording is released, a mic coming up cancelled") + func termFieldClosesDictation() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let commands = rig.commands + model.micDown() + model.micUp() + model.apply(snapshot(.recording)) + model.beginAddingTerm() + #expect(commands.sent.map(\.kind) == [.press, .release]) + #expect(model.gate.isIdle) + #expect(model.termDraft == "") + model.cancelAddingTerm() + model.apply(snapshot(.idle)) + model.micDown() + model.micUp() + model.apply(snapshot(.connecting)) + model.beginAddingTerm() + #expect(commands.sent.map(\.kind) == [.press, .release, .press, .cancel]) + #expect(model.gate.isIdle) + // Nothing in flight: the field opens and nothing is sent. + model.cancelAddingTerm() + model.apply(snapshot(.idle)) + model.beginAddingTerm() + #expect(commands.sent.count == 4) + } + + @Test("the + while words are being transcribed lets them land; from a highlighted word it cancels them") + func termFieldDuringProcessing() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let commands = rig.commands + // A tap latched the gate; the app auto-released, with no finger event. + model.micDown() + model.micUp() + model.apply(snapshot(.recording)) + model.apply(snapshot(.processing)) + #expect(!model.gate.isIdle) + model.beginAddingTerm() + #expect(commands.sent.map(\.kind) == [.press]) + #expect(model.gate.isIdle) + model.cancelAddingTerm() + rig.proxy.selected = "Rizz" + model.beginAddingTerm() + #expect(commands.sent.map(\.kind) == [.press, .cancel]) + } + + @Test("leaving the screen while the words are being transcribed lets them finish") + func disappearDuringProcessing() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let commands = rig.commands + model.micDown() + model.micUp() + model.apply(snapshot(.recording)) + model.micDown() + model.micUp() + model.apply(snapshot(.processing)) + model.disappeared() + #expect(commands.sent.map(\.kind) == [.press, .release]) + #expect(model.gate.isIdle) + } + + @Test("a swipe off the orb undoes only its own press: a fresh one is cancelled, a latched recording kept") + func undoPress() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let commands = rig.commands + // A swipe whose press went out from idle: taken back. + model.micDown() + model.undoPress() + #expect(commands.sent.map(\.kind) == [.press, .cancel]) + #expect(model.gate.isIdle) + // A tap latches a recording; a swipe across the orb leaves it running, + // and the next tap is still the stop. + model.apply(snapshot(.idle)) + model.micDown() + model.micUp() + model.apply(snapshot(.recording)) + model.micDown() + model.undoPress() + #expect(commands.sent.map(\.kind) == [.press, .cancel, .press]) + #expect(!model.gate.isIdle) + model.micDown() + model.micUp() + #expect(commands.sent.map(\.kind) == [.press, .cancel, .press, .release]) + } + + @Test("a swipe over a recording this keyboard just released cancels nothing: those words are being transcribed") + func undoPressOverReleasedRecording() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let commands = rig.commands + model.micDown() + model.micUp() + model.apply(snapshot(.recording)) + // Leaving the screen releases it; the app hasn't published `.processing` + // yet, so the picture still says recording and the gate is idle. + model.disappeared() + #expect(commands.sent.map(\.kind) == [.press, .release]) + // A swipe across the orb in that moment: its press went out over a + // dictation the app was still running, so it started nothing to undo. + model.micDown() + model.undoPress() + #expect(commands.sent.map(\.kind) == [.press, .release, .press]) + #expect(model.gate.isIdle) + } + + @Test("a press the app never answers stops holding the gate once it is too old for the app to take") + func unansweredPressExpires() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + model.micDown() + model.micUp() + #expect(!model.expireUnansweredPress()) + #expect(model.unansweredPress != nil) + // A settled phase that doesn't answer it leaves the latch while it is fresh… + model.apply(snapshot(.pasted)) + #expect(!model.gate.isIdle) + // …and lets it go once the app would drop it unread. + agePress(model) + model.apply(snapshot(.idle)) + #expect(model.unansweredPress == nil) + #expect(model.gate.isIdle) + } + + @Test("an appearance lets go of a press too old to be answered, so hands-free can start") + func appearanceExpiresPress() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let commands = rig.commands + SharedStore.autoDictate = true + model.micDown() + model.micUp() + agePress(model) + model.appeared() + // Hands-free pressed afresh rather than skipping over a dead latch. + #expect(commands.sent.map(\.kind) == [.press, .press]) + #expect(model.unansweredPress == commands.sent.last?.id) + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/KeyboardModelTests.swift b/App/BlurtiOS/BlurtiOSTests/KeyboardModelTests.swift new file mode 100644 index 00000000..6f9f3e60 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/KeyboardModelTests.swift @@ -0,0 +1,359 @@ +import Foundation +import Testing +import UIKit + +@testable import BlurtiOSCore + +/// A host text field the model can type into, remembering what it did. +final class FakeProxy: NSObject, UITextDocumentProxy { + var before = "" + var after = "" + var selected: String? + var autocapitalizationType: UITextAutocapitalizationType = .sentences + var keyboardType: UIKeyboardType = .default + var returnKeyType: UIReturnKeyType = .default + + var documentContextBeforeInput: String? { before } + var documentContextAfterInput: String? { after } + var selectedText: String? { selected } + var documentInputMode: UITextInputMode? { nil } + var documentIdentifier: UUID { UUID() } + var hasText: Bool { !before.isEmpty || !after.isEmpty } + + func insertText(_ text: String) { + if let selected, !selected.isEmpty { + self.selected = nil + } + before += text + } + + func deleteBackward() { _ = before.popLast() } + func adjustTextPosition(byCharacterOffset offset: Int) {} + func setMarkedText(_ markedText: String, selectedRange: NSRange) {} + func unmarkText() {} +} + +@Suite("Keyboard model: typing rules") +struct KeyboardModelTypingTests { + private func model(before: String = "", selected: String? = nil) -> (KeyboardModel, FakeProxy) { + let proxy = FakeProxy() + proxy.before = before + proxy.selected = selected + let model = KeyboardModel() + model.proxyOverride = proxy + return (model, proxy) + } + + @Test("sentence start: empty, after a full stop and a space, after a newline") + func sentenceStart() { + #expect(KeyboardModel.isSentenceStart("")) + #expect(KeyboardModel.isSentenceStart("Hello. ")) + #expect(KeyboardModel.isSentenceStart("Hello?")) + #expect(KeyboardModel.isSentenceStart("Line\n")) + #expect(!KeyboardModel.isSentenceStart("Hello ")) + #expect(!KeyboardModel.isSentenceStart("Hello, ")) + } + + @Test("a letter goes to the field and lowers shift") + func typing() { + let (model, proxy) = model() + model.shifted = true + model.type("H") + #expect(proxy.before == "H") + #expect(!model.shifted) + } + + @Test("two quick spaces after a word become a full stop and a space") + func doubleSpace() { + let (model, proxy) = model(before: "hello") + model.space() + #expect(proxy.before == "hello ") + model.space() + #expect(proxy.before == "hello. ") + } + + @Test("two quick spaces after punctuation stay spaces") + func doubleSpaceAfterPunctuation() { + let (model, proxy) = model(before: "hello.") + model.space() + model.space() + #expect(proxy.before == "hello. ") + } + + @Test("the term field takes the keys instead of the host, and return saves nothing empty") + func termField() { + let (model, proxy) = model(before: "yo ") + model.beginAddingTerm() + #expect(model.termDraft == "") + #expect(model.effectiveLayout == .full) + model.type("R") + model.type("i") + model.space() + model.type("z") + model.deleteBackward() + #expect(model.termDraft == "Ri ") + #expect(proxy.before == "yo ") + model.cancelAddingTerm() + #expect(model.termDraft == nil) + } + + @Test("a swipe while a term is open flips nothing; the panel is back on the mic when the term closes") + func noFlipDuringTerm() { + let (model, _) = model() + model.layout = .panel + model.flipPanel(towardsLeading: true) + #expect(model.panelShowsKeys) + model.flipPanel(towardsLeading: false) + #expect(!model.panelShowsKeys) + model.beginAddingTerm() + model.flipPanel(towardsLeading: true) + #expect(!model.panelShowsKeys) + #expect(model.effectiveLayout == .full) + model.cancelAddingTerm() + #expect(model.effectiveLayout == .panel) + } + + @Test("a selection seeds the term and is replaced on save") + func termFromSelection() { + let (model, proxy) = model(before: "he said ", selected: "riz") + model.beginAddingTerm() + #expect(model.termDraft == "riz") + model.type("z") + model.saveTerm() + #expect(model.termDraft == nil) + #expect(proxy.before == "he said rizz") + } + + @Test("typing that reached the host field while the term is open moves into the term") + func hardwareTypingClaimed() { + let (model, proxy) = model(before: "yo ") + model.beginAddingTerm() + // A hardware key is one character per change. + proxy.before = "yo R" + model.contextChanged() + proxy.before = "yo i" + model.contextChanged() + #expect(model.termDraft == "Ri") + #expect(proxy.before == "yo ") + // A block landing at once (a paste, the system's dictation) is not typing. + proxy.before = "yo hello" + model.contextChanged() + #expect(model.termDraft == "Ri") + #expect(proxy.before == "yo hello") + } + + @Test("the return key wears the field's word") + func returnLabel() { + let (model, proxy) = model() + proxy.returnKeyType = .send + model.contextChanged() + #expect(model.returnLabel == "send") + proxy.returnKeyType = .default + model.contextChanged() + #expect(model.returnLabel == nil) + } + + // MARK: The chip for a highlighted word + + @Test("a selection is a key-term candidate when trimmed, on one line and short") + func termCandidate() { + #expect(KeyboardModel.termCandidate(from: " Rizz ") == "Rizz") + #expect(KeyboardModel.termCandidate(from: "Neil Bisht") == "Neil Bisht") + #expect(KeyboardModel.termCandidate(from: nil) == nil) + #expect(KeyboardModel.termCandidate(from: " ") == nil) + #expect(KeyboardModel.termCandidate(from: "two\nlines") == nil) + #expect(KeyboardModel.termCandidate(from: String(repeating: "a", count: KeyboardModel.termLengthCap)) != nil) + #expect(KeyboardModel.termCandidate(from: String(repeating: "a", count: KeyboardModel.termLengthCap + 1)) == nil) + } + + @Test("a highlighted word is the chip; one tap adds it as it stands and leaves the text alone") + func addSelectedTerm() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let (model, proxy) = model(before: "he said ", selected: "Rizz") + model.hasFullAccess = true + model.contextChanged() + #expect(model.selectedTerm == "Rizz") + #expect(!model.selectedTermIsKnown) + model.addSelectedTerm() + #expect(SharedStore.keyTerms == ["Rizz"]) + #expect(model.selectedTermIsKnown) + #expect(model.termSavedAt != nil) + #expect(model.termDraft == nil) + #expect(proxy.before == "he said ") + #expect(proxy.selected == "Rizz") + // A second tap adds nothing twice. + model.addSelectedTerm() + #expect(SharedStore.keyTerms == ["Rizz"]) + } + + @Test("a word already in the key terms is known on selection; clearing the highlight takes the chip away") + func knownSelection() { + let suite = ScratchSuite() + defer { suite.tearDown() } + SharedStore.addKeyTerm("rizz") + let (model, proxy) = model(before: "he said ", selected: "Rizz") + model.hasFullAccess = true + model.contextChanged() + #expect(model.selectedTerm == "Rizz") + #expect(model.selectedTermIsKnown) + proxy.selected = nil + model.contextChanged() + #expect(model.selectedTerm == nil) + #expect(!model.selectedTermIsKnown) + } + + @Test("no chip without Full Access, or for a selection that is no term") + func noChip() { + let (model, proxy) = model(before: "", selected: "Rizz") + model.contextChanged() + #expect(model.selectedTerm == nil) + model.hasFullAccess = true + proxy.selected = "a whole\nparagraph" + model.contextChanged() + #expect(model.selectedTerm == nil) + } + + @Test("holding the chip opens the field with the word; saving puts the + back though the word stays highlighted") + func chipToField() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let (model, proxy) = model(before: "he said ", selected: "riz") + model.hasFullAccess = true + model.contextChanged() + model.beginAddingTerm() + #expect(model.termDraft == "riz") + model.saveTerm() + #expect(SharedStore.keyTerms == ["riz"]) + #expect(proxy.before == "he said ") + #expect(proxy.selected == "riz") + #expect(model.selectedTerm == nil) + #expect(model.termSavedAt != nil) + // The host still reports the highlight on the next change: no chip. + model.contextChanged() + #expect(model.selectedTerm == nil) + } +} + +@Suite("Keyboard model: the chip and the term field") +struct KeyboardModelChipTests { + private func model(before: String = "", selected: String? = nil) -> (KeyboardModel, FakeProxy) { + let proxy = FakeProxy() + proxy.before = before + proxy.selected = selected + let model = KeyboardModel() + model.proxyOverride = proxy + return (model, proxy) + } + + @Test( + "a tap on a chip for a word Blurt has puts the + back; the chip returns for another word, or the same one again") + func dismissKnownChip() { + let suite = ScratchSuite() + defer { suite.tearDown() } + SharedStore.addKeyTerm("Rizz") + let (model, proxy) = model(before: "he said ", selected: "Rizz") + model.hasFullAccess = true + model.contextChanged() + #expect(model.selectedTerm == "Rizz") + #expect(model.selectedTermIsKnown) + model.addSelectedTerm() + #expect(model.selectedTerm == nil) + #expect(SharedStore.keyTerms == ["Rizz"]) + model.contextChanged() + #expect(model.selectedTerm == nil) + // Another word highlighted: its chip. + proxy.selected = "Gyatt" + model.contextChanged() + #expect(model.selectedTerm == "Gyatt") + #expect(!model.selectedTermIsKnown) + // The highlight cleared and the first word highlighted again: known again. + proxy.selected = nil + model.contextChanged() + proxy.selected = "Rizz" + model.contextChanged() + #expect(model.selectedTerm == "Rizz") + #expect(model.selectedTermIsKnown) + } + + @Test("the + coming back after a term was added from the chip keeps the highlight's word out of the chip") + func dismissAfterAdd() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let (model, _) = model(before: "he said ", selected: "Rizz") + model.hasFullAccess = true + model.contextChanged() + model.addSelectedTerm() + // While the check shows, the chip is still up with it. + #expect(model.selectedTerm == "Rizz") + #expect(model.selectedTermIsKnown) + model.dismissSelection() + #expect(model.selectedTerm == nil) + model.contextChanged() + #expect(model.selectedTerm == nil) + } + + @Test("collapsing the highlight the field opened over is not typing: the word stays in the text") + func collapseIsNotTyping() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let (model, proxy) = model(before: "he said ", selected: "Rizz") + proxy.after = " today" + model.hasFullAccess = true + model.contextChanged() + model.beginAddingTerm() + #expect(model.termDraft == "Rizz") + // The user taps at the end of the word: the highlight collapses there. + proxy.selected = nil + proxy.before = "he said Rizz" + model.contextChanged() + #expect(proxy.before == "he said Rizz") + #expect(model.termDraft == "Rizz") + } + + @Test("saving does not replace the word once the cursor has moved off the highlight") + func noReplaceAfterCursorMove() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let (model, proxy) = model(before: "he said ", selected: "riz") + model.hasFullAccess = true + model.contextChanged() + model.beginAddingTerm() + model.type("z") + // The host moved the cursor but still reports the old selection for a moment. + proxy.before = "he said riz and" + model.saveTerm() + #expect(SharedStore.keyTerms == ["rizz"]) + #expect(proxy.before == "he said riz and") + } + + @Test("a key term capitalises after a space too") + func termShift() { + let (model, _) = model() + model.hasFullAccess = true + model.beginAddingTerm() + #expect(model.shifted) + model.type("N") + #expect(!model.shifted) + model.space() + #expect(model.shifted) + model.type("B") + #expect(!model.shifted) + #expect(model.termDraft == "N B") + } + + @Test("123 opens the symbols, #+= the second page, 123 there the first, ABC the letters") + func symbolPages() { + let (model, _) = model() + #expect(!model.symbolsPage && !model.morePage) + model.toggleSymbols() + #expect(model.symbolsPage && !model.morePage) + model.toggleMore() + #expect(model.symbolsPage && model.morePage) + model.toggleMore() + #expect(model.symbolsPage && !model.morePage) + model.toggleMore() + model.toggleSymbols() + #expect(!model.symbolsPage && !model.morePage) + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/PanelCarouselTests.swift b/App/BlurtiOS/BlurtiOSTests/PanelCarouselTests.swift new file mode 100644 index 00000000..73bf0f30 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/PanelCarouselTests.swift @@ -0,0 +1,73 @@ +import Testing + +@testable import BlurtiOS + +@Suite("Panel carousel") +struct PanelCarouselTests { + @Test("a flip slides a page towards the finger, and settling puts the other page in the middle") + func flipAndSettle() { + var strip = PanelCarouselState(keysUp: false) + let left = strip.flip(to: true, towardsLeading: true, animated: true) + #expect(left == PanelCarouselState.Motion(slide: -1, generation: 1)) + #expect(strip.incomingEdge == .trailing) + #expect(!strip.keysUp && strip.target && strip.inFlight) + let settled = strip.settle(1) + #expect(settled && strip.keysUp && !strip.inFlight) + // From the keys, a swipe right brings the panel in from the leading side. + let right = strip.flip(to: false, towardsLeading: false, animated: true) + #expect(right == PanelCarouselState.Motion(slide: 1, generation: 2)) + #expect(strip.incomingEdge == .leading) + let back = strip.settle(2) + #expect(back && !strip.keysUp) + } + + @Test("two swipes the same way wrap: the incoming page is set down on the same side each time") + func wrap() { + var strip = PanelCarouselState(keysUp: false) + let first = strip.flip(to: true, towardsLeading: true, animated: true) + #expect(first?.slide == -1) + let firstSettled = strip.settle(1) + #expect(firstSettled) + let second = strip.flip(to: false, towardsLeading: true, animated: true) + #expect(second?.slide == -1) + #expect(strip.incomingEdge == .trailing) + let secondSettled = strip.settle(2) + #expect(secondSettled && !strip.keysUp) + } + + @Test("a flip back mid-slide retreats, and the overtaken slide cannot settle the strip") + func retreat() { + var strip = PanelCarouselState(keysUp: false) + let out = strip.flip(to: true, towardsLeading: true, animated: true) + #expect(out?.generation == 1) + let back = strip.flip(to: false, towardsLeading: false, animated: true) + #expect(back == PanelCarouselState.Motion(slide: 0, generation: 2)) + #expect(strip.incomingEdge == .trailing, "the page on its way in retreats where it came from") + let late = strip.settle(1) + #expect(!late, "the first slide's ending is too late") + #expect(strip.inFlight && !strip.keysUp) + let settled = strip.settle(2) + #expect(settled && !strip.keysUp && !strip.inFlight) + } + + @Test("a retreat cut short by another flip keeps the incoming page's side") + func retreatCutShort() { + var strip = PanelCarouselState(keysUp: false) + _ = strip.flip(to: true, towardsLeading: true, animated: true) + _ = strip.flip(to: false, towardsLeading: false, animated: true) + let again = strip.flip(to: true, towardsLeading: false, animated: true) + #expect(again == PanelCarouselState.Motion(slide: -1, generation: 3)) + #expect(strip.incomingEdge == .trailing) + let settled = strip.settle(3) + #expect(settled && strip.keysUp) + } + + @Test("without motion a flip settles at once, with nothing to animate") + func still() { + var strip = PanelCarouselState(keysUp: false) + let still = strip.flip(to: true, towardsLeading: true, animated: false) + #expect(still == nil && strip.keysUp && !strip.inFlight) + let same = strip.flip(to: true, towardsLeading: true, animated: true) + #expect(same == nil, "already there: nothing moves") + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/ProtocolTests.swift b/App/BlurtiOS/BlurtiOSTests/ProtocolTests.swift new file mode 100644 index 00000000..57dc517c --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/ProtocolTests.swift @@ -0,0 +1,289 @@ +import BlurtEngine +import Foundation +import Testing +import UIKit + +@testable import BlurtiOSCore + +@Suite("The two-process contract", .serialized) +@MainActor +struct ProtocolTests { + @Test("the key-term key is the engine's own, so the app's and the keyboard's readers agree") + func keyTermsKey() { + #expect(SharedStore.keyTermsKey == KeyTermsStore.defaultsKey) + } + + @Test("listening needs both a live window and a live app") + func isListening() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let now = Date() + SharedStore.listeningUntil = now.addingTimeInterval(600) + SharedStore.appSeenAt = now.addingTimeInterval(-2) + #expect(SharedStore.isListening(now: now)) + SharedStore.appSeenAt = now.addingTimeInterval(-(SharedStore.presenceWindow + 1)) + #expect(!SharedStore.isListening(now: now)) + SharedStore.appSeenAt = now + SharedStore.listeningUntil = now.addingTimeInterval(-1) + #expect(!SharedStore.isListening(now: now)) + } + + @Test("the presence windows outlast two missed heartbeats") + func presenceRatio() { + #expect(SharedStore.presenceWindow > 2 * SharedStore.appHeartbeatInterval) + #expect(SharedStore.presenceWindow > 2 * SharedStore.keyboardHeartbeatInterval) + } + + @Test("addKeyTerm trims and refuses a duplicate in any case") + func addKeyTerm() { + let suite = ScratchSuite() + defer { suite.tearDown() } + #expect(SharedStore.addKeyTerm(" Rizz ")) + #expect(!SharedStore.addKeyTerm("rizz")) + #expect(!SharedStore.addKeyTerm(" ")) + #expect(SharedStore.keyTerms == ["Rizz"]) + } + + @Test("with no keyboard on screen the words go to the clipboard and the phase is the quiet one") + func relayFallsBackToClipboard() async { + let suite = ScratchSuite() + defer { suite.tearDown() } + SharedStore.keyboardSeenAt = nil + UIPasteboard.general.string = "" + let injector = KeyboardRelayInjector() + await #expect(throws: BlurtError.noEditableTarget) { + try await injector.insert("hello there", after: nil, windowTitle: nil) + } + #expect(UIPasteboard.general.string == "hello there") + #expect(SharedStore.read(DictationResult.self, forKey: BlurtShared.Key.result) == nil) + } + + @Test("a result the keyboard never takes is copied out after the delivery timeout") + func relayTimesOut() async { + let suite = ScratchSuite() + defer { suite.tearDown() } + SharedStore.keyboardSeenAt = Date() + SharedStore.keyboardInstance = "kb-1" + UIPasteboard.general.string = "" + let injector = KeyboardRelayInjector() + let started = Date() + await #expect(throws: BlurtError.noEditableTarget) { + try await injector.insert("late words", after: nil, windowTitle: nil) + } + #expect(Date().timeIntervalSince(started) >= KeyboardRelayInjector.deliveryTimeout - 0.5) + #expect(UIPasteboard.general.string == "late words") + #expect(SharedStore.read(DictationResult.self, forKey: BlurtShared.Key.result) == nil) + } + + @Test("the words go back to the keyboard that pressed; another keyboard up in its place means the clipboard") + func relayToPresser() async throws { + let suite = ScratchSuite() + defer { suite.tearDown() } + SharedStore.keyboardSeenAt = Date() + SharedStore.keyboardInstance = "kb-2" + UIPasteboard.general.string = "" + let elsewhere = KeyboardRelayInjector(presser: { "kb-1" }) + await #expect(throws: BlurtError.noEditableTarget) { + try await elsewhere.insert("wrong app", after: nil, windowTitle: nil) + } + #expect(UIPasteboard.general.string == "wrong app") + #expect(SharedStore.read(DictationResult.self, forKey: BlurtShared.Key.result) == nil) + let same = KeyboardRelayInjector(presser: { "kb-2" }) + let taker = Task { + try? await Task.sleep(for: .milliseconds(300)) + let pending = SharedStore.read(DictationResult.self, forKey: BlurtShared.Key.result) + SharedStore.remove(forKey: BlurtShared.Key.result) + return pending + } + try await same.insert("right app", after: nil, windowTitle: nil) + #expect(await taker.value?.recipient == "kb-2") + } + + @Test("a result taken by the keyboard counts as delivered") + func relayDelivered() async throws { + let suite = ScratchSuite() + defer { suite.tearDown() } + SharedStore.keyboardSeenAt = Date() + SharedStore.keyboardInstance = "kb-1" + let injector = KeyboardRelayInjector() + let taker = Task { + try? await Task.sleep(for: .milliseconds(300)) + let pending = SharedStore.read(DictationResult.self, forKey: BlurtShared.Key.result) + SharedStore.remove(forKey: BlurtShared.Key.result) + return pending + } + try await injector.insert("on time", after: nil, windowTitle: nil) + let taken = await taker.value + #expect(taken?.text == "on time") + #expect(taken?.recipient == "kb-1") + } +} + +@Suite("Keyboard model: the mic, the phase and results", .serialized) +@MainActor +struct KeyboardModelProtocolTests { + + @Test("a tap presses and latches; the next tap releases") + func tapThenTap() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let commands = rig.commands + model.micDown() + model.micUp() + #expect(commands.sent.map(\.kind) == [.press]) + model.apply(snapshot(.recording)) + model.micDown() + model.micUp() + #expect(commands.sent.map(\.kind) == [.press, .release]) + } + + @Test("a dictation started elsewhere latches the gate, so one tap stops it") + func latchOnInFlight() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let commands = rig.commands + model.apply(snapshot(.recording)) + #expect(!model.gate.isIdle) + model.micDown() + model.micUp() + #expect(commands.sent.map(\.kind) == [.release]) + } + + @Test("a press while transcribing is refused rather than dropped by the engine") + func noPressWhileProcessing() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let commands = rig.commands + model.apply(snapshot(.processing)) + model.micDown() + model.micUp() + #expect(commands.sent.isEmpty) + #expect(model.gate.isIdle) + } + + @Test("leaving the screen releases a latched recording and cancels anything earlier") + func disappearEndsDictation() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let commands = rig.commands + model.micDown() + model.micUp() + model.apply(snapshot(.recording)) + model.disappeared() + #expect(commands.sent.map(\.kind) == [.press, .release]) + #expect(model.gate.isIdle) + #expect(SharedStore.keyboardSeenAt == nil) + } + + @Test("a result is inserted once, only while fresh, only if addressed here") + func results() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let proxy = rig.proxy + proxy.before = "Hi" + func deliver(_ text: String, age: TimeInterval = 0, to recipient: String?) { + SharedStore.write( + DictationResult(id: UUID(), text: text, deliveredAt: Date().addingTimeInterval(-age), recipient: recipient), + forKey: BlurtShared.Key.result) + } + deliver("one", to: nil) + model.resultArrived() + #expect(proxy.before == "Hi one") + #expect(SharedStore.read(DictationResult.self, forKey: BlurtShared.Key.result) == nil) + deliver("two", age: 20, to: nil) + model.resultArrived() + #expect(proxy.before == "Hi one") + deliver("three", to: "someone-else") + model.resultArrived() + #expect(proxy.before == "Hi one") + deliver("four", to: model.instanceID) + model.resultArrived() + #expect(proxy.before == "Hi one four") + } + + @Test("the term field never claims a cursor move or a selection") + func termFieldIgnoresCursorMoves() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let proxy = rig.proxy + proxy.before = "Hello wor" + proxy.after = "ld" + model.beginAddingTerm() + // The cursor moved two to the right: both sides changed. + proxy.before = "Hello world" + proxy.after = "" + model.contextChanged() + #expect(model.termDraft == "") + #expect(proxy.before == "Hello world") + // A selection appeared with the same growth. + proxy.before = "Hello wor" + proxy.after = "ld" + model.beginAddingTerm() + proxy.before = "Hello world" + proxy.selected = "ld" + model.contextChanged() + #expect(model.termDraft == "") + } + + @Test("a selection with spaces around it is replaced with the spaces kept") + func termReplaceKeepsSpaces() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let proxy = rig.proxy + proxy.before = "he said" + proxy.selected = " riz " + model.beginAddingTerm() + #expect(model.termDraft == "riz") + model.type("z") + model.saveTerm() + #expect(proxy.before == "he said rizz ") + #expect(SharedStore.keyTerms == ["rizz"]) + } + + @Test("shift follows the field: words, all characters, none") + func shiftModes() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let rig = KeyboardRig() + let model = rig.model + let proxy = rig.proxy + proxy.before = "hello " + proxy.autocapitalizationType = .words + model.contextChanged() + #expect(model.shifted) + proxy.before = "hello w" + model.contextChanged() + #expect(!model.shifted) + proxy.autocapitalizationType = .allCharacters + model.contextChanged() + #expect(model.shifted) + proxy.autocapitalizationType = .none + model.shifted = false + proxy.before = "" + model.contextChanged() + #expect(!model.shifted) + } + + @Test("number fields open on the symbols page") + func symbolsForNumbers() { + #expect(KeyboardModel.wantsSymbols(.numberPad)) + #expect(KeyboardModel.wantsSymbols(.phonePad)) + #expect(KeyboardModel.wantsSymbols(.decimalPad)) + #expect(!KeyboardModel.wantsSymbols(.default)) + #expect(!KeyboardModel.wantsSymbols(nil)) + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/SharedStateTests.swift b/App/BlurtiOS/BlurtiOSTests/SharedStateTests.swift new file mode 100644 index 00000000..134d6ad8 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/SharedStateTests.swift @@ -0,0 +1,200 @@ +import Foundation +import SwiftUI +import Testing + +@testable import BlurtiOS +@testable import BlurtiOSCore + +@Suite("Mic alignment setting") +struct MicAlignmentSettingTests { + @Test("the mic's side round-trips through the App Group; unset or unknown is the middle") + func roundTrip() { + let suite = ScratchSuite() + defer { suite.tearDown() } + #expect(SharedStore.micAlignment == .center) + SharedStore.micAlignment = .right + #expect(SharedStore.micAlignment == .right) + SharedStore.defaults.set("sideways", forKey: BlurtShared.Key.micAlignment) + #expect(SharedStore.micAlignment == .center) + } + + @Test("left and right are the phone's sides whatever the language reads in; the + leads only at the trailing edge") + func sides() { + #expect(MicAlignment.left.alignment(in: .leftToRight) == .leading) + #expect(MicAlignment.left.alignment(in: .rightToLeft) == .trailing) + #expect(MicAlignment.right.alignment(in: .leftToRight) == .trailing) + #expect(MicAlignment.right.alignment(in: .rightToLeft) == .leading) + #expect(MicAlignment.center.alignment(in: .rightToLeft) == .center) + #expect(MicAlignment.right.plusLeads(in: .leftToRight)) + #expect(MicAlignment.left.plusLeads(in: .rightToLeft)) + #expect(!MicAlignment.left.plusLeads(in: .leftToRight)) + #expect(!MicAlignment.center.plusLeads(in: .leftToRight)) + } + + @Test("the gallery and probe words name a side") + func parse() { + #expect(MicAlignment.parse(["-BlurtGallery", "panel", "idle", "-BlurtGalleryAlign", "right"]) == .right) + #expect(MicAlignment.parse(["-BlurtGallery", "panel", "idle"]) == nil) + #expect(MicAlignment(rawValue: "left") == .left) + } +} + +@Suite("Key term list") +struct KeyTermListTests { + @Test("parse trims, drops blanks, and deduplicates case-insensitively in first-seen order") + func parse() { + #expect(KeyTermList.parse(" Rizz , skibidi,,RIZZ, Blurt ,") == ["Rizz", "skibidi", "Blurt"]) + } + + @Test("join is what parse reads back") + func roundTrip() { + let terms = ["Neil Bisht", "AssemblyAI", "rizz"] + #expect(KeyTermList.parse(KeyTermList.join(terms)) == terms) + } + + @Test("an empty or all-blank list is empty") + func empty() { + #expect(KeyTermList.parse("").isEmpty) + #expect(KeyTermList.parse(" , ,").isEmpty) + } +} + +@Suite("Phase snapshot staleness") +struct PhaseSnapshotTests { + private func snapshot(_ state: PhaseSnapshot.State, ageSeconds: TimeInterval) -> PhaseSnapshot { + PhaseSnapshot(state: state, message: nil, level: 0, at: Date().addingTimeInterval(-ageSeconds)) + } + + @Test("idle is never stale") + func idle() { + #expect(!snapshot(.idle, ageSeconds: 100_000).isStale) + } + + @Test("a notice is over after three seconds") + func notices() { + for state in [PhaseSnapshot.State.pasted, .copied, .error] { + #expect(!snapshot(state, ageSeconds: 2).isStale) + #expect(snapshot(state, ageSeconds: 4).isStale) + } + } + + @Test("an in-flight state outlives no dictation past 130 seconds") + func inFlight() { + for state in [PhaseSnapshot.State.connecting, .recording, .processing] { + #expect(!snapshot(state, ageSeconds: 120).isStale) + #expect(snapshot(state, ageSeconds: 131).isStale) + } + } + + @Test("the dwell matches the design: 1.2 s pasted, 2 s copied and error, none otherwise") + func dwell() { + #expect(snapshot(.pasted, ageSeconds: 0).noticeDwellSeconds == 1.2) + #expect(snapshot(.copied, ageSeconds: 0).noticeDwellSeconds == 2) + #expect(snapshot(.error, ageSeconds: 0).noticeDwellSeconds == 2) + #expect(snapshot(.recording, ageSeconds: 0).noticeDwellSeconds == nil) + } +} + +@Suite("Voice element setting") +struct VoiceElementSettingTests { + @Test("the mic concept round-trips through the App Group; unset or unknown is the shipped one") + func roundTrip() { + let suite = ScratchSuite() + defer { suite.tearDown() } + #expect(SharedStore.voiceElementKind == .shipped) + SharedStore.voiceElementKind = .streak + #expect(SharedStore.voiceElementKind == .streak) + SharedStore.defaults.set("z", forKey: BlurtShared.Key.voiceElement) + #expect(SharedStore.voiceElementKind == .shipped) + } +} + +@Suite("Shared defaults") +@MainActor +struct SharedDefaultsTests { + @Test("a payload round-trips through the App Group; unreadable or removed is nothing") + func payloads() { + let suite = ScratchSuite() + defer { suite.tearDown() } + let result = DictationResult(id: UUID(), text: "hi", deliveredAt: Date(), recipient: "kb") + SharedStore.write(result, forKey: BlurtShared.Key.result) + #expect(SharedStore.read(DictationResult.self, forKey: BlurtShared.Key.result)?.id == result.id) + SharedStore.defaults.set(Data("not json".utf8), forKey: BlurtShared.Key.result) + #expect(SharedStore.read(DictationResult.self, forKey: BlurtShared.Key.result) == nil) + SharedStore.write(result, forKey: BlurtShared.Key.result) + SharedStore.remove(forKey: BlurtShared.Key.result) + #expect(SharedStore.read(DictationResult.self, forKey: BlurtShared.Key.result) == nil) + } + + @Test("each setting has its default until chosen, and keeps what was chosen") + func settings() { + let suite = ScratchSuite() + defer { suite.tearDown() } + #expect(SharedStore.layout == .panel) + SharedStore.layout = .full + #expect(SharedStore.layout == .full) + #expect(SharedStore.themeID == "system") + SharedStore.themeID = "blurt" + #expect(SharedStore.themeID == "blurt") + #expect(SharedStore.autoDictate) + SharedStore.autoDictate = false + #expect(!SharedStore.autoDictate) + #expect(SharedStore.windowMinutes == 15) + // 0 is a real choice (until closed), not the unset default. + SharedStore.windowMinutes = 0 + #expect(SharedStore.windowMinutes == 0) + #expect(SharedStore.lexiconRefreshedAt == nil) + } + + @Test("a contact name has both sides equal; a text replacement does not") + func lexiconEntries() { + #expect(LexiconEntry(userInput: "Neil Bisht", documentText: "Neil Bisht").isName) + #expect(!LexiconEntry(userInput: "omw", documentText: "On my way!").isName) + } +} + +@Suite("Layouts and palettes") +struct LayoutTests { + @Test("every layout and mic side has its own name and description for Settings") + func settingsCopy() { + for cases in [ + KeyboardLayout.allCases.map { ($0.id, $0.title, $0.summary) }, + MicAlignment.allCases.map { ($0.id, $0.title, $0.summary) }, + ] { + #expect(Set(cases.map(\.0)).count == cases.count) + #expect(Set(cases.map(\.1)).count == cases.count) + #expect(Set(cases.map(\.2)).count == cases.count) + #expect(cases.allSatisfy { !$0.1.isEmpty && !$0.2.isEmpty }) + } + #expect(KeyboardLayout.full.id == "full") + #expect(MicAlignment.left.id == "left") + } + + @Test("heights are the rows at the iPhone keyboard's spacing (DESIGN.md)") + func heights() { + #expect(KeyboardLayout.slimBar.height == 60) + #expect(KeyboardLayout.panel.height == 216) + #expect(KeyboardLayout.full.height == 270) + } + + @Test("letter rows follow the phone's first language") + func letterRows() { + #expect(LetterLayout.forPreferredLanguages(["en-US", "fr-FR"]) == LetterLayout.qwerty) + #expect(LetterLayout.forPreferredLanguages(["fr-CA"]) == LetterLayout.azerty) + #expect(LetterLayout.forPreferredLanguages(["de-DE"]) == LetterLayout.qwertz) + #expect(LetterLayout.forPreferredLanguages(["cs"]) == LetterLayout.qwertz) + #expect(LetterLayout.forPreferredLanguages([]) == LetterLayout.qwerty) + } + + @Test("every theme has a distinct id; the brand has two faces; an unknown or retired id is the brand's") + func palettes() { + let ids = KeyboardPalette.all.map(\.id) + #expect(Set(ids).count == ids.count) + #expect(KeyboardPalette.resolve("blurt", dark: true).face == .dark) + #expect(KeyboardPalette.resolve("blurt", dark: false).keyText == DesignTokens.Themes.lightLegend) + #expect(KeyboardPalette.resolve("blurt", dark: true).keyText == DesignTokens.Themes.darkLegend) + #expect(KeyboardPalette.resolve("nope", dark: false).id == "blurt") + #expect(KeyboardPalette.resolve("system", dark: true) == .brandDark) + #expect(KeyboardPalette.brandLight != KeyboardPalette.brandDark) + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/TermPackTests.swift b/App/BlurtiOS/BlurtiOSTests/TermPackTests.swift new file mode 100644 index 00000000..ff564f6e --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/TermPackTests.swift @@ -0,0 +1,61 @@ +import Foundation +import Testing + +@testable import BlurtiOSCore + +@Suite("Term packs") +struct TermPackTests { + @Test("a blurt://terms link carries terms, a name and a sender") + func link() throws { + let url = try #require(URL(string: "blurt://terms?add=Rizz,Skibidi,%20rizz%20&name=Group%20chat&from=Neil")) + let pack = try #require(TermPack.from(url)) + #expect(pack.terms == ["Rizz", "Skibidi"]) + #expect(pack.name == "Group chat") + #expect(pack.from == "Neil") + } + + @Test("a link with no terms, or any other link, is not a pack") + func notAPack() throws { + #expect(TermPack.from(try #require(URL(string: "blurt://terms?add="))) == nil) + #expect(TermPack.from(try #require(URL(string: "blurt://start"))) == nil) + #expect(TermPack.from(try #require(URL(string: "https://example.com/?add=a"))) == nil) + } + + @Test("a .blurtterms file round-trips and is parsed by the key-term rules") + func file() throws { + let pack = TermPack(name: "Chat", from: "Neil", terms: ["Rizz", "rizz", " Blurt "]) + let url = FileManager.default.temporaryDirectory.appendingPathComponent("test-\(UUID().uuidString).blurtterms") + try JSONEncoder().encode(pack).write(to: url) + defer { try? FileManager.default.removeItem(at: url) } + let read = try #require(TermPack.from(url)) + #expect(read.terms == ["Rizz", "Blurt"]) + #expect(read.name == "Chat") + } + + @Test("the shared file is named for the list, with what a file name can't carry taken out") + func fileName() { + #expect(TermPack(name: "Gym / crew!", from: nil, terms: ["a"]).fileStem == "Gym crew") + #expect(TermPack(name: " ", from: nil, terms: ["a"]).fileStem == "Blurt key terms") + #expect(TermPack(name: "../../", from: nil, terms: ["a"]).fileStem == "Blurt key terms") + } + + @Test("the share sheet's file is the pack, as JSON a friend's Blurt reads back") + func shared() async throws { + // `exported(as:)` is iOS 18.2; the suite runs on the current simulator. + guard #available(iOS 18.2, *) else { + Issue.record("needs iOS 18.2 to export a Transferable") + return + } + let pack = TermPack(name: "Chat", from: "Neil", terms: ["Rizz", "Blurt"]) + let data = try await pack.exported(as: .blurtTerms) + let read = try JSONDecoder().decode(TermPack.self, from: data) + #expect(read.id == pack.id) + #expect(TermPack(name: "Chat", from: "Neil", terms: ["Blurt", "Rizz"]).id != pack.id) + } + + @Test("the plain text names the sender and lists the words") + func plainText() { + let pack = TermPack(name: "Chat", from: "Neil", terms: ["Rizz", "Blurt"]) + #expect(pack.plainText == "Neil's Blurt key terms (Chat): Rizz, Blurt") + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/TestSuite.swift b/App/BlurtiOS/BlurtiOSTests/TestSuite.swift new file mode 100644 index 00000000..3460a107 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/TestSuite.swift @@ -0,0 +1,62 @@ +import Foundation + +@testable import BlurtiOSCore + +/// A throwaway App Group for one test: nothing a test writes reaches the real +/// suite the installed app and keyboard share (the engine's own rule about +/// never touching the real Keychain in tests, applied here). +@MainActor +struct ScratchSuite { + let name: String + + init() { + name = "test-\(UUID().uuidString)" + SharedStore.override = UserDefaults(suiteName: name) + } + + func tearDown() { + SharedStore.override?.removePersistentDomain(forName: name) + SharedStore.override = nil + } +} + +/// A keyboard that is ready to dictate — a fake field, Full Access, the app +/// listening for the next ten minutes — with every command it sends captured +/// instead of reaching the App Group. Build it after `ScratchSuite`: the +/// listening window it opens is written to the store. +@MainActor +struct KeyboardRig { + final class Commands { + var sent: [KeyboardCommand] = [] + } + + let model: KeyboardModel + let proxy: FakeProxy + let commands: Commands + + init(proxy: FakeProxy = FakeProxy()) { + let model = KeyboardModel() + model.proxyOverride = proxy + model.hasFullAccess = true + Self.listening(true) + model.isListening = true + let commands = Commands() + model.transport = { commands.sent.append($0) } + self.model = model + self.proxy = proxy + self.commands = commands + } + + /// The app listening — an open window and a fresh heartbeat — or not. + static func listening(_ on: Bool) { + let now = Date() + SharedStore.listeningUntil = on ? now.addingTimeInterval(600) : nil + SharedStore.appSeenAt = on ? now : nil + } +} + +/// A phase as the app would publish it, now. +@MainActor +func snapshot(_ state: PhaseSnapshot.State, level: Double = 0) -> PhaseSnapshot { + PhaseSnapshot(state: state, message: nil, level: level, at: Date()) +} diff --git a/App/BlurtiOS/BlurtiOSTests/VoiceElementTests.swift b/App/BlurtiOS/BlurtiOSTests/VoiceElementTests.swift new file mode 100644 index 00000000..e84b8500 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/VoiceElementTests.swift @@ -0,0 +1,67 @@ +import Foundation +import Testing + +@testable import BlurtiOS +@testable import BlurtiOSCore + +@Suite("Voice element") +struct VoiceElementTests { + @Test("the kind is read from -BlurtGalleryVoice, and only a, b or c") + func kind() { + #expect(VoiceElementKind.parse(["app"]) == nil) + #expect(VoiceElementKind.parse(["app", "-BlurtGalleryVoice"]) == nil) + #expect(VoiceElementKind.parse(["app", "-BlurtGalleryVoice", "a"]) == .grille) + #expect(VoiceElementKind.parse(["app", "-BlurtGalleryVoice", "b"]) == .ribs) + #expect(VoiceElementKind.parse(["app", "-BlurtGalleryVoice", "c"]) == .streak) + #expect(VoiceElementKind.parse(["app", "-BlurtGalleryVoice", "d"]) == nil) + } + + @Test("the landing glint runs 0 → 1 over motion.landing and is gone after") + func landing() { + let landedAt = Date() + #expect(VoiceClock.landing(landedAt: nil, now: landedAt) == nil) + #expect(VoiceClock.landing(landedAt: landedAt, now: landedAt.addingTimeInterval(-0.1)) == nil) + #expect(VoiceClock.landing(landedAt: landedAt, now: landedAt) == 0) + let half = VoiceClock.landing(landedAt: landedAt, now: landedAt.addingTimeInterval(DesignTokens.Motion.landing / 2)) + #expect(half != nil && abs((half ?? 0) - 0.5) < 0.001) + #expect( + VoiceClock.landing(landedAt: landedAt, now: landedAt.addingTimeInterval(DesignTokens.Motion.landing)) == nil) + } + + @Test("the sheen runs on the rest period, faster while working, and the facet hash is stable") + func clocks() { + #expect(VoiceClock.sheen(time: 0, working: false) == 0) + #expect(abs(VoiceClock.sheen(time: DesignTokens.Motion.sheen / 2, working: false) - 0.5) < 0.001) + #expect(abs(VoiceClock.sheen(time: DesignTokens.Motion.sheenWorking / 4, working: true) - 0.25) < 0.001) + #expect(VoiceClock.hash(3, 7) == VoiceClock.hash(3, 7)) + #expect(VoiceClock.hash(3, 7) != VoiceClock.hash(4, 7)) + #expect((0..<64).allSatisfy { (0...1).contains(VoiceClock.hash($0, 1)) }) + } + + @Test("the bar pairs the + with what shows: the grille's own width, the ribs asleep or awake, the streak's line") + func visibleWidth() { + let metrics = DesignTokens.Metrics.self + #expect(VoiceElementKind.grille.visibleWidth(slot: .bar, recording: false) == metrics.grilleBarWidth) + #expect(VoiceElementKind.grille.visibleWidth(slot: .bar, recording: true) == metrics.grilleBarWidth) + #expect(VoiceElementKind.grille.visibleWidth(slot: .panel, recording: false) == metrics.grillePanelWidth) + #expect(VoiceElementKind.ribs.visibleWidth(slot: .bar, recording: false) == metrics.ribsRestWidth) + #expect(VoiceElementKind.ribs.visibleWidth(slot: .bar, recording: true) == metrics.voiceBarWidth) + #expect(VoiceElementKind.streak.visibleWidth(slot: .bar, recording: false) == metrics.voiceBarWidth) + // Every candidate fits its slot's box. + for kind in VoiceElementKind.allCases { + for slot in VoiceSlot.allCases { + #expect(kind.visibleWidth(slot: slot, recording: false) <= slot.box.width) + #expect(kind.visibleWidth(slot: slot, recording: true) <= slot.box.width) + } + } + } + + @Test("every slot has a box, from the tokens") + func slots() { + #expect(VoiceSlot.bar.box.width == DesignTokens.Metrics.voiceBarWidth) + #expect(VoiceSlot.panel.box.height == DesignTokens.Metrics.voicePanelHeight) + #expect( + VoiceSlot.home.box + == CGSize(width: DesignTokens.Metrics.voiceHomeWidth, height: DesignTokens.Metrics.voiceHomeHeight)) + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/VoiceRowLayoutTests.swift b/App/BlurtiOS/BlurtiOSTests/VoiceRowLayoutTests.swift new file mode 100644 index 00000000..14b63e35 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/VoiceRowLayoutTests.swift @@ -0,0 +1,132 @@ +import CoreFoundation +import Testing + +@testable import BlurtiOS + +/// The voice row's placement rules (`VoiceRowLayout`), at the iPhone 18 Pro's +/// row of 389 pt unless a case needs another: the mic dead centre or at an +/// edge, the + hanging off it, the chip growing away from it, the pair +/// centring when a chip has no room, and nothing ever negative. +@Suite("Voice row layout") +struct VoiceRowLayoutTests { + private let row: CGFloat = 389 + private let mic: CGFloat = 64 + private let key: CGFloat = 43.333 + private let hit = DesignTokens.Metrics.glyphHit + private let clearance = DesignTokens.Metrics.voicebarAddtermClearance + + private func layout(_ side: VoiceRowLayout.Side, row: CGFloat? = nil) -> VoiceRowLayout { + VoiceRowLayout(rowWidth: row ?? self.row, side: side) + } + + @Test("centred: the mic's centre is the row's; the + hangs a clearance off its trailing edge, the × mirrors it") + func centred() { + let placed = layout(.center).place(micWidth: mic, minMicWidth: key, chip: false) + #expect(!placed.paired) + #expect(placed.micWidth == mic) + #expect(abs(placed.micX + mic / 2 - row / 2) < 0.001) + #expect(placed.addX == placed.micX + mic + clearance) + #expect(placed.addWidth == hit) + #expect(placed.addGrowsTrailing) + #expect(placed.cancelX == placed.micX - clearance - hit) + // The + and the × sit the same distance from the mic on either side. + let plusGap = placed.addX - (placed.micX + mic) + let cancelGap = placed.micX - ((placed.cancelX ?? 0) + hit) + #expect(abs(plusGap - cancelGap) < 0.001) + } + + @Test("centred with a chip: the chip takes everything from the mic to the row's edge, the mic stays put") + func centredChip() { + let plain = layout(.center).place(micWidth: mic, minMicWidth: key, chip: false) + let chip = layout(.center).place(micWidth: mic, minMicWidth: key, chip: true) + #expect(!chip.paired) + #expect(chip.micX == plain.micX) + #expect(chip.addX == plain.addX) + #expect(abs(chip.addX + chip.addWidth - row) < 0.001) + #expect(chip.addWidth >= DesignTokens.Metrics.addtermChipMinWidth) + } + + @Test("the panel's 160 pt grille still leaves a chip its minimum on the iPhone 18 Pro") + func panelChip() { + let placed = layout(.center).place(micWidth: 160, minMicWidth: key, chip: true) + #expect(!placed.paired) + #expect(placed.addWidth >= DesignTokens.Metrics.addtermChipMinWidth) + } + + @Test( + "the slim bar's row cannot hold a chip beside a centred mic: the pair centres, the chip takes what the mic leaves") + func slimBarPairs() { + // The row between the globe, delete and return at the full keyboard's + // widths (191.67), and without the globe (247.5). + for width in [191.667, 247.5] as [CGFloat] { + let placed = layout(.center, row: width).place(micWidth: mic, minMicWidth: key, chip: true) + #expect(placed.paired, "\(width)") + #expect(placed.micWidth == mic) + #expect(abs(placed.addWidth - (width - mic - clearance)) < 0.001) + #expect(placed.addWidth > 0) + } + // The + alone always fits beside the centred mic there. + let plus = layout(.center, row: 191.667).place(micWidth: mic, minMicWidth: key, chip: false) + #expect(!plus.paired) + #expect(plus.addX + plus.addWidth <= 191.667) + } + + @Test("the mic key is never narrower than a key nor wider than the row, and the + never leaves the row") + func clamps() { + let narrow = layout(.center).place(micWidth: 10, minMicWidth: key, chip: false) + #expect(narrow.micWidth == key) + let wide = layout(.center).place(micWidth: 1000, minMicWidth: key, chip: false) + #expect(wide.micWidth == row) + #expect(wide.micX == 0) + #expect(wide.addX + wide.addWidth <= row) + #expect(wide.cancelX == 0) + // A tiny row with a chip: still nothing negative. + let tiny = layout(.center, row: 60).place(micWidth: mic, minMicWidth: key, chip: true) + #expect(tiny.micWidth == 60) + #expect(tiny.addWidth == 0) + #expect(tiny.paired) + } + + @Test("at the leading edge the mic is at the edge and the + on its inner side; a chip grows to the far edge") + func leading() { + let plus = layout(.leading).place(micWidth: mic, minMicWidth: key, chip: false) + #expect(plus.micX == 0) + #expect(plus.addX == mic + clearance) + #expect(plus.addWidth == hit) + #expect(plus.addGrowsTrailing) + #expect(plus.cancelX == nil) + let chip = layout(.leading).place(micWidth: mic, minMicWidth: key, chip: true) + #expect(abs(chip.addX + chip.addWidth - row) < 0.001) + } + + @Test("at the trailing edge the mic is at the edge and the + on its inner side, growing towards the leading edge") + func trailing() { + let plus = layout(.trailing).place(micWidth: mic, minMicWidth: key, chip: false) + #expect(abs(plus.micX + mic - row) < 0.001) + #expect(plus.addWidth == hit) + #expect(abs(plus.addX + hit + clearance - plus.micX) < 0.001) + #expect(!plus.addGrowsTrailing) + #expect(plus.cancelX == nil) + let chip = layout(.trailing).place(micWidth: mic, minMicWidth: key, chip: true) + #expect(chip.addX == 0) + #expect(abs(chip.addWidth + clearance - chip.micX) < 0.001) + } + + @Test("the row's side follows the alignment Settings chose, flipped for a right-to-left language") + func sides() { + #expect(VoiceRow.side(.center, in: .leftToRight) == .center) + #expect(VoiceRow.side(.left, in: .leftToRight) == .leading) + #expect(VoiceRow.side(.right, in: .leftToRight) == .trailing) + #expect(VoiceRow.side(.left, in: .rightToLeft) == .trailing) + #expect(VoiceRow.side(.right, in: .rightToLeft) == .leading) + } + + @Test("the full keyboard's symbol pages: five punctuation keys fill what seven letters do") + func punctuation() { + let geometry = KeyGeometry(rowWidth: row) + let letters = 7 * geometry.letterWidth + 6 * geometry.gap + let punctuation = 5 * geometry.punctuationWidth + 4 * geometry.gap + #expect(abs(letters - punctuation) < 0.001) + #expect(geometry.punctuationWidth > geometry.letterWidth) + } +} diff --git a/App/BlurtiOS/BlurtiOSTests/VoiceStateTests.swift b/App/BlurtiOS/BlurtiOSTests/VoiceStateTests.swift new file mode 100644 index 00000000..df1eab48 --- /dev/null +++ b/App/BlurtiOS/BlurtiOSTests/VoiceStateTests.swift @@ -0,0 +1,64 @@ +import BlurtEngine +import Foundation +import Testing + +@testable import BlurtiOSCore + +@Suite("Voice state") +struct VoiceStateTests { + private func snapshot(_ state: PhaseSnapshot.State, level: Double = 0, message: String? = nil) -> PhaseSnapshot { + PhaseSnapshot(state: state, message: message, level: level, at: Date()) + } + + @Test("ready and idle: the circle, still, nothing else") + func idle() { + let state = VoiceState(snapshot: snapshot(.idle), isReady: true) + #expect(state.ring == .still) + #expect(state.glyph == nil) + #expect(!state.isRecording && !state.isWorking && !state.isNotice && !state.dimmed && !state.canCancel) + #expect(state.accessibilityLabel == "Dictate") + #expect(state.accessibilityValue == "") + } + + @Test("the mic coming up and the words being made sweep the ring; recording is the meter, ring still") + func inFlight() { + let connecting = VoiceState(snapshot: snapshot(.connecting), isReady: true) + #expect(connecting.ring == .sweeping && connecting.isWorking && connecting.canCancel && !connecting.isRecording) + let recording = VoiceState(snapshot: snapshot(.recording, level: 0.62), isReady: true) + #expect(recording.ring == .still && recording.isRecording && recording.isWorking && recording.level == 0.62) + #expect(recording.accessibilityLabel == "Stop dictation") + let processing = VoiceState(snapshot: snapshot(.processing), isReady: true) + #expect(processing.ring == .sweeping && processing.isWorking && !processing.isRecording) + } + + @Test("the notices: green ring for the words landing, a clipboard when copied, orange and an exclamation on error") + func notices() { + let pasted = VoiceState(snapshot: snapshot(.pasted), isReady: true) + #expect(pasted.ring == .solid(.ok) && pasted.glyph == nil && pasted.isNotice && !pasted.canCancel) + let copied = VoiceState(snapshot: snapshot(.copied), isReady: true) + #expect(copied.ring == .solid(.ok) && copied.glyph == .clipboard) + let error = VoiceState(snapshot: snapshot(.error, message: "The microphone didn't start."), isReady: true) + #expect(error.ring == .solid(.error) && error.glyph == .exclamation) + #expect(error.accessibilityValue == "The microphone didn't start.") + #expect(VoiceState(snapshot: snapshot(.error), isReady: true).accessibilityValue == "Dictation failed.") + } + + @Test("not ready: dimmed, no meter, no glyph, no sweep; a tap starts Blurt") + func notReady() { + for phase in [PhaseSnapshot.State.idle, .connecting, .recording, .processing, .copied] { + let state = VoiceState(snapshot: snapshot(phase, level: 0.5), isReady: false) + #expect(state.dimmed && !state.isRecording && !state.isWorking && state.glyph == nil) + #expect(state.ring != .sweeping) + #expect(state.accessibilityLabel == "Start Blurt") + } + } + + @Test("the app's overlay state maps onto the same phases; no target is copied") + func overlay() { + #expect(VoiceState(overlay: .noTarget, windowOpen: true, level: 0).phase == .copied) + #expect(VoiceState(overlay: .error(message: "x"), windowOpen: true, level: 0).message == "x") + #expect(VoiceState(overlay: .recording, windowOpen: true, level: 0.4).level == 0.4) + #expect(VoiceState(overlay: .idle, windowOpen: false, level: 0).dimmed) + #expect(VoiceState(overlay: .processing, windowOpen: true, level: 0).ring == .sweeping) + } +} diff --git a/App/BlurtiOS/DESIGN.md b/App/BlurtiOS/DESIGN.md new file mode 100644 index 00000000..d09a961a --- /dev/null +++ b/App/BlurtiOS/DESIGN.md @@ -0,0 +1,758 @@ +# Blurt for iPhone — design + +The iPhone app and its keyboard are Blurt's brand on a phone: Cam Neri's +"warped futurism" as it appears on assemblyai.com/blurt — an ink ground +under film grain, a warm paper one in the light, serif headlines, uppercase +mono labels, green as the one signal, rectangular corners — at the iPhone +keyboard's own measured geometry, so nothing is unfamiliar under the fingers. +Nothing here is decoration: every number and colour is a token, every rule +has a test, and every state can be captured from a launch argument. + +Sources of truth: the brand design system +(`https://www.assemblyai.com/brand-design-system/assemblyai-brand-design-system.md`) +and the Blurt landing page for the look; `Design/apple-geometry.json` for the +keys; `Design/tokens.json` for every value the views draw with. From the +tokens, `scripts/design-tokens.swift` writes `Shared/DesignTokens.swift` +(what the views read), the tables in this file marked `tokens:begin` / +`tokens:end`, and the app's asset-catalog colour sets. Edit the JSON, run `scripts/design-sync.sh`, commit; `check.sh` +runs `design-sync.sh --check` and fails on drift or on a design literal in a +view (`// literal-ok: ` for a genuine derivation). + +## Reproducing it + +Every layout and state renders inside the app, from a launch argument, so a +screenshot never depends on tapping through Notes: + +```bash +BLURT_LAUNCH_ARGS="-BlurtGallery panel idle,recording,processing dark" scripts/ios-sim.sh --screenshot panel.png +BLURT_LAUNCH_ARGS="-BlurtGallery full idle,symbols,send light" scripts/ios-sim.sh --screenshot full.png +BLURT_LAUNCH_ARGS="-BlurtSettings" scripts/ios-sim.sh --screenshot settings.png +``` + +Layouts: `slimBar`, `panel`, `full`. Faces: `light`, `dark` (the older theme +words still land on one). States: `off` (no Full Access), `start` (the app +isn't listening), the pipeline's `idle`, `connecting`, `recording` (level +0.62), `processing`, `pasted`, `copied`, `error`; `landed` (the glint, timed +for a 3 s screenshot); `live` (a whole dictation walked on a clock, over and +over, for watching or recording); and the keyboard's own `keys` (the panel +flipped to its keyboard page), `symbols`, `term` (the key-term field +mid-typing), `send` (a return key with a label), `saved` (the + just after a +term was saved), `selected` (a word highlighted in the text: the + is the chip +holding it) and `known` (the same, for a word already in the key terms). +`-BlurtGalleryVoice a|b|c` picks the mic concept and `-BlurtGalleryAlign +left|center|right` the mic's side (One hand, below) +(below). The gallery is `BlurtiOS/Sources/KeyboardGalleryView.swift`, debug +builds only; the keyboard's sources are compiled into the app for it. + +For the design loop a capture has to be the same pixels every time, and it +has to be the keyboard's pixels alone: `-BlurtGalleryStill` holds every +motion at time zero (what Reduce Motion does) and `-BlurtGalleryBare` draws +the first row by itself inside a 2 pt `#FF00FF` registration border. +`-BlurtVoice [light|dark]` renders one voice +element the same way. The scripts on top: + +```bash +scripts/design-capture.sh --layout panel --states idle,recording,error,term --themes light,dark [--voice a] + # builds once, captures each still, crops it → .build/design/captures/--[-]@3x.png +scripts/ios-record.sh --layout panel --theme dark [--voice a] + # one 11.7 s loop of the `live` walk, cropped +swift scripts/design-diff.swift diff a.png b.png --out triptych.png --json metrics.json + # sRGB, AE and RMSE; `sheet` makes a contact sheet; `measure` reads a keyboard's caps off a screenshot +scripts/apple-geometry.sh # measures the iPhone's own keyboard → Design/apple-geometry.json +scripts/ios-keyboard-shot.sh --face dark --voice b --layout panel + # the real keyboard — the extension in its own process — over a field in the app: what the gallery + # cannot vouch for (a canvas that renders asynchronously never presents in an extension) +scripts/design-sync.sh [--check] # tokens.json → DesignTokens.swift, these tables, the colour sets +``` + +The capture device is the iPhone 18 Pro (402 pt → 1206 px at 3×), pinned in +`scripts/ios-sim.sh`; a crop that is not the layout's height fails rather +than being resampled. + +## The two faces + +The keyboard follows the field it is typing into, as the iPhone's does: +**dark** — ink under the brand's film grain, the website's ground — when the +host field's `keyboardAppearance` is dark, else the app's own appearance; +**light** — warm paper — otherwise. Both are the brand; neither is Apple's +grey. A face changes how the keyboard looks and never how it types. The +curated themes come back later as value sets through the same +`KeyboardPalette` shape (`all`, `resolve(_:dark:)`); a retired id resolves to +the brand's face. Ten roles each (`BlurtKeyboard/Sources/KeyboardPalette.swift`): + + + +| Face | Surface | Key | Modifier | Legend | Secondary | Signal | Pop-up | Field | Field border | Notice | Key in container | Modifier in container | Host material | +| ------- | --------- | --------- | --------- | --------- | --------- | --------- | --------- | --------- | ------------ | --------- | ---------------- | --------------------- | ------------- | +| `light` | `#ECEBE5` | `#FFFFFF` | `#DAD7CB` | `#1D1B16` | `#777673` | `#01762F` | `#FFFFFF` | `#FFFFFF` | `#C7C3B2` | `#E67F36` | `#FFFFFF` | `#DAD7CB` | `#E2E3E8` | +| `dark` | `#1D1B16` | `#33302A` | `#26231E` | `#F5F3EB` | `#A5A4A2` | `#67AD82` | `#33302A` | `#33302A` | `#3A362F` | `#E67F36` | `#575757` | `#474747` | `#373737` | + + + +**The surface is the host's.** iOS 26 wraps a third-party keyboard in the host +app's own rounded keyboard material, inset above and below by about 16 pt, +and the host's traits arrive on their own schedule: the face is read in +`viewWillAppear` and again in `viewIsAppearing`, never from a trait-change +registration — that callback runs inside the host's layout pass, and a model +change made there left the mic element's drawing uncommitted in Messages +(the grille never drew, every time; the probe app never fires it), +and nothing the keyboard draws reaches the inset (Apple: working as +intended). A painted surface therefore shows a band of the host's grey above +it, in every Liquid Glass app. So the surface is **clear** by default +(`surface/paint` 0): the host's material is the surface, seamlessly, and the +keys take the face's `container-key` / `container-key-modifier` colours, +which read on the system's light and dark material (measured from the iPhone +keyboard: `#575757` keys on the dark one). The brand lives in the keys, the +labels and the voice element. `surface/paint` 1 paints the face's surface +and grain under the keys — and accepts the band. The keyboard's height is a +constraint at required − 1, activated from `updateViewConstraints` once the +view has appeared; activated earlier it fights the host's first layout. +`scripts/ios-keyboard-shot.sh` and the `MessagesKeyboardShot` UI test +capture the real keyboard in the app and in Messages. + +The app's screens follow the phone's appearance through the asset catalog's +colour sets (generated from the `keyboard.app/*` tokens): `Page`, `Text`, +`Muted`, `CardFill`, `CardBorder`, `AccentColor`, `CTA`, `CTAText`. + +## Apple's geometry, measured + +The key tokens were assumed until 2026-09-29; now they are measured. +`scripts/apple-geometry.sh` runs the `BlurtiOSProbe` UI test (XCTest, the one +allowed use of it; its own scheme, never in the app's test run or the +periphery scan): the app launches on `-BlurtProbeField light|dark`, a bare +text field takes the system keyboard, the test reads every key's frame and +screenshots both faces. XCUITest reports touch cells, which tile a row with +no gaps, so `design-diff.swift measure` reads the caps themselves — widths, +gaps, height, the corner radius fitted to the top-left inset — off the +screenshot. The result is `Design/apple-geometry.json`; `AppleGeometryTests` +ties the tokens to it, `DesignTokensTests` the geometry to the tokens. + +iPhone 18 Pro, iOS 27, 402 pt: caps 33.5 wide and 43 tall, 6 between, 6.5 at +the sides, 11 between rows, corners at 8; shift and delete 45.58 with a 13.67 +gap to the letters; 123 and the globe 43.33, return two of those and a gap +(92.67); a 52 pt band above the keys (the suggestion bar's — where the voice +row goes: 8 + 44) and 13 under them; 270 in all. The dark face is identical. +The letter legend estimates at 24 pt from the l glyph. `KeyGeometry` +(`BlurtKeyboard/Sources/KeyGeometry.swift`) is the arithmetic between the +tokens: ten letters and nine gaps across the row, the side keys taking what +seven letters and the side gaps leave, the bottom row's widths scaling from +the reference width; the full keyboard draws from it and carries no +arithmetic of its own. iOS draws its own globe-and-mic bar under a custom +keyboard on a Face ID phone (58 pt), so the globe appears in ours only when +`needsInputModeSwitchKey` says so. + +## Keys + +Flat, one colour, at the iPhone's corner (8); no edge, no drop, no gloss. A +key brightens 15 % while pressed (0.08 s) rather than dimming — a gesture +state, so a touch the system takes away (a call, the keyboard dismissed +mid-press) unlights it too; the letter pop-up carries the one shadow on the +keyboard, a whisper (12 %, radius 8), being the one thing that floats, and +rises `popup/offset` from the key's top so it overlaps the key and never +leaves the keyboard on the top row. A key acts on release when the finger +barely moved or ended on the key within `tapSlop` (8) of its edges — a thumb +that rolls still types, a swipe that left the key does not +(`KeyboardInteraction.isTap`, tested). Every layout's keys are `KeyGeometry` +widths: 123, globe and delete 43.33, return 92.67, so a key is the same size +on every page. 123 opens the numbers and symbols, #+= the second symbol page +(brackets, currency), 123 there the first, ABC the letters; on the symbol +pages the third row's five punctuation keys fill what seven letters do. + +Type: the letters are SF Pro at the iPhone's size, and the glyph keys — shift, +delete, globe, the +, × and ✓ — SF Symbols, so the keyboard feels native +under the fingers. Every **word** on a key — `123`, `ABC`, `#+=`, `space`, +`return` and the field's own `send` / `go` / `done` / `search` / `next` / `join` — +is the brand's eyebrow: the system monospaced face, medium, 12 pt, uppercase, +1.2 +tracking, in the face's secondary legend, a step quieter than a letter. Fixed +point sizes throughout, as the system keyboard's; no Dynamic Type. + + + +| Token | Value | Swift | Use | +| -------------------- | ---------- | ----------------------------- | --------------------------------------------------------------------------------- | +| `ratio/voice-glyph` | `0.34` | `Typography.ratioVoiceGlyph` | the clipboard and exclamation glyphs, as a fraction of the voice element's height | +| `size/body` | `16` | `Typography.sizeBody` | the app's body text | +| `size/caption` | `14` | `Typography.sizeCaption` | the app's small text | +| `size/cta` | `14` | `Typography.sizeCta` | a mono button label in the app (E2) | +| `size/eyebrow` | `12` | `Typography.sizeEyebrow` | a mono eyebrow in the app (E1) | +| `size/glyph` | `17` | `Typography.sizeGlyph` | the +, × and ✓ | +| `size/label` | `12` | `Typography.sizeLabel` | the mono word labels on keys: 123, ABC, #+=, space, the return label | +| `size/legend` | `16` | `Typography.sizeLegend` | every other key | +| `size/letter` | `24` | `Typography.sizeLetter` | letter keys: the iPhone's (estimated from the l glyph, apple-geometry.json) | +| `size/popup` | `32` | `Typography.sizePopup` | the letter pop-up | +| `size/term` | `17` | `Typography.sizeTerm` | the key-term field | +| `size/title` | `34` | `Typography.sizeTitle` | the app's serif headline | +| `tracking/cta` | `1.4` | `Typography.trackingCta` | uppercase mono at 14: the brand's E2 tracking | +| `tracking/eyebrow` | `1.2` | `Typography.trackingEyebrow` | uppercase mono at 12: the brand's E1 tracking | +| `weight/glyph` | `medium` | `Typography.weightGlyph` | — | +| `weight/label` | `medium` | `Typography.weightLabel` | — | +| `weight/legend` | `medium` | `Typography.weightLegend` | — | +| `weight/letter` | `regular` | `Typography.weightLetter` | — | +| `weight/popup` | `regular` | `Typography.weightPopup` | — | +| `weight/term` | `regular` | `Typography.weightTerm` | — | +| `weight/voice-glyph` | `semibold` | `Typography.weightVoiceGlyph` | — | + + + +No font is bundled: every face is the system's, reached only through +`BlurtType` by role — `mono` (the system's monospaced design) for eyebrows, +CTAs and the keyboard's word labels, `heading` (its serif design) for the +app's headlines, `body` (the default face) for body text — so a call site names +the role and the faces can change in one place. + +## The voice element + +The mic control is the one part of the keyboard that says what is +happening, and it says it without a word: the orb is gone. Three concepts +are built side by side behind one seam, `VoiceElementView` +(`BlurtKeyboard/Sources/VoiceElement.swift`), so they can be judged on sight +from the same states, the same level and the same landing moment; the one +that ships stays and the others go. Every candidate draws from +`VoiceElementInputs` — the `VoiceState`, the level inside it, when the words +landed, whether motion is allowed, the face, the slot (bar, panel, home) — +and nothing else. `VoiceClock` holds the clocks they share: the landing +glint (0 → 1 over `motion.landing`), the sheen (one pass per `motion.sheen` +at rest, per `sheen-working` while something is happening), and a facet hash +that makes the glints random to the eye and the same in every capture. + +| Kind | At rest | Recording | Working | Landed | Error | +| ------------------------------ | ---------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------- | ----------------------------------------------- | +| **a — Grille** (`VoiceGrille`) | a dot lattice (2 pt dots, 4 pt pitch) in a 4 pt-corner rectangle, a diagonal sheen crossing it | the lattice is the meter: columns light bottom-up in green with the engine's envelope, quantised to whole dots; facets on the peaks flash white | one lit dot runs the perimeter on the ring's cadence | a thin white cross glint with a faint prismatic fringe sweeps across once | the lattice orange, an exclamation mark over it | +| **b — Ribs** (`VoiceRibs`) | a short block of the wave's own bars under a cosine envelope, the sheen passing rib to rib | the ribs are the wave: they widen to the box and follow the voice | the ribs breathe with the sheen | a white glint runs the ribs | the ribs orange | +| **c — Streak** (`VoiceStreak`) | a hairline the width of the box with a bright point at its centre | the point blooms into a light streak whose reach follows the level, green along it, white at the heart, violet at the tips | a short streak sweeps the line back and forth | the cross's vertical arm flashes through the point | the line orange | + +**Temporary, while one is chosen:** Settings → Keyboard → Mic switches the +concept in use (`SharedStore.voiceElementKind`, read by the keyboard on every +appearance and by the home screen live). The switch, the two losers and +`-BlurtGalleryVoice` go together once the direction is picked. + +What is common: the box (`voice/*`), dimming to `opacity/off` when Blurt +isn't ready (a tap opens the app), the two glyphs that need saying (a +clipboard when the words went there, an exclamation mark on a failure) drawn +by `MicControl` over the element, every change fading over `state-fade`, and +the press answering at once. **The grille is drawn as plain SwiftUI paths, +never a `Canvas`**: a Canvas has its own render surface, and in a keyboard +extension that surface has failed to present three times (asynchronous +first; then in Messages, whose host swaps scenes as the keyboard appears) — +the key was there, the dots were not. The frame is a handful of `Path` +layers, one per colour, with the sheen's lift quantised to four steps. The +two candidates still draw with a Canvas and inherit the risk; they go, or get +the same treatment, when one is picked. The probes check the element's +_pixels_ (`MicPixels`, the share of the key's box that is not the material), +not only its accessibility frame, on appearance and after every return. Under Reduce Motion the sheen and the glints +hold still; the level still drives the meter. The home screen draws the same +element at home size in the app's face. + +| State | Element | Haptic | +| ----------------------- | ---------------------------------------------------------------------- | ---------------------------- | +| no Full Access | the note "Allow Full Access in Settings" is the key; a tap opens Blurt | — | +| app not listening | the element at `opacity/off`; a tap opens Blurt | — | +| idle | at rest, the sheen | — | +| connecting / processing | working; the cancel × only while the app is there to stop it | — / light impact on the stop | +| recording | the meter | medium impact | +| pasted | the landing glint, the element in green for 1.2 s | success | +| copied | the same, with the clipboard glyph, 2 s | success | +| error | orange, the exclamation mark, 2 s; the message on VoiceOver | error | + +The gesture is `MicPressSequencer` (`KeyboardInteraction.swift`), tested: a +press waits 90 ms so a swipe that starts on the key never starts a dictation +it must then cancel; a tap quicker than that is still a tap; a swipe past +24 pt cancels the timer, or undoes a press that went out; a touch the system +takes away before the timer fires presses nothing. Finger down and up drive +the engine's `DictationKeyGate`, so a tap latches and a hold is push-to-talk +exactly as on the Mac. A release made while the mic is still coming up (the +engine drops one before it records) is held and sent on the first recording +phase, so a short hold over a slow mic still stops. Every phase the app +publishes carries the id of the keyboard command it last took +(`PhaseSnapshot.command`), and the keyboard settles its gate only for a phase +that answers the press it is waiting on — the previous dictation's notice +landing a moment after a new press used to reset the gate and turn the next +tap into a second press instead of the stop. An in-flight phase with +the app gone (killed, the mic taken) reads as idle, a notice is never +replayed once its dwell ran out, and a keyboard that comes up without Full +Access starts from idle with the gate reset. `KeyboardModelFuzzTests` drives +the model through ten thousand random event sequences and checks these +invariants after every one. + +## Layouts and their heights (`KeyboardLayout.height`) + +Side margins 6.5; top 8; under a row of keys 13 (the iPhone's), under the +slim bar 8. + +| Layout | Rows | Height | +| --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------ | +| `slimBar` | globe (when the phone draws none) · voice bar · delete · return, at the full keyboard's widths and gap, the keys as tall as the row | 60 | +| `panel` | the voice element, dead centre between the keyboard's top edge and the keys — or at an edge for one hand (Settings, below) — with the + beside it on its centre line and the cancel × mirroring the + on the other side while something is in flight (× · mic · +) · globe · space · delete · return at the full keyboard's widths (43.33 / 92.67), so nothing changes size when the carousel flips. A two-page carousel: a horizontal swipe flips between the panel and the full keyboard, and it is back to the element each time the keyboard appears. The pages are a strip one page wide (`PanelCarousel`): the other page is set down a page away on the side the finger moved towards and the strip slides, so the two never cross whichever way the flips come; a flip back mid-slide retreats (`PanelCarouselState`, tested). | 216, or 270 when flipped | +| `full` | voice row 44, then the first key row right under it as the iPhone's sits under its suggestions · three letter rows 43 · 123 / globe / space / return | 270 | + +The voice row (`VoiceBar`) is where the system keyboard puts its suggestion +bar. One rule places it everywhere (`VoiceRowLayout`, tested; `VoiceRow` +draws it in the bar and the panel): the mic key is what is placed — dead +centre of the row, or at the edge Settings chose — and the add key (the +, +or the chip for a highlighted word) hangs off it a clearance away on its +centre line: on its trailing side in the middle, on its inner side at an +edge, so the mic is always the thing at the edge. A chip grows away from the +mic as far as the row allows; when that is less than `addterm/chip-min-width` +beside a centred mic (the slim bar's row), the mic and the chip centre as a +pair instead, the mic moving over rather than the chip collapsing onto it. +The mic key is as wide as the element _shows_ (`VoiceElementKind.visibleWidth`, +tested: the grille's own width, the ribs asleep or the wave awake, the +streak's line — never the slot's box, which would put the + off at the edge; +never narrower than a key, never wider than the row) and as tall as the row, +so the whole row under the element is the key. The +, × and ✓ are one glyph +key (`GlyphKey`): a `size/glyph` symbol in a `glyph/hit` square, the HIG's +44 pt. Or the row is the key-term field. A swipe flips the panel past +48 pt sideways and more sideways than up or down by 1.5×; a key acts on +release and only under 12 pt of travel (`KeyboardInteraction`, tested). The +swipe is recognised on the input view itself (`KeyboardViewController`), not +in SwiftUI, and the keyboard paints a floor under everything at 1 % of the +host material's own colour (`material/*`, so nothing shows; 1 % black read as +a shade darker than the border around it): +the host hands a keyboard only the touches that land on pixels it drew, so +with the surface clear a swipe that started on the panel's empty space never +reached the keyboard at all (`scripts/ios-keyboard-flows.sh` swipes from the +gap, the mic, the space key and the letters, both ways). + +## Quick-add key term + +The most-asked-for thing: Blurt mishears a niche word, and the fix has to take +seconds, inside the app you're typing in, never a trip to Settings. Two ways +in, both from the small **+** beside the element, on its centre line, in +every layout. + +**Highlight a word, tap once.** Select a word (or a short phrase) in your text +and the + grows into a chip — the field's shape, a rectangle at 8 pt with a +hairline — holding the word, with a green **+** before it (`AddTermKey`, +`KeyboardModel.selectedTerm`). One tap adds the word as it stands to Blurt's +key terms; the text is not touched, a success haptic, and the chip shows ✓ +for 1.2 s — then gives way to the + even though the host may keep the word +highlighted for as long as you leave it (a chip that stayed with the +highlight read as stuck; `dismissedSelection`, tested). Only that word's +chip: another word highlighted within the 1.2 s keeps its own. A word already in the +list shows a quiet ✓ from the start, so selecting anything tells you whether +Blurt has it; a tap on that ✓ puts the + back. Highlight another word and +the chip is back for it. The selection is read off the host field on every +change (`selectionDidChange`, `textDidChange`); what counts as a word is +trimmed, one line, at most 48 characters (`termCandidate`, tested), and only +with Full Access, since the list lives in the App Group. The chip grows away +from the mic to the row's edge (`VoiceRowLayout`, above). **Hold the chip** +(0.6 s, the system's long press, so a touch the system takes away never +opens it) to open the field with the word in it instead — for the case +below. + +**Type it.** Tap the + with nothing selected (or hold the chip) and the bar +becomes a field — × · what you type · ✓ — pre-filled with the selection if +there was one, and the keys type into it (the panel and slim bar flip to the +letter keys for it). The field takes the mic key's row, so a dictation in +flight is closed first rather than running on with no key to stop it: from +the plain + a recording is released and the words still land; from a +highlighted word it is cancelled, since a result replaces the selection and +would land over the very word being fixed (tested). Shift is on for the first letter and after a space. Type the right +spelling; return or ✓ saves. The term goes into Blurt's key terms +(`BlurtKeyTerms` in the App Group, `KeyTermList` rules: trimmed, deduplicated +case-insensitively) and rides `keyterms_prompt` on the very next dictation. If +it began as a selection and you changed it, the misheard word in your text is +replaced with what you typed — only while that highlight still stands where +it was (a host can report a selection a moment after the cursor moved, and +inserting then would put the word in twice). A success haptic, and the + +shows a check for 1.2 s. + +× or an empty save leaves the mode, and so does the keyboard going away. A +hardware keyboard types past the on-screen keys into the app's field; while +the term field is open a short run that appears there right after where the +cursor was is moved into the term and taken back out of the field — one +character at a time, as a key types; a block (a paste, the system's own +dictation under the keyboard) is never typing and stays — and never when the +field opened over a highlighted word, since collapsing that highlight grows +the text before the cursor by exactly the word (tested). The +list itself is edited in Settings → Transcription, which shows the count +against the request's cap of 100 terms. + +## Sharing key terms + +Settings → Transcription → **Share key terms…** puts the list in the share +sheet as a `.blurtterms` file (JSON — `name`, `from`, `terms`; a declared +document type, `dev.alex.blurt.terms`) with the words as the message text. A +tap on the file in Messages opens Blurt with "Add N key terms?": every term +ticked, untick any, Add merges the rest without duplicates. A +`blurt://terms?add=a,b,c&name=…&from=…` link does the same (`TermPack`, +`ImportTermsView`). A pack is refused over 64 KB, past 100 terms or with a +term, name or sender over 80 characters. + +## Adapting to the phone's own keyboard settings + +Nothing to configure: key clicks play only if the user has keyboard clicks +on; the return key says what the field asks — send, search, go, done, next, +join — and `return` otherwise; a number, decimal or phone field opens on +the symbols page; capitalisation follows the field's `autocapitalizationType`; +the letter rows follow the phone's first language (AZERTY for French, QWERTZ +for German, Czech, Slovak and Hungarian, QWERTY otherwise); the face follows +the field's `keyboardAppearance`. + +## Hands-free + +On by default, with a toggle in Settings: a dictation starts the moment the +keyboard comes up in a text field — the same synthetic tap through the +engine's gate, so it latches and the next tap of the element stops it. Only +when the app is listening and nothing is in flight; otherwise the element +sits dimmed and the first tap opens Blurt. When the keyboard leaves the +screen with a dictation it started, it releases a recording (the words still +land, on the clipboard if no keyboard is there) and cancels anything earlier. + +## One hand: the mic at an edge + +Settings → Keyboard, under the layout, the same three-segment shape with a +line beneath: **Left alignment · Default · Right alignment** (`MicAlignment`, +`micAlignment` in the App Group, read on every appearance like the layout). +Default is the middle, as everywhere above. At an edge the mic key is the +thing at the edge, in every layout (`VoiceRowLayout`): the element sits at +the edge as wide as it _shows_ (`visibleWidth`, so the grille is under the +thumb, not an empty half of its box), the add key beside it on its inner +side, and — in the panel — cancel in the far top corner, clear of them both, +where in the middle it mirrors the + on the mic's other side. +Left and right are the phone's sides: in a right-to-left language the +leading/trailing mapping flips so the mic stays under the thumb chosen +(`BlurtiOSCore/MicAlignment.swift` and `Shared/MicAlignment+Layout.swift`, tested). Gallery: `-BlurtGalleryAlign +left|center|right`; the real extension: `scripts/ios-keyboard-flows.sh +--align left|right` checks the mic is at that edge with the add key inside it. + +## Tokens (`Design/tokens.json` → `Shared/DesignTokens.swift`) + +The brand's colours, as the design system defines them and as the phone +uses them. `BlurtBrand` keeps a few of the Mac's names as aliases. + + + +| Token | Value | Swift | Use | +| ------------------------ | --------- | ---------------------------- | ------------------------------------------------------------------------------------------------------ | +| `apple/dark-key` | `#6B6B6B` | `Brand.appleDarkKey` | the iPhone keyboard's dark key | +| `apple/dark-modifier` | `#464646` | `Brand.appleDarkModifier` | the iPhone keyboard's dark modifier key | +| `apple/dark-surface` | `#2B2B2B` | `Brand.appleDarkSurface` | the iPhone keyboard's dark surface | +| `apple/light-modifier` | `#ADB3BC` | `Brand.appleLightModifier` | the iPhone keyboard's light modifier key | +| `apple/light-surface` | `#D1D5DB` | `Brand.appleLightSurface` | the iPhone keyboard's light surface | +| `black` | `#000000` | `Brand.black` | — | +| `card` | `#EBE8E8` | `Brand.card` | the warm card fill; the paper theme's surface | +| `card-border` | `#DEDBDB` | `Brand.cardBorder` | the card's hairline; the paper theme's modifier key | +| `cobolt` | `#3923C7` | `Brand.cobolt` | AssemblyAI's primary violet: the drop, never a key or a surface | +| `green/100` | `#CCE4D5` | `Brand.green100` | the design system's UI-and-code green fill (unused on the phone yet) | +| `green/200` | `#99C8AC` | `Brand.green200` | the design system's UI-and-code green highlight (unused on the phone yet) | +| `green/400` | `#67AD82` | `Brand.green400` | the brand hue lifted for dark chrome: the meter, the ring, the wave on ink | +| `green/700` | `#01762F` | `Brand.green700` | the wordmark green, light chrome (the app's accent in light) | +| `green/mist` | `#DBF5E6` | `Brand.greenMist` | the pale green the orb lightens into | +| `ink` | `#1D1B16` | `Brand.ink` | the brand ink: the Mac pill's body, the ink theme's surface | +| `ink/600` | `#3A362F` | `Brand.ink600` | the Mac's dark card border | +| `ink/700` | `#33302A` | `Brand.ink700` | an ordinary key on ink, one step up | +| `ink/800` | `#26231E` | `Brand.ink800` | a modifier key on ink; the Mac's dark card fill | +| `material/dark-key` | `#575757` | `Brand.materialDarkKey` | a key on the system's dark keyboard material (measured, iOS 27): the container case | +| `material/dark-modifier` | `#474747` | `Brand.materialDarkModifier` | a modifier key on the system's dark material | +| `material/dark-surface` | `#373737` | `Brand.materialDarkSurface` | the system's dark keyboard material (measured, iOS 27) | +| `material/light-surface` | `#E2E3E8` | `Brand.materialLightSurface` | the system's light keyboard material (measured, iOS 27): what the gallery stands under a clear surface | +| `neutral/300` | `#D2D1D0` | `Brand.neutral300` | black-100: the lightest warm grey | +| `neutral/500` | `#A5A4A2` | `Brand.neutral500` | black-200: mono labels on the dark face | +| `neutral/700` | `#777673` | `Brand.neutral700` | black-300: muted text and mono labels on the light face | +| `neutral/900` | `#4A4945` | `Brand.neutral900` | black-400: body text on the light face | +| `orange` | `#E67F36` | `Brand.orange` | the error word and ring, cancel, the Full Access note; never a red body | +| `paper/200` | `#ECEBE5` | `Brand.paper200` | neutral-100: the light face's surface | +| `paper/300` | `#DAD7CB` | `Brand.paper300` | neutral-200: a modifier key on the light face | +| `paper/400` | `#C7C3B2` | `Brand.paper400` | neutral-300: hairlines and field borders on the light face | +| `paper/page` | `#FDFCF8` | `Brand.paperPage` | the design system's page: the app's light ground | +| `paper/tint` | `#F5F3EB` | `Brand.paperTint` | the warm off-white one step in: light cards, dark legends | +| `violet/iris` | `#887BDD` | `Brand.violetIris` | the orb gradient's upper violet stop | +| `violet/lavender` | `#D7D3F4` | `Brand.violetLavender` | the orb's lightest violet: its top and the gradient's ends | +| `violet/periwinkle` | `#B0A7E9` | `Brand.violetPeriwinkle` | the orb's mid violet: the drop's halo | +| `warm-white` | `#F2EEE6` | `Brand.warmWhite` | key legends on ink: warm white on warm ink | +| `white` | `#FFFFFF` | `Brand.white` | — | + + + +The keyboard's semantic colours and the app's catalog colours: + + + +| Token | Value | Swift | Use | +| ----------------------- | ------------------------------------------ | ----------------------------- | ---------------------------------------------------------------- | +| `app/accent-dark` | `#67AD82` (`brand.green/400`) | `Keyboard.appAccentDark` | the catalog's AccentColor in dark | +| `app/accent-light` | `#01762F` (`brand.green/700`) | `Keyboard.appAccentLight` | the catalog's AccentColor in light | +| `app/card-border-dark` | `#3A362F` (`brand.ink/600`) | `Keyboard.appCardBorderDark` | the catalog's CardBorder in dark | +| `app/card-border-light` | `#C7C3B2` (`brand.paper/400`) | `Keyboard.appCardBorderLight` | the catalog's CardBorder in light | +| `app/card-fill-dark` | `#26231E` (`brand.ink/800`) | `Keyboard.appCardFillDark` | the catalog's CardFill in dark | +| `app/card-fill-light` | `#F5F3EB` (`brand.paper/tint`) | `Keyboard.appCardFillLight` | the catalog's CardFill in light | +| `app/cta-dark` | `#67AD82` (`brand.green/400`) | `Keyboard.appCtaDark` | the catalog's CTA in dark: the site's button green | +| `app/cta-light` | `#01762F` (`brand.green/700`) | `Keyboard.appCtaLight` | the catalog's CTA in light: the one green button | +| `app/cta-text-dark` | `#1D1B16` (`brand.ink`) | `Keyboard.appCtaTextDark` | the catalog's CTAText in dark: ink on green, as the site | +| `app/cta-text-light` | `#FFFFFF` (`brand.white`) | `Keyboard.appCtaTextLight` | the catalog's CTAText in light | +| `app/muted-dark` | `#A5A4A2` (`brand.neutral/500`) | `Keyboard.appMutedDark` | the catalog's Muted in dark | +| `app/muted-light` | `#777673` (`brand.neutral/700`) | `Keyboard.appMutedLight` | the catalog's Muted in light: eyebrows, captions, secondary text | +| `app/page-dark` | `#1D1B16` (`brand.ink`) | `Keyboard.appPageDark` | the catalog's Page in dark: ink, the website's ground | +| `app/page-light` | `#FDFCF8` (`brand.paper/page`) | `Keyboard.appPageLight` | the catalog's Page in light: the design system's page | +| `app/text-dark` | `#F5F3EB` (`brand.paper/tint`) | `Keyboard.appTextDark` | the catalog's Text in dark | +| `app/text-light` | `#1D1B16` (`brand.ink`) | `Keyboard.appTextLight` | the catalog's Text in light | +| `kb/cancel` | `#E67F36` (`brand.orange`) | `Keyboard.kbCancel` | the panel's cancel × | +| `kb/field` | `#33302A` (`themes.dark/field`) | `Keyboard.kbField` | the key-term field | +| `kb/field-border` | `#3A362F` (`themes.dark/field-border`) | `Keyboard.kbFieldBorder` | the field's hairline | +| `kb/full-access-note` | `#E67F36` (`brand.orange`) | `Keyboard.kbFullAccessNote` | the one line of words the keyboard ever shows | +| `kb/key` | `#33302A` (`themes.dark/key`) | `Keyboard.kbKey` | an ordinary key | +| `kb/key-modifier` | `#26231E` (`themes.dark/key-modifier`) | `Keyboard.kbKeyModifier` | shift, delete, globe, return, 123, cancel | +| `kb/legend` | `#F5F3EB` (`themes.dark/legend`) | `Keyboard.kbLegend` | key legends and bare glyphs | +| `kb/legend-secondary` | `#A5A4A2` (`themes.dark/legend-secondary`) | `Keyboard.kbLegendSecondary` | the mono word labels (123, ABC, return, space) and the + at rest | +| `kb/notice-error` | `#E67F36` (`brand.orange`) | `Keyboard.kbNoticeError` | the solid ring and glyph for an error | +| `kb/popup` | `#33302A` (`themes.dark/popup`) | `Keyboard.kbPopup` | the letter pop-up | +| `kb/signal` | `#67AD82` (`themes.dark/signal`) | `Keyboard.kbSignal` | the wave, the caret, the saved check | +| `kb/surface` | `#1D1B16` (`themes.dark/surface`) | `Keyboard.kbSurface` | the design face's surface (Swift reads the palette) | + + + + + + + +## Metrics and motion + +Every size, gap, radius, opacity and duration the views use. The heights in +`KeyboardLayout.height` are derived from these, and `LayoutArithmeticTests` +pins the derivation. + + + +| Token | Points | Swift | Use | +| ---------------------------- | --------- | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `addterm/chip-gap` | `6` | `Metrics.addtermChipGap` | between the chip's glyph and the word | +| `addterm/chip-min-width` | `96` | `Metrics.addtermChipMinWidth` | the least room a chip needs beside a centred mic; with less, the mic and the chip centre as a pair | +| `addterm/chip-pad` | `10` | `Metrics.addtermChipPad` | the selected-word chip's side padding | +| `addterm/inset` | `6` | `Metrics.addtermInset` | the cancel × from the panel's far corner when the mic is at an edge | +| `app/button-height` | `40` | `Metrics.appButtonHeight` | the brand's button | +| `app/button-pad` | `12` | `Metrics.appButtonPad` | its side padding | +| `app/card-pad` | `16` | `Metrics.appCardPad` | inside a card | +| `app/chip-gap` | `8` | `Metrics.appChipGap` | between chips | +| `app/chip-pad-x` | `12` | `Metrics.appChipPadX` | a style chip's side padding | +| `app/chip-pad-y` | `8` | `Metrics.appChipPadY` | and its top and bottom | +| `app/header-height` | `44` | `Metrics.appHeaderHeight` | the home header: wordmark and gear | +| `app/hero-gap` | `14` | `Metrics.appHeroGap` | between the hero's pieces | +| `app/hero-pad` | `20` | `Metrics.appHeroPad` | inside the hero card | +| `app/icon` | `20` | `Metrics.appIcon` | the header's gear, the copy glyph | +| `app/line-gap` | `6` | `Metrics.appLineGap` | between lines of one thought | +| `app/page-pad` | `20` | `Metrics.appPagePad` | the screens' side padding | +| `app/preview-gap` | `8` | `Metrics.appPreviewGap` | between the theme card's two faces | +| `app/section-gap` | `24` | `Metrics.appSectionGap` | between the home screen's sections | +| `app/setup-number-width` | `24` | `Metrics.appSetupNumberWidth` | the setup steps' mono numerals | +| `app/stack-gap` | `12` | `Metrics.appStackGap` | between a section's eyebrow and its content, and between cards | +| `card/border` | `1` | `Metrics.cardBorder` | the card's hairline | +| `card/radius` | `12` | `Metrics.cardRadius` | the app's cards (radius/card) | +| `caret/height` | `20` | `Metrics.caretHeight` | — | +| `caret/lead` | `1` | `Metrics.caretLead` | the caret's gap from the text | +| `caret/radius` | `1` | `Metrics.caretRadius` | — | +| `caret/width` | `2` | `Metrics.caretWidth` | — | +| `glyph/hit` | `44` | `Metrics.glyphHit` | the +, × and ✓ touch targets: the HIG's 44 pt square around a size/glyph symbol | +| `grille/bar-height` | `28` | `Metrics.grilleBarHeight` | — | +| `grille/bar-width` | `64` | `Metrics.grilleBarWidth` | the grille in the voice bar | +| `grille/dot` | `2` | `Metrics.grilleDot` | a dot's diameter | +| `grille/glint-arm` | `1.6` | `Metrics.grilleGlintArm` | the cross glint's vertical arm, as a fraction of the grille's height | +| `grille/glint-width` | `1` | `Metrics.grilleGlintWidth` | the glint's arms | +| `grille/home-height` | `108` | `Metrics.grilleHomeHeight` | — | +| `grille/home-width` | `240` | `Metrics.grilleHomeWidth` | the grille on the home screen | +| `grille/panel-height` | `72` | `Metrics.grillePanelHeight` | — | +| `grille/panel-width` | `160` | `Metrics.grillePanelWidth` | the grille in the panel | +| `grille/pitch` | `4` | `Metrics.grillePitch` | the dot lattice: one dot every | +| `grille/sheen-width` | `0.35` | `Metrics.grilleSheenWidth` | the sheen's band, as a fraction of the width | +| `key/abc-width-402` | `43.333` | `Metrics.keyAbcWidth402` | 123 and the globe at 402 pt: the iPhone's | +| `key/gap` | `6` | `Metrics.keyGap` | between keys, the iPhone's | +| `key/height` | `43` | `Metrics.keyHeight` | every key: the iPhone's cap height (apple-geometry.json) | +| `key/letter-width-402` | `33.5` | `Metrics.keyLetterWidth402` | a letter key at 402 pt: (width − 2 × margin/side − 9 × key/gap) / 10 | +| `key/min-width` | `43.333` | `Metrics.keyMinWidth` | a key that isn't given a width: the iPhone's 123 key at 402 pt | +| `key/pad` | `4` | `Metrics.keyPad` | a legend's side padding on a min-width key | +| `key/radius` | `8` | `Metrics.keyRadius` | flat keys, one colour at this corner: the iPhone's (fitted 8.17) | +| `key/reference-width` | `402` | `Metrics.keyReferenceWidth` | the width the @402 tokens were measured at (iPhone 18 Pro); widths scale from it | +| `key/return-width-402` | `92.667` | `Metrics.keyReturnWidth402` | return at 402 pt: two 123 keys and a gap | +| `key/side-gap` | `13.667` | `Metrics.keySideGap` | shift and delete stand this far from the letters: the iPhone's | +| `key/side-width-402` | `45.583` | `Metrics.keySideWidth402` | shift and delete at 402 pt: what seven letters and the side gaps leave, halved (measured 45.5) | +| `key/space-width-402` | `191.667` | `Metrics.keySpaceWidth402` | space at 402 pt beside 123, the globe and return: what is left (measured 191.333) | +| `layout/full` | `270` | `Metrics.layoutFull` | margin/vertical + voicebar/height + 4 × key/height + 3 × row/gap + margin/bottom-keys | +| `layout/panel` | `216` | `Metrics.layoutPanel` | chosen: room for the 96 orb over one key row | +| `layout/slim` | `60` | `Metrics.layoutSlim` | 2 × margin/vertical + voicebar/height | +| `margin/bottom-keys` | `13` | `Metrics.marginBottomKeys` | under a bottom row of keys (the full keyboard, the panel): the iPhone's | +| `margin/side` | `6.5` | `Metrics.marginSide` | at the keyboard's sides: the iPhone's cap margin, (402 − 10 letters − 9 gaps) / 2 | +| `margin/vertical` | `8` | `Metrics.marginVertical` | the keyboard's top, and the slim bar's bottom | +| `opacity/disabled` | `0.4` | `Metrics.opacityDisabled` | the + without Full Access | +| `opacity/glint` | `0.95` | `Metrics.opacityGlint` | the cross glint when the words land | +| `opacity/grain-dark` | `0.14` | `Metrics.opacityGrainDark` | film grain over the dark face: matte, the brand's texture | +| `opacity/grain-light` | `0.08` | `Metrics.opacityGrainLight` | film grain over the light face: paper, lighter | +| `opacity/grille-rest` | `0.55` | `Metrics.opacityGrilleRest` | the dots at rest, over the key colour | +| `opacity/hairline` | `0.5` | `Metrics.opacityHairline` | — | +| `opacity/legend` | `1` | `Metrics.opacityLegend` | key legends over the cap; below 1 they sit back | +| `opacity/legend-muted` | `0.5` | `Metrics.opacityLegendMuted` | the + at rest | +| `opacity/off` | `0.6` | `Metrics.opacityOff` | the element when Blurt isn't ready — dim and still, but there: a tap opens the app | +| `opacity/placeholder` | `0.4` | `Metrics.opacityPlaceholder` | the field's placeholder | +| `opacity/popup-shadow` | `0.12` | `Metrics.opacityPopupShadow` | — | +| `opacity/press-brighten` | `0.15` | `Metrics.opacityPressBrighten` | a key lightens this much while pressed | +| `opacity/pressed` | `0.85` | `Metrics.opacityPressed` | a brand button while pressed: a colour change, no lift | +| `opacity/sheen` | `0.22` | `Metrics.opacitySheen` | the light passing over the element at rest: chrome catching light | +| `opacity/surface-vignette` | `0` | `Metrics.opacitySurfaceVignette` | a soft darkening toward the surface's edges; 0 is none | +| `opacity/term-cancel` | `0.7` | `Metrics.opacityTermCancel` | the field's × | +| `panel/spacing` | `8` | `Metrics.panelSpacing` | under the panel's voice row, above its keys: the top margin again, so the mic is centred between the keyboard's top edge and the keys | +| `picker/preview-width` | `393` | `Metrics.pickerPreviewWidth` | the theme card draws the keyboard at this width | +| `picker/radius` | `10` | `Metrics.pickerRadius` | the theme card's preview corners | +| `picker/scale` | `0.42` | `Metrics.pickerScale` | then scales it to fit two across | +| `popup/extra-width` | `18` | `Metrics.popupExtraWidth` | the letter pop-up is the key width plus this | +| `popup/height` | `56` | `Metrics.popupHeight` | — | +| `popup/offset` | `50` | `Metrics.popupOffset` | the pop-up rises this far from the key's top: it overlaps the key by popup/height minus this and stays inside the keyboard on the top row | +| `popup/radius` | `10` | `Metrics.popupRadius` | — | +| `popup/shadow-radius` | `8` | `Metrics.popupShadowRadius` | the one thing that floats | +| `popup/shadow-y` | `2` | `Metrics.popupShadowY` | — | +| `radius/button` | `4` | `Metrics.radiusButton` | the brand's buttons and the grille: rectangular, never a pill | +| `radius/card` | `12` | `Metrics.radiusCard` | the brand's cards | +| `radius/hero` | `16` | `Metrics.radiusHero` | the brand's featured cards | +| `radius/input` | `8` | `Metrics.radiusInput` | the brand's inputs: the key-term field | +| `ribs/rest-height` | `16` | `Metrics.ribsRestHeight` | — | +| `ribs/rest-width` | `40` | `Metrics.ribsRestWidth` | the ribs asleep: the wave's own bars at rest | +| `row/gap` | `11` | `Metrics.rowGap` | between rows, the iPhone's | +| `streak/arm` | `1.4` | `Metrics.streakArm` | the landing flash's vertical arm, as a fraction of the box's height | +| `streak/height` | `2` | `Metrics.streakHeight` | the light streak while recording | +| `streak/line` | `1` | `Metrics.streakLine` | the hairline at rest | +| `streak/point` | `4` | `Metrics.streakPoint` | the bright point at the centre | +| `surface/paint` | `0` | `Metrics.surfacePaint` | 1 paints the face's surface under the keys; 0 leaves it clear, so the host's own keyboard material (iOS 26's rounded container, which insets a third-party keyboard and cannot be painted over) is the surface, seamlessly, and the keys take the container colours | +| `surface/vignette-end` | `0.75` | `Metrics.surfaceVignetteEnd` | where it is full | +| `surface/vignette-start` | `0.3` | `Metrics.surfaceVignetteStart` | where the vignette begins, as a fraction of the keyboard's width | +| `term/border` | `1` | `Metrics.termBorder` | the field's hairline | +| `term/gap` | `8` | `Metrics.termGap` | × · field · ✓ | +| `term/height` | `36` | `Metrics.termHeight` | the key-term field's capsule | +| `term/inset` | `2` | `Metrics.termInset` | the field row's side inset | +| `term/pad` | `14` | `Metrics.termPad` | the field's side padding | +| `term/radius` | `8` | `Metrics.termRadius` | the key-term field: the brand's input corners (radius/input) | +| `voice/bar-height` | `32` | `Metrics.voiceBarHeight` | — | +| `voice/bar-width` | `160` | `Metrics.voiceBarWidth` | the voice element's box in the voice bar — the most it takes (the ribs awake, the streak); the + sits beside what shows | +| `voice/home-height` | `112` | `Metrics.voiceHomeHeight` | — | +| `voice/home-width` | `280` | `Metrics.voiceHomeWidth` | the box on the home screen | +| `voice/panel-height` | `88` | `Metrics.voicePanelHeight` | — | +| `voice/panel-width` | `300` | `Metrics.voicePanelWidth` | the box in the panel | +| `voice/press-scale` | `0.94` | `Metrics.voicePressScale` | the element while pressed | +| `voicebar/addterm-clearance` | `6` | `Metrics.voicebarAddtermClearance` | between the mic key's edge and the add key's box (the + or the chip), the key gap | +| `voicebar/height` | `44` | `Metrics.voicebarHeight` | the voice row, where the system puts its suggestion bar | +| `voicebar/note-gap` | `10` | `Metrics.voicebarNoteGap` | between the orb and the Full Access note | +| `wave/bar` | `2` | `Metrics.waveBar` | a wave bar's width | +| `wave/bar-height` | `24` | `Metrics.waveBarHeight` | — | +| `wave/gap` | `2` | `Metrics.waveGap` | between wave bars | +| `wave/home-height` | `44` | `Metrics.waveHomeHeight` | — | +| `wave/panel-height` | `40` | `Metrics.wavePanelHeight` | — | +| `wordmark/height` | `22` | `Metrics.wordmarkHeight` | — | + + + +Motion. Nothing springs or snaps; only the press answers at once. Colour and +fill changes ease over the brand's transition; everything else moves on the +house curve (`ease/signature`, `cubic-bezier(0.22, 1, 0.36, 1)`). + + + +| Token | Value | Swift | Use | +| ---------------- | --------------- | ---------------------- | -------------------------------------------------------------------------- | +| `caret` | `0.5` | `Motion.caret` | the caret's blink | +| `chip-hold` | `0.6` | `Motion.chipHold` | holding the chip opens the field instead of adding; longer than a firm tap | +| `colour` | `0.2` | `Motion.colour` | a colour or fill changing state: the brand's transition | +| `ease/signature` | `0.22,1,0.36,1` | `Motion.easeSignature` | the house curve for everything that moves; nothing springs | +| `flip` | `0.25` | `Motion.flip` | the panel's carousel | +| `glint-life` | `0.5` | `Motion.glintLife` | a facet's flash while recording | +| `height-change` | `0.25` | `Motion.heightChange` | the keyboard resizing | +| `key-press` | `0.08` | `Motion.keyPress` | a key lighting | +| `landing` | `0.6` | `Motion.landing` | the glint when the words land | +| `press` | `0.1` | `Motion.press` | the orb's press; the one thing that answers at once | +| `ring-period` | `1.6` | `Motion.ringPeriod` | one turn of the ring, the Mac's cadence | +| `sheen` | `4` | `Motion.sheen` | one pass of the light over the element at rest | +| `sheen-working` | `1.6` | `Motion.sheenWorking` | and while something is happening: the ring's cadence | +| `state-fade` | `0.5` | `Motion.stateFade` | every other change on the key: the ring, a glyph, the dimming | +| `term-swap` | `0.4` | `Motion.termSwap` | the voice bar becoming the field | +| `wave-fade` | `0.7` | `Motion.waveFade` | the orb and the wave crossing, either way | + + + +## The app's screens (`BlurtiOS/Sources/`) + +The Blurt landing page on a phone, in the phone's appearance: the page under +the brand's grain; the wordmark (tinted with the accent) and the gear across +the top; sections under mono eyebrows; serif headlines (the system serif, 34), +body in the system face (16, captions 14); cards with 12 pt corners and a 1 pt +hairline (the hero 16), no shadows; chips and buttons rectangular at 4 pt, +buttons 40 pt tall with their label in uppercase mono, one green button per +row (Cam's rule) and a hairline one beside it, a colour change on press and +never a lift. Glyphs are SF Symbols — the platform's — where the web brand +uses Material Symbols. + +**Home** (`HomeView`, `HomeHero`, `SetupCard`, `StyleChips`, `RecentSection`, +`HomeStatus` for the words): the setup card only while something is missing, +its three steps numbered `01 02 03` in mono; the hero — `STATUS`, the +headline (Not listening · Ready to dictate · Connecting… · Listening… · +Transcribing… · Pasted · Copied · the error), the line under it, the voice +element at home size, `START LISTENING` (refused without a key, with a line +saying so) or the hairline `STOP LISTENING`, and "Dictate here, to the +clipboard" as a text button; `STYLE` and the chips, the active one filled; +`RECENT` and the cards (three lines, the style as a green mono tag, the +time, a copy button); `POWERED BY ASSEMBLYAI`. + +**Settings** (`SettingsView`): a form on the page, its sections under +eyebrows — Keyboard (layout, theme, hands-free), Listening (the window), +Transcription (enhanced transcripts, output styles, key terms with the +contact-name count, sharing), Account (sign-in stub; the API key in debug +builds), About. The theme picker shows each theme's two faces side by side. +Styles, the import sheet and the key entry share the form's chrome +(`brandForm()`). + +`-BlurtSettings` opens Settings as the root for a screenshot; `xcrun simctl +ui booted appearance dark` before any capture for the dark appearance; +`-BlurtStartListening` opens the mic at launch. + +## Feedback and accessibility + +Haptics (`KeyboardModel.haptics`): medium impact on `recording`, light on the +stop, `.success` on pasted/copied, `.error` on error — with no words on the +keyboard, they are half of the feedback. The DX7 start/stop cues play in the +app, not the keyboard. The mic control is a button labelled Dictate / Stop +dictation / Start Blurt and speaks the error message; the element itself is +hidden from VoiceOver. Reduce Motion stops the sheen and the glints; heights +still follow the level. + +## The two processes, and every number + +The app listens and transcribes; the keyboard is a remote control. They talk +over the App Group's `UserDefaults` (payloads as JSON) plus Darwin +notifications that carry nothing and just say "look" (`BlurtiOSCore/SharedState.swift`). + +| Rule | Value | Why | +| ---------------------- | ---------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | +| presence | heartbeat 5 s (app), 4 s (keyboard); window 15 s | more than twice a heartbeat, so one missed beat isn't absence; a killed process is absent within seconds | +| listening | window not lapsed **and** app seen within 15 s | a killed app, or a phone call, with the window's end still in the future | +| result freshness | 10 s | a notification held for a suspended keyboard arrives minutes late | +| result addressing | `recipient` = the keyboard instance last seen | two live keyboards (two host apps) must not both insert | +| result delivery | the app waits 3 s (poll 250 ms) for the keyboard to take the result; else clipboard + "Copied" | "Pasted" is never a guess | +| command freshness | 10 s | a held press from minutes ago must not start a dictation on resume | +| command retry | one re-signal after 600 ms with no phase change | a missed Darwin notification would leave the gate latched over nothing | +| phase staleness | notices 3 s (max dwell 2 s + 1); in flight 130 s (the 120 s cap + 10) | the keyboard reads the snapshot on every appearance | +| notice dwell | pasted 1.2 s; copied, error 2 s | the Mac's 0.8 / 1.6 s, a little longer with no hover | +| press delay on the mic | 90 ms | a swipe that starts on the key never starts a dictation it must then cancel | +| tap travel | 12 pt (keys), 24 pt (mic), swipe ≥ 48 pt horizontal and 1.5× the vertical | the carousel's swipe never types or dictates | +| level publish | every 80 ms while recording | the element's meter, off the capture path's back | +| lexicon refresh | hourly | thousands of contacts on a keyboard memory budget | +| height change | 0.25 s; constraint priority 999 | iOS honours a keyboard's height at just under required | +| the visual numbers | see Metrics and motion above | generated from `Design/tokens.json`; the views carry no literals | +| term packs | ≤ 64 KB, ≤ 100 terms, ≤ 80 characters each | the request's cap; unbounded input from a stranger | + +Tests pin the rules that can be pinned (`BlurtiOSTests`: the contract, the +gate, results, the term field, the feed, the layout arithmetic, Apple's +geometry, the interaction rules, the voice state, the home's words, the +tokens); anything that gains a number here should gain a line +there. + +## Where Blurt diverges from the brand system, on purpose + +- Green is the UI signal (the system says cobolt-only): Cam's rule for the + collab brand — inside the Assembly palette, off primary cobolt. Cobolt and + its violets appear only as light: the glint's fringe, the streak's tips. +- SF Pro for the letter keys and SF Symbols for every glyph (the web uses + Material Symbols): a keyboard has to be native under the fingers. +- The keys' corners are the iPhone's (8), not the brand's 4: Apple's exact + geometry wins on the keyboard; the brand's radii hold everywhere else. + +## Next + +The pick between a, b and c; the losers and the `-BlurtGalleryVoice` switch +go, and the winner's tokens lose their prefix. Then: the keyboard's autocorrect +and suggestion bar; onboarding as a stepped flow once sign-in exists; the +app's cues on the phase edges; the curated themes as brand value sets. diff --git a/App/BlurtiOS/Design/apple-geometry.json b/App/BlurtiOS/Design/apple-geometry.json new file mode 100644 index 00000000..df7df064 --- /dev/null +++ b/App/BlurtiOS/Design/apple-geometry.json @@ -0,0 +1,58 @@ +{ + "cap": { + "abc": 43.333, + "bottomMargin": 13.0, + "gap": 6.0, + "height": 43, + "keyboardHeight": 270.667, + "letter": 33.5, + "marginRight": 6.333, + "marginSide": 6.667, + "radius": 8.17, + "return": 93, + "rowGap": 11.0, + "side": 45.5, + "sideGap": 13.667, + "space": 191.333, + "systemBarBelow": 58.0, + "topBand": 52.667 + }, + "cell": { + "delete": 51.667, + "firstRowTop": 597.0, + "keyboardHeight": 226.0, + "keyboardTop": 590.0, + "letter": 39.333, + "more": 49.333, + "return": 99.0, + "row": 54.0, + "shift": 51.333, + "space": 197.333 + }, + "colors": { + "dark": { + "key": "#575757", + "modifier": "#575757", + "surface": "#373737" + }, + "light": { + "key": "#FFFFFF", + "modifier": "#FFFFFF", + "surface": "#E2E3E8" + } + }, + "darkFaceIdentical": true, + "device": "iPhone 18 Pro", + "legend": { + "lHeight": 17.667, + "pointSizeEstimate": 24.201 + }, + "measured": "2026-09-29", + "sources": { + "caps": "pixel scan of the light screenshot (scripts/design-diff.swift measure)", + "cells": "XCUITest key frames (BlurtiOSProbe) \u2014 touch cells, no gaps", + "legend": "the l glyph's height on the light face; the point size is an estimate (\u00f7 0.73, SF's ascender)", + "radius": "circle fitted to the top-left inset of the q cap over its first 12 pt (\u00b10.25)" + }, + "width": 402 +} diff --git a/App/BlurtiOS/Design/tokens.json b/App/BlurtiOS/Design/tokens.json new file mode 100644 index 00000000..29e2f484 --- /dev/null +++ b/App/BlurtiOS/Design/tokens.json @@ -0,0 +1,894 @@ +{ + "version": 1, + "brand": { + "green/700": { + "value": "#01762F", + "use": "the wordmark green, light chrome (the app's accent in light)" + }, + "green/400": { + "value": "#67AD82", + "use": "the brand hue lifted for dark chrome: the meter, the ring, the wave on ink" + }, + "green/200": { + "value": "#99C8AC", + "use": "the design system's UI-and-code green highlight (unused on the phone yet)" + }, + "green/100": { + "value": "#CCE4D5", + "use": "the design system's UI-and-code green fill (unused on the phone yet)" + }, + "green/mist": { + "value": "#DBF5E6", + "use": "the pale green the orb lightens into" + }, + "cobolt": { + "value": "#3923C7", + "use": "AssemblyAI's primary violet: the drop, never a key or a surface" + }, + "violet/lavender": { + "value": "#D7D3F4", + "use": "the orb's lightest violet: its top and the gradient's ends" + }, + "violet/periwinkle": { + "value": "#B0A7E9", + "use": "the orb's mid violet: the drop's halo" + }, + "violet/iris": { + "value": "#887BDD", + "use": "the orb gradient's upper violet stop" + }, + "ink": { + "value": "#1D1B16", + "use": "the brand ink: the Mac pill's body, the ink theme's surface" + }, + "ink/800": { + "value": "#26231E", + "use": "a modifier key on ink; the Mac's dark card fill" + }, + "ink/700": { + "value": "#33302A", + "use": "an ordinary key on ink, one step up" + }, + "ink/600": { + "value": "#3A362F", + "use": "the Mac's dark card border" + }, + "warm-white": { + "value": "#F2EEE6", + "use": "key legends on ink: warm white on warm ink" + }, + "card": { + "value": "#EBE8E8", + "use": "the warm card fill; the paper theme's surface" + }, + "card-border": { + "value": "#DEDBDB", + "use": "the card's hairline; the paper theme's modifier key" + }, + "orange": { + "value": "#E67F36", + "use": "the error word and ring, cancel, the Full Access note; never a red body" + }, + "white": { + "value": "#FFFFFF" + }, + "black": { + "value": "#000000" + }, + "apple/light-surface": { + "value": "#D1D5DB", + "use": "the iPhone keyboard's light surface" + }, + "apple/light-modifier": { + "value": "#ADB3BC", + "use": "the iPhone keyboard's light modifier key" + }, + "apple/dark-surface": { + "value": "#2B2B2B", + "use": "the iPhone keyboard's dark surface" + }, + "apple/dark-key": { + "value": "#6B6B6B", + "use": "the iPhone keyboard's dark key" + }, + "apple/dark-modifier": { + "value": "#464646", + "use": "the iPhone keyboard's dark modifier key" + }, + "paper/page": { + "value": "#FDFCF8", + "use": "the design system's page: the app's light ground" + }, + "paper/tint": { + "value": "#F5F3EB", + "use": "the warm off-white one step in: light cards, dark legends" + }, + "paper/200": { + "value": "#ECEBE5", + "use": "neutral-100: the light face's surface" + }, + "paper/300": { + "value": "#DAD7CB", + "use": "neutral-200: a modifier key on the light face" + }, + "paper/400": { + "value": "#C7C3B2", + "use": "neutral-300: hairlines and field borders on the light face" + }, + "neutral/900": { + "value": "#4A4945", + "use": "black-400: body text on the light face" + }, + "neutral/700": { + "value": "#777673", + "use": "black-300: muted text and mono labels on the light face" + }, + "neutral/500": { + "value": "#A5A4A2", + "use": "black-200: mono labels on the dark face" + }, + "neutral/300": { + "value": "#D2D1D0", + "use": "black-100: the lightest warm grey" + }, + "material/dark-key": { + "value": "#575757", + "use": "a key on the system's dark keyboard material (measured, iOS 27): the container case" + }, + "material/dark-modifier": { + "value": "#474747", + "use": "a modifier key on the system's dark material" + }, + "material/light-surface": { + "value": "#E2E3E8", + "use": "the system's light keyboard material (measured, iOS 27): what the gallery stands under a clear surface" + }, + "material/dark-surface": { + "value": "#373737", + "use": "the system's dark keyboard material (measured, iOS 27)" + } + }, + "themes": { + "light/surface": "{brand.paper/200}", + "light/key": "{brand.white}", + "light/key-modifier": "{brand.paper/300}", + "light/legend": "{brand.ink}", + "light/legend-secondary": "{brand.neutral/700}", + "light/signal": "{brand.green/700}", + "light/popup": "{brand.white}", + "light/field": "{brand.white}", + "light/field-border": "{brand.paper/400}", + "light/notice": "{brand.orange}", + "light/container-key": "{brand.white}", + "light/container-key-modifier": "{brand.paper/300}", + "light/material": "{brand.material/light-surface}", + "dark/surface": "{brand.ink}", + "dark/key": "{brand.ink/700}", + "dark/key-modifier": "{brand.ink/800}", + "dark/legend": "{brand.paper/tint}", + "dark/legend-secondary": "{brand.neutral/500}", + "dark/signal": "{brand.green/400}", + "dark/popup": "{brand.ink/700}", + "dark/field": "{brand.ink/700}", + "dark/field-border": "{brand.ink/600}", + "dark/notice": "{brand.orange}", + "dark/container-key": "{brand.material/dark-key}", + "dark/container-key-modifier": "{brand.material/dark-modifier}", + "dark/material": "{brand.material/dark-surface}" + }, + "keyboard": { + "kb/surface": { + "value": "{themes.dark/surface}", + "use": "the design face's surface (Swift reads the palette)" + }, + "kb/key": { + "value": "{themes.dark/key}", + "use": "an ordinary key" + }, + "kb/key-modifier": { + "value": "{themes.dark/key-modifier}", + "use": "shift, delete, globe, return, 123, cancel" + }, + "kb/legend": { + "value": "{themes.dark/legend}", + "use": "key legends and bare glyphs" + }, + "kb/signal": { + "value": "{themes.dark/signal}", + "use": "the wave, the caret, the saved check" + }, + "kb/popup": { + "value": "{themes.dark/popup}", + "use": "the letter pop-up" + }, + "kb/notice-error": { + "value": "{brand.orange}", + "use": "the solid ring and glyph for an error" + }, + "kb/cancel": { + "value": "{brand.orange}", + "use": "the panel's cancel ×" + }, + "kb/full-access-note": { + "value": "{brand.orange}", + "use": "the one line of words the keyboard ever shows" + }, + "app/accent-light": { + "value": "{brand.green/700}", + "use": "the catalog's AccentColor in light" + }, + "app/accent-dark": { + "value": "{brand.green/400}", + "use": "the catalog's AccentColor in dark" + }, + "app/card-fill-light": { + "value": "{brand.paper/tint}", + "use": "the catalog's CardFill in light" + }, + "app/card-fill-dark": { + "value": "{brand.ink/800}", + "use": "the catalog's CardFill in dark" + }, + "app/card-border-light": { + "value": "{brand.paper/400}", + "use": "the catalog's CardBorder in light" + }, + "app/card-border-dark": { + "value": "{brand.ink/600}", + "use": "the catalog's CardBorder in dark" + }, + "kb/legend-secondary": { + "value": "{themes.dark/legend-secondary}", + "use": "the mono word labels (123, ABC, return, space) and the + at rest" + }, + "kb/field": { + "value": "{themes.dark/field}", + "use": "the key-term field" + }, + "kb/field-border": { + "value": "{themes.dark/field-border}", + "use": "the field's hairline" + }, + "app/page-light": { + "value": "{brand.paper/page}", + "use": "the catalog's Page in light: the design system's page" + }, + "app/page-dark": { + "value": "{brand.ink}", + "use": "the catalog's Page in dark: ink, the website's ground" + }, + "app/text-light": { + "value": "{brand.ink}", + "use": "the catalog's Text in light" + }, + "app/text-dark": { + "value": "{brand.paper/tint}", + "use": "the catalog's Text in dark" + }, + "app/muted-light": { + "value": "{brand.neutral/700}", + "use": "the catalog's Muted in light: eyebrows, captions, secondary text" + }, + "app/muted-dark": { + "value": "{brand.neutral/500}", + "use": "the catalog's Muted in dark" + }, + "app/cta-light": { + "value": "{brand.green/700}", + "use": "the catalog's CTA in light: the one green button" + }, + "app/cta-dark": { + "value": "{brand.green/400}", + "use": "the catalog's CTA in dark: the site's button green" + }, + "app/cta-text-light": { + "value": "{brand.white}", + "use": "the catalog's CTAText in light" + }, + "app/cta-text-dark": { + "value": "{brand.ink}", + "use": "the catalog's CTAText in dark: ink on green, as the site" + } + }, + "gradients": {}, + "metrics": { + "key/height": { + "value": 43, + "use": "every key: the iPhone's cap height (apple-geometry.json)" + }, + "key/gap": { + "value": 6, + "use": "between keys, the iPhone's" + }, + "row/gap": { + "value": 11, + "use": "between rows, the iPhone's" + }, + "margin/side": { + "value": 6.5, + "use": "at the keyboard's sides: the iPhone's cap margin, (402 − 10 letters − 9 gaps) / 2" + }, + "margin/vertical": { + "value": 8, + "use": "the keyboard's top, and the slim bar's bottom" + }, + "key/radius": { + "value": 8, + "use": "flat keys, one colour at this corner: the iPhone's (fitted 8.17)" + }, + "key/min-width": { + "value": 43.333, + "use": "a key that isn't given a width: the iPhone's 123 key at 402 pt" + }, + "key/letter-width-402": { + "value": 33.5, + "use": "a letter key at 402 pt: (width − 2 × margin/side − 9 × key/gap) / 10" + }, + "key/side-width-402": { + "value": 45.583, + "use": "shift and delete at 402 pt: what seven letters and the side gaps leave, halved (measured 45.5)" + }, + "key/return-width-402": { + "value": 92.667, + "use": "return at 402 pt: two 123 keys and a gap" + }, + "voicebar/height": { + "value": 44, + "use": "the voice row, where the system puts its suggestion bar" + }, + "voicebar/note-gap": { + "value": 10, + "use": "between the orb and the Full Access note" + }, + "voicebar/addterm-clearance": { + "value": 6, + "use": "between the mic key's edge and the add key's box (the + or the chip), the key gap" + }, + "wave/bar-height": { + "value": 24 + }, + "wave/panel-height": { + "value": 40 + }, + "wave/home-height": { + "value": 44 + }, + "wave/bar": { + "value": 2, + "use": "a wave bar's width" + }, + "wave/gap": { + "value": 2, + "use": "between wave bars" + }, + "popup/extra-width": { + "value": 18, + "use": "the letter pop-up is the key width plus this" + }, + "popup/height": { + "value": 56 + }, + "popup/radius": { + "value": 10 + }, + "popup/offset": { + "value": 50, + "use": "the pop-up rises this far from the key's top: it overlaps the key by popup/height minus this and stays inside the keyboard on the top row" + }, + "popup/shadow-radius": { + "value": 8, + "use": "the one thing that floats" + }, + "popup/shadow-y": { + "value": 2 + }, + "term/height": { + "value": 36, + "use": "the key-term field's capsule" + }, + "term/pad": { + "value": 14, + "use": "the field's side padding" + }, + "term/gap": { + "value": 8, + "use": "× · field · ✓" + }, + "term/inset": { + "value": 2, + "use": "the field row's side inset" + }, + "caret/width": { + "value": 2 + }, + "caret/height": { + "value": 20 + }, + "caret/radius": { + "value": 1 + }, + "caret/lead": { + "value": 1, + "use": "the caret's gap from the text" + }, + "glyph/hit": { + "value": 44, + "use": "the +, × and ✓ touch targets: the HIG's 44 pt square around a size/glyph symbol" + }, + "panel/spacing": { + "value": 8, + "use": "under the panel's voice row, above its keys: the top margin again, so the mic is centred between the keyboard's top edge and the keys" + }, + "addterm/inset": { + "value": 6, + "use": "the cancel × from the panel's far corner when the mic is at an edge" + }, + "addterm/chip-pad": { + "value": 10, + "use": "the selected-word chip's side padding" + }, + "addterm/chip-gap": { + "value": 6, + "use": "between the chip's glyph and the word" + }, + "addterm/chip-min-width": { + "value": 96, + "use": "the least room a chip needs beside a centred mic; with less, the mic and the chip centre as a pair" + }, + "card/radius": { + "value": 12, + "use": "the app's cards (radius/card)" + }, + "card/border": { + "value": 1, + "use": "the card's hairline" + }, + "wordmark/height": { + "value": 22 + }, + "picker/preview-width": { + "value": 393, + "use": "the theme card draws the keyboard at this width" + }, + "picker/scale": { + "value": 0.42, + "use": "then scales it to fit two across" + }, + "picker/radius": { + "value": 10, + "use": "the theme card's preview corners" + }, + "layout/slim": { + "value": 60, + "use": "2 × margin/vertical + voicebar/height" + }, + "layout/panel": { + "value": 216, + "use": "chosen: room for the 96 orb over one key row" + }, + "layout/full": { + "value": 270, + "use": "margin/vertical + voicebar/height + 4 × key/height + 3 × row/gap + margin/bottom-keys" + }, + "opacity/legend-muted": { + "value": 0.5, + "use": "the + at rest" + }, + "opacity/term-cancel": { + "value": 0.7, + "use": "the field's ×" + }, + "opacity/placeholder": { + "value": 0.4, + "use": "the field's placeholder" + }, + "opacity/disabled": { + "value": 0.4, + "use": "the + without Full Access" + }, + "opacity/press-brighten": { + "value": 0.15, + "use": "a key lightens this much while pressed" + }, + "opacity/legend": { + "value": 1, + "use": "key legends over the cap; below 1 they sit back" + }, + "opacity/popup-shadow": { + "value": 0.12 + }, + "opacity/surface-vignette": { + "value": 0.0, + "use": "a soft darkening toward the surface's edges; 0 is none" + }, + "surface/vignette-start": { + "value": 0.3, + "use": "where the vignette begins, as a fraction of the keyboard's width" + }, + "surface/vignette-end": { + "value": 0.75, + "use": "where it is full" + }, + "margin/bottom-keys": { + "value": 13, + "use": "under a bottom row of keys (the full keyboard, the panel): the iPhone's" + }, + "key/side-gap": { + "value": 13.667, + "use": "shift and delete stand this far from the letters: the iPhone's" + }, + "key/reference-width": { + "value": 402, + "use": "the width the @402 tokens were measured at (iPhone 18 Pro); widths scale from it" + }, + "key/abc-width-402": { + "value": 43.333, + "use": "123 and the globe at 402 pt: the iPhone's" + }, + "key/space-width-402": { + "value": 191.667, + "use": "space at 402 pt beside 123, the globe and return: what is left (measured 191.333)" + }, + "voice/bar-width": { + "value": 160, + "use": "the voice element's box in the voice bar — the most it takes (the ribs awake, the streak); the + sits beside what shows" + }, + "voice/bar-height": { + "value": 32 + }, + "voice/panel-width": { + "value": 300, + "use": "the box in the panel" + }, + "voice/panel-height": { + "value": 88 + }, + "voice/home-width": { + "value": 280, + "use": "the box on the home screen" + }, + "voice/home-height": { + "value": 112 + }, + "voice/press-scale": { + "value": 0.94, + "use": "the element while pressed" + }, + "opacity/off": { + "value": 0.6, + "use": "the element when Blurt isn't ready — dim and still, but there: a tap opens the app" + }, + "opacity/sheen": { + "value": 0.22, + "use": "the light passing over the element at rest: chrome catching light" + }, + "opacity/glint": { + "value": 0.95, + "use": "the cross glint when the words land" + }, + "radius/button": { + "value": 4, + "use": "the brand's buttons and the grille: rectangular, never a pill" + }, + "radius/input": { + "value": 8, + "use": "the brand's inputs: the key-term field" + }, + "radius/card": { + "value": 12, + "use": "the brand's cards" + }, + "radius/hero": { + "value": 16, + "use": "the brand's featured cards" + }, + "grille/pitch": { + "value": 4, + "use": "the dot lattice: one dot every" + }, + "grille/dot": { + "value": 2, + "use": "a dot's diameter" + }, + "grille/bar-width": { + "value": 64, + "use": "the grille in the voice bar" + }, + "grille/bar-height": { + "value": 28 + }, + "grille/panel-width": { + "value": 160, + "use": "the grille in the panel" + }, + "grille/panel-height": { + "value": 72 + }, + "grille/home-width": { + "value": 240, + "use": "the grille on the home screen" + }, + "grille/home-height": { + "value": 108 + }, + "opacity/grille-rest": { + "value": 0.55, + "use": "the dots at rest, over the key colour" + }, + "grille/glint-arm": { + "value": 1.6, + "use": "the cross glint's vertical arm, as a fraction of the grille's height" + }, + "grille/glint-width": { + "value": 1, + "use": "the glint's arms" + }, + "grille/sheen-width": { + "value": 0.35, + "use": "the sheen's band, as a fraction of the width" + }, + "ribs/rest-width": { + "value": 40, + "use": "the ribs asleep: the wave's own bars at rest" + }, + "ribs/rest-height": { + "value": 16 + }, + "streak/line": { + "value": 1, + "use": "the hairline at rest" + }, + "streak/height": { + "value": 2, + "use": "the light streak while recording" + }, + "streak/point": { + "value": 4, + "use": "the bright point at the centre" + }, + "streak/arm": { + "value": 1.4, + "use": "the landing flash's vertical arm, as a fraction of the box's height" + }, + "opacity/hairline": { + "value": 0.5 + }, + "opacity/grain-dark": { + "value": 0.14, + "use": "film grain over the dark face: matte, the brand's texture" + }, + "opacity/grain-light": { + "value": 0.08, + "use": "film grain over the light face: paper, lighter" + }, + "term/radius": { + "value": 8, + "use": "the key-term field: the brand's input corners (radius/input)" + }, + "term/border": { + "value": 1, + "use": "the field's hairline" + }, + "key/pad": { + "value": 4, + "use": "a legend's side padding on a min-width key" + }, + "app/page-pad": { + "value": 20, + "use": "the screens' side padding" + }, + "app/header-height": { + "value": 44, + "use": "the home header: wordmark and gear" + }, + "app/section-gap": { + "value": 24, + "use": "between the home screen's sections" + }, + "app/stack-gap": { + "value": 12, + "use": "between a section's eyebrow and its content, and between cards" + }, + "app/line-gap": { + "value": 6, + "use": "between lines of one thought" + }, + "app/card-pad": { + "value": 16, + "use": "inside a card" + }, + "app/hero-pad": { + "value": 20, + "use": "inside the hero card" + }, + "app/hero-gap": { + "value": 14, + "use": "between the hero's pieces" + }, + "app/button-height": { + "value": 40, + "use": "the brand's button" + }, + "app/button-pad": { + "value": 12, + "use": "its side padding" + }, + "app/chip-pad-x": { + "value": 12, + "use": "a style chip's side padding" + }, + "app/chip-pad-y": { + "value": 8, + "use": "and its top and bottom" + }, + "app/chip-gap": { + "value": 8, + "use": "between chips" + }, + "app/icon": { + "value": 20, + "use": "the header's gear, the copy glyph" + }, + "app/setup-number-width": { + "value": 24, + "use": "the setup steps' mono numerals" + }, + "app/preview-gap": { + "value": 8, + "use": "between the theme card's two faces" + }, + "opacity/pressed": { + "value": 0.85, + "use": "a brand button while pressed: a colour change, no lift" + }, + "surface/paint": { + "value": 0, + "use": "1 paints the face's surface under the keys; 0 leaves it clear, so the host's own keyboard material (iOS 26's rounded container, which insets a third-party keyboard and cannot be painted over) is the surface, seamlessly, and the keys take the container colours" + } + }, + "type": { + "size/letter": { + "value": 24, + "use": "letter keys: the iPhone's (estimated from the l glyph, apple-geometry.json)" + }, + "size/legend": { + "value": 16, + "use": "every other key" + }, + "size/popup": { + "value": 32, + "use": "the letter pop-up" + }, + "size/term": { + "value": 17, + "use": "the key-term field" + }, + "size/glyph": { + "value": 17, + "use": "the +, × and ✓" + }, + "weight/letter": { + "value": "regular" + }, + "weight/legend": { + "value": "medium" + }, + "weight/popup": { + "value": "regular" + }, + "weight/term": { + "value": "regular" + }, + "weight/glyph": { + "value": "medium" + }, + "size/label": { + "value": 12, + "use": "the mono word labels on keys: 123, ABC, #+=, space, the return label" + }, + "size/eyebrow": { + "value": 12, + "use": "a mono eyebrow in the app (E1)" + }, + "size/cta": { + "value": 14, + "use": "a mono button label in the app (E2)" + }, + "size/title": { + "value": 34, + "use": "the app's serif headline" + }, + "size/body": { + "value": 16, + "use": "the app's body text" + }, + "size/caption": { + "value": 14, + "use": "the app's small text" + }, + "weight/label": { + "value": "medium" + }, + "tracking/eyebrow": { + "value": 1.2, + "use": "uppercase mono at 12: the brand's E1 tracking" + }, + "tracking/cta": { + "value": 1.4, + "use": "uppercase mono at 14: the brand's E2 tracking" + }, + "ratio/voice-glyph": { + "value": 0.34, + "use": "the clipboard and exclamation glyphs, as a fraction of the voice element's height" + }, + "weight/voice-glyph": { + "value": "semibold" + } + }, + "motion": { + "wave-fade": { + "value": 0.7, + "use": "the orb and the wave crossing, either way" + }, + "state-fade": { + "value": 0.5, + "use": "every other change on the key: the ring, a glyph, the dimming" + }, + "term-swap": { + "value": 0.4, + "use": "the voice bar becoming the field" + }, + "chip-hold": { + "value": 0.6, + "use": "holding the chip opens the field instead of adding; longer than a firm tap" + }, + "press": { + "value": 0.1, + "use": "the orb's press; the one thing that answers at once" + }, + "key-press": { + "value": 0.08, + "use": "a key lighting" + }, + "ring-period": { + "value": 1.6, + "use": "one turn of the ring, the Mac's cadence" + }, + "caret": { + "value": 0.5, + "use": "the caret's blink" + }, + "flip": { + "value": 0.25, + "use": "the panel's carousel" + }, + "height-change": { + "value": 0.25, + "use": "the keyboard resizing" + }, + "colour": { + "value": 0.2, + "use": "a colour or fill changing state: the brand's transition" + }, + "landing": { + "value": 0.6, + "use": "the glint when the words land" + }, + "ease/signature": { + "value": [0.22, 1, 0.36, 1], + "use": "the house curve for everything that moves; nothing springs" + }, + "sheen": { + "value": 4, + "use": "one pass of the light over the element at rest" + }, + "sheen-working": { + "value": 1.6, + "use": "and while something is happening: the ring's cadence" + }, + "glint-life": { + "value": 0.5, + "use": "a facet's flash while recording" + } + } +} diff --git a/App/BlurtiOS/README.md b/App/BlurtiOS/README.md new file mode 100644 index 00000000..ae271b0b --- /dev/null +++ b/App/BlurtiOS/README.md @@ -0,0 +1,160 @@ +# Blurt for iPhone + +The iPhone app and the keyboard it ships. Same engine as the Mac app +(`Sources/BlurtEngine`), a different shell, because iPhone leaves no other shape: + +- **The app listens and transcribes.** iOS lets no keyboard use the microphone. The + app opens the microphone while it is in front (the keyboard sends it there with + `blurt://start`), keeps it open for a _listening window_ (`ListeningWindow`, + default 15 minutes, extended by every dictation), and runs the engine's + `DictationSession` over that always-on capture (`WindowedAudioSource`, which + converts the phone's native audio to the 16 kHz mono PCM the dictation API wants). +- **The keyboard is a remote control.** It sends press / release / cancel with the + text around the cursor (the same context the Mac reads through Accessibility), + shows the phase the app publishes, inserts the words that come back through + `textDocumentProxy`, and reads the phone's word list (`UILexicon`: contact names + and text replacements) so names come back spelled right with no setup. +- **They talk through an App Group** (`BlurtiOSCore/SharedState.swift`): payloads in the + shared defaults, Darwin notifications to wake each other. Full Access is what + lets the keyboard reach the group — without it the keyboard still types, and says + what it needs. +- **Three layouts, the user's choice**: a slim mic bar, a mic panel (the Wispr / + Aqua shape), or a full keyboard with the mic as the main key. One keyboard + target, one setting, one set of plumbing. + +## Build + +```bash +cd App/BlurtiOS && xcodegen generate # the project is generated, not committed +open BlurtiOS.xcodeproj # scheme BlurtiOS builds and embeds the keyboard +``` + +CI builds it for the simulator on every PR (`check.yml`'s `ios-build` job). On a +Mac with only the Command Line Tools — no iOS SDK — `scripts/ios-typecheck.sh` +typechecks both targets against the Mac Catalyst frameworks with the same +flags, which catches the Swift 6 isolation errors before CI does. Running +on a phone needs Xcode with a team that can provision the App Group; every id in +`project.yml` (`dev.alex.blurt.ios`, `group.dev.alex.blurt`) is a placeholder +inherited from the Mac app and must become org-owned before the App Store. + +## Design + +`DESIGN.md` — the tokens, the orb, the meter, the three layouts, the +app's screens, all traced to the Mac app's sources, plus the gallery that renders every layout +and state for a screenshot and the launch flags that reach every app state. + +## Getting Xcode + +Any Xcode from 26.6 (CI's macos-26 runner has it) works; a phone on iOS 27 needs Xcode 27, which +needs macOS 26.6. The Mac App Store is one way. The other is Apple's developer site through +[`xcodes`](https://github.com/XcodesOrg/xcodes): Homebrew cannot build it without Xcode +already present (chicken and egg), so take the prebuilt binary — +`gh release download --repo XcodesOrg/xcodes --pattern xcodes.zip`, unzip, put `xcodes` on +`PATH` — then `xcodes install 27.0 --select`. It signs in with an Apple ID that has accepted +the Apple Developer Agreement at developer.apple.com/account; a company-managed Apple ID +(Apple Business Manager) cannot accept it and gets a 403, so use a personal one for the +download. The archive lands in `~/Library/Application Support/com.robotsandpencils.xcodes/`; +an 83 KB "xip" there is a saved error page, delete it and retry. + +## Logic and UI + +The iPhone code splits the way the Mac's does. **`BlurtiOSCore/`** is the logic — the App Group +contract, the keyboard's model and rules, the voice state, term packs, the relay injector — and +plays the part `BlurtEngine` plays for the Mac shell: no SwiftUI, no views, no audio (the keyboard +links it, and the keyboard never hears anything; `check-invariants.sh` enforces the last). The +app (`BlurtiOS/Sources`) and the keyboard (`BlurtKeyboard/Sources`) are the UI on top of it, with +`Shared/` holding what both draw with (design tokens, type, brand colours). + +It is a static framework, so the app and the keyboard each link their own copy beside the +engine's, and its API is `package` (one `SWIFT_PACKAGE_NAME` across the project), not `public`: +nothing outside this project can see it, and periphery still reports what nothing uses. A +preview hands the model a `ThemeFace`, not a palette — the colours are the UI's +(`KeyboardModel+Palette.swift`). + +## Unit tests + +`scripts/ios-test.sh` runs `BlurtiOSTests` on a simulator (CI's `ios-build` job runs it after +the build): the core's logic — the App Group contract (`KeyTermList`, stale snapshots, layouts), +the keyboard's rules (sentence start, the double space, the term field, return labels, letter +rows) against a fake text field, the gate against the app's phases, the Darwin signals, term +packs — and what the UI builds on it (palettes, the voice element, the gallery). The engine's +own tests stay `swift test`. Anything with a rule in `DESIGN.md` should have a test here. + +The suite holds the same gates as the engine's: warnings are errors, a plain run fails when +`BlurtiOSCore` drops below the engine's own line-coverage floor (`MIN_IOS_COVERAGE`, 88%, in the +script — raise it as coverage grows, never lower it to land a change), and CI's `ios-sanitizers` +job runs it again under ThreadSanitizer and AddressSanitizer (`BLURT_IOS_SANITIZER=thread` or +`address` locally). `scripts/ios-check.sh` runs the whole iPhone bar in one go — the tests and +the coverage gate, then `swiftlint analyze` and periphery over this project — and is what CI's +`ios-build` job and a local `scripts/check.sh` run. The views are not in that figure, as the Mac shell's aren't in the engine's: +they are checked on sight, through the gallery and the probe's screenshot flows. + +## Testing in the simulator + +Signing and the team don't matter here, and the App Group works, so the whole +loop runs (`scripts/ios-lib.sh` holds the one signing recipe and the simulator pick). `AVCaptureSession` carries no audio in the simulator, so the app uses +`SimulatorAudioSource` (an Audio Queue on the Mac's microphone) there and +`WindowedAudioSource` on a phone. + +```bash +scripts/ios-sim.sh # generate, build, boot, install, grant mic, launch +scripts/ios-sim.sh --screenshot shot.png # …and capture the screen +``` + +Xcode 27 replaced Simulator.app with Device Hub (`Xcode.app/Contents/Applications/DeviceHub.app`), +which the script opens; turn off its **Always simulate hardware keyboard** setting (Device Hub → +Settings…) or no on-screen keyboard ever appears. Then: Blurt → Open Settings, and Apps → Blurt → +Keyboards → Blurt on, Allow Full Access (the simulator's Settings opens at its top level, not on +Blurt's page); Start listening; in Notes, hold the globe key, pick Blurt, tap the +mic. The keyboard's crash logs, if any, land in `~/Library/Logs/DiagnosticReports/BlurtKeyboard-*`, +and `xcrun simctl spawn booted log show --last 5m --predicate 'process == "BlurtKeyboard"'` +shows its console. + +## Testing on a phone + +1. Install Xcode 26.6 or newer — CI's runner has 26.6 (`xcodes install 26.6 --select`, or + the Mac App Store). A phone on iOS 27 needs Xcode 27, which needs macOS 26.6. Then once: + + ```bash + sudo xcode-select -s /Applications/Xcode.app + sudo xcodebuild -license accept + sudo xcodebuild -runFirstLaunch + xcodebuild -downloadPlatform iOS # only for the simulator; a phone needs no download + ``` + + `scripts/bootstrap.sh` from the repo root installs xcodegen and the linters if `brew` has not. + +2. Sign in to Xcode with your Apple ID (Settings → Accounts) and note your team ID (the ten + characters after the team's name). The App Group needs a paid team — a free Personal Team + cannot register one, so use the org's team, not your own. +3. `cd App/BlurtiOS && BLURT_TEAM= xcodegen generate && open BlurtiOS.xcodeproj`. + Automatic signing registers `dev.alex.blurt.ios`, `dev.alex.blurt.ios.keyboard` and + `group.dev.alex.blurt` under that team on the first build; it fails if another team already + owns them. +4. Plug the phone in (Developer Mode on: Settings → Privacy & Security), pick it as the run + destination, run the `BlurtiOS` scheme. On the phone, trust the developer app (Settings → + General → VPN & Device Management). In the app: **Use an API key instead** (debug builds + only), **Allow** the microphone, then **Open Settings** and go to **Apps › Blurt › Keyboards** + (a phone may open straight on Blurt's page): turn on Blurt and **Allow Full Access**. +5. Back in the app, **Start listening**, then go to Messages, hold the globe key, pick Blurt, + and tap the mic. Tap again to stop, or hold it while you talk. The words land in the field. +6. Try the other two layouts from the app's Keyboard picker; the keyboard reads the choice the + next time it comes up. + +## What is still a stub + +- **Autocorrect and the suggestion bar** on the full keyboard. + +- **Sign in with AssemblyAI.** The shipping app signs the user in and gets a key + behind the scenes; that service does not exist yet. Debug builds show "Use an API + key instead" (`KeyEntryView`) so the pipeline can be tested now. +- **Returning to the host app.** After "Start Blurt" the user swipes back; iOS 26.4 + ended automatic switch-back for everyone. +- **The full keyboard** has no autocorrect or suggestions yet. + +## What to verify on a phone first + +The listening window's life in the background (capture keeps flowing after the +swipe back; battery over an hour); keyboard → app → back on iOS 26 and 27; the +keyboard's memory with the orb growing into the wave; how long the lexicon takes on a phone +with thousands of contacts; and, once the service exists, sign-in end to end. diff --git a/App/BlurtiOS/Shared/BlurtBrand.swift b/App/BlurtiOS/Shared/BlurtBrand.swift new file mode 100644 index 00000000..99da6498 --- /dev/null +++ b/App/BlurtiOS/Shared/BlurtBrand.swift @@ -0,0 +1,20 @@ +/// Blurt's brand, as the Mac app defines it (`App/Blurt/Blurt/Branding/BlurtBrand.swift`, +/// from the 2026-08-31 comps), plus the handful of shades the keyboard needs +/// for a surface the Mac never draws. Compiled into the app and the keyboard. +/// +/// The values live in `Design/tokens.json` and arrive here through the +/// generated `DesignTokens`; these are the Mac's names for them, kept so the +/// call sites still read in the Mac's vocabulary. Add a colour to the tokens, +/// not here. +/// +/// The keyboard's surface is fixed per theme, the way the Mac's overlay pill is +/// fixed: it floats over whatever app the user is typing in, so it cannot take +/// its cue from that app's appearance, and it must read the same on a white +/// Notes page and a black Messages thread. Everything on it therefore names +/// the raw shades — `greenOnDark`, `ink` — and never `accent`. +enum BlurtBrand { + /// `#01762F` — the wordmark green, for light chrome. + nonisolated static let green = DesignTokens.Brand.green700 + /// `#E67F36` — the error word. The body stays ink; the word carries the alarm. + nonisolated static let errorOrange = DesignTokens.Brand.orange +} diff --git a/App/BlurtiOS/Shared/BlurtType.swift b/App/BlurtiOS/Shared/BlurtType.swift new file mode 100644 index 00000000..43b19c1e --- /dev/null +++ b/App/BlurtiOS/Shared/BlurtType.swift @@ -0,0 +1,22 @@ +import SwiftUI + +/// The type roles the app and the keyboard draw with, in the system's own +/// faces — no font is bundled. Each role keeps the design it stands for: mono +/// for eyebrows, CTAs and the keyboard's word labels; serif for the app's +/// headlines; the default face for body text. Call sites name the role, never +/// a face. Fixed point sizes throughout, as the system keyboard's are — no +/// Dynamic Type. +enum BlurtType { + /// Eyebrows, CTAs, the keyboard's word labels. + nonisolated static func mono(_ size: CGFloat, weight: Font.Weight = .regular) -> Font { + .system(size: size, weight: weight, design: .monospaced) + } + + /// The app's headlines, sentence case. + nonisolated static func heading(_ size: CGFloat) -> Font { .system(size: size, design: .serif) } + + /// The app's body text. + nonisolated static func body(_ size: CGFloat, bold: Bool = false) -> Font { + .system(size: size, weight: bold ? .bold : .regular) + } +} diff --git a/App/BlurtiOS/Shared/DesignTokens.swift b/App/BlurtiOS/Shared/DesignTokens.swift new file mode 100644 index 00000000..dad8de06 --- /dev/null +++ b/App/BlurtiOS/Shared/DesignTokens.swift @@ -0,0 +1,750 @@ +// Generated by scripts/design-tokens.swift from App/BlurtiOS/Design/tokens.json — do not edit. +// Regenerate with scripts/design-sync.sh; scripts/check.sh fails on drift. +// periphery:ignore:all +// swiftlint:disable file_length +import SwiftUI + +// swiftlint:disable type_body_length +/// The design's colours and numbers, from their one source +/// (`Design/tokens.json`): what the views draw with, what DESIGN.md's tables +/// say, and what the asset catalog holds, all from the same file. Swift names +/// follow the token names by one rule (`kb/key-modifier` → `kbKeyModifier`). +enum DesignTokens { + enum Brand { + /// The iPhone keyboard's dark key. + nonisolated static let appleDarkKey: Color = Color(hex: 0x6B6B6B) + /// The iPhone keyboard's dark modifier key. + nonisolated static let appleDarkModifier: Color = Color(hex: 0x464646) + /// The iPhone keyboard's dark surface. + nonisolated static let appleDarkSurface: Color = Color(hex: 0x2B2B2B) + /// The iPhone keyboard's light modifier key. + nonisolated static let appleLightModifier: Color = Color(hex: 0xADB3BC) + /// The iPhone keyboard's light surface. + nonisolated static let appleLightSurface: Color = Color(hex: 0xD1D5DB) + nonisolated static let black: Color = Color(hex: 0x000000) + /// The warm card fill; the paper theme's surface. + nonisolated static let card: Color = Color(hex: 0xEBE8E8) + /// The card's hairline; the paper theme's modifier key. + nonisolated static let cardBorder: Color = Color(hex: 0xDEDBDB) + /// AssemblyAI's primary violet: the drop, never a key or a surface. + nonisolated static let cobolt: Color = Color(hex: 0x3923C7) + /// The design system's UI-and-code green fill (unused on the phone yet). + nonisolated static let green100: Color = Color(hex: 0xCCE4D5) + /// The design system's UI-and-code green highlight (unused on the phone yet). + nonisolated static let green200: Color = Color(hex: 0x99C8AC) + /// The brand hue lifted for dark chrome: the meter, the ring, the wave on ink. + nonisolated static let green400: Color = Color(hex: 0x67AD82) + /// The wordmark green, light chrome (the app's accent in light). + nonisolated static let green700: Color = Color(hex: 0x01762F) + /// The pale green the orb lightens into. + nonisolated static let greenMist: Color = Color(hex: 0xDBF5E6) + /// The brand ink: the Mac pill's body, the ink theme's surface. + nonisolated static let ink: Color = Color(hex: 0x1D1B16) + /// The Mac's dark card border. + nonisolated static let ink600: Color = Color(hex: 0x3A362F) + /// An ordinary key on ink, one step up. + nonisolated static let ink700: Color = Color(hex: 0x33302A) + /// A modifier key on ink; the Mac's dark card fill. + nonisolated static let ink800: Color = Color(hex: 0x26231E) + /// A key on the system's dark keyboard material (measured, iOS 27): the container case. + nonisolated static let materialDarkKey: Color = Color(hex: 0x575757) + /// A modifier key on the system's dark material. + nonisolated static let materialDarkModifier: Color = Color(hex: 0x474747) + /// The system's dark keyboard material (measured, iOS 27). + nonisolated static let materialDarkSurface: Color = Color(hex: 0x373737) + /// The system's light keyboard material (measured, iOS 27): what the gallery stands under a clear surface. + nonisolated static let materialLightSurface: Color = Color(hex: 0xE2E3E8) + /// Black-100: the lightest warm grey. + nonisolated static let neutral300: Color = Color(hex: 0xD2D1D0) + /// Black-200: mono labels on the dark face. + nonisolated static let neutral500: Color = Color(hex: 0xA5A4A2) + /// Black-300: muted text and mono labels on the light face. + nonisolated static let neutral700: Color = Color(hex: 0x777673) + /// Black-400: body text on the light face. + nonisolated static let neutral900: Color = Color(hex: 0x4A4945) + /// The error word and ring, cancel, the Full Access note; never a red body. + nonisolated static let orange: Color = Color(hex: 0xE67F36) + /// Neutral-100: the light face's surface. + nonisolated static let paper200: Color = Color(hex: 0xECEBE5) + /// Neutral-200: a modifier key on the light face. + nonisolated static let paper300: Color = Color(hex: 0xDAD7CB) + /// Neutral-300: hairlines and field borders on the light face. + nonisolated static let paper400: Color = Color(hex: 0xC7C3B2) + /// The design system's page: the app's light ground. + nonisolated static let paperPage: Color = Color(hex: 0xFDFCF8) + /// The warm off-white one step in: light cards, dark legends. + nonisolated static let paperTint: Color = Color(hex: 0xF5F3EB) + /// The orb gradient's upper violet stop. + nonisolated static let violetIris: Color = Color(hex: 0x887BDD) + /// The orb's lightest violet: its top and the gradient's ends. + nonisolated static let violetLavender: Color = Color(hex: 0xD7D3F4) + /// The orb's mid violet: the drop's halo. + nonisolated static let violetPeriwinkle: Color = Color(hex: 0xB0A7E9) + /// Key legends on ink: warm white on warm ink. + nonisolated static let warmWhite: Color = Color(hex: 0xF2EEE6) + nonisolated static let white: Color = Color(hex: 0xFFFFFF) + } + + enum Themes { + /// `brand.material/dark-key`. + nonisolated static let darkContainerKey: Color = Brand.materialDarkKey + /// `brand.material/dark-modifier`. + nonisolated static let darkContainerKeyModifier: Color = Brand.materialDarkModifier + /// `brand.ink/700`. + nonisolated static let darkField: Color = Brand.ink700 + /// `brand.ink/600`. + nonisolated static let darkFieldBorder: Color = Brand.ink600 + /// `brand.ink/700`. + nonisolated static let darkKey: Color = Brand.ink700 + /// `brand.ink/800`. + nonisolated static let darkKeyModifier: Color = Brand.ink800 + /// `brand.paper/tint`. + nonisolated static let darkLegend: Color = Brand.paperTint + /// `brand.neutral/500`. + nonisolated static let darkLegendSecondary: Color = Brand.neutral500 + /// `brand.material/dark-surface`. + nonisolated static let darkMaterial: Color = Brand.materialDarkSurface + /// `brand.orange`. + nonisolated static let darkNotice: Color = Brand.orange + /// `brand.ink/700`. + nonisolated static let darkPopup: Color = Brand.ink700 + /// `brand.green/400`. + nonisolated static let darkSignal: Color = Brand.green400 + /// `brand.ink`. + nonisolated static let darkSurface: Color = Brand.ink + /// `brand.white`. + nonisolated static let lightContainerKey: Color = Brand.white + /// `brand.paper/300`. + nonisolated static let lightContainerKeyModifier: Color = Brand.paper300 + /// `brand.white`. + nonisolated static let lightField: Color = Brand.white + /// `brand.paper/400`. + nonisolated static let lightFieldBorder: Color = Brand.paper400 + /// `brand.white`. + nonisolated static let lightKey: Color = Brand.white + /// `brand.paper/300`. + nonisolated static let lightKeyModifier: Color = Brand.paper300 + /// `brand.ink`. + nonisolated static let lightLegend: Color = Brand.ink + /// `brand.neutral/700`. + nonisolated static let lightLegendSecondary: Color = Brand.neutral700 + /// `brand.material/light-surface`. + nonisolated static let lightMaterial: Color = Brand.materialLightSurface + /// `brand.orange`. + nonisolated static let lightNotice: Color = Brand.orange + /// `brand.white`. + nonisolated static let lightPopup: Color = Brand.white + /// `brand.green/700`. + nonisolated static let lightSignal: Color = Brand.green700 + /// `brand.paper/200`. + nonisolated static let lightSurface: Color = Brand.paper200 + } + + enum Keyboard { + /// The catalog's AccentColor in dark. + nonisolated static let appAccentDark: Color = Brand.green400 + /// The catalog's AccentColor in light. + nonisolated static let appAccentLight: Color = Brand.green700 + /// The catalog's CardBorder in dark. + nonisolated static let appCardBorderDark: Color = Brand.ink600 + /// The catalog's CardBorder in light. + nonisolated static let appCardBorderLight: Color = Brand.paper400 + /// The catalog's CardFill in dark. + nonisolated static let appCardFillDark: Color = Brand.ink800 + /// The catalog's CardFill in light. + nonisolated static let appCardFillLight: Color = Brand.paperTint + /// The catalog's CTA in dark: the site's button green. + nonisolated static let appCtaDark: Color = Brand.green400 + /// The catalog's CTA in light: the one green button. + nonisolated static let appCtaLight: Color = Brand.green700 + /// The catalog's CTAText in dark: ink on green, as the site. + nonisolated static let appCtaTextDark: Color = Brand.ink + /// The catalog's CTAText in light. + nonisolated static let appCtaTextLight: Color = Brand.white + /// The catalog's Muted in dark. + nonisolated static let appMutedDark: Color = Brand.neutral500 + /// The catalog's Muted in light: eyebrows, captions, secondary text. + nonisolated static let appMutedLight: Color = Brand.neutral700 + /// The catalog's Page in dark: ink, the website's ground. + nonisolated static let appPageDark: Color = Brand.ink + /// The catalog's Page in light: the design system's page. + nonisolated static let appPageLight: Color = Brand.paperPage + /// The catalog's Text in dark. + nonisolated static let appTextDark: Color = Brand.paperTint + /// The catalog's Text in light. + nonisolated static let appTextLight: Color = Brand.ink + /// The panel's cancel ×. + nonisolated static let kbCancel: Color = Brand.orange + /// The key-term field. + nonisolated static let kbField: Color = Themes.darkField + /// The field's hairline. + nonisolated static let kbFieldBorder: Color = Themes.darkFieldBorder + /// The one line of words the keyboard ever shows. + nonisolated static let kbFullAccessNote: Color = Brand.orange + /// An ordinary key. + nonisolated static let kbKey: Color = Themes.darkKey + /// Shift, delete, globe, return, 123, cancel. + nonisolated static let kbKeyModifier: Color = Themes.darkKeyModifier + /// Key legends and bare glyphs. + nonisolated static let kbLegend: Color = Themes.darkLegend + /// The mono word labels (123, ABC, return, space) and the + at rest. + nonisolated static let kbLegendSecondary: Color = Themes.darkLegendSecondary + /// The solid ring and glyph for an error. + nonisolated static let kbNoticeError: Color = Brand.orange + /// The letter pop-up. + nonisolated static let kbPopup: Color = Themes.darkPopup + /// The wave, the caret, the saved check. + nonisolated static let kbSignal: Color = Themes.darkSignal + /// The design face's surface (Swift reads the palette). + nonisolated static let kbSurface: Color = Themes.darkSurface + } + + enum Metrics { + /// Between the chip's glyph and the word. + nonisolated static let addtermChipGap: CGFloat = 6 + /// The least room a chip needs beside a centred mic; with less, the mic and the chip centre as a pair. + nonisolated static let addtermChipMinWidth: CGFloat = 96 + /// The selected-word chip's side padding. + nonisolated static let addtermChipPad: CGFloat = 10 + /// The cancel × from the panel's far corner when the mic is at an edge. + nonisolated static let addtermInset: CGFloat = 6 + /// The brand's button. + nonisolated static let appButtonHeight: CGFloat = 40 + /// Its side padding. + nonisolated static let appButtonPad: CGFloat = 12 + /// Inside a card. + nonisolated static let appCardPad: CGFloat = 16 + /// Between chips. + nonisolated static let appChipGap: CGFloat = 8 + /// A style chip's side padding. + nonisolated static let appChipPadX: CGFloat = 12 + /// And its top and bottom. + nonisolated static let appChipPadY: CGFloat = 8 + /// The home header: wordmark and gear. + nonisolated static let appHeaderHeight: CGFloat = 44 + /// Between the hero's pieces. + nonisolated static let appHeroGap: CGFloat = 14 + /// Inside the hero card. + nonisolated static let appHeroPad: CGFloat = 20 + /// The header's gear, the copy glyph. + nonisolated static let appIcon: CGFloat = 20 + /// Between lines of one thought. + nonisolated static let appLineGap: CGFloat = 6 + /// The screens' side padding. + nonisolated static let appPagePad: CGFloat = 20 + /// Between the theme card's two faces. + nonisolated static let appPreviewGap: CGFloat = 8 + /// Between the home screen's sections. + nonisolated static let appSectionGap: CGFloat = 24 + /// The setup steps' mono numerals. + nonisolated static let appSetupNumberWidth: CGFloat = 24 + /// Between a section's eyebrow and its content, and between cards. + nonisolated static let appStackGap: CGFloat = 12 + /// The card's hairline. + nonisolated static let cardBorder: CGFloat = 1 + /// The app's cards (radius/card). + nonisolated static let cardRadius: CGFloat = 12 + nonisolated static let caretHeight: CGFloat = 20 + /// The caret's gap from the text. + nonisolated static let caretLead: CGFloat = 1 + nonisolated static let caretRadius: CGFloat = 1 + nonisolated static let caretWidth: CGFloat = 2 + /// The +, × and ✓ touch targets: the HIG's 44 pt square around a size/glyph symbol. + nonisolated static let glyphHit: CGFloat = 44 + nonisolated static let grilleBarHeight: CGFloat = 28 + /// The grille in the voice bar. + nonisolated static let grilleBarWidth: CGFloat = 64 + /// A dot's diameter. + nonisolated static let grilleDot: CGFloat = 2 + /// The cross glint's vertical arm, as a fraction of the grille's height. + nonisolated static let grilleGlintArm: CGFloat = 1.6 + /// The glint's arms. + nonisolated static let grilleGlintWidth: CGFloat = 1 + nonisolated static let grilleHomeHeight: CGFloat = 108 + /// The grille on the home screen. + nonisolated static let grilleHomeWidth: CGFloat = 240 + nonisolated static let grillePanelHeight: CGFloat = 72 + /// The grille in the panel. + nonisolated static let grillePanelWidth: CGFloat = 160 + /// The dot lattice: one dot every. + nonisolated static let grillePitch: CGFloat = 4 + /// The sheen's band, as a fraction of the width. + nonisolated static let grilleSheenWidth: CGFloat = 0.35 + /// 123 and the globe at 402 pt: the iPhone's. + nonisolated static let keyAbcWidth402: CGFloat = 43.333 + /// Between keys, the iPhone's. + nonisolated static let keyGap: CGFloat = 6 + /// Every key: the iPhone's cap height (apple-geometry.json). + nonisolated static let keyHeight: CGFloat = 43 + /// A letter key at 402 pt: (width − 2 × margin/side − 9 × key/gap) / 10. + nonisolated static let keyLetterWidth402: CGFloat = 33.5 + /// A key that isn't given a width: the iPhone's 123 key at 402 pt. + nonisolated static let keyMinWidth: CGFloat = 43.333 + /// A legend's side padding on a min-width key. + nonisolated static let keyPad: CGFloat = 4 + /// Flat keys, one colour at this corner: the iPhone's (fitted 8.17). + nonisolated static let keyRadius: CGFloat = 8 + /// The width the @402 tokens were measured at (iPhone 18 Pro); widths scale from it. + nonisolated static let keyReferenceWidth: CGFloat = 402 + /// Return at 402 pt: two 123 keys and a gap. + nonisolated static let keyReturnWidth402: CGFloat = 92.667 + /// Shift and delete stand this far from the letters: the iPhone's. + nonisolated static let keySideGap: CGFloat = 13.667 + /// Shift and delete at 402 pt: what seven letters and the side gaps leave, halved (measured 45.5). + nonisolated static let keySideWidth402: CGFloat = 45.583 + /// Space at 402 pt beside 123, the globe and return: what is left (measured 191.333). + nonisolated static let keySpaceWidth402: CGFloat = 191.667 + /// Margin/vertical + voicebar/height + 4 × key/height + 3 × row/gap + margin/bottom-keys. + nonisolated static let layoutFull: CGFloat = 270 + /// Chosen: room for the 96 orb over one key row. + nonisolated static let layoutPanel: CGFloat = 216 + /// 2 × margin/vertical + voicebar/height. + nonisolated static let layoutSlim: CGFloat = 60 + /// Under a bottom row of keys (the full keyboard, the panel): the iPhone's. + nonisolated static let marginBottomKeys: CGFloat = 13 + /// At the keyboard's sides: the iPhone's cap margin, (402 − 10 letters − 9 gaps) / 2. + nonisolated static let marginSide: CGFloat = 6.5 + /// The keyboard's top, and the slim bar's bottom. + nonisolated static let marginVertical: CGFloat = 8 + /// The + without Full Access. + nonisolated static let opacityDisabled: CGFloat = 0.4 + /// The cross glint when the words land. + nonisolated static let opacityGlint: CGFloat = 0.95 + /// Film grain over the dark face: matte, the brand's texture. + nonisolated static let opacityGrainDark: CGFloat = 0.14 + /// Film grain over the light face: paper, lighter. + nonisolated static let opacityGrainLight: CGFloat = 0.08 + /// The dots at rest, over the key colour. + nonisolated static let opacityGrilleRest: CGFloat = 0.55 + nonisolated static let opacityHairline: CGFloat = 0.5 + /// Key legends over the cap; below 1 they sit back. + nonisolated static let opacityLegend: CGFloat = 1 + /// The + at rest. + nonisolated static let opacityLegendMuted: CGFloat = 0.5 + /// The element when Blurt isn't ready — dim and still, but there: a tap opens the app. + nonisolated static let opacityOff: CGFloat = 0.6 + /// The field's placeholder. + nonisolated static let opacityPlaceholder: CGFloat = 0.4 + nonisolated static let opacityPopupShadow: CGFloat = 0.12 + /// A key lightens this much while pressed. + nonisolated static let opacityPressBrighten: CGFloat = 0.15 + /// A brand button while pressed: a colour change, no lift. + nonisolated static let opacityPressed: CGFloat = 0.85 + /// The light passing over the element at rest: chrome catching light. + nonisolated static let opacitySheen: CGFloat = 0.22 + /// A soft darkening toward the surface's edges; 0 is none. + nonisolated static let opacitySurfaceVignette: CGFloat = 0 + /// The field's ×. + nonisolated static let opacityTermCancel: CGFloat = 0.7 + /// Under the panel's voice row, above its keys: the top margin again, so the mic is centred between the keyboard's top edge and the keys. + nonisolated static let panelSpacing: CGFloat = 8 + /// The theme card draws the keyboard at this width. + nonisolated static let pickerPreviewWidth: CGFloat = 393 + /// The theme card's preview corners. + nonisolated static let pickerRadius: CGFloat = 10 + /// Then scales it to fit two across. + nonisolated static let pickerScale: CGFloat = 0.42 + /// The letter pop-up is the key width plus this. + nonisolated static let popupExtraWidth: CGFloat = 18 + nonisolated static let popupHeight: CGFloat = 56 + /// The pop-up rises this far from the key's top: it overlaps the key by popup/height minus this and stays inside the keyboard on the top row. + nonisolated static let popupOffset: CGFloat = 50 + nonisolated static let popupRadius: CGFloat = 10 + /// The one thing that floats. + nonisolated static let popupShadowRadius: CGFloat = 8 + nonisolated static let popupShadowY: CGFloat = 2 + /// The brand's buttons and the grille: rectangular, never a pill. + nonisolated static let radiusButton: CGFloat = 4 + /// The brand's cards. + nonisolated static let radiusCard: CGFloat = 12 + /// The brand's featured cards. + nonisolated static let radiusHero: CGFloat = 16 + /// The brand's inputs: the key-term field. + nonisolated static let radiusInput: CGFloat = 8 + nonisolated static let ribsRestHeight: CGFloat = 16 + /// The ribs asleep: the wave's own bars at rest. + nonisolated static let ribsRestWidth: CGFloat = 40 + /// Between rows, the iPhone's. + nonisolated static let rowGap: CGFloat = 11 + /// The landing flash's vertical arm, as a fraction of the box's height. + nonisolated static let streakArm: CGFloat = 1.4 + /// The light streak while recording. + nonisolated static let streakHeight: CGFloat = 2 + /// The hairline at rest. + nonisolated static let streakLine: CGFloat = 1 + /// The bright point at the centre. + nonisolated static let streakPoint: CGFloat = 4 + /// 1 paints the face's surface under the keys; 0 leaves it clear, so the host's own keyboard material (iOS 26's rounded container, which insets a third-party keyboard and cannot be painted over) is the surface, seamlessly, and the keys take the container colours. + nonisolated static let surfacePaint: CGFloat = 0 + /// Where it is full. + nonisolated static let surfaceVignetteEnd: CGFloat = 0.75 + /// Where the vignette begins, as a fraction of the keyboard's width. + nonisolated static let surfaceVignetteStart: CGFloat = 0.3 + /// The field's hairline. + nonisolated static let termBorder: CGFloat = 1 + /// × · field · ✓. + nonisolated static let termGap: CGFloat = 8 + /// The key-term field's capsule. + nonisolated static let termHeight: CGFloat = 36 + /// The field row's side inset. + nonisolated static let termInset: CGFloat = 2 + /// The field's side padding. + nonisolated static let termPad: CGFloat = 14 + /// The key-term field: the brand's input corners (radius/input). + nonisolated static let termRadius: CGFloat = 8 + nonisolated static let voiceBarHeight: CGFloat = 32 + /// The voice element's box in the voice bar — the most it takes (the ribs awake, the streak); the + sits beside what shows. + nonisolated static let voiceBarWidth: CGFloat = 160 + nonisolated static let voiceHomeHeight: CGFloat = 112 + /// The box on the home screen. + nonisolated static let voiceHomeWidth: CGFloat = 280 + nonisolated static let voicePanelHeight: CGFloat = 88 + /// The box in the panel. + nonisolated static let voicePanelWidth: CGFloat = 300 + /// The element while pressed. + nonisolated static let voicePressScale: CGFloat = 0.94 + /// Between the mic key's edge and the add key's box (the + or the chip), the key gap. + nonisolated static let voicebarAddtermClearance: CGFloat = 6 + /// The voice row, where the system puts its suggestion bar. + nonisolated static let voicebarHeight: CGFloat = 44 + /// Between the orb and the Full Access note. + nonisolated static let voicebarNoteGap: CGFloat = 10 + /// A wave bar's width. + nonisolated static let waveBar: CGFloat = 2 + nonisolated static let waveBarHeight: CGFloat = 24 + /// Between wave bars. + nonisolated static let waveGap: CGFloat = 2 + nonisolated static let waveHomeHeight: CGFloat = 44 + nonisolated static let wavePanelHeight: CGFloat = 40 + nonisolated static let wordmarkHeight: CGFloat = 22 + } + + enum Typography { + /// The clipboard and exclamation glyphs, as a fraction of the voice element's height. + nonisolated static let ratioVoiceGlyph: CGFloat = 0.34 + /// The app's body text. + nonisolated static let sizeBody: CGFloat = 16 + /// The app's small text. + nonisolated static let sizeCaption: CGFloat = 14 + /// A mono button label in the app (E2). + nonisolated static let sizeCta: CGFloat = 14 + /// A mono eyebrow in the app (E1). + nonisolated static let sizeEyebrow: CGFloat = 12 + /// The +, × and ✓. + nonisolated static let sizeGlyph: CGFloat = 17 + /// The mono word labels on keys: 123, ABC, #+=, space, the return label. + nonisolated static let sizeLabel: CGFloat = 12 + /// Every other key. + nonisolated static let sizeLegend: CGFloat = 16 + /// Letter keys: the iPhone's (estimated from the l glyph, apple-geometry.json). + nonisolated static let sizeLetter: CGFloat = 24 + /// The letter pop-up. + nonisolated static let sizePopup: CGFloat = 32 + /// The key-term field. + nonisolated static let sizeTerm: CGFloat = 17 + /// The app's serif headline. + nonisolated static let sizeTitle: CGFloat = 34 + /// Uppercase mono at 14: the brand's E2 tracking. + nonisolated static let trackingCta: CGFloat = 1.4 + /// Uppercase mono at 12: the brand's E1 tracking. + nonisolated static let trackingEyebrow: CGFloat = 1.2 + nonisolated static let weightGlyph: Font.Weight = .medium + nonisolated static let weightLabel: Font.Weight = .medium + nonisolated static let weightLegend: Font.Weight = .medium + nonisolated static let weightLetter: Font.Weight = .regular + nonisolated static let weightPopup: Font.Weight = .regular + nonisolated static let weightTerm: Font.Weight = .regular + nonisolated static let weightVoiceGlyph: Font.Weight = .semibold + } + + enum Motion { + /// The caret's blink. + nonisolated static let caret: Double = 0.5 + /// Holding the chip opens the field instead of adding; longer than a firm tap. + nonisolated static let chipHold: Double = 0.6 + /// A colour or fill changing state: the brand's transition. + nonisolated static let colour: Double = 0.2 + /// The house curve for everything that moves; nothing springs. + nonisolated static let easeSignature: UnitCurve = UnitCurve.bezier( + startControlPoint: UnitPoint(x: 0.22, y: 1), endControlPoint: UnitPoint(x: 0.36, y: 1)) + /// The panel's carousel. + nonisolated static let flip: Double = 0.25 + /// A facet's flash while recording. + nonisolated static let glintLife: Double = 0.5 + /// The keyboard resizing. + nonisolated static let heightChange: Double = 0.25 + /// A key lighting. + nonisolated static let keyPress: Double = 0.08 + /// The glint when the words land. + nonisolated static let landing: Double = 0.6 + /// The orb's press; the one thing that answers at once. + nonisolated static let press: Double = 0.1 + /// One turn of the ring, the Mac's cadence. + nonisolated static let ringPeriod: Double = 1.6 + /// One pass of the light over the element at rest. + nonisolated static let sheen: Double = 4 + /// And while something is happening: the ring's cadence. + nonisolated static let sheenWorking: Double = 1.6 + /// Every other change on the key: the ring, a glyph, the dimming. + nonisolated static let stateFade: Double = 0.5 + /// The voice bar becoming the field. + nonisolated static let termSwap: Double = 0.4 + /// The orb and the wave crossing, either way. + nonisolated static let waveFade: Double = 0.7 + } + + enum Gradients { + } + + /// Every token as `group.name: value`, the way tokens.json renders it, so a + /// test can pin the compiled values to the file (`DesignTokensTests`). + nonisolated static let manifest: [String: String] = [ + "brand.apple/dark-key": "#6B6B6B", + "brand.apple/dark-modifier": "#464646", + "brand.apple/dark-surface": "#2B2B2B", + "brand.apple/light-modifier": "#ADB3BC", + "brand.apple/light-surface": "#D1D5DB", + "brand.black": "#000000", + "brand.card": "#EBE8E8", + "brand.card-border": "#DEDBDB", + "brand.cobolt": "#3923C7", + "brand.green/100": "#CCE4D5", + "brand.green/200": "#99C8AC", + "brand.green/400": "#67AD82", + "brand.green/700": "#01762F", + "brand.green/mist": "#DBF5E6", + "brand.ink": "#1D1B16", + "brand.ink/600": "#3A362F", + "brand.ink/700": "#33302A", + "brand.ink/800": "#26231E", + "brand.material/dark-key": "#575757", + "brand.material/dark-modifier": "#474747", + "brand.material/dark-surface": "#373737", + "brand.material/light-surface": "#E2E3E8", + "brand.neutral/300": "#D2D1D0", + "brand.neutral/500": "#A5A4A2", + "brand.neutral/700": "#777673", + "brand.neutral/900": "#4A4945", + "brand.orange": "#E67F36", + "brand.paper/200": "#ECEBE5", + "brand.paper/300": "#DAD7CB", + "brand.paper/400": "#C7C3B2", + "brand.paper/page": "#FDFCF8", + "brand.paper/tint": "#F5F3EB", + "brand.violet/iris": "#887BDD", + "brand.violet/lavender": "#D7D3F4", + "brand.violet/periwinkle": "#B0A7E9", + "brand.warm-white": "#F2EEE6", + "brand.white": "#FFFFFF", + "themes.dark/container-key": "#575757", + "themes.dark/container-key-modifier": "#474747", + "themes.dark/field": "#33302A", + "themes.dark/field-border": "#3A362F", + "themes.dark/key": "#33302A", + "themes.dark/key-modifier": "#26231E", + "themes.dark/legend": "#F5F3EB", + "themes.dark/legend-secondary": "#A5A4A2", + "themes.dark/material": "#373737", + "themes.dark/notice": "#E67F36", + "themes.dark/popup": "#33302A", + "themes.dark/signal": "#67AD82", + "themes.dark/surface": "#1D1B16", + "themes.light/container-key": "#FFFFFF", + "themes.light/container-key-modifier": "#DAD7CB", + "themes.light/field": "#FFFFFF", + "themes.light/field-border": "#C7C3B2", + "themes.light/key": "#FFFFFF", + "themes.light/key-modifier": "#DAD7CB", + "themes.light/legend": "#1D1B16", + "themes.light/legend-secondary": "#777673", + "themes.light/material": "#E2E3E8", + "themes.light/notice": "#E67F36", + "themes.light/popup": "#FFFFFF", + "themes.light/signal": "#01762F", + "themes.light/surface": "#ECEBE5", + "keyboard.app/accent-dark": "#67AD82", + "keyboard.app/accent-light": "#01762F", + "keyboard.app/card-border-dark": "#3A362F", + "keyboard.app/card-border-light": "#C7C3B2", + "keyboard.app/card-fill-dark": "#26231E", + "keyboard.app/card-fill-light": "#F5F3EB", + "keyboard.app/cta-dark": "#67AD82", + "keyboard.app/cta-light": "#01762F", + "keyboard.app/cta-text-dark": "#1D1B16", + "keyboard.app/cta-text-light": "#FFFFFF", + "keyboard.app/muted-dark": "#A5A4A2", + "keyboard.app/muted-light": "#777673", + "keyboard.app/page-dark": "#1D1B16", + "keyboard.app/page-light": "#FDFCF8", + "keyboard.app/text-dark": "#F5F3EB", + "keyboard.app/text-light": "#1D1B16", + "keyboard.kb/cancel": "#E67F36", + "keyboard.kb/field": "#33302A", + "keyboard.kb/field-border": "#3A362F", + "keyboard.kb/full-access-note": "#E67F36", + "keyboard.kb/key": "#33302A", + "keyboard.kb/key-modifier": "#26231E", + "keyboard.kb/legend": "#F5F3EB", + "keyboard.kb/legend-secondary": "#A5A4A2", + "keyboard.kb/notice-error": "#E67F36", + "keyboard.kb/popup": "#33302A", + "keyboard.kb/signal": "#67AD82", + "keyboard.kb/surface": "#1D1B16", + "metrics.addterm/chip-gap": "6", + "metrics.addterm/chip-min-width": "96", + "metrics.addterm/chip-pad": "10", + "metrics.addterm/inset": "6", + "metrics.app/button-height": "40", + "metrics.app/button-pad": "12", + "metrics.app/card-pad": "16", + "metrics.app/chip-gap": "8", + "metrics.app/chip-pad-x": "12", + "metrics.app/chip-pad-y": "8", + "metrics.app/header-height": "44", + "metrics.app/hero-gap": "14", + "metrics.app/hero-pad": "20", + "metrics.app/icon": "20", + "metrics.app/line-gap": "6", + "metrics.app/page-pad": "20", + "metrics.app/preview-gap": "8", + "metrics.app/section-gap": "24", + "metrics.app/setup-number-width": "24", + "metrics.app/stack-gap": "12", + "metrics.card/border": "1", + "metrics.card/radius": "12", + "metrics.caret/height": "20", + "metrics.caret/lead": "1", + "metrics.caret/radius": "1", + "metrics.caret/width": "2", + "metrics.glyph/hit": "44", + "metrics.grille/bar-height": "28", + "metrics.grille/bar-width": "64", + "metrics.grille/dot": "2", + "metrics.grille/glint-arm": "1.6", + "metrics.grille/glint-width": "1", + "metrics.grille/home-height": "108", + "metrics.grille/home-width": "240", + "metrics.grille/panel-height": "72", + "metrics.grille/panel-width": "160", + "metrics.grille/pitch": "4", + "metrics.grille/sheen-width": "0.35", + "metrics.key/abc-width-402": "43.333", + "metrics.key/gap": "6", + "metrics.key/height": "43", + "metrics.key/letter-width-402": "33.5", + "metrics.key/min-width": "43.333", + "metrics.key/pad": "4", + "metrics.key/radius": "8", + "metrics.key/reference-width": "402", + "metrics.key/return-width-402": "92.667", + "metrics.key/side-gap": "13.667", + "metrics.key/side-width-402": "45.583", + "metrics.key/space-width-402": "191.667", + "metrics.layout/full": "270", + "metrics.layout/panel": "216", + "metrics.layout/slim": "60", + "metrics.margin/bottom-keys": "13", + "metrics.margin/side": "6.5", + "metrics.margin/vertical": "8", + "metrics.opacity/disabled": "0.4", + "metrics.opacity/glint": "0.95", + "metrics.opacity/grain-dark": "0.14", + "metrics.opacity/grain-light": "0.08", + "metrics.opacity/grille-rest": "0.55", + "metrics.opacity/hairline": "0.5", + "metrics.opacity/legend": "1", + "metrics.opacity/legend-muted": "0.5", + "metrics.opacity/off": "0.6", + "metrics.opacity/placeholder": "0.4", + "metrics.opacity/popup-shadow": "0.12", + "metrics.opacity/press-brighten": "0.15", + "metrics.opacity/pressed": "0.85", + "metrics.opacity/sheen": "0.22", + "metrics.opacity/surface-vignette": "0", + "metrics.opacity/term-cancel": "0.7", + "metrics.panel/spacing": "8", + "metrics.picker/preview-width": "393", + "metrics.picker/radius": "10", + "metrics.picker/scale": "0.42", + "metrics.popup/extra-width": "18", + "metrics.popup/height": "56", + "metrics.popup/offset": "50", + "metrics.popup/radius": "10", + "metrics.popup/shadow-radius": "8", + "metrics.popup/shadow-y": "2", + "metrics.radius/button": "4", + "metrics.radius/card": "12", + "metrics.radius/hero": "16", + "metrics.radius/input": "8", + "metrics.ribs/rest-height": "16", + "metrics.ribs/rest-width": "40", + "metrics.row/gap": "11", + "metrics.streak/arm": "1.4", + "metrics.streak/height": "2", + "metrics.streak/line": "1", + "metrics.streak/point": "4", + "metrics.surface/paint": "0", + "metrics.surface/vignette-end": "0.75", + "metrics.surface/vignette-start": "0.3", + "metrics.term/border": "1", + "metrics.term/gap": "8", + "metrics.term/height": "36", + "metrics.term/inset": "2", + "metrics.term/pad": "14", + "metrics.term/radius": "8", + "metrics.voice/bar-height": "32", + "metrics.voice/bar-width": "160", + "metrics.voice/home-height": "112", + "metrics.voice/home-width": "280", + "metrics.voice/panel-height": "88", + "metrics.voice/panel-width": "300", + "metrics.voice/press-scale": "0.94", + "metrics.voicebar/addterm-clearance": "6", + "metrics.voicebar/height": "44", + "metrics.voicebar/note-gap": "10", + "metrics.wave/bar": "2", + "metrics.wave/bar-height": "24", + "metrics.wave/gap": "2", + "metrics.wave/home-height": "44", + "metrics.wave/panel-height": "40", + "metrics.wordmark/height": "22", + "type.ratio/voice-glyph": "0.34", + "type.size/body": "16", + "type.size/caption": "14", + "type.size/cta": "14", + "type.size/eyebrow": "12", + "type.size/glyph": "17", + "type.size/label": "12", + "type.size/legend": "16", + "type.size/letter": "24", + "type.size/popup": "32", + "type.size/term": "17", + "type.size/title": "34", + "type.tracking/cta": "1.4", + "type.tracking/eyebrow": "1.2", + "type.weight/glyph": "medium", + "type.weight/label": "medium", + "type.weight/legend": "medium", + "type.weight/letter": "regular", + "type.weight/popup": "regular", + "type.weight/term": "regular", + "type.weight/voice-glyph": "semibold", + "motion.caret": "0.5", + "motion.chip-hold": "0.6", + "motion.colour": "0.2", + "motion.ease/signature": "0.22,1,0.36,1", + "motion.flip": "0.25", + "motion.glint-life": "0.5", + "motion.height-change": "0.25", + "motion.key-press": "0.08", + "motion.landing": "0.6", + "motion.press": "0.1", + "motion.ring-period": "1.6", + "motion.sheen": "4", + "motion.sheen-working": "1.6", + "motion.state-fade": "0.5", + "motion.term-swap": "0.4", + "motion.wave-fade": "0.7", + ] +} +// swiftlint:enable type_body_length diff --git a/App/BlurtiOS/Shared/KeyboardLayout+Height.swift b/App/BlurtiOS/Shared/KeyboardLayout+Height.swift new file mode 100644 index 00000000..af27d156 --- /dev/null +++ b/App/BlurtiOS/Shared/KeyboardLayout+Height.swift @@ -0,0 +1,17 @@ +import BlurtiOSCore +import CoreFoundation + +extension KeyboardLayout { + /// The keyboard's height on screen, from the layout's rows at the iPhone + /// keyboard's own spacing: see `App/BlurtiOS/DESIGN.md` for the arithmetic. + var height: CGFloat { + let metrics = DesignTokens.Metrics.self + return switch self { + case .slimBar: 2 * metrics.marginVertical + metrics.voicebarHeight + case .panel: metrics.layoutPanel + case .full: + metrics.marginVertical + metrics.voicebarHeight + 4 * metrics.keyHeight + 3 * metrics.rowGap + + metrics.marginBottomKeys + } + } +} diff --git a/App/BlurtiOS/Shared/MicAlignment+Layout.swift b/App/BlurtiOS/Shared/MicAlignment+Layout.swift new file mode 100644 index 00000000..716bf598 --- /dev/null +++ b/App/BlurtiOS/Shared/MicAlignment+Layout.swift @@ -0,0 +1,21 @@ +import BlurtiOSCore +import SwiftUI + +extension MicAlignment { + /// The side of the row, as SwiftUI counts sides: left is the leading edge + /// in a left-to-right language and the trailing one in a right-to-left + /// one, so the mic stays under the thumb the user chose either way. + func alignment(in direction: LayoutDirection) -> Alignment { + switch self { + case .center: .center + case .left: direction == .leftToRight ? .leading : .trailing + case .right: direction == .leftToRight ? .trailing : .leading + } + } + + /// Whether the + goes before the mic in the row: at the trailing edge, so + /// the mic is the thing at the edge and the + sits on its inner side. + func plusLeads(in direction: LayoutDirection) -> Bool { + alignment(in: direction) == .trailing + } +} diff --git a/App/BlurtiOS/project.yml b/App/BlurtiOS/project.yml new file mode 100644 index 00000000..8f45c3f7 --- /dev/null +++ b/App/BlurtiOS/project.yml @@ -0,0 +1,279 @@ +# XcodeGen spec for Blurt on iPhone: the app (listens and transcribes) plus the +# keyboard it ships (drops the words into whatever app you are typing in). +# +# Like App/Blurt/project.yml this is the source of truth, but unlike it the +# generated BlurtiOS.xcodeproj is NOT committed: nobody opens a stale one, CI +# runs `xcodegen generate` right before it builds, and a local checkout does the +# same (`cd App/BlurtiOS && xcodegen generate`). The Info.plist and entitlements +# files below are generated from the `info:`/`entitlements:` blocks the same way. +# +# Two processes, one App Group. iOS lets no keyboard use the microphone, so the +# app records and talks to AssemblyAI while the keyboard is a remote control that +# inserts the result; they share `group.dev.alex.blurt` (see Shared/). Every id +# here is a placeholder namespace inherited from the Mac app — the App Store +# submission needs org-owned bundle ids, an org App Group and a team that can +# provision them. +name: BlurtiOS +options: + bundleIdPrefix: dev.alex + deploymentTarget: + # Matches Package.swift's `.iOS(.v18)`: an app floor below the package's fails + # to resolve the dependency at all. + iOS: "18.0" + developmentLanguage: en +settings: + base: + SWIFT_VERSION: "6.0" + IPHONEOS_DEPLOYMENT_TARGET: "18.0" + # iPhone first. iPad needs its own keyboard sizing pass. + TARGETED_DEVICE_FAMILY: "1" + # Whoever generates the project signs with their own team: XcodeGen expands + # `${BLURT_TEAM}` from the environment (`BLURT_TEAM=XXXXXXXXXX xcodegen + # generate`). Unset, it stays a reference that resolves to no team, which is + # what CI wants — it builds with signing off. App Groups need a paid team. + DEVELOPMENT_TEAM: ${BLURT_TEAM} + CODE_SIGN_STYLE: Automatic + # One package access identifier for every target, so BlurtiOSCore's + # `package` declarations reach the app, the keyboard and the tests (and + # nothing outside them) without becoming `public` API. + SWIFT_PACKAGE_NAME: BlurtiOS +packages: + # The same local package the Mac app carries, reached from App/BlurtiOS. No + # remote packages: check.sh's dependency guard reads every App/*/project.yml. + BlurtEngine: + path: ../.. +targets: + # The iPhone code's logic, apart from its UI, the way BlurtEngine sits apart + # from the Mac shell: the App Group contract, the keyboard's model and rules, + # the voice state, term packs. No SwiftUI and no views (those stay in the app + # and the keyboard), and no audio: the keyboard links this, and it never + # hears anything. Static, so the app and the keyboard each carry their own + # copy alongside the engine's, with no framework to embed and no second copy + # of the engine inside one process. ios-test.sh gates its line coverage. + BlurtiOSCore: + type: framework.static + platform: iOS + sources: + - path: BlurtiOSCore + settings: + base: + PRODUCT_BUNDLE_IDENTIFIER: dev.alex.blurt.ios.core + GENERATE_INFOPLIST_FILE: YES + SWIFT_TREAT_WARNINGS_AS_ERRORS: YES + SWIFT_DEFAULT_ACTOR_ISOLATION: MainActor + # The keyboard links it, so only extension-safe API. + APPLICATION_EXTENSION_API_ONLY: YES + dependencies: + - package: BlurtEngine + product: BlurtEngine + BlurtiOS: + type: application + platform: iOS + sources: + - path: BlurtiOS/Sources + # The Mac's brand colours (accent, card fill and border), copied verbatim + # from App/Blurt/Blurt/Assets.xcassets — JSON can't reference JSON. + - path: BlurtiOS/Assets.xcassets + # Apple's required-reason declarations: App Group UserDefaults (1C8F.1) + # and the bundle's own (CA92.1), the only such API either target uses. + - path: BlurtiOS/PrivacyInfo.xcprivacy + buildPhase: resources + # The Mac's app icon and ready-screen wordmark, shared rather than copied: + # the Icon Composer file targets iOS too (`squares: shared`). + - path: ../Blurt/Blurt/AppIcon.icon + - path: ../Blurt/Blurt/Branding/blurt-ready-logo.png + buildPhase: resources + # The UI both the app and the keyboard draw with (design tokens, type, + # brand colours), compiled into each — an extension can't import the + # app's module. The App Group contract itself is BlurtiOSCore. + - path: Shared + # The keyboard's views and model: the theme picker draws the real keyboard + # on its cards, the home screen draws the orb and the meter, and debug + # builds render every layout and state (KeyboardGalleryView) for design + # work and screenshots. Only the extension's entry point stays out; the + # keyboard itself never runs here. + - path: BlurtKeyboard/Sources + excludes: + - KeyboardViewController.swift + settings: + base: + PRODUCT_BUNDLE_IDENTIFIER: dev.alex.blurt.ios + ASSETCATALOG_COMPILER_APPICON_NAME: AppIcon + # The app-level accent, from the catalog, so system-drawn chrome and our + # views agree (see the Mac's project.yml and BlurtBrand). + ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME: AccentColor + # Warnings are failures for our own code, scoped to the target for the + # reason App/Blurt/project.yml gives (the project base collides with the + # -suppress-warnings Xcode applies to SPM packages). + SWIFT_TREAT_WARNINGS_AS_ERRORS: YES + # UI code defaults to the main actor (Swift 6.2), like the Mac shell; + # the capture path and the App Group plumbing opt out with `nonisolated`. + SWIFT_DEFAULT_ACTOR_ISOLATION: MainActor + info: + path: BlurtiOS/Info.plist + properties: + CFBundleDisplayName: Blurt + CFBundleShortVersionString: "0.1.0" + CFBundleVersion: "1" + LSApplicationCategoryType: public.app-category.productivity + NSMicrophoneUsageDescription: Blurt listens only while you dictate, and sends that audio to AssemblyAI to turn it into text. + # Keeps the listening window alive while the user is back in the app + # they are typing in: iOS ends a backgrounded app's capture without it. + UIBackgroundModes: [audio] + # The icon zooms into this before the first frame: the app's own page + # colour (Assets.xcassets › Page, light and dark), not a blank system + # white that reads as a broken icon mid-launch. + UILaunchScreen: + UIColorName: Page + UIApplicationSceneManifest: + UIApplicationSupportsMultipleScenes: false + UISupportedInterfaceOrientations: [UIInterfaceOrientationPortrait] + ITSAppUsesNonExemptEncryption: false + # `blurt://start` — what the keyboard opens to bring the app forward, the + # one moment iOS insists on before the microphone may open. + CFBundleURLTypes: + - CFBundleURLName: dev.alex.blurt.ios + CFBundleURLSchemes: [blurt] + # A shared key-term list (`.blurtterms`, JSON): declared so a tap on + # one in Messages offers Blurt, which opens it as an import. + # Not in place: iOS hands the app a copy in its Inbox, which + # `TermPack.from` reads and then deletes — an import, never an edit of + # the sender's file (and no security-scoped access to manage). + LSSupportsOpeningDocumentsInPlace: false + CFBundleDocumentTypes: + - CFBundleTypeName: Blurt key terms + CFBundleTypeRole: Viewer + LSHandlerRank: Owner + LSItemContentTypes: [dev.alex.blurt.terms] + UTExportedTypeDeclarations: + - UTTypeIdentifier: dev.alex.blurt.terms + UTTypeDescription: Blurt key terms + UTTypeConformsTo: [public.json] + UTTypeTagSpecification: + public.filename-extension: [blurtterms] + public.mime-type: application/vnd.blurt.terms+json + entitlements: + path: BlurtiOS/BlurtiOS.entitlements + properties: + com.apple.security.application-groups: [group.dev.alex.blurt] + dependencies: + - package: BlurtEngine + product: BlurtEngine + - target: BlurtiOSCore + embed: false + - target: BlurtKeyboard + # The keyboard. It types, it starts and stops a dictation, it inserts the + # words; it never hears anything. `RequestsOpenAccess` asks the user for Full + # Access, which is what lets it reach the App Group and the app. + BlurtKeyboard: + type: app-extension + platform: iOS + sources: + - path: BlurtKeyboard/Sources + - path: Shared + - path: BlurtKeyboard/PrivacyInfo.xcprivacy + buildPhase: resources + settings: + base: + PRODUCT_BUNDLE_IDENTIFIER: dev.alex.blurt.ios.keyboard + SWIFT_TREAT_WARNINGS_AS_ERRORS: YES + SWIFT_DEFAULT_ACTOR_ISOLATION: MainActor + APPLICATION_EXTENSION_API_ONLY: YES + info: + path: BlurtKeyboard/Info.plist + properties: + CFBundleDisplayName: Blurt + CFBundleShortVersionString: "0.1.0" + CFBundleVersion: "1" + NSExtension: + NSExtensionPointIdentifier: com.apple.keyboard-service + NSExtensionPrincipalClass: $(PRODUCT_MODULE_NAME).KeyboardViewController + NSExtensionAttributes: + IsASCIICapable: false + PrefersRightToLeft: false + PrimaryLanguage: en-US + RequestsOpenAccess: true + entitlements: + path: BlurtKeyboard/BlurtKeyboard.entitlements + properties: + com.apple.security.application-groups: [group.dev.alex.blurt] + dependencies: + - package: BlurtEngine + product: BlurtEngine + - target: BlurtiOSCore + embed: false + # Unit tests for the iPhone code — BlurtiOSCore's logic, and what the app and + # keyboard build on it — hosted by the app so `@testable import BlurtiOS` + # reaches the shell too (the keyboard's sources are compiled into the app). + # Core is linked into the host already, so the bundle only imports it. + # Swift Testing; `scripts/ios-test.sh` runs them on a simulator. + BlurtiOSTests: + type: bundle.unit-test + platform: iOS + sources: + - path: BlurtiOSTests + # The design's one source, bundled so DesignTokensTests can pin the + # compiled DesignTokens.swift to the file it was generated from. + - path: Design/tokens.json + buildPhase: resources + # The iPhone keyboard's measured geometry, so AppleGeometryTests can pin + # the key/* tokens to what scripts/apple-geometry.sh read off the phone. + - path: Design/apple-geometry.json + buildPhase: resources + dependencies: + - target: BlurtiOS + settings: + base: + PRODUCT_BUNDLE_IDENTIFIER: dev.alex.blurt.ios.tests + GENERATE_INFOPLIST_FILE: YES + SWIFT_TREAT_WARNINGS_AS_ERRORS: YES + SWIFT_DEFAULT_ACTOR_ISOLATION: MainActor + # The Apple-keyboard probe: one XCUITest that launches the app on + # `-BlurtProbeField` and reads the system keyboard's key frames, so the + # keyboard's geometry tokens are measured, not assumed + # (scripts/apple-geometry.sh → Design/apple-geometry.json). Its own scheme, + # so `xcodebuild test -scheme BlurtiOS` and the periphery scan never see it. + BlurtiOSProbe: + type: bundle.ui-testing + platform: iOS + sources: + - path: BlurtiOSProbe + dependencies: + - target: BlurtiOS + settings: + base: + PRODUCT_BUNDLE_IDENTIFIER: dev.alex.blurt.ios.probe + TEST_TARGET_NAME: BlurtiOS + GENERATE_INFOPLIST_FILE: YES + SWIFT_TREAT_WARNINGS_AS_ERRORS: YES +# XcodeGen emits a scheme only where one is declared, and `xcodebuild -scheme` +# needs a shared one. Building the app builds the keyboard it embeds; testing +# runs the unit tests against it. +schemes: + BlurtiOS: + build: + targets: + BlurtiOS: all + BlurtiOSTests: [test] + run: + config: Debug + test: + config: Debug + targets: + # Serial, on purpose: tests swap `SharedStore.override` (a global) for a + # scratch App Group, and an async test suspended mid-await would read + # another suite's — or the real one — if a second ran in the gap. + # `.serialized` only orders tests inside one suite. + - name: BlurtiOSTests + parallelizable: false + BlurtiOSProbe: + build: + targets: + BlurtiOS: all + BlurtiOSProbe: [test] + run: + config: Debug + test: + config: Debug + targets: + - BlurtiOSProbe diff --git a/CLAUDE.md b/CLAUDE.md index 44c6b238..1eeac300 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,7 +11,7 @@ This file adds only what is specific to Claude Code: the tooling wired up under | Hook | What it does | | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `SessionStart` → `session-start.sh` | Web/Linux sandboxes only (`CLAUDE_CODE_REMOTE=true`): installs the portable linters so `scripts/check.sh --portable` works, then prints a preflight noting CI is the authority on green. SwiftLint and swift-format are _not_ installed — their Linux builds ship as GitHub release binaries, which the default network policy blocks. Local macOS sessions exit immediately. | -| `PreToolUse` → `protect-pbxproj.sh` | Blocks edits to `App/Blurt/Blurt.xcodeproj/project.pbxproj` (exit 2). Edit `App/Blurt/project.yml` and run `xcodegen generate` instead. | +| `PreToolUse` → `protect-pbxproj.sh` | Blocks edits to `App/Blurt/Blurt.xcodeproj/project.pbxproj` (exit 2). Edit `App/Blurt/project.yml` and run `xcodegen generate` instead. (The iPhone project, `App/BlurtiOS/BlurtiOS.xcodeproj`, isn't committed at all: edit `App/BlurtiOS/project.yml`; every script regenerates it.) | | `PostToolUse` → `swift-format.sh` | Formats every edited `*.swift` file with the repo's `.swift-format` config, so edits are CI-clean by construction. No-ops when the tool is absent. | | `PostToolUse` → `swiftlint.sh` | Lints edited Swift files and surfaces violations immediately rather than at `check.sh` time. | @@ -59,9 +59,12 @@ globs go in one invocation so `hook-lib.sh` and `release-lib.sh` are in the inpu ## Subagents (`.claude/agents/`) -- **`swift6-concurrency-reviewer`** — run after editing engine or app code that touches actors, - async, the mic capture path, or the dictation pipeline. +- **`swift6-concurrency-reviewer`** — run after editing engine, Mac app or iPhone code that + touches actors, async, the mic capture path, the dictation pipeline, or the keyboard's hand-off + with the iPhone app (`BlurtiOSCore`'s model, `DictationCoordinator`, `ListeningWindow`). - **`macos-hig-reviewer`** — run after editing views in `App/Blurt/` (wizard, settings, overlay). + Mac only: the iPhone's views answer to `App/BlurtiOS/DESIGN.md`, checked on sight through the + gallery and the probe's screenshots (`scripts/design-capture.sh`, `scripts/ios-keyboard-shot.sh`). - **`cleanup-reviewer`** — quality-only review (reuse, simplification/dead code, efficiency, altitude), one agent per angle. Use it for `/simplify`-style passes instead of writing the guardrails into an ad-hoc prompt: it loads `project-guardrails` itself and already knows the @@ -89,9 +92,27 @@ not raw `brew install` or hand-rolled `xcodebuild` invocations: full Xcode — not just the Command Line Tools, Homebrew, a free Apple Development certificate) and why the scripts work the way they do. +For the iPhone app (`App/BlurtiOS`, the keyboard included), from the repo root: + +1. `xcodebuild -downloadPlatform iOS` — one-time: the simulator runtime **matching this Xcode's + iOS SDK**. An older runtime is refused at the destination ("iOS 26.5 is not installed"), even + with simulators for it listed. +2. `scripts/ios-sim.sh` — build, boot a simulator, install, grant the microphone, launch. + `BLURT_SIM_DEVICE="iPhone Air"` picks one; avoid the one a test run is using. +3. In the simulator: Settings › Apps › Blurt › Keyboards (or General › Keyboard › Keyboards › + Add New Keyboard) — Blurt on, Allow Full Access; then **Start listening** in Blurt. + +The simulator records through the Mac's microphone (`SimulatorAudioSource`), so dictation also +needs macOS to allow the **Simulator** app the microphone (Privacy & Security › Microphone) and +the right input under the Simulator's I/O › Audio Input — a refused microphone records silence +rather than failing. `scripts/ios-check.sh` runs the iPhone's quality gates (tests with the +`BlurtiOSCore` coverage floor, `swiftlint analyze`, periphery, the probe build); `check.sh` runs it +itself when this Mac has the matching runtime. + ## Permissions `.claude/settings.json` pre-allows the read-only and verification commands this repo needs (`swift test`, `swift build`, `xcodegen generate`, the Debug `xcodebuild` invocation, the linters, -`scripts/check.sh` with and without `--portable`, `scripts/dev-build.sh`). If you find yourself +`scripts/check.sh` with and without `--portable`, `scripts/dev-build.sh`, and the iPhone scripts: +`ios-sim.sh`, `ios-test.sh`, `ios-check.sh`, `ios-typecheck.sh`). If you find yourself prompted for a command that belongs in that set, add it there rather than working around it. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ff2deb44..641960eb 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -105,6 +105,31 @@ Run `xcodegen generate` and commit the regenerated `Blurt.xcodeproj/project.pbxp alongside it. `check.sh` fails on drift between the two. Never hand-edit the `.pbxproj` — it's generated. +### The iPhone app + +`App/BlurtiOS` is the iPhone app and the keyboard it ships: the app listens and transcribes, the +keyboard inserts the words (iOS lets no keyboard use the microphone). Its logic lives in +`BlurtiOSCore`, apart from its views the way `BlurtEngine` is apart from the Mac shell. + +```bash +xcodebuild -downloadPlatform iOS # one-time: the simulator runtime this Xcode builds for +scripts/ios-sim.sh # build, boot, install, grant the mic, launch (repo root) +scripts/ios-check.sh # its gates: tests + coverage floor, swiftlint analyze, periphery +``` + +No team or signing is needed for the simulator. Two things trip people up: + +- **The runtime has to match Xcode's iOS SDK.** Simulators on an older runtime still show in the + list but are refused at build time. +- **The simulator dictates through your Mac's microphone,** so macOS has to allow the + **Simulator** app the microphone (System Settings › Privacy & Security › Microphone), with the + right input picked under the Simulator's **I/O › Audio Input**. Refused, it records silence + rather than failing. + +To use the keyboard: Settings › Apps › Blurt › Keyboards — turn on Blurt, then Allow Full +Access — and **Start listening** in Blurt. A phone needs a paid team for the App Group; +[`App/BlurtiOS/README.md`](./App/BlurtiOS/README.md) has that, the tests and the design loop. + ### Working without a Mac You can read and edit Swift, and `scripts/check.sh --portable` fully verifies diff --git a/Package.swift b/Package.swift index badb5a39..69b49e7a 100644 --- a/Package.swift +++ b/Package.swift @@ -3,7 +3,12 @@ import PackageDescription let package = Package( name: "BlurtEngine", - platforms: [.macOS(.v15)], + // iOS 18 is the era-matching floor for macOS 15: the engine's `Synchronization` + // imports (`Mutex`) need it, and nothing portable here wants anything newer. + // The macOS-only capture, injection, Accessibility and TCC files are fenced + // behind `#if os(macOS)`; the pipeline, STT client and settings stores compile + // for both platforms. CI's `ios-build` job is what keeps that true. + platforms: [.macOS(.v15), .iOS(.v18)], products: [ .library(name: "BlurtEngine", targets: ["BlurtEngine"]) ], diff --git a/README.md b/README.md index e7f8ea99..e8612d1c 100644 --- a/README.md +++ b/README.md @@ -241,6 +241,13 @@ App/Blurt/ AppKit/SwiftUI shell (Xcode project generated by XcodeG Overlay/, MenuBar/ floating status pill, menu bar dictation indicator Update/ UpdaterModel: the Sparkle updater behind every "Check for Updates" Wizard/ setup wizard + settings window + +App/BlurtiOS/ the iPhone app and its keyboard (in development; README.md there) + BlurtiOSCore/ the logic, apart from the UI: the App Group contract between the + app and the keyboard, the keyboard's model and rules + BlurtiOS/Sources/ the app: listens and transcribes over the engine's pipeline + BlurtKeyboard/Sources/ the keyboard: a remote control that inserts the words, since + iOS lets no keyboard use the microphone ``` The engine is a standalone package you can embed to build your own dictation diff --git a/Sources/BlurtEngine/Audio/AudioInputDevices.swift b/Sources/BlurtEngine/Audio/AudioInputDevices.swift index 7d681ceb..3f7bb08b 100644 --- a/Sources/BlurtEngine/Audio/AudioInputDevices.swift +++ b/Sources/BlurtEngine/Audio/AudioInputDevices.swift @@ -102,7 +102,19 @@ public enum AudioInputDevices { /// `AudioTransport` matches against CoreAudio's `kAudioDeviceTransportType*` /// constants. Bit-pattern converted rather than numerically: these are packed /// ASCII, and the sign of the `Int32` spelling is an accident of the API. + /// + /// macOS only: `transportType` is not in the iOS SDK, and iOS exposes a single + /// microphone whose physical route the audio session decides. There this + /// answers nil — which `AudioTransport` reads as "unclassified", the middle + /// liveness cap and no tail linger — and, through `transportType(forUID:)`, + /// also reads as "the pinned device is absent", so an iOS capture always + /// follows the system route. Both hold until the iOS capture path classifies + /// `AVAudioSession` ports instead. private static func transportType(of device: AVCaptureDevice?) -> UInt32? { - device.map { UInt32(bitPattern: $0.transportType) } + #if os(macOS) + return device.map { UInt32(bitPattern: $0.transportType) } + #else + return nil + #endif } } diff --git a/Sources/BlurtEngine/Audio/AudioRoute.swift b/Sources/BlurtEngine/Audio/AudioRoute.swift index 142315ff..e64c7130 100644 --- a/Sources/BlurtEngine/Audio/AudioRoute.swift +++ b/Sources/BlurtEngine/Audio/AudioRoute.swift @@ -1,71 +1,73 @@ -import CoreAudio +#if os(macOS) + import CoreAudio -/// The CoreAudio reads that are left: which device is the default *output*, and -/// the property addressing `AudioRouteMonitor` shares with it. -/// -/// The input side used to live here too — which device is the default input, and -/// what transport it is on — and both have moved to `AVCaptureDevice` -/// (`AudioInputDevices`), which answers them off the same object the recorder -/// opens. That move waited on confirming `AVCaptureDevice.transportType` reports -/// `blue` for AirPods, because a transport that failed to read as Bluetooth -/// silently costs the 2.5 s liveness cap and the 220 ms tail linger — the -/// missing-last-word bug both exist to fix. It does; the parallel HAL read went. -/// -/// Output has no such replacement: `AVCaptureDevice` describes *capture* -/// devices, and what `AudioRouteMonitor` watches is the output route the cue -/// players render into. -/// -/// Raw reads only — no policy. Needs real hardware to answer anything, so it is -/// excluded from the coverage gate and must not be where a decision hides. -/// -/// Internal, not public: the app never asks these directly (it observes route -/// *changes* through `AudioRouteMonitor`), and `.periphery.yml` runs with -/// `retain_public: false`, so a `public` symbol only the engine reaches fails -/// the unused-code scan. -enum AudioRoute { - /// The system's current default *output* device — what `AudioRouteMonitor` - /// hangs its format listener on. Nil when there is none, or the read failed. - static func defaultOutputDeviceID() -> AudioDeviceID? { - defaultDeviceID(for: kAudioHardwarePropertyDefaultOutputDevice) - } + /// The CoreAudio reads that are left: which device is the default *output*, and + /// the property addressing `AudioRouteMonitor` shares with it. + /// + /// The input side used to live here too — which device is the default input, and + /// what transport it is on — and both have moved to `AVCaptureDevice` + /// (`AudioInputDevices`), which answers them off the same object the recorder + /// opens. That move waited on confirming `AVCaptureDevice.transportType` reports + /// `blue` for AirPods, because a transport that failed to read as Bluetooth + /// silently costs the 2.5 s liveness cap and the 220 ms tail linger — the + /// missing-last-word bug both exist to fix. It does; the parallel HAL read went. + /// + /// Output has no such replacement: `AVCaptureDevice` describes *capture* + /// devices, and what `AudioRouteMonitor` watches is the output route the cue + /// players render into. + /// + /// Raw reads only — no policy. Needs real hardware to answer anything, so it is + /// excluded from the coverage gate and must not be where a decision hides. + /// + /// Internal, not public: the app never asks these directly (it observes route + /// *changes* through `AudioRouteMonitor`), and `.periphery.yml` runs with + /// `retain_public: false`, so a `public` symbol only the engine reaches fails + /// the unused-code scan. + enum AudioRoute { + /// The system's current default *output* device — what `AudioRouteMonitor` + /// hangs its format listener on. Nil when there is none, or the read failed. + static func defaultOutputDeviceID() -> AudioDeviceID? { + defaultDeviceID(for: kAudioHardwarePropertyDefaultOutputDevice) + } - // MARK: - CoreAudio addressing + // MARK: - CoreAudio addressing - /// The system-wide audio object, which owns the default-device properties. - static var systemObject: AudioObjectID { AudioObjectID(kAudioObjectSystemObject) } + /// The system-wide audio object, which owns the default-device properties. + static var systemObject: AudioObjectID { AudioObjectID(kAudioObjectSystemObject) } - /// A global-scope address for `selector`. Returned fresh per call rather than - /// stored, because every caller needs its own copy to pass `inout` to - /// CoreAudio. Shared with `AudioRouteMonitor`, which addresses the same object - /// graph and would otherwise restate this three-field literal. - static func globalAddress(_ selector: AudioObjectPropertySelector) -> AudioObjectPropertyAddress { - AudioObjectPropertyAddress( - mSelector: selector, - mScope: kAudioObjectPropertyScopeGlobal, - mElement: kAudioObjectPropertyElementMain) - } + /// A global-scope address for `selector`. Returned fresh per call rather than + /// stored, because every caller needs its own copy to pass `inout` to + /// CoreAudio. Shared with `AudioRouteMonitor`, which addresses the same object + /// graph and would otherwise restate this three-field literal. + static func globalAddress(_ selector: AudioObjectPropertySelector) -> AudioObjectPropertyAddress { + AudioObjectPropertyAddress( + mSelector: selector, + mScope: kAudioObjectPropertyScopeGlobal, + mElement: kAudioObjectPropertyElementMain) + } - // MARK: - CoreAudio reads + // MARK: - CoreAudio reads - /// The device the system object reports for `selector` (a default-device - /// property). Nil covers both a failed read and the "no such device" sentinel, - /// which callers treat identically. - /// - /// Spelled out rather than shared behind a generic `read(_:from:initial:)`. - /// That reads better but doesn't compile: `&value` on an unconstrained `T` is - /// "forming 'UnsafeMutableRawPointer' to a variable of type 'T'; this is likely - /// incorrect because 'T' may contain an object reference". Making it work means - /// constraining to `BitwiseCopyable` and going through `withUnsafeMutableBytes` - /// — more machinery than one five-line read is worth, in a file the coverage - /// gate can't check anyway. - private static func defaultDeviceID(for selector: AudioObjectPropertySelector) -> AudioDeviceID? { - var address = globalAddress(selector) - var deviceID = AudioDeviceID(0) - var size = UInt32(MemoryLayout.size) - let status = AudioObjectGetPropertyData(systemObject, &address, 0, nil, &size, &deviceID) - // 0 is `kAudioObjectUnknown` — "there is no such device" — spelled as the - // literal so this doesn't depend on how the constant imports. - guard status == noErr, deviceID != 0 else { return nil } - return deviceID + /// The device the system object reports for `selector` (a default-device + /// property). Nil covers both a failed read and the "no such device" sentinel, + /// which callers treat identically. + /// + /// Spelled out rather than shared behind a generic `read(_:from:initial:)`. + /// That reads better but doesn't compile: `&value` on an unconstrained `T` is + /// "forming 'UnsafeMutableRawPointer' to a variable of type 'T'; this is likely + /// incorrect because 'T' may contain an object reference". Making it work means + /// constraining to `BitwiseCopyable` and going through `withUnsafeMutableBytes` + /// — more machinery than one five-line read is worth, in a file the coverage + /// gate can't check anyway. + private static func defaultDeviceID(for selector: AudioObjectPropertySelector) -> AudioDeviceID? { + var address = globalAddress(selector) + var deviceID = AudioDeviceID(0) + var size = UInt32(MemoryLayout.size) + let status = AudioObjectGetPropertyData(systemObject, &address, 0, nil, &size, &deviceID) + // 0 is `kAudioObjectUnknown` — "there is no such device" — spelled as the + // literal so this doesn't depend on how the constant imports. + guard status == noErr, deviceID != 0 else { return nil } + return deviceID + } } -} +#endif diff --git a/Sources/BlurtEngine/Audio/AudioRouteMonitor.swift b/Sources/BlurtEngine/Audio/AudioRouteMonitor.swift index a1a24fa3..31f1cf93 100644 --- a/Sources/BlurtEngine/Audio/AudioRouteMonitor.swift +++ b/Sources/BlurtEngine/Audio/AudioRouteMonitor.swift @@ -1,152 +1,154 @@ -import CoreAudio -import Dispatch -import os +#if os(macOS) + import CoreAudio + import Dispatch + import os -/// Ticks whenever the system's audio **output** route changes in a way that -/// invalidates an already-pre-rolled `AVAudioPlayer`. -/// -/// This exists for the record cue chimes. `CueSoundPlayer` decodes and -/// `prepareToPlay()`s them once at launch so the first chime never stalls the -/// pill — but that pre-roll is bound to the output route it was made against, -/// and Blurt's own capture is what invalidates it: opening the mic flips AirPods -/// out of their output-only profile into the bidirectional one, which drops the -/// output format underneath the primed players. The first chime after such a -/// flip is exactly the one that stalls, which is the chime at the start of a -/// dictation. -/// -/// Two properties are watched, because "the route changed" has two shapes: -/// -/// - `kAudioHardwarePropertyDefaultOutputDevice` on the system object — the user -/// switched output devices (built-in speakers → AirPods). -/// - `kAudioDevicePropertyNominalSampleRate` on whichever device is *currently* -/// default — the same device renegotiated its format, which is what the -/// profile flip looks like from CoreAudio. That listener is re-targeted -/// whenever the first one fires, so it always tracks the live device. -/// -/// Public because the app owns the cue players; the engine's own use of -/// CoreAudio routing (`AudioRoute`) stays internal. -/// -/// `@unchecked Sendable` because the listener registrations below are confined -/// to `queue` rather than protected by a lock — see their declarations. -public final class AudioRouteMonitor: @unchecked Sendable { - private static let logger = HostIdentity.current.logger("AudioRoute") + /// Ticks whenever the system's audio **output** route changes in a way that + /// invalidates an already-pre-rolled `AVAudioPlayer`. + /// + /// This exists for the record cue chimes. `CueSoundPlayer` decodes and + /// `prepareToPlay()`s them once at launch so the first chime never stalls the + /// pill — but that pre-roll is bound to the output route it was made against, + /// and Blurt's own capture is what invalidates it: opening the mic flips AirPods + /// out of their output-only profile into the bidirectional one, which drops the + /// output format underneath the primed players. The first chime after such a + /// flip is exactly the one that stalls, which is the chime at the start of a + /// dictation. + /// + /// Two properties are watched, because "the route changed" has two shapes: + /// + /// - `kAudioHardwarePropertyDefaultOutputDevice` on the system object — the user + /// switched output devices (built-in speakers → AirPods). + /// - `kAudioDevicePropertyNominalSampleRate` on whichever device is *currently* + /// default — the same device renegotiated its format, which is what the + /// profile flip looks like from CoreAudio. That listener is re-targeted + /// whenever the first one fires, so it always tracks the live device. + /// + /// Public because the app owns the cue players; the engine's own use of + /// CoreAudio routing (`AudioRoute`) stays internal. + /// + /// `@unchecked Sendable` because the listener registrations below are confined + /// to `queue` rather than protected by a lock — see their declarations. + public final class AudioRouteMonitor: @unchecked Sendable { + private static let logger = HostIdentity.current.logger("AudioRoute") - /// Fires once per observed route change. `.bufferingNewest(1)` because this is - /// an invalidation signal, not a log: a consumer that was busy through three - /// changes needs to re-prime once, not three times. - public let outputRouteChanges: AsyncStream - private let continuation: AsyncStream.Continuation + /// Fires once per observed route change. `.bufferingNewest(1)` because this is + /// an invalidation signal, not a log: a consumer that was busy through three + /// changes needs to re-prime once, not three times. + public let outputRouteChanges: AsyncStream + private let continuation: AsyncStream.Continuation - /// The queue CoreAudio delivers every listener callback on, and the one place - /// the registrations below are touched. Serial, so a re-target triggered by a - /// default-device change can't interleave with itself. - private let queue: DispatchQueue + /// The queue CoreAudio delivers every listener callback on, and the one place + /// the registrations below are touched. Serial, so a re-target triggered by a + /// default-device change can't interleave with itself. + private let queue: DispatchQueue - /// The registered listener blocks, kept so they can be handed back to - /// CoreAudio — removal matches on block identity, so a re-created block would - /// deregister nothing. - /// - /// `nonisolated(unsafe)` rather than lock-guarded: every read and write happens - /// inside a `queue` block, including the initial registration (`init` wraps it - /// in `queue.sync` precisely so a listener can't fire before the property - /// recording it has been written). Dispatch's serial ordering supplies both the - /// exclusion and the memory barriers a lock would. - private nonisolated(unsafe) var systemListener: AudioObjectPropertyListenerBlock? - private nonisolated(unsafe) var deviceListener: (id: AudioDeviceID, block: AudioObjectPropertyListenerBlock)? + /// The registered listener blocks, kept so they can be handed back to + /// CoreAudio — removal matches on block identity, so a re-created block would + /// deregister nothing. + /// + /// `nonisolated(unsafe)` rather than lock-guarded: every read and write happens + /// inside a `queue` block, including the initial registration (`init` wraps it + /// in `queue.sync` precisely so a listener can't fire before the property + /// recording it has been written). Dispatch's serial ordering supplies both the + /// exclusion and the memory barriers a lock would. + private nonisolated(unsafe) var systemListener: AudioObjectPropertyListenerBlock? + private nonisolated(unsafe) var deviceListener: (id: AudioDeviceID, block: AudioObjectPropertyListenerBlock)? - public init() { - let (stream, continuation) = AsyncStream.makeStream(bufferingPolicy: .bufferingNewest(1)) - self.outputRouteChanges = stream - self.continuation = continuation - self.queue = DispatchQueue(label: HostIdentity.current.queueLabel("AudioRoute")) - queue.sync { - installDefaultDeviceListener() - retargetFormatListener() + public init() { + let (stream, continuation) = AsyncStream.makeStream(bufferingPolicy: .bufferingNewest(1)) + self.outputRouteChanges = stream + self.continuation = continuation + self.queue = DispatchQueue(label: HostIdentity.current.queueLabel("AudioRoute")) + queue.sync { + installDefaultDeviceListener() + retargetFormatListener() + } } - } - /// The monitor is owned for the app's lifetime, so this never runs in - /// practice — but deregistering mirrors the `[weak self]` care below and - /// documents that the CoreAudio registrations are owned rather than leaked: a - /// listener left behind outlives the monitor, since CoreAudio retains the block - /// and nothing else would ever hand it back. - /// - /// Removal happens **inline, with no hop onto `queue`** — deliberately. - /// - /// A `queue.sync` here can self-deadlock. The listener blocks capture `self` - /// weakly, but `guard let self` upgrades that to a strong reference for the - /// body's duration, so while a block is running `queue` *is* an owner. If the - /// last other reference is dropped in that window, the block's release is the - /// final one and this `deinit` runs **on `queue`** — where `queue.sync` - /// deadlocks against itself. - /// - /// Inline removal is also race-free without the hop. `deinit` only runs once - /// the last reference is gone, so no block can be *inside* its `guard let self` - /// concurrently with this — a block that starts now fails the upgrade and - /// touches nothing. That leaves these reads of the queue-confined - /// registrations unopposed. (`queue` is still passed to CoreAudio, because - /// removal matches on the queue the listener was added with; that's an argument, - /// not an execution context.) - deinit { - continuation.finish() - if let systemListener { - var address = AudioRoute.globalAddress(kAudioHardwarePropertyDefaultOutputDevice) - _ = AudioObjectRemovePropertyListenerBlock( - AudioRoute.systemObject, &address, queue, systemListener) - } - if let deviceListener { - var address = AudioRoute.globalAddress(kAudioDevicePropertyNominalSampleRate) - _ = AudioObjectRemovePropertyListenerBlock( - deviceListener.id, &address, queue, deviceListener.block) + /// The monitor is owned for the app's lifetime, so this never runs in + /// practice — but deregistering mirrors the `[weak self]` care below and + /// documents that the CoreAudio registrations are owned rather than leaked: a + /// listener left behind outlives the monitor, since CoreAudio retains the block + /// and nothing else would ever hand it back. + /// + /// Removal happens **inline, with no hop onto `queue`** — deliberately. + /// + /// A `queue.sync` here can self-deadlock. The listener blocks capture `self` + /// weakly, but `guard let self` upgrades that to a strong reference for the + /// body's duration, so while a block is running `queue` *is* an owner. If the + /// last other reference is dropped in that window, the block's release is the + /// final one and this `deinit` runs **on `queue`** — where `queue.sync` + /// deadlocks against itself. + /// + /// Inline removal is also race-free without the hop. `deinit` only runs once + /// the last reference is gone, so no block can be *inside* its `guard let self` + /// concurrently with this — a block that starts now fails the upgrade and + /// touches nothing. That leaves these reads of the queue-confined + /// registrations unopposed. (`queue` is still passed to CoreAudio, because + /// removal matches on the queue the listener was added with; that's an argument, + /// not an execution context.) + deinit { + continuation.finish() + if let systemListener { + var address = AudioRoute.globalAddress(kAudioHardwarePropertyDefaultOutputDevice) + _ = AudioObjectRemovePropertyListenerBlock( + AudioRoute.systemObject, &address, queue, systemListener) + } + if let deviceListener { + var address = AudioRoute.globalAddress(kAudioDevicePropertyNominalSampleRate) + _ = AudioObjectRemovePropertyListenerBlock( + deviceListener.id, &address, queue, deviceListener.block) + } } - } - // MARK: - Registration (queue-confined) + // MARK: - Registration (queue-confined) - /// Watches for the default output device itself changing. Registered once and - /// never re-targeted — the system object is always there. - private func installDefaultDeviceListener() { - var address = AudioRoute.globalAddress(kAudioHardwarePropertyDefaultOutputDevice) - // `[weak self]`, so CoreAudio's strong hold on the block doesn't keep the - // monitor alive forever — and so a callback landing during teardown finds - // nil rather than a half-destroyed object. - let block: AudioObjectPropertyListenerBlock = { [weak self] _, _ in - guard let self else { return } - // Re-target first, then publish: a consumer that re-primes on this tick - // should already be behind a listener pointed at the new device. - self.retargetFormatListener() - self.continuation.yield() - } - let status = AudioObjectAddPropertyListenerBlock(AudioRoute.systemObject, &address, queue, block) - guard status == noErr else { - Self.logger.error("default-output listener failed: \(status)") - return + /// Watches for the default output device itself changing. Registered once and + /// never re-targeted — the system object is always there. + private func installDefaultDeviceListener() { + var address = AudioRoute.globalAddress(kAudioHardwarePropertyDefaultOutputDevice) + // `[weak self]`, so CoreAudio's strong hold on the block doesn't keep the + // monitor alive forever — and so a callback landing during teardown finds + // nil rather than a half-destroyed object. + let block: AudioObjectPropertyListenerBlock = { [weak self] _, _ in + guard let self else { return } + // Re-target first, then publish: a consumer that re-primes on this tick + // should already be behind a listener pointed at the new device. + self.retargetFormatListener() + self.continuation.yield() + } + let status = AudioObjectAddPropertyListenerBlock(AudioRoute.systemObject, &address, queue, block) + guard status == noErr else { + Self.logger.error("default-output listener failed: \(status)") + return + } + systemListener = block } - systemListener = block - } - /// Points the format listener at the current default output device, removing - /// the one on the previous device. A no-op when the device hasn't actually - /// changed, so a notification that resolves to the same device doesn't churn - /// the registration. - private func retargetFormatListener() { - let device = AudioRoute.defaultOutputDeviceID() - if let existing = deviceListener { - guard existing.id != device else { return } + /// Points the format listener at the current default output device, removing + /// the one on the previous device. A no-op when the device hasn't actually + /// changed, so a notification that resolves to the same device doesn't churn + /// the registration. + private func retargetFormatListener() { + let device = AudioRoute.defaultOutputDeviceID() + if let existing = deviceListener { + guard existing.id != device else { return } + var address = AudioRoute.globalAddress(kAudioDevicePropertyNominalSampleRate) + _ = AudioObjectRemovePropertyListenerBlock(existing.id, &address, queue, existing.block) + deviceListener = nil + } + guard let device else { return } + let block: AudioObjectPropertyListenerBlock = { [weak self] _, _ in + self?.continuation.yield() + } var address = AudioRoute.globalAddress(kAudioDevicePropertyNominalSampleRate) - _ = AudioObjectRemovePropertyListenerBlock(existing.id, &address, queue, existing.block) - deviceListener = nil - } - guard let device else { return } - let block: AudioObjectPropertyListenerBlock = { [weak self] _, _ in - self?.continuation.yield() - } - var address = AudioRoute.globalAddress(kAudioDevicePropertyNominalSampleRate) - let status = AudioObjectAddPropertyListenerBlock(device, &address, queue, block) - guard status == noErr else { - Self.logger.error("output-format listener failed: \(status)") - return + let status = AudioObjectAddPropertyListenerBlock(device, &address, queue, block) + guard status == noErr else { + Self.logger.error("output-format listener failed: \(status)") + return + } + deviceListener = (id: device, block: block) } - deviceListener = (id: device, block: block) } -} +#endif diff --git a/Sources/BlurtEngine/Audio/AudioTransport.swift b/Sources/BlurtEngine/Audio/AudioTransport.swift index 99046b11..eee1bd94 100644 --- a/Sources/BlurtEngine/Audio/AudioTransport.swift +++ b/Sources/BlurtEngine/Audio/AudioTransport.swift @@ -1,4 +1,6 @@ -import CoreAudio +#if os(macOS) + import CoreAudio +#endif /// Classification of a CoreAudio device's transport type /// (`kAudioDevicePropertyTransportType`). @@ -12,6 +14,13 @@ import CoreAudio /// bring-up gate. /// - `MicCapture`'s tail linger, which keeps capturing past key-up so the last /// word doesn't get truncated by the link's buffering. +/// +/// The transport codes are HAL constants, which exist on macOS only. On iOS +/// nothing produces one — `AudioInputDevices` answers nil there — so both +/// classifiers answer `false`, which lands every iOS capture on the middle +/// liveness cap with no tail linger: the conservative reading, and a +/// placeholder until the iOS capture path classifies `AVAudioSession` ports +/// (`.bluetoothHFP` is the AirPods case that matters) the same way. enum AudioTransport { /// Whether the transport is a Bluetooth one. Both types count: AirPods and /// other wireless headsets report the classic `bluetooth` transport, LE Audio @@ -23,9 +32,13 @@ enum AudioTransport { /// and no linger. Padding every wired capture with a delay would be a worse /// regression than losing the tail on a device we couldn't classify. static func isBluetooth(_ transportType: UInt32?) -> Bool { - guard let transportType else { return false } - return transportType == kAudioDeviceTransportTypeBluetooth - || transportType == kAudioDeviceTransportTypeBluetoothLE + #if os(macOS) + guard let transportType else { return false } + return transportType == kAudioDeviceTransportTypeBluetooth + || transportType == kAudioDeviceTransportTypeBluetoothLE + #else + return false + #endif } /// Whether the transport is a wired or on-board one — a device that begins @@ -45,19 +58,23 @@ enum AudioTransport { /// the middle cap, which is the safe direction — a slightly longer wait on an /// exotic input, never a truncated one. static func isLocal(_ transportType: UInt32?) -> Bool { - guard let transportType else { return false } - switch transportType { - case kAudioDeviceTransportTypeBuiltIn, - kAudioDeviceTransportTypeUSB, - kAudioDeviceTransportTypePCI, - kAudioDeviceTransportTypeFireWire, - kAudioDeviceTransportTypeThunderbolt, - kAudioDeviceTransportTypeDisplayPort, - kAudioDeviceTransportTypeHDMI: - return true - default: + #if os(macOS) + guard let transportType else { return false } + switch transportType { + case kAudioDeviceTransportTypeBuiltIn, + kAudioDeviceTransportTypeUSB, + kAudioDeviceTransportTypePCI, + kAudioDeviceTransportTypeFireWire, + kAudioDeviceTransportTypeThunderbolt, + kAudioDeviceTransportTypeDisplayPort, + kAudioDeviceTransportTypeHDMI: + return true + default: + return false + } + #else return false - } + #endif } /// How much longer capture runs past the key-up that ends it, for a device of diff --git a/Sources/BlurtEngine/Audio/CaptureSessionRecorder+AudioSettings.swift b/Sources/BlurtEngine/Audio/CaptureSessionRecorder+AudioSettings.swift new file mode 100644 index 00000000..ccdd43e4 --- /dev/null +++ b/Sources/BlurtEngine/Audio/CaptureSessionRecorder+AudioSettings.swift @@ -0,0 +1,30 @@ +@preconcurrency import AVFoundation + +// The output's format request, in its own file so `CaptureSessionRecorder.swift` +// stays within the lint file-length budget — the same split `DictationSession` +// makes. macOS only: `audioSettings` is not in the iOS SDK (see the recorder). +#if os(macOS) + extension CaptureSessionRecorder { + /// The dictation API's geometry, converted by the output itself — the same six + /// keys the retired WAV recorder asked of its file, so capture still lands in + /// upload-ready S16LE with no resample pass anywhere. + /// + /// Every number comes from `SyncSTTLimits`, which also owns the byte math the + /// upload side applies to the result. They are one contract: a stereo or + /// 8-bit recorder would silently halve or double every duration the pipeline + /// computes. Device-free, so `MicCaptureFormatTests` can assert it despite + /// this file being coverage-excluded; a function rather than a stored static + /// because `[String: Any]` is not `Sendable`, and it is built once per + /// recorder regardless. + static func audioSettings() -> [String: Any] { + [ + AVFormatIDKey: kAudioFormatLinearPCM, + AVSampleRateKey: Double(SyncSTTLimits.sampleRate), + AVNumberOfChannelsKey: SyncSTTLimits.channelCount, + AVLinearPCMBitDepthKey: SyncSTTLimits.bitDepth, + AVLinearPCMIsFloatKey: false, + AVLinearPCMIsBigEndianKey: false, + ] + } + } +#endif diff --git a/Sources/BlurtEngine/Audio/CaptureSessionRecorder.swift b/Sources/BlurtEngine/Audio/CaptureSessionRecorder.swift index 4fceab0d..b573cca4 100644 --- a/Sources/BlurtEngine/Audio/CaptureSessionRecorder.swift +++ b/Sources/BlurtEngine/Audio/CaptureSessionRecorder.swift @@ -154,7 +154,11 @@ final class CaptureSessionRecorder: NSObject, @unchecked Sendable { "session refused the input device \(device.localizedName, privacy: .public)") } } - output.audioSettings = Self.audioSettings() + // macOS only: `audioSettings` is not in the iOS SDK, so there the output + // vends the device's native format — an iOS host brings its own capture. + #if os(macOS) + output.audioSettings = Self.audioSettings() + #endif output.setSampleBufferDelegate(self, queue: delegateQueue) if session.canAddOutput(output) { session.addOutput(output) @@ -189,28 +193,6 @@ final class CaptureSessionRecorder: NSObject, @unchecked Sendable { return AudioInputDevices.systemDefaultDevice } - /// The dictation API's geometry, converted by the output itself — the same six - /// keys the retired WAV recorder asked of its file, so capture still lands in - /// upload-ready S16LE with no resample pass anywhere. - /// - /// Every number comes from `SyncSTTLimits`, which also owns the byte math the - /// upload side applies to the result. They are one contract: a stereo or - /// 8-bit recorder would silently halve or double every duration the pipeline - /// computes. Device-free, so `MicCaptureFormatTests` can assert it despite - /// this file being coverage-excluded; a function rather than a stored static - /// because `[String: Any]` is not `Sendable`, and it is built once per - /// recorder regardless. - static func audioSettings() -> [String: Any] { - [ - AVFormatIDKey: kAudioFormatLinearPCM, - AVSampleRateKey: Double(SyncSTTLimits.sampleRate), - AVNumberOfChannelsKey: SyncSTTLimits.channelCount, - AVLinearPCMBitDepthKey: SyncSTTLimits.bitDepth, - AVLinearPCMIsFloatKey: false, - AVLinearPCMIsBigEndianKey: false, - ] - } - /// Opens the device and starts capture, answering false when there is no /// usable input: nothing was attached at build time, or the session refused to /// run. diff --git a/Sources/BlurtEngine/Config/InstallReset.swift b/Sources/BlurtEngine/Config/InstallReset.swift index 30cbb6ad..19927547 100644 --- a/Sources/BlurtEngine/Config/InstallReset.swift +++ b/Sources/BlurtEngine/Config/InstallReset.swift @@ -80,7 +80,15 @@ public struct InstallReset { UserDefaults.standard.removePersistentDomain(forName: bundleID) }, clearAPIKey: { keyStore.save(nil) }, - resetPermissions: { PermissionsReset.resetAll(bundleID: bundleID) }, + resetPermissions: { + #if os(macOS) + return PermissionsReset.resetAll(bundleID: bundleID) + #else + // iOS lets no app revoke its own grants — they go with the install — + // so there is nothing for this step to do, and nothing to report. + return true + #endif + }, clearLogs: { DictationLog.removeStoredLogs() }) } diff --git a/Sources/BlurtEngine/FocusCapture/FocusCapture+Editability.swift b/Sources/BlurtEngine/FocusCapture/FocusCapture+Editability.swift index 3e461613..4377f16e 100644 --- a/Sources/BlurtEngine/FocusCapture/FocusCapture+Editability.swift +++ b/Sources/BlurtEngine/FocusCapture/FocusCapture+Editability.swift @@ -1,160 +1,162 @@ -import AppKit -import ApplicationServices +#if os(macOS) + import AppKit + import ApplicationServices -// The "can this target accept a paste?" half of `FocusCapture`, split out of -// `FocusCapture.swift` to stay within the lint file-length budget — the same -// reason `DictationSession` is split across `+Commands`/`+Observation`/`+Pipeline`. -// -// Cohesive as its own file: everything here serves `KeyInjector`'s pre-paste -// "no beep" guard, whereas `FocusCapture.swift` proper serves the press-time -// context capture that primes the STT request. -extension FocusCapture { - /// AX roles a focused element reports when it accepts typed/pasted text. - /// Includes `secureFieldRole`: a password field is a valid *paste* target even - /// though its contents are never read (see `isSecureField`). - private static let editableRoles: Set = [ - "AXTextField", "AXTextArea", "AXComboBox", secureFieldRole, "AXSearchField", - ] + // The "can this target accept a paste?" half of `FocusCapture`, split out of + // `FocusCapture.swift` to stay within the lint file-length budget — the same + // reason `DictationSession` is split across `+Commands`/`+Observation`/`+Pipeline`. + // + // Cohesive as its own file: everything here serves `KeyInjector`'s pre-paste + // "no beep" guard, whereas `FocusCapture.swift` proper serves the press-time + // context capture that primes the STT request. + extension FocusCapture { + /// AX roles a focused element reports when it accepts typed/pasted text. + /// Includes `secureFieldRole`: a password field is a valid *paste* target even + /// though its contents are never read (see `isSecureField`). + private static let editableRoles: Set = [ + "AXTextField", "AXTextArea", "AXComboBox", secureFieldRole, "AXSearchField", + ] - /// Pure decision: do these signals, read off a focused element, mean it accepts - /// pasted text? The injector calls this just before a synthesized ⌘V — if it - /// returns false the paste is skipped (so macOS doesn't beep into a non-editable - /// target) and the transcript is left on the clipboard with a quiet "Copied" - /// notice. The "is anything focused at all?" question is answered by the caller, - /// which never gets this far without an element (see `hasEditableFocusedElement`). - /// - /// Requires a *positive* editability signal: a known text role, a settable value, - /// or an insertion point. Anything else — a non-text control, an unknown role, or - /// no readable role — is treated as not editable, so we copy rather than beep a - /// ⌘V into a target that can't take it. - /// - /// AX-opaque apps — Electron editors (VS Code, Slack) and web browsers — can - /// expose *none* of these signals even for a genuine text field, so this - /// returns false for them too. The injector still pastes into those via a - /// separate app-identity check (see `isAXOpaqueApp` / `KeyInjector.insert`), - /// so the user's words aren't dropped to copy-only there. - static func isEditableTarget(role: String?, valueSettable: Bool, hasInsertionPoint: Bool) -> Bool { - if let role, editableRoles.contains(role) { return true } - return valueSettable || hasInsertionPoint - } + /// Pure decision: do these signals, read off a focused element, mean it accepts + /// pasted text? The injector calls this just before a synthesized ⌘V — if it + /// returns false the paste is skipped (so macOS doesn't beep into a non-editable + /// target) and the transcript is left on the clipboard with a quiet "Copied" + /// notice. The "is anything focused at all?" question is answered by the caller, + /// which never gets this far without an element (see `hasEditableFocusedElement`). + /// + /// Requires a *positive* editability signal: a known text role, a settable value, + /// or an insertion point. Anything else — a non-text control, an unknown role, or + /// no readable role — is treated as not editable, so we copy rather than beep a + /// ⌘V into a target that can't take it. + /// + /// AX-opaque apps — Electron editors (VS Code, Slack) and web browsers — can + /// expose *none* of these signals even for a genuine text field, so this + /// returns false for them too. The injector still pastes into those via a + /// separate app-identity check (see `isAXOpaqueApp` / `KeyInjector.insert`), + /// so the user's words aren't dropped to copy-only there. + static func isEditableTarget(role: String?, valueSettable: Bool, hasInsertionPoint: Bool) -> Bool { + if let role, editableRoles.contains(role) { return true } + return valueSettable || hasInsertionPoint + } - /// Whether `app` is an Electron/Chromium-based app, detected by the bundled - /// Electron framework. Such apps ship with their accessibility tree off, so even - /// a focused text field exposes no editable AX signal and - /// `hasEditableFocusedElement` reads them as non-editable. A native app with - /// genuinely no editable focus bundles no such framework and correctly falls - /// back to copy. - static func isElectronApp(_ app: NSRunningApplication?) -> Bool { - isElectronBundle(app?.bundleURL) - } + /// Whether `app` is an Electron/Chromium-based app, detected by the bundled + /// Electron framework. Such apps ship with their accessibility tree off, so even + /// a focused text field exposes no editable AX signal and + /// `hasEditableFocusedElement` reads them as non-editable. A native app with + /// genuinely no editable focus bundles no such framework and correctly falls + /// back to copy. + static func isElectronApp(_ app: NSRunningApplication?) -> Bool { + isElectronBundle(app?.bundleURL) + } - /// Pure decision behind `isElectronApp`: does the bundle at `bundleURL` ship the - /// Electron framework? Split from the `NSRunningApplication` wrapper for the same - /// reason as `isBrowserBundleID` — the detection is then unit-testable against a - /// fixture bundle, instead of requiring an Electron app to be installed *and* - /// running on the machine under test. - /// - /// Rebranded Electron apps rename the framework (Codex ships - /// `Codex Framework.framework`), so the `ElectronAsarIntegrity` Info.plist key - /// Electron's packager writes is checked as a fallback. - static func isElectronBundle(_ bundleURL: URL?) -> Bool { - guard let bundleURL else { return false } - let electronFramework = bundleURL.appendingPathComponent( - "Contents/Frameworks/Electron Framework.framework") - if FileManager.default.fileExists(atPath: electronFramework.path) { return true } - return Bundle(url: bundleURL)?.object(forInfoDictionaryKey: "ElectronAsarIntegrity") != nil - } + /// Pure decision behind `isElectronApp`: does the bundle at `bundleURL` ship the + /// Electron framework? Split from the `NSRunningApplication` wrapper for the same + /// reason as `isBrowserBundleID` — the detection is then unit-testable against a + /// fixture bundle, instead of requiring an Electron app to be installed *and* + /// running on the machine under test. + /// + /// Rebranded Electron apps rename the framework (Codex ships + /// `Codex Framework.framework`), so the `ElectronAsarIntegrity` Info.plist key + /// Electron's packager writes is checked as a fallback. + static func isElectronBundle(_ bundleURL: URL?) -> Bool { + guard let bundleURL else { return false } + let electronFramework = bundleURL.appendingPathComponent( + "Contents/Frameworks/Electron Framework.framework") + if FileManager.default.fileExists(atPath: electronFramework.path) { return true } + return Bundle(url: bundleURL)?.object(forInfoDictionaryKey: "ElectronAsarIntegrity") != nil + } - /// Bundle-identifier prefixes of known web browsers. Prefix-matched so channel - /// variants classify with their stable siblings (`com.google.Chrome.beta`, - /// `com.apple.SafariTechnologyPreview`). - private static let browserBundleIDPrefixes: [String] = [ - "com.apple.Safari", // Safari + Safari Technology Preview - "com.google.Chrome", // Chrome + Beta/Dev/Canary - "org.chromium.Chromium", - "com.microsoft.edgemac", // Edge + Beta/Dev/Canary - "com.brave.Browser", // Brave + Beta/Nightly - "com.operasoftware.Opera", - "com.vivaldi.Vivaldi", - "company.thebrowser.Browser", // Arc - "org.mozilla.firefox", - "com.duckduckgo.macos.browser", - "com.kagi.kagimacOS", // Orion - ] + /// Bundle-identifier prefixes of known web browsers. Prefix-matched so channel + /// variants classify with their stable siblings (`com.google.Chrome.beta`, + /// `com.apple.SafariTechnologyPreview`). + private static let browserBundleIDPrefixes: [String] = [ + "com.apple.Safari", // Safari + Safari Technology Preview + "com.google.Chrome", // Chrome + Beta/Dev/Canary + "org.chromium.Chromium", + "com.microsoft.edgemac", // Edge + Beta/Dev/Canary + "com.brave.Browser", // Brave + Beta/Nightly + "com.operasoftware.Opera", + "com.vivaldi.Vivaldi", + "company.thebrowser.Browser", // Arc + "org.mozilla.firefox", + "com.duckduckgo.macos.browser", + "com.kagi.kagimacOS", // Orion + ] - /// Pure decision behind `isBrowserApp`: does this bundle identifier belong to a - /// known browser? Split from the `NSRunningApplication` wrapper so the - /// classification is unit-testable without live running apps. - static func isBrowserBundleID(_ bundleID: String?) -> Bool { - guard let bundleID else { return false } - return browserBundleIDPrefixes.contains { bundleID.hasPrefix($0) } - } + /// Pure decision behind `isBrowserApp`: does this bundle identifier belong to a + /// known browser? Split from the `NSRunningApplication` wrapper so the + /// classification is unit-testable without live running apps. + static func isBrowserBundleID(_ bundleID: String?) -> Bool { + guard let bundleID else { return false } + return browserBundleIDPrefixes.contains { bundleID.hasPrefix($0) } + } - /// Whether `app` is a known web browser. Web content is AX-opaque in practice: - /// Chromium builds its accessibility tree lazily (the first query after launch - /// resolves only a bare `AXWebArea` with no editable signal), and even with the - /// tree live, a `contenteditable` composer (ChatGPT's ProseMirror field) can - /// surface as a generic group with no settable value. So "no editable signal" - /// in a browser usually means "AX can't see the field," not "no field." - static func isBrowserApp(_ app: NSRunningApplication?) -> Bool { - isBrowserBundleID(app?.bundleIdentifier) - } + /// Whether `app` is a known web browser. Web content is AX-opaque in practice: + /// Chromium builds its accessibility tree lazily (the first query after launch + /// resolves only a bare `AXWebArea` with no editable signal), and even with the + /// tree live, a `contenteditable` composer (ChatGPT's ProseMirror field) can + /// surface as a generic group with no settable value. So "no editable signal" + /// in a browser usually means "AX can't see the field," not "no field." + static func isBrowserApp(_ app: NSRunningApplication?) -> Bool { + isBrowserBundleID(app?.bundleIdentifier) + } - /// Whether `app` is AX-opaque — an Electron editor or a web browser — where a - /// focused text field can expose no editable AX signal at all. These are the - /// one case the injector still pastes into on no signal: dropping the user's - /// words into a copy-only fallback there would be the worse mistake. The - /// accepted trade-off is a rare ⌘V beep when such an app truly has nothing - /// editable focused. - static func isAXOpaqueApp(_ app: NSRunningApplication?) -> Bool { - // Browser first: it's a string prefix check, whereas isElectronApp probes - // the disk (FileManager.fileExists) — skip that I/O for the common case. - isBrowserApp(app) || isElectronApp(app) - } + /// Whether `app` is AX-opaque — an Electron editor or a web browser — where a + /// focused text field can expose no editable AX signal at all. These are the + /// one case the injector still pastes into on no signal: dropping the user's + /// words into a copy-only fallback there would be the worse mistake. The + /// accepted trade-off is a rare ⌘V beep when such an app truly has nothing + /// editable focused. + static func isAXOpaqueApp(_ app: NSRunningApplication?) -> Bool { + // Browser first: it's a string prefix check, whereas isElectronApp probes + // the disk (FileManager.fileExists) — skip that I/O for the common case. + isBrowserApp(app) || isElectronApp(app) + } - /// Whether the system-wide focused element can accept pasted text right now. - /// Read by `KeyInjector` (off the main actor, after it has activated the target - /// app) just before pasting — the Accessibility *client* read APIs are - /// thread-safe. Returns `true` whenever AX can't be consulted (process not - /// trusted) or can't resolve a focused element, so an unknowable state still - /// attempts the paste — the injector's own trust check then handles the - /// missing-permission case. - nonisolated static func hasEditableFocusedElement() -> Bool { - guard AXIsProcessTrusted() else { return true } + /// Whether the system-wide focused element can accept pasted text right now. + /// Read by `KeyInjector` (off the main actor, after it has activated the target + /// app) just before pasting — the Accessibility *client* read APIs are + /// thread-safe. Returns `true` whenever AX can't be consulted (process not + /// trusted) or can't resolve a focused element, so an unknowable state still + /// attempts the paste — the injector's own trust check then handles the + /// missing-permission case. + nonisolated static func hasEditableFocusedElement() -> Bool { + guard AXIsProcessTrusted() else { return true } - guard let element = systemFocusedElement() else { - // AX is trusted but reports no focused element — e.g. a native app frontmost - // with nothing editable focused (Finder, the desktop, a button-only window). - // Posting ⌘V there only beeps, so treat it as non-editable and copy instead. - // AX-opaque apps (Electron editors like VS Code/Slack, and browsers before - // Chromium's lazy accessibility tree is built) also expose no focused - // element here, but the injector's app-identity check still pastes into - // those (see `KeyInjector.insert` / `isAXOpaqueApp`). - return false - } + guard let element = systemFocusedElement() else { + // AX is trusted but reports no focused element — e.g. a native app frontmost + // with nothing editable focused (Finder, the desktop, a button-only window). + // Posting ⌘V there only beeps, so treat it as non-editable and copy instead. + // AX-opaque apps (Electron editors like VS Code/Slack, and browsers before + // Chromium's lazy accessibility tree is built) also expose no focused + // element here, but the injector's app-identity check still pastes into + // those (see `KeyInjector.insert` / `isAXOpaqueApp`). + return false + } - // Same checked reader `captureFieldContext` uses for this attribute, so the - // editability path and the secure-field path can't disagree about what a - // misbehaving app's role reads as. - let role = stringValue(element, kAXRoleAttribute) + // Same checked reader `captureFieldContext` uses for this attribute, so the + // editability path and the secure-field path can't disagree about what a + // misbehaving app's role reads as. + let role = stringValue(element, kAXRoleAttribute) - var settable = DarwinBoolean(false) - let valueSettable = - AXUIElementIsAttributeSettable(element, kAXValueAttribute as CFString, &settable) == .success - && settable.boolValue + var settable = DarwinBoolean(false) + let valueSettable = + AXUIElementIsAttributeSettable(element, kAXValueAttribute as CFString, &settable) == .success + && settable.boolValue - // A readable selected-text *range* means the element has an insertion point — - // the hallmark of a text input even when its value isn't reported settable. - // Require the range to actually decode, not merely that the read succeeded: a - // `.success` carrying a non-CFRange payload is not an insertion point, and this - // signal is one of the two that can green-light a ⌘V on an unknown role. - var rangeRef: CFTypeRef? - let hasInsertionPoint = - AXUIElementCopyAttributeValue( - element, kAXSelectedTextRangeAttribute as CFString, &rangeRef) == .success - && rangeRef.flatMap(axRange) != nil + // A readable selected-text *range* means the element has an insertion point — + // the hallmark of a text input even when its value isn't reported settable. + // Require the range to actually decode, not merely that the read succeeded: a + // `.success` carrying a non-CFRange payload is not an insertion point, and this + // signal is one of the two that can green-light a ⌘V on an unknown role. + var rangeRef: CFTypeRef? + let hasInsertionPoint = + AXUIElementCopyAttributeValue( + element, kAXSelectedTextRangeAttribute as CFString, &rangeRef) == .success + && rangeRef.flatMap(axRange) != nil - return isEditableTarget( - role: role, valueSettable: valueSettable, hasInsertionPoint: hasInsertionPoint) + return isEditableTarget( + role: role, valueSettable: valueSettable, hasInsertionPoint: hasInsertionPoint) + } } -} +#endif diff --git a/Sources/BlurtEngine/FocusCapture/FocusCapture+Pure.swift b/Sources/BlurtEngine/FocusCapture/FocusCapture+Pure.swift index aee3e060..b2f6a8e7 100644 --- a/Sources/BlurtEngine/FocusCapture/FocusCapture+Pure.swift +++ b/Sources/BlurtEngine/FocusCapture/FocusCapture+Pure.swift @@ -50,12 +50,12 @@ extension FocusCapture { /// /// Google Docs (and some other web editors) expose the text before the caret as /// a lone U+200B ZERO WIDTH SPACE — invisible, not real content, yet crucially - /// *not* `Character.isWhitespace`. Left as-is, `KeyInjector.withLeadingSeparator` + /// *not* `Character.isWhitespace`. Left as-is, `InsertionSeparator.withLeadingSeparator` /// reads it as substantive preceding text that doesn't end in whitespace and /// prepends a stray separator space on every dictation. Treating an /// entirely-invisible read as "nothing readable here" (`nil`) instead routes the /// field into the same-window separator fallback like any other - /// Accessibility-opaque editor (see `KeyInjector.separatorBasis`). + /// Accessibility-opaque editor (see `InsertionSeparator.basis`). /// /// "Invisible" means every scalar is a default-ignorable code point (zero-width /// spaces, joiners, bidi marks, BOM, …). Regular whitespace (space/tab/newline) diff --git a/Sources/BlurtEngine/FocusCapture/FocusCapture.swift b/Sources/BlurtEngine/FocusCapture/FocusCapture.swift index a6a37a51..2b9ca144 100644 --- a/Sources/BlurtEngine/FocusCapture/FocusCapture.swift +++ b/Sources/BlurtEngine/FocusCapture/FocusCapture.swift @@ -1,333 +1,299 @@ -import AppKit -import ApplicationServices +#if os(macOS) + import AppKit + import ApplicationServices -struct CapturedFocus: Sendable { - let pid: pid_t - let processName: String? -} - -enum FocusCapture { - @MainActor - static func captureFrontmost() -> CapturedFocus? { - guard let app = NSWorkspace.shared.frontmostApplication else { return nil } - return CapturedFocus( - pid: app.processIdentifier, - processName: app.localizedName - ) - } - - static func runningApp(for captured: CapturedFocus) -> NSRunningApplication? { - NSRunningApplication(processIdentifier: captured.pid) - } + // The Accessibility-backed half of the press-time focus capture. `FocusCapture` + // itself, `CapturedFocus` and `FocusedFieldContext` are declared in + // `FocusContext.swift`, which builds on every platform; this file is macOS only. + extension FocusCapture { + @MainActor + static func captureFrontmost() -> CapturedFocus? { + guard let app = NSWorkspace.shared.frontmostApplication else { return nil } + return CapturedFocus( + pid: app.processIdentifier, + processName: app.localizedName + ) + } - /// Accessibility-derived priming read from the system-wide focused UI element - /// at dictation start (see `TranscriptionContext`). Every field is - /// best-effort: any signal that can't be read is `nil`, and a fully-empty - /// result simply means less context, never an error. - struct FocusedFieldContext: Sendable { - /// Text immediately preceding the insertion point ("prior chunk context"). - let priorText: String? - /// The text currently selected in the focused field — the dictation will - /// replace it, so it primes the model on what the utterance is about. - let selectedText: String? - /// The focused window's title — a dense topic hint. - let windowTitle: String? - /// A short label for the focused field ("To", "Search", "Message"). - let fieldLabel: String? - /// Whether `mustRedactContents` refused to read this field — a password input, - /// or an element whose role couldn't be read at all (that guard fails closed). - /// `priorText`/`selectedText` are always nil when this is true, but the flag is - /// carried separately because "we read nothing" and "we refused to read" - /// license different downstream behaviour: only the latter means the transcript - /// the user is about to dictate must not be remembered as history (see - /// `DictationSession.recentDictations`). + /// Reads window title, field label, and up to `maxPriorChars` of text before + /// the cursor from the focused UI element in a single Accessibility traversal. + /// + /// Returns `.empty` whenever nothing can be read — the process lacks + /// Accessibility trust or no element is focused. Each field is independently + /// best-effort. Requires the same Accessibility permission the app already + /// holds for paste injection, so it adds no new prompt. /// - /// Defaults to `false` so a test fixture describes an ordinary field without - /// restating it; the production capture always passes its real verdict. - var isSecure: Bool = false + /// Secure text fields (password inputs) are detected by role **or** subrole and + /// never have their contents read. This guard is what keeps a typed password out + /// of the developer-mode log and — since the text before the cursor is sent as + /// the request's contextual prompt (`STTPrompt`) — off the wire + /// entirely. The check + /// fails closed: an unreadable role is treated as secure, since it can't be + /// shown not to be. + /// + /// Deliberately `nonisolated`: each read below is a synchronous cross-process + /// IPC round trip into the frontmost app, and an unresponsive app blocks the + /// calling thread until the AX messaging timeout. On the main actor that froze + /// the overlay and menu bar right at hotkey press; callers run this off-main + /// (the AX *client* read APIs are thread-safe — see `systemFocusedElement`). + nonisolated static func captureFieldContext(maxPriorChars: Int = 320, maxSelectedChars: Int = 320) + -> FocusedFieldContext + { + guard let (element, isSecure) = focusedElementForReading() else { return .empty } + // Both text slices stay nil for a secure field — the whole point of the guard + // above — so the reads that feed them are skipped wholesale rather than each + // being individually conditional. + var prior: String? + var selected: String? + if !isSecure { + // One read of the selected range serves both slices: its location *is* the + // insertion point the prior-text slice ends at, and its extent is what the + // selected-text slice asks for. Reading it here rather than inside each slice + // costs the capture one cross-process round trip for it instead of two. + let selection = selectedTextRange(of: element) + // `visibleTextOrNil` collapses an all-invisible read (e.g. Google Docs' lone + // U+200B before the caret) to nil so it can't masquerade as real prior text. + prior = visibleTextOrNil(priorText(of: element, selection: selection, maxChars: maxPriorChars)) + // The selected range's text (empty when there's no selection). Capped like + // prior text so a huge highlight can't dominate the context budget. + selected = visibleTextOrNil(selectedText(of: element, selection: selection, maxChars: maxSelectedChars)) + } + return FocusedFieldContext( + priorText: prior, + selectedText: selected, + windowTitle: clip(windowTitle(of: element), to: 120), + fieldLabel: clip(fieldLabel(of: element), to: 80), + isSecure: isSecure) + } - static let empty = FocusedFieldContext( - priorText: nil, selectedText: nil, windowTitle: nil, fieldLabel: nil) - } + /// What an Accessibility read of the selection found. + enum SelectionRead: Equatable { + case text(String) + /// A readable focused element with no selection (a bare caret), or a secure + /// field. A definite answer: the press should dictate. + case none + /// No focused element could be reached. Electron apps such as Claude.app + /// answer the system-wide focus query with `cannotComplete`, and the + /// selection in their web content is invisible to AX. Only here may the + /// caller fall back to copying (`SelectionCopy`). + case unreadable + } - /// Reads window title, field label, and up to `maxPriorChars` of text before - /// the cursor from the focused UI element in a single Accessibility traversal. - /// - /// Returns `.empty` whenever nothing can be read — the process lacks - /// Accessibility trust or no element is focused. Each field is independently - /// best-effort. Requires the same Accessibility permission the app already - /// holds for paste injection, so it adds no new prompt. - /// - /// Secure text fields (password inputs) are detected by role **or** subrole and - /// never have their contents read. This guard is what keeps a typed password out - /// of the developer-mode log and — since the text before the cursor is sent as - /// the request's contextual prompt (`STTPrompt`) — off the wire - /// entirely. The check - /// fails closed: an unreadable role is treated as secure, since it can't be - /// shown not to be. - /// - /// Deliberately `nonisolated`: each read below is a synchronous cross-process - /// IPC round trip into the frontmost app, and an unresponsive app blocks the - /// calling thread until the AX messaging timeout. On the main actor that froze - /// the overlay and menu bar right at hotkey press; callers run this off-main - /// (the AX *client* read APIs are thread-safe — see `systemFocusedElement`). - nonisolated static func captureFieldContext(maxPriorChars: Int = 320, maxSelectedChars: Int = 320) - -> FocusedFieldContext - { - guard let (element, isSecure) = focusedElementForReading() else { return .empty } - // Both text slices stay nil for a secure field — the whole point of the guard - // above — so the reads that feed them are skipped wholesale rather than each - // being individually conditional. - var prior: String? - var selected: String? - if !isSecure { - // One read of the selected range serves both slices: its location *is* the - // insertion point the prior-text slice ends at, and its extent is what the - // selected-text slice asks for. Reading it here rather than inside each slice - // costs the capture one cross-process round trip for it instead of two. + /// Just the focused field's selected text, up to `maxChars` — the read the + /// experimental read-aloud press makes (`SelectionSpeaker.focusedSelection`). + /// A narrower traversal than `captureFieldContext` because it sits on the + /// press path ahead of the dictate-or-speak decision, so every round trip it + /// skips is latency a dictation press doesn't pay. Same secure-field guard, + /// same off-main rule. + nonisolated static func captureSelectedText(maxChars: Int) -> SelectionRead { + guard AXIsProcessTrusted() else { return .none } + guard let (element, isSecure) = focusedElementForReading() else { return .unreadable } + guard !isSecure else { return .none } let selection = selectedTextRange(of: element) - // `visibleTextOrNil` collapses an all-invisible read (e.g. Google Docs' lone - // U+200B before the caret) to nil so it can't masquerade as real prior text. - prior = visibleTextOrNil(priorText(of: element, selection: selection, maxChars: maxPriorChars)) - // The selected range's text (empty when there's no selection). Capped like - // prior text so a huge highlight can't dominate the context budget. - selected = visibleTextOrNil(selectedText(of: element, selection: selection, maxChars: maxSelectedChars)) + // A bare caret is the common case (every dictation press while read-aloud + // is on), so skip `selectedText`'s `kAXSelectedText` fallback read for it. + // That fallback is for elements that expose no range at all. + if let selection, selection.length == 0 { return .none } + let text = visibleTextOrNil(selectedText(of: element, selection: selection, maxChars: maxChars)) + return text.map(SelectionRead.text) ?? .none } - return FocusedFieldContext( - priorText: prior, - selectedText: selected, - windowTitle: clip(windowTitle(of: element), to: 120), - fieldLabel: clip(fieldLabel(of: element), to: 80), - isSecure: isSecure) - } - /// What an Accessibility read of the selection found. - enum SelectionRead: Equatable { - case text(String) - /// A readable focused element with no selection (a bare caret), or a secure - /// field. A definite answer: the press should dictate. - case none - /// No focused element could be reached. Electron apps such as Claude.app - /// answer the system-wide focus query with `cannotComplete`, and the - /// selection in their web content is invisible to AX. Only here may the - /// caller fall back to copying (`SelectionCopy`). - case unreadable - } - - /// Just the focused field's selected text, up to `maxChars` — the read the - /// experimental read-aloud press makes (`SelectionSpeaker.focusedSelection`). - /// A narrower traversal than `captureFieldContext` because it sits on the - /// press path ahead of the dictate-or-speak decision, so every round trip it - /// skips is latency a dictation press doesn't pay. Same secure-field guard, - /// same off-main rule. - nonisolated static func captureSelectedText(maxChars: Int) -> SelectionRead { - guard AXIsProcessTrusted() else { return .none } - guard let (element, isSecure) = focusedElementForReading() else { return .unreadable } - guard !isSecure else { return .none } - let selection = selectedTextRange(of: element) - // A bare caret is the common case (every dictation press while read-aloud - // is on), so skip `selectedText`'s `kAXSelectedText` fallback read for it. - // That fallback is for elements that expose no range at all. - if let selection, selection.length == 0 { return .none } - let text = visibleTextOrNil(selectedText(of: element, selection: selection, maxChars: maxChars)) - return text.map(SelectionRead.text) ?? .none - } - - /// The focused element and whether its contents must stay unread, or nil when - /// the process isn't trusted or nothing is focused. Shared by both captures, - /// so the password-field guard is spelled once. - /// - /// The whole decision, including the fail-closed arm, lives in - /// `mustRedactContents`, where it is unit-tested. This needs a live AX element, - /// so anything decided inline here would be covered by nothing. - private nonisolated static func focusedElementForReading() -> (element: AXUIElement, isSecure: Bool)? { - guard AXIsProcessTrusted(), let element = systemFocusedElement() else { return nil } - let isSecure = mustRedactContents( - role: stringValue(element, kAXRoleAttribute), - subrole: stringValue(element, kAXSubroleAttribute)) - return (element, isSecure) - } + /// The focused element and whether its contents must stay unread, or nil when + /// the process isn't trusted or nothing is focused. Shared by both captures, + /// so the password-field guard is spelled once. + /// + /// The whole decision, including the fail-closed arm, lives in + /// `mustRedactContents`, where it is unit-tested. This needs a live AX element, + /// so anything decided inline here would be covered by nothing. + private nonisolated static func focusedElementForReading() -> (element: AXUIElement, isSecure: Bool)? { + guard AXIsProcessTrusted(), let element = systemFocusedElement() else { return nil } + let isSecure = mustRedactContents( + role: stringValue(element, kAXRoleAttribute), + subrole: stringValue(element, kAXSubroleAttribute)) + return (element, isSecure) + } - /// Cap on each cross-process AX round trip this process makes. An unresponsive - /// frontmost app costs a read this long, not the ~6 s system default; the - /// context capture is best-effort priming, so partial answers beat waiting. - private static let axMessagingTimeoutSeconds: Float = 1 + /// Cap on each cross-process AX round trip this process makes. An unresponsive + /// frontmost app costs a read this long, not the ~6 s system default; the + /// context capture is best-effort priming, so partial answers beat waiting. + private static let axMessagingTimeoutSeconds: Float = 1 - // Internal, not private: `hasEditableFocusedElement` in - // FocusCapture+Editability.swift calls it from another file. - /// The system-wide focused UI element, or `nil` when none is resolvable - /// (process not trusted, or nothing focused). The Accessibility *client* read - /// APIs are thread-safe, so this serves both the off-main context capture - /// and the injector's off-main editability check. - nonisolated static func systemFocusedElement() -> AXUIElement? { - let system = AXUIElementCreateSystemWide() - // Setting the timeout on the system-wide element applies it process-wide - // (per AXUIElement.h), bounding this focused-element lookup AND every later - // read — including ones on *other* element refs a per-element timeout would - // miss (the window element behind `windowTitle`, the editability probes). - AXUIElementSetMessagingTimeout(system, axMessagingTimeoutSeconds) - var focusedRef: CFTypeRef? - guard - AXUIElementCopyAttributeValue(system, kAXFocusedUIElementAttribute as CFString, &focusedRef) - == .success, - let focusedRef - else { return nil } - return axElement(focusedRef) - } + // Internal, not private: `hasEditableFocusedElement` in + // FocusCapture+Editability.swift calls it from another file. + /// The system-wide focused UI element, or `nil` when none is resolvable + /// (process not trusted, or nothing focused). The Accessibility *client* read + /// APIs are thread-safe, so this serves both the off-main context capture + /// and the injector's off-main editability check. + nonisolated static func systemFocusedElement() -> AXUIElement? { + let system = AXUIElementCreateSystemWide() + // Setting the timeout on the system-wide element applies it process-wide + // (per AXUIElement.h), bounding this focused-element lookup AND every later + // read — including ones on *other* element refs a per-element timeout would + // miss (the window element behind `windowTitle`, the editability probes). + AXUIElementSetMessagingTimeout(system, axMessagingTimeoutSeconds) + var focusedRef: CFTypeRef? + guard + AXUIElementCopyAttributeValue(system, kAXFocusedUIElementAttribute as CFString, &focusedRef) + == .success, + let focusedRef + else { return nil } + return axElement(focusedRef) + } - // MARK: - Checked CF downcasts - // - // AX attribute values arrive as CFTypeRef from *other apps'* accessibility - // implementations, so a misbehaving app returning the wrong CF type must read - // as "nothing readable" (nil), never flow onward mistyped. CF bridging makes - // `as?` a compile-time "always succeeds" warning (an error under - // -warnings-as-errors), so the runtime check is CFGetTypeID; after it the - // force-cast below each guard is provably safe — these two helpers are the - // only force_cast sites in the repo. Internal (not private) so the unit tests - // can exercise both arms without Accessibility trust. + // MARK: - Checked CF downcasts + // + // AX attribute values arrive as CFTypeRef from *other apps'* accessibility + // implementations, so a misbehaving app returning the wrong CF type must read + // as "nothing readable" (nil), never flow onward mistyped. CF bridging makes + // `as?` a compile-time "always succeeds" warning (an error under + // -warnings-as-errors), so the runtime check is CFGetTypeID; after it the + // force-cast below each guard is provably safe — these two helpers are the + // only force_cast sites in the repo. Internal (not private) so the unit tests + // can exercise both arms without Accessibility trust. - /// `ref` as an `AXUIElement`, or `nil` when it's some other CF type. - nonisolated static func axElement(_ ref: CFTypeRef) -> AXUIElement? { - guard CFGetTypeID(ref) == AXUIElementGetTypeID() else { return nil } - // swiftlint:disable:next force_cast - return (ref as! AXUIElement) - } + /// `ref` as an `AXUIElement`, or `nil` when it's some other CF type. + nonisolated static func axElement(_ ref: CFTypeRef) -> AXUIElement? { + guard CFGetTypeID(ref) == AXUIElementGetTypeID() else { return nil } + // swiftlint:disable:next force_cast + return (ref as! AXUIElement) + } - /// `ref` decoded as an `AXValue`-wrapped `CFRange` (the selected-text-range - /// payload), or `nil` when it's some other CF type or a non-range `AXValue` - /// (`AXValueGetValue` checks the payload type and refuses a mismatch). - nonisolated static func axRange(_ ref: CFTypeRef) -> CFRange? { - guard CFGetTypeID(ref) == AXValueGetTypeID() else { return nil } - // swiftlint:disable:next force_cast - let value = ref as! AXValue - var range = CFRange() - guard AXValueGetValue(value, .cfRange, &range) else { return nil } - return range - } + /// `ref` decoded as an `AXValue`-wrapped `CFRange` (the selected-text-range + /// payload), or `nil` when it's some other CF type or a non-range `AXValue` + /// (`AXValueGetValue` checks the payload type and refuses a mismatch). + nonisolated static func axRange(_ ref: CFTypeRef) -> CFRange? { + guard CFGetTypeID(ref) == AXValueGetTypeID() else { return nil } + // swiftlint:disable:next force_cast + let value = ref as! AXValue + var range = CFRange() + guard AXValueGetValue(value, .cfRange, &range) else { return nil } + return range + } - /// The element's selected text range — a zero-length range when there's just a - /// caret — or `nil` when it exposes no readable selection. Its `location` is the - /// insertion point. - private nonisolated static func selectedTextRange(of element: AXUIElement) -> CFRange? { - var rangeRef: CFTypeRef? - guard - AXUIElementCopyAttributeValue(element, kAXSelectedTextRangeAttribute as CFString, &rangeRef) - == .success, - let rangeRef - else { return nil } - return axRange(rangeRef) - } + /// The element's selected text range — a zero-length range when there's just a + /// caret — or `nil` when it exposes no readable selection. Its `location` is the + /// insertion point. + private nonisolated static func selectedTextRange(of element: AXUIElement) -> CFRange? { + var rangeRef: CFTypeRef? + guard + AXUIElementCopyAttributeValue(element, kAXSelectedTextRangeAttribute as CFString, &rangeRef) + == .success, + let rangeRef + else { return nil } + return axRange(rangeRef) + } - /// The element's text for `range` via the parameterized "string for range" - /// attribute, so only the slice asked for crosses Accessibility IPC rather than - /// the whole field value. Shared by the prior-text and selected-text reads, - /// which differ only in the range they request and how they post-process it. - private nonisolated static func string(of element: AXUIElement, in range: CFRange) -> String? { - var range = range - guard let axRange = AXValueCreate(.cfRange, &range) else { return nil } - var sliceRef: CFTypeRef? - guard - AXUIElementCopyParameterizedAttributeValue( - element, kAXStringForRangeParameterizedAttribute as CFString, axRange, &sliceRef) - == .success, - let slice = sliceRef as? String - else { return nil } - return slice - } + /// The element's text for `range` via the parameterized "string for range" + /// attribute, so only the slice asked for crosses Accessibility IPC rather than + /// the whole field value. Shared by the prior-text and selected-text reads, + /// which differ only in the range they request and how they post-process it. + private nonisolated static func string(of element: AXUIElement, in range: CFRange) -> String? { + var range = range + guard let axRange = AXValueCreate(.cfRange, &range) else { return nil } + var sliceRef: CFTypeRef? + guard + AXUIElementCopyParameterizedAttributeValue( + element, kAXStringForRangeParameterizedAttribute as CFString, axRange, &sliceRef) + == .success, + let slice = sliceRef as? String + else { return nil } + return slice + } - /// Up to `maxChars` of text immediately preceding the insertion point, or - /// `nil` when the element exposes no readable text before the cursor. - /// `selection` is the already-read selected range (see `selectedTextRange`). - private nonisolated static func priorText( - of element: AXUIElement, selection: CFRange?, maxChars: Int - ) -> String? { - // Insertion point = the location of the (possibly empty) selected range. - let caret = selection?.location ?? -1 + /// Up to `maxChars` of text immediately preceding the insertion point, or + /// `nil` when the element exposes no readable text before the cursor. + /// `selection` is the already-read selected range (see `selectedTextRange`). + private nonisolated static func priorText( + of element: AXUIElement, selection: CFRange?, maxChars: Int + ) -> String? { + // Insertion point = the location of the (possibly empty) selected range. + let caret = selection?.location ?? -1 - // Prefer the parameterized "string for range" so we read only the slice we - // need (cheap even in huge documents) rather than the whole field value. - if caret > 0 { - let start = max(0, caret - maxChars) - if let slice = string(of: element, in: CFRange(location: start, length: caret - start)), - !slice.isEmpty - { - return slice + // Prefer the parameterized "string for range" so we read only the slice we + // need (cheap even in huge documents) rather than the whole field value. + if caret > 0 { + let start = max(0, caret - maxChars) + if let slice = string(of: element, in: CFRange(location: start, length: caret - start)), + !slice.isEmpty + { + return slice + } } - } - // Fallback: read the full value and clip to the caret (or the tail). Read it - // RAW — `caret` is a UTF-16 offset into the untrimmed value, so a trimmed - // string would be indexed with offsets that no longer refer to it. - return priorSlice(full: rawStringValue(element, kAXValueAttribute) ?? "", caret: caret, maxChars: maxChars) - } + // Fallback: read the full value and clip to the caret (or the tail). Read it + // RAW — `caret` is a UTF-16 offset into the untrimmed value, so a trimmed + // string would be indexed with offsets that no longer refer to it. + return priorSlice(full: rawStringValue(element, kAXValueAttribute) ?? "", caret: caret, maxChars: maxChars) + } - /// Up to `maxChars` of selected text, or `nil` when the element exposes no - /// readable selection. Slices `selection` through the parameterized - /// string-for-range attribute first so a huge highlight does not copy the full - /// selection across Accessibility IPC before being clipped locally; falls back - /// to `kAXSelectedText` for elements that expose no string-for-range. - private nonisolated static func selectedText( - of element: AXUIElement, selection: CFRange?, maxChars: Int - ) -> String? { - if var selectedRange = selection, selectedRange.length > 0 { - selectedRange.length = min(selectedRange.length, maxChars) - if let slice = string(of: element, in: selectedRange) { - return clip(slice.trimmedNonEmpty(), to: maxChars) + /// Up to `maxChars` of selected text, or `nil` when the element exposes no + /// readable selection. Slices `selection` through the parameterized + /// string-for-range attribute first so a huge highlight does not copy the full + /// selection across Accessibility IPC before being clipped locally; falls back + /// to `kAXSelectedText` for elements that expose no string-for-range. + private nonisolated static func selectedText( + of element: AXUIElement, selection: CFRange?, maxChars: Int + ) -> String? { + if var selectedRange = selection, selectedRange.length > 0 { + selectedRange.length = min(selectedRange.length, maxChars) + if let slice = string(of: element, in: selectedRange) { + return clip(slice.trimmedNonEmpty(), to: maxChars) + } } - } - return clip(stringValue(element, kAXSelectedTextAttribute), to: maxChars) - } + return clip(stringValue(element, kAXSelectedTextAttribute), to: maxChars) + } - /// The title of the window containing the focused element, if exposed. - private nonisolated static func windowTitle(of element: AXUIElement) -> String? { - guard let window = elementValue(element, kAXWindowAttribute) else { return nil } - return stringValue(window, kAXTitleAttribute) - } + /// The title of the window containing the focused element, if exposed. + private nonisolated static func windowTitle(of element: AXUIElement) -> String? { + guard let window = elementValue(element, kAXWindowAttribute) else { return nil } + return stringValue(window, kAXTitleAttribute) + } - /// A short, human-meaningful label for the field, chosen by priority from the - /// attributes the focused element exposes. - private nonisolated static func fieldLabel(of element: AXUIElement) -> String? { - selectLabel( - placeholder: stringValue(element, kAXPlaceholderValueAttribute), - description: stringValue(element, kAXDescriptionAttribute), - title: stringValue(element, kAXTitleAttribute), - roleDescription: stringValue(element, kAXRoleDescriptionAttribute)) - } + /// A short, human-meaningful label for the field, chosen by priority from the + /// attributes the focused element exposes. + private nonisolated static func fieldLabel(of element: AXUIElement) -> String? { + selectLabel( + placeholder: stringValue(element, kAXPlaceholderValueAttribute), + description: stringValue(element, kAXDescriptionAttribute), + title: stringValue(element, kAXTitleAttribute), + roleDescription: stringValue(element, kAXRoleDescriptionAttribute)) + } - // Internal for the same reason as `systemFocusedElement`: the editability path in - // FocusCapture+Editability.swift reads the role through it. - /// Reads a `String`-valued AX attribute, returning `nil` for missing, - /// non-string, or blank values. Trims, which is what the *label-ish* attributes - /// want (role, title, placeholder, description). Do **not** use it for - /// `kAXValueAttribute` when a caret offset will index the result — see - /// `rawStringValue`. - nonisolated static func stringValue(_ element: AXUIElement, _ attribute: String) -> String? { - rawStringValue(element, attribute)?.trimmedNonEmpty() - } + // Internal for the same reason as `systemFocusedElement`: the editability path in + // FocusCapture+Editability.swift reads the role through it. + /// Reads a `String`-valued AX attribute, returning `nil` for missing, + /// non-string, or blank values. Trims, which is what the *label-ish* attributes + /// want (role, title, placeholder, description). Do **not** use it for + /// `kAXValueAttribute` when a caret offset will index the result — see + /// `rawStringValue`. + nonisolated static func stringValue(_ element: AXUIElement, _ attribute: String) -> String? { + rawStringValue(element, attribute)?.trimmedNonEmpty() + } - /// Reads a `String`-valued AX attribute **verbatim** — no trimming, so a - /// caret offset taken from the same element still indexes it correctly. - /// - /// `kAXSelectedTextRange` locations are UTF-16 offsets into the element's - /// *original* value. Trimming shifts every offset past a leading whitespace run - /// and shortens the string, so slicing a trimmed value with an untrimmed caret - /// silently returns the wrong text (or falls back to the whole tail, which - /// destroys the trailing-whitespace signal `withLeadingSeparator` reads). - private nonisolated static func rawStringValue(_ element: AXUIElement, _ attribute: String) -> String? { - var ref: CFTypeRef? - guard AXUIElementCopyAttributeValue(element, attribute as CFString, &ref) == .success, - let value = ref as? String - else { return nil } - return value - } + /// Reads a `String`-valued AX attribute **verbatim** — no trimming, so a + /// caret offset taken from the same element still indexes it correctly. + /// + /// `kAXSelectedTextRange` locations are UTF-16 offsets into the element's + /// *original* value. Trimming shifts every offset past a leading whitespace run + /// and shortens the string, so slicing a trimmed value with an untrimmed caret + /// silently returns the wrong text (or falls back to the whole tail, which + /// destroys the trailing-whitespace signal `withLeadingSeparator` reads). + private nonisolated static func rawStringValue(_ element: AXUIElement, _ attribute: String) -> String? { + var ref: CFTypeRef? + guard AXUIElementCopyAttributeValue(element, attribute as CFString, &ref) == .success, + let value = ref as? String + else { return nil } + return value + } - /// Reads an `AXUIElement`-valued AX attribute (e.g. the containing window). - private nonisolated static func elementValue(_ element: AXUIElement, _ attribute: String) -> AXUIElement? { - var ref: CFTypeRef? - guard AXUIElementCopyAttributeValue(element, attribute as CFString, &ref) == .success, - let ref - else { return nil } - return axElement(ref) + /// Reads an `AXUIElement`-valued AX attribute (e.g. the containing window). + private nonisolated static func elementValue(_ element: AXUIElement, _ attribute: String) -> AXUIElement? { + var ref: CFTypeRef? + guard AXUIElementCopyAttributeValue(element, attribute as CFString, &ref) == .success, + let ref + else { return nil } + return axElement(ref) + } } -} +#endif diff --git a/Sources/BlurtEngine/FocusCapture/FocusContext.swift b/Sources/BlurtEngine/FocusCapture/FocusContext.swift new file mode 100644 index 00000000..eee50ad5 --- /dev/null +++ b/Sources/BlurtEngine/FocusCapture/FocusContext.swift @@ -0,0 +1,74 @@ +/// The application that was frontmost when the user pressed the trigger — the +/// paste target, and the context's app name. +/// +/// A pid (an `Int32`, which is what `pid_t` is) and a display name rather than +/// an `NSRunningApplication`, so the value crosses the engine's public seams +/// (`InjectorProtocol.setTarget`, `DictationSession.HostFocusCapture`) on every +/// platform the engine builds for. `KeyInjector` resolves the process handle +/// from the pid on macOS; a host on iOS has no process to name — a keyboard +/// cannot see which app hosts it — and passes nil. +public struct CapturedFocus: Sendable { + public let pid: Int32 + public let processName: String? + + public init(pid: Int32, processName: String?) { + self.pid = pid + self.processName = processName + } +} + +// periphery:ignore - public for hosts outside this repo (the iOS app); the Mac app never names it. +/// Priming read from the focused field at dictation start (see +/// `TranscriptionContext`). Every field is best-effort: any signal that can't be +/// read is `nil`, and a fully-empty result simply means less context, never an +/// error. +/// +/// On macOS `FocusCapture.captureFieldContext` fills it from Accessibility. A +/// host on another platform builds it from whatever its own text surface +/// exposes — an iPhone keyboard's `documentContextBeforeInput` is `priorText`, +/// its `selectedText` is `selectedText`, and the app name, window title and +/// field label stay nil because a keyboard cannot see them — and hands it to +/// `DictationSession` through `HostFocusCapture`. +public struct FocusedFieldContext: Sendable { + /// Text immediately preceding the insertion point ("prior chunk context"). + public let priorText: String? + /// The text currently selected in the focused field — the dictation will + /// replace it, so it primes the model on what the utterance is about. + public let selectedText: String? + /// The focused window's title — a dense topic hint. + public let windowTitle: String? + /// A short label for the focused field ("To", "Search", "Message"). + public let fieldLabel: String? + /// Whether `mustRedactContents` refused to read this field — a password input, + /// or an element whose role couldn't be read at all (that guard fails closed). + /// `priorText`/`selectedText` are always nil when this is true, but the flag is + /// carried separately because "we read nothing" and "we refused to read" + /// license different downstream behaviour: only the latter means the transcript + /// the user is about to dictate must not be remembered as history (see + /// `DictationSession.recentDictations`). + /// + /// Defaults to `false` so a test fixture describes an ordinary field without + /// restating it; the production capture always passes its real verdict. + public var isSecure: Bool + + public init( + priorText: String?, selectedText: String?, windowTitle: String?, fieldLabel: String?, + isSecure: Bool = false + ) { + self.priorText = priorText + self.selectedText = selectedText + self.windowTitle = windowTitle + self.fieldLabel = fieldLabel + self.isSecure = isSecure + } + + public static let empty = FocusedFieldContext( + priorText: nil, selectedText: nil, windowTitle: nil, fieldLabel: nil) +} + +/// The press-time focus capture. The Accessibility-backed reads live in +/// `FocusCapture.swift` and `FocusCapture+Editability.swift`, which exist on +/// macOS only; the decisions that need no Accessibility call +/// (`FocusCapture+Pure.swift`) build on every platform, which is why the type +/// itself is declared here rather than beside the reads. +enum FocusCapture {} diff --git a/Sources/BlurtEngine/Injection/InjectorProtocol.swift b/Sources/BlurtEngine/Injection/InjectorProtocol.swift index 79d7eda4..d50155ff 100644 --- a/Sources/BlurtEngine/Injection/InjectorProtocol.swift +++ b/Sources/BlurtEngine/Injection/InjectorProtocol.swift @@ -1,14 +1,17 @@ -import AppKit - public protocol InjectorProtocol: Sendable { - func setTargetApp(_ app: NSRunningApplication?) async + /// Pins where the next `insert` lands: the focus captured when the trigger + /// went down, so the paste follows the user even if focus moves during + /// transcription. `nil` means "wherever focus is now". A host with no notion + /// of a target app — an iPhone keyboard inserts into the field it is already + /// attached to — receives nil and ignores it. + func setTarget(_ focus: CapturedFocus?) async /// Insert text into whatever the OS currently treats as the focus target. /// `priorText` is the text immediately before the caret (captured at press time), /// used to decide whether a separating space is needed so consecutive dictations /// don't run together; nil when the field is empty or its contents are opaque. /// `windowTitle` is the focused window's title at that same press-time capture — /// used to recognize a *continuing* dictation into the same window when - /// `priorText` is unreadable (see `KeyInjector.separatorBasis`); nil when the + /// `priorText` is unreadable (see `InsertionSeparator.basis`); nil when the /// window exposes no title. func insert(_ text: String, after priorText: String?, windowTitle: String?) async throws } diff --git a/Sources/BlurtEngine/Injection/KeyInjector+Separator.swift b/Sources/BlurtEngine/Injection/InsertionSeparator.swift similarity index 52% rename from Sources/BlurtEngine/Injection/KeyInjector+Separator.swift rename to Sources/BlurtEngine/Injection/InsertionSeparator.swift index 79aec915..717ed660 100644 --- a/Sources/BlurtEngine/Injection/KeyInjector+Separator.swift +++ b/Sources/BlurtEngine/Injection/InsertionSeparator.swift @@ -1,13 +1,16 @@ -// `KeyInjector`'s separator decision: the pure text rules for joining a new -// dictation onto whatever already sits before the caret. Split out of -// `KeyInjector.swift` to stay within the lint file-length budget, on the seam its -// tests already use — `KeyInjectorLeadingSeparatorTests` and the -// `KeyInjector.separatorBasis` suite cover exactly these two functions, and -// neither touches the pasteboard, the event system, or the actor's state. -// `resolveInsert`, which composes them with the window-identity decision, stays -// beside that state in `KeyInjector.swift` (it needs `pid_t`, and this file -// deliberately imports nothing). -extension KeyInjector { +// periphery:ignore - public for the iOS keyboard (App/BlurtiOS), which the Mac scan doesn't index. +/// The pure text rules for joining a new dictation onto whatever already sits +/// before the caret. +/// +/// Shared by `KeyInjector` — the macOS clipboard paste — and by any host that +/// inserts text itself: an iPhone keyboard writing the transcript through +/// `insertText` faces exactly this decision, with `documentContextBeforeInput` +/// as the prior text. So it lives on its own type with no platform imports +/// rather than as `KeyInjector`'s extension (where it started; the tests moved +/// with it). `KeyInjector.resolveInsert` composes these two with the +/// window-identity decision, which stays beside the actor's state. +public enum InsertionSeparator { + // periphery:ignore - public for the iOS keyboard (App/BlurtiOS), which the Mac scan doesn't index. /// Joins `text` to whatever precedes the caret with exactly one separating space, /// so consecutive dictations don't run together. Prepends a *leading* space only /// when there's preceding text (`priorText`) that doesn't already end in @@ -33,15 +36,15 @@ extension KeyInjector { /// Code — or a browser tab like Google Docs, whose canvas-rendered body is just /// as opaque) — we can't tell those apart from AX alone, so we fall back to the /// text we last pasted, but only when this dictation targets the *same window* - /// as last time (see `WindowIdentity`): that's the in-progress-run case where - /// our own paste is what now sits before the caret. This is deliberately - /// app-agnostic rather than an allowlist of "known opaque editors": a window - /// match is a reasonable proxy for "still the same document" across *any* app, - /// opaque or not, whereas a shared process id alone isn't (one browser process - /// hosts many unrelated tabs/documents). Otherwise (a different window or - /// nothing pasted yet) we return nil rather than risk a stray leading space - /// into what may be a genuinely fresh field. - static func separatorBasis(priorText: String?, lastInserted: String?, sameWindow: Bool) -> String? { + /// as last time (see `KeyInjector.WindowIdentity`): that's the in-progress-run + /// case where our own paste is what now sits before the caret. This is + /// deliberately app-agnostic rather than an allowlist of "known opaque + /// editors": a window match is a reasonable proxy for "still the same document" + /// across *any* app, opaque or not, whereas a shared process id alone isn't + /// (one browser process hosts many unrelated tabs/documents). Otherwise (a + /// different window or nothing pasted yet) we return nil rather than risk a + /// stray leading space into what may be a genuinely fresh field. + static func basis(priorText: String?, lastInserted: String?, sameWindow: Bool) -> String? { if priorText != nil { return priorText } return sameWindow ? lastInserted : nil } diff --git a/Sources/BlurtEngine/Injection/KeyInjector+SystemActions.swift b/Sources/BlurtEngine/Injection/KeyInjector+SystemActions.swift index 0315801e..2b119ea0 100644 --- a/Sources/BlurtEngine/Injection/KeyInjector+SystemActions.swift +++ b/Sources/BlurtEngine/Injection/KeyInjector+SystemActions.swift @@ -1,115 +1,117 @@ -import AppKit -import ApplicationServices -import CoreGraphics +#if os(macOS) + import AppKit + import ApplicationServices + import CoreGraphics -// The system side effects behind `KeyInjector`'s injectable seams: the default -// implementations of app activation, the frontmost-wait, the Accessibility-trust -// check, and the synthesized ⌘V. Split out from the actor's paste orchestration -// (`KeyInjector.swift`) because these are pure AppKit/CoreGraphics glue with no -// actor state — the only place `KeyInjector` needs CoreGraphics or -// ApplicationServices at all. `static`, not `private`, so the initializers in -// `KeyInjector.swift` can wire them as the defaults across the file boundary. -extension KeyInjector { - static func activate(_ app: NSRunningApplication) -> Bool { - app.activate() - } + // The system side effects behind `KeyInjector`'s injectable seams: the default + // implementations of app activation, the frontmost-wait, the Accessibility-trust + // check, and the synthesized ⌘V. Split out from the actor's paste orchestration + // (`KeyInjector.swift`) because these are pure AppKit/CoreGraphics glue with no + // actor state — the only place `KeyInjector` needs CoreGraphics or + // ApplicationServices at all. `static`, not `private`, so the initializers in + // `KeyInjector.swift` can wire them as the defaults across the file boundary. + extension KeyInjector { + static func activate(_ app: NSRunningApplication) -> Bool { + app.activate() + } - static func waitUntilFrontmost(_ app: NSRunningApplication) async -> Bool { - let pid = app.processIdentifier - let clock = ContinuousClock() - let deadline = clock.now.advanced(by: .milliseconds(350)) - while clock.now < deadline { - if await isFrontmost(pid) { return true } - try? await Task.sleep(for: .milliseconds(10)) + static func waitUntilFrontmost(_ app: NSRunningApplication) async -> Bool { + let pid = app.processIdentifier + let clock = ContinuousClock() + let deadline = clock.now.advanced(by: .milliseconds(350)) + while clock.now < deadline { + if await isFrontmost(pid) { return true } + try? await Task.sleep(for: .milliseconds(10)) + } + // One last read: the final sleep may have carried us past the deadline just + // before the activation landed. + return await isFrontmost(pid) } - // One last read: the final sleep may have carried us past the deadline just - // before the activation landed. - return await isFrontmost(pid) - } - /// Whether `pid` owns the frontmost application right now. Named rather than - /// inlined so the poll and the post-deadline check share one expression — and so - /// neither needs `MainActor.run`'s explicit `body:` label, which a trailing - /// closure in an `if` condition can't use. - private static func isFrontmost(_ pid: pid_t) async -> Bool { - await MainActor.run { - NSWorkspace.shared.frontmostApplication?.processIdentifier == pid + /// Whether `pid` owns the frontmost application right now. Named rather than + /// inlined so the poll and the post-deadline check share one expression — and so + /// neither needs `MainActor.run`'s explicit `body:` label, which a trailing + /// closure in an `if` condition can't use. + private static func isFrontmost(_ pid: pid_t) async -> Bool { + await MainActor.run { + NSWorkspace.shared.frontmostApplication?.processIdentifier == pid + } } - } - static func accessibilityTrusted() -> Bool { - AXIsProcessTrusted() - } + static func accessibilityTrusted() -> Bool { + AXIsProcessTrusted() + } - /// Seconds since the user last pressed a key or clicked, anywhere. Reads the - /// HID system state's idle counters: no permission needed (no event tap, no - /// Input Monitoring), and only hardware input moves them — verified that our - /// own synthesized ⌘V leaves them untouched, so a paste can't make it look - /// like the user typed after it. Relies on every `TriggerKey` being a modifier - /// (`flagsChanged`, not `keyDown`): a non-modifier trigger would count as input - /// on every dictation and silently disable the same-window separator. - static func secondsSinceHardwareInput() -> TimeInterval { - let types: [CGEventType] = [.keyDown, .leftMouseDown, .rightMouseDown, .otherMouseDown] - return types.map { CGEventSource.secondsSinceLastEventType(.hidSystemState, eventType: $0) } - .min() ?? .infinity - } + /// Seconds since the user last pressed a key or clicked, anywhere. Reads the + /// HID system state's idle counters: no permission needed (no event tap, no + /// Input Monitoring), and only hardware input moves them — verified that our + /// own synthesized ⌘V leaves them untouched, so a paste can't make it look + /// like the user typed after it. Relies on every `TriggerKey` being a modifier + /// (`flagsChanged`, not `keyDown`): a non-modifier trigger would count as input + /// on every dictation and silently disable the same-window separator. + static func secondsSinceHardwareInput() -> TimeInterval { + let types: [CGEventType] = [.keyDown, .leftMouseDown, .rightMouseDown, .otherMouseDown] + return types.map { CGEventSource.secondsSinceLastEventType(.hidSystemState, eventType: $0) } + .min() ?? .infinity + } - /// The Cmd-V key-down/key-up pair, or `nil` when CoreGraphics refuses to build - /// them. Split from `postCmdV` because only the *posting* is untestable: building - /// an event needs no Accessibility trust, while posting one sends a live - /// keystroke into whatever app has focus — not something `swift test` may do to - /// the machine it runs on. So the part carrying an actual invariant (the ⌘ flag - /// on both events and the `kVK_ANSI_V` keycode, which is what makes the paste a - /// paste) is asserted in `KeyInjectorSystemActionsTests`, and only the two - /// `.post` calls below stay covered by running the app. - static func cmdVEvents() -> (down: CGEvent, up: CGEvent)? { - commandEvents(virtualKey: 0x09) // kVK_ANSI_V - } + /// The Cmd-V key-down/key-up pair, or `nil` when CoreGraphics refuses to build + /// them. Split from `postCmdV` because only the *posting* is untestable: building + /// an event needs no Accessibility trust, while posting one sends a live + /// keystroke into whatever app has focus — not something `swift test` may do to + /// the machine it runs on. So the part carrying an actual invariant (the ⌘ flag + /// on both events and the `kVK_ANSI_V` keycode, which is what makes the paste a + /// paste) is asserted in `KeyInjectorSystemActionsTests`, and only the two + /// `.post` calls below stay covered by running the app. + static func cmdVEvents() -> (down: CGEvent, up: CGEvent)? { + commandEvents(virtualKey: 0x09) // kVK_ANSI_V + } - /// The Cmd-C pair, for the read-aloud press's copy fallback (`SelectionCopy`). - static func cmdCEvents() -> (down: CGEvent, up: CGEvent)? { - commandEvents(virtualKey: 0x08) // kVK_ANSI_C - } + /// The Cmd-C pair, for the read-aloud press's copy fallback (`SelectionCopy`). + static func cmdCEvents() -> (down: CGEvent, up: CGEvent)? { + commandEvents(virtualKey: 0x08) // kVK_ANSI_C + } - private static func commandEvents(virtualKey: CGKeyCode) -> (down: CGEvent, up: CGEvent)? { - guard let source = CGEventSource(stateID: .combinedSessionState), - let down = CGEvent(keyboardEventSource: source, virtualKey: virtualKey, keyDown: true), - let up = CGEvent(keyboardEventSource: source, virtualKey: virtualKey, keyDown: false) - else { return nil } - // Set on both: a key-up carrying no ⌘ reads as the modifier having been - // released mid-chord, which some apps treat as cancelling the shortcut. - down.flags = .maskCommand - up.flags = .maskCommand - return (down, up) - } + private static func commandEvents(virtualKey: CGKeyCode) -> (down: CGEvent, up: CGEvent)? { + guard let source = CGEventSource(stateID: .combinedSessionState), + let down = CGEvent(keyboardEventSource: source, virtualKey: virtualKey, keyDown: true), + let up = CGEvent(keyboardEventSource: source, virtualKey: virtualKey, keyDown: false) + else { return nil } + // Set on both: a key-up carrying no ⌘ reads as the modifier having been + // released mid-chord, which some apps treat as cancelling the shortcut. + down.flags = .maskCommand + up.flags = .maskCommand + return (down, up) + } - /// Posts Cmd-V. Returns `false` if the events couldn't be built. The real side - /// effect (a keystroke into the focused app) is why this is the injectable seam - /// tests replace. - static func postCmdV() -> Bool { - post(cmdVEvents()) - } + /// Posts Cmd-V. Returns `false` if the events couldn't be built. The real side + /// effect (a keystroke into the focused app) is why this is the injectable seam + /// tests replace. + static func postCmdV() -> Bool { + post(cmdVEvents()) + } - /// Posts Cmd-C, the same way and for the same reasons as `postCmdV`. The - /// annotated session tap also sits *after* the trigger's own session tap, so - /// `DictationKeyTap` never sees this keystroke. Otherwise its `keyDown` would - /// read as a ⌘-combo and cancel the very press that asked for the copy. - static func postCmdC() -> Bool { - post(cmdCEvents()) - } + /// Posts Cmd-C, the same way and for the same reasons as `postCmdV`. The + /// annotated session tap also sits *after* the trigger's own session tap, so + /// `DictationKeyTap` never sees this keystroke. Otherwise its `keyDown` would + /// read as a ⌘-combo and cancel the very press that asked for the copy. + static func postCmdC() -> Bool { + post(cmdCEvents()) + } - private static func post(_ events: (down: CGEvent, up: CGEvent)?) -> Bool { - guard let (down, up) = events else { return false } - // Post to the annotated session tap rather than the HID tap: the session tap - // honors exactly the flags set above instead of OR-ing in the live hardware - // modifier state, so a still-held hotkey modifier can't corrupt Cmd-V into a - // combo the target app ignores. (We deliberately don't suppress local events - // during the post: `setLocalEventsFilterDuringSuppressionState` lingers for - // the source's ~0.25s suppression interval and would swallow the user's next - // dictation keypress right after a paste — the annotated tap already prevents - // the modifier merge that suppression was guarding against.) - down.post(tap: .cgAnnotatedSessionEventTap) - up.post(tap: .cgAnnotatedSessionEventTap) - return true + private static func post(_ events: (down: CGEvent, up: CGEvent)?) -> Bool { + guard let (down, up) = events else { return false } + // Post to the annotated session tap rather than the HID tap: the session tap + // honors exactly the flags set above instead of OR-ing in the live hardware + // modifier state, so a still-held hotkey modifier can't corrupt Cmd-V into a + // combo the target app ignores. (We deliberately don't suppress local events + // during the post: `setLocalEventsFilterDuringSuppressionState` lingers for + // the source's ~0.25s suppression interval and would swallow the user's next + // dictation keypress right after a paste — the annotated tap already prevents + // the modifier merge that suppression was guarding against.) + down.post(tap: .cgAnnotatedSessionEventTap) + up.post(tap: .cgAnnotatedSessionEventTap) + return true + } } -} +#endif diff --git a/Sources/BlurtEngine/Injection/KeyInjector.swift b/Sources/BlurtEngine/Injection/KeyInjector.swift index 1517a584..5ae8a5f4 100644 --- a/Sources/BlurtEngine/Injection/KeyInjector.swift +++ b/Sources/BlurtEngine/Injection/KeyInjector.swift @@ -1,318 +1,326 @@ -import AppKit +#if os(macOS) + import AppKit -public actor KeyInjector: InjectorProtocol { - private var targetApp: NSRunningApplication? + public actor KeyInjector: InjectorProtocol { + private var targetApp: NSRunningApplication? - /// How long to wait after posting Cmd-V before clearing and restoring the - /// pasteboard. The paste is asynchronous: clear too soon and a slow target - /// (VNC/Remote Desktop, heavy Electron apps) reads an already-cleared - /// clipboard. 400 ms is a pragmatic margin over local apps' near-instant read. - /// This wait runs in a background settle task (see `pendingSettle`), *not* on - /// `insert`'s caller, so it no longer delays the pipeline re-arming. - private let pasteSettleDuration: Duration + /// How long to wait after posting Cmd-V before clearing and restoring the + /// pasteboard. The paste is asynchronous: clear too soon and a slow target + /// (VNC/Remote Desktop, heavy Electron apps) reads an already-cleared + /// clipboard. 400 ms is a pragmatic margin over local apps' near-instant read. + /// This wait runs in a background settle task (see `pendingSettle`), *not* on + /// `insert`'s caller, so it no longer delays the pipeline re-arming. + private let pasteSettleDuration: Duration - /// Synthesizes the paste keystroke. Returns `false` if the event subsystem - /// couldn't build the events. Injectable so tests exercise the clipboard - /// save/restore logic without posting a real Cmd-V into the focused app. - private let postPaste: @Sendable () -> Bool + /// Synthesizes the paste keystroke. Returns `false` if the event subsystem + /// couldn't build the events. Injectable so tests exercise the clipboard + /// save/restore logic without posting a real Cmd-V into the focused app. + private let postPaste: @Sendable () -> Bool - /// Identifies a window by its app's pid plus its title — a pid alone isn't - /// enough (one browser process hosts many unrelated tabs/documents), and a - /// title alone isn't stable across apps, so both travel together as one - /// value rather than two optionals that would have to stay in sync by hand. - /// Title equality is itself a heuristic, not a strict identity check (two - /// tabs can coincidentally share a title, or a title can change mid-session - /// for reasons unrelated to the document, e.g. an unsaved-changes marker); - /// accepted here because the safe failure mode is just a missing separator. - /// - /// Internal rather than private so `resolveInsert` — the pure form of the - /// decision it feeds — can be driven from tests with plain pids and titles. - struct WindowIdentity: Equatable { - let pid: pid_t - let title: String + /// Identifies a window by its app's pid plus its title — a pid alone isn't + /// enough (one browser process hosts many unrelated tabs/documents), and a + /// title alone isn't stable across apps, so both travel together as one + /// value rather than two optionals that would have to stay in sync by hand. + /// Title equality is itself a heuristic, not a strict identity check (two + /// tabs can coincidentally share a title, or a title can change mid-session + /// for reasons unrelated to the document, e.g. an unsaved-changes marker); + /// accepted here because the safe failure mode is just a missing separator. + /// + /// Internal rather than private so `resolveInsert` — the pure form of the + /// decision it feeds — can be driven from tests with plain pids and titles. + struct WindowIdentity: Equatable { + let pid: pid_t + let title: String - // Hand-written rather than synthesized: Periphery's static analysis can't - // see through a compiler-synthesized `==`, so it flags `pid`/`title` as - // assigned-but-unused. Spelling out the comparison gives it a real usage - // site to index. - static func == (lhs: WindowIdentity, rhs: WindowIdentity) -> Bool { - lhs.pid == rhs.pid && lhs.title == rhs.title + // Hand-written rather than synthesized: Periphery's static analysis can't + // see through a compiler-synthesized `==`, so it flags `pid`/`title` as + // assigned-but-unused. Spelling out the comparison gives it a real usage + // site to index. + static func == (lhs: WindowIdentity, rhs: WindowIdentity) -> Bool { + lhs.pid == rhs.pid && lhs.title == rhs.title + } } - } - - /// The previous insert's resolution: the exact text it pasted (including any - /// leading separator it added) and the window it landed in. One value rather - /// than two optionals kept in sync by hand — the same reasoning as - /// `WindowIdentity` above, and they are only ever written together. - /// - /// Lets `insert` recover spacing in Accessibility-opaque editors - /// (Electron/Monaco, e.g. VS Code — and, just as opaque, a browser tab like - /// Google Docs) where no prior text can be read: if the next dictation targets - /// the same window, the text we just pasted is what now precedes the caret, so - /// it drives the separator decision (see `separatorBasis`). A different window - /// — a different tab, a different file, or an unreadable title — means a - /// different field, so the fallback doesn't fire. - /// - /// `at` is when the paste was posted, so `performInsert` can tell whether the - /// user has typed or clicked since (see `secondsSinceUserInput`): pressing - /// Return for a new paragraph, or clicking elsewhere, leaves the window title - /// unchanged while the caret no longer follows our paste. - private var lastInserted: (resolved: ResolvedInsert, at: ContinuousClock.Instant)? - - /// Seconds since the user last pressed a key or clicked. Once that input is - /// newer than the last paste, the caret may be anywhere (a fresh line in - /// Google Docs, issue #206), so the same-window fallback is dropped rather - /// than risk a stray leading space. Injectable so tests don't depend on the - /// host's real keyboard and mouse (defaults to "no input" there). - private let secondsSinceUserInput: @Sendable () -> TimeInterval - - /// Tail of the paste chain: each insert links behind the previous insert's - /// ENTIRE critical section — paste *plus* its backgrounded settle/restore — so - /// two pastes can never interleave on the global `NSPasteboard`. `insert` - /// itself returns as soon as its paste is posted (the settle trails inside the - /// chain link), so the pipeline re-arms immediately while the next paste still - /// waits out the restore window. Chaining replaces a hand-rolled continuation - /// mutex whose lock had to be handed off to the settle task. Exposed - /// (internal get) so tests can await the deferred restore before asserting - /// clipboard state; production never reads it. - private(set) var pendingSettle: Task? - - /// Brings the captured target app back to the foreground before pasting. - /// Injectable so tests can cover activation failure without depending on - /// another live application. - private let activateTarget: @Sendable (NSRunningApplication) -> Bool - /// Waits until the app activation has actually become visible to - /// `NSWorkspace` before editability is checked and Cmd-V is posted. Kept - /// injectable so tests that stub activation don't depend on the host's live - /// foreground app. - private let waitForTargetActivation: @Sendable (NSRunningApplication) async -> Bool + /// The previous insert's resolution: the exact text it pasted (including any + /// leading separator it added) and the window it landed in. One value rather + /// than two optionals kept in sync by hand — the same reasoning as + /// `WindowIdentity` above, and they are only ever written together. + /// + /// Lets `insert` recover spacing in Accessibility-opaque editors + /// (Electron/Monaco, e.g. VS Code — and, just as opaque, a browser tab like + /// Google Docs) where no prior text can be read: if the next dictation targets + /// the same window, the text we just pasted is what now precedes the caret, so + /// it drives the separator decision (see `separatorBasis`). A different window + /// — a different tab, a different file, or an unreadable title — means a + /// different field, so the fallback doesn't fire. + /// + /// `at` is when the paste was posted, so `performInsert` can tell whether the + /// user has typed or clicked since (see `secondsSinceUserInput`): pressing + /// Return for a new paragraph, or clicking elsewhere, leaves the window title + /// unchanged while the caret no longer follows our paste. + private var lastInserted: (resolved: ResolvedInsert, at: ContinuousClock.Instant)? - /// Whether the process is trusted for Accessibility. Posting the synthesized - /// Cmd-V requires it (macOS 10.14+); without it `CGEvent.post` is silently - /// dropped, so we check first and fail loudly instead of reporting a paste that - /// never happened. Injectable so tests don't depend on the host's AX state - /// (defaults to trusted there). - private let isAccessibilityTrusted: @Sendable () -> Bool + /// Seconds since the user last pressed a key or clicked. Once that input is + /// newer than the last paste, the caret may be anywhere (a fresh line in + /// Google Docs, issue #206), so the same-window fallback is dropped rather + /// than risk a stray leading space. Injectable so tests don't depend on the + /// host's real keyboard and mouse (defaults to "no input" there). + private let secondsSinceUserInput: @Sendable () -> TimeInterval - /// Whether something editable is focused to receive the paste. Checked right - /// before the synthesized ⌘V (after the target app is activated): when false, - /// the paste is skipped and the transcript is left on the clipboard instead of - /// beeping into a non-editable target. Injectable so tests don't depend on the - /// host's live focus (defaults to "editable" there). - private let hasEditableTarget: @Sendable () -> Bool + /// Tail of the paste chain: each insert links behind the previous insert's + /// ENTIRE critical section — paste *plus* its backgrounded settle/restore — so + /// two pastes can never interleave on the global `NSPasteboard`. `insert` + /// itself returns as soon as its paste is posted (the settle trails inside the + /// chain link), so the pipeline re-arms immediately while the next paste still + /// waits out the restore window. Chaining replaces a hand-rolled continuation + /// mutex whose lock had to be handed off to the settle task. Exposed + /// (internal get) so tests can await the deferred restore before asserting + /// clipboard state; production never reads it. + private(set) var pendingSettle: Task? - /// Whether the captured target app is AX-opaque — an Electron/Chromium editor - /// (VS Code, Slack) or a web browser — which can expose no editable AX signal - /// even for a real text field (Chromium builds its accessibility tree lazily, - /// and `contenteditable` composers like ChatGPT's surface no editable role). - /// When `hasEditableTarget` reads false but this is true, we still paste - /// rather than copy — dropping the user's words into a field they're clearly - /// typing in would be the worse mistake. Injectable so tests don't depend on - /// which apps are installed (defaults to "not opaque"). - private let isAXOpaqueApp: @Sendable (NSRunningApplication?) -> Bool + /// Brings the captured target app back to the foreground before pasting. + /// Injectable so tests can cover activation failure without depending on + /// another live application. + private let activateTarget: @Sendable (NSRunningApplication) -> Bool - /// The pasteboard the paste reads, writes, and restores. Behind a seam so - /// tests exercise the save/restore + changeCount logic against an in-memory - /// fake instead of the real system pasteboard — whose contents another process - /// can change mid-test, which (correctly) suppresses the restore and would - /// otherwise flake. - private let clipboard: any ClipboardAccess + /// Waits until the app activation has actually become visible to + /// `NSWorkspace` before editability is checked and Cmd-V is posted. Kept + /// injectable so tests that stub activation don't depend on the host's live + /// foreground app. + private let waitForTargetActivation: @Sendable (NSRunningApplication) async -> Bool - public init(pasteSettleDuration: Duration = .milliseconds(400)) { - self.init( - pasteSettleDuration: pasteSettleDuration, - postPaste: KeyInjector.postCmdV, - activateTarget: KeyInjector.activate, - waitForTargetActivation: KeyInjector.waitUntilFrontmost, - isAccessibilityTrusted: KeyInjector.accessibilityTrusted, - hasEditableTarget: FocusCapture.hasEditableFocusedElement, - isAXOpaqueApp: FocusCapture.isAXOpaqueApp, - secondsSinceUserInput: KeyInjector.secondsSinceHardwareInput) - } + /// Whether the process is trusted for Accessibility. Posting the synthesized + /// Cmd-V requires it (macOS 10.14+); without it `CGEvent.post` is silently + /// dropped, so we check first and fail loudly instead of reporting a paste that + /// never happened. Injectable so tests don't depend on the host's AX state + /// (defaults to trusted there). + private let isAccessibilityTrusted: @Sendable () -> Bool - init( - pasteSettleDuration: Duration, - postPaste: @escaping @Sendable () -> Bool, - activateTarget: @escaping @Sendable (NSRunningApplication) -> Bool = KeyInjector.activate, - waitForTargetActivation: @escaping @Sendable (NSRunningApplication) async -> Bool = { _ in true }, - isAccessibilityTrusted: @escaping @Sendable () -> Bool = { true }, - hasEditableTarget: @escaping @Sendable () -> Bool = { true }, - isAXOpaqueApp: @escaping @Sendable (NSRunningApplication?) -> Bool = { _ in false }, - secondsSinceUserInput: @escaping @Sendable () -> TimeInterval = { .infinity }, - clipboard: any ClipboardAccess = SystemClipboard() - ) { - self.pasteSettleDuration = pasteSettleDuration - self.postPaste = postPaste - self.activateTarget = activateTarget - self.waitForTargetActivation = waitForTargetActivation - self.isAccessibilityTrusted = isAccessibilityTrusted - self.hasEditableTarget = hasEditableTarget - self.isAXOpaqueApp = isAXOpaqueApp - self.secondsSinceUserInput = secondsSinceUserInput - self.clipboard = clipboard - } + /// Whether something editable is focused to receive the paste. Checked right + /// before the synthesized ⌘V (after the target app is activated): when false, + /// the paste is skipped and the transcript is left on the clipboard instead of + /// beeping into a non-editable target. Injectable so tests don't depend on the + /// host's live focus (defaults to "editable" there). + private let hasEditableTarget: @Sendable () -> Bool - public func setTargetApp(_ app: NSRunningApplication?) async { - targetApp = app - } + /// Whether the captured target app is AX-opaque — an Electron/Chromium editor + /// (VS Code, Slack) or a web browser — which can expose no editable AX signal + /// even for a real text field (Chromium builds its accessibility tree lazily, + /// and `contenteditable` composers like ChatGPT's surface no editable role). + /// When `hasEditableTarget` reads false but this is true, we still paste + /// rather than copy — dropping the user's words into a field they're clearly + /// typing in would be the worse mistake. Injectable so tests don't depend on + /// which apps are installed (defaults to "not opaque"). + private let isAXOpaqueApp: @Sendable (NSRunningApplication?) -> Bool - /// Brings the captured target app frontmost before a paste, then waits a beat - /// for the activation to settle. No-op when no target was captured. Throws - /// `.targetAppLost` if the app quit between capture and now, or if activation - /// fails — pasting into whatever currently has focus would land the - /// keystrokes in the wrong place. (`performInsert` puts the transcript on the - /// clipboard before letting this error propagate, so the words survive.) - /// Takes the target as a parameter (the caller's snapshot) rather than - /// re-reading `targetApp`, so a `setTargetApp` racing in across the settle - /// sleep can't swap it mid-paste. - private func activateTargetApp(_ target: NSRunningApplication?) async throws(BlurtError) { - guard let target else { return } - guard !target.isTerminated else { throw BlurtError.targetAppLost } - guard activateTarget(target) else { throw BlurtError.targetAppLost } - guard await waitForTargetActivation(target) else { throw BlurtError.targetAppLost } - } + /// The pasteboard the paste reads, writes, and restores. Behind a seam so + /// tests exercise the save/restore + changeCount logic against an in-memory + /// fake instead of the real system pasteboard — whose contents another process + /// can change mid-test, which (correctly) suppresses the restore and would + /// otherwise flake. + private let clipboard: any ClipboardAccess - public func insert(_ text: String, after priorText: String? = nil, windowTitle: String? = nil) async throws { - guard !text.isEmpty else { return } - // Serialize the whole paste critical section by chaining behind the previous - // insert's link (which includes its settle/restore — see `pendingSettle`). - // The actor is reentrant across the `await`s in `performInsert`, so an - // unserialized second insert would snapshot the pasteboard while it still - // holds this insert's transcript and later restore that instead of the - // user's original clipboard. - let previous = pendingSettle - let timeout = pasteSettleDuration - let paste = Task<(@Sendable () -> Void)?, any Error> { - await previous?.value - return try await self.performInsert(text, after: priorText, windowTitle: windowTitle) - } - // Strong `self` captures, deliberately: each link is bounded (one paste, one - // settle sleep — no cycle), and a weak capture would let an injector torn - // down mid-window skip the paste. The restore closure captures the clipboard - // (not `self`), so the user's contents come back even if the injector is - // gone by the time the settle fires. - pendingSettle = Task { - guard let restore = try? await paste.value else { return } - try? await Task.sleep(for: timeout) - restore() + public init(pasteSettleDuration: Duration = .milliseconds(400)) { + self.init( + pasteSettleDuration: pasteSettleDuration, + postPaste: KeyInjector.postCmdV, + activateTarget: KeyInjector.activate, + waitForTargetActivation: KeyInjector.waitUntilFrontmost, + isAccessibilityTrusted: KeyInjector.accessibilityTrusted, + hasEditableTarget: FocusCapture.hasEditableFocusedElement, + isAXOpaqueApp: FocusCapture.isAXOpaqueApp, + secondsSinceUserInput: KeyInjector.secondsSinceHardwareInput) } - // Forward the caller's cancellation into the chain link: the paste task is - // unstructured, so `pipelineTask.cancel()` in the session wouldn't otherwise - // reach `performInsert`'s cancellation gates. - try await withTaskCancellationHandler { - _ = try await paste.value - } onCancel: { - paste.cancel() + + init( + pasteSettleDuration: Duration, + postPaste: @escaping @Sendable () -> Bool, + activateTarget: @escaping @Sendable (NSRunningApplication) -> Bool = KeyInjector.activate, + waitForTargetActivation: @escaping @Sendable (NSRunningApplication) async -> Bool = { _ in true }, + isAccessibilityTrusted: @escaping @Sendable () -> Bool = { true }, + hasEditableTarget: @escaping @Sendable () -> Bool = { true }, + isAXOpaqueApp: @escaping @Sendable (NSRunningApplication?) -> Bool = { _ in false }, + secondsSinceUserInput: @escaping @Sendable () -> TimeInterval = { .infinity }, + clipboard: any ClipboardAccess = SystemClipboard() + ) { + self.pasteSettleDuration = pasteSettleDuration + self.postPaste = postPaste + self.activateTarget = activateTarget + self.waitForTargetActivation = waitForTargetActivation + self.isAccessibilityTrusted = isAccessibilityTrusted + self.hasEditableTarget = hasEditableTarget + self.isAXOpaqueApp = isAXOpaqueApp + self.secondsSinceUserInput = secondsSinceUserInput + self.clipboard = clipboard } - } - /// The paste critical section: runs with the chain's guarantee that no other - /// insert (or its settle) is mid-flight. Returns the deferred clipboard-restore - /// action for the settle link to run once the paste has landed. - private func performInsert( - _ text: String, after priorText: String?, windowTitle: String? - ) async throws -> @Sendable () -> Void { - try Task.checkCancellation() - // Snapshot the target at entry and use only the local below: this method - // suspends (activation settle), the actor is reentrant, and a - // setTargetApp() interleaving mid-insert must not make us activate one app - // while judging editability and recording `lastInserted` for another. - let target = targetApp - // Our last paste only still precedes the caret if the user hasn't typed or - // clicked since; otherwise forget it, same as a window change. - let continuity = lastInserted.flatMap { last in - secondsSinceUserInput() >= last.at.duration(to: .now) / .seconds(1) ? last.resolved : nil + /// Pins the paste target: the app that was frontmost at press time, resolved + /// from its pid here because `NSRunningApplication` is the handle activation + /// needs while a pid is what `CapturedFocus` can carry across the engine's + /// public seam on every platform. A pid that no longer resolves (the app quit + /// before the press was even joined) leaves no target, and `insert` then + /// pastes wherever focus is — the same thing a nil capture always meant. + public func setTarget(_ focus: CapturedFocus?) async { + targetApp = focus.flatMap { NSRunningApplication(processIdentifier: $0.pid) } } - let resolved = KeyInjector.resolveInsert( - text: text, priorText: priorText, windowTitle: windowTitle, - targetPID: target?.processIdentifier, - lastInserted: continuity) - let finalText = resolved.text - do { - try await activateTargetApp(target) - } catch { - // The target app quit or refused activation between capture and paste. - // Transcription already succeeded, so leave the words on the clipboard — - // the pipeline degrades this to the quiet "copied" notice instead of a - // hard failure that would lose the dictation. - clipboard.write(finalText) - throw error + + /// Brings the captured target app frontmost before a paste, then waits a beat + /// for the activation to settle. No-op when no target was captured. Throws + /// `.targetAppLost` if the app quit between capture and now, or if activation + /// fails — pasting into whatever currently has focus would land the + /// keystrokes in the wrong place. (`performInsert` puts the transcript on the + /// clipboard before letting this error propagate, so the words survive.) + /// Takes the target as a parameter (the caller's snapshot) rather than + /// re-reading `targetApp`, so a `setTarget` racing in across the settle + /// sleep can't swap it mid-paste. + private func activateTargetApp(_ target: NSRunningApplication?) async throws(BlurtError) { + guard let target else { return } + guard !target.isTerminated else { throw BlurtError.targetAppLost } + guard activateTarget(target) else { throw BlurtError.targetAppLost } + guard await waitForTargetActivation(target) else { throw BlurtError.targetAppLost } } - // Final cancellation gate before the irreversible paste: a cancel() that - // landed during activation must not type into the focused app. - try Task.checkCancellation() - // Nothing editable is focused (checked now that the target app is frontmost): - // a synthesized ⌘V would just make macOS beep. Leave the transcript on the - // clipboard so the user can paste it by hand, and signal the pipeline to show - // a quiet "copied" notice instead of typing. The exception is an AX-opaque - // app — an Electron editor (VS Code, Slack) or a web browser — which can - // report no editable signal even for a real text field; there we still paste - // rather than drop the user's words. - guard hasEditableTarget() || isAXOpaqueApp(target) else { - clipboard.write(finalText) - throw BlurtError.noEditableTarget + + public func insert(_ text: String, after priorText: String? = nil, windowTitle: String? = nil) async throws { + guard !text.isEmpty else { return } + // Serialize the whole paste critical section by chaining behind the previous + // insert's link (which includes its settle/restore — see `pendingSettle`). + // The actor is reentrant across the `await`s in `performInsert`, so an + // unserialized second insert would snapshot the pasteboard while it still + // holds this insert's transcript and later restore that instead of the + // user's original clipboard. + let previous = pendingSettle + let timeout = pasteSettleDuration + let paste = Task<(@Sendable () -> Void)?, any Error> { + await previous?.value + return try await self.performInsert(text, after: priorText, windowTitle: windowTitle) + } + // Strong `self` captures, deliberately: each link is bounded (one paste, one + // settle sleep — no cycle), and a weak capture would let an injector torn + // down mid-window skip the paste. The restore closure captures the clipboard + // (not `self`), so the user's contents come back even if the injector is + // gone by the time the settle fires. + pendingSettle = Task { + guard let restore = try? await paste.value else { return } + try? await Task.sleep(for: timeout) + restore() + } + // Forward the caller's cancellation into the chain link: the paste task is + // unstructured, so `pipelineTask.cancel()` in the session wouldn't otherwise + // reach `performInsert`'s cancellation gates. + try await withTaskCancellationHandler { + _ = try await paste.value + } onCancel: { + paste.cancel() + } } - // Bail before touching the pasteboard if we can't actually paste: without - // Accessibility trust the Cmd-V post below is silently dropped, which would - // otherwise clobber-and-restore the clipboard for a paste that never lands. - guard isAccessibilityTrusted() else { throw BlurtError.accessibilityPermissionMissing } - // Put the transcript on the clipboard and keep the deferred restore that - // brings the user's contents back once the paste settles. - let restore = clipboard.writeAndPrepareRestore(finalText) - // If the event subsystem won't synthesize the keystroke, the paste can't - // happen. The transcript is already on the pasteboard — leave it there (the - // user's words beat the stale pre-paste snapshot) so the failure degrades - // to the "copied" notice, matching the lost-target path above. - guard postPaste() else { throw BlurtError.targetAppLost } - // The paste is posted and the text is visible. Record what landed (including - // any leading separator) and which window it landed in so a following - // dictation into the same window can recover its spacing, then hand the - // deferred restore back to the chain link (see `insert`). `insert` returns - // now — so the pipeline reaches `.idle` and re-arms without waiting out the - // restore window — while the next paste still serializes behind the settle. - lastInserted = (resolved, .now) - return restore - } + /// The paste critical section: runs with the chain's guarantee that no other + /// insert (or its settle) is mid-flight. Returns the deferred clipboard-restore + /// action for the settle link to run once the paste has landed. + private func performInsert( + _ text: String, after priorText: String?, windowTitle: String? + ) async throws -> @Sendable () -> Void { + try Task.checkCancellation() + // Snapshot the target at entry and use only the local below: this method + // suspends (activation settle), the actor is reentrant, and a + // setTarget() interleaving mid-insert must not make us activate one app + // while judging editability and recording `lastInserted` for another. + let target = targetApp + // Our last paste only still precedes the caret if the user hasn't typed or + // clicked since; otherwise forget it, same as a window change. + let continuity = lastInserted.flatMap { last in + secondsSinceUserInput() >= last.at.duration(to: .now) / .seconds(1) ? last.resolved : nil + } + let resolved = KeyInjector.resolveInsert( + text: text, priorText: priorText, windowTitle: windowTitle, + targetPID: target?.processIdentifier, + lastInserted: continuity) + let finalText = resolved.text + do { + try await activateTargetApp(target) + } catch { + // The target app quit or refused activation between capture and paste. + // Transcription already succeeded, so leave the words on the clipboard — + // the pipeline degrades this to the quiet "copied" notice instead of a + // hard failure that would lose the dictation. + clipboard.write(finalText) + throw error + } + // Final cancellation gate before the irreversible paste: a cancel() that + // landed during activation must not type into the focused app. + try Task.checkCancellation() + // Nothing editable is focused (checked now that the target app is frontmost): + // a synthesized ⌘V would just make macOS beep. Leave the transcript on the + // clipboard so the user can paste it by hand, and signal the pipeline to show + // a quiet "copied" notice instead of typing. The exception is an AX-opaque + // app — an Electron editor (VS Code, Slack) or a web browser — which can + // report no editable signal even for a real text field; there we still paste + // rather than drop the user's words. + guard hasEditableTarget() || isAXOpaqueApp(target) else { + clipboard.write(finalText) + throw BlurtError.noEditableTarget + } + // Bail before touching the pasteboard if we can't actually paste: without + // Accessibility trust the Cmd-V post below is silently dropped, which would + // otherwise clobber-and-restore the clipboard for a paste that never lands. + guard isAccessibilityTrusted() else { throw BlurtError.accessibilityPermissionMissing } - /// What one insert resolves to before anything is activated or pasted: the - /// exact text to write (any leading separator included) and the window - /// identity to remember it against. Fed straight back in as the next insert's - /// `lastInserted` — a resolution and the memory it becomes are the same pair, - /// so there is one type for both. - struct ResolvedInsert: Equatable { - let text: String - let window: WindowIdentity? - } + // Put the transcript on the clipboard and keep the deferred restore that + // brings the user's contents back once the paste settles. + let restore = clipboard.writeAndPrepareRestore(finalText) + // If the event subsystem won't synthesize the keystroke, the paste can't + // happen. The transcript is already on the pasteboard — leave it there (the + // user's words beat the stale pre-paste snapshot) so the failure degrades + // to the "copied" notice, matching the lost-target path above. + guard postPaste() else { throw BlurtError.targetAppLost } + // The paste is posted and the text is visible. Record what landed (including + // any leading separator) and which window it landed in so a following + // dictation into the same window can recover its spacing, then hand the + // deferred restore back to the chain link (see `insert`). `insert` returns + // now — so the pipeline reaches `.idle` and re-arms without waiting out the + // restore window — while the next paste still serializes behind the settle. + lastInserted = (resolved, .now) + return restore + } + + /// What one insert resolves to before anything is activated or pasted: the + /// exact text to write (any leading separator included) and the window + /// identity to remember it against. Fed straight back in as the next insert's + /// `lastInserted` — a resolution and the memory it becomes are the same pair, + /// so there is one type for both. + struct ResolvedInsert: Equatable { + let text: String + let window: WindowIdentity? + } - /// Both of `performInsert`'s continuity decisions, as one pure function taking - /// the paste target as a plain pid rather than an `NSRunningApplication`: - /// which text to write, and which window to remember writing it into. - /// - /// A pid is all the decision ever needed. Computed inline in `performInsert`, - /// it could only be reached through a full `insert`, so its four cases were - /// covered by scraping `NSWorkspace` for live processes — one case required - /// the host to be running two — which activated the developer's foreground app - /// and failed whenever the unordered pick landed on a background-only process. - /// Here the same cases are plain values, and `insert` keeps one end-to-end test - /// for the wiring. - static func resolveInsert( - text: String, - priorText: String?, - windowTitle: String?, - targetPID: pid_t?, - lastInserted: ResolvedInsert? - ) -> ResolvedInsert { - let currentWindow = targetPID.flatMap { pid in - windowTitle.map { WindowIdentity(pid: pid, title: $0) } + /// Both of `performInsert`'s continuity decisions, as one pure function taking + /// the paste target as a plain pid rather than an `NSRunningApplication`: + /// which text to write, and which window to remember writing it into. + /// + /// A pid is all the decision ever needed. Computed inline in `performInsert`, + /// it could only be reached through a full `insert`, so its four cases were + /// covered by scraping `NSWorkspace` for live processes — one case required + /// the host to be running two — which activated the developer's foreground app + /// and failed whenever the unordered pick landed on a background-only process. + /// Here the same cases are plain values, and `insert` keeps one end-to-end test + /// for the wiring. + static func resolveInsert( + text: String, + priorText: String?, + windowTitle: String?, + targetPID: pid_t?, + lastInserted: ResolvedInsert? + ) -> ResolvedInsert { + let currentWindow = targetPID.flatMap { pid in + windowTitle.map { WindowIdentity(pid: pid, title: $0) } + } + // `currentWindow.map { ... } ?? false` rather than `currentWindow == lastInserted?.window`: + // both sides being nil (nothing readable this time, nothing pasted last time) + // must not count as a match. + let sameWindow = currentWindow.map { $0 == lastInserted?.window } ?? false + let basis = InsertionSeparator.basis( + priorText: priorText, lastInserted: lastInserted?.text, sameWindow: sameWindow) + return ResolvedInsert(text: InsertionSeparator.withLeadingSeparator(text, after: basis), window: currentWindow) } - // `currentWindow.map { ... } ?? false` rather than `currentWindow == lastInserted?.window`: - // both sides being nil (nothing readable this time, nothing pasted last time) - // must not count as a match. - let sameWindow = currentWindow.map { $0 == lastInserted?.window } ?? false - let basis = separatorBasis( - priorText: priorText, lastInserted: lastInserted?.text, sameWindow: sameWindow) - return ResolvedInsert(text: withLeadingSeparator(text, after: basis), window: currentWindow) } -} +#endif diff --git a/Sources/BlurtEngine/Injection/SystemClipboard.swift b/Sources/BlurtEngine/Injection/SystemClipboard.swift index 5e7390c9..a750a3f9 100644 --- a/Sources/BlurtEngine/Injection/SystemClipboard.swift +++ b/Sources/BlurtEngine/Injection/SystemClipboard.swift @@ -1,159 +1,161 @@ -import AppKit -import Foundation +#if os(macOS) + import AppKit + import Foundation -/// A thread-safe, value-type representation of a pasteboard item containing its data -/// keyed by pasteboard types, allowing it to cross concurrency boundaries safely. -struct SendablePasteboardItem: Sendable { - let dataMap: [NSPasteboard.PasteboardType: Data] -} + /// A thread-safe, value-type representation of a pasteboard item containing its data + /// keyed by pasteboard types, allowing it to cross concurrency boundaries safely. + struct SendablePasteboardItem: Sendable { + let dataMap: [NSPasteboard.PasteboardType: Data] + } -/// A best-effort snapshot of the whole pasteboard, taken before a paste so the -/// user's contents can be put back afterwards. -/// -/// The distinction that matters: this type is only ever produced when the -/// pasteboard was *readable*. `SystemClipboard.snapshot()` returns `nil` when the -/// read itself failed, so a failure can never be mistaken for "the clipboard was -/// empty" — conflating those two is what made the restore clear the user's -/// clipboard instead of restoring it. -/// -/// Known, inherent limitation: pasteboard data can be *promised* (provided lazily -/// by the owning app on demand). A promise cannot be copied, only materialized, and -/// an app may decline. Types that decline are absent from `dataMap`, so a restore -/// of promise-backed contents is a best-effort downgrade, not a byte-faithful -/// round trip — `plainText` exists to keep at least the text when that happens. -struct PasteboardSnapshot: Sendable { - /// One entry per item the pasteboard held, in order. An entry with an empty - /// `dataMap` means the item existed but none of its representations could be - /// materialized — distinct from the pasteboard having held no items at all, - /// which is `items.isEmpty`. - let items: [SendablePasteboardItem] - /// The pasteboard's plain-string flavor, kept separately as a restore floor for - /// when no item's representations could be materialized. - let plainText: String? -} + /// A best-effort snapshot of the whole pasteboard, taken before a paste so the + /// user's contents can be put back afterwards. + /// + /// The distinction that matters: this type is only ever produced when the + /// pasteboard was *readable*. `SystemClipboard.snapshot()` returns `nil` when the + /// read itself failed, so a failure can never be mistaken for "the clipboard was + /// empty" — conflating those two is what made the restore clear the user's + /// clipboard instead of restoring it. + /// + /// Known, inherent limitation: pasteboard data can be *promised* (provided lazily + /// by the owning app on demand). A promise cannot be copied, only materialized, and + /// an app may decline. Types that decline are absent from `dataMap`, so a restore + /// of promise-backed contents is a best-effort downgrade, not a byte-faithful + /// round trip — `plainText` exists to keep at least the text when that happens. + struct PasteboardSnapshot: Sendable { + /// One entry per item the pasteboard held, in order. An entry with an empty + /// `dataMap` means the item existed but none of its representations could be + /// materialized — distinct from the pasteboard having held no items at all, + /// which is `items.isEmpty`. + let items: [SendablePasteboardItem] + /// The pasteboard's plain-string flavor, kept separately as a restore floor for + /// when no item's representations could be materialized. + let plainText: String? + } -/// The two clipboard operations `KeyInjector` actually performs around a paste — -/// a plain overwrite, or an overwrite that can later restore what it displaced — -/// rather than exposing NSPasteboard's raw change-count/multi-item bookkeeping. -/// A seam so tests substitute a trivial in-memory fake that never has to -/// re-derive the pasteboard's change-count semantics to stay faithful. -protocol ClipboardAccess: Sendable { - /// Overwrite the clipboard with a single plain-string item, discarding the - /// previous contents. The degraded paste paths call this to leave the - /// transcript on the clipboard for a manual paste. - func write(_ text: String) - /// Overwrite the clipboard with `text`, returning an action that restores the - /// previous contents — but only if nothing else has written to the clipboard - /// in the meantime (so a user copy during the paste-settle window survives). - /// Call the returned action once the paste has settled. - func writeAndPrepareRestore(_ text: String) -> @Sendable () -> Void -} + /// The two clipboard operations `KeyInjector` actually performs around a paste — + /// a plain overwrite, or an overwrite that can later restore what it displaced — + /// rather than exposing NSPasteboard's raw change-count/multi-item bookkeeping. + /// A seam so tests substitute a trivial in-memory fake that never has to + /// re-derive the pasteboard's change-count semantics to stay faithful. + protocol ClipboardAccess: Sendable { + /// Overwrite the clipboard with a single plain-string item, discarding the + /// previous contents. The degraded paste paths call this to leave the + /// transcript on the clipboard for a manual paste. + func write(_ text: String) + /// Overwrite the clipboard with `text`, returning an action that restores the + /// previous contents — but only if nothing else has written to the clipboard + /// in the meantime (so a user copy during the paste-settle window survives). + /// Call the returned action once the paste has settled. + func writeAndPrepareRestore(_ text: String) -> @Sendable () -> Void + } -/// `ClipboardAccess` backed by the real `NSPasteboard.general`. The change-count -/// comparison that gates the deferred restore lives here, behind the seam, so a -/// fake never re-implements it. -struct SystemClipboard: ClipboardAccess { - func write(_ text: String) { setString(text) } + /// `ClipboardAccess` backed by the real `NSPasteboard.general`. The change-count + /// comparison that gates the deferred restore lives here, behind the seam, so a + /// fake never re-implements it. + struct SystemClipboard: ClipboardAccess { + func write(_ text: String) { setString(text) } - func writeAndPrepareRestore(_ text: String) -> @Sendable () -> Void { - let saved = snapshot() - setString(text) - // Snapshot the change count our own write produced. If anything else writes - // to the pasteboard before the restore fires (e.g. the user copies - // something), the count moves and the restore leaves their newer contents - // alone rather than clobbering them with the stale pre-paste snapshot. - let ourChangeCount = changeCount - return { [self] in - guard changeCount == ourChangeCount else { return } - // A nil snapshot means the pasteboard could not be read at all. There is - // nothing to put back, so leave the transcript on the clipboard (the same - // degraded-but-recoverable outcome as the `.noTarget` path) rather than - // clearing the user's clipboard to nothing. - guard let saved else { return } - restore(saved) + func writeAndPrepareRestore(_ text: String) -> @Sendable () -> Void { + let saved = snapshot() + setString(text) + // Snapshot the change count our own write produced. If anything else writes + // to the pasteboard before the restore fires (e.g. the user copies + // something), the count moves and the restore leaves their newer contents + // alone rather than clobbering them with the stale pre-paste snapshot. + let ourChangeCount = changeCount + return { [self] in + guard changeCount == ourChangeCount else { return } + // A nil snapshot means the pasteboard could not be read at all. There is + // nothing to put back, so leave the transcript on the clipboard (the same + // degraded-but-recoverable outcome as the `.noTarget` path) rather than + // clearing the user's clipboard to nothing. + guard let saved else { return } + restore(saved) + } } - } - // MARK: - NSPasteboard building blocks (also exercised directly by SystemClipboardTests) + // MARK: - NSPasteboard building blocks (also exercised directly by SystemClipboardTests) - var changeCount: Int { NSPasteboard.general.changeCount } + var changeCount: Int { NSPasteboard.general.changeCount } - /// Snapshots the pasteboard, or `nil` when it can't be read at all - /// (`pasteboardItems` is documented to return nil on error). Callers must treat - /// nil as "don't restore" — never as an empty clipboard. - func snapshot() -> PasteboardSnapshot? { - let pasteboard = NSPasteboard.general - guard let items = pasteboard.pasteboardItems else { return nil } - let captured = items.map { item in - var dataMap: [NSPasteboard.PasteboardType: Data] = [:] - for type in item.types { - // A nil read is a promised representation the owning app declined to - // materialize; record what we did get and let `plainText` be the floor. - if let data = item.data(forType: type) { - dataMap[type] = data + /// Snapshots the pasteboard, or `nil` when it can't be read at all + /// (`pasteboardItems` is documented to return nil on error). Callers must treat + /// nil as "don't restore" — never as an empty clipboard. + func snapshot() -> PasteboardSnapshot? { + let pasteboard = NSPasteboard.general + guard let items = pasteboard.pasteboardItems else { return nil } + let captured = items.map { item in + var dataMap: [NSPasteboard.PasteboardType: Data] = [:] + for type in item.types { + // A nil read is a promised representation the owning app declined to + // materialize; record what we did get and let `plainText` be the floor. + if let data = item.data(forType: type) { + dataMap[type] = data + } } + return SendablePasteboardItem(dataMap: dataMap) } - return SendablePasteboardItem(dataMap: dataMap) + return PasteboardSnapshot(items: captured, plainText: pasteboard.string(forType: .string)) } - return PasteboardSnapshot(items: captured, plainText: pasteboard.string(forType: .string)) - } - - func setString(_ text: String) { - let pasteboard = NSPasteboard.general - pasteboard.clearContents() - pasteboard.setString(text, forType: .string) - } - func restore(_ saved: PasteboardSnapshot) { - let pasteboard = NSPasteboard.general - - // The pasteboard genuinely held nothing, so restoring it means emptying it. - // Safe to clear because the snapshot succeeded — a *failed* read is nil and - // never reaches here. - guard !saved.items.isEmpty else { + func setString(_ text: String) { + let pasteboard = NSPasteboard.general pasteboard.clearContents() - return + pasteboard.setString(text, forType: .string) } - // Build the items BEFORE clearing. Clearing first and then discovering there - // is nothing to write is how the user's clipboard got destroyed whenever a - // snapshot came back degraded. - let rebuilt = saved.items.compactMap { item -> NSPasteboardItem? in - guard !item.dataMap.isEmpty else { return nil } - let pasteboardItem = NSPasteboardItem() - for (type, data) in item.dataMap { - pasteboardItem.setData(data, forType: type) + func restore(_ saved: PasteboardSnapshot) { + let pasteboard = NSPasteboard.general + + // The pasteboard genuinely held nothing, so restoring it means emptying it. + // Safe to clear because the snapshot succeeded — a *failed* read is nil and + // never reaches here. + guard !saved.items.isEmpty else { + pasteboard.clearContents() + return } - return pasteboardItem - } - if !rebuilt.isEmpty { - pasteboard.clearContents() - // `writeObjects` can refuse the batch; fall through to the text floor - // rather than leaving the pasteboard empty. - if pasteboard.writeObjects(rebuilt) { return } - } + // Build the items BEFORE clearing. Clearing first and then discovering there + // is nothing to write is how the user's clipboard got destroyed whenever a + // snapshot came back degraded. + let rebuilt = saved.items.compactMap { item -> NSPasteboardItem? in + guard !item.dataMap.isEmpty else { return nil } + let pasteboardItem = NSPasteboardItem() + for (type, data) in item.dataMap { + pasteboardItem.setData(data, forType: type) + } + return pasteboardItem + } - // Either nothing was materializable, or the write above was refused. Put the - // text back if we have it. - // - // Precise about the one case this does NOT recover: if items DID materialize, - // `writeObjects` refused them, and there was no plain-string flavor, the - // pasteboard has already been cleared and stays empty. That is not gated on - // `plainText` being non-nil on purpose — skipping the item write whenever - // there's no text flavor would refuse to restore an image-only or - // file-only clipboard, which is a far more common clipboard than a refused - // batch write. `writeObjects` failing on a freshly-cleared pasteboard holding - // valid `NSPasteboardItem`s is a programming error, not a runtime condition, - // and NSPasteboard offers no way to test a write before clearing. - if let plainText = saved.plainText { - pasteboard.clearContents() - pasteboard.setString(plainText, forType: .string) + if !rebuilt.isEmpty { + pasteboard.clearContents() + // `writeObjects` can refuse the batch; fall through to the text floor + // rather than leaving the pasteboard empty. + if pasteboard.writeObjects(rebuilt) { return } + } + + // Either nothing was materializable, or the write above was refused. Put the + // text back if we have it. + // + // Precise about the one case this does NOT recover: if items DID materialize, + // `writeObjects` refused them, and there was no plain-string flavor, the + // pasteboard has already been cleared and stays empty. That is not gated on + // `plainText` being non-nil on purpose — skipping the item write whenever + // there's no text flavor would refuse to restore an image-only or + // file-only clipboard, which is a far more common clipboard than a refused + // batch write. `writeObjects` failing on a freshly-cleared pasteboard holding + // valid `NSPasteboardItem`s is a programming error, not a runtime condition, + // and NSPasteboard offers no way to test a write before clearing. + if let plainText = saved.plainText { + pasteboard.clearContents() + pasteboard.setString(plainText, forType: .string) + } } } -} -/// The read-aloud copy fallback's view of the pasteboard (see `SelectionCopy`). -extension SystemClipboard: CopyPasteboard { - func currentString() -> String? { NSPasteboard.general.string(forType: .string) } -} + /// The read-aloud copy fallback's view of the pasteboard (see `SelectionCopy`). + extension SystemClipboard: CopyPasteboard { + func currentString() -> String? { NSPasteboard.general.string(forType: .string) } + } +#endif diff --git a/Sources/BlurtEngine/Permissions/PermissionStatus.swift b/Sources/BlurtEngine/Permissions/PermissionStatus.swift new file mode 100644 index 00000000..5f1e87d1 --- /dev/null +++ b/Sources/BlurtEngine/Permissions/PermissionStatus.swift @@ -0,0 +1,26 @@ +/// The grants a Blurt install needs, as one reading. A pure value — two Bools +/// and their derivations — consumed by the portable `SetupReadiness` policy, +/// which is why it lives apart from `PermissionsChecker`: only *reading* the +/// grants needs the macOS-only TCC probes, and the rule that decides what +/// "ready" means must build on every platform the engine does. +public struct PermissionStatus: Equatable, Sendable { + public let microphone: Bool + public let accessibility: Bool + + public init(microphone: Bool, accessibility: Bool) { + self.microphone = microphone + self.accessibility = accessibility + } + + public var allGranted: Bool { microphone && accessibility } + + /// True when `previous` had every permission and this reading no longer does — + /// i.e. the user revoked one in System Settings, possibly while no window was + /// open. The shell reacts by pulling them back into onboarding rather than + /// leaving a dead overlay, so this is a behavioural edge worth a test; it lives + /// next to `allGranted`, the derivation it's built from, rather than being + /// spelled out at the one call site that watches for it. + public func lostGrant(since previous: PermissionStatus) -> Bool { + previous.allGranted && !allGranted + } +} diff --git a/Sources/BlurtEngine/Permissions/PermissionsChecker.swift b/Sources/BlurtEngine/Permissions/PermissionsChecker.swift index 190cb6ff..1c06b8b1 100644 --- a/Sources/BlurtEngine/Permissions/PermissionsChecker.swift +++ b/Sources/BlurtEngine/Permissions/PermissionsChecker.swift @@ -1,115 +1,95 @@ -import AVFoundation -import AppKit -import ApplicationServices -import Foundation +#if os(macOS) + import AVFoundation + import AppKit + import ApplicationServices + import Foundation -public struct PermissionStatus: Equatable, Sendable { - public let microphone: Bool - public let accessibility: Bool - - public init(microphone: Bool, accessibility: Bool) { - self.microphone = microphone - self.accessibility = accessibility - } - - public var allGranted: Bool { microphone && accessibility } - - /// True when `previous` had every permission and this reading no longer does — - /// i.e. the user revoked one in System Settings, possibly while no window was - /// open. The shell reacts by pulling them back into onboarding rather than - /// leaving a dead overlay, so this is a behavioural edge worth a test; it lives - /// next to `allGranted`, the derivation it's built from, rather than being - /// spelled out at the one call site that watches for it. - public func lostGrant(since previous: PermissionStatus) -> Bool { - previous.allGranted && !allGranted - } -} - -public enum PermissionsChecker { - /// The current grant state, read without prompting. - public static func check() -> PermissionStatus { - check(micGranted: { micGranted() }, axTrusted: { AXIsProcessTrusted() }) - } + public enum PermissionsChecker { + /// The current grant state, read without prompting. + public static func check() -> PermissionStatus { + check(micGranted: { micGranted() }, axTrusted: { AXIsProcessTrusted() }) + } - /// The composition `check()` performs, with both probes injected. The probes - /// themselves read process-global TCC state the test host can't set, so with - /// them hard-coded the only assertable claim about `check()` was that it - /// doesn't crash — and the test that made it ended up restating `allGranted`'s - /// own definition instead. Injected, the wiring is assertable: each probe's - /// answer has to land in its own field, so a swapped pair fails. - static func check(micGranted: () -> Bool, axTrusted: () -> Bool) -> PermissionStatus { - PermissionStatus( - microphone: micGranted(), - accessibility: axTrusted() - ) - } + /// The composition `check()` performs, with both probes injected. The probes + /// themselves read process-global TCC state the test host can't set, so with + /// them hard-coded the only assertable claim about `check()` was that it + /// doesn't crash — and the test that made it ended up restating `allGranted`'s + /// own definition instead. Injected, the wiring is assertable: each probe's + /// answer has to land in its own field, so a swapped pair fails. + static func check(micGranted: () -> Bool, axTrusted: () -> Bool) -> PermissionStatus { + PermissionStatus( + microphone: micGranted(), + accessibility: axTrusted() + ) + } - private static func micGranted() -> Bool { - AVAudioApplication.shared.recordPermission == .granted - } + private static func micGranted() -> Bool { + AVAudioApplication.shared.recordPermission == .granted + } - public static func requestMicrophone() async -> Bool { - await AVAudioApplication.requestRecordPermission() - } + public static func requestMicrophone() async -> Bool { + await AVAudioApplication.requestRecordPermission() + } - /// Opens System Settings to Privacy › Microphone. The fallback when the in-app - /// `requestMicrophone()` prompt can't grant access — the user declined it, or - /// the system won't re-present it once the status is determined — so the - /// Microphone row still has a way forward, mirroring the Accessibility row's - /// "open Settings" flow rather than being a dead-end button. - @MainActor - public static func openMicrophoneSettings() { - guard - let url = URL( - string: - "x-apple.systempreferences:com.apple.settings.PrivacySecurity.extension?Privacy_Microphone") - else { return } - NSWorkspace.shared.open(url) - } + /// Opens System Settings to Privacy › Microphone. The fallback when the in-app + /// `requestMicrophone()` prompt can't grant access — the user declined it, or + /// the system won't re-present it once the status is determined — so the + /// Microphone row still has a way forward, mirroring the Accessibility row's + /// "open Settings" flow rather than being a dead-end button. + @MainActor + public static func openMicrophoneSettings() { + guard + let url = URL( + string: + "x-apple.systempreferences:com.apple.settings.PrivacySecurity.extension?Privacy_Microphone") + else { return } + NSWorkspace.shared.open(url) + } - /// Trigger the Accessibility permission flow: - /// 1. Make a *real* AX-protected call against another app's UI tree — - /// this is what reliably registers Blurt in TCC on macOS 26. - /// `AXIsProcessTrustedWithOptions` and the system-wide query don't - /// appear to count as "activity" for registration purposes. - /// 2. Show the trust prompt with an "Open System Settings" button. - @MainActor - public static func openAccessibilitySettings() { - forceAccessibilityActivity() - // The literal spells out `kAXTrustedCheckOptionPrompt`'s value: the SDK - // header declares the constant as a non-const `extern CFStringRef`, so it - // imports into Swift as a global `var` that strict concurrency refuses to - // reference ("shared mutable state") — the framework constant cannot be - // used here. - let prompt: NSDictionary = ["AXTrustedCheckOptionPrompt": true] - _ = AXIsProcessTrustedWithOptions(prompt) - } + /// Trigger the Accessibility permission flow: + /// 1. Make a *real* AX-protected call against another app's UI tree — + /// this is what reliably registers Blurt in TCC on macOS 26. + /// `AXIsProcessTrustedWithOptions` and the system-wide query don't + /// appear to count as "activity" for registration purposes. + /// 2. Show the trust prompt with an "Open System Settings" button. + @MainActor + public static func openAccessibilitySettings() { + forceAccessibilityActivity() + // The literal spells out `kAXTrustedCheckOptionPrompt`'s value: the SDK + // header declares the constant as a non-const `extern CFStringRef`, so it + // imports into Swift as a global `var` that strict concurrency refuses to + // reference ("shared mutable state") — the framework constant cannot be + // used here. + let prompt: NSDictionary = ["AXTrustedCheckOptionPrompt": true] + _ = AXIsProcessTrustedWithOptions(prompt) + } - /// Makes a *real* AX-protected call against another app's UI tree — what - /// reliably registers Blurt in TCC on macOS 26, and the no-prompt first step - /// of `openAccessibilitySettings`. Internal rather than private so the engine - /// tests can exercise it directly, without the trust prompt that - /// `openAccessibilitySettings` adds on top. - @MainActor - static func forceAccessibilityActivity() { - // AX reads against our own pid are NOT TCC-protected — apps can always - // read their own UI. We must target a different process so tccd sees - // a denied request and registers Blurt in the Accessibility list. - // `frontmostApplication` is Blurt itself when this runs from a - // button in our own window, which is why prior versions never worked. - let myPid = ProcessInfo.processInfo.processIdentifier - let others = NSWorkspace.shared.runningApplications.filter { - $0.processIdentifier > 0 && $0.processIdentifier != myPid + /// Makes a *real* AX-protected call against another app's UI tree — what + /// reliably registers Blurt in TCC on macOS 26, and the no-prompt first step + /// of `openAccessibilitySettings`. Internal rather than private so the engine + /// tests can exercise it directly, without the trust prompt that + /// `openAccessibilitySettings` adds on top. + @MainActor + static func forceAccessibilityActivity() { + // AX reads against our own pid are NOT TCC-protected — apps can always + // read their own UI. We must target a different process so tccd sees + // a denied request and registers Blurt in the Accessibility list. + // `frontmostApplication` is Blurt itself when this runs from a + // button in our own window, which is why prior versions never worked. + let myPid = ProcessInfo.processInfo.processIdentifier + let others = NSWorkspace.shared.runningApplications.filter { + $0.processIdentifier > 0 && $0.processIdentifier != myPid + } + let target = + others.first(where: { $0.bundleIdentifier == "com.apple.finder" }) + ?? others.first(where: { $0.activationPolicy == .regular }) + ?? others.first + guard let pid = target?.processIdentifier else { return } + let element = AXUIElementCreateApplication(pid) + var value: CFTypeRef? + _ = AXUIElementCopyAttributeValue( + element, kAXFocusedUIElementAttribute as CFString, &value) } - let target = - others.first(where: { $0.bundleIdentifier == "com.apple.finder" }) - ?? others.first(where: { $0.activationPolicy == .regular }) - ?? others.first - guard let pid = target?.processIdentifier else { return } - let element = AXUIElementCreateApplication(pid) - var value: CFTypeRef? - _ = AXUIElementCopyAttributeValue( - element, kAXFocusedUIElementAttribute as CFString, &value) - } -} + } +#endif diff --git a/Sources/BlurtEngine/Permissions/PermissionsReset.swift b/Sources/BlurtEngine/Permissions/PermissionsReset.swift index 15216a15..693d5c97 100644 --- a/Sources/BlurtEngine/Permissions/PermissionsReset.swift +++ b/Sources/BlurtEngine/Permissions/PermissionsReset.swift @@ -1,83 +1,85 @@ -import Foundation -import os +#if os(macOS) + import Foundation + import os -/// Clears the TCC grants an install holds, by running `tccutil reset ` -/// against a bundle id. The in-process half of what `scripts/reset-install.sh` -/// does from a terminal, so the Settings window's reset can offer the same thing -/// without one. -/// -/// Resetting a bundle's *own* grants needs no admin rights, which is what makes -/// this usable from the app at all. It is still a system call with no dry run: -/// pass the **running** bundle id (`Bundle.main.bundleIdentifier`), never -/// `HostIdentity.current.subsystem` — debug builds ship under -/// `dev.alex.blurt.dev` (see `project.yml`), so the constant would have a dev -/// build clearing the released Blurt's grants, the one app whose permissions -/// this process has no business touching. -public enum PermissionsReset { - private static let log = HostIdentity.current.logger("PermissionsReset") - - /// The TCC services a Blurt install can hold a grant under — the set - /// `resetAll` sweeps, and the same three `scripts/reset-install.sh` lists - /// (bash can't read this enum, so the two are kept in step by hand). + /// Clears the TCC grants an install holds, by running `tccutil reset ` + /// against a bundle id. The in-process half of what `scripts/reset-install.sh` + /// does from a terminal, so the Settings window's reset can offer the same thing + /// without one. /// - /// Raw values are `tccutil`'s own service names, which are not all the names - /// System Settings shows. - public enum Service: String, CaseIterable, Sendable { - /// Typing into other apps, and the focused-field reads that feed the paste - /// path (`KeyInjector`, `FocusCapture`). - case accessibility = "Accessibility" - /// Recording (`MicCapture`). - case microphone = "Microphone" - /// Input Monitoring — the `CGEventTap` behind the hold-to-dictate hotkey - /// (`DictationKeyTap`). `tccutil` knows it by its internal name, - /// `ListenEvent`. - case inputMonitoring = "ListenEvent" - } + /// Resetting a bundle's *own* grants needs no admin rights, which is what makes + /// this usable from the app at all. It is still a system call with no dry run: + /// pass the **running** bundle id (`Bundle.main.bundleIdentifier`), never + /// `HostIdentity.current.subsystem` — debug builds ship under + /// `dev.alex.blurt.dev` (see `project.yml`), so the constant would have a dev + /// build clearing the released Blurt's grants, the one app whose permissions + /// this process has no business touching. + public enum PermissionsReset { + private static let log = HostIdentity.current.logger("PermissionsReset") + + /// The TCC services a Blurt install can hold a grant under — the set + /// `resetAll` sweeps, and the same three `scripts/reset-install.sh` lists + /// (bash can't read this enum, so the two are kept in step by hand). + /// + /// Raw values are `tccutil`'s own service names, which are not all the names + /// System Settings shows. + public enum Service: String, CaseIterable, Sendable { + /// Typing into other apps, and the focused-field reads that feed the paste + /// path (`KeyInjector`, `FocusCapture`). + case accessibility = "Accessibility" + /// Recording (`MicCapture`). + case microphone = "Microphone" + /// Input Monitoring — the `CGEventTap` behind the hold-to-dictate hotkey + /// (`DictationKeyTap`). `tccutil` knows it by its internal name, + /// `ListenEvent`. + case inputMonitoring = "ListenEvent" + } - /// Clears `service` for `bundleID`. Returns whether `tccutil` exited 0 — a - /// non-zero exit (or a launch failure) is reported rather than thrown, since - /// every caller's answer to a failed reset is the same: say so, and leave the - /// rest of the sweep to run. - @discardableResult - public static func reset(_ service: Service, bundleID: String) -> Bool { - let proc = Process() - proc.executableURL = URL(fileURLWithPath: "/usr/bin/tccutil") - proc.arguments = ["reset", service.rawValue, bundleID] - do { - try proc.run() - proc.waitUntilExit() - let ok = proc.terminationStatus == 0 - if !ok { - log.warning( - "tccutil reset \(service.rawValue, privacy: .public) exited \(proc.terminationStatus)") + /// Clears `service` for `bundleID`. Returns whether `tccutil` exited 0 — a + /// non-zero exit (or a launch failure) is reported rather than thrown, since + /// every caller's answer to a failed reset is the same: say so, and leave the + /// rest of the sweep to run. + @discardableResult + public static func reset(_ service: Service, bundleID: String) -> Bool { + let proc = Process() + proc.executableURL = URL(fileURLWithPath: "/usr/bin/tccutil") + proc.arguments = ["reset", service.rawValue, bundleID] + do { + try proc.run() + proc.waitUntilExit() + let ok = proc.terminationStatus == 0 + if !ok { + log.warning( + "tccutil reset \(service.rawValue, privacy: .public) exited \(proc.terminationStatus)") + } + return ok + } catch { + let reason = error.localizedDescription + log.error( + "tccutil reset \(service.rawValue, privacy: .public) failed to launch: \(reason, privacy: .public)") + return false } - return ok - } catch { - let reason = error.localizedDescription - log.error( - "tccutil reset \(service.rawValue, privacy: .public) failed to launch: \(reason, privacy: .public)") - return false } - } - /// The services a full reset sweeps. Spelled out case by case rather than - /// taken from `allCases` because Periphery can't follow a case that is only - /// ever reached through `allCases` and reports it as dead; - /// `PermissionsResetTests` pins this list against `allCases`, so a service - /// added to the enum and forgotten here fails there rather than silently - /// surviving every "full" reset. - static let sweep: [Service] = [.accessibility, .microphone, .inputMonitoring] + /// The services a full reset sweeps. Spelled out case by case rather than + /// taken from `allCases` because Periphery can't follow a case that is only + /// ever reached through `allCases` and reports it as dead; + /// `PermissionsResetTests` pins this list against `allCases`, so a service + /// added to the enum and forgotten here fails there rather than silently + /// surviving every "full" reset. + static let sweep: [Service] = [.accessibility, .microphone, .inputMonitoring] - /// Clears every service in `sweep` for `bundleID`, returning true only when - /// all of them succeeded. Every service is attempted even after one fails: a - /// half-reset install is the state this exists to get *out* of, so stopping at - /// the first failure would leave more behind than reporting it does. - @discardableResult - static func resetAll(bundleID: String) -> Bool { - var allCleared = true - for service in sweep where !reset(service, bundleID: bundleID) { - allCleared = false + /// Clears every service in `sweep` for `bundleID`, returning true only when + /// all of them succeeded. Every service is attempted even after one fails: a + /// half-reset install is the state this exists to get *out* of, so stopping at + /// the first failure would leave more behind than reporting it does. + @discardableResult + static func resetAll(bundleID: String) -> Bool { + var allCleared = true + for service in sweep where !reset(service, bundleID: bundleID) { + allCleared = false + } + return allCleared } - return allCleared } -} +#endif diff --git a/Sources/BlurtEngine/Permissions/SigningIdentity.swift b/Sources/BlurtEngine/Permissions/SigningIdentity.swift index c12e0bf2..67424cf8 100644 --- a/Sources/BlurtEngine/Permissions/SigningIdentity.swift +++ b/Sources/BlurtEngine/Permissions/SigningIdentity.swift @@ -1,94 +1,96 @@ -import Security +#if os(macOS) + import Security -/// The integration adapter for the signing-identity migration: reads the -/// identity this process's signature pins into its designated requirement. Kept -/// separate from the pure `SigningIdentityMigration` so the decision logic stays -/// testable and this `Security` handshake stays thin. Clearing the stale grant -/// the migration decides on is `PermissionsReset`'s half. -public enum SigningIdentity { - /// Namespace marker on a recorded identity. Every build before this one recorded - /// a bare Team ID (10 alphanumerics, no colon), so the prefix keeps the two - /// shapes disjoint — a marker left by an older build can never accidentally - /// compare equal to a requirement — and makes a dumped `defaults read` value - /// self-describing. - static let requirementPrefix = "dr:" + /// The integration adapter for the signing-identity migration: reads the + /// identity this process's signature pins into its designated requirement. Kept + /// separate from the pure `SigningIdentityMigration` so the decision logic stays + /// testable and this `Security` handshake stays thin. Clearing the stale grant + /// the migration decides on is `PermissionsReset`'s half. + public enum SigningIdentity { + /// Namespace marker on a recorded identity. Every build before this one recorded + /// a bare Team ID (10 alphanumerics, no colon), so the prefix keeps the two + /// shapes disjoint — a marker left by an older build can never accidentally + /// compare equal to a requirement — and makes a dumped `defaults read` value + /// self-describing. + static let requirementPrefix = "dr:" - /// Whatever an Accessibility grant taken *right now* would be pinned to: this - /// binary's **designated requirement**, serialized. - /// - /// The DR is what `tccd` stores alongside a grant and re-checks the running - /// binary against, so it is the identity — not a proxy for it. That distinction - /// is the whole bug this used to have: it recorded the *Team ID*, and the two - /// ways Blurt is team-signed carry the same team while pinning different - /// requirements. - /// - /// - **Dev builds** (`Apple Development`, re-signed by the `project.yml` - /// post-build install) pin an explicit team-based requirement: - /// `identifier … and anchor apple generic and certificate leaf[subject.OU] = `. - /// Cert rotation inside the team leaves that string byte-identical, which is - /// why the explicit requirement is stamped in the first place. - /// - **Releases** (`Developer ID`, `scripts/release-build.sh`) carry codesign's - /// *default* requirement, which additionally pins the leaf cert's Common Name - /// and the Developer ID marker OIDs — so re-issuing that certificate moves the - /// requirement and orphans every installed user's grant. - /// - /// `nil` for **unsigned or ad-hoc code**, which the migration reads as "no - /// action". Ad-hoc signatures pin a cdhash, so honouring them would mean every - /// `uitest.sh` / `check.sh` run — each one a throwaway ad-hoc binary under the - /// debug bundle id — resetting the developer's own Blurt Dev grant. Refusing - /// them here makes that structural rather than a flag the call site has to - /// remember to pass. The cost is a contributor who builds ad-hoc *and* copies - /// the result to /Applications by hand: they re-grant per rebuild, and - /// `tccutil reset Accessibility dev.alex.blurt.dev` is the way out. - public static func current() -> String? { - guard let code = staticCodeForSelf() else { return nil } - // A team identifier is present exactly when the signature isn't ad-hoc, so it - // is the probe for that carve-out — never the recorded value. - guard teamIdentifier(of: code) != nil else { return nil } - guard let requirement = designatedRequirement(of: code) else { return nil } - return requirementPrefix + requirement - } + /// Whatever an Accessibility grant taken *right now* would be pinned to: this + /// binary's **designated requirement**, serialized. + /// + /// The DR is what `tccd` stores alongside a grant and re-checks the running + /// binary against, so it is the identity — not a proxy for it. That distinction + /// is the whole bug this used to have: it recorded the *Team ID*, and the two + /// ways Blurt is team-signed carry the same team while pinning different + /// requirements. + /// + /// - **Dev builds** (`Apple Development`, re-signed by the `project.yml` + /// post-build install) pin an explicit team-based requirement: + /// `identifier … and anchor apple generic and certificate leaf[subject.OU] = `. + /// Cert rotation inside the team leaves that string byte-identical, which is + /// why the explicit requirement is stamped in the first place. + /// - **Releases** (`Developer ID`, `scripts/release-build.sh`) carry codesign's + /// *default* requirement, which additionally pins the leaf cert's Common Name + /// and the Developer ID marker OIDs — so re-issuing that certificate moves the + /// requirement and orphans every installed user's grant. + /// + /// `nil` for **unsigned or ad-hoc code**, which the migration reads as "no + /// action". Ad-hoc signatures pin a cdhash, so honouring them would mean every + /// `uitest.sh` / `check.sh` run — each one a throwaway ad-hoc binary under the + /// debug bundle id — resetting the developer's own Blurt Dev grant. Refusing + /// them here makes that structural rather than a flag the call site has to + /// remember to pass. The cost is a contributor who builds ad-hoc *and* copies + /// the result to /Applications by hand: they re-grant per rebuild, and + /// `tccutil reset Accessibility dev.alex.blurt.dev` is the way out. + public static func current() -> String? { + guard let code = staticCodeForSelf() else { return nil } + // A team identifier is present exactly when the signature isn't ad-hoc, so it + // is the probe for that carve-out — never the recorded value. + guard teamIdentifier(of: code) != nil else { return nil } + guard let requirement = designatedRequirement(of: code) else { return nil } + return requirementPrefix + requirement + } - /// The on-disk code object backing this process, or `nil` when any step of the - /// `Security` handshake fails. - static func staticCodeForSelf() -> SecStaticCode? { - var code: SecCode? - guard SecCodeCopySelf(SecCSFlags(), &code) == errSecSuccess, let code else { return nil } - var staticCode: SecStaticCode? - guard SecCodeCopyStaticCode(code, SecCSFlags(), &staticCode) == errSecSuccess, let staticCode - else { return nil } - return staticCode - } + /// The on-disk code object backing this process, or `nil` when any step of the + /// `Security` handshake fails. + static func staticCodeForSelf() -> SecStaticCode? { + var code: SecCode? + guard SecCodeCopySelf(SecCSFlags(), &code) == errSecSuccess, let code else { return nil } + var staticCode: SecStaticCode? + guard SecCodeCopyStaticCode(code, SecCSFlags(), &staticCode) == errSecSuccess, let staticCode + else { return nil } + return staticCode + } - /// The signature's team identifier — `nil` for unsigned or ad-hoc code. - private static func teamIdentifier(of code: SecStaticCode) -> String? { - var info: CFDictionary? - let flags = SecCSFlags(rawValue: kSecCSSigningInformation) - guard SecCodeCopySigningInformation(code, flags, &info) == errSecSuccess, - let dict = info as? [String: Any] - else { return nil } - return dict[kSecCodeInfoTeamIdentifier as String] as? String - } + /// The signature's team identifier — `nil` for unsigned or ad-hoc code. + private static func teamIdentifier(of code: SecStaticCode) -> String? { + var info: CFDictionary? + let flags = SecCSFlags(rawValue: kSecCSSigningInformation) + guard SecCodeCopySigningInformation(code, flags, &info) == errSecSuccess, + let dict = info as? [String: Any] + else { return nil } + return dict[kSecCodeInfoTeamIdentifier as String] as? String + } - /// The designated requirement, serialized to the same text `codesign -d -r-` - /// prints. Read through `SecCodeCopyDesignatedRequirement` rather than the - /// `kSecCodeInfoDesignatedRequirement` key of the signing-information - /// dictionary: that dictionary is `[String: Any]`, and narrowing an `Any` back - /// to a CoreFoundation type needs a cast the compiler flags as always-succeeding - /// (or a banned force cast). This spelling is typed end to end. - /// - /// Takes the code object rather than reading `self` so the tests can point it at - /// a binary whose requirement is known — the whole `Security` handshake is the - /// one part of this file that isn't pure logic, and the test host's own - /// signature varies by how the suite was launched. - static func designatedRequirement(of code: SecStaticCode) -> String? { - var requirement: SecRequirement? - guard SecCodeCopyDesignatedRequirement(code, SecCSFlags(), &requirement) == errSecSuccess, - let requirement - else { return nil } - var text: CFString? - guard SecRequirementCopyString(requirement, SecCSFlags(), &text) == errSecSuccess, let text - else { return nil } - return text as String + /// The designated requirement, serialized to the same text `codesign -d -r-` + /// prints. Read through `SecCodeCopyDesignatedRequirement` rather than the + /// `kSecCodeInfoDesignatedRequirement` key of the signing-information + /// dictionary: that dictionary is `[String: Any]`, and narrowing an `Any` back + /// to a CoreFoundation type needs a cast the compiler flags as always-succeeding + /// (or a banned force cast). This spelling is typed end to end. + /// + /// Takes the code object rather than reading `self` so the tests can point it at + /// a binary whose requirement is known — the whole `Security` handshake is the + /// one part of this file that isn't pure logic, and the test host's own + /// signature varies by how the suite was launched. + static func designatedRequirement(of code: SecStaticCode) -> String? { + var requirement: SecRequirement? + guard SecCodeCopyDesignatedRequirement(code, SecCSFlags(), &requirement) == errSecSuccess, + let requirement + else { return nil } + var text: CFString? + guard SecRequirementCopyString(requirement, SecCSFlags(), &text) == errSecSuccess, let text + else { return nil } + return text as String + } } -} +#endif diff --git a/Sources/BlurtEngine/Pipeline/DictationLog.swift b/Sources/BlurtEngine/Pipeline/DictationLog.swift index 5f6041ac..4b4a51d9 100644 --- a/Sources/BlurtEngine/Pipeline/DictationLog.swift +++ b/Sources/BlurtEngine/Pipeline/DictationLog.swift @@ -31,7 +31,7 @@ public enum DictationLog { /// chunk is *trimmed* and joined to the history; /// this records it raw. The difference is load-bearing twice over. Its trailing /// whitespace is the entire input to the paste's leading-separator decision - /// (`KeyInjector.withLeadingSeparator` branches on `prior.last.isWhitespace`), + /// (`InsertionSeparator.withLeadingSeparator` branches on `prior.last.isWhitespace`), /// so a spacing bug is undiagnosable from a trimmed copy. And a nil here /// distinguishes "no prior text was read" from "the prompt ends with the /// newest recent dictation", which `sttPrompt` alone cannot say. diff --git a/Sources/BlurtEngine/Pipeline/DictationSession+Press.swift b/Sources/BlurtEngine/Pipeline/DictationSession+Press.swift index b5226bc2..eec635c1 100644 --- a/Sources/BlurtEngine/Pipeline/DictationSession+Press.swift +++ b/Sources/BlurtEngine/Pipeline/DictationSession+Press.swift @@ -124,7 +124,16 @@ extension DictationSession { if !consumeCancelRequest() { setPhase(.cancelled) } return } - setPhase(.failed(.audioCaptureFailed(underlying: error))) + // `MicCapture.start()` already throws a `BlurtError` for its own failures + // (no input device, the liveness gate failing closed); wrapping that a + // second time read as "Audio capture failed: Audio capture failed: …" on + // the pill and in `errors.jsonl`. Only a conformer's untyped error needs + // the wrap — the same rule `awaitUpload` applies to the transcriber's. + if let blurtError = error as? BlurtError { + setPhase(.failed(blurtError)) + } else { + setPhase(.failed(.audioCaptureFailed(underlying: error))) + } } } @@ -142,7 +151,7 @@ extension DictationSession { // above) so the call is a Sendable closure rather than isolated state. let captureFrontmost = seams.captureFrontmost let captured = await captureFrontmost() - await injector.setTargetApp(captured.flatMap { FocusCapture.runningApp(for: $0) }) + await injector.setTarget(captured) // Key terms are read synchronously at press (cheap UserDefaults read), so // each dictation observably re-reads Settings edits at press time. let keyTerms = keyTermsProvider() diff --git a/Sources/BlurtEngine/Pipeline/DictationSession+Seams.swift b/Sources/BlurtEngine/Pipeline/DictationSession+Seams.swift index 460eb9a0..765aabc3 100644 --- a/Sources/BlurtEngine/Pipeline/DictationSession+Seams.swift +++ b/Sources/BlurtEngine/Pipeline/DictationSession+Seams.swift @@ -10,11 +10,13 @@ extension DictationSession { /// `~/Library/Logs/Blurt`, so on a machine with developer mode switched on /// `swift test` appended its fixtures to the user's own dictation corpus. /// - /// Bundled into one value rather than four initializer parameters: the - /// public initializer can't name these types (`CapturedFocus` and - /// `FocusedFieldContext` are internal), so the seams ride on the internal - /// initializer, and one required parameter there keeps the two initializers - /// unambiguous at every call site. + /// Bundled into one value rather than four initializer parameters: the log + /// seams have no business on the public initializer, so the whole set rides + /// on the internal one, and one required parameter there keeps the two + /// initializers unambiguous at every call site. The focus half *is* public — + /// as `HostFocusCapture`, below — because a host on another platform has to + /// supply it: the production defaults here are Accessibility reads, and only + /// macOS has those. /// /// Spelled with `var` properties: the synthesized memberwise initializer omits /// `let` properties that already carry a default value, so a test could not @@ -22,17 +24,29 @@ extension DictationSession { /// session stores one and never writes to it. struct Seams: Sendable { /// Captures the frontmost application — the paste target, and the context's - /// app name. On the main actor because it's an AppKit read. + /// app name. On the main actor because it's an AppKit read. Off macOS there + /// is no frontmost app to read (a keyboard cannot see its host), so the + /// default reports nothing and a host with real context supplies its own. var captureFrontmost: @Sendable () async -> CapturedFocus? = { - await MainActor.run { FocusCapture.captureFrontmost() } + #if os(macOS) + return await MainActor.run { FocusCapture.captureFrontmost() } + #else + return nil + #endif } /// Reads the focused field's Accessibility context. Deliberately /// synchronous: `performPress` runs it on a Dispatch queue precisely because /// it blocks a thread against an unresponsive app (see its call site), and - /// that stays true of the production capture behind this seam. - var captureFieldContext: @Sendable () -> FocusCapture.FocusedFieldContext = { - FocusCapture.captureFieldContext() + /// that stays true of the production capture behind this seam. Off macOS + /// the default reads nothing; an iPhone host hands the keyboard's text + /// before the cursor in through `HostFocusCapture` instead. + var captureFieldContext: @Sendable () -> FocusedFieldContext = { + #if os(macOS) + return FocusCapture.captureFieldContext() + #else + return .empty + #endif } /// Records a completed dictation in the developer-mode log. Gated on the @@ -49,7 +63,42 @@ extension DictationSession { } /// The real focus capture and the real log — what the public initializer - /// passes, and the only value production ever uses. + /// passes when the host supplies no `HostFocusCapture`, and the only value + /// production on macOS ever uses. static let production = Seams() } + + // periphery:ignore - public for hosts outside this repo (the iOS app); the Mac app relies on the defaults. + /// What a host that is not this repo's Mac app knows about the field being + /// dictated into, supplied to the public initializer in place of the + /// Accessibility reads the engine performs on macOS. + /// + /// `field` is read once per press, off the actor, and consumed when the + /// upload opens (bounded by `contextWaitBudget`) — the same contract the + /// Accessibility capture has, so a host's read may block briefly but must + /// never hang. An iPhone keyboard's `documentContextBeforeInput` is the + /// `priorText` the request's `stt_prompt` continues from; everything a + /// keyboard cannot see stays nil. `frontmost` is the paste target handed to + /// `InjectorProtocol.setTarget`; a host with no such notion returns nil. + public struct HostFocusCapture: Sendable { + public var frontmost: @Sendable () async -> CapturedFocus? + public var field: @Sendable () -> FocusedFieldContext + + public init( + frontmost: @escaping @Sendable () async -> CapturedFocus?, + field: @escaping @Sendable () -> FocusedFieldContext + ) { + self.frontmost = frontmost + self.field = field + } + } +} + +extension DictationSession.Seams { + /// The production seams with the focus half replaced by the host's. In an + /// extension so the struct keeps its memberwise initializer, which the tests + /// build their seams with. + init(hostFocusCapture host: DictationSession.HostFocusCapture) { + self.init(captureFrontmost: host.frontmost, captureFieldContext: host.field) + } } diff --git a/Sources/BlurtEngine/Pipeline/DictationSession.swift b/Sources/BlurtEngine/Pipeline/DictationSession.swift index 0133e646..11cadcb2 100644 --- a/Sources/BlurtEngine/Pipeline/DictationSession.swift +++ b/Sources/BlurtEngine/Pipeline/DictationSession.swift @@ -162,15 +162,13 @@ public actor DictationSession { /// `pipelineTask` alone would abandon only the *wait* — the request itself /// would keep streaming and complete against a dictation the user dismissed. /// - /// A bare task, because that is now all a live request is. It was an - /// `InFlightUpload` pairing the task with the context channel its `config` - /// part parked on, so abandoning it meant closing that too; the streaming - /// route settles the context up front, so `cancel()` is now the whole of it. + /// A bare task: the streaming route settles the context up front, so `cancel()` is the whole of it. var upload: Task? - /// The production entry point: the real focus capture and the real - /// developer-mode log. Delegates to the seam-carrying initializer below, which - /// can't be public because it names internal types. + /// The production entry point: the real log, and the real focus capture + /// unless the host supplies its own (`hostFocusCapture`: the iOS app, whose + /// keyboard knows the text before the cursor). Delegates to the seam-carrying + /// initializer below, which can't be public (it names internal seams). public init( mic: MicCaptureProtocol, transcriber: TranscriberProtocol, @@ -181,14 +179,16 @@ public actor DictationSession { styleNameProvider: (@Sendable () -> String?)? = nil, textShortcutsProvider: (@Sendable () -> [TextShortcut])? = nil, readinessCheck: @escaping @Sendable () -> BlurtError? = { nil }, - onTranscriptDelivered: (@Sendable (String, RecentDictations) -> Void)? = nil + onTranscriptDelivered: (@Sendable (String, RecentDictations) -> Void)? = nil, + hostFocusCapture: HostFocusCapture? = nil ) { self.init( mic: mic, transcriber: transcriber, injector: injector, maxRecordingSeconds: maxRecordingSeconds, clock: clock, keyTermsProvider: keyTermsProvider, styleNameProvider: styleNameProvider, textShortcutsProvider: textShortcutsProvider, readinessCheck: readinessCheck, - onTranscriptDelivered: onTranscriptDelivered, seams: .production) + onTranscriptDelivered: onTranscriptDelivered, + seams: hostFocusCapture.map(Seams.init(hostFocusCapture:)) ?? .production) } /// `seams` is deliberately required rather than defaulted: it's what keeps this diff --git a/Sources/BlurtEngine/Pipeline/MeterBarGeometry.swift b/Sources/BlurtEngine/Pipeline/MeterBarGeometry.swift index 93a02e16..a7524495 100644 --- a/Sources/BlurtEngine/Pipeline/MeterBarGeometry.swift +++ b/Sources/BlurtEngine/Pipeline/MeterBarGeometry.swift @@ -122,9 +122,19 @@ public struct MeterBarRow: Sendable, Equatable { /// Resolves the row for the frame the meter has been given. public init(availableSize: CGSize) { - let count = MeterBarGeometry.barCount(availableWidth: availableSize.width) + self.init( + count: MeterBarGeometry.barCount(availableWidth: availableSize.width), + availableHeight: availableSize.height) + } + + // periphery:ignore - public for the iOS keyboard (App/BlurtiOS), which the Mac scan doesn't index. + /// Resolves the row for a caller that lays out its own pitch — the iPhone + /// keyboard's thin wave draws 2 pt bars 2 pt apart — with `count` bars in a + /// frame `availableHeight` tall. Never fewer than `minBarCount` bars. + public init(count: Int, availableHeight: CGFloat) { + let count = max(MeterBarGeometry.minBarCount, count) self.count = count - self.maxBarHeight = MeterBarGeometry.maxBarHeight(availableHeight: availableSize.height) + self.maxBarHeight = MeterBarGeometry.maxBarHeight(availableHeight: availableHeight) self.weights = (0.. Bool - /// How long to wait for the app to answer the ⌘C. Measured copies land in - /// tens of milliseconds, and nothing landing means nothing was selected. - private let timeout: Duration +#if os(macOS) + /// The read-aloud press's fallback for apps whose selection Accessibility can't + /// see. It sends ⌘C, reads what landed on the clipboard, and puts the user's + /// clipboard back. + /// + /// Measured against Claude.app (Electron) on 2026-09-30: the system-wide focus + /// query fails with `cannotComplete`, the app-level one has no value, and the web + /// area exposes no selected text, even with `AXManualAccessibility` or + /// `AXEnhancedUserInterface` set. ⌘C copied the highlighted reply text on the + /// first try. + /// + /// Runs only when the AX read was `.unreadable`, never when AX saw a bare caret. + /// Some editors copy the whole current line on a ⌘C with no selection, and a + /// fallback on every empty selection would turn an ordinary dictation press into + /// reading that line aloud. An app that is unreadable to AX *and* copies lines + /// (VS Code with accessibility off) still has that problem. It's accepted while + /// the feature is experimental. + struct SelectionCopy: Sendable { + private let pasteboard: any CopyPasteboard + private let postCopy: @Sendable () -> Bool + /// How long to wait for the app to answer the ⌘C. Measured copies land in + /// tens of milliseconds, and nothing landing means nothing was selected. + private let timeout: Duration - init( - pasteboard: any CopyPasteboard = SystemClipboard(), - postCopy: @escaping @Sendable () -> Bool = { KeyInjector.postCmdC() }, - timeout: Duration = .milliseconds(250) - ) { - self.pasteboard = pasteboard - self.postCopy = postCopy - self.timeout = timeout - } + init( + pasteboard: any CopyPasteboard = SystemClipboard(), + postCopy: @escaping @Sendable () -> Bool = { KeyInjector.postCmdC() }, + timeout: Duration = .milliseconds(250) + ) { + self.pasteboard = pasteboard + self.postCopy = postCopy + self.timeout = timeout + } - /// The copied text, up to `maxCharacters`, or nil when the copy produced - /// nothing readable. The clipboard is restored whenever the copy changed it. - func copySelection(maxCharacters: Int) async -> String? { - // An unreadable pasteboard can't be restored, so don't disturb it. - guard let saved = pasteboard.snapshot() else { return nil } - let before = pasteboard.changeCount - guard postCopy() else { return nil } - let deadline = ContinuousClock.now + timeout - while pasteboard.changeCount == before, ContinuousClock.now < deadline { - try? await Task.sleep(for: .milliseconds(10)) + /// The copied text, up to `maxCharacters`, or nil when the copy produced + /// nothing readable. The clipboard is restored whenever the copy changed it. + func copySelection(maxCharacters: Int) async -> String? { + // An unreadable pasteboard can't be restored, so don't disturb it. + guard let saved = pasteboard.snapshot() else { return nil } + let before = pasteboard.changeCount + guard postCopy() else { return nil } + let deadline = ContinuousClock.now + timeout + while pasteboard.changeCount == before, ContinuousClock.now < deadline { + try? await Task.sleep(for: .milliseconds(10)) + } + let copiedCount = pasteboard.changeCount + guard copiedCount != before else { return nil } + let copied = pasteboard.currentString() + // Put the user's clipboard back, unless something wrote to it after the copy + // did. The same rule the paste path's restore follows. + if pasteboard.changeCount == copiedCount { pasteboard.restore(saved) } + return copied?.trimmedNonEmpty().map { String($0.prefix(maxCharacters)) } } - let copiedCount = pasteboard.changeCount - guard copiedCount != before else { return nil } - let copied = pasteboard.currentString() - // Put the user's clipboard back, unless something wrote to it after the copy - // did. The same rule the paste path's restore follows. - if pasteboard.changeCount == copiedCount { pasteboard.restore(saved) } - return copied?.trimmedNonEmpty().map { String($0.prefix(maxCharacters)) } } -} -/// The pasteboard operations `SelectionCopy` needs, so tests never touch the real -/// clipboard. `SystemClipboard` is the production conformance. -protocol CopyPasteboard: Sendable { - var changeCount: Int { get } - func snapshot() -> PasteboardSnapshot? - func currentString() -> String? - func restore(_ saved: PasteboardSnapshot) -} + /// The pasteboard operations `SelectionCopy` needs, so tests never touch the real + /// clipboard. `SystemClipboard` is the production conformance. + protocol CopyPasteboard: Sendable { + var changeCount: Int { get } + func snapshot() -> PasteboardSnapshot? + func currentString() -> String? + func restore(_ saved: PasteboardSnapshot) + } +#endif diff --git a/Sources/BlurtEngine/TTS/SelectionSpeaker.swift b/Sources/BlurtEngine/TTS/SelectionSpeaker.swift index 7add7165..e41c2a9c 100644 --- a/Sources/BlurtEngine/TTS/SelectionSpeaker.swift +++ b/Sources/BlurtEngine/TTS/SelectionSpeaker.swift @@ -1,72 +1,74 @@ -/// Reads text aloud with AssemblyAI's streaming TTS. This is the engine half of -/// the experimental read-selection-aloud press (`SelectionSpeechStore`); the app -/// decides when to call it. -/// -/// `speak` runs until the last sample has played. To stop early, cancel the task -/// that is awaiting it: cancellation silences playback at once and closes the -/// socket. -public struct SelectionSpeaker: Sendable { - /// The most text one press will read. Enough for several paragraphs, while - /// keeping a whole-document select-all from queueing minutes of audio (and - /// the TTS usage that comes with it) behind a single tap. - public static let maxCharacters = 4_000 +#if os(macOS) + /// Reads text aloud with AssemblyAI's streaming TTS. This is the engine half of + /// the experimental read-selection-aloud press (`SelectionSpeechStore`); the app + /// decides when to call it. + /// + /// `speak` runs until the last sample has played. To stop early, cancel the task + /// that is awaiting it: cancellation silences playback at once and closes the + /// socket. + public struct SelectionSpeaker: Sendable { + /// The most text one press will read. Enough for several paragraphs, while + /// keeping a whole-document select-all from queueing minutes of audio (and + /// the TTS usage that comes with it) behind a single tap. + public static let maxCharacters = 4_000 - private let synthesizer: AssemblyAISpeechSynthesizer - private let makeSink: @Sendable () -> any PCMSink + private let synthesizer: AssemblyAISpeechSynthesizer + private let makeSink: @Sendable () -> any PCMSink - public init() { - self.init(synthesizer: AssemblyAISpeechSynthesizer()) { - StreamingPCMPlayer(sampleRate: AssemblyAISpeechSynthesizer.sampleRate) + public init() { + self.init(synthesizer: AssemblyAISpeechSynthesizer()) { + StreamingPCMPlayer(sampleRate: AssemblyAISpeechSynthesizer.sampleRate) + } } - } - /// `makeSink` is the playback seam. There is one sink per `speak`, so a - /// cancelled read can't bleed into the next one. - init(synthesizer: AssemblyAISpeechSynthesizer, makeSink: @escaping @Sendable () -> any PCMSink) { - self.synthesizer = synthesizer - self.makeSink = makeSink - } + /// `makeSink` is the playback seam. There is one sink per `speak`, so a + /// cancelled read can't bleed into the next one. + init(synthesizer: AssemblyAISpeechSynthesizer, makeSink: @escaping @Sendable () -> any PCMSink) { + self.synthesizer = synthesizer + self.makeSink = makeSink + } - /// The selected text in the frontmost app, or nil when there is none, when - /// the field is secure, or when it can't be read. - /// - /// Accessibility first. The read is synchronous cross-process IPC that can - /// block a thread for the AX timeout against a hung app, so it runs on - /// `DictationSession.contextQueue`, the press-time capture's queue, and never - /// on the cooperative pool. When AX can't reach a focused element at all, it - /// falls back to copying the selection (`SelectionCopy`). - public static func focusedSelection() async -> String? { - let read = await withCheckedContinuation { continuation in - DictationSession.contextQueue.async { - continuation.resume(returning: FocusCapture.captureSelectedText(maxChars: maxCharacters)) + /// The selected text in the frontmost app, or nil when there is none, when + /// the field is secure, or when it can't be read. + /// + /// Accessibility first. The read is synchronous cross-process IPC that can + /// block a thread for the AX timeout against a hung app, so it runs on + /// `DictationSession.contextQueue`, the press-time capture's queue, and never + /// on the cooperative pool. When AX can't reach a focused element at all, it + /// falls back to copying the selection (`SelectionCopy`). + public static func focusedSelection() async -> String? { + let read = await withCheckedContinuation { continuation in + DictationSession.contextQueue.async { + continuation.resume(returning: FocusCapture.captureSelectedText(maxChars: maxCharacters)) + } } + return await resolve(read, copy: SelectionCopy()) } - return await resolve(read, copy: SelectionCopy()) - } - /// Split from `focusedSelection` so the fallback rule is testable without AX. - static func resolve(_ read: FocusCapture.SelectionRead, copy: SelectionCopy) async -> String? { - switch read { - case .text(let text): text - case .none: nil - case .unreadable: await copy.copySelection(maxCharacters: maxCharacters) + /// Split from `focusedSelection` so the fallback rule is testable without AX. + static func resolve(_ read: FocusCapture.SelectionRead, copy: SelectionCopy) async -> String? { + switch read { + case .text(let text): text + case .none: nil + case .unreadable: await copy.copySelection(maxCharacters: maxCharacters) + } } - } - public func speak(_ text: String) async throws { - let player = makeSink() - try await withTaskCancellationHandler { - do { - for try await chunk in synthesizer.synthesize(text) { - player.enqueue(chunk) + public func speak(_ text: String) async throws { + let player = makeSink() + try await withTaskCancellationHandler { + do { + for try await chunk in synthesizer.synthesize(text) { + player.enqueue(chunk) + } + } catch { + player.stop() + throw error } - } catch { + await player.drain() + } onCancel: { player.stop() - throw error } - await player.drain() - } onCancel: { - player.stop() } } -} +#endif diff --git a/Sources/BlurtEngine/UserAgent.swift b/Sources/BlurtEngine/UserAgent.swift index 1a39d58e..eeb78ee5 100644 --- a/Sources/BlurtEngine/UserAgent.swift +++ b/Sources/BlurtEngine/UserAgent.swift @@ -79,14 +79,21 @@ enum UserAgent { return "\(name) (\(comment.joined(separator: "; ")))" } - /// `macOS 15.5` — major and minor, for the reason the type doc gives. + /// `macOS 15.5` — major and minor, for the reason the type doc gives — or + /// `iOS 18.1` when the engine is running on a phone: the OS name is the one + /// fact about the platform the service-side baseline has to split on. /// /// `ProcessInfo` rather than AppKit's `NSProcessInfo`-adjacent niceties or a /// `sw_vers` shell-out, because the engine is a dependency-free Swift package - /// and this is the one Foundation API that answers it. + /// and this is the one Foundation API that answers it on both platforms. private static func osDetail() -> String { let version = ProcessInfo.processInfo.operatingSystemVersion - return "macOS \(version.majorVersion).\(version.minorVersion)" + #if os(macOS) + let name = "macOS" + #else + let name = "iOS" + #endif + return "\(name) \(version.majorVersion).\(version.minorVersion)" } /// `dev` for anything that isn't a release build, and nothing at all for a diff --git a/Tests/BlurtEngineTests/AXDowncastTests.swift b/Tests/BlurtEngineTests/AXDowncastTests.swift index ba50daca..07f80bf4 100644 --- a/Tests/BlurtEngineTests/AXDowncastTests.swift +++ b/Tests/BlurtEngineTests/AXDowncastTests.swift @@ -1,44 +1,46 @@ -import ApplicationServices -import Testing +#if os(macOS) + import ApplicationServices + import Testing -@testable import BlurtEngine + @testable import BlurtEngine -/// The checked CFTypeRef downcasts behind the Accessibility reads -/// (`FocusCapture.axElement` / `.axRange`). Attribute values arrive from -/// *other apps'* AX implementations, so a wrong CF type must decode to `nil`, -/// never flow onward mistyped. Pure type checks — no Accessibility trust, no -/// focused element, and no cross-process IPC needed. -@Suite("FocusCapture checked AX downcasts") -struct AXDowncastTests { - @Test("axElement passes a real AXUIElement through") - func elementAccepted() { - // The system-wide element is just a local ref — creating it needs no trust. - #expect(FocusCapture.axElement(AXUIElementCreateSystemWide()) != nil) - } + /// The checked CFTypeRef downcasts behind the Accessibility reads + /// (`FocusCapture.axElement` / `.axRange`). Attribute values arrive from + /// *other apps'* AX implementations, so a wrong CF type must decode to `nil`, + /// never flow onward mistyped. Pure type checks — no Accessibility trust, no + /// focused element, and no cross-process IPC needed. + @Suite("FocusCapture checked AX downcasts") + struct AXDowncastTests { + @Test("axElement passes a real AXUIElement through") + func elementAccepted() { + // The system-wide element is just a local ref — creating it needs no trust. + #expect(FocusCapture.axElement(AXUIElementCreateSystemWide()) != nil) + } - @Test("axElement rejects a non-element CF value") - func elementWrongTypeRejected() { - #expect(FocusCapture.axElement("not an element" as CFString) == nil) - } + @Test("axElement rejects a non-element CF value") + func elementWrongTypeRejected() { + #expect(FocusCapture.axElement("not an element" as CFString) == nil) + } - @Test("axRange decodes an AXValue-wrapped CFRange") - func rangeDecoded() throws { - var range = CFRange(location: 4, length: 2) - let wrapped = try #require(AXValueCreate(.cfRange, &range)) - let decoded = try #require(FocusCapture.axRange(wrapped)) - #expect(decoded.location == 4) - #expect(decoded.length == 2) - } + @Test("axRange decodes an AXValue-wrapped CFRange") + func rangeDecoded() throws { + var range = CFRange(location: 4, length: 2) + let wrapped = try #require(AXValueCreate(.cfRange, &range)) + let decoded = try #require(FocusCapture.axRange(wrapped)) + #expect(decoded.location == 4) + #expect(decoded.length == 2) + } - @Test("axRange rejects a non-AXValue CF value") - func rangeWrongTypeRejected() { - #expect(FocusCapture.axRange("not a range" as CFString) == nil) - } + @Test("axRange rejects a non-AXValue CF value") + func rangeWrongTypeRejected() { + #expect(FocusCapture.axRange("not a range" as CFString) == nil) + } - @Test("axRange rejects an AXValue holding a non-range payload") - func rangeWrongPayloadRejected() throws { - var point = CGPoint(x: 1, y: 2) - let wrapped = try #require(AXValueCreate(.cgPoint, &point)) - #expect(FocusCapture.axRange(wrapped) == nil) + @Test("axRange rejects an AXValue holding a non-range payload") + func rangeWrongPayloadRejected() throws { + var point = CGPoint(x: 1, y: 2) + let wrapped = try #require(AXValueCreate(.cgPoint, &point)) + #expect(FocusCapture.axRange(wrapped) == nil) + } } -} +#endif diff --git a/Tests/BlurtEngineTests/AudioInputDevicesTests.swift b/Tests/BlurtEngineTests/AudioInputDevicesTests.swift index e4940e3b..52b74bca 100644 --- a/Tests/BlurtEngineTests/AudioInputDevicesTests.swift +++ b/Tests/BlurtEngineTests/AudioInputDevicesTests.swift @@ -1,186 +1,188 @@ -@preconcurrency import AVFoundation -import CoreAudio -import Foundation -import Testing - -@testable import BlurtEngine - -/// Live-hardware checks for the microphone-selection plumbing: the device -/// enumeration the Settings picker lists, the UID lookups the capture path -/// resolves a pin with, and the session recorder itself. -/// -/// Gated on BLURT_LIVE_AUDIO_TESTS=1 like the other capture suites — every test -/// here talks to real devices (and two of them open one), which a headless CI -/// runner cannot answer for. Documents and locks the behavior for a human -/// running it on a Mac with a microphone; `AudioInputDevices.swift` and -/// `CaptureSessionRecorder.swift` are excluded from the coverage gate for the -/// same reason `MicCapture.swift` is. -@Suite( - "AudioInputDevices & capture session (live)", - ConditionTrait.requiresLiveAudio, - .tags(.liveAudio), - // Serialized: three of these open the default input, and the engagement test - // reads whether *anyone* has it open. Run in parallel they'd flag each other. - .serialized, - .timeLimit(.minutes(1))) -struct AudioInputDevicesTests { - @Test("enumeration lists named, uniquely-identified input devices") - func enumerationListsInputDevices() throws { - let devices = AudioInputDevices.all() - try #require(!devices.isEmpty, "a machine running this suite needs at least one input device") - - // Every entry must be renderable in the picker and pinnable by the store. - // Closures rather than key paths inside the macros — the rethrows + key-path - // combination is the documented `#expect` trap (see AGENTS.md's testing notes). - #expect(devices.allSatisfy { !$0.uid.isEmpty && !$0.name.isEmpty }) - let uids = devices.map { $0.uid } - #expect(Set(uids).count == uids.count, "device UIDs must be unique") - - // The default input has a name for the "Same as system (…)" label. - #expect(AudioInputDevices.systemDefaultInputName() != nil) - } - - @Test("a listed UID classifies; a bogus one reads as absent") - func uidLookupsAgreeWithTheEnumeration() throws { - let first = try #require(AudioInputDevices.all().first) +#if os(macOS) + @preconcurrency import AVFoundation + import CoreAudio + import Foundation + import Testing + + @testable import BlurtEngine + + /// Live-hardware checks for the microphone-selection plumbing: the device + /// enumeration the Settings picker lists, the UID lookups the capture path + /// resolves a pin with, and the session recorder itself. + /// + /// Gated on BLURT_LIVE_AUDIO_TESTS=1 like the other capture suites — every test + /// here talks to real devices (and two of them open one), which a headless CI + /// runner cannot answer for. Documents and locks the behavior for a human + /// running it on a Mac with a microphone; `AudioInputDevices.swift` and + /// `CaptureSessionRecorder.swift` are excluded from the coverage gate for the + /// same reason `MicCapture.swift` is. + @Suite( + "AudioInputDevices & capture session (live)", + ConditionTrait.requiresLiveAudio, + .tags(.liveAudio), + // Serialized: three of these open the default input, and the engagement test + // reads whether *anyone* has it open. Run in parallel they'd flag each other. + .serialized, + .timeLimit(.minutes(1))) + struct AudioInputDevicesTests { + @Test("enumeration lists named, uniquely-identified input devices") + func enumerationListsInputDevices() throws { + let devices = AudioInputDevices.all() + try #require(!devices.isEmpty, "a machine running this suite needs at least one input device") + + // Every entry must be renderable in the picker and pinnable by the store. + // Closures rather than key paths inside the macros — the rethrows + key-path + // combination is the documented `#expect` trap (see AGENTS.md's testing notes). + #expect(devices.allSatisfy { !$0.uid.isEmpty && !$0.name.isEmpty }) + let uids = devices.map { $0.uid } + #expect(Set(uids).count == uids.count, "device UIDs must be unique") + + // The default input has a name for the "Same as system (…)" label. + #expect(AudioInputDevices.systemDefaultInputName() != nil) + } - // One read answers both questions `MicCapture.resolveInput` asks of a pin — - // is the device there, and what transport is it on. That it holds for every - // enumerated device is the load-bearing part: the picker must not offer a - // device the capture path would then treat as missing (or fail to classify, - // which silently loses the Bluetooth caps). - #expect(AudioInputDevices.transportType(forUID: first.uid) != nil) + @Test("a listed UID classifies; a bogus one reads as absent") + func uidLookupsAgreeWithTheEnumeration() throws { + let first = try #require(AudioInputDevices.all().first) - // The missing-device signal `MicDeviceSelection.effective` falls back on. - #expect(AudioInputDevices.transportType(forUID: "blurt-test-no-such-device") == nil) - } + // One read answers both questions `MicCapture.resolveInput` asks of a pin — + // is the device there, and what transport is it on. That it holds for every + // enumerated device is the load-bearing part: the picker must not offer a + // device the capture path would then treat as missing (or fail to classify, + // which silently loses the Bluetooth caps). + #expect(AudioInputDevices.transportType(forUID: first.uid) != nil) - @Test("building a recorder — and warming up — leaves the microphone closed") - func buildingASessionDoesNotEngageTheDevice() async throws { - await LiveAudioDevice.acquire() - defer { LiveAudioDevice.release() } - - // The measurement the warm-up design rests on, as a regression test: a built - // but unstarted session must not open the device, or `warmUp()` at launch - // would light the input indicator and pin AirPods into their mic-capable - // profile (where *output* audio is degraded) for as long as the app runs. - let device = try #require(AVCaptureDevice.default(for: .audio)) - // Wait for the device to be free rather than demanding it already is: this - // suite is `.serialized`, but the other live suites are separate suites and - // run in parallel with it, and two of them open this same device. Polling - // makes the precondition deterministic instead of dependent on which suite - // got there first; a device that never frees up is a real failure with a - // message that says so. - try #require( - await Self.poll(upTo: .seconds(3)) { Self.isRunningSomewhere(uid: device.uniqueID) == false }, - "the default input stayed busy — another app (or suite) is capturing from it") - - let recorder = try await CaptureSessionRecorder.make(pinnedUID: device.uniqueID) - #expect(Self.isRunningSomewhere(uid: device.uniqueID) == false, "building must not open the device") - - await MicCapture(deviceSelection: { .systemDefault }).warmUp() - #expect(Self.isRunningSomewhere(uid: device.uniqueID) == false, "warm-up must not open the device") - - // And the same recorder still opens it on demand, so "closed" isn't just a - // recorder that was never usable. - // - // Polled rather than asserted outright, because when the open *completes* - // is transport-dependent — the measurement that shaped the liveness gate. - // On a USB interface `startRunning()` blocks until the device is running, so - // this holds the instant `record()` returns; on AirPods it returns in ~80 ms - // and the device comes up ~400 ms later. Asserting immediately passes on - // wired hardware and fails on the transport the gate exists for. - try #require(await recorder.record()) - #expect( - await Self.waitForRunning(uid: device.uniqueID), - "record() is what opens the device, even if the link finishes opening it later") - recorder.stopAndDiscard() - } + // The missing-device signal `MicDeviceSelection.effective` falls back on. + #expect(AudioInputDevices.transportType(forUID: "blurt-test-no-such-device") == nil) + } - @Test("the session recorder captures S16LE audio from the device it was pinned to") - func pinnedRecorderCapturesAudio() async throws { - await LiveAudioDevice.acquire() - defer { LiveAudioDevice.release() } - - // Pin to the current default input — the one device a machine running this - // suite is known to have working — by the same UID the picker would store. - let uid = try #require(AVCaptureDevice.default(for: .audio)?.uniqueID) - #expect(AudioInputDevices.all().contains { $0.uid == uid }, "the default input should be listed") - - let recorder = try await CaptureSessionRecorder.make(pinnedUID: uid) - try #require(await recorder.record(), "the session recorder should start on a live device") - - // Wait for frames the way the liveness gate does, rather than sleeping a - // fixed 500 ms: on AirPods the first buffer lands ~490 ms after - // `startRunning()` returns, so a fixed sleep sits right on the edge and this - // suite would flake on a cold link. - try #require( - await Self.poll(upTo: MicLiveness.bluetoothTimeout) { recorder.deliveredFrames > 0 }, - "buffers should be arriving") - // The meter must read as a live analog input, not digital silence — this is - // the reading `MicLiveness.silenceFloorDB` gates on, and the assertion that - // proves the session meter's floor semantics on real hardware. - // - // Polled for the same reason the frame wait is, and this is the sharper - // case: the channel's power lags the first frames, reporting - // `-Float.greatestFiniteMagnitude` until its first update (measured on - // AirPods). That is exactly why the gate polls *both* terms instead of - // metering once frames appear. - #expect( - await Self.poll(upTo: MicLiveness.bluetoothTimeout) { - recorder.meteredPowerDB() > MicLiveness.silenceFloorDB - }, "a live mic must out-read the silence floor") - - let byteCount = recorder.stopAndReadByteCount() - #expect(byteCount > 0, "half a second of capture should deliver samples") - #expect( - byteCount % SyncSTTLimits.bytesPerSample == 0, "the capture must be whole S16LE samples") - } + @Test("building a recorder — and warming up — leaves the microphone closed") + func buildingASessionDoesNotEngageTheDevice() async throws { + await LiveAudioDevice.acquire() + defer { LiveAudioDevice.release() } + + // The measurement the warm-up design rests on, as a regression test: a built + // but unstarted session must not open the device, or `warmUp()` at launch + // would light the input indicator and pin AirPods into their mic-capable + // profile (where *output* audio is degraded) for as long as the app runs. + let device = try #require(AVCaptureDevice.default(for: .audio)) + // Wait for the device to be free rather than demanding it already is: this + // suite is `.serialized`, but the other live suites are separate suites and + // run in parallel with it, and two of them open this same device. Polling + // makes the precondition deterministic instead of dependent on which suite + // got there first; a device that never frees up is a real failure with a + // message that says so. + try #require( + await Self.poll(upTo: .seconds(3)) { Self.isRunningSomewhere(uid: device.uniqueID) == false }, + "the default input stayed busy — another app (or suite) is capturing from it") + + let recorder = try await CaptureSessionRecorder.make(pinnedUID: device.uniqueID) + #expect(Self.isRunningSomewhere(uid: device.uniqueID) == false, "building must not open the device") + + await MicCapture(deviceSelection: { .systemDefault }).warmUp() + #expect(Self.isRunningSomewhere(uid: device.uniqueID) == false, "warm-up must not open the device") + + // And the same recorder still opens it on demand, so "closed" isn't just a + // recorder that was never usable. + // + // Polled rather than asserted outright, because when the open *completes* + // is transport-dependent — the measurement that shaped the liveness gate. + // On a USB interface `startRunning()` blocks until the device is running, so + // this holds the instant `record()` returns; on AirPods it returns in ~80 ms + // and the device comes up ~400 ms later. Asserting immediately passes on + // wired hardware and fails on the transport the gate exists for. + try #require(await recorder.record()) + #expect( + await Self.waitForRunning(uid: device.uniqueID), + "record() is what opens the device, even if the link finishes opening it later") + recorder.stopAndDiscard() + } - /// Polls `condition` on a short tick until it holds or `timeout` elapses, - /// answering whether it ever held. The suite's own miniature of - /// `MicLiveness.waitUntilLive`, for the same reason that exists: nothing about - /// a capture device is synchronous just because the call that started it - /// returned. - private static func poll( - upTo timeout: Duration, _ condition: @Sendable () -> Bool - ) async -> Bool { - let deadline = ContinuousClock().now.advanced(by: timeout) - while ContinuousClock().now < deadline { - if condition() { return true } - try? await Task.sleep(for: .milliseconds(20)) + @Test("the session recorder captures S16LE audio from the device it was pinned to") + func pinnedRecorderCapturesAudio() async throws { + await LiveAudioDevice.acquire() + defer { LiveAudioDevice.release() } + + // Pin to the current default input — the one device a machine running this + // suite is known to have working — by the same UID the picker would store. + let uid = try #require(AVCaptureDevice.default(for: .audio)?.uniqueID) + #expect(AudioInputDevices.all().contains { $0.uid == uid }, "the default input should be listed") + + let recorder = try await CaptureSessionRecorder.make(pinnedUID: uid) + try #require(await recorder.record(), "the session recorder should start on a live device") + + // Wait for frames the way the liveness gate does, rather than sleeping a + // fixed 500 ms: on AirPods the first buffer lands ~490 ms after + // `startRunning()` returns, so a fixed sleep sits right on the edge and this + // suite would flake on a cold link. + try #require( + await Self.poll(upTo: MicLiveness.bluetoothTimeout) { recorder.deliveredFrames > 0 }, + "buffers should be arriving") + // The meter must read as a live analog input, not digital silence — this is + // the reading `MicLiveness.silenceFloorDB` gates on, and the assertion that + // proves the session meter's floor semantics on real hardware. + // + // Polled for the same reason the frame wait is, and this is the sharper + // case: the channel's power lags the first frames, reporting + // `-Float.greatestFiniteMagnitude` until its first update (measured on + // AirPods). That is exactly why the gate polls *both* terms instead of + // metering once frames appear. + #expect( + await Self.poll(upTo: MicLiveness.bluetoothTimeout) { + recorder.meteredPowerDB() > MicLiveness.silenceFloorDB + }, "a live mic must out-read the silence floor") + + let byteCount = recorder.stopAndReadByteCount() + #expect(byteCount > 0, "half a second of capture should deliver samples") + #expect( + byteCount % SyncSTTLimits.bytesPerSample == 0, "the capture must be whole S16LE samples") } - return condition() - } - /// Whether the device carrying `uid` comes up as capturing within the - /// Bluetooth cap — the widest the product itself ever waits. - private static func waitForRunning(uid: String) async -> Bool { - await poll(upTo: MicLiveness.bluetoothTimeout) { isRunningSomewhere(uid: uid) } - } + /// Polls `condition` on a short tick until it holds or `timeout` elapses, + /// answering whether it ever held. The suite's own miniature of + /// `MicLiveness.waitUntilLive`, for the same reason that exists: nothing about + /// a capture device is synchronous just because the call that started it + /// returned. + private static func poll( + upTo timeout: Duration, _ condition: @Sendable () -> Bool + ) async -> Bool { + let deadline = ContinuousClock().now.advanced(by: timeout) + while ContinuousClock().now < deadline { + if condition() { return true } + try? await Task.sleep(for: .milliseconds(20)) + } + return condition() + } - /// CoreAudio's own answer to "is this device capturing right now" — the bit - /// behind the system input indicator. Test-local rather than a production - /// helper: nothing in the app needs to ask, and the point of the assertions - /// above is to check the *device*, not our bookkeeping about it. - private static func isRunningSomewhere(uid: String) -> Bool { - var translate = AudioRoute.globalAddress(kAudioHardwarePropertyTranslateUIDToDevice) - var cfUID = uid as CFString - var deviceID = AudioDeviceID(0) - var idSize = UInt32(MemoryLayout.size) - let translated = withUnsafeMutablePointer(to: &cfUID) { qualifier in - AudioObjectGetPropertyData( - AudioRoute.systemObject, &translate, UInt32(MemoryLayout.size), qualifier, - &idSize, &deviceID) + /// Whether the device carrying `uid` comes up as capturing within the + /// Bluetooth cap — the widest the product itself ever waits. + private static func waitForRunning(uid: String) async -> Bool { + await poll(upTo: MicLiveness.bluetoothTimeout) { isRunningSomewhere(uid: uid) } } - guard translated == noErr, deviceID != 0 else { return false } - var address = AudioRoute.globalAddress(kAudioDevicePropertyDeviceIsRunningSomewhere) - var running = UInt32(0) - var size = UInt32(MemoryLayout.size) - guard AudioObjectGetPropertyData(deviceID, &address, 0, nil, &size, &running) == noErr else { - return false + /// CoreAudio's own answer to "is this device capturing right now" — the bit + /// behind the system input indicator. Test-local rather than a production + /// helper: nothing in the app needs to ask, and the point of the assertions + /// above is to check the *device*, not our bookkeeping about it. + private static func isRunningSomewhere(uid: String) -> Bool { + var translate = AudioRoute.globalAddress(kAudioHardwarePropertyTranslateUIDToDevice) + var cfUID = uid as CFString + var deviceID = AudioDeviceID(0) + var idSize = UInt32(MemoryLayout.size) + let translated = withUnsafeMutablePointer(to: &cfUID) { qualifier in + AudioObjectGetPropertyData( + AudioRoute.systemObject, &translate, UInt32(MemoryLayout.size), qualifier, + &idSize, &deviceID) + } + guard translated == noErr, deviceID != 0 else { return false } + + var address = AudioRoute.globalAddress(kAudioDevicePropertyDeviceIsRunningSomewhere) + var running = UInt32(0) + var size = UInt32(MemoryLayout.size) + guard AudioObjectGetPropertyData(deviceID, &address, 0, nil, &size, &running) == noErr else { + return false + } + return running != 0 } - return running != 0 } -} +#endif diff --git a/Tests/BlurtEngineTests/BrowserBundleIDTests.swift b/Tests/BlurtEngineTests/BrowserBundleIDTests.swift index 183398da..0e59e4e7 100644 --- a/Tests/BlurtEngineTests/BrowserBundleIDTests.swift +++ b/Tests/BlurtEngineTests/BrowserBundleIDTests.swift @@ -1,164 +1,166 @@ -import AppKit -import Foundation -import Testing - -@testable import BlurtEngine - -/// Pins the browser classification behind the injector's AX-opaque exemption -/// (see `FocusCapture.isAXOpaqueApp`): a known browser pastes even when the -/// focused element exposes no editable AX signal, because web content is -/// routinely opaque (Chromium's lazy accessibility tree, `contenteditable` -/// composers like ChatGPT's) — "no signal" there means "AX can't see the -/// field," not "no field." -@Suite("FocusCapture.isBrowserBundleID") -struct BrowserBundleIDTests { - @Test( - "known browser bundle IDs classify as browsers", - arguments: [ - "com.apple.Safari", - "com.google.Chrome", - "org.chromium.Chromium", - "com.microsoft.edgemac", - "com.brave.Browser", - "com.operasoftware.Opera", - "com.vivaldi.Vivaldi", - "company.thebrowser.Browser", - "org.mozilla.firefox", - "com.duckduckgo.macos.browser", - "com.kagi.kagimacOS", - ]) - func knownBrowsers(bundleID: String) { - #expect(FocusCapture.isBrowserBundleID(bundleID)) +#if os(macOS) + import AppKit + import Foundation + import Testing + + @testable import BlurtEngine + + /// Pins the browser classification behind the injector's AX-opaque exemption + /// (see `FocusCapture.isAXOpaqueApp`): a known browser pastes even when the + /// focused element exposes no editable AX signal, because web content is + /// routinely opaque (Chromium's lazy accessibility tree, `contenteditable` + /// composers like ChatGPT's) — "no signal" there means "AX can't see the + /// field," not "no field." + @Suite("FocusCapture.isBrowserBundleID") + struct BrowserBundleIDTests { + @Test( + "known browser bundle IDs classify as browsers", + arguments: [ + "com.apple.Safari", + "com.google.Chrome", + "org.chromium.Chromium", + "com.microsoft.edgemac", + "com.brave.Browser", + "com.operasoftware.Opera", + "com.vivaldi.Vivaldi", + "company.thebrowser.Browser", + "org.mozilla.firefox", + "com.duckduckgo.macos.browser", + "com.kagi.kagimacOS", + ]) + func knownBrowsers(bundleID: String) { + #expect(FocusCapture.isBrowserBundleID(bundleID)) + } + + @Test( + "channel variants classify with their stable siblings (prefix match)", + arguments: [ + "com.apple.SafariTechnologyPreview", + "com.google.Chrome.beta", + "com.google.Chrome.canary", + "com.microsoft.edgemac.Dev", + "com.brave.Browser.nightly", + ]) + func channelVariants(bundleID: String) { + #expect(FocusCapture.isBrowserBundleID(bundleID)) + } + + @Test( + "non-browser apps are not browsers — they keep the copy-don't-beep fallback", + arguments: [ + "com.apple.finder", + "com.apple.TextEdit", + "com.apple.dt.Xcode", + "com.microsoft.VSCode", // Electron: exempted by isElectronApp, not here + "com.googlecode.iterm2", // "com.google" lookalike must not prefix-match + ]) + func nonBrowsers(bundleID: String) { + #expect(!FocusCapture.isBrowserBundleID(bundleID)) + } + + @Test("a nil bundle ID is not a browser") + func nilBundleID() { + #expect(!FocusCapture.isBrowserBundleID(nil)) + } } - @Test( - "channel variants classify with their stable siblings (prefix match)", - arguments: [ - "com.apple.SafariTechnologyPreview", - "com.google.Chrome.beta", - "com.google.Chrome.canary", - "com.microsoft.edgemac.Dev", - "com.brave.Browser.nightly", - ]) - func channelVariants(bundleID: String) { - #expect(FocusCapture.isBrowserBundleID(bundleID)) + /// The other half of the AX-opaque exemption: Electron detection, and the + /// `isAXOpaqueApp` disjunction the injector actually calls. + /// + /// Electron apps are classified by the framework they bundle rather than by + /// bundle ID, because the set is open-ended (every Electron app ever shipped), + /// so the fixtures here are directory trees rather than identifier strings — + /// `isElectronBundle` is split out of the `NSRunningApplication` wrapper for + /// exactly that reason. + @Suite("FocusCapture AX-opaque app classification") + struct AXOpaqueAppTests { + + /// An app bundle skeleton in a temp directory, with the Electron framework + /// present or absent. Only the *path* matters to the check — nothing is loaded — + /// so an empty directory at the framework's location is a faithful fixture. + private func makeBundle(withElectron: Bool) throws -> URL { + let bundle = URL.temporaryDirectory.appending(path: "Blurt-\(UUID().uuidString).app") + let contents = + withElectron + ? bundle.appending(path: "Contents/Frameworks/Electron Framework.framework") + : bundle.appending(path: "Contents/Frameworks") + try FileManager.default.createDirectory(at: contents, withIntermediateDirectories: true) + return bundle + } + + // MARK: isElectronBundle + + @Test("a bundle shipping the Electron framework is Electron") + func electronBundleDetected() throws { + let bundle = try makeBundle(withElectron: true) + defer { try? FileManager.default.removeItem(at: bundle) } + // The true arm is what keeps VS Code and Slack on the paste path: their focused + // text fields expose no editable AX signal, so without this they'd fall back to + // copy-only and the user's words would never land. + #expect(FocusCapture.isElectronBundle(bundle)) + } + + @Test("a native bundle with no Electron framework is not Electron") + func nativeBundleRejected() throws { + let bundle = try makeBundle(withElectron: false) + defer { try? FileManager.default.removeItem(at: bundle) } + // The false arm matters just as much: a native app with genuinely nothing + // editable focused must fall back to copy rather than beep a ⌘V. + #expect(!FocusCapture.isElectronBundle(bundle)) + } + + @Test("a rebranded Electron bundle is detected by its asar-integrity key") + func rebrandedElectronBundleDetected() throws { + let bundle = URL.temporaryDirectory.appending(path: "Blurt-\(UUID().uuidString).app") + defer { try? FileManager.default.removeItem(at: bundle) } + // Codex renames the framework to `Codex Framework.framework`, so only the + // Info.plist key identifies it as Electron. + let contents = bundle.appending(path: "Contents") + try FileManager.default.createDirectory( + at: contents.appending(path: "Frameworks/Codex Framework.framework"), + withIntermediateDirectories: true) + let plist: [String: Any] = [ + "CFBundleIdentifier": "com.example.rebranded", + "CFBundlePackageType": "APPL", + "ElectronAsarIntegrity": ["Resources/app.asar": ["algorithm": "SHA256"]], + ] + let data = try PropertyListSerialization.data(fromPropertyList: plist, format: .xml, options: 0) + try data.write(to: contents.appending(path: "Info.plist")) + #expect(FocusCapture.isElectronBundle(bundle)) + } + + @Test("a bundle URL that doesn't exist is not Electron") + func missingBundleRejected() { + #expect(!FocusCapture.isElectronBundle(URL(filePath: "/nonexistent/Ghost.app"))) + } + + @Test("a nil bundle URL is not Electron") + func nilBundleURLRejected() { + #expect(!FocusCapture.isElectronBundle(nil)) + } + + // MARK: NSRunningApplication wrappers + + @Test("the test host is neither a browser nor Electron") + func testHostIsNotOpaque() { + // The one live `NSRunningApplication` a unit test can count on. Weak as an + // assertion about *this* process, but it pins the wrappers as pass-throughs to + // the two pure checks rather than, say, defaulting to opaque — which would make + // the injector paste into every non-editable target and beep. + let current = NSRunningApplication.current + #expect(!FocusCapture.isBrowserApp(current)) + #expect(!FocusCapture.isElectronApp(current)) + #expect(!FocusCapture.isAXOpaqueApp(current)) + } + + @Test("no app at all is not AX-opaque") + func nilAppIsNotOpaque() { + // `KeyInjector` passes its captured target, which is nil when nothing was + // captured — that must not be treated as opaque, or a paste with no known + // target would be attempted anyway. + #expect(!FocusCapture.isBrowserApp(nil)) + #expect(!FocusCapture.isElectronApp(nil)) + #expect(!FocusCapture.isAXOpaqueApp(nil)) + } } - - @Test( - "non-browser apps are not browsers — they keep the copy-don't-beep fallback", - arguments: [ - "com.apple.finder", - "com.apple.TextEdit", - "com.apple.dt.Xcode", - "com.microsoft.VSCode", // Electron: exempted by isElectronApp, not here - "com.googlecode.iterm2", // "com.google" lookalike must not prefix-match - ]) - func nonBrowsers(bundleID: String) { - #expect(!FocusCapture.isBrowserBundleID(bundleID)) - } - - @Test("a nil bundle ID is not a browser") - func nilBundleID() { - #expect(!FocusCapture.isBrowserBundleID(nil)) - } -} - -/// The other half of the AX-opaque exemption: Electron detection, and the -/// `isAXOpaqueApp` disjunction the injector actually calls. -/// -/// Electron apps are classified by the framework they bundle rather than by -/// bundle ID, because the set is open-ended (every Electron app ever shipped), -/// so the fixtures here are directory trees rather than identifier strings — -/// `isElectronBundle` is split out of the `NSRunningApplication` wrapper for -/// exactly that reason. -@Suite("FocusCapture AX-opaque app classification") -struct AXOpaqueAppTests { - - /// An app bundle skeleton in a temp directory, with the Electron framework - /// present or absent. Only the *path* matters to the check — nothing is loaded — - /// so an empty directory at the framework's location is a faithful fixture. - private func makeBundle(withElectron: Bool) throws -> URL { - let bundle = URL.temporaryDirectory.appending(path: "Blurt-\(UUID().uuidString).app") - let contents = - withElectron - ? bundle.appending(path: "Contents/Frameworks/Electron Framework.framework") - : bundle.appending(path: "Contents/Frameworks") - try FileManager.default.createDirectory(at: contents, withIntermediateDirectories: true) - return bundle - } - - // MARK: isElectronBundle - - @Test("a bundle shipping the Electron framework is Electron") - func electronBundleDetected() throws { - let bundle = try makeBundle(withElectron: true) - defer { try? FileManager.default.removeItem(at: bundle) } - // The true arm is what keeps VS Code and Slack on the paste path: their focused - // text fields expose no editable AX signal, so without this they'd fall back to - // copy-only and the user's words would never land. - #expect(FocusCapture.isElectronBundle(bundle)) - } - - @Test("a native bundle with no Electron framework is not Electron") - func nativeBundleRejected() throws { - let bundle = try makeBundle(withElectron: false) - defer { try? FileManager.default.removeItem(at: bundle) } - // The false arm matters just as much: a native app with genuinely nothing - // editable focused must fall back to copy rather than beep a ⌘V. - #expect(!FocusCapture.isElectronBundle(bundle)) - } - - @Test("a rebranded Electron bundle is detected by its asar-integrity key") - func rebrandedElectronBundleDetected() throws { - let bundle = URL.temporaryDirectory.appending(path: "Blurt-\(UUID().uuidString).app") - defer { try? FileManager.default.removeItem(at: bundle) } - // Codex renames the framework to `Codex Framework.framework`, so only the - // Info.plist key identifies it as Electron. - let contents = bundle.appending(path: "Contents") - try FileManager.default.createDirectory( - at: contents.appending(path: "Frameworks/Codex Framework.framework"), - withIntermediateDirectories: true) - let plist: [String: Any] = [ - "CFBundleIdentifier": "com.example.rebranded", - "CFBundlePackageType": "APPL", - "ElectronAsarIntegrity": ["Resources/app.asar": ["algorithm": "SHA256"]], - ] - let data = try PropertyListSerialization.data(fromPropertyList: plist, format: .xml, options: 0) - try data.write(to: contents.appending(path: "Info.plist")) - #expect(FocusCapture.isElectronBundle(bundle)) - } - - @Test("a bundle URL that doesn't exist is not Electron") - func missingBundleRejected() { - #expect(!FocusCapture.isElectronBundle(URL(filePath: "/nonexistent/Ghost.app"))) - } - - @Test("a nil bundle URL is not Electron") - func nilBundleURLRejected() { - #expect(!FocusCapture.isElectronBundle(nil)) - } - - // MARK: NSRunningApplication wrappers - - @Test("the test host is neither a browser nor Electron") - func testHostIsNotOpaque() { - // The one live `NSRunningApplication` a unit test can count on. Weak as an - // assertion about *this* process, but it pins the wrappers as pass-throughs to - // the two pure checks rather than, say, defaulting to opaque — which would make - // the injector paste into every non-editable target and beep. - let current = NSRunningApplication.current - #expect(!FocusCapture.isBrowserApp(current)) - #expect(!FocusCapture.isElectronApp(current)) - #expect(!FocusCapture.isAXOpaqueApp(current)) - } - - @Test("no app at all is not AX-opaque") - func nilAppIsNotOpaque() { - // `KeyInjector` passes its captured target, which is nil when nothing was - // captured — that must not be treated as opaque, or a paste with no known - // target would be attempted anyway. - #expect(!FocusCapture.isBrowserApp(nil)) - #expect(!FocusCapture.isElectronApp(nil)) - #expect(!FocusCapture.isAXOpaqueApp(nil)) - } -} +#endif diff --git a/Tests/BlurtEngineTests/CancelRaceTests.swift b/Tests/BlurtEngineTests/CancelRaceTests.swift index 67fcc3de..5e4d9a2b 100644 --- a/Tests/BlurtEngineTests/CancelRaceTests.swift +++ b/Tests/BlurtEngineTests/CancelRaceTests.swift @@ -1,4 +1,3 @@ -import AppKit import Foundation import Testing @@ -325,7 +324,7 @@ private actor GatedInjector: InjectorProtocol { self.onRecord = onRecord } - func setTargetApp(_ app: NSRunningApplication?) async {} + func setTarget(_ focus: CapturedFocus?) async {} func insert(_ text: String, after priorText: String?, windowTitle: String?) async throws { await gate.enter() diff --git a/Tests/BlurtEngineTests/ChunkedUploadTests.swift b/Tests/BlurtEngineTests/ChunkedUploadTests.swift index 77a456c4..fc5a0716 100644 --- a/Tests/BlurtEngineTests/ChunkedUploadTests.swift +++ b/Tests/BlurtEngineTests/ChunkedUploadTests.swift @@ -140,7 +140,7 @@ struct ChunkedUploadTests { let session = makeSession( transcriber: FailsBeforeContext(), seams: testSeams( - field: FocusCapture.FocusedFieldContext( + field: FocusedFieldContext( priorText: "Dear Sam,", selectedText: nil, windowTitle: nil, fieldLabel: nil), log: log)) diff --git a/Tests/BlurtEngineTests/DictationLogTests.swift b/Tests/BlurtEngineTests/DictationLogTests.swift index b86e24ec..bc2a769b 100644 --- a/Tests/BlurtEngineTests/DictationLogTests.swift +++ b/Tests/BlurtEngineTests/DictationLogTests.swift @@ -114,7 +114,7 @@ struct DictationLogEntryTests { func priorIsRawWhereTheTurnIsTrimmed() { // Why `prior` isn't redundant with the prompt's tail. The trailing space is the // whole input to the paste's leading-separator decision - // (`KeyInjector.withLeadingSeparator` branches on `prior.last.isWhitespace`), + // (`InsertionSeparator.withLeadingSeparator` branches on `prior.last.isWhitespace`), // and the wire copy has it trimmed off — so dropping `prior` for being a // duplicate would take the only record of that bit with it. let entry = DictationLog.makeEntry( @@ -122,8 +122,8 @@ struct DictationLogEntryTests { now: Date()) #expect(entry.prior == "Hi Sam, ") #expect(entry.sttPrompt == "Hi Sam,") - #expect(KeyInjector.withLeadingSeparator("p", after: entry.prior) == "p") - #expect(KeyInjector.withLeadingSeparator("p", after: entry.sttPrompt) == " p") + #expect(InsertionSeparator.withLeadingSeparator("p", after: entry.prior) == "p") + #expect(InsertionSeparator.withLeadingSeparator("p", after: entry.sttPrompt) == " p") } @Test("a nil prior tells a history-only prompt apart from a prior chunk") diff --git a/Tests/BlurtEngineTests/DictationSessionContextTests.swift b/Tests/BlurtEngineTests/DictationSessionContextTests.swift index 3f68cf9a..610616cb 100644 --- a/Tests/BlurtEngineTests/DictationSessionContextTests.swift +++ b/Tests/BlurtEngineTests/DictationSessionContextTests.swift @@ -11,14 +11,14 @@ import Testing /// the seams (`testSeams`) both are ordinary values. @Suite("DictationSession captured context", .timeLimit(.minutes(1))) struct DictationSessionContextTests { - private static let field = FocusCapture.FocusedFieldContext( + private static let field = FocusedFieldContext( priorText: "Hi Sam,", selectedText: "the old plan", windowTitle: "Re: Q3 pricing", fieldLabel: "Body") /// The secure-target fixture, shared by the two tests that assert a password /// is never remembered — one where the read lands in time, one where it lands /// after the upload's budget. - private static let secureField = FocusCapture.FocusedFieldContext( + private static let secureField = FocusedFieldContext( priorText: nil, selectedText: nil, windowTitle: "1Password", fieldLabel: "Password", isSecure: true) @@ -222,7 +222,7 @@ struct DictationSessionContextTests { // `isEmpty` has to count `targetIsSecure`: a secure field yields no prior or // selected text, so without that the snapshot would collapse to `nil` on the // way through `contextStream` and the guard above would never see the flag. - let secure = FocusCapture.FocusedFieldContext( + let secure = FocusedFieldContext( priorText: nil, selectedText: nil, windowTitle: nil, fieldLabel: nil, isSecure: true) #expect(!TranscriptionContext(appName: nil, priorText: nil, targetIsSecure: true).isEmpty) let fixture = makeSession(mode: .transcript("hunter2"), field: secure, frontmost: nil) @@ -318,7 +318,7 @@ struct DictationSessionFailureLogTests { func failureIsLogged() async throws { let fixture = makeSession( mode: .throwError(BlurtError.apiKeyMissing), - field: FocusCapture.FocusedFieldContext( + field: FocusedFieldContext( priorText: nil, selectedText: nil, windowTitle: "Vault", fieldLabel: "Password")) await fixture.session.press() diff --git a/Tests/BlurtEngineTests/EditableTargetTests.swift b/Tests/BlurtEngineTests/EditableTargetTests.swift index 27e45b7d..48297d38 100644 --- a/Tests/BlurtEngineTests/EditableTargetTests.swift +++ b/Tests/BlurtEngineTests/EditableTargetTests.swift @@ -1,76 +1,78 @@ -import Testing +#if os(macOS) + import Testing -@testable import BlurtEngine + @testable import BlurtEngine -@Suite("FocusCapture.isEditableTarget") -struct EditableTargetTests { - @Test("a known text role is editable") - func textRole() { - #expect( - FocusCapture.isEditableTarget( - role: "AXTextArea", valueSettable: false, hasInsertionPoint: false)) - } + @Suite("FocusCapture.isEditableTarget") + struct EditableTargetTests { + @Test("a known text role is editable") + func textRole() { + #expect( + FocusCapture.isEditableTarget( + role: "AXTextArea", valueSettable: false, hasInsertionPoint: false)) + } - @Test("a secure (password) field is still a paste target despite prompt redaction") - func secureFieldIsEditable() { - // Redaction (never read a password into the STT prompt) and editability - // (the paste may land there) are independent: dictating into a password - // field must type, even though its contents are never captured. - #expect( - FocusCapture.isEditableTarget( - role: FocusCapture.secureFieldRole, valueSettable: false, hasInsertionPoint: false)) - } + @Test("a secure (password) field is still a paste target despite prompt redaction") + func secureFieldIsEditable() { + // Redaction (never read a password into the STT prompt) and editability + // (the paste may land there) are independent: dictating into a password + // field must type, even though its contents are never captured. + #expect( + FocusCapture.isEditableTarget( + role: FocusCapture.secureFieldRole, valueSettable: false, hasInsertionPoint: false)) + } - @Test("a settable value is editable even with an unknown role") - func settableValue() { - #expect( - FocusCapture.isEditableTarget( - role: "AXUnknown", valueSettable: true, hasInsertionPoint: false)) - } + @Test("a settable value is editable even with an unknown role") + func settableValue() { + #expect( + FocusCapture.isEditableTarget( + role: "AXUnknown", valueSettable: true, hasInsertionPoint: false)) + } - @Test("an insertion point is editable even with an unknown role") - func insertionPoint() { - #expect( - FocusCapture.isEditableTarget( - role: nil, valueSettable: false, hasInsertionPoint: true)) - } + @Test("an insertion point is editable even with an unknown role") + func insertionPoint() { + #expect( + FocusCapture.isEditableTarget( + role: nil, valueSettable: false, hasInsertionPoint: true)) + } - @Test("a non-text control with no editable signal is not editable") - func nonEditableControl() { - #expect( - !FocusCapture.isEditableTarget( - role: "AXButton", valueSettable: false, hasInsertionPoint: false)) - } + @Test("a non-text control with no editable signal is not editable") + func nonEditableControl() { + #expect( + !FocusCapture.isEditableTarget( + role: "AXButton", valueSettable: false, hasInsertionPoint: false)) + } - @Test("an unknown role with no editable signal is not editable (copy, don't beep)") - func unknownRoleWithoutSignalCopies() { - // A focused element that reports an unrecognized role and exposes no settable - // value or insertion point isn't a text target — copy rather than beep a ⌘V - // into it. (AX-opaque apps — Electron editors and browsers — also land here, - // but are pasted into via the injector's separate app-identity check, not - // this signal test.) - #expect( - !FocusCapture.isEditableTarget( - role: "AXWebArea", valueSettable: false, hasInsertionPoint: false)) - } + @Test("an unknown role with no editable signal is not editable (copy, don't beep)") + func unknownRoleWithoutSignalCopies() { + // A focused element that reports an unrecognized role and exposes no settable + // value or insertion point isn't a text target — copy rather than beep a ⌘V + // into it. (AX-opaque apps — Electron editors and browsers — also land here, + // but are pasted into via the injector's separate app-identity check, not + // this signal test.) + #expect( + !FocusCapture.isEditableTarget( + role: "AXWebArea", valueSettable: false, hasInsertionPoint: false)) + } - @Test("a focused element with an unreadable role is not editable (copy, don't beep)") - func nilRoleWithoutSignalCopies() { - #expect( - !FocusCapture.isEditableTarget( - role: nil, valueSettable: false, hasInsertionPoint: false)) + @Test("a focused element with an unreadable role is not editable (copy, don't beep)") + func nilRoleWithoutSignalCopies() { + #expect( + !FocusCapture.isEditableTarget( + role: nil, valueSettable: false, hasInsertionPoint: false)) + } } -} -@Suite("noTarget phase + overlay mapping") -struct NoTargetPhaseTests { - @Test("noTarget is terminal") - func terminal() { - #expect(PipelinePhase.noTarget.isTerminal) - } + @Suite("noTarget phase + overlay mapping") + struct NoTargetPhaseTests { + @Test("noTarget is terminal") + func terminal() { + #expect(PipelinePhase.noTarget.isTerminal) + } - @Test("noTarget maps to the quiet overlay state, not an error") - func overlayMapping() { - #expect(PipelinePhase.noTarget.overlayState == .noTarget) + @Test("noTarget maps to the quiet overlay state, not an error") + func overlayMapping() { + #expect(PipelinePhase.noTarget.overlayState == .noTarget) + } } -} +#endif diff --git a/Tests/BlurtEngineTests/FocusCaptureTests.swift b/Tests/BlurtEngineTests/FocusCaptureTests.swift index c2fdbe3a..35b1e6b7 100644 --- a/Tests/BlurtEngineTests/FocusCaptureTests.swift +++ b/Tests/BlurtEngineTests/FocusCaptureTests.swift @@ -222,11 +222,13 @@ struct FocusCaptureTests { // MARK: isElectronApp - @Test("isElectronApp is false for a missing app") - func electronNilApp() { - // The positive branch probes the app bundle on disk and is host-dependent; - // the nil guard (no captured target → never the Electron paste exception) - // is the deterministic half. - #expect(!FocusCapture.isElectronApp(nil)) - } + #if os(macOS) + @Test("isElectronApp is false for a missing app") + func electronNilApp() { + // The positive branch probes the app bundle on disk and is host-dependent; + // the nil guard (no captured target → never the Electron paste exception) + // is the deterministic half. + #expect(!FocusCapture.isElectronApp(nil)) + } + #endif } diff --git a/Tests/BlurtEngineTests/InsertionSeparatorTests.swift b/Tests/BlurtEngineTests/InsertionSeparatorTests.swift new file mode 100644 index 00000000..97198f25 --- /dev/null +++ b/Tests/BlurtEngineTests/InsertionSeparatorTests.swift @@ -0,0 +1,164 @@ +// Foundation for `pid_t`: the resolve-insert suite below names it, and the two +// pure text-rule suites this file started as needed no imports at all. +import Foundation +import Testing + +@testable import BlurtEngine + +@Suite("InsertionSeparator.withLeadingSeparator") +struct InsertionSeparatorTests { + @Test("prepends a space when prior text doesn't end in whitespace") + func prependsSpace() { + #expect(InsertionSeparator.withLeadingSeparator("Second.", after: "First.") == " Second.") + } + + @Test("no separator when prior text already ends in a space") + func priorEndsInSpace() { + #expect(InsertionSeparator.withLeadingSeparator("Second.", after: "First. ") == "Second.") + } + + @Test("no separator when prior text ends in a newline") + func priorEndsInNewline() { + #expect(InsertionSeparator.withLeadingSeparator("Second.", after: "First.\n") == "Second.") + } + + @Test("no leading space into an empty field (nil prior text)") + func nilPrior() { + #expect(InsertionSeparator.withLeadingSeparator("Second.", after: nil) == "Second.") + } + + @Test("no leading space when prior text is empty") + func emptyPrior() { + #expect(InsertionSeparator.withLeadingSeparator("Second.", after: "") == "Second.") + } + + @Test("doesn't double up when the new text already starts with whitespace") + func textStartsWithSpace() { + #expect(InsertionSeparator.withLeadingSeparator(" Second.", after: "First.") == " Second.") + } + + @Test("returns empty text unchanged") + func emptyText() { + #expect(InsertionSeparator.withLeadingSeparator("", after: "First.") == "") + } + + @Test("every whitespace class counts, not just space and newline") + func otherWhitespaceClasses() { + // The rules key on Character.isWhitespace: a trailing tab, carriage return, + // or non-breaking space suppresses the separator like a plain space does… + #expect(InsertionSeparator.withLeadingSeparator("Second.", after: "First.\t") == "Second.") + #expect(InsertionSeparator.withLeadingSeparator("Second.", after: "First.\r") == "Second.") + #expect(InsertionSeparator.withLeadingSeparator("Second.", after: "First.\u{00A0}") == "Second.") + // …and text already leading with a tab or non-breaking space isn't doubled. + #expect(InsertionSeparator.withLeadingSeparator("\tSecond.", after: "First.") == "\tSecond.") + #expect(InsertionSeparator.withLeadingSeparator("\u{00A0}Second.", after: "First.") == "\u{00A0}Second.") + } +} + +@Suite("InsertionSeparator.basis") +struct InsertionSeparatorBasisTests { + @Test("AX-read prior text wins even when a prior paste is on record") + func priorTextWins() { + #expect( + InsertionSeparator.basis(priorText: "AX.", lastInserted: "Old.", sameWindow: false) == "AX.") + } + + @Test("falls back to the last paste when AX is opaque and the window is unchanged") + func opaqueSameWindowFallsBack() { + #expect( + InsertionSeparator.basis(priorText: nil, lastInserted: "First.", sameWindow: true) + == "First.") + } + + @Test("does not carry a prior paste across a different window") + func opaqueDifferentWindowNoFallback() { + #expect( + InsertionSeparator.basis(priorText: nil, lastInserted: "First.", sameWindow: false) == nil) + } + + @Test("no basis when AX is opaque and nothing was pasted yet") + func opaqueNoPriorPaste() { + #expect(InsertionSeparator.basis(priorText: nil, lastInserted: nil, sameWindow: true) == nil) + } +} + +#if os(macOS) + /// `resolveInsert` — everything `performInsert` decides before it activates + /// anything: the text to write and the window to remember writing it into. The + /// same-window derivation used to be inline in `performInsert`, so these cases + /// could only be driven by scraping `NSWorkspace` for live applications (and one + /// of them by requiring two). Here the target is just a pid. + @Suite("KeyInjector.resolveInsert") + struct KeyInjectorResolveInsertTests { + private let editor = KeyInjector.WindowIdentity(pid: 501, title: "notes.txt — Editor") + + /// A second dictation with no AX prior text, resolved against `editor`. + private func secondInsert( + targetPID: pid_t?, windowTitle: String?, priorText: String? = nil + ) -> KeyInjector.ResolvedInsert { + KeyInjector.resolveInsert( + text: "Second.", priorText: priorText, windowTitle: windowTitle, targetPID: targetPID, + lastInserted: KeyInjector.ResolvedInsert(text: "First.", window: editor)) + } + + @Test("opaque editor: the same pid and title recovers the spacing from the last paste") + func sameWindowSeparates() { + // Two back-to-back dictations into the same VS Code file or Google Docs tab: + // AX gives no prior text, so what we pasted a moment ago is what precedes + // the caret. + #expect( + secondInsert(targetPID: editor.pid, windowTitle: editor.title) + == KeyInjector.ResolvedInsert(text: " Second.", window: editor)) + } + + @Test("opaque editor: a changed title is a different field, so no phantom space") + func changedTitleDoesNotSeparate() { + // One browser process (or one Electron window) but a different tab or file — + // a shared pid alone must not carry spacing into an unrelated document. The + // new window is still remembered, so a *third* dictation into it separates. + let resolved = secondInsert(targetPID: editor.pid, windowTitle: "Untitled document — Docs") + #expect(resolved.text == "Second.") + #expect(resolved.window == KeyInjector.WindowIdentity(pid: editor.pid, title: "Untitled document — Docs")) + } + + @Test("no readable window title means no match and nothing to remember") + func missingTitleDoesNotSeparate() { + // Can't confirm it's the same window, so the fallback stays off rather than + // guessing — and with no identity to record, the next dictation can't match + // this one either. + #expect( + secondInsert(targetPID: editor.pid, windowTitle: nil) + == KeyInjector.ResolvedInsert(text: "Second.", window: nil)) + } + + @Test("a different target app never inherits the previous paste's spacing") + func changedTargetDoesNotSeparate() { + let resolved = secondInsert(targetPID: 777, windowTitle: editor.title) + #expect(resolved.text == "Second.") + #expect(resolved.window == KeyInjector.WindowIdentity(pid: 777, title: editor.title)) + } + + @Test("no captured target at all: nothing to match, nothing to remember") + func noTargetDoesNotSeparate() { + #expect( + secondInsert(targetPID: nil, windowTitle: editor.title) + == KeyInjector.ResolvedInsert(text: "Second.", window: nil)) + } + + @Test("AX prior text wins over the remembered paste, in the same window or not") + func priorTextWinsOverMemory() { + // The field is readable, so the caret's real neighbour decides — here it + // already ends in whitespace, which the remembered "First." would not have. + #expect(secondInsert(targetPID: editor.pid, windowTitle: editor.title, priorText: "Hello ").text == "Second.") + #expect(secondInsert(targetPID: 777, windowTitle: "Other", priorText: "Hello").text == " Second.") + } + + @Test("a first-ever insert has no memory to fall back on") + func firstInsertHasNoBasis() { + let resolved = KeyInjector.resolveInsert( + text: "First.", priorText: nil, windowTitle: editor.title, targetPID: editor.pid, + lastInserted: nil) + #expect(resolved == KeyInjector.ResolvedInsert(text: "First.", window: editor)) + } + } +#endif diff --git a/Tests/BlurtEngineTests/KeyInjectorFallbackTests.swift b/Tests/BlurtEngineTests/KeyInjectorFallbackTests.swift index f6978ec7..f7f80b22 100644 --- a/Tests/BlurtEngineTests/KeyInjectorFallbackTests.swift +++ b/Tests/BlurtEngineTests/KeyInjectorFallbackTests.swift @@ -1,97 +1,99 @@ -import AppKit -import Foundation -import Testing +#if os(macOS) + import AppKit + import Foundation + import Testing -@testable import BlurtEngine + @testable import BlurtEngine -/// Exercises `KeyInjector.insert`'s abort paths: the mid-activation cancel gate -/// and the copy-to-clipboard fallbacks. Split from `KeyInjectorInsertTests` -/// (same seams, same shared fixtures in `Stubs/InjectorTestSupport.swift`) to -/// stay within the lint file-length budget. -@Suite("KeyInjector.insert fallback & cancel") -struct KeyInjectorFallbackTests { + /// Exercises `KeyInjector.insert`'s abort paths: the mid-activation cancel gate + /// and the copy-to-clipboard fallbacks. Split from `KeyInjectorInsertTests` + /// (same seams, same shared fixtures in `Stubs/InjectorTestSupport.swift`) to + /// stay within the lint file-length budget. + @Suite("KeyInjector.insert fallback & cancel") + struct KeyInjectorFallbackTests { - @Test("a cancel landing during target activation aborts before the paste") - func cancelDuringActivationSkipsPaste() async throws { - let clip = FakeClipboard(string: "user-clipboard") - let posted = ValueBox(false) - let insertTask = ValueBox?>(nil) - let injector = KeyInjector( - pasteSettleDuration: .zero, - postPaste: { - posted.value = true - return true - }, - activateTarget: { _ in - // The cancel lands while activation is in flight; activation itself - // still succeeds, so only the post-activation cancellation gate stands - // between the cancel and the irreversible ⌘V. - insertTask.value?.cancel() - return true - }, - clipboard: clip) - await injector.setTargetApp(try liveTargetApp()) + @Test("a cancel landing during target activation aborts before the paste") + func cancelDuringActivationSkipsPaste() async throws { + let clip = FakeClipboard(string: "user-clipboard") + let posted = ValueBox(false) + let insertTask = ValueBox?>(nil) + let injector = KeyInjector( + pasteSettleDuration: .zero, + postPaste: { + posted.value = true + return true + }, + activateTarget: { _ in + // The cancel lands while activation is in flight; activation itself + // still succeeds, so only the post-activation cancellation gate stands + // between the cancel and the irreversible ⌘V. + insertTask.value?.cancel() + return true + }, + clipboard: clip) + await injector.setTarget(try liveTargetFocus()) - // Park the insert behind a gate until the task handle is stored, so the - // activation closure deterministically has something to cancel. - let gate = AsyncGate() - let task = Task { - await gate.wait() - try await injector.insert("text") - } - insertTask.value = task - gate.open() + // Park the insert behind a gate until the task handle is stored, so the + // activation closure deterministically has something to cancel. + let gate = AsyncGate() + let task = Task { + await gate.wait() + try await injector.insert("text") + } + insertTask.value = task + gate.open() - await #expect(throws: CancellationError.self) { - try await task.value + await #expect(throws: CancellationError.self) { + try await task.value + } + // No ⌘V was posted and the user's clipboard was never touched — the abort + // happened before the save/overwrite stage. + #expect(posted.value == false) + #expect(clip.string == "user-clipboard") } - // No ⌘V was posted and the user's clipboard was never touched — the abort - // happened before the save/overwrite stage. - #expect(posted.value == false) - #expect(clip.string == "user-clipboard") - } - @Test("the clipboard fallback carries the leading separator, not the raw text") - func fallbackKeepsLeadingSeparator() async throws { - let clip = FakeClipboard(string: "user-clipboard") - let injector = KeyInjector( - pasteSettleDuration: .zero, - postPaste: { true }, - activateTarget: { _ in false }, - clipboard: clip) - await injector.setTargetApp(try liveTargetApp()) + @Test("the clipboard fallback carries the leading separator, not the raw text") + func fallbackKeepsLeadingSeparator() async throws { + let clip = FakeClipboard(string: "user-clipboard") + let injector = KeyInjector( + pasteSettleDuration: .zero, + postPaste: { true }, + activateTarget: { _ in false }, + clipboard: clip) + await injector.setTarget(try liveTargetFocus()) - await #expect(throws: BlurtError.targetAppLost) { - try await injector.insert("Second.", after: "First.") + await #expect(throws: BlurtError.targetAppLost) { + try await injector.insert("Second.", after: "First.") + } + // What lands on the clipboard is the final text a paste would have typed — + // separator included — so a manual ⌘V still joins the prior text correctly. + #expect(clip.string == " Second.") } - // What lands on the clipboard is the final text a paste would have typed — - // separator included — so a manual ⌘V still joins the prior text correctly. - #expect(clip.string == " Second.") - } - @Test("a thrown insert releases the paste lock for the next dictation", .timeLimit(.minutes(1))) - func failedInsertReleasesLock() async throws { - let clip = FakeClipboard(string: "user-clipboard") - let editable = ValueBox(false) - let injector = KeyInjector( - pasteSettleDuration: .zero, - postPaste: { true }, - hasEditableTarget: { editable.value }, - clipboard: clip) + @Test("a thrown insert releases the paste lock for the next dictation", .timeLimit(.minutes(1))) + func failedInsertReleasesLock() async throws { + let clip = FakeClipboard(string: "user-clipboard") + let editable = ValueBox(false) + let injector = KeyInjector( + pasteSettleDuration: .zero, + postPaste: { true }, + hasEditableTarget: { editable.value }, + clipboard: clip) - // First insert: nothing editable focused → copy fallback throws. - await #expect(throws: BlurtError.noEditableTarget) { - try await injector.insert("first") - } + // First insert: nothing editable focused → copy fallback throws. + await #expect(throws: BlurtError.noEditableTarget) { + try await injector.insert("first") + } - // The throw path must have released the paste lock: a following insert - // proceeds (rather than deadlocking behind a leaked lock, which the time - // limit would surface) and completes a normal paste + deferred restore. - editable.value = true - try await injector.insert("second") - await injector.pendingSettle?.value - // The restore brings back the pre-paste contents — which the failed insert - // deliberately left as its copied transcript. - #expect(clip.string == "first") + // The throw path must have released the paste lock: a following insert + // proceeds (rather than deadlocking behind a leaked lock, which the time + // limit would surface) and completes a normal paste + deferred restore. + editable.value = true + try await injector.insert("second") + await injector.pendingSettle?.value + // The restore brings back the pre-paste contents — which the failed insert + // deliberately left as its copied transcript. + #expect(clip.string == "first") + } } -} +#endif diff --git a/Tests/BlurtEngineTests/KeyInjectorInsertTests.swift b/Tests/BlurtEngineTests/KeyInjectorInsertTests.swift index a5ba9570..2749c567 100644 --- a/Tests/BlurtEngineTests/KeyInjectorInsertTests.swift +++ b/Tests/BlurtEngineTests/KeyInjectorInsertTests.swift @@ -1,305 +1,307 @@ -import AppKit -import Foundation -import Testing - -@testable import BlurtEngine - -/// Exercises `KeyInjector.insert` and its clipboard save/restore. The real Cmd-V -/// poster is replaced with an injected closure so no keystroke is sent to the -/// focused app, and an in-memory `FakeClipboard` stands in for the system -/// pasteboard so the save/restore + changeCount logic is tested in full -/// isolation — no dependency on (or races with) the host's real clipboard. -@Suite("KeyInjector.insert") -struct KeyInjectorInsertTests { - - @Test("restores the prior pasteboard contents after pasting") - func restoresClipboard() async throws { - let clip = FakeClipboard(string: "user-clipboard") - let injector = KeyInjector(pasteSettleDuration: .zero, postPaste: { true }, clipboard: clip) - try await injector.insert("dictated text") - // The restore is deferred to the background settle task; await it before - // asserting the user's clipboard came back. - await injector.pendingSettle?.value - - #expect(clip.string == "user-clipboard") - } +#if os(macOS) + import AppKit + import Foundation + import Testing + + @testable import BlurtEngine + + /// Exercises `KeyInjector.insert` and its clipboard save/restore. The real Cmd-V + /// poster is replaced with an injected closure so no keystroke is sent to the + /// focused app, and an in-memory `FakeClipboard` stands in for the system + /// pasteboard so the save/restore + changeCount logic is tested in full + /// isolation — no dependency on (or races with) the host's real clipboard. + @Suite("KeyInjector.insert") + struct KeyInjectorInsertTests { + + @Test("restores the prior pasteboard contents after pasting") + func restoresClipboard() async throws { + let clip = FakeClipboard(string: "user-clipboard") + let injector = KeyInjector(pasteSettleDuration: .zero, postPaste: { true }, clipboard: clip) + try await injector.insert("dictated text") + // The restore is deferred to the background settle task; await it before + // asserting the user's clipboard came back. + await injector.pendingSettle?.value - @Test("insert returns before the deferred clipboard restore runs") - func deferRestoreReArmsEarly() async throws { - let clip = FakeClipboard(string: "user-clipboard") - // A 1s settle: long enough that the assertions below run well before the - // background restore fires, so the "still pending" checks are deterministic. - let injector = KeyInjector( - pasteSettleDuration: .seconds(1), postPaste: { true }, clipboard: clip) - - try await injector.insert("dictated text") - - // insert() has returned, but the restore is deferred: the pasted text is - // still on the clipboard and a settle task is in flight. This is what lets - // the pipeline re-arm immediately instead of waiting out the settle. - let settle = await injector.pendingSettle - #expect(clip.string == "dictated text") - #expect(settle != nil) - - // The backgrounded settle eventually restores the user's clipboard. - await settle?.value - #expect(clip.string == "user-clipboard") - } + #expect(clip.string == "user-clipboard") + } - @Test("activates a live target app before pasting") - func activatesTargetApp() async throws { - let activated = ValueBox(false) - let injector = KeyInjector( - pasteSettleDuration: .zero, - postPaste: { true }, - activateTarget: { _ in - activated.value = true - return true - }, - clipboard: FakeClipboard(string: nil)) - await injector.setTargetApp(try liveTargetApp()) - try await injector.insert("dictated text") // must not throw - #expect(activated.value) - } + @Test("insert returns before the deferred clipboard restore runs") + func deferRestoreReArmsEarly() async throws { + let clip = FakeClipboard(string: "user-clipboard") + // A 1s settle: long enough that the assertions below run well before the + // background restore fires, so the "still pending" checks are deterministic. + let injector = KeyInjector( + pasteSettleDuration: .seconds(1), postPaste: { true }, clipboard: clip) + + try await injector.insert("dictated text") + + // insert() has returned, but the restore is deferred: the pasted text is + // still on the clipboard and a settle task is in flight. This is what lets + // the pipeline re-arm immediately instead of waiting out the settle. + let settle = await injector.pendingSettle + #expect(clip.string == "dictated text") + #expect(settle != nil) - @Test("activation failure: skips the paste but leaves the transcript on the clipboard") - func activationFailureSkipsPaste() async throws { - let clip = FakeClipboard(string: "user-clipboard") - let posted = ValueBox(false) - let injector = KeyInjector( - pasteSettleDuration: .zero, - postPaste: { - posted.value = true - return true - }, - activateTarget: { _ in false }, - clipboard: clip) - await injector.setTargetApp(try liveTargetApp()) - - await #expect(throws: BlurtError.targetAppLost) { - try await injector.insert("text") + // The backgrounded settle eventually restores the user's clipboard. + await settle?.value + #expect(clip.string == "user-clipboard") } - // No ⌘V was posted into whatever now has focus, but the transcript survives - // on the clipboard so the failure degrades to a "copied" notice. - #expect(posted.value == false) - #expect(clip.string == "text") - } - @Test("empty text is a no-op (no paste posted)") - func emptyTextNoOp() async throws { - let posted = ValueBox(false) - let injector = KeyInjector( - pasteSettleDuration: .zero, - postPaste: { - posted.value = true - return true - }) - try await injector.insert("") - #expect(posted.value == false) - } + @Test("activates a live target app before pasting") + func activatesTargetApp() async throws { + let activated = ValueBox(false) + let injector = KeyInjector( + pasteSettleDuration: .zero, + postPaste: { true }, + activateTarget: { _ in + activated.value = true + return true + }, + clipboard: FakeClipboard(string: nil)) + await injector.setTarget(try liveTargetFocus()) + try await injector.insert("dictated text") // must not throw + #expect(activated.value) + } - @Test("paste synthesis failure: throws .targetAppLost, transcript stays on the clipboard") - func pasteSynthesisFailureThrows() async { - let clip = FakeClipboard(string: "user-clipboard") - let injector = KeyInjector(pasteSettleDuration: .zero, postPaste: { false }, clipboard: clip) - await #expect(throws: BlurtError.targetAppLost) { - try await injector.insert("text") + @Test("activation failure: skips the paste but leaves the transcript on the clipboard") + func activationFailureSkipsPaste() async throws { + let clip = FakeClipboard(string: "user-clipboard") + let posted = ValueBox(false) + let injector = KeyInjector( + pasteSettleDuration: .zero, + postPaste: { + posted.value = true + return true + }, + activateTarget: { _ in false }, + clipboard: clip) + await injector.setTarget(try liveTargetFocus()) + + await #expect(throws: BlurtError.targetAppLost) { + try await injector.insert("text") + } + // No ⌘V was posted into whatever now has focus, but the transcript survives + // on the clipboard so the failure degrades to a "copied" notice. + #expect(posted.value == false) + #expect(clip.string == "text") } - // The paste never happened, so the transcript is deliberately left on the - // clipboard (not restored away) — the user's words beat the stale snapshot. - #expect(clip.string == "text") - } - @Test("throws .accessibilityPermissionMissing and leaves the clipboard untouched") - func notAccessibilityTrustedThrows() async throws { - let clip = FakeClipboard(string: "user-clipboard") - let posted = ValueBox(false) - let injector = KeyInjector( - pasteSettleDuration: .zero, - postPaste: { - posted.value = true - return true - }, - isAccessibilityTrusted: { false }, - clipboard: clip) - - await #expect(throws: BlurtError.accessibilityPermissionMissing) { - try await injector.insert("text") + @Test("empty text is a no-op (no paste posted)") + func emptyTextNoOp() async throws { + let posted = ValueBox(false) + let injector = KeyInjector( + pasteSettleDuration: .zero, + postPaste: { + posted.value = true + return true + }) + try await injector.insert("") + #expect(posted.value == false) } - // No paste posted, and the user's clipboard is left exactly as it was. - #expect(posted.value == false) - #expect(clip.string == "user-clipboard") - } - @Test("no editable target: skips the paste and leaves the transcript on the clipboard") - func noEditableTargetKeepsClipboard() async throws { - let clip = FakeClipboard(string: "user-clipboard") - let posted = ValueBox(false) - let injector = KeyInjector( - pasteSettleDuration: .zero, - postPaste: { - posted.value = true - return true - }, - hasEditableTarget: { false }, - clipboard: clip) - - await #expect(throws: BlurtError.noEditableTarget) { - try await injector.insert("dictated text") + @Test("paste synthesis failure: throws .targetAppLost, transcript stays on the clipboard") + func pasteSynthesisFailureThrows() async { + let clip = FakeClipboard(string: "user-clipboard") + let injector = KeyInjector(pasteSettleDuration: .zero, postPaste: { false }, clipboard: clip) + await #expect(throws: BlurtError.targetAppLost) { + try await injector.insert("text") + } + // The paste never happened, so the transcript is deliberately left on the + // clipboard (not restored away) — the user's words beat the stale snapshot. + #expect(clip.string == "text") } - // No ⌘V was posted (so macOS never beeps), and the transcript is left on the - // clipboard for a manual paste rather than being restored away. - #expect(posted.value == false) - #expect(clip.string == "dictated text") - } - @Test("AX-opaque Electron editor still pastes despite no editable signal") - func electronEditorPastesWithoutSignal() async throws { - let clip = FakeClipboard(string: "user-clipboard") - let posted = ValueBox(false) - let injector = KeyInjector( - pasteSettleDuration: .zero, - postPaste: { - posted.value = true - return true - }, - hasEditableTarget: { false }, // Electron/Chromium exposes no editable AX signal - isAXOpaqueApp: { _ in true }, // …but it *is* an AX-opaque app (Electron editor) - clipboard: clip) - - // Must not throw noEditableTarget: the Electron exception keeps the paste. - try await injector.insert("dictated text") - await injector.pendingSettle?.value - - // The ⌘V was posted (words pasted, not copy-only), and the user's clipboard - // is restored after the settle. - #expect(posted.value == true) - #expect(clip.string == "user-clipboard") - } + @Test("throws .accessibilityPermissionMissing and leaves the clipboard untouched") + func notAccessibilityTrustedThrows() async throws { + let clip = FakeClipboard(string: "user-clipboard") + let posted = ValueBox(false) + let injector = KeyInjector( + pasteSettleDuration: .zero, + postPaste: { + posted.value = true + return true + }, + isAccessibilityTrusted: { false }, + clipboard: clip) + + await #expect(throws: BlurtError.accessibilityPermissionMissing) { + try await injector.insert("text") + } + // No paste posted, and the user's clipboard is left exactly as it was. + #expect(posted.value == false) + #expect(clip.string == "user-clipboard") + } - @Test("opaque editor: a second dictation into the same window gets a separating space") - func opaqueEditorSameWindowInsertsSeparated() async throws { - let (injector, pasted) = makeRecordingInjector() - // Same target app AND window title for both dictations — e.g. two - // back-to-back dictations into the same VS Code file, or the same Google - // Docs tab. Prior text is nil because both surfaces are Accessibility-opaque. - await injector.setTargetApp(try liveTargetApp()) - - try await injector.insert("First.", after: nil, windowTitle: "notes.txt — Editor") - try await injector.insert("Second.", after: nil, windowTitle: "notes.txt — Editor") - - // First paste lands as-is; the second is separated from it even though AX - // gave us no prior text — the injector remembers what it just pasted into - // this same window. - // - // The one end-to-end case: it pins that `insert` actually threads its - // resolution through (feeding the target's pid in, recording the resolved - // window after the paste). The rules themselves — a changed title, a missing - // title, a changed target — are cases of `resolveInsert` and live in - // `KeyInjectorResolveInsertTests`, which needs no live process at all. - #expect(pasted.values == ["First.", " Second."]) - } + @Test("no editable target: skips the paste and leaves the transcript on the clipboard") + func noEditableTargetKeepsClipboard() async throws { + let clip = FakeClipboard(string: "user-clipboard") + let posted = ValueBox(false) + let injector = KeyInjector( + pasteSettleDuration: .zero, + postPaste: { + posted.value = true + return true + }, + hasEditableTarget: { false }, + clipboard: clip) + + await #expect(throws: BlurtError.noEditableTarget) { + try await injector.insert("dictated text") + } + // No ⌘V was posted (so macOS never beeps), and the transcript is left on the + // clipboard for a manual paste rather than being restored away. + #expect(posted.value == false) + #expect(clip.string == "dictated text") + } - @Test("opaque editor: typing or clicking after a paste drops the same-window separator") - func opaqueEditorUserInputDropsSeparator() async throws { - // Issue #206: in Google Docs the user dictates a sentence, presses Return - // twice for a new paragraph, and dictates again. The window title hasn't - // changed, but the caret no longer follows our paste — a leading space - // there is a stray indent. Input "0 seconds ago" is always newer than the - // first paste, which is what pressing Return between dictations looks like. - let (injector, pasted) = makeRecordingInjector(secondsSinceUserInput: { 0 }) - await injector.setTargetApp(try liveTargetApp()) + @Test("AX-opaque Electron editor still pastes despite no editable signal") + func electronEditorPastesWithoutSignal() async throws { + let clip = FakeClipboard(string: "user-clipboard") + let posted = ValueBox(false) + let injector = KeyInjector( + pasteSettleDuration: .zero, + postPaste: { + posted.value = true + return true + }, + hasEditableTarget: { false }, // Electron/Chromium exposes no editable AX signal + isAXOpaqueApp: { _ in true }, // …but it *is* an AX-opaque app (Electron editor) + clipboard: clip) + + // Must not throw noEditableTarget: the Electron exception keeps the paste. + try await injector.insert("dictated text") + await injector.pendingSettle?.value - try await injector.insert("First.", after: nil, windowTitle: "Doc - Google Docs") - try await injector.insert("Second.", after: nil, windowTitle: "Doc - Google Docs") + // The ⌘V was posted (words pasted, not copy-only), and the user's clipboard + // is restored after the settle. + #expect(posted.value == true) + #expect(clip.string == "user-clipboard") + } - #expect(pasted.values == ["First.", "Second."]) - } + @Test("opaque editor: a second dictation into the same window gets a separating space") + func opaqueEditorSameWindowInsertsSeparated() async throws { + let (injector, pasted) = makeRecordingInjector() + // Same target app AND window title for both dictations — e.g. two + // back-to-back dictations into the same VS Code file, or the same Google + // Docs tab. Prior text is nil because both surfaces are Accessibility-opaque. + await injector.setTarget(try liveTargetFocus()) + + try await injector.insert("First.", after: nil, windowTitle: "notes.txt — Editor") + try await injector.insert("Second.", after: nil, windowTitle: "notes.txt — Editor") + + // First paste lands as-is; the second is separated from it even though AX + // gave us no prior text — the injector remembers what it just pasted into + // this same window. + // + // The one end-to-end case: it pins that `insert` actually threads its + // resolution through (feeding the target's pid in, recording the resolved + // window after the paste). The rules themselves — a changed title, a missing + // title, a changed target — are cases of `resolveInsert` and live in + // `KeyInjectorResolveInsertTests`, which needs no live process at all. + #expect(pasted.values == ["First.", " Second."]) + } - @Test("overlapping inserts still restore the user's original clipboard") - func overlappingInsertsRestoreOriginal() async throws { - let clip = FakeClipboard(string: "user-clipboard") - // A non-zero settle keeps insert #1's backgrounded settle task holding the - // paste lock while a second insert arrives. Without serialization, insert #2 - // would snapshot the clipboard while it still holds insert #1's "first" text - // and later restore *that* instead of the user's original. `postPaste` opens - // the gate while #1 still holds the lock (insert #1 then hands the lock to - // its settle task and returns), so when `wait()` returns insert #2 - // deterministically blocks on the lock until #1's settle restores and - // releases — rather than interleaving. No timing margin needed. - let firstParked = AsyncGate() - let injector = KeyInjector( - pasteSettleDuration: .milliseconds(100), - postPaste: { - firstParked.open() - return true - }, - clipboard: clip) - - let first = Task { try await injector.insert("first") } - await firstParked.wait() - try await injector.insert("second") - try await first.value - // Await the second insert's deferred settle (insert returns before it runs). - await injector.pendingSettle?.value - - #expect(clip.string == "user-clipboard") - } + @Test("opaque editor: typing or clicking after a paste drops the same-window separator") + func opaqueEditorUserInputDropsSeparator() async throws { + // Issue #206: in Google Docs the user dictates a sentence, presses Return + // twice for a new paragraph, and dictates again. The window title hasn't + // changed, but the caret no longer follows our paste — a leading space + // there is a stray indent. Input "0 seconds ago" is always newer than the + // first paste, which is what pressing Return between dictations looks like. + let (injector, pasted) = makeRecordingInjector(secondsSinceUserInput: { 0 }) + await injector.setTarget(try liveTargetFocus()) - @Test("a copy during the settle window is preserved, not clobbered by restore") - func copyDuringSettleIsPreserved() async throws { - let clip = FakeClipboard(string: "user-clipboard") - // The paste posts, then the deferred settle waits. During that wait the user - // copies something new. The restore must not blow that away with the stale - // pre-paste snapshot — once the pasteboard changed under us, the right move - // is to leave the newer contents alone. - let pasted = AsyncGate() - let injector = KeyInjector( - pasteSettleDuration: .milliseconds(100), - postPaste: { - pasted.open() - return true - }, - clipboard: clip) - - let task = Task { try await injector.insert("dictated") } - await pasted.wait() - clip.externalWrite("copied-mid-paste") - try await task.value - // Await the deferred settle: it must see the changed changeCount and skip the - // restore, leaving the user's mid-paste copy intact. - await injector.pendingSettle?.value - - #expect(clip.string == "copied-mid-paste") - } + try await injector.insert("First.", after: nil, windowTitle: "Doc - Google Docs") + try await injector.insert("Second.", after: nil, windowTitle: "Doc - Google Docs") + + #expect(pasted.values == ["First.", "Second."]) + } - @Test("a cancelled task throws before posting any paste") - func cancelledBeforePaste() async throws { - let posted = ValueBox(false) - let injector = KeyInjector( - pasteSettleDuration: .zero, - postPaste: { - posted.value = true - return true - }) - // Gate the body so `cancel()` is guaranteed to land before `insert` reaches - // its first cancellation check. Without this, a busy machine can run the - // task past that check before `cancel()` arrives, posting the paste and - // flaking the assertion. The gate opens only after the cancel. - let gate = AsyncGate() - let task = Task { - await gate.wait() - try await injector.insert("text") + @Test("overlapping inserts still restore the user's original clipboard") + func overlappingInsertsRestoreOriginal() async throws { + let clip = FakeClipboard(string: "user-clipboard") + // A non-zero settle keeps insert #1's backgrounded settle task holding the + // paste lock while a second insert arrives. Without serialization, insert #2 + // would snapshot the clipboard while it still holds insert #1's "first" text + // and later restore *that* instead of the user's original. `postPaste` opens + // the gate while #1 still holds the lock (insert #1 then hands the lock to + // its settle task and returns), so when `wait()` returns insert #2 + // deterministically blocks on the lock until #1's settle restores and + // releases — rather than interleaving. No timing margin needed. + let firstParked = AsyncGate() + let injector = KeyInjector( + pasteSettleDuration: .milliseconds(100), + postPaste: { + firstParked.open() + return true + }, + clipboard: clip) + + let first = Task { try await injector.insert("first") } + await firstParked.wait() + try await injector.insert("second") + try await first.value + // Await the second insert's deferred settle (insert returns before it runs). + await injector.pendingSettle?.value + + #expect(clip.string == "user-clipboard") } - task.cancel() - gate.open() - await #expect(throws: CancellationError.self) { + + @Test("a copy during the settle window is preserved, not clobbered by restore") + func copyDuringSettleIsPreserved() async throws { + let clip = FakeClipboard(string: "user-clipboard") + // The paste posts, then the deferred settle waits. During that wait the user + // copies something new. The restore must not blow that away with the stale + // pre-paste snapshot — once the pasteboard changed under us, the right move + // is to leave the newer contents alone. + let pasted = AsyncGate() + let injector = KeyInjector( + pasteSettleDuration: .milliseconds(100), + postPaste: { + pasted.open() + return true + }, + clipboard: clip) + + let task = Task { try await injector.insert("dictated") } + await pasted.wait() + clip.externalWrite("copied-mid-paste") try await task.value + // Await the deferred settle: it must see the changed changeCount and skip the + // restore, leaving the user's mid-paste copy intact. + await injector.pendingSettle?.value + + #expect(clip.string == "copied-mid-paste") + } + + @Test("a cancelled task throws before posting any paste") + func cancelledBeforePaste() async throws { + let posted = ValueBox(false) + let injector = KeyInjector( + pasteSettleDuration: .zero, + postPaste: { + posted.value = true + return true + }) + // Gate the body so `cancel()` is guaranteed to land before `insert` reaches + // its first cancellation check. Without this, a busy machine can run the + // task past that check before `cancel()` arrives, posting the paste and + // flaking the assertion. The gate opens only after the cancel. + let gate = AsyncGate() + let task = Task { + await gate.wait() + try await injector.insert("text") + } + task.cancel() + gate.open() + await #expect(throws: CancellationError.self) { + try await task.value + } + #expect(posted.value == false) } - #expect(posted.value == false) } -} // The shared fixtures these tests drive the injector with — `liveTargetApp`, // `AsyncGate`, `StringListBox`, `ValueBox` — live in `Stubs/InjectorTestSupport.swift` // so the fallback/cancel suite (a separate file) can reuse them. +#endif diff --git a/Tests/BlurtEngineTests/KeyInjectorLeadingSeparatorTests.swift b/Tests/BlurtEngineTests/KeyInjectorLeadingSeparatorTests.swift deleted file mode 100644 index 311b9ab5..00000000 --- a/Tests/BlurtEngineTests/KeyInjectorLeadingSeparatorTests.swift +++ /dev/null @@ -1,162 +0,0 @@ -// Foundation for `pid_t`: the resolve-insert suite below names it, and the two -// pure text-rule suites this file started as needed no imports at all. -import Foundation -import Testing - -@testable import BlurtEngine - -@Suite("KeyInjector.withLeadingSeparator") -struct KeyInjectorLeadingSeparatorTests { - @Test("prepends a space when prior text doesn't end in whitespace") - func prependsSpace() { - #expect(KeyInjector.withLeadingSeparator("Second.", after: "First.") == " Second.") - } - - @Test("no separator when prior text already ends in a space") - func priorEndsInSpace() { - #expect(KeyInjector.withLeadingSeparator("Second.", after: "First. ") == "Second.") - } - - @Test("no separator when prior text ends in a newline") - func priorEndsInNewline() { - #expect(KeyInjector.withLeadingSeparator("Second.", after: "First.\n") == "Second.") - } - - @Test("no leading space into an empty field (nil prior text)") - func nilPrior() { - #expect(KeyInjector.withLeadingSeparator("Second.", after: nil) == "Second.") - } - - @Test("no leading space when prior text is empty") - func emptyPrior() { - #expect(KeyInjector.withLeadingSeparator("Second.", after: "") == "Second.") - } - - @Test("doesn't double up when the new text already starts with whitespace") - func textStartsWithSpace() { - #expect(KeyInjector.withLeadingSeparator(" Second.", after: "First.") == " Second.") - } - - @Test("returns empty text unchanged") - func emptyText() { - #expect(KeyInjector.withLeadingSeparator("", after: "First.") == "") - } - - @Test("every whitespace class counts, not just space and newline") - func otherWhitespaceClasses() { - // The rules key on Character.isWhitespace: a trailing tab, carriage return, - // or non-breaking space suppresses the separator like a plain space does… - #expect(KeyInjector.withLeadingSeparator("Second.", after: "First.\t") == "Second.") - #expect(KeyInjector.withLeadingSeparator("Second.", after: "First.\r") == "Second.") - #expect(KeyInjector.withLeadingSeparator("Second.", after: "First.\u{00A0}") == "Second.") - // …and text already leading with a tab or non-breaking space isn't doubled. - #expect(KeyInjector.withLeadingSeparator("\tSecond.", after: "First.") == "\tSecond.") - #expect(KeyInjector.withLeadingSeparator("\u{00A0}Second.", after: "First.") == "\u{00A0}Second.") - } -} - -@Suite("KeyInjector.separatorBasis") -struct KeyInjectorSeparatorBasisTests { - @Test("AX-read prior text wins even when a prior paste is on record") - func priorTextWins() { - #expect( - KeyInjector.separatorBasis(priorText: "AX.", lastInserted: "Old.", sameWindow: false) == "AX.") - } - - @Test("falls back to the last paste when AX is opaque and the window is unchanged") - func opaqueSameWindowFallsBack() { - #expect( - KeyInjector.separatorBasis(priorText: nil, lastInserted: "First.", sameWindow: true) - == "First.") - } - - @Test("does not carry a prior paste across a different window") - func opaqueDifferentWindowNoFallback() { - #expect( - KeyInjector.separatorBasis(priorText: nil, lastInserted: "First.", sameWindow: false) == nil) - } - - @Test("no basis when AX is opaque and nothing was pasted yet") - func opaqueNoPriorPaste() { - #expect(KeyInjector.separatorBasis(priorText: nil, lastInserted: nil, sameWindow: true) == nil) - } -} - -/// `resolveInsert` — everything `performInsert` decides before it activates -/// anything: the text to write and the window to remember writing it into. The -/// same-window derivation used to be inline in `performInsert`, so these cases -/// could only be driven by scraping `NSWorkspace` for live applications (and one -/// of them by requiring two). Here the target is just a pid. -@Suite("KeyInjector.resolveInsert") -struct KeyInjectorResolveInsertTests { - private let editor = KeyInjector.WindowIdentity(pid: 501, title: "notes.txt — Editor") - - /// A second dictation with no AX prior text, resolved against `editor`. - private func secondInsert( - targetPID: pid_t?, windowTitle: String?, priorText: String? = nil - ) -> KeyInjector.ResolvedInsert { - KeyInjector.resolveInsert( - text: "Second.", priorText: priorText, windowTitle: windowTitle, targetPID: targetPID, - lastInserted: KeyInjector.ResolvedInsert(text: "First.", window: editor)) - } - - @Test("opaque editor: the same pid and title recovers the spacing from the last paste") - func sameWindowSeparates() { - // Two back-to-back dictations into the same VS Code file or Google Docs tab: - // AX gives no prior text, so what we pasted a moment ago is what precedes - // the caret. - #expect( - secondInsert(targetPID: editor.pid, windowTitle: editor.title) - == KeyInjector.ResolvedInsert(text: " Second.", window: editor)) - } - - @Test("opaque editor: a changed title is a different field, so no phantom space") - func changedTitleDoesNotSeparate() { - // One browser process (or one Electron window) but a different tab or file — - // a shared pid alone must not carry spacing into an unrelated document. The - // new window is still remembered, so a *third* dictation into it separates. - let resolved = secondInsert(targetPID: editor.pid, windowTitle: "Untitled document — Docs") - #expect(resolved.text == "Second.") - #expect(resolved.window == KeyInjector.WindowIdentity(pid: editor.pid, title: "Untitled document — Docs")) - } - - @Test("no readable window title means no match and nothing to remember") - func missingTitleDoesNotSeparate() { - // Can't confirm it's the same window, so the fallback stays off rather than - // guessing — and with no identity to record, the next dictation can't match - // this one either. - #expect( - secondInsert(targetPID: editor.pid, windowTitle: nil) - == KeyInjector.ResolvedInsert(text: "Second.", window: nil)) - } - - @Test("a different target app never inherits the previous paste's spacing") - func changedTargetDoesNotSeparate() { - let resolved = secondInsert(targetPID: 777, windowTitle: editor.title) - #expect(resolved.text == "Second.") - #expect(resolved.window == KeyInjector.WindowIdentity(pid: 777, title: editor.title)) - } - - @Test("no captured target at all: nothing to match, nothing to remember") - func noTargetDoesNotSeparate() { - #expect( - secondInsert(targetPID: nil, windowTitle: editor.title) - == KeyInjector.ResolvedInsert(text: "Second.", window: nil)) - } - - @Test("AX prior text wins over the remembered paste, in the same window or not") - func priorTextWinsOverMemory() { - // The field is readable, so the caret's real neighbour decides — here it - // already ends in whitespace, which the remembered "First." would not have. - #expect(secondInsert(targetPID: editor.pid, windowTitle: editor.title, priorText: "Hello ").text == "Second.") - #expect(secondInsert(targetPID: 777, windowTitle: "Other", priorText: "Hello").text == " Second.") - } - - @Test("a first-ever insert has no memory to fall back on") - func firstInsertHasNoBasis() { - let resolved = KeyInjector.resolveInsert( - text: "First.", priorText: nil, windowTitle: editor.title, targetPID: editor.pid, - lastInserted: nil) - #expect(resolved == KeyInjector.ResolvedInsert(text: "First.", window: editor)) - } -} diff --git a/Tests/BlurtEngineTests/KeyInjectorSystemActionsTests.swift b/Tests/BlurtEngineTests/KeyInjectorSystemActionsTests.swift index da8ac56e..5e6c7c58 100644 --- a/Tests/BlurtEngineTests/KeyInjectorSystemActionsTests.swift +++ b/Tests/BlurtEngineTests/KeyInjectorSystemActionsTests.swift @@ -1,118 +1,120 @@ -import AppKit -import CoreGraphics -import Testing - -@testable import BlurtEngine - -/// The system side of `KeyInjector`'s seams (`KeyInjector+SystemActions.swift`), -/// covering the parts that can be asserted without changing the state of the -/// machine running the suite. -/// -/// The line this suite draws: *reads* of process-global state are fair game -/// (`accessibilityTrusted`, the frontmost-app poll), and so is *building* a -/// CGEvent. What is deliberately left to running the real app is anything that -/// mutates the session — `activate` steals focus, and `postCmdV`'s two `.post` -/// calls would fire a live ⌘V into whatever the developer had open. That is the -/// same reason `check.sh` runs the XCUITest suite on CI only: it commandeers the -/// GUI session. -@Suite("KeyInjector system actions") -struct KeyInjectorSystemActionsTests { - - // MARK: - Cmd-V event construction - - @Test("cmdVEvents builds a V key-down/key-up pair") - func cmdVEventsBuildsPair() throws { - let events = try #require(KeyInjector.cmdVEvents()) - - #expect(events.down.type == .keyDown) - #expect(events.up.type == .keyUp) - // 0x09 is kVK_ANSI_V. Asserted numerically because the Carbon constant isn't - // importable here, which is why the source spells it as a literal too — this - // is the check that keeps that literal honest. - #expect(events.down.getIntegerValueField(.keyboardEventKeycode) == 0x09) - #expect(events.up.getIntegerValueField(.keyboardEventKeycode) == 0x09) - } +#if os(macOS) + import AppKit + import CoreGraphics + import Testing + + @testable import BlurtEngine + + /// The system side of `KeyInjector`'s seams (`KeyInjector+SystemActions.swift`), + /// covering the parts that can be asserted without changing the state of the + /// machine running the suite. + /// + /// The line this suite draws: *reads* of process-global state are fair game + /// (`accessibilityTrusted`, the frontmost-app poll), and so is *building* a + /// CGEvent. What is deliberately left to running the real app is anything that + /// mutates the session — `activate` steals focus, and `postCmdV`'s two `.post` + /// calls would fire a live ⌘V into whatever the developer had open. That is the + /// same reason `check.sh` runs the XCUITest suite on CI only: it commandeers the + /// GUI session. + @Suite("KeyInjector system actions") + struct KeyInjectorSystemActionsTests { + + // MARK: - Cmd-V event construction + + @Test("cmdVEvents builds a V key-down/key-up pair") + func cmdVEventsBuildsPair() throws { + let events = try #require(KeyInjector.cmdVEvents()) + + #expect(events.down.type == .keyDown) + #expect(events.up.type == .keyUp) + // 0x09 is kVK_ANSI_V. Asserted numerically because the Carbon constant isn't + // importable here, which is why the source spells it as a literal too — this + // is the check that keeps that literal honest. + #expect(events.down.getIntegerValueField(.keyboardEventKeycode) == 0x09) + #expect(events.up.getIntegerValueField(.keyboardEventKeycode) == 0x09) + } - @Test("cmdCEvents builds a ⌘C pair on kVK_ANSI_C") - func cmdCEventsBuildsPair() throws { - let events = try #require(KeyInjector.cmdCEvents()) - #expect(events.down.type == .keyDown) - #expect(events.up.type == .keyUp) - #expect(events.down.getIntegerValueField(.keyboardEventKeycode) == 0x08) - #expect(events.down.flags.contains(.maskCommand)) - #expect(events.up.flags.contains(.maskCommand)) - } + @Test("cmdCEvents builds a ⌘C pair on kVK_ANSI_C") + func cmdCEventsBuildsPair() throws { + let events = try #require(KeyInjector.cmdCEvents()) + #expect(events.down.type == .keyDown) + #expect(events.up.type == .keyUp) + #expect(events.down.getIntegerValueField(.keyboardEventKeycode) == 0x08) + #expect(events.down.flags.contains(.maskCommand)) + #expect(events.up.flags.contains(.maskCommand)) + } - @Test("both Cmd-V events carry the command flag") - func cmdVEventsCarryCommand() throws { - let events = try #require(KeyInjector.cmdVEvents()) + @Test("both Cmd-V events carry the command flag") + func cmdVEventsCarryCommand() throws { + let events = try #require(KeyInjector.cmdVEvents()) - // A ⌘-less key-down is a plain "v" — it types a character into the target - // instead of pasting, which is the visible failure this pins. - #expect(events.down.flags.contains(.maskCommand)) - // And a ⌘-less key-up reads as the modifier having been released mid-chord. - #expect(events.up.flags.contains(.maskCommand)) - } + // A ⌘-less key-down is a plain "v" — it types a character into the target + // instead of pasting, which is the visible failure this pins. + #expect(events.down.flags.contains(.maskCommand)) + // And a ⌘-less key-up reads as the modifier having been released mid-chord. + #expect(events.up.flags.contains(.maskCommand)) + } - @Test("Cmd-V events carry no modifier beyond command") - func cmdVEventsCarryNoOtherModifier() throws { - let events = try #require(KeyInjector.cmdVEvents()) - - // ⌘⌥V and ⌘⇧V are "paste and match style" in most apps, and ⌃⌘V is bound - // elsewhere again — so a stray extra modifier doesn't fail loudly, it pastes - // the wrong way. `flags` is assigned (not OR-ed) in `cmdVEvents`, and this is - // what keeps it that way. - for flags in [events.down.flags, events.up.flags] { - #expect(!flags.contains(.maskAlternate)) - #expect(!flags.contains(.maskShift)) - #expect(!flags.contains(.maskControl)) - #expect(!flags.contains(.maskSecondaryFn)) + @Test("Cmd-V events carry no modifier beyond command") + func cmdVEventsCarryNoOtherModifier() throws { + let events = try #require(KeyInjector.cmdVEvents()) + + // ⌘⌥V and ⌘⇧V are "paste and match style" in most apps, and ⌃⌘V is bound + // elsewhere again — so a stray extra modifier doesn't fail loudly, it pastes + // the wrong way. `flags` is assigned (not OR-ed) in `cmdVEvents`, and this is + // what keeps it that way. + for flags in [events.down.flags, events.up.flags] { + #expect(!flags.contains(.maskAlternate)) + #expect(!flags.contains(.maskShift)) + #expect(!flags.contains(.maskControl)) + #expect(!flags.contains(.maskSecondaryFn)) + } } - } - // MARK: - Accessibility trust probe + // MARK: - Accessibility trust probe - @Test("accessibilityTrusted reports the process-wide AX trust state") - func accessibilityTrustedMatchesSystem() { - // The test host's trust state isn't ours to set, so the assertable claim is - // that the seam is a pass-through and not, say, a hard-coded `true` that would - // make `KeyInjector` skip its permission check in production. - #expect(KeyInjector.accessibilityTrusted() == AXIsProcessTrusted()) - } + @Test("accessibilityTrusted reports the process-wide AX trust state") + func accessibilityTrustedMatchesSystem() { + // The test host's trust state isn't ours to set, so the assertable claim is + // that the seam is a pass-through and not, say, a hard-coded `true` that would + // make `KeyInjector` skip its permission check in production. + #expect(KeyInjector.accessibilityTrusted() == AXIsProcessTrusted()) + } - // MARK: - User-input idle probe - - @Test("secondsSinceHardwareInput reports the newest key or click across types") - func secondsSinceHardwareInputIsNewestOfTypes() { - // The host's real input history isn't ours to set, so the assertable claim is - // that the seam takes the *newest* input (the minimum idle time) — a keypress - // alone must be enough to drop the same-window separator fallback. - let keyIdle = CGEventSource.secondsSinceLastEventType(.hidSystemState, eventType: .keyDown) - let idle = KeyInjector.secondsSinceHardwareInput() - #expect(idle >= 0) - #expect(idle <= keyIdle + 1) - } + // MARK: - User-input idle probe + + @Test("secondsSinceHardwareInput reports the newest key or click across types") + func secondsSinceHardwareInputIsNewestOfTypes() { + // The host's real input history isn't ours to set, so the assertable claim is + // that the seam takes the *newest* input (the minimum idle time) — a keypress + // alone must be enough to drop the same-window separator fallback. + let keyIdle = CGEventSource.secondsSinceLastEventType(.hidSystemState, eventType: .keyDown) + let idle = KeyInjector.secondsSinceHardwareInput() + #expect(idle >= 0) + #expect(idle <= keyIdle + 1) + } - // MARK: - Frontmost wait + // MARK: - Frontmost wait - @Test("waitUntilFrontmost reports failure for an app that never comes frontmost") - func waitUntilFrontmostGivesUp() async { - // The test host is a command-line process with no windows, so the window - // server never reports it frontmost — the deterministic "activation didn't - // land" case. `KeyInjector.activateTargetApp` turns this `false` into - // `.targetAppLost` rather than pasting into the wrong app. - #expect(await KeyInjector.waitUntilFrontmost(.current) == false) - } + @Test("waitUntilFrontmost reports failure for an app that never comes frontmost") + func waitUntilFrontmostGivesUp() async { + // The test host is a command-line process with no windows, so the window + // server never reports it frontmost — the deterministic "activation didn't + // land" case. `KeyInjector.activateTargetApp` turns this `false` into + // `.targetAppLost` rather than pasting into the wrong app. + #expect(await KeyInjector.waitUntilFrontmost(.current) == false) + } - @Test("waitUntilFrontmost gives up on a bounded deadline instead of hanging") - func waitUntilFrontmostIsBounded() async { - // Sits on the press→paste path, so an unbounded wait would freeze the paste, - // not just slow it. 350 ms budget; the ceiling leaves room for a loaded CI - // box's scheduling without being loose enough to pass an unbounded loop. - let clock = ContinuousClock() - let elapsed = await clock.measure { - _ = await KeyInjector.waitUntilFrontmost(.current) + @Test("waitUntilFrontmost gives up on a bounded deadline instead of hanging") + func waitUntilFrontmostIsBounded() async { + // Sits on the press→paste path, so an unbounded wait would freeze the paste, + // not just slow it. 350 ms budget; the ceiling leaves room for a loaded CI + // box's scheduling without being loose enough to pass an unbounded loop. + let clock = ContinuousClock() + let elapsed = await clock.measure { + _ = await KeyInjector.waitUntilFrontmost(.current) + } + #expect(elapsed < .seconds(3)) } - #expect(elapsed < .seconds(3)) } -} +#endif diff --git a/Tests/BlurtEngineTests/MemoryLeakTests.swift b/Tests/BlurtEngineTests/MemoryLeakTests.swift index d5d988e1..90617fa4 100644 --- a/Tests/BlurtEngineTests/MemoryLeakTests.swift +++ b/Tests/BlurtEngineTests/MemoryLeakTests.swift @@ -44,10 +44,12 @@ struct MemoryLeakTests { } } - @Test("KeyInjector deallocates") - func keyInjectorNoLeak() async { - await expectNoLeak("KeyInjector") { - KeyInjector() + #if os(macOS) + @Test("KeyInjector deallocates") + func keyInjectorNoLeak() async { + await expectNoLeak("KeyInjector") { + KeyInjector() + } } - } + #endif } diff --git a/Tests/BlurtEngineTests/MicLivenessTests.swift b/Tests/BlurtEngineTests/MicLivenessTests.swift index 26730b15..61e1def8 100644 --- a/Tests/BlurtEngineTests/MicLivenessTests.swift +++ b/Tests/BlurtEngineTests/MicLivenessTests.swift @@ -1,307 +1,309 @@ -import CoreAudio -import Foundation -import Synchronization -import Testing +#if os(macOS) + import CoreAudio + import Foundation + import Synchronization + import Testing -@testable import BlurtEngine + @testable import BlurtEngine -/// What a live input reads even with nobody talking: a mic's own self-noise sits -/// far above `MicLiveness.silenceFloorDB` (one least-significant bit of a 16-bit -/// sample is already about -90 dBFS). The floor separates *digital* silence from -/// any analog input, so this is what a quiet room must look like — live. -private let quietRoomPowerDB: Float = -90 + /// What a live input reads even with nobody talking: a mic's own self-noise sits + /// far above `MicLiveness.silenceFloorDB` (one least-significant bit of a 16-bit + /// sample is already about -90 dBFS). The floor separates *digital* silence from + /// any analog input, so this is what a quiet room must look like — live. + private let quietRoomPowerDB: Float = -90 -/// What a zero-filled buffer from a device that hasn't finished switching reads. -private let digitalSilencePowerDB: Float = -160 + /// What a zero-filled buffer from a device that hasn't finished switching reads. + private let digitalSilencePowerDB: Float = -160 -/// The pure half of `MicCapture.start()`'s liveness gate: the transport-aware -/// wait cap and the poll-until-the-recorder-is-clocking-*and*-metering loop. Driven -/// against `TestClock` so the Bluetooth cap is exercised without waiting real -/// seconds. -@Suite("MicLiveness", .timeLimit(.minutes(1))) -struct MicLivenessTests { - @Test("each transport class gets its own cap: Bluetooth, local, unclassifiable") - func transportTimeouts() { - // The profile switch is the whole reason the gate exists — both Bluetooth - // transport types must get the multi-second budget. - #expect( - MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeBluetooth) - == MicLiveness.bluetoothTimeout) - #expect( - MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeBluetoothLE) - == MicLiveness.bluetoothTimeout) - // Wired/built-in inputs deliver frames near-instantly; a long cap there - // would make a genuinely broken mic feel like a hang. - #expect( - MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeBuiltIn) - == MicLiveness.defaultTimeout) - #expect( - MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeUSB) - == MicLiveness.defaultTimeout) - // Every other local transport takes the same budget — the whole set is - // asserted in `AudioTransportTests.localTransports`. - #expect( - MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeThunderbolt) - == MicLiveness.defaultTimeout) - // An aggregate or virtual input, an unnamed type, and an unreadable read all - // take the middle cap rather than the wired one: the wrapper reports its own - // transport, so real AirPods can be switching underneath it, and 300 ms - // cannot cover that. (This is the assertion that read `defaultTimeout` for - // nil before — the second half of the lost-first-words bug.) - #expect( - MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeAggregate) - == MicLiveness.unknownTransportTimeout) - #expect( - MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeVirtual) - == MicLiveness.unknownTransportTimeout) - #expect( - MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeUnknown) - == MicLiveness.unknownTransportTimeout) - #expect(MicLiveness.timeout(forTransportType: nil) == MicLiveness.unknownTransportTimeout) - // It has to sit strictly between the two caps it splits, or it is just one of - // them under a second name. - #expect(MicLiveness.defaultTimeout < MicLiveness.unknownTransportTimeout) - #expect(MicLiveness.unknownTransportTimeout < MicLiveness.bluetoothTimeout) - } + /// The pure half of `MicCapture.start()`'s liveness gate: the transport-aware + /// wait cap and the poll-until-the-recorder-is-clocking-*and*-metering loop. Driven + /// against `TestClock` so the Bluetooth cap is exercised without waiting real + /// seconds. + @Suite("MicLiveness", .timeLimit(.minutes(1))) + struct MicLivenessTests { + @Test("each transport class gets its own cap: Bluetooth, local, unclassifiable") + func transportTimeouts() { + // The profile switch is the whole reason the gate exists — both Bluetooth + // transport types must get the multi-second budget. + #expect( + MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeBluetooth) + == MicLiveness.bluetoothTimeout) + #expect( + MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeBluetoothLE) + == MicLiveness.bluetoothTimeout) + // Wired/built-in inputs deliver frames near-instantly; a long cap there + // would make a genuinely broken mic feel like a hang. + #expect( + MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeBuiltIn) + == MicLiveness.defaultTimeout) + #expect( + MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeUSB) + == MicLiveness.defaultTimeout) + // Every other local transport takes the same budget — the whole set is + // asserted in `AudioTransportTests.localTransports`. + #expect( + MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeThunderbolt) + == MicLiveness.defaultTimeout) + // An aggregate or virtual input, an unnamed type, and an unreadable read all + // take the middle cap rather than the wired one: the wrapper reports its own + // transport, so real AirPods can be switching underneath it, and 300 ms + // cannot cover that. (This is the assertion that read `defaultTimeout` for + // nil before — the second half of the lost-first-words bug.) + #expect( + MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeAggregate) + == MicLiveness.unknownTransportTimeout) + #expect( + MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeVirtual) + == MicLiveness.unknownTransportTimeout) + #expect( + MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeUnknown) + == MicLiveness.unknownTransportTimeout) + #expect(MicLiveness.timeout(forTransportType: nil) == MicLiveness.unknownTransportTimeout) + // It has to sit strictly between the two caps it splits, or it is just one of + // them under a second name. + #expect(MicLiveness.defaultTimeout < MicLiveness.unknownTransportTimeout) + #expect(MicLiveness.unknownTransportTimeout < MicLiveness.bluetoothTimeout) + } - @Test("frames already arriving confirm immediately, without sleeping") - func immediateLiveness() async { - let clock = TestClock() - // Never advanced: a sleep would park forever, so returning at all proves - // the fast path never sleeps (the suite's time limit backs that up). - let gap = await MicLiveness.waitUntilLive( - timeout: .seconds(1), clock: clock, deliveredFrames: { 1024 }, - inputPowerDB: { quietRoomPowerDB }) - #expect(gap == .zero) - } + @Test("frames already arriving confirm immediately, without sleeping") + func immediateLiveness() async { + let clock = TestClock() + // Never advanced: a sleep would park forever, so returning at all proves + // the fast path never sleeps (the suite's time limit backs that up). + let gap = await MicLiveness.waitUntilLive( + timeout: .seconds(1), clock: clock, deliveredFrames: { 1024 }, + inputPowerDB: { quietRoomPowerDB }) + #expect(gap == .zero) + } - @Test("the poll interval doubles, so a slow bring-up isn't hundreds of wakeups") - func livenessBacksOff() async { - let clock = TestClock() - let polls = Mutex(0) - async let gap = MicLiveness.waitUntilLive( - timeout: MicLiveness.bluetoothTimeout, clock: clock, - deliveredFrames: { - // Nothing delivered for the first three checks — the route still - // switching — then buffers start arriving. - polls.withLock { polls in - polls += 1 - return polls < 4 ? 0 : 1024 - } - }, inputPowerDB: { quietRoomPowerDB }) - // Each wait is twice the last: 1 ms, 2 ms, 4 ms. Driving the clock by the - // exact expected delay is also the assertion — `waitUntilSleeping` matches on - // the deadline, so a loop that slept a fixed quantum would hang here rather - // than quietly pass. - var expected = MicLiveness.initialPollInterval - var elapsed = Duration.zero - for _ in 1...3 { - await clock.waitUntilSleeping(for: expected) - clock.advance(by: expected) - elapsed += expected - expected = min(expected * 2, MicLiveness.maxPollInterval) + @Test("the poll interval doubles, so a slow bring-up isn't hundreds of wakeups") + func livenessBacksOff() async { + let clock = TestClock() + let polls = Mutex(0) + async let gap = MicLiveness.waitUntilLive( + timeout: MicLiveness.bluetoothTimeout, clock: clock, + deliveredFrames: { + // Nothing delivered for the first three checks — the route still + // switching — then buffers start arriving. + polls.withLock { polls in + polls += 1 + return polls < 4 ? 0 : 1024 + } + }, inputPowerDB: { quietRoomPowerDB }) + // Each wait is twice the last: 1 ms, 2 ms, 4 ms. Driving the clock by the + // exact expected delay is also the assertion — `waitUntilSleeping` matches on + // the deadline, so a loop that slept a fixed quantum would hang here rather + // than quietly pass. + var expected = MicLiveness.initialPollInterval + var elapsed = Duration.zero + for _ in 1...3 { + await clock.waitUntilSleeping(for: expected) + clock.advance(by: expected) + elapsed += expected + expected = min(expected * 2, MicLiveness.maxPollInterval) + } + #expect(await gap == elapsed) } - #expect(await gap == elapsed) - } - @Test("the backoff is capped rather than doubling without limit") - func backoffIsCapped() async { - // Otherwise a 2.5 s Bluetooth cap would end in multi-second sleeps and - // overshoot the deadline it is supposed to respect. - let clock = TestClock() - let polls = Mutex(0) - async let gap = MicLiveness.waitUntilLive( - timeout: .seconds(30), clock: clock, - deliveredFrames: { - polls.withLock { polls in - polls += 1 - return polls < 12 ? 0 : 1024 - } - }, inputPowerDB: { quietRoomPowerDB }) - var expected = MicLiveness.initialPollInterval - for _ in 1...11 { - await clock.waitUntilSleeping(for: expected) - clock.advance(by: expected) - expected = min(expected * 2, MicLiveness.maxPollInterval) + @Test("the backoff is capped rather than doubling without limit") + func backoffIsCapped() async { + // Otherwise a 2.5 s Bluetooth cap would end in multi-second sleeps and + // overshoot the deadline it is supposed to respect. + let clock = TestClock() + let polls = Mutex(0) + async let gap = MicLiveness.waitUntilLive( + timeout: .seconds(30), clock: clock, + deliveredFrames: { + polls.withLock { polls in + polls += 1 + return polls < 12 ? 0 : 1024 + } + }, inputPowerDB: { quietRoomPowerDB }) + var expected = MicLiveness.initialPollInterval + for _ in 1...11 { + await clock.waitUntilSleeping(for: expected) + clock.advance(by: expected) + expected = min(expected * 2, MicLiveness.maxPollInterval) + } + #expect(expected == MicLiveness.maxPollInterval) + #expect(await gap != nil) } - #expect(expected == MicLiveness.maxPollInterval) - #expect(await gap != nil) - } - @Test("frames arriving as digital silence are not live — the shipped bug") - func zeroFilledBuffersAreNotLive() async { - // The gate's original signal was frame arrival alone, and macOS can hand - // a stale or not-yet-switched device's queue all-zero buffers (the same - // failure that retired the `AVAudioEngine` capture path). Frames of digital - // silence arrive and count exactly like real audio, so the wait - // was satisfied on the first ~1 ms poll while the AirPods link was still - // renegotiating: the "Connecting…" pill flashed past and the start chime - // fired over a dead mic. With the power term the wait holds, and a device - // that never delivers ends in nil — which `MicCapture` fails the press on. - #expect(await waitToCap(powerDB: { digitalSilencePowerDB }) == nil) - } + @Test("frames arriving as digital silence are not live — the shipped bug") + func zeroFilledBuffersAreNotLive() async { + // The gate's original signal was frame arrival alone, and macOS can hand + // a stale or not-yet-switched device's queue all-zero buffers (the same + // failure that retired the `AVAudioEngine` capture path). Frames of digital + // silence arrive and count exactly like real audio, so the wait + // was satisfied on the first ~1 ms poll while the AirPods link was still + // renegotiating: the "Connecting…" pill flashed past and the start chime + // fired over a dead mic. With the power term the wait holds, and a device + // that never delivers ends in nil — which `MicCapture` fails the press on. + #expect(await waitToCap(powerDB: { digitalSilencePowerDB }) == nil) + } - @Test("the floor is exclusive, and a non-finite reading is not live either") - func silenceFloorEdges() async { - // At the floor, not above it. - #expect(await waitToCap(powerDB: { MicLiveness.silenceFloorDB }) == nil) - // NaN fails both comparisons; -infinity is what a truly muted device can - // report. Both must read as not-live, so the press fails rather than - // confirming a route that is delivering nothing. - #expect(await waitToCap(powerDB: { Float.nan }) == nil) - #expect(await waitToCap(powerDB: { -Float.infinity }) == nil) - } + @Test("the floor is exclusive, and a non-finite reading is not live either") + func silenceFloorEdges() async { + // At the floor, not above it. + #expect(await waitToCap(powerDB: { MicLiveness.silenceFloorDB }) == nil) + // NaN fails both comparisons; -infinity is what a truly muted device can + // report. Both must read as not-live, so the press fails rather than + // confirming a route that is delivering nothing. + #expect(await waitToCap(powerDB: { Float.nan }) == nil) + #expect(await waitToCap(powerDB: { -Float.infinity }) == nil) + } - @Test("delivered frames plus real input power confirm at once") - func realInputPowerConfirmsPromptly() async { - // The other half of the floor's job: it must not become voice-activity - // detection. A live mic in a silent room reads its own self-noise, well above - // the floor, so a user who presses and says nothing is confirmed immediately - // instead of failing at the cap — no grace period needed. The clock is never - // advanced here, so a sleep would park forever and the suite's time limit - // would fail this rather than let it pass slowly. - let clock = TestClock() - let gap = await MicLiveness.waitUntilLive( - timeout: MicLiveness.bluetoothTimeout, clock: clock, deliveredFrames: { 1024 }, - inputPowerDB: { quietRoomPowerDB }) - #expect(gap == .zero) - } + @Test("delivered frames plus real input power confirm at once") + func realInputPowerConfirmsPromptly() async { + // The other half of the floor's job: it must not become voice-activity + // detection. A live mic in a silent room reads its own self-noise, well above + // the floor, so a user who presses and says nothing is confirmed immediately + // instead of failing at the cap — no grace period needed. The clock is never + // advanced here, so a sleep would park forever and the suite's time limit + // would fail this rather than let it pass slowly. + let clock = TestClock() + let gap = await MicLiveness.waitUntilLive( + timeout: MicLiveness.bluetoothTimeout, clock: clock, deliveredFrames: { 1024 }, + inputPowerDB: { quietRoomPowerDB }) + #expect(gap == .zero) + } - @Test("the gate's log line carries the transport, the wait, and the meter") - func logSummaryCarriesTheDiagnosis() { - // This line is the field evidence for the zero-buffer premise, which can't be - // reproduced without a Mac and AirPods — so what it must contain is pinned - // here rather than left to an untested call site in `MicCapture`. - let confirmed = MicLiveness.logSummary( - gap: .milliseconds(1420), timeout: MicLiveness.bluetoothTimeout, - transportType: kAudioDeviceTransportTypeBluetooth, powerDB: -37.5) - #expect(confirmed.contains("live after 1420 ms")) - #expect(confirmed.contains("transport=\(kAudioDeviceTransportTypeBluetooth)")) - #expect(confirmed.contains("powerDB=-37.5")) + @Test("the gate's log line carries the transport, the wait, and the meter") + func logSummaryCarriesTheDiagnosis() { + // This line is the field evidence for the zero-buffer premise, which can't be + // reproduced without a Mac and AirPods — so what it must contain is pinned + // here rather than left to an untested call site in `MicCapture`. + let confirmed = MicLiveness.logSummary( + gap: .milliseconds(1420), timeout: MicLiveness.bluetoothTimeout, + transportType: kAudioDeviceTransportTypeBluetooth, powerDB: -37.5) + #expect(confirmed.contains("live after 1420 ms")) + #expect(confirmed.contains("transport=\(kAudioDeviceTransportTypeBluetooth)")) + #expect(confirmed.contains("powerDB=-37.5")) - // A timeout reports the cap it waited out. An unreadable transport has to - // stay distinguishable from `kAudioDeviceTransportTypeUnknown`, which is 0. - let failedOpen = MicLiveness.logSummary( - gap: nil, timeout: MicLiveness.unknownTransportTimeout, transportType: nil, - powerDB: digitalSilencePowerDB) - #expect(failedOpen.contains("unconfirmed after 1000 ms")) - #expect(failedOpen.contains("transport=nil")) - #expect(failedOpen.contains("powerDB=-160.0")) - } + // A timeout reports the cap it waited out. An unreadable transport has to + // stay distinguishable from `kAudioDeviceTransportTypeUnknown`, which is 0. + let failedOpen = MicLiveness.logSummary( + gap: nil, timeout: MicLiveness.unknownTransportTimeout, transportType: nil, + powerDB: digitalSilencePowerDB) + #expect(failedOpen.contains("unconfirmed after 1000 ms")) + #expect(failedOpen.contains("transport=nil")) + #expect(failedOpen.contains("powerDB=-160.0")) + } - @Test("a recorder that never delivers times out with nil — the not-live verdict") - func timeoutReturnsNil() async { - // nil is pure information — the *caller* decides what it means, and - // `MicCapture.start()` fails the press closed on it (tears the recorder - // down, throws `.audioCaptureFailed`, the error pill). What is pinned here - // is only that the deadline actually produces the verdict. - let clock = TestClock() - let timeout = MicLiveness.initialPollInterval * 3 - async let gap = MicLiveness.waitUntilLive( - timeout: timeout, clock: clock, deliveredFrames: { 0 }, - inputPowerDB: { digitalSilencePowerDB }) - var expected = MicLiveness.initialPollInterval - for _ in 1...2 { - await clock.waitUntilSleeping(for: expected) - clock.advance(by: expected) - expected = min(expected * 2, MicLiveness.maxPollInterval) + @Test("a recorder that never delivers times out with nil — the not-live verdict") + func timeoutReturnsNil() async { + // nil is pure information — the *caller* decides what it means, and + // `MicCapture.start()` fails the press closed on it (tears the recorder + // down, throws `.audioCaptureFailed`, the error pill). What is pinned here + // is only that the deadline actually produces the verdict. + let clock = TestClock() + let timeout = MicLiveness.initialPollInterval * 3 + async let gap = MicLiveness.waitUntilLive( + timeout: timeout, clock: clock, deliveredFrames: { 0 }, + inputPowerDB: { digitalSilencePowerDB }) + var expected = MicLiveness.initialPollInterval + for _ in 1...2 { + await clock.waitUntilSleeping(for: expected) + clock.advance(by: expected) + expected = min(expected * 2, MicLiveness.maxPollInterval) + } + #expect(await gap == nil) } - #expect(await gap == nil) - } - /// Drives the wait to its cap against a clock this test owns, for a power probe - /// that never reports live while frames are already arriving. - /// Shared so each "not live" reading is one assertion rather than a copy of the - /// backoff-driving loop. - private func waitToCap(powerDB: @escaping @Sendable () -> Float) async -> Duration? { - let clock = TestClock() - let timeout = MicLiveness.initialPollInterval * 3 - async let gap = MicLiveness.waitUntilLive( - timeout: timeout, clock: clock, deliveredFrames: { 1024 }, inputPowerDB: powerDB) - var expected = MicLiveness.initialPollInterval - for _ in 1...2 { - await clock.waitUntilSleeping(for: expected) - clock.advance(by: expected) - expected = min(expected * 2, MicLiveness.maxPollInterval) + /// Drives the wait to its cap against a clock this test owns, for a power probe + /// that never reports live while frames are already arriving. + /// Shared so each "not live" reading is one assertion rather than a copy of the + /// backoff-driving loop. + private func waitToCap(powerDB: @escaping @Sendable () -> Float) async -> Duration? { + let clock = TestClock() + let timeout = MicLiveness.initialPollInterval * 3 + async let gap = MicLiveness.waitUntilLive( + timeout: timeout, clock: clock, deliveredFrames: { 1024 }, inputPowerDB: powerDB) + var expected = MicLiveness.initialPollInterval + for _ in 1...2 { + await clock.waitUntilSleeping(for: expected) + clock.advance(by: expected) + expected = min(expected * 2, MicLiveness.maxPollInterval) + } + return await gap } - return await gap } -} -/// The transport classification both the liveness cap and `MicCapture`'s tail -/// linger hang off. Pure and pinned here because `AudioRoute`, which reads the -/// raw value, needs real hardware and is excluded from the coverage gate — so -/// this is the only place the decision can be tested. -@Suite("AudioTransport") -struct AudioTransportTests { - @Test("both Bluetooth transport types count") - func bluetoothTransports() { - #expect(AudioTransport.isBluetooth(kAudioDeviceTransportTypeBluetooth)) - #expect(AudioTransport.isBluetooth(kAudioDeviceTransportTypeBluetoothLE)) - } + /// The transport classification both the liveness cap and `MicCapture`'s tail + /// linger hang off. Pure and pinned here because `AudioRoute`, which reads the + /// raw value, needs real hardware and is excluded from the coverage gate — so + /// this is the only place the decision can be tested. + @Suite("AudioTransport") + struct AudioTransportTests { + @Test("both Bluetooth transport types count") + func bluetoothTransports() { + #expect(AudioTransport.isBluetooth(kAudioDeviceTransportTypeBluetooth)) + #expect(AudioTransport.isBluetooth(kAudioDeviceTransportTypeBluetoothLE)) + } - @Test("the tail linger is Bluetooth-only") - func tailLinger() { - // The other transport-conditional policy, here rather than in `MicCapture` - // so it is reachable by `swift test` at all — the capture actor needs real - // hardware and is excluded from the coverage gate. - #expect( - AudioTransport.tailLinger(forTransportType: kAudioDeviceTransportTypeBluetooth) - == AudioTransport.bluetoothTailLinger) - // `.zero`, not a small duration: `stop()` skips the sleep entirely on a - // wired input rather than awaiting a nominal one. - #expect(AudioTransport.tailLinger(forTransportType: kAudioDeviceTransportTypeBuiltIn) == .zero) - #expect(AudioTransport.tailLinger(forTransportType: nil) == .zero) - } + @Test("the tail linger is Bluetooth-only") + func tailLinger() { + // The other transport-conditional policy, here rather than in `MicCapture` + // so it is reachable by `swift test` at all — the capture actor needs real + // hardware and is excluded from the coverage gate. + #expect( + AudioTransport.tailLinger(forTransportType: kAudioDeviceTransportTypeBluetooth) + == AudioTransport.bluetoothTailLinger) + // `.zero`, not a small duration: `stop()` skips the sleep entirely on a + // wired input rather than awaiting a nominal one. + #expect(AudioTransport.tailLinger(forTransportType: kAudioDeviceTransportTypeBuiltIn) == .zero) + #expect(AudioTransport.tailLinger(forTransportType: nil) == .zero) + } - @Test("every wired and on-board transport counts as local") - func localTransports() { - // Each constant asserted individually: `isLocal` is a switch over all of - // them, and a value that only matches the first pattern never executes the - // rest — so this is also what keeps those lines covered. - #expect(AudioTransport.isLocal(kAudioDeviceTransportTypeBuiltIn)) - #expect(AudioTransport.isLocal(kAudioDeviceTransportTypeUSB)) - #expect(AudioTransport.isLocal(kAudioDeviceTransportTypePCI)) - #expect(AudioTransport.isLocal(kAudioDeviceTransportTypeFireWire)) - #expect(AudioTransport.isLocal(kAudioDeviceTransportTypeThunderbolt)) - #expect(AudioTransport.isLocal(kAudioDeviceTransportTypeDisplayPort)) - #expect(AudioTransport.isLocal(kAudioDeviceTransportTypeHDMI)) - } + @Test("every wired and on-board transport counts as local") + func localTransports() { + // Each constant asserted individually: `isLocal` is a switch over all of + // them, and a value that only matches the first pattern never executes the + // rest — so this is also what keeps those lines covered. + #expect(AudioTransport.isLocal(kAudioDeviceTransportTypeBuiltIn)) + #expect(AudioTransport.isLocal(kAudioDeviceTransportTypeUSB)) + #expect(AudioTransport.isLocal(kAudioDeviceTransportTypePCI)) + #expect(AudioTransport.isLocal(kAudioDeviceTransportTypeFireWire)) + #expect(AudioTransport.isLocal(kAudioDeviceTransportTypeThunderbolt)) + #expect(AudioTransport.isLocal(kAudioDeviceTransportTypeDisplayPort)) + #expect(AudioTransport.isLocal(kAudioDeviceTransportTypeHDMI)) + } - @Test("Bluetooth, wrapped, unnamed and unreadable transports are not local") - func nonLocalTransports() { - // `isLocal` is deliberately not `!isBluetooth`: these all fall through to - // `MicLiveness.unknownTransportTimeout`, because an aggregate or virtual - // device reports its own transport while real AirPods switch underneath it. - #expect(!AudioTransport.isLocal(kAudioDeviceTransportTypeAggregate)) - #expect(!AudioTransport.isLocal(kAudioDeviceTransportTypeVirtual)) - #expect(!AudioTransport.isLocal(kAudioDeviceTransportTypeUnknown)) - #expect(!AudioTransport.isLocal(kAudioDeviceTransportTypeBluetooth)) - #expect(!AudioTransport.isLocal(nil)) - } + @Test("Bluetooth, wrapped, unnamed and unreadable transports are not local") + func nonLocalTransports() { + // `isLocal` is deliberately not `!isBluetooth`: these all fall through to + // `MicLiveness.unknownTransportTimeout`, because an aggregate or virtual + // device reports its own transport while real AirPods switch underneath it. + #expect(!AudioTransport.isLocal(kAudioDeviceTransportTypeAggregate)) + #expect(!AudioTransport.isLocal(kAudioDeviceTransportTypeVirtual)) + #expect(!AudioTransport.isLocal(kAudioDeviceTransportTypeUnknown)) + #expect(!AudioTransport.isLocal(kAudioDeviceTransportTypeBluetooth)) + #expect(!AudioTransport.isLocal(nil)) + } - @Test("wired, built-in, and unreadable transports do not") - func nonBluetoothTransports() { - #expect(!AudioTransport.isBluetooth(kAudioDeviceTransportTypeBuiltIn)) - #expect(!AudioTransport.isBluetooth(kAudioDeviceTransportTypeUSB)) - #expect(!AudioTransport.isBluetooth(kAudioDeviceTransportTypeAggregate)) - // nil is the conservative answer for both consumers: the short wait cap and - // no tail linger. Padding every wired capture with a delay would be a worse - // regression than losing the tail on a device we couldn't classify. - #expect(!AudioTransport.isBluetooth(nil)) + @Test("wired, built-in, and unreadable transports do not") + func nonBluetoothTransports() { + #expect(!AudioTransport.isBluetooth(kAudioDeviceTransportTypeBuiltIn)) + #expect(!AudioTransport.isBluetooth(kAudioDeviceTransportTypeUSB)) + #expect(!AudioTransport.isBluetooth(kAudioDeviceTransportTypeAggregate)) + // nil is the conservative answer for both consumers: the short wait cap and + // no tail linger. Padding every wired capture with a delay would be a worse + // regression than losing the tail on a device we couldn't classify. + #expect(!AudioTransport.isBluetooth(nil)) + } } -} -/// `Duration.milliseconds` backs the latency lines `MicCapture` logs for the -/// liveness gap — the field evidence for whether the gate is doing anything — -/// so a silently wrong conversion would make those logs lie. -@Suite("Duration.milliseconds") -struct DurationMillisecondsTests { - @Test func convertsWholeAndFractionalDurations() { - #expect(Duration.milliseconds(250).milliseconds == 250) - #expect(Duration.seconds(2).milliseconds == 2000) - #expect(Duration.zero.milliseconds == 0) - // Sub-millisecond durations keep their fraction rather than truncating to 0. - #expect(Duration.microseconds(500).milliseconds == 0.5) + /// `Duration.milliseconds` backs the latency lines `MicCapture` logs for the + /// liveness gap — the field evidence for whether the gate is doing anything — + /// so a silently wrong conversion would make those logs lie. + @Suite("Duration.milliseconds") + struct DurationMillisecondsTests { + @Test func convertsWholeAndFractionalDurations() { + #expect(Duration.milliseconds(250).milliseconds == 250) + #expect(Duration.seconds(2).milliseconds == 2000) + #expect(Duration.zero.milliseconds == 0) + // Sub-millisecond durations keep their fraction rather than truncating to 0. + #expect(Duration.microseconds(500).milliseconds == 0.5) + } } -} +#endif diff --git a/Tests/BlurtEngineTests/PermissionsCheckerTests.swift b/Tests/BlurtEngineTests/PermissionsCheckerTests.swift index 02d5c50f..9ba23e5c 100644 --- a/Tests/BlurtEngineTests/PermissionsCheckerTests.swift +++ b/Tests/BlurtEngineTests/PermissionsCheckerTests.swift @@ -1,80 +1,82 @@ -import Foundation -import Testing +#if os(macOS) + import Foundation + import Testing -@testable import BlurtEngine + @testable import BlurtEngine -@Suite("PermissionsChecker") -struct PermissionsCheckerTests { + @Suite("PermissionsChecker") + struct PermissionsCheckerTests { - @Test("allGranted requires both microphone and accessibility") - func allGrantedLogic() { - #expect(PermissionStatus(microphone: true, accessibility: true).allGranted) - #expect(!PermissionStatus(microphone: true, accessibility: false).allGranted) - #expect(!PermissionStatus(microphone: false, accessibility: true).allGranted) - #expect(!PermissionStatus(microphone: false, accessibility: false).allGranted) - } + @Test("allGranted requires both microphone and accessibility") + func allGrantedLogic() { + #expect(PermissionStatus(microphone: true, accessibility: true).allGranted) + #expect(!PermissionStatus(microphone: true, accessibility: false).allGranted) + #expect(!PermissionStatus(microphone: false, accessibility: true).allGranted) + #expect(!PermissionStatus(microphone: false, accessibility: false).allGranted) + } - @Test("PermissionStatus is value-equatable") - func equatable() { - #expect( - PermissionStatus(microphone: true, accessibility: false) - == PermissionStatus(microphone: true, accessibility: false)) - #expect( - PermissionStatus(microphone: true, accessibility: false) - != PermissionStatus(microphone: false, accessibility: false)) - } + @Test("PermissionStatus is value-equatable") + func equatable() { + #expect( + PermissionStatus(microphone: true, accessibility: false) + == PermissionStatus(microphone: true, accessibility: false)) + #expect( + PermissionStatus(microphone: true, accessibility: false) + != PermissionStatus(microphone: false, accessibility: false)) + } - /// Every combination of the two probes, so the field each one feeds is pinned. - static let probeCases: [(mic: Bool, accessibility: Bool)] = [ - (true, true), (true, false), (false, true), (false, false), - ] + /// Every combination of the two probes, so the field each one feeds is pinned. + static let probeCases: [(mic: Bool, accessibility: Bool)] = [ + (true, true), (true, false), (false, true), (false, false), + ] - @Test("check reports each probe in its own field", arguments: probeCases) - func checkWiresProbesToFields(mic: Bool, accessibility: Bool) { - // The one thing `check()` does that can be wrong: which probe feeds which - // field. Driving both probes pins it — swap them and half these rows fail. - #expect( - PermissionsChecker.check(micGranted: { mic }, axTrusted: { accessibility }) - == PermissionStatus(microphone: mic, accessibility: accessibility)) - } - - @Test("check consults both probes exactly once") - func checkReadsEachProbeOnce() { - // Each real probe is a TCC read on the permission-poll timer; re-reading one - // per call would double that traffic for a struct with two fields. - let micReads = Counter() - let axReads = Counter() - _ = PermissionsChecker.check( - micGranted: { - _ = micReads.next() - return true - }, - axTrusted: { - _ = axReads.next() - return true - }) - #expect(micReads.value == 1) - #expect(axReads.value == 1) - } + @Test("check reports each probe in its own field", arguments: probeCases) + func checkWiresProbesToFields(mic: Bool, accessibility: Bool) { + // The one thing `check()` does that can be wrong: which probe feeds which + // field. Driving both probes pins it — swap them and half these rows fail. + #expect( + PermissionsChecker.check(micGranted: { mic }, axTrusted: { accessibility }) + == PermissionStatus(microphone: mic, accessibility: accessibility)) + } - /// Smoke tests, deliberately assertion-free: both entry points read - /// process-global TCC state this host can't set, so all they can establish is - /// that the real probes run without throwing or prompting. The behaviour they - /// compose is covered above, against injected probes. - @Suite("PermissionsChecker smoke") - struct SmokeTests { - @Test("the production check() runs against the real probes without prompting") - func productionCheckRuns() { - _ = PermissionsChecker.check() + @Test("check consults both probes exactly once") + func checkReadsEachProbeOnce() { + // Each real probe is a TCC read on the permission-poll timer; re-reading one + // per call would double that traffic for a struct with two fields. + let micReads = Counter() + let axReads = Counter() + _ = PermissionsChecker.check( + micGranted: { + _ = micReads.next() + return true + }, + axTrusted: { + _ = axReads.next() + return true + }) + #expect(micReads.value == 1) + #expect(axReads.value == 1) } - @Test("forceAccessibilityActivity runs without prompting") - @MainActor - func forceAccessibilityActivityRuns() { - // Best-effort, side-effect-light (a read-only AX query against another - // process). This is the no-prompt half of the Accessibility flow — - // `openAccessibilitySettings` adds the trust prompt. - PermissionsChecker.forceAccessibilityActivity() + /// Smoke tests, deliberately assertion-free: both entry points read + /// process-global TCC state this host can't set, so all they can establish is + /// that the real probes run without throwing or prompting. The behaviour they + /// compose is covered above, against injected probes. + @Suite("PermissionsChecker smoke") + struct SmokeTests { + @Test("the production check() runs against the real probes without prompting") + func productionCheckRuns() { + _ = PermissionsChecker.check() + } + + @Test("forceAccessibilityActivity runs without prompting") + @MainActor + func forceAccessibilityActivityRuns() { + // Best-effort, side-effect-light (a read-only AX query against another + // process). This is the no-prompt half of the Accessibility flow — + // `openAccessibilitySettings` adds the trust prompt. + PermissionsChecker.forceAccessibilityActivity() + } } } -} +#endif diff --git a/Tests/BlurtEngineTests/PermissionsResetTests.swift b/Tests/BlurtEngineTests/PermissionsResetTests.swift index 4fd0eee0..72b96e3d 100644 --- a/Tests/BlurtEngineTests/PermissionsResetTests.swift +++ b/Tests/BlurtEngineTests/PermissionsResetTests.swift @@ -1,31 +1,33 @@ -import Testing +#if os(macOS) + import Testing -@testable import BlurtEngine + @testable import BlurtEngine -/// The service names are the whole contract with `tccutil`: it exits 0 for a -/// name it doesn't know as readily as for one it does, so a typo here is a reset -/// that silently clears nothing while reporting success. `reset` itself isn't -/// exercised — running it would clear the developer's own grants — so the names -/// and the roster are what a test can hold. -@Suite("PermissionsReset.Service") -struct PermissionsResetTests { - @Test("names the services tccutil knows") - func serviceNames() { - #expect(PermissionsReset.Service.accessibility.rawValue == "Accessibility") - #expect(PermissionsReset.Service.microphone.rawValue == "Microphone") - // Input Monitoring's internal name — what `tccutil` takes, and not what - // System Settings calls the row. - #expect(PermissionsReset.Service.inputMonitoring.rawValue == "ListenEvent") - } + /// The service names are the whole contract with `tccutil`: it exits 0 for a + /// name it doesn't know as readily as for one it does, so a typo here is a reset + /// that silently clears nothing while reporting success. `reset` itself isn't + /// exercised — running it would clear the developer's own grants — so the names + /// and the roster are what a test can hold. + @Suite("PermissionsReset.Service") + struct PermissionsResetTests { + @Test("names the services tccutil knows") + func serviceNames() { + #expect(PermissionsReset.Service.accessibility.rawValue == "Accessibility") + #expect(PermissionsReset.Service.microphone.rawValue == "Microphone") + // Input Monitoring's internal name — what `tccutil` takes, and not what + // System Settings calls the row. + #expect(PermissionsReset.Service.inputMonitoring.rawValue == "ListenEvent") + } - /// `resetAll` walks the hand-written `sweep` (Periphery can't follow a case - /// reached only through `allCases`), so this is what keeps that list honest: a - /// service added to the enum and forgotten there would otherwise survive every - /// "full" reset in silence. The three are also the ones - /// `scripts/reset-install.sh` lists — a fourth belongs in both places. - @Test("the sweep covers every service the enum names") - func sweepCoversEveryService() { - #expect(Set(PermissionsReset.sweep) == Set(PermissionsReset.Service.allCases)) - #expect(Set(PermissionsReset.sweep) == [.accessibility, .microphone, .inputMonitoring]) + /// `resetAll` walks the hand-written `sweep` (Periphery can't follow a case + /// reached only through `allCases`), so this is what keeps that list honest: a + /// service added to the enum and forgotten there would otherwise survive every + /// "full" reset in silence. The three are also the ones + /// `scripts/reset-install.sh` lists — a fourth belongs in both places. + @Test("the sweep covers every service the enum names") + func sweepCoversEveryService() { + #expect(Set(PermissionsReset.sweep) == Set(PermissionsReset.Service.allCases)) + #expect(Set(PermissionsReset.sweep) == [.accessibility, .microphone, .inputMonitoring]) + } } -} +#endif diff --git a/Tests/BlurtEngineTests/SigningIdentityMigrationTests.swift b/Tests/BlurtEngineTests/SigningIdentityMigrationTests.swift index b50fa30a..2b913895 100644 --- a/Tests/BlurtEngineTests/SigningIdentityMigrationTests.swift +++ b/Tests/BlurtEngineTests/SigningIdentityMigrationTests.swift @@ -1,111 +1,113 @@ -import Testing +#if os(macOS) + import Testing -@testable import BlurtEngine + @testable import BlurtEngine -@Suite struct SigningIdentityMigrationTests { - private typealias Migration = SigningIdentityMigration + @Suite struct SigningIdentityMigrationTests { + private typealias Migration = SigningIdentityMigration - // The identities `SigningIdentity.current()` produces, one per way Blurt is - // signed. The migration treats them alike — it only ever compares strings — but - // which pairs of them differ is the whole reason the cases below exist, so they - // are spelled out rather than left implied. - // - // A dev build's requirement is explicit and team-based (stamped by the - // `project.yml` post-build install, stable across cert rotation); a release - // carries codesign's default, which names the leaf certificate — so re-issuing - // that certificate produces `reissuedRelease` and orphans the grant. - private static let prefix = SigningIdentity.requirementPrefix - private static let anchor = "identifier \"dev.alex.blurt\" and anchor apple generic" - private static let devBuild = "\(prefix)\(anchor) and certificate leaf[subject.OU] = \"B2VQF7Q2QY\"" - private static let release = "\(prefix)\(anchor) and certificate leaf[subject.CN] = \"Dev ID 2024\"" - private static let reissuedRelease = - "\(prefix)\(anchor) and certificate leaf[subject.CN] = \"Dev ID 2029\"" + // The identities `SigningIdentity.current()` produces, one per way Blurt is + // signed. The migration treats them alike — it only ever compares strings — but + // which pairs of them differ is the whole reason the cases below exist, so they + // are spelled out rather than left implied. + // + // A dev build's requirement is explicit and team-based (stamped by the + // `project.yml` post-build install, stable across cert rotation); a release + // carries codesign's default, which names the leaf certificate — so re-issuing + // that certificate produces `reissuedRelease` and orphans the grant. + private static let prefix = SigningIdentity.requirementPrefix + private static let anchor = "identifier \"dev.alex.blurt\" and anchor apple generic" + private static let devBuild = "\(prefix)\(anchor) and certificate leaf[subject.OU] = \"B2VQF7Q2QY\"" + private static let release = "\(prefix)\(anchor) and certificate leaf[subject.CN] = \"Dev ID 2024\"" + private static let reissuedRelease = + "\(prefix)\(anchor) and certificate leaf[subject.CN] = \"Dev ID 2029\"" - // decide(): unsigned builds never act. - @Test func unsignedNeverActs() { - #expect(Migration.decide(lastIdentity: nil, currentIdentity: nil, isTrusted: false) == .noAction) - #expect(Migration.decide(lastIdentity: "OLD", currentIdentity: nil, isTrusted: true) == .noAction) - } + // decide(): unsigned builds never act. + @Test func unsignedNeverActs() { + #expect(Migration.decide(lastIdentity: nil, currentIdentity: nil, isTrusted: false) == .noAction) + #expect(Migration.decide(lastIdentity: "OLD", currentIdentity: nil, isTrusted: true) == .noAction) + } - // decide(): same identity as last launch is steady state, regardless of trust. - @Test func steadyStateIsNoAction() { - #expect(Migration.decide(lastIdentity: "NEW", currentIdentity: "NEW", isTrusted: false) == .noAction) - #expect(Migration.decide(lastIdentity: "NEW", currentIdentity: "NEW", isTrusted: true) == .noAction) - } + // decide(): same identity as last launch is steady state, regardless of trust. + @Test func steadyStateIsNoAction() { + #expect(Migration.decide(lastIdentity: "NEW", currentIdentity: "NEW", isTrusted: false) == .noAction) + #expect(Migration.decide(lastIdentity: "NEW", currentIdentity: "NEW", isTrusted: true) == .noAction) + } - // decide(): no marker (every build predating it) is treated as changed. - @Test func firstSeenUntrustedResets() { - let decision = Migration.decide(lastIdentity: nil, currentIdentity: "NEW", isTrusted: false) - #expect(decision == .resetThenRecord("NEW")) - } - @Test func firstSeenTrustedRecordsOnly() { - let decision = Migration.decide(lastIdentity: nil, currentIdentity: "NEW", isTrusted: true) - #expect(decision == .record("NEW")) - } + // decide(): no marker (every build predating it) is treated as changed. + @Test func firstSeenUntrustedResets() { + let decision = Migration.decide(lastIdentity: nil, currentIdentity: "NEW", isTrusted: false) + #expect(decision == .resetThenRecord("NEW")) + } + @Test func firstSeenTrustedRecordsOnly() { + let decision = Migration.decide(lastIdentity: nil, currentIdentity: "NEW", isTrusted: true) + #expect(decision == .record("NEW")) + } - // decide(): any identity change at all. - @Test func identityChangedUntrustedResets() { - let decision = Migration.decide(lastIdentity: "OLD", currentIdentity: "NEW", isTrusted: false) - #expect(decision == .resetThenRecord("NEW")) - } - @Test func identityChangedTrustedRecordsOnly() { - let decision = Migration.decide(lastIdentity: "OLD", currentIdentity: "NEW", isTrusted: true) - #expect(decision == .record("NEW")) - } + // decide(): any identity change at all. + @Test func identityChangedUntrustedResets() { + let decision = Migration.decide(lastIdentity: "OLD", currentIdentity: "NEW", isTrusted: false) + #expect(decision == .resetThenRecord("NEW")) + } + @Test func identityChangedTrustedRecordsOnly() { + let decision = Migration.decide(lastIdentity: "OLD", currentIdentity: "NEW", isTrusted: true) + #expect(decision == .record("NEW")) + } - // decide(): the case this exists for now that debug builds carry their own - // bundle id. Re-issuing the Developer ID certificate (RELEASE.md's rotation - // procedure) changes the leaf its default requirement names, so every installed - // user's grant is suddenly pinned to a requirement the update cannot satisfy — - // switched on in System Settings, denied by `AXIsProcessTrusted()`, with no way - // past the wizard. Nothing at signing time can pre-empt it: the requirement was - // handed to `tccd` before the rotation existed. - @Test func reissuedReleaseCertificateResets() { - let decision = Migration.decide( - lastIdentity: Self.release, currentIdentity: Self.reissuedRelease, isTrusted: false) - #expect(decision == .resetThenRecord(Self.reissuedRelease)) - } + // decide(): the case this exists for now that debug builds carry their own + // bundle id. Re-issuing the Developer ID certificate (RELEASE.md's rotation + // procedure) changes the leaf its default requirement names, so every installed + // user's grant is suddenly pinned to a requirement the update cannot satisfy — + // switched on in System Settings, denied by `AXIsProcessTrusted()`, with no way + // past the wizard. Nothing at signing time can pre-empt it: the requirement was + // handed to `tccd` before the rotation existed. + @Test func reissuedReleaseCertificateResets() { + let decision = Migration.decide( + lastIdentity: Self.release, currentIdentity: Self.reissuedRelease, isTrusted: false) + #expect(decision == .resetThenRecord(Self.reissuedRelease)) + } - // decide(): the flip side — the identity must be *stable* for the loop that - // rebuilds the same way over and over. Two `dev-build.sh` runs pin the same - // team-based requirement, so a working grant survives a rebuild; nothing here - // may reset on every launch. - @Test func rebuildingTheSameWayIsNoAction() { - #expect( - Migration.decide(lastIdentity: Self.devBuild, currentIdentity: Self.devBuild, isTrusted: true) - == .noAction) - #expect( - Migration.decide(lastIdentity: Self.release, currentIdentity: Self.release, isTrusted: false) - == .noAction) - } + // decide(): the flip side — the identity must be *stable* for the loop that + // rebuilds the same way over and over. Two `dev-build.sh` runs pin the same + // team-based requirement, so a working grant survives a rebuild; nothing here + // may reset on every launch. + @Test func rebuildingTheSameWayIsNoAction() { + #expect( + Migration.decide(lastIdentity: Self.devBuild, currentIdentity: Self.devBuild, isTrusted: true) + == .noAction) + #expect( + Migration.decide(lastIdentity: Self.release, currentIdentity: Self.release, isTrusted: false) + == .noAction) + } - // run(): steady state / unsigned persist nothing and never reset. - @Test func runNoActionPersistsNothingAndSkipsReset() { - let out = Migration.run(lastIdentity: "NEW", currentIdentity: "NEW", isTrusted: false) { - Issue.record("reset must not run for .noAction") - return true + // run(): steady state / unsigned persist nothing and never reset. + @Test func runNoActionPersistsNothingAndSkipsReset() { + let out = Migration.run(lastIdentity: "NEW", currentIdentity: "NEW", isTrusted: false) { + Issue.record("reset must not run for .noAction") + return true + } + #expect(out == nil) } - #expect(out == nil) - } - // run(): a trusted identity change records without resetting. - @Test func runRecordsWithoutResetting() { - var didReset = false - let out = Migration.run(lastIdentity: "OLD", currentIdentity: "NEW", isTrusted: true) { - didReset = true - return true + // run(): a trusted identity change records without resetting. + @Test func runRecordsWithoutResetting() { + var didReset = false + let out = Migration.run(lastIdentity: "OLD", currentIdentity: "NEW", isTrusted: true) { + didReset = true + return true + } + #expect(out == "NEW") + #expect(didReset == false) } - #expect(out == "NEW") - #expect(didReset == false) - } - // run(): an untrusted identity change resets, then persists only on success. - @Test func runResetsThenPersistsOnSuccess() { - let out = Migration.run(lastIdentity: "OLD", currentIdentity: "NEW", isTrusted: false) { true } - #expect(out == "NEW") - } - @Test func runDoesNotPersistWhenResetFails() { - let out = Migration.run(lastIdentity: "OLD", currentIdentity: "NEW", isTrusted: false) { false } - #expect(out == nil) + // run(): an untrusted identity change resets, then persists only on success. + @Test func runResetsThenPersistsOnSuccess() { + let out = Migration.run(lastIdentity: "OLD", currentIdentity: "NEW", isTrusted: false) { true } + #expect(out == "NEW") + } + @Test func runDoesNotPersistWhenResetFails() { + let out = Migration.run(lastIdentity: "OLD", currentIdentity: "NEW", isTrusted: false) { false } + #expect(out == nil) + } } -} +#endif diff --git a/Tests/BlurtEngineTests/SigningIdentityTests.swift b/Tests/BlurtEngineTests/SigningIdentityTests.swift index d93d58a5..f7b41bb3 100644 --- a/Tests/BlurtEngineTests/SigningIdentityTests.swift +++ b/Tests/BlurtEngineTests/SigningIdentityTests.swift @@ -1,59 +1,61 @@ -import Foundation -import Security -import Testing +#if os(macOS) + import Foundation + import Security + import Testing -@testable import BlurtEngine + @testable import BlurtEngine -@Suite("SigningIdentity") -struct SigningIdentityTests { + @Suite("SigningIdentity") + struct SigningIdentityTests { - // The value the migration compares must be a property of the *binary*, not of - // the moment it was read — an identity that varied per call would reset the - // Accessibility grant on every launch. - @Test("current() is stable across reads") - func currentIsStable() { - #expect(SigningIdentity.current() == SigningIdentity.current()) - } + // The value the migration compares must be a property of the *binary*, not of + // the moment it was read — an identity that varied per call would reset the + // Accessibility grant on every launch. + @Test("current() is stable across reads") + func currentIsStable() { + #expect(SigningIdentity.current() == SigningIdentity.current()) + } - // What the test host reports depends on how it was signed (ad-hoc under - // `swift test`, team-signed under Xcode), so this asserts the shape of whatever - // this host gives rather than a literal — including the shape of "nothing", - // which is the answer for the ad-hoc host and the one the migration reads as - // "no action". - @Test("current() is either absent or a namespaced requirement") - func currentIsNamespacedWhenPresent() { - guard let identity = SigningIdentity.current() else { - return // ad-hoc or unsigned host: refusing to answer is the contract + // What the test host reports depends on how it was signed (ad-hoc under + // `swift test`, team-signed under Xcode), so this asserts the shape of whatever + // this host gives rather than a literal — including the shape of "nothing", + // which is the answer for the ad-hoc host and the one the migration reads as + // "no action". + @Test("current() is either absent or a namespaced requirement") + func currentIsNamespacedWhenPresent() { + guard let identity = SigningIdentity.current() else { + return // ad-hoc or unsigned host: refusing to answer is the contract + } + #expect(identity.hasPrefix(SigningIdentity.requirementPrefix)) + // Never a bare Team ID (10 alphanumerics, no colon): the marker recorded by + // builds that predate this shape has to stay distinguishable from one + // recorded now, or an upgrade would read as steady state. + #expect(identity.contains(":")) + #expect(identity.count > SigningIdentity.requirementPrefix.count) } - #expect(identity.hasPrefix(SigningIdentity.requirementPrefix)) - // Never a bare Team ID (10 alphanumerics, no colon): the marker recorded by - // builds that predate this shape has to stay distinguishable from one - // recorded now, or an upgrade would read as steady state. - #expect(identity.contains(":")) - #expect(identity.count > SigningIdentity.requirementPrefix.count) - } - // The `Security` handshake is the one part of this file that isn't pure logic, - // and the host's own signature can't exercise it (ad-hoc under `swift test`). - // `/bin/ls` can: it is present on every Mac and Apple-signed, so its designated - // requirement is both readable and known. If this returns nil the migration - // silently degrades to "never act" — a grant orphaned by a re-issued release - // certificate would then strand every user with no way past the wizard. - @Test("a designated requirement can actually be read") - func designatedRequirementIsReadable() throws { - var code: SecStaticCode? - let status = SecStaticCodeCreateWithPath( - URL(fileURLWithPath: "/bin/ls") as CFURL, SecCSFlags(), &code) - #expect(status == errSecSuccess) - let staticCode = try #require(code) - let requirement = try #require(SigningIdentity.designatedRequirement(of: staticCode)) - #expect(requirement.contains("anchor apple")) - } + // The `Security` handshake is the one part of this file that isn't pure logic, + // and the host's own signature can't exercise it (ad-hoc under `swift test`). + // `/bin/ls` can: it is present on every Mac and Apple-signed, so its designated + // requirement is both readable and known. If this returns nil the migration + // silently degrades to "never act" — a grant orphaned by a re-issued release + // certificate would then strand every user with no way past the wizard. + @Test("a designated requirement can actually be read") + func designatedRequirementIsReadable() throws { + var code: SecStaticCode? + let status = SecStaticCodeCreateWithPath( + URL(fileURLWithPath: "/bin/ls") as CFURL, SecCSFlags(), &code) + #expect(status == errSecSuccess) + let staticCode = try #require(code) + let requirement = try #require(SigningIdentity.designatedRequirement(of: staticCode)) + #expect(requirement.contains("anchor apple")) + } - // The process's own code object has to resolve, or `current()` can only ever - // answer nil and the migration is dead weight in every build. - @Test("this process's code object resolves") - func selfStaticCodeResolves() throws { - _ = try #require(SigningIdentity.staticCodeForSelf()) + // The process's own code object has to resolve, or `current()` can only ever + // answer nil and the migration is dead weight in every build. + @Test("this process's code object resolves") + func selfStaticCodeResolves() throws { + _ = try #require(SigningIdentity.staticCodeForSelf()) + } } -} +#endif diff --git a/Tests/BlurtEngineTests/Stubs/DictationSessionTestSupport.swift b/Tests/BlurtEngineTests/Stubs/DictationSessionTestSupport.swift index aee4b9da..71d4407e 100644 --- a/Tests/BlurtEngineTests/Stubs/DictationSessionTestSupport.swift +++ b/Tests/BlurtEngineTests/Stubs/DictationSessionTestSupport.swift @@ -23,7 +23,7 @@ struct SessionFixture { func makeSession( mode: StubTranscriber.Mode = .transcript("Hello world."), maxRecordingSeconds: Double = SyncSTTLimits.autoReleaseSeconds, - field: FocusCapture.FocusedFieldContext = .empty, + field: FocusedFieldContext = .empty, frontmost: CapturedFocus? = nil, keyTerms: [String] = [], textShortcuts: [TextShortcut] = [], @@ -52,7 +52,7 @@ func makeSession( /// the press-time context, so a test can assert on it instead of counting that /// *something* host-dependent arrived. func testSeams( - field: FocusCapture.FocusedFieldContext = .empty, + field: FocusedFieldContext = .empty, frontmost: CapturedFocus? = nil, log: RecordedLog = RecordedLog() ) -> DictationSession.Seams { @@ -71,7 +71,7 @@ func testSeams( /// did, and both copies quietly dropped the transcript recording `testSeams` /// wires up, so a later assertion on `log.transcripts` would have seen nothing. func hungFieldSeams( - field: FocusCapture.FocusedFieldContext = .empty, + field: FocusedFieldContext = .empty, log: RecordedLog = RecordedLog() ) -> (seams: DictationSession.Seams, release: DispatchSemaphore) { let hung = DispatchSemaphore(value: 0) diff --git a/Tests/BlurtEngineTests/Stubs/FakeClipboard.swift b/Tests/BlurtEngineTests/Stubs/FakeClipboard.swift index de0b9617..563feffd 100644 --- a/Tests/BlurtEngineTests/Stubs/FakeClipboard.swift +++ b/Tests/BlurtEngineTests/Stubs/FakeClipboard.swift @@ -1,58 +1,60 @@ -import Foundation -import Synchronization +#if os(macOS) + import Foundation + import Synchronization -@testable import BlurtEngine + @testable import BlurtEngine -/// In-memory `ClipboardAccess` for tests: holds a single plain string without -/// touching the system pasteboard, and lets a test simulate another process -/// overwriting the clipboard via `externalWrite`. Shared by the KeyInjector -/// insert and delete suites. A `Mutex` guards the state, making the `Sendable` -/// conformance compiler-checked. -/// -/// The narrowed `ClipboardAccess` seam keeps the pasteboard's change-count and -/// restore-if-unchanged policy inside `SystemClipboard`, so this fake only has -/// to hold a string and a write counter — it never re-derives that policy. -final class FakeClipboard: ClipboardAccess, Sendable { - private struct State { - var text: String? - var count = 0 - } - private let state: Mutex - - init(string: String?) { - state = Mutex(State(text: string)) - } + /// In-memory `ClipboardAccess` for tests: holds a single plain string without + /// touching the system pasteboard, and lets a test simulate another process + /// overwriting the clipboard via `externalWrite`. Shared by the KeyInjector + /// insert and delete suites. A `Mutex` guards the state, making the `Sendable` + /// conformance compiler-checked. + /// + /// The narrowed `ClipboardAccess` seam keeps the pasteboard's change-count and + /// restore-if-unchanged policy inside `SystemClipboard`, so this fake only has + /// to hold a string and a write counter — it never re-derives that policy. + final class FakeClipboard: ClipboardAccess, Sendable { + private struct State { + var text: String? + var count = 0 + } + private let state: Mutex - func write(_ text: String) { - state.withLock { - $0.text = text - $0.count += 1 + init(string: String?) { + state = Mutex(State(text: string)) } - } - func writeAndPrepareRestore(_ text: String) -> @Sendable () -> Void { - let (saved, mark) = state.withLock { s -> (String?, Int) in - let previous = s.text - s.text = text - s.count += 1 - return (previous, s.count) + func write(_ text: String) { + state.withLock { + $0.text = text + $0.count += 1 + } } - return { [self] in - state.withLock { s in - // Another writer bumped the count during the settle window — leave their - // newer contents alone (mirrors SystemClipboard's changeCount guard). - guard s.count == mark else { return } - s.text = saved + + func writeAndPrepareRestore(_ text: String) -> @Sendable () -> Void { + let (saved, mark) = state.withLock { s -> (String?, Int) in + let previous = s.text + s.text = text s.count += 1 + return (previous, s.count) + } + return { [self] in + state.withLock { s in + // Another writer bumped the count during the settle window — leave their + // newer contents alone (mirrors SystemClipboard's changeCount guard). + guard s.count == mark else { return } + s.text = saved + s.count += 1 + } } } - } - /// Simulate another process writing the clipboard mid-paste. - func externalWrite(_ text: String) { write(text) } + /// Simulate another process writing the clipboard mid-paste. + func externalWrite(_ text: String) { write(text) } - /// Current plain-string content, for assertions. - var string: String? { - state.withLock { $0.text } + /// Current plain-string content, for assertions. + var string: String? { + state.withLock { $0.text } + } } -} +#endif diff --git a/Tests/BlurtEngineTests/Stubs/InjectorTestSupport.swift b/Tests/BlurtEngineTests/Stubs/InjectorTestSupport.swift index e61cf20c..abfa47c2 100644 --- a/Tests/BlurtEngineTests/Stubs/InjectorTestSupport.swift +++ b/Tests/BlurtEngineTests/Stubs/InjectorTestSupport.swift @@ -1,120 +1,129 @@ -import AppKit -import Synchronization -import Testing +#if os(macOS) + import AppKit + import Synchronization + import Testing -@testable import BlurtEngine + @testable import BlurtEngine -/// Shared fixtures for the `KeyInjector.insert` suites (the main insert suite -/// and the fallback/cancel suite live in separate files to stay within the -/// lint file-length budget, but drive the injector the same way). -/// -/// The boxes are classes over `Mutex` (not actors) because they're poked from -/// synchronous `@Sendable` seams like `postPaste`; the `Mutex` makes each -/// `Sendable` conformance compiler-checked instead of `@unchecked`-asserted. + /// Shared fixtures for the `KeyInjector.insert` suites (the main insert suite + /// and the fallback/cancel suite live in separate files to stay within the + /// lint file-length budget, but drive the injector the same way). + /// + /// The boxes are classes over `Mutex` (not actors) because they're poked from + /// synchronous `@Sendable` seams like `postPaste`; the `Mutex` makes each + /// `Sendable` conformance compiler-checked instead of `@unchecked`-asserted. -/// Some live application to stand in as the captured paste target. -func liveTargetApp() throws -> NSRunningApplication { - try #require( - NSWorkspace.shared.runningApplications.first { - $0.processIdentifier > 0 && !$0.isTerminated - }) -} + /// Some live application to stand in as the captured paste target. + func liveTargetApp() throws -> NSRunningApplication { + try #require( + NSWorkspace.shared.runningApplications.first { + $0.processIdentifier > 0 && !$0.isTerminated + }) + } -/// A `KeyInjector` wired to an in-memory clipboard, with `postPaste` recording -/// each pasted string (captured after `setString`, before the deferred -/// restore) into the returned box — for the end-to-end case that pins `insert` -/// threading its resolution through to the paste. (The continuity rules -/// themselves are cases of `resolveInsert` and need no injector at all.) -/// `secondsSinceUserInput` stands in for the HID idle counters; the default -/// reports no keyboard or mouse input ever. -func makeRecordingInjector( - secondsSinceUserInput: @escaping @Sendable () -> TimeInterval = { .infinity } -) -> (injector: KeyInjector, pasted: StringListBox) { - let clip = FakeClipboard(string: nil) - let pasted = StringListBox() - let injector = KeyInjector( - pasteSettleDuration: .zero, - postPaste: { - pasted.append(clip.string) - return true - }, - // Stub the activation. Omitting this defaulted to `KeyInjector.activate`, a - // REAL `NSRunningApplication.activate()` — so this test yanked the - // developer's foreground app, and failed spuriously whenever - // `liveTargetApp()`'s unordered pick landed on a background-only process - // whose activate() returns false (the injector then throws `.targetAppLost` - // for reasons unrelated to separator logic). It needs a stable non-nil app - // identity, not activation. - activateTarget: { _ in true }, - secondsSinceUserInput: secondsSinceUserInput, - clipboard: clip) - return (injector, pasted) -} + /// That application as the `CapturedFocus` `KeyInjector.setTarget` takes, so the + /// insert suites pin a target the way `DictationSession` does. + func liveTargetFocus() throws -> CapturedFocus { + let app = try liveTargetApp() + return CapturedFocus(pid: app.processIdentifier, processName: app.localizedName) + } -/// One-shot async gate: `wait()` suspends until `open()` is called. Tolerates -/// `open()` racing ahead of `wait()` (the waiter then returns immediately), and -/// any number of concurrent waiters — `Gate`, which is built from a pair of -/// these, needs that for the stubs whose blocked method is called twice by the -/// regression under test. -/// -/// `open()` is synchronous (a `Mutex`-guarded class, not an actor) because the -/// seams that trip these gates are synchronous `@Sendable` closures like -/// `KeyInjector.postPaste`. -final class AsyncGate: Sendable { - private struct State { - var waiters: [CheckedContinuation] = [] - var opened = false + /// A `KeyInjector` wired to an in-memory clipboard, with `postPaste` recording + /// each pasted string (captured after `setString`, before the deferred + /// restore) into the returned box — for the end-to-end case that pins `insert` + /// threading its resolution through to the paste. (The continuity rules + /// themselves are cases of `resolveInsert` and need no injector at all.) + /// `secondsSinceUserInput` stands in for the HID idle counters; the default + /// reports no keyboard or mouse input ever. + func makeRecordingInjector( + secondsSinceUserInput: @escaping @Sendable () -> TimeInterval = { .infinity } + ) -> (injector: KeyInjector, pasted: StringListBox) { + let clip = FakeClipboard(string: nil) + let pasted = StringListBox() + let injector = KeyInjector( + pasteSettleDuration: .zero, + postPaste: { + pasted.append(clip.string) + return true + }, + // Stub the activation. Omitting this defaulted to `KeyInjector.activate`, a + // REAL `NSRunningApplication.activate()` — so this test yanked the + // developer's foreground app, and failed spuriously whenever + // `liveTargetApp()`'s unordered pick landed on a background-only process + // whose activate() returns false (the injector then throws `.targetAppLost` + // for reasons unrelated to separator logic). It needs a stable non-nil app + // identity, not activation. + activateTarget: { _ in true }, + secondsSinceUserInput: secondsSinceUserInput, + clipboard: clip) + return (injector, pasted) } - private let state = Mutex(State()) - func wait() async { - await withCheckedContinuation { cont in - let openedAlready = state.withLock { s -> Bool in - if s.opened { return true } - s.waiters.append(cont) - return false + /// One-shot async gate: `wait()` suspends until `open()` is called. Tolerates + /// `open()` racing ahead of `wait()` (the waiter then returns immediately), and + /// any number of concurrent waiters — `Gate`, which is built from a pair of + /// these, needs that for the stubs whose blocked method is called twice by the + /// regression under test. + /// + /// `open()` is synchronous (a `Mutex`-guarded class, not an actor) because the + /// seams that trip these gates are synchronous `@Sendable` closures like + /// `KeyInjector.postPaste`. + final class AsyncGate: Sendable { + private struct State { + var waiters: [CheckedContinuation] = [] + var opened = false + } + private let state = Mutex(State()) + + func wait() async { + await withCheckedContinuation { cont in + let openedAlready = state.withLock { s -> Bool in + if s.opened { return true } + s.waiters.append(cont) + return false + } + if openedAlready { cont.resume() } } - if openedAlready { cont.resume() } } - } - func open() { - let waiters = state.withLock { s -> [CheckedContinuation] in - s.opened = true - let pending = s.waiters - s.waiters.removeAll() - return pending + func open() { + let waiters = state.withLock { s -> [CheckedContinuation] in + s.opened = true + let pending = s.waiters + s.waiters.removeAll() + return pending + } + for waiter in waiters { waiter.resume() } } - for waiter in waiters { waiter.resume() } } -} -/// Thread-safe ordered list of strings recorded inside a `@Sendable` closure, -/// for asserting the sequence of texts a test observed being pasted. Not a -/// `ValueBox<[String]>`: the atomic append is the point — appending through a -/// get-then-set property would race two recorders against each other. -final class StringListBox: Sendable { - private let items = Mutex<[String]>([]) - func append(_ value: String?) { - items.withLock { $0.append(value ?? "") } - } - var values: [String] { - items.withLock { $0 } + /// Thread-safe ordered list of strings recorded inside a `@Sendable` closure, + /// for asserting the sequence of texts a test observed being pasted. Not a + /// `ValueBox<[String]>`: the atomic append is the point — appending through a + /// get-then-set property would race two recorders against each other. + final class StringListBox: Sendable { + private let items = Mutex<[String]>([]) + func append(_ value: String?) { + items.withLock { $0.append(value ?? "") } + } + var values: [String] { + items.withLock { $0 } + } } -} -/// Thread-safe single-value cell for capturing an arbitrary value written inside -/// a `@Sendable` closure and reading it back after the awaited call returns — -/// also how a closure holds the handle of the very task executing it, so it can -/// cancel it (a `Task` is `Sendable`, so that needs no separate box). -final class ValueBox: Sendable { - private let stored: Mutex - init(_ initial: T) { stored = Mutex(initial) } - /// One settable property rather than a `value` getter beside a `set(_:)` — the - /// `Mutex` is what makes the class `Sendable`, so both accessors can go through - /// it and callers read as ordinary assignment. - var value: T { - get { stored.withLock { $0 } } - set { stored.withLock { $0 = newValue } } + /// Thread-safe single-value cell for capturing an arbitrary value written inside + /// a `@Sendable` closure and reading it back after the awaited call returns — + /// also how a closure holds the handle of the very task executing it, so it can + /// cancel it (a `Task` is `Sendable`, so that needs no separate box). + final class ValueBox: Sendable { + private let stored: Mutex + init(_ initial: T) { stored = Mutex(initial) } + /// One settable property rather than a `value` getter beside a `set(_:)` — the + /// `Mutex` is what makes the class `Sendable`, so both accessors can go through + /// it and callers read as ordinary assignment. + var value: T { + get { stored.withLock { $0 } } + set { stored.withLock { $0 = newValue } } + } } -} +#endif diff --git a/Tests/BlurtEngineTests/Stubs/StubInjector.swift b/Tests/BlurtEngineTests/Stubs/StubInjector.swift index 0c5f2ef6..eddb56cf 100644 --- a/Tests/BlurtEngineTests/Stubs/StubInjector.swift +++ b/Tests/BlurtEngineTests/Stubs/StubInjector.swift @@ -1,4 +1,3 @@ -import AppKit import Foundation @testable import BlurtEngine @@ -20,6 +19,6 @@ actor StubInjector: InjectorProtocol { insertedPrior.append(priorText) insertedWindowTitles.append(windowTitle) } - func setTargetApp(_ app: NSRunningApplication?) async {} + func setTarget(_ focus: CapturedFocus?) async {} func setError(_ error: (any Error & Sendable)?) { self.error = error } } diff --git a/Tests/BlurtEngineTests/SystemClipboardTests.swift b/Tests/BlurtEngineTests/SystemClipboardTests.swift index 1564d87a..fa46eee3 100644 --- a/Tests/BlurtEngineTests/SystemClipboardTests.swift +++ b/Tests/BlurtEngineTests/SystemClipboardTests.swift @@ -1,211 +1,213 @@ -import AppKit -import Foundation -import Testing - -@testable import BlurtEngine - -/// Covers `SystemClipboard`, the real-`NSPasteboard` implementation of the -/// `ClipboardAccess` seam (`KeyInjector` uses a fake in its own tests). These -/// run synchronously with no settle window, so they don't race other processes' -/// clipboard activity; `.serialized` because they touch `NSPasteboard.general`. -@Suite("SystemClipboard", .serialized) -struct SystemClipboardTests { - - /// Snapshot/restore the user's clipboard around a test body so the suite leaves - /// the real pasteboard as it found it. - /// - /// Uses the full `snapshot()`/`restore()` primitives rather than just the string - /// flavor: snapshotting only `.string` and then clearing unconditionally meant - /// running these tests wiped a developer's real clipboard whenever it held an - /// image, a file, or styled text. - private func withClipboardRestored(_ body: () throws -> Void) rethrows { - let clip = SystemClipboard() - let saved = clip.snapshot() - defer { - if let saved { clip.restore(saved) } +#if os(macOS) + import AppKit + import Foundation + import Testing + + @testable import BlurtEngine + + /// Covers `SystemClipboard`, the real-`NSPasteboard` implementation of the + /// `ClipboardAccess` seam (`KeyInjector` uses a fake in its own tests). These + /// run synchronously with no settle window, so they don't race other processes' + /// clipboard activity; `.serialized` because they touch `NSPasteboard.general`. + @Suite("SystemClipboard", .serialized) + struct SystemClipboardTests { + + /// Snapshot/restore the user's clipboard around a test body so the suite leaves + /// the real pasteboard as it found it. + /// + /// Uses the full `snapshot()`/`restore()` primitives rather than just the string + /// flavor: snapshotting only `.string` and then clearing unconditionally meant + /// running these tests wiped a developer's real clipboard whenever it held an + /// image, a file, or styled text. + private func withClipboardRestored(_ body: () throws -> Void) rethrows { + let clip = SystemClipboard() + let saved = clip.snapshot() + defer { + if let saved { clip.restore(saved) } + } + try body() } - try body() - } - @Test("setString writes the string and advances changeCount") - func setStringWrites() { - withClipboardRestored { - let pb = NSPasteboard.general - let clip = SystemClipboard() + @Test("setString writes the string and advances changeCount") + func setStringWrites() { + withClipboardRestored { + let pb = NSPasteboard.general + let clip = SystemClipboard() - let before = clip.changeCount - clip.setString("written") + let before = clip.changeCount + clip.setString("written") - #expect(pb.string(forType: .string) == "written") - #expect(clip.changeCount > before) + #expect(pb.string(forType: .string) == "written") + #expect(clip.changeCount > before) + } } - } - @Test("snapshot captures contents that restore brings back") - func snapshotRestoreRoundTrips() throws { - try withClipboardRestored { - let pb = NSPasteboard.general - let clip = SystemClipboard() + @Test("snapshot captures contents that restore brings back") + func snapshotRestoreRoundTrips() throws { + try withClipboardRestored { + let pb = NSPasteboard.general + let clip = SystemClipboard() - pb.clearContents() - pb.setString("original", forType: .string) - let snapshot = try #require(clip.snapshot()) + pb.clearContents() + pb.setString("original", forType: .string) + let snapshot = try #require(clip.snapshot()) - clip.setString("overwritten") - #expect(pb.string(forType: .string) == "overwritten") + clip.setString("overwritten") + #expect(pb.string(forType: .string) == "overwritten") - clip.restore(snapshot) - #expect(pb.string(forType: .string) == "original") + clip.restore(snapshot) + #expect(pb.string(forType: .string) == "original") + } } - } - - @Test("snapshot/restore preserves every representation of multi-type, multi-item contents") - func multiTypeSnapshotRoundTrips() throws { - // The reason `SendablePasteboardItem` keys data by pasteboard *type*: a copy - // of styled text carries several representations (plain string + RTF), and - // the restore must bring all of them back — a string-only round trip would - // silently downgrade the user's clipboard to plain text. - try withClipboardRestored { - let pb = NSPasteboard.general - let clip = SystemClipboard() - let styled = NSPasteboardItem() - styled.setString("styled", forType: .string) - styled.setData(Data("{\\rtf1 styled}".utf8), forType: .rtf) - let plain = NSPasteboardItem() - plain.setString("second item", forType: .string) - pb.clearContents() - pb.writeObjects([styled, plain]) - let snapshot = try #require(clip.snapshot()) - - clip.setString("overwritten") - clip.restore(snapshot) - - let restored = pb.pasteboardItems ?? [] - #expect(restored.count == 2) - #expect(restored.first?.string(forType: .string) == "styled") - #expect(restored.first?.data(forType: .rtf) == Data("{\\rtf1 styled}".utf8)) - #expect(restored.last?.string(forType: .string) == "second item") + @Test("snapshot/restore preserves every representation of multi-type, multi-item contents") + func multiTypeSnapshotRoundTrips() throws { + // The reason `SendablePasteboardItem` keys data by pasteboard *type*: a copy + // of styled text carries several representations (plain string + RTF), and + // the restore must bring all of them back — a string-only round trip would + // silently downgrade the user's clipboard to plain text. + try withClipboardRestored { + let pb = NSPasteboard.general + let clip = SystemClipboard() + + let styled = NSPasteboardItem() + styled.setString("styled", forType: .string) + styled.setData(Data("{\\rtf1 styled}".utf8), forType: .rtf) + let plain = NSPasteboardItem() + plain.setString("second item", forType: .string) + pb.clearContents() + pb.writeObjects([styled, plain]) + let snapshot = try #require(clip.snapshot()) + + clip.setString("overwritten") + clip.restore(snapshot) + + let restored = pb.pasteboardItems ?? [] + #expect(restored.count == 2) + #expect(restored.first?.string(forType: .string) == "styled") + #expect(restored.first?.data(forType: .rtf) == Data("{\\rtf1 styled}".utf8)) + #expect(restored.last?.string(forType: .string) == "second item") + } } - } - - @Test("a degraded snapshot never clears the clipboard it cannot replace") - func degradedSnapshotDoesNotDestroy() { - // The failure this guards: an item that exists but whose representations can't - // be materialized (a promise the owning app declines) used to produce an - // all-empty snapshot, and `restore` cleared first and wrote nothing — so the - // user's clipboard came back EMPTY instead of unchanged. - withClipboardRestored { - let pb = NSPasteboard.general - let clip = SystemClipboard() - - // An item with a declared type but no retrievable data models the promise. - let degraded = PasteboardSnapshot( - items: [SendablePasteboardItem(dataMap: [:])], plainText: nil) - clip.setString("transcript") - clip.restore(degraded) - - // Left alone rather than emptied — the transcript is still recoverable. - #expect(pb.string(forType: .string) == "transcript") + @Test("a degraded snapshot never clears the clipboard it cannot replace") + func degradedSnapshotDoesNotDestroy() { + // The failure this guards: an item that exists but whose representations can't + // be materialized (a promise the owning app declines) used to produce an + // all-empty snapshot, and `restore` cleared first and wrote nothing — so the + // user's clipboard came back EMPTY instead of unchanged. + withClipboardRestored { + let pb = NSPasteboard.general + let clip = SystemClipboard() + + // An item with a declared type but no retrievable data models the promise. + let degraded = PasteboardSnapshot( + items: [SendablePasteboardItem(dataMap: [:])], plainText: nil) + + clip.setString("transcript") + clip.restore(degraded) + + // Left alone rather than emptied — the transcript is still recoverable. + #expect(pb.string(forType: .string) == "transcript") + } } - } - @Test("a partly-readable snapshot falls back to the plain-text floor") - func degradedSnapshotUsesTextFloor() { - withClipboardRestored { - let pb = NSPasteboard.general - let clip = SystemClipboard() + @Test("a partly-readable snapshot falls back to the plain-text floor") + func degradedSnapshotUsesTextFloor() { + withClipboardRestored { + let pb = NSPasteboard.general + let clip = SystemClipboard() - let degraded = PasteboardSnapshot( - items: [SendablePasteboardItem(dataMap: [:])], plainText: "original text") + let degraded = PasteboardSnapshot( + items: [SendablePasteboardItem(dataMap: [:])], plainText: "original text") - clip.setString("transcript") - clip.restore(degraded) + clip.setString("transcript") + clip.restore(degraded) - // Not byte-faithful (the richer flavors are unrecoverable), but the user's - // text survives instead of their clipboard being emptied. - #expect(pb.string(forType: .string) == "original text") + // Not byte-faithful (the richer flavors are unrecoverable), but the user's + // text survives instead of their clipboard being emptied. + #expect(pb.string(forType: .string) == "original text") + } } - } - @Test("writeAndPrepareRestore writes the text and its restore puts the saved contents back") - func writeAndPrepareRestoreRoundTrips() { - // The entry point `KeyInjector` actually pastes through, and the one primitive - // in this type that no test reached: the injector suites drive `FakeClipboard`, - // which holds a string and a counter and deliberately does *not* re-derive the - // save-write-restore policy — the whole point of narrowing the - // `ClipboardAccess` seam. So the policy only exists here, and only here can it - // be checked. - withClipboardRestored { - let pb = NSPasteboard.general - let clip = SystemClipboard() - - pb.clearContents() - pb.setString("original", forType: .string) - - let restore = clip.writeAndPrepareRestore("transcript") - // The transcript is on the clipboard for the target app's ⌘V to read. - #expect(pb.string(forType: .string) == "transcript") - - restore() - #expect(pb.string(forType: .string) == "original") + @Test("writeAndPrepareRestore writes the text and its restore puts the saved contents back") + func writeAndPrepareRestoreRoundTrips() { + // The entry point `KeyInjector` actually pastes through, and the one primitive + // in this type that no test reached: the injector suites drive `FakeClipboard`, + // which holds a string and a counter and deliberately does *not* re-derive the + // save-write-restore policy — the whole point of narrowing the + // `ClipboardAccess` seam. So the policy only exists here, and only here can it + // be checked. + withClipboardRestored { + let pb = NSPasteboard.general + let clip = SystemClipboard() + + pb.clearContents() + pb.setString("original", forType: .string) + + let restore = clip.writeAndPrepareRestore("transcript") + // The transcript is on the clipboard for the target app's ⌘V to read. + #expect(pb.string(forType: .string) == "transcript") + + restore() + #expect(pb.string(forType: .string) == "original") + } } - } - - @Test("a clipboard changed after our paste is left alone rather than restored over") - func restoreSkipsAfterAnExternalWrite() { - // The change-count guard. The restore is deferred by `pasteSettleDuration`, and - // during that window the user may copy something new — restoring then would - // silently destroy what they just copied, which is a worse outcome than leaving - // the transcript behind. So a pasteboard that has moved on since our write is - // not touched. - withClipboardRestored { - let pb = NSPasteboard.general - let clip = SystemClipboard() - - pb.clearContents() - pb.setString("original", forType: .string) - let restore = clip.writeAndPrepareRestore("transcript") - // Someone else copies inside the settle window. - pb.clearContents() - pb.setString("user copied this", forType: .string) - - restore() - #expect(pb.string(forType: .string) == "user copied this") + @Test("a clipboard changed after our paste is left alone rather than restored over") + func restoreSkipsAfterAnExternalWrite() { + // The change-count guard. The restore is deferred by `pasteSettleDuration`, and + // during that window the user may copy something new — restoring then would + // silently destroy what they just copied, which is a worse outcome than leaving + // the transcript behind. So a pasteboard that has moved on since our write is + // not touched. + withClipboardRestored { + let pb = NSPasteboard.general + let clip = SystemClipboard() + + pb.clearContents() + pb.setString("original", forType: .string) + let restore = clip.writeAndPrepareRestore("transcript") + + // Someone else copies inside the settle window. + pb.clearContents() + pb.setString("user copied this", forType: .string) + + restore() + #expect(pb.string(forType: .string) == "user copied this") + } } - } - - @Test("write overwrites the clipboard with just the text") - func writeOverwrites() { - withClipboardRestored { - let pb = NSPasteboard.general - let clip = SystemClipboard() - pb.clearContents() - pb.setString("previous", forType: .string) - // The `ClipboardAccess` half the degraded paste paths use: no restore is - // prepared, because the transcript is meant to *stay* on the clipboard for - // the user to paste by hand. So the previous contents must be gone. - clip.write("transcript") - - #expect(pb.string(forType: .string) == "transcript") + @Test("write overwrites the clipboard with just the text") + func writeOverwrites() { + withClipboardRestored { + let pb = NSPasteboard.general + let clip = SystemClipboard() + + pb.clearContents() + pb.setString("previous", forType: .string) + // The `ClipboardAccess` half the degraded paste paths use: no restore is + // prepared, because the transcript is meant to *stay* on the clipboard for + // the user to paste by hand. So the previous contents must be gone. + clip.write("transcript") + + #expect(pb.string(forType: .string) == "transcript") + } } - } - @Test("restore of an empty snapshot leaves the cleared pasteboard empty") - func restoreEmptyIsNoOp() throws { - try withClipboardRestored { - let pb = NSPasteboard.general - let clip = SystemClipboard() + @Test("restore of an empty snapshot leaves the cleared pasteboard empty") + func restoreEmptyIsNoOp() throws { + try withClipboardRestored { + let pb = NSPasteboard.general + let clip = SystemClipboard() - pb.clearContents() // an empty pasteboard has no items to snapshot - let empty = try #require(clip.snapshot()) - clip.setString("temp") - clip.restore(empty) + pb.clearContents() // an empty pasteboard has no items to snapshot + let empty = try #require(clip.snapshot()) + clip.setString("temp") + clip.restore(empty) - #expect(pb.string(forType: .string) == nil) + #expect(pb.string(forType: .string) == nil) + } } } -} +#endif diff --git a/Tests/BlurtEngineTests/TextShortcutTests.swift b/Tests/BlurtEngineTests/TextShortcutTests.swift index 6d3c09f9..53787c53 100644 --- a/Tests/BlurtEngineTests/TextShortcutTests.swift +++ b/Tests/BlurtEngineTests/TextShortcutTests.swift @@ -203,7 +203,7 @@ struct DictationSessionTextShortcutTests { @Test("the text before the caret goes out with expansions put back to triggers") func priorTextRedacted() async { let fixture = makeSession( - field: FocusCapture.FocusedFieldContext( + field: FocusedFieldContext( priorText: "Mail me@example.com.", selectedText: nil, windowTitle: nil, fieldLabel: nil), textShortcuts: [TextShortcut(trigger: "personal email", expansion: "me@example.com")]) diff --git a/scripts/apple-geometry.sh b/scripts/apple-geometry.sh new file mode 100755 index 00000000..59740bb4 --- /dev/null +++ b/scripts/apple-geometry.sh @@ -0,0 +1,171 @@ +#!/bin/bash +# Measure the iPhone's own keyboard on the simulator and write what it found: +# the BlurtiOSProbe UI test launches the app on `-BlurtProbeField`, reads every +# key's frame off the system keyboard (both faces), and this turns the +# attachment into App/BlurtiOS/Design/apple-geometry.json — the table the +# keyboard's `key/*`, `row/gap` and `margin/*` tokens are pinned to +# (AppleGeometryTests) and the crop scripts/design-diff.swift measures the +# corner radius from. +# +# scripts/apple-geometry.sh # iPhone 18 Pro, writes the JSON and prints the table +# BLURT_SIM_DEVICE="iPhone 17" scripts/apple-geometry.sh --out /tmp/geometry.json +# +# Needs the on-screen keyboard: in Device Hub / Simulator, "Always simulate +# hardware keyboard" must be off, or no keyboard ever appears and the test fails. +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)" +cd "$REPO_ROOT" +# shellcheck source=scripts/ios-lib.sh +source "$REPO_ROOT/scripts/ios-lib.sh" +DERIVED="$IOS_DERIVED" +OUT="$REPO_ROOT/App/BlurtiOS/Design/apple-geometry.json" +WORK="$REPO_ROOT/.build/design/probe" +while [ $# -gt 0 ]; do + case "$1" in + --out) + OUT="${2:?}" + shift 2 + ;; + *) + echo "usage: apple-geometry.sh [--out FILE.json]" >&2 + exit 2 + ;; + esac +done + +PICKED="$(ios_pick_device "${BLURT_SIM_DEVICE:-iPhone 18 Pro}")" +[ -n "$PICKED" ] || { + echo "apple-geometry: no iPhone simulator available" >&2 + exit 1 +} +UDID="${PICKED%% *}" +NAME="${PICKED#* }" +ios_wait_booted "$UDID" +rm -rf "$WORK" +mkdir -p "$WORK" + +echo "==> measuring the system keyboard on $NAME" +(cd "$REPO_ROOT/App/BlurtiOS" && xcodegen generate --quiet) +RESULTS="$WORK/probe.xcresult" +xcodebuild test -project "$REPO_ROOT/App/BlurtiOS/BlurtiOS.xcodeproj" -scheme BlurtiOSProbe \ + -destination "platform=iOS Simulator,id=$UDID" -derivedDataPath "$DERIVED" \ + -resultBundlePath "$RESULTS" "${IOS_SIM_SIGNING[@]}" -collect-test-diagnostics never -quiet +xcrun xcresulttool export attachments --path "$RESULTS" --output-path "$WORK/attachments" >/dev/null + +# The attachments come out under generated names; the manifest maps them back. +# XCUITest reports touch cells, which tile a row with no gaps, so the caps +# themselves — widths, gaps, height, radius, the surface's top edge and the +# margins — are read off the light screenshot by design-diff.swift measure, +# with the probes placed inside the caps from the cells. +python3 - "$WORK/attachments" "$OUT" "$NAME" "$WORK" "$REPO_ROOT" <<'PY' +import json, os, shutil, subprocess, sys +from datetime import date +attachments, out, device, work, root = sys.argv[1:6] +manifest = json.load(open(os.path.join(attachments, "manifest.json"))) +found = {} +for test in manifest: + for a in test.get("attachments", []): + found[a["suggestedHumanReadableName"]] = os.path.join(attachments, a["exportedFileName"]) +faces, shots = {}, {} +for face in ("light", "dark"): + name = next(k for k in found if k.startswith(f"apple-geometry-{face}")) + faces[face] = json.load(open(found[name])) + shot = next(k for k in found if k.startswith(f"apple-keyboard-{face}")) + shots[face] = os.path.join(work, f"apple-keyboard-{face}.png") + shutil.copy(found[shot], shots[face]) + +light = faces["light"] +keys, kb, win = light["keys"], light["keyboard"], light["window"] +def cell(name): + if name not in keys: + raise SystemExit(f"apple-geometry: the keyboard has no key called {name!r}; keys: {sorted(keys)}") + return keys[name] +q, w, a, z, space, more = cell("q"), cell("w"), cell("a"), cell("z"), cell("space"), cell("more") +others = light.get("others", {}) +shift = others.get("button:shift") or cell("shift") +ret = next((v for k, v in others.items() if k.lower().startswith("button:return")), None) +r3 = lambda v: round(v * 1000) / 1000 + +# Probes at each cell's centre: the cap's middle, which the scan reads +# through (a glyph is inside the cap; the edges are found past it). +def probe(c): + return f'{c["x"] + c["w"] / 2},{c["y"] + c["h"] / 2}' +scale = 3 # the iPhone 18 Pro; the capture's pixels per point +def measure(face, probes, columns): + args = ["swift", os.path.join(root, "scripts/design-diff.swift"), "measure", shots[face], "--scale", str(scale), + "--surface", f'{space["x"] + space["w"] / 2},{q["y"] - 3}'] + for p in probes: args += ["--probe", p] + for c in columns: args += ["--column", c] + return json.loads(subprocess.run(args, capture_output=True, text=True, check=True).stdout) +row_l = {"w": keys["l"]["w"], "x": keys["l"]["x"], "y": keys["l"]["y"]} +# The surface's top edge is read down the keyboard's middle (its top corners +# are rounded); the l glyph's height down the column nearest its stroke. +mid = f'{win["w"] / 2}' +l_cx = keys["l"]["x"] + keys["l"]["w"] / 2 +l_columns = [f'{l_cx + dx / 3}:{keys["l"]["y"]}:{keys["l"]["y"] + keys["l"]["h"]}' for dx in range(-4, 5)] +m = measure("light", [probe(q), probe(shift), probe(space), probe(ret) if ret else probe(more), probe(more)], + [f'{mid}:{q["y"] - 80}:{q["y"]}'] + l_columns) +dark = measure("dark", [probe(q), probe(more)], [f'{mid}:{q["y"] - 80}:{q["y"]}']) +caps = {c["probe"]: c for c in m["caps"] if "error" not in c} +missing = [c for c in m["caps"] if "error" in c] +if missing: + raise SystemExit(f"apple-geometry: probes missed a cap: {missing}") +letter, side, spacebar, returnkey, abc = (caps[probe(q)], caps[probe(shift)], caps[probe(space)], + caps[probe(ret)] if ret else None, caps[probe(more)]) +row1 = letter["row"] +surface_top = m["columns"][0]["changes"][0]["y"] +# Four changes down the column: cap top, glyph top, glyph bottom, cap bottom. +glyph_height = None +for column in m["columns"][1:]: + changes = column["changes"] + if len(changes) >= 4: + glyph_height = changes[-2]["y"] - changes[1]["y"] + break +cap = { + "letter": r3(sum(c["width"] for c in row1) / len(row1)), + "height": letter["height"], + "gap": r3(sum(letter["gaps"]) / len(letter["gaps"])), + "marginSide": row1[0]["left"], + "marginRight": r3(win["w"] - row1[-1]["left"] - row1[-1]["width"]), + "rowGap": r3(a["y"] - q["y"] - letter["height"]), + "radius": letter["radius"], + "side": r3((side["row"][0]["width"] + side["row"][-1]["width"]) / 2), + "sideGap": side["gaps"][0], + "abc": abc["width"], + "space": spacebar["width"], + "return": returnkey["width"] if returnkey else None, + "topBand": r3(letter["top"] - surface_top), + "bottomMargin": r3(kb["y"] + kb["h"] - letter["bottom"] - 3 * (a["y"] - q["y"])), + "keyboardHeight": r3(kb["y"] + kb["h"] - surface_top), + "systemBarBelow": r3(win["h"] - kb["y"] - kb["h"]), +} +geometry = { + "device": device, "width": win["w"], "measured": date.today().isoformat(), + "sources": { + "cells": "XCUITest key frames (BlurtiOSProbe) — touch cells, no gaps", + "caps": "pixel scan of the light screenshot (scripts/design-diff.swift measure)", + "radius": "circle fitted to the top-left inset of the q cap over its first 12 pt (±0.25)", + "legend": "the l glyph's height on the light face; the point size is an estimate (÷ 0.73, SF's ascender)", + }, + "cell": {"letter": r3(q["w"]), "row": r3(a["y"] - q["y"]), "shift": r3(shift["w"]), "delete": r3(cell("delete")["w"]), + "more": r3(more["w"]), "space": r3(space["w"]), "return": r3(ret["w"]) if ret else None, + "firstRowTop": r3(q["y"]), "keyboardTop": r3(kb["y"]), "keyboardHeight": r3(kb["h"])}, + "cap": cap, + "legend": {"lHeight": r3(glyph_height) if glyph_height else None, + "pointSizeEstimate": r3(glyph_height / 0.73) if glyph_height else None}, + "colors": {"light": {"surface": m["surface"], "key": letter["fill"], "modifier": abc["fill"]}, + "dark": {"surface": dark["surface"], "key": dark["caps"][0]["fill"], "modifier": dark["caps"][1]["fill"]}}, + "darkFaceIdentical": all( + abs(faces["dark"]["keys"].get(n, {}).get(kk, -1) - vv) < 0.01 for n, v in keys.items() for kk, vv in v.items()), +} +json.dump(geometry, open(out, "w"), indent=2, sort_keys=True) +open(out, "a").write("\n") +print(f"apple-geometry: {out}") +for k, v in cap.items(): + print(f" cap.{k}: {v}") +print(f" legend: {geometry['legend']}") +print(f" colors: {geometry['colors']}") +print(f" dark face identical: {geometry['darkFaceIdentical']}") +PY +echo "screenshots: $WORK/apple-keyboard-light.png, $WORK/apple-keyboard-dark.png" diff --git a/scripts/check-invariants.sh b/scripts/check-invariants.sh index e1725580..b070fc5d 100755 --- a/scripts/check-invariants.sh +++ b/scripts/check-invariants.sh @@ -75,8 +75,16 @@ GUARDRAILS=".claude/skills/project-guardrails/SKILL.md" # code extensions also keeps the app scope off the .png and .m4a resources it # was otherwise grepping byte by byte. ENGINE="Sources/*.swift" -APP="App/Blurt/*.swift App/Blurt/*.yml App/Blurt/*.plist :!App/Blurt/Blurt.xcodeproj" -TESTS="Tests/*.swift App/Blurt/BlurtUITests/*.swift" +# The app scope is `App/`, not `App/Blurt/`: there is a second app spec now +# (the iOS app), and a scope pinned to one app silently stops covering the +# next one. +APP="App/*.swift App/*.yml App/*.plist :!App/Blurt/Blurt.xcodeproj :!App/BlurtiOS/BlurtiOSTests" +TESTS="Tests/*.swift App/Blurt/BlurtUITests/*.swift App/BlurtiOS/BlurtiOSTests/*.swift" +# The keyboard extension and what it compiles or links — its own sources, the +# Shared UI, and BlurtiOSCore: iOS lets no keyboard use the microphone, and the +# design keeps every audio type out of it, so an import there is a decision +# being reversed by accident. +KEYBOARD="App/BlurtiOS/BlurtKeyboard/*.swift App/BlurtiOS/Shared/*.swift App/BlurtiOS/BlurtiOSCore/*.swift" # The streaming-STT rule's one carve-out: the experimental read-selection-aloud # feature speaks through AssemblyAI's streaming TTS, which is WebSocket-only. # The rule is about *dictation* staying one request/one response, so the TTS @@ -100,6 +108,7 @@ PATTERNS=( "import KeyboardShortcuts" "KeychainStore\\(service: *(HostIdentity\\.current\\.keychainService|\"blurt\")" "@available\\(\\*, *deprecated" + "import AVFoundation|import AVFAudio|AVCaptureSession|AVAudioSession|AVAudioApplication|AudioQueueNewInput" ) SCOPES=( "$ENGINE $APP" @@ -114,6 +123,7 @@ SCOPES=( "$ENGINE $APP" "$TESTS" "$ENGINE $APP" + "$KEYBOARD" ) ADVICE=( "MicCapture builds a fresh AVCaptureSession recorder per capture — a long-lived engine goes stale on a device switch" @@ -128,6 +138,7 @@ ADVICE=( "the trigger is a home-grown lone modifier (CGEventTap + DictationKeyGate)" "use an isolated service (see KeychainStoreTests) or InMemoryAPIKeyStore" "deleted types stay deleted — no deprecated re-exports" + "the keyboard never hears anything: the app listens and transcribes, the keyboard inserts (KeyboardRelayInjector)" ) # One known-bad line per rule, same order. `--self-test` asserts each pattern @@ -147,6 +158,7 @@ PROBES=( "import KeyboardShortcuts" "let store = KeychainStore(service: HostIdentity.current.keychainService, account: \"AssemblyAIAPIKey\")" "@available(*, deprecated, renamed: \"NewName\")" + "let session = AVCaptureSession()" ) # A verbatim slice of the row in AGENTS.md's "Settled decisions" table that each @@ -168,6 +180,7 @@ TABLE_ANCHORS=( "Add a \`KeyboardShortcuts\` package or a key+modifier chord" "Touch the real Keychain in tests" "Add backwards-compat shims for removed types" + "Give the keyboard extension a microphone or any audio type" ) # The same rule as the guardrails skill words it. Deliberately not the table's @@ -188,6 +201,7 @@ SKILL_ANCHORS=( "No \`KeyboardShortcuts\` package" "the real Keychain in tests" "Don't add backwards-compat shims for removed types." + "The keyboard never hears anything." ) if [ "${#SCOPES[@]}" -ne "${#PATTERNS[@]}" ] \ @@ -204,7 +218,7 @@ fi if [ "${1:-}" = "--self-test" ]; then failed=0 for i in "${!PATTERNS[@]}"; do - if printf '%s\n' "${PROBES[$i]}" | grep -qE "${PATTERNS[$i]}"; then + if grep -qE -- "${PATTERNS[$i]}" <<<"${PROBES[$i]}"; then echo " ok rule $i flags: ${PROBES[$i]}" else echo " FAIL rule $i does not match its own probe: ${PROBES[$i]}" >&2 @@ -225,7 +239,7 @@ if [ "${1:-}" = "--self-test" ]; then for good in "${GOOD[@]}"; do hit="" for i in "${!PATTERNS[@]}"; do - if printf '%s\n' "$good" | grep -qE "${PATTERNS[$i]}"; then + if grep -qE -- "${PATTERNS[$i]}" <<<"$good"; then hit="$i" break fi @@ -251,8 +265,12 @@ if [ "${1:-}" = "--self-test" ]; then exit 1 fi + # Here-strings, never `printf | grep -q`, throughout this file: grep -q + # exits at its first match, and a producer still writing gets EPIPE — + # under pipefail, a false "no match". The table (~30 KB, past a macOS pipe's + # 16 KB first buffer) did exactly that on CI: rule 3 failing at random. for i in "${!PATTERNS[@]}"; do - if printf '%s\n' "$TABLE" | grep -qF -- "${TABLE_ANCHORS[$i]}"; then + if grep -qF -- "${TABLE_ANCHORS[$i]}" <<<"$TABLE"; then echo " ok rule $i is pinned to its $GUIDE row" else echo " FAIL rule $i has no row in $GUIDE's settled-decisions table:" >&2 @@ -285,7 +303,7 @@ fi # here until it is staged — and a scan that skips the file you just wrote reads # exactly like a scan that approved it. Warn rather than fail, because an # untracked file is a normal state mid-edit; `git add -N` brings it into scope. -UNTRACKED=$(git ls-files --others --exclude-standard -- Sources Tests App/Blurt) +UNTRACKED=$(git ls-files --others --exclude-standard -- Sources Tests App) if [ -n "$UNTRACKED" ]; then echo "note: untracked files are NOT scanned (git add -N to include them):" printf '%s\n' "$UNTRACKED" | sed 's/^/ /' diff --git a/scripts/check.sh b/scripts/check.sh index d570f5b5..9f877c97 100755 --- a/scripts/check.sh +++ b/scripts/check.sh @@ -238,27 +238,64 @@ check_no_external_deps() { violation=1 fi - # App: the local BlurtEngine (path:) package plus the one allowlisted remote. - # A remote package is declared with a url:/github: key inside project.yml's - # `packages:` block, so extract that block and reject any such key that isn't - # exactly the allowlisted URL. A github: shorthand is rejected outright, so the - # allowlist has one spelling to match. - local app_packages remote allowed_re + # Apps: the local BlurtEngine (path:) package plus the one allowlisted remote + # (Sparkle, the Mac app's updater). A remote package is declared with a + # url:/github: key inside project.yml's `packages:` block, so extract that + # block and reject any such key that isn't exactly the allowlisted URL. A + # github: shorthand is rejected outright, so the allowlist has one spelling to + # match. + # + # Every App/*/project.yml, not just the mac app's: the iOS app carries its own + # spec with its own `packages:` block, and a guard that names one file is a + # guard the next app walks around without anyone noticing. + local spec spec_path app_packages remote allowed_re allowed_re="${ALLOWED_APP_PACKAGE_URL//./\\.}" - app_packages="$(awk '/^packages:/{f=1;next} /^[^[:space:]]/{f=0} f' "$APP_DIR/project.yml")" - remote="$(printf '%s\n' "$app_packages" | grep -nE '(^|[[:space:]])(url|github):' \ - | grep -vE "^[0-9]+:[[:space:]]*url:[[:space:]]*${allowed_re}[[:space:]]*$" || true)" - if [ -n "$remote" ]; then - echo "error: App/Blurt/project.yml declares a remote SPM package other than Sparkle:" >&2 - printf '%s\n' "$remote" >&2 - violation=1 - fi + for spec in "$REPO_ROOT"/App/*/project.yml; do + spec_path="${spec#"$REPO_ROOT"/}" + app_packages="$(awk '/^packages:/{f=1;next} /^[^[:space:]]/{f=0} f' "$spec")" + remote="$(printf '%s\n' "$app_packages" | grep -nE '(^|[[:space:]])(url|github):' \ + | grep -vE "^[0-9]+:[[:space:]]*url:[[:space:]]*${allowed_re}[[:space:]]*$" || true)" + if [ -n "$remote" ]; then + echo "error: $spec_path declares a remote SPM package other than Sparkle:" >&2 + printf '%s\n' "$remote" >&2 + violation=1 + fi + done [ "$violation" -eq 0 ] || return 1 echo "no unexpected dependencies (engine dependency-free; app carries BlurtEngine + Sparkle only)" } run_check "no-external-dependencies guard" check_no_external_deps +# Whether this run builds the iPhone app for real (scripts/ios-check.sh, near +# the end): off CI — there the ios-build job runs the same script, and running +# it here too would double the macos-26 bill — and only with an iPhone simulator +# on the runtime this Xcode's SDK builds for (an older one is refused at the +# destination). The list is read into a variable before it is searched: piped +# into `grep -q` under pipefail, an early match could leave simctl writing into +# a closed pipe and read as "no runtime". +IOS_GATES=0 +if [ "$PORTABLE" -eq 0 ] && ! is_ci; then + IOS_SDK_VERSION="$(xcrun --sdk iphonesimulator --show-sdk-version 2>/dev/null || true)" + IOS_RUNTIMES="$(xcrun simctl list runtimes available 2>/dev/null || true)" + if [ -n "$IOS_SDK_VERSION" ] && grep -q "^iOS $IOS_SDK_VERSION " <<<"$IOS_RUNTIMES"; then + IOS_GATES=1 + fi +fi + +# The iPhone targets, typechecked against the Mac Catalyst frameworks — the +# closest thing to an iOS SDK a Mac without Xcode has (scripts/ios-typecheck.sh +# explains). Not when ios-check.sh builds them for real below: that build +# catches all this does. Skipped, not failed, where those frameworks are +# absent; CI's ios-build job builds and tests the real thing. +if [ "$IOS_GATES" -eq 1 ]; then + echo "==> ios-typecheck skipped: ios-check builds App/BlurtiOS for real below" +elif [ -d "$(xcrun --show-sdk-path 2>/dev/null)/System/iOSSupport/System/Library/Frameworks/UIKit.framework" ]; then + run_check "ios-typecheck (App/BlurtiOS, Mac Catalyst stand-in)" "$REPO_ROOT/scripts/ios-typecheck.sh" +else + echo "==> ios-typecheck skipped: no Mac Catalyst frameworks in the selected SDK" +fi + # Ignore rules must not shadow tracked files. A .gitignore pattern only suppresses # files that are *untracked* — one that also matches something already committed # leaves it tracked but invisible to `git status`, so later edits to it stop @@ -391,6 +428,19 @@ check_invariants() { } run_check "settled decisions (AGENTS.md invariants)" check_invariants +# The design's one source (App/BlurtiOS/Design/tokens.json) and everything +# generated from it — Shared/DesignTokens.swift, DESIGN.md's token tables, the +# app's three colour sets — must agree, and the keyboard's views must carry no +# design literal that belongs in the tokens. scripts/design-sync.sh --check +# regenerates into .build/ and diffs, the same shape as the xcodegen drift +# check further down. It runs scripts/design-tokens.swift as a `swift` script, +# which --portable mode does without. +if [ "$PORTABLE" -eq 0 ]; then + run_check "design tokens in sync (scripts/design-sync.sh --check)" bash scripts/design-sync.sh --check +else + echo "==> design-sync.sh --check skipped in portable mode (needs swift)" +fi + # Mutation-testing target list. The full run stays out of this script (minutes, and # survivors need judgement — see mutate.sh's header), but that also meant nothing # noticed when #186 renamed ConversationContext.swift out from under its default @@ -591,7 +641,9 @@ else echo "==> coverage gate (>= ${MIN_COVERAGE}% engine lines)" BIN="$(swift build --show-bin-path)" PROFDATA="$BIN/codecov/default.profdata" - XCTEST_BUNDLE="$(find "$BIN" -maxdepth 1 -name '*PackageTests.xctest' -print -quit)" + # `BlurtPackageTests.xctest` from SwiftPM's own build system; Xcode 27's + # SwiftPM names the bundle after the test target, `BlurtEngineTests.xctest`. + XCTEST_BUNDLE="$(find "$BIN" -maxdepth 1 -name '*Tests.xctest' -print -quit)" XCTEST_BIN="$XCTEST_BUNDLE/Contents/MacOS/$(basename "$XCTEST_BUNDLE" .xctest)" # These must exist after `swift test --enable-code-coverage`. Previously a # missing one (a renamed test bundle, a coverage build that didn't happen) @@ -783,6 +835,17 @@ if [ "$PORTABLE" -eq 0 ]; then fi fi +# The iPhone code's gates — its tests with BlurtiOSCore's coverage gate, +# `swiftlint analyze` and periphery over App/BlurtiOS (scripts/ios-check.sh) — so +# a local green covers both platforms. When IOS_GATES (above) says no, skipped +# with a note, as the UI suite is, and CI stays the authority. +if [ "$IOS_GATES" -eq 1 ]; then + run_check "ios-check (App/BlurtiOS tests, coverage, analyze, periphery)" "$REPO_ROOT/scripts/ios-check.sh" +elif [ "$PORTABLE" -eq 0 ] && ! is_ci; then + IOS_SKIPPED=1 + echo "==> ios-check skipped: no iOS ${IOS_SDK_VERSION:-?} simulator runtime (xcodebuild -downloadPlatform iOS)" +fi + # The closing line, and only when there is nothing to report — otherwise the exit # trap has the last word. Printing "ok" above a list of failures would be worse # than printing nothing at all. @@ -792,7 +855,7 @@ if [ "${#FAILED_CHECKS[@]}" -eq 0 ]; then elif [ "${INTEGRATION:-0}" -eq 0 ]; then # Say it at the end too, where the reader is deciding whether this run means # "green": everything else passed, but the UI suite and leak scan did not run. - echo "==> ok (UI suite + leak scan NOT run — CI on macos-26 covers those)" + echo "==> ok (UI suite + leak scan NOT run${IOS_SKIPPED:+, nor the iPhone gates} — CI on macos-26 covers those)" else echo "==> ok" fi diff --git a/scripts/design-capture.sh b/scripts/design-capture.sh new file mode 100755 index 00000000..3d8ca68a --- /dev/null +++ b/scripts/design-capture.sh @@ -0,0 +1,110 @@ +#!/bin/bash +# Capture the keyboard from the simulator, one still per layout × state × theme, +# cropped to the keyboard's own pixels — the simulator side of the design loop +# (DESIGN.md › Reproducing it). Each capture is deterministic: the gallery's +# -BlurtGalleryStill holds the orb, the ring and the wave at time zero, and +# -BlurtGalleryBare draws one row with a magenta registration border that +# scripts/design-diff.swift crops to. +# +# scripts/design-capture.sh # panel × the golden states × both faces +# scripts/design-capture.sh --layout full --states idle,recording,error --themes dark +# scripts/design-capture.sh --voice b # the ribs candidate (-BlurtGalleryVoice) +# scripts/design-capture.sh --out .build/design/captures --no-build +# +# Writes /--@3x.png (1206 px wide on iPhone 18 Pro, +# the same pixel size as any 3× reference, so the diff never resamples) +# plus /--.raw.png, the whole screen it was cut from. +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)" +cd "$REPO_ROOT" + +LAYOUT=panel +STATES=idle,recording,error,term +THEMES=light,dark +OUT="$REPO_ROOT/.build/design/captures" +VOICE="" +BUILD=1 +while [ $# -gt 0 ]; do + case "$1" in + --layout) + LAYOUT="${2:?}" + shift 2 + ;; + --states) + STATES="${2:?}" + shift 2 + ;; + --themes) + THEMES="${2:?}" + shift 2 + ;; + --out) + OUT="${2:?}" + shift 2 + ;; + --voice) + VOICE="${2:?}" + shift 2 + ;; + --no-build) + BUILD=0 + shift + ;; + *) + echo "usage: design-capture.sh [--layout slimBar|panel|full] [--states a,b] [--themes a,b] [--voice a|b|c] [--out DIR] [--no-build]" >&2 + exit 2 + ;; + esac +done +mkdir -p "$OUT" + +# The layout's height in points, from the tokens the views are built on, so +# the crop can assert it got the whole keyboard and nothing else. +height_pt() { + python3 - "$1" <<'PY' +import json, sys +m = json.load(open("App/BlurtiOS/Design/tokens.json"))["metrics"] +v = lambda k: m[k]["value"] if isinstance(m[k], dict) else m[k] +print({"slimBar": v("layout/slim"), "panel": v("layout/panel"), "full": v("layout/full")}[sys.argv[1]]) +PY +} +HEIGHT_PT="$(height_pt "$LAYOUT")" + +# Build once; every capture after the first reuses it. +first=1 +IFS=, read -ra STATE_LIST <<<"$STATES" +IFS=, read -ra THEME_LIST <<<"$THEMES" +for theme in "${THEME_LIST[@]}"; do + for state in "${STATE_LIST[@]}"; do + name="$LAYOUT-$state-$theme${VOICE:+-$VOICE}" + raw="$OUT/$name.raw.png" + flags=(--screenshot "$raw") + if [ "$first" -eq 0 ] || [ "$BUILD" -eq 0 ]; then flags=(--no-build "${flags[@]}"); fi + first=0 + # `landed` is a moment, not a state: the drop peaks 0.6 s after the gallery's + # 2.4 s offset, so the default 3 s delay catches it; everything else is + # still by construction and needs only the launch to settle. + delay=1.5 + still=-BlurtGalleryStill + if [ "$state" = "landed" ]; then + delay=3 + still="" + fi + echo "==> $name" + BLURT_LAUNCH_ARGS="-BlurtGallery $LAYOUT $state $theme $still -BlurtGalleryBare${VOICE:+ -BlurtGalleryVoice $VOICE}" \ + BLURT_SHOT_DELAY="$delay" scripts/ios-sim.sh "${flags[@]}" >/dev/null + # The capture's scale: pixels across ÷ the device's points across. + scale="$(python3 -c ' +import subprocess, sys +out = subprocess.run(["sips", "-g", "pixelWidth", sys.argv[1]], capture_output=True, text=True).stdout +px = int(out.split()[-1]) +print(3 if px > 1000 else 2) +' "$raw")" + # `term` and `keys` show the full keyboard whatever layout was asked for. + expect=$((HEIGHT_PT * scale)) + case "$state" in term | keys | symbols) expect=$(($(height_pt full) * scale)) ;; esac + swift scripts/design-diff.swift crop "$raw" "$OUT/$name@${scale}x.png" --expect-height "$expect" + done +done +echo "design-capture: ${#STATE_LIST[@]} × ${#THEME_LIST[@]} captures of $LAYOUT in $OUT" diff --git a/scripts/design-diff.swift b/scripts/design-diff.swift new file mode 100755 index 00000000..248cb03b --- /dev/null +++ b/scripts/design-diff.swift @@ -0,0 +1,450 @@ +#!/usr/bin/env swift // Pixel tools for the design loop, with no dependencies beyond AppKit — the // same shape as beautify.swift and screenshot.swift. Four commands: // // swift scripts/design-diff.swift crop [--expect-height ] // Cuts the keyboard out of a whole-screen simulator capture: finds the // magenta (#FF00FF) registration border the gallery's -BlurtGalleryBare // mode draws round the row and writes what is inside it. With +// --expect-height, fails unless the crop is that tall (±1 px). +// +// swift scripts/design-diff.swift diff --out --json +// [--mask x,y,w,h]... [--threshold 8] +// Compares two same-sized images pixel by pixel, both converted to sRGB +// first (a simulator capture carries the display's profile, a design +// export is sRGB). Reports, for the whole image and for everything +// outside the masks, the fraction of pixels whose largest channel +// difference exceeds the threshold (AE) and the root-mean-square +// difference (RMSE, 0–1); each mask is reported on its own. Writes a +// triptych: a | b | heat map (difference in red, masks tinted blue). +// Masks are in pixels of the images given. +// +// swift scripts/design-diff.swift sheet --out --columns [--scale ] =... +// A contact sheet: the images in a grid with their captions under them. +// +// swift scripts/design-diff.swift measure --scale 3 --surface x,y --probe x,y... [--column x:y0:y1]... +// The visual geometry of the keys in a screenshot of the system keyboard +// (widths, gaps, heights, corner radius, the surface's edges), in points. +// +// Every image is read into an 8-bit sRGB RGBA buffer through CoreGraphics, so +// the numbers do not depend on how the PNG was tagged. + +import AppKit + +struct Pixels { + let width: Int + let height: Int + var bytes: [UInt8] // RGBA, sRGB, not premultiplied in any way that matters for opaque captures + + subscript(x: Int, y: Int) -> (r: Int, g: Int, b: Int) { + let i = (y * width + x) * 4 + return (Int(bytes[i]), Int(bytes[i + 1]), Int(bytes[i + 2])) + } +} + +func fail(_ message: String) -> Never { + FileHandle.standardError.write(Data("design-diff: \(message)\n".utf8)) + exit(1) +} + +func load(_ path: String) -> Pixels { + guard let image = NSImage(contentsOfFile: path), + let cgImage = image.cgImage(forProposedRect: nil, context: nil, hints: nil) + else { fail("could not read \(path)") } + let width = cgImage.width + let height = cgImage.height + var bytes = [UInt8](repeating: 0, count: width * height * 4) + guard let space = CGColorSpace(name: CGColorSpace.sRGB), + let context = CGContext( + data: &bytes, width: width, height: height, bitsPerComponent: 8, bytesPerRow: width * 4, space: space, + bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue) + else { fail("could not make an sRGB context for \(path)") } + context.draw(cgImage, in: CGRect(x: 0, y: 0, width: width, height: height)) + return Pixels(width: width, height: height, bytes: bytes) +} + +func save(_ pixels: Pixels, to path: String) { + var bytes = pixels.bytes + guard let space = CGColorSpace(name: CGColorSpace.sRGB), + let context = CGContext( + data: &bytes, width: pixels.width, height: pixels.height, bitsPerComponent: 8, bytesPerRow: pixels.width * 4, + space: space, bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue), + let image = context.makeImage() + else { fail("could not make an image for \(path)") } + let rep = NSBitmapImageRep(cgImage: image) + guard let data = rep.representation(using: .png, properties: [:]) else { fail("could not encode \(path)") } + do { try data.write(to: URL(fileURLWithPath: path)) } catch { fail("could not write \(path): \(error)") } +} + +func flag(_ name: String, in args: inout [String]) -> String? { + guard let i = args.firstIndex(of: name), i + 1 < args.count else { return nil } + let value = args[i + 1] + args.removeSubrange(i...(i + 1)) + return value +} + +func flags(_ name: String, in args: inout [String]) -> [String] { + var values: [String] = [] + while let value = flag(name, in: &args) { values.append(value) } + return values +} + +// MARK: - crop + +func isSentinel(_ p: (r: Int, g: Int, b: Int)) -> Bool { + abs(p.r - 255) <= 6 && p.g <= 6 && abs(p.b - 255) <= 6 +} + +func crop(_ args: [String]) { + var args = args + let expected = flag("--expect-height", in: &args).map { Int($0) ?? -1 } + guard args.count == 2 else { fail("crop [--expect-height ]") } + let source = load(args[0]) + var minX = Int.max + var minY = Int.max + var maxX = -1 + var maxY = -1 + for y in 0..= 0 else { fail("no #FF00FF registration border in \(args[0]) — launch with -BlurtGalleryBare") } + // The border's thickness: walk down from the top edge along the middle + // column. The sides may have no border at all — the keyboard rows are drawn + // at the phone's full width, bordered above and below only — so each side is + // inset only where the middle row does start with the sentinel. + let midX = (minX + maxX) / 2 + let midY = (minY + maxY) / 2 + var thickness = 0 + while minY + thickness <= maxY, isSentinel(source[midX, minY + thickness]) { thickness += 1 } + guard thickness > 0, thickness < 40 else { fail("registration border looks wrong (thickness \(thickness))") } + let leftInset = isSentinel(source[minX, midY]) ? thickness : 0 + let rightInset = isSentinel(source[maxX, midY]) ? thickness : 0 + let x0 = minX + leftInset + let y0 = minY + thickness + let width = maxX - minX + 1 - leftInset - rightInset + let height = maxY - minY + 1 - 2 * thickness + guard width > 0, height > 0 else { fail("registration border encloses nothing") } + if let expected, abs(height - expected) > 1 { + fail("crop is \(height) px tall, expected \(expected) (the layout height × the device scale)") + } + var out = Pixels(width: width, height: height, bytes: [UInt8](repeating: 0, count: width * height * 4)) + for y in 0.. Bool { px >= x && px < x + w && py >= y && py < y + h } +} + +func diff(_ args: [String]) { + var args = args + guard let outPath = flag("--out", in: &args), let jsonPath = flag("--json", in: &args) else { + fail("diff --out --json [--mask x,y,w,h]... [--threshold n]") + } + let threshold = Int(flag("--threshold", in: &args) ?? "8") ?? 8 + let masks = flags("--mask", in: &args).map { spec -> Rect in + let parts = spec.split(separator: ",").compactMap { Int($0) } + guard parts.count == 4 else { fail("mask must be x,y,w,h: \(spec)") } + return Rect(x: parts[0], y: parts[1], w: parts[2], h: parts[3]) + } + guard args.count == 2 else { fail("diff needs two images") } + let a = load(args[0]) + let b = load(args[1]) + guard a.width == b.width, a.height == b.height else { + fail("sizes differ: \(a.width)×\(a.height) vs \(b.width)×\(b.height) — never resample; fix the export scale") + } + var total = 0 + var differing = 0 + var squares = 0.0 + var chromeTotal = 0 + var chromeDiffering = 0 + var chromeSquares = 0.0 + var maskDiffering = [Int](repeating: 0, count: masks.count) + var maskTotal = [Int](repeating: 0, count: masks.count) + var heat = Pixels( + width: a.width * 3, height: a.height, bytes: [UInt8](repeating: 255, count: a.width * 3 * a.height * 4)) + for y in 0.. threshold + let normalized = Double(delta) / 255 + total += 1 + if over { differing += 1 } + squares += normalized * normalized + let maskIndex = masks.firstIndex { $0.contains(x, y) } + if let maskIndex { + maskTotal[maskIndex] += 1 + if over { maskDiffering[maskIndex] += 1 } + } else { + chromeTotal += 1 + if over { chromeDiffering += 1 } + chromeSquares += normalized * normalized + } + // Triptych: a, b, heat. + let ia = (y * heat.width + x) * 4 + heat.bytes[ia] = UInt8(pa.r) + heat.bytes[ia + 1] = UInt8(pa.g) + heat.bytes[ia + 2] = UInt8(pa.b) + let ib = (y * heat.width + a.width + x) * 4 + heat.bytes[ib] = UInt8(pb.r) + heat.bytes[ib + 1] = UInt8(pb.g) + heat.bytes[ib + 2] = UInt8(pb.b) + let ih = (y * heat.width + 2 * a.width + x) * 4 + let strength = UInt8(min(255, delta * 4)) + if maskIndex != nil { + heat.bytes[ih] = 230 - strength / 2 + heat.bytes[ih + 1] = 236 - strength / 2 + heat.bytes[ih + 2] = 255 + } else { + heat.bytes[ih] = 255 + heat.bytes[ih + 1] = 255 - strength + heat.bytes[ih + 2] = 255 - strength + } + } + } + save(heat, to: outPath) + func fraction(_ n: Int, _ d: Int) -> Double { d == 0 ? 0 : Double(n) / Double(d) } + var metrics: [String: Any] = [ + "width": a.width, "height": a.height, "threshold": threshold, + "all": ["ae": fraction(differing, total), "rmse": (squares / Double(max(total, 1))).squareRoot()], + "chrome": [ + "ae": fraction(chromeDiffering, chromeTotal), "rmse": (chromeSquares / Double(max(chromeTotal, 1))).squareRoot(), + "pixels": chromeTotal, + ], + ] + metrics["masks"] = masks.enumerated().map { i, rect in + ["x": rect.x, "y": rect.y, "w": rect.w, "h": rect.h, "ae": fraction(maskDiffering[i], maskTotal[i])] + } + guard let data = try? JSONSerialization.data(withJSONObject: metrics, options: [.prettyPrinted, .sortedKeys]) else { + fail("could not encode metrics") + } + do { try data.write(to: URL(fileURLWithPath: jsonPath)) } catch { fail("could not write \(jsonPath): \(error)") } + func percent(_ value: Double) -> String { String(format: "%.3f%%", value * 100) } + let chromeRMSE = String(format: "%.4f", (chromeSquares / Double(max(chromeTotal, 1))).squareRoot()) + print( + "diff: chrome AE \(percent(fraction(chromeDiffering, chromeTotal))) RMSE \(chromeRMSE)" + + " · all AE \(percent(fraction(differing, total))) · \(masks.count) masks → \(outPath)") +} + +// MARK: - sheet + +func sheet(_ args: [String]) { + var args = args + guard let outPath = flag("--out", in: &args), let columns = Int(flag("--columns", in: &args) ?? "") else { + fail("sheet --out --columns [--scale ] =...") + } + let scale = CGFloat(Double(flag("--scale", in: &args) ?? "0.5") ?? 0.5) + let entries = args.map { arg -> (String, NSImage) in + guard let eq = arg.firstIndex(of: "=") else { fail("each cell is caption=path: \(arg)") } + let path = String(arg[arg.index(after: eq)...]) + guard let image = NSImage(contentsOfFile: path) else { fail("could not read \(path)") } + return (String(arg[.. --scale 3 --surface x,y --probe x,y [--probe x,y]... [--column x:y0:y1]... +/// +/// `--surface` samples the keyboard's ground colour; a pixel that differs +/// from it by more than the threshold is part of a cap. Each `--probe` names a +/// point inside a cap: the cap's top and bottom are found up and down that +/// column, then the row 3 pt below the top is scanned for every cap on it +/// (left, width, and the gap to the next), and the corner radius is fitted +/// from the cap's top-left inset over its first rows. `--column` lists where +/// the colour changes along a column, for finding the surface's top edge. +func measure(_ args: [String]) { + var args = args + guard let scaleText = flag("--scale", in: &args), let scale = Double(scaleText), + let surfaceText = flag("--surface", in: &args) + else { fail("measure --scale --surface x,y [--probe x,y]... [--column x:y0:y1]...") } + let probes = flags("--probe", in: &args) + let columns = flags("--column", in: &args) + guard args.count == 1 else { fail("measure: one png") } + let image = load(args[0]) + func point(_ text: String) -> (x: Int, y: Int) { + let parts = text.split(separator: ",").compactMap { Double($0) } + guard parts.count == 2 else { fail("measure: a point is x,y in points, got \(text)") } + return (Int((parts[0] * scale).rounded()), Int((parts[1] * scale).rounded())) + } + let surfaceAt = point(surfaceText) + let surface = image[surfaceAt.x, surfaceAt.y] + let threshold = 14 + func isCap(_ x: Int, _ y: Int) -> Bool { + guard x >= 0, y >= 0, x < image.width, y < image.height else { return false } + let p = image[x, y] + return max(abs(p.r - surface.r), abs(p.g - surface.g), abs(p.b - surface.b)) > threshold + } + func pt(_ px: Int) -> Double { (Double(px) / scale * 1000).rounded() / 1000 } + var report: [String: Any] = [ + "surface": String(format: "#%02X%02X%02X", surface.r, surface.g, surface.b), "scale": scale, + ] + var caps: [[String: Any]] = [] + for probeText in probes { + let probe = point(probeText) + guard isCap(probe.x, probe.y) else { + caps.append(["probe": probeText, "error": "not on a cap"]) + continue + } + // The cap's top and bottom along the probe's column. A glyph's + // anti-aliased edge can land on the surface's colour for a pixel, so an + // edge is only an edge when the surface holds for a few pixels. + let hold = Int(scale) + 1 + func surfaceHolds(_ x: Int, _ y: Int, step: Int) -> Bool { + (0.. 0, !surfaceHolds(probe.x, top - 1, step: -1) { top -= 1 } + var bottom = probe.y + while bottom < image.height - 1, !surfaceHolds(probe.x, bottom + 1, step: 1) { bottom += 1 } + // Every cap along the row through the cap's middle (a glyph is inside a + // cap, never the surface's colour for long): runs of cap pixels, merged + // across glyph pixels, with the gaps between. + let scanY = (top + bottom) / 2 + var runs: [(left: Int, right: Int)] = [] + var x = 0 + while x < image.width { + guard isCap(x, scanY) else { + x += 1 + continue + } + let left = x + var lastCap = x + while x < image.width { + if isCap(x, scanY) { + lastCap = x + } else if x - lastCap >= hold { + break + } + x += 1 + } + runs.append((left, lastCap)) + x = lastCap + 1 + } + guard let own = runs.first(where: { $0.left <= probe.x && probe.x <= $0.right }) else { + caps.append(["probe": probeText, "error": "no run under the probe"]) + continue + } + // The top-left corner: the inset of the first cap pixel on each of the + // first rows, fitted to a circle of radius r (inset = r − √(2ry − y²)). + var insets: [(y: Double, inset: Double)] = [] + let rowsToFit = Int(12 * scale) + for i in 0..= candidate ? 0 : candidate - (2 * candidate * sample.y - sample.y * sample.y).squareRoot() + return sum + (predicted - sample.inset) * (predicted - sample.inset) + } + if error < bestError { + bestError = error + bestRadius = candidate + } + candidate += 0.25 + } + // The fill, read just under the top edge where no glyph is. + let color = image[probe.x, min(bottom, top + Int(3 * scale))] + caps.append([ + "probe": probeText, + "fill": String(format: "#%02X%02X%02X", color.r, color.g, color.b), + "top": pt(top), "bottom": pt(bottom + 1), "height": pt(bottom + 1 - top), + "left": pt(own.left), "width": pt(own.right + 1 - own.left), + "radius": (bestRadius / scale * 100).rounded() / 100, + "row": runs.map { ["left": pt($0.left), "width": pt($0.right + 1 - $0.left)] }, + "gaps": zip(runs, runs.dropFirst()).map { pt($1.left - $0.right - 1) }, + ]) + } + report["caps"] = caps + var edges: [[String: Any]] = [] + for columnText in columns { + let parts = columnText.split(separator: ":").compactMap { Double($0) } + guard parts.count == 3 else { fail("measure: a column is x:y0:y1 in points") } + let x = Int((parts[0] * scale).rounded()) + var changes: [[String: Any]] = [] + let y0 = max(0, min(Int((parts[1] * scale).rounded()), image.height - 1)) + let y1 = max(y0, min(Int((parts[2] * scale).rounded()), image.height - 1)) + var previous = image[x, y0] + for y in y0...y1 { + let p = image[x, y] + if max(abs(p.r - previous.r), abs(p.g - previous.g), abs(p.b - previous.b)) > threshold { + changes.append(["y": pt(y), "to": String(format: "#%02X%02X%02X", p.r, p.g, p.b)]) + } + previous = p + } + edges.append(["column": columnText, "changes": changes]) + } + report["columns"] = edges + guard let data = try? JSONSerialization.data(withJSONObject: report, options: [.prettyPrinted, .sortedKeys]) else { + fail("measure: could not encode the report") + } + print(String(decoding: data, as: UTF8.self)) +} + +var arguments = Array(CommandLine.arguments.dropFirst()) +guard let command = arguments.first else { fail("usage: design-diff.swift crop|diff|sheet|measure …") } +arguments.removeFirst() +switch command { +case "crop": crop(arguments) +case "diff": diff(arguments) +case "sheet": sheet(arguments) +case "measure": measure(arguments) +default: fail("unknown command \(command)") +} diff --git a/scripts/design-sync.sh b/scripts/design-sync.sh new file mode 100755 index 00000000..cd4c0bcf --- /dev/null +++ b/scripts/design-sync.sh @@ -0,0 +1,136 @@ +#!/bin/bash +# Fan the design tokens out, or prove they already are. +# +# scripts/design-sync.sh # regenerate DesignTokens.swift, DESIGN.md's tables, the colour sets +# scripts/design-sync.sh --check # regenerate into .build/ and diff; non-zero on drift +# +# `App/BlurtiOS/Design/tokens.json` is the one source for the keyboard's colours +# and numbers, edited by hand. Everything +# that repeats a value from it is generated by scripts/design-tokens.swift, and +# check.sh runs `--check` so a hand edit to a generated file, or a token change +# without a regeneration, fails the way a stale .pbxproj does. +# +# Both modes also lint the keyboard's view files for design literals: a number +# inside a layout or motion modifier that isn't 0 or 1 belongs in tokens.json, +# not in the view. End the line with `// literal-ok: ` when it is +# genuinely local (a divide-by-two, a geometry derivation). +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)" +cd "$REPO_ROOT" + +CHECK=0 +if [ "${1:-}" = "--check" ]; then + CHECK=1 +elif [ -n "${1:-}" ]; then + echo "usage: scripts/design-sync.sh [--check]" >&2 + exit 2 +fi + +if command -v xcrun >/dev/null 2>&1 && xcrun --find swift-format >/dev/null 2>&1; then + SWIFT_FORMAT=(xcrun swift-format) +elif command -v swift-format >/dev/null 2>&1; then + SWIFT_FORMAT=(swift-format) +else + SWIFT_FORMAT=() +fi + +OUTPUTS=( + App/BlurtiOS/Shared/DesignTokens.swift + App/BlurtiOS/DESIGN.md + App/BlurtiOS/BlurtiOS/Assets.xcassets/AccentColor.colorset/Contents.json + App/BlurtiOS/BlurtiOS/Assets.xcassets/CardFill.colorset/Contents.json + App/BlurtiOS/BlurtiOS/Assets.xcassets/CardBorder.colorset/Contents.json + App/BlurtiOS/BlurtiOS/Assets.xcassets/Page.colorset/Contents.json + App/BlurtiOS/BlurtiOS/Assets.xcassets/Text.colorset/Contents.json + App/BlurtiOS/BlurtiOS/Assets.xcassets/Muted.colorset/Contents.json + App/BlurtiOS/BlurtiOS/Assets.xcassets/CTA.colorset/Contents.json + App/BlurtiOS/BlurtiOS/Assets.xcassets/CTAText.colorset/Contents.json +) + +# Format the generated files the way check.sh expects the tree to be formatted, +# so a regeneration never fails swift-format or prettier on its own output. +# `--ignore-path /dev/null`: .prettierignore excludes .build/, where --check +# writes, and an explicitly named ignored file is skipped, not formatted. +format_outputs() { + local root="$1" + if [ "${#SWIFT_FORMAT[@]}" -gt 0 ]; then + "${SWIFT_FORMAT[@]}" format --in-place "$root/App/BlurtiOS/Shared/DesignTokens.swift" + fi + if command -v prettier >/dev/null 2>&1; then + prettier --log-level warn --ignore-path /dev/null --write "$root/App/BlurtiOS/DESIGN.md" >/dev/null + fi +} + +# The design-literal lint. Scope: every file under the keyboard's and the +# app's sources that draws — new views are linted by default — minus the few +# that compute rather than draw (the host controller, the palette, the +# geometry) and Grain.swift, whose body is the grain's recipe (density, +# the two shades), with only its opacity as a token. Numbers 0 and 1 pass: `Spacer(minLength: 0)`, +# `.opacity(on ? 1 : 0)` and `scaleEffect(1 + …)` are structure, not design. +lint_literals() { + python3 - <<'PY' +import glob, os, re, sys +# BlurtiOSCore (the model, the voice state, the interaction rules) is not +# globbed at all: it draws nothing, so it has no design literals to carry. +exempt = { + "KeyboardViewController.swift", "KeyboardPalette.swift", "KeyGeometry.swift", "MotionHeld.swift", "Grain.swift", +} +app_exempt = { + "BlurtiOSApp.swift", "DictationCoordinator.swift", "ListeningWindow.swift", "ListeningSource.swift", + "AudioSource.swift", "SimulatorAudioSource.swift", "KeyboardGalleryView.swift", "KeyboardProbeView.swift", +} +files = sorted( + [path for path in glob.glob("App/BlurtiOS/BlurtKeyboard/Sources/**/*.swift", recursive=True) + if os.path.basename(path) not in exempt] + + [path for path in glob.glob("App/BlurtiOS/BlurtiOS/Sources/*.swift") if os.path.basename(path) not in app_exempt] +) +modifiers = re.compile( + r"\.(padding|frame|font|blur|scaleEffect|offset|shadow|opacity|cornerRadius|saturation|brightness" + r"|strokeBorder|stroke|lineWidth)\(|\b(spacing|duration|minimumDistance|lineWidth|radius|size|width|height|by):" +) +number = re.compile(r"(?/dev/null +format_outputs "$STAGE" +DRIFT=0 +for file in "${OUTPUTS[@]}"; do + if ! diff -u "$file" "$STAGE/$file" >"$STAGE/diff.txt" 2>&1; then + echo "error: $file is out of sync with App/BlurtiOS/Design/tokens.json:" >&2 + head -40 "$STAGE/diff.txt" >&2 + DRIFT=1 + fi +done +if [ "$DRIFT" -ne 0 ]; then + echo " run scripts/design-sync.sh and commit the result" >&2 + exit 1 +fi +echo "design-sync: ${#OUTPUTS[@]} generated files match tokens.json" +lint_literals diff --git a/scripts/design-tokens.swift b/scripts/design-tokens.swift new file mode 100755 index 00000000..b0109256 --- /dev/null +++ b/scripts/design-tokens.swift @@ -0,0 +1,451 @@ +#!/usr/bin/env swift // The design's one source of numbers and colours, fanned out: reads // App/BlurtiOS/Design/tokens.json (hand-edited: it is the // source, not an export) and writes everything that used to repeat those values by // hand — `App/BlurtiOS/Shared/DesignTokens.swift`, the generated tables in // `App/BlurtiOS/DESIGN.md`, and the app's three asset-catalog colour sets. Run +// through scripts/design-sync.sh, which also formats the output and, with +// --check, fails on drift the way check.sh fails on a stale .pbxproj. +// +// swift scripts/design-tokens.swift [--root ] +// +// Inputs are always read from the repo; `--root` redirects the outputs (the +// drift check writes into .build/ and diffs). A `swift` script run from disk +// can't import the app, so this is plain Foundation. +// +// Token values: a hex string ("#01762F"), a number, a weight name, or an alias +// to another token in braces ("{brand.green/700}", the DTCG spelling). A value +// may be wrapped as {"value": …, "use": "one line for DESIGN.md"}. Swift names +// derive from token names by one rule — `kb/key-modifier` → `kbKeyModifier`, +// `green/700` → `green700`. + +import Foundation + +// MARK: - Arguments and paths + +let scriptURL = URL(fileURLWithPath: CommandLine.arguments[0]).standardizedFileURL +let repoRoot = scriptURL.deletingLastPathComponent().deletingLastPathComponent() +var outputRoot = repoRoot +var arguments = Array(CommandLine.arguments.dropFirst()) +while !arguments.isEmpty { + let flag = arguments.removeFirst() + switch flag { + case "--root": + guard !arguments.isEmpty else { fail("--root needs a directory") } + outputRoot = URL(fileURLWithPath: arguments.removeFirst()).standardizedFileURL + default: + fail("unknown argument \(flag)") + } +} + +let tokensPath = "App/BlurtiOS/Design/tokens.json" +let swiftPath = "App/BlurtiOS/Shared/DesignTokens.swift" +let designPath = "App/BlurtiOS/DESIGN.md" +let catalogPath = "App/BlurtiOS/BlurtiOS/Assets.xcassets" + +func fail(_ message: String) -> Never { + FileHandle.standardError.write(Data("design-tokens: \(message)\n".utf8)) + exit(1) +} + +// MARK: - The model + +enum Value { + case color(String) // "#RRGGBB" + case number(Double) + case word(String) // a font weight + case curve([Double]) // a cubic-bezier easing: four numbers + case alias(String) // "group.name" +} + +struct Token { + var group: String + var name: String + var value: Value + var use: String? + + var qualified: String { "\(group).\(name)" } + var swiftName: String { camel(name) } +} + +struct Stop { + let color: String // alias or hex + let location: Double +} + +struct Gradient { + let name: String + let use: String? + let start: String + let end: String + let stops: [Stop] +} + +/// `kb/key-modifier` → `kbKeyModifier`; `green/700` → `green700`. +func camel(_ name: String) -> String { + let parts = name.split(whereSeparator: { "/-.".contains($0) }).map(String.init) + guard let first = parts.first else { return name } + return ([first.lowercased()] + parts.dropFirst().map { $0.prefix(1).uppercased() + $0.dropFirst() }).joined() +} + +/// A group name to its Swift enum: `brand` → `Brand`. `type` becomes +/// `Typography`, since a member named `Type` is not allowed in Swift. +func typeName(_ group: String) -> String { + group == "type" ? "Typography" : group.prefix(1).uppercased() + group.dropFirst() +} + +func parseValue(_ raw: Any) -> Value { + if let number = raw as? NSNumber, CFGetTypeID(number) != CFBooleanGetTypeID() { + return .number(number.doubleValue) + } + if let numbers = raw as? [NSNumber] { + guard numbers.count == 4 else { fail("a curve is four numbers, got \(numbers)") } + return .curve(numbers.map(\.doubleValue)) + } + guard let string = raw as? String else { fail("a token value must be a string or a number, got \(raw)") } + if string.hasPrefix("{"), string.hasSuffix("}") { return .alias(String(string.dropFirst().dropLast())) } + if string.hasPrefix("#") { + guard string.count == 7, string.dropFirst().allSatisfy(\.isHexDigit) else { fail("bad colour \(string)") } + return .color(string.uppercased()) + } + return .word(string) +} + +// MARK: - Read + +let tokensURL = repoRoot.appendingPathComponent(tokensPath) +guard let data = FileManager.default.contents(atPath: tokensURL.path), + let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any] +else { fail("could not read \(tokensPath)") } + +let groups = ["brand", "themes", "keyboard", "metrics", "type", "motion"] +var tokens: [Token] = [] +var byQualified: [String: Token] = [:] +for group in groups { + guard let entries = json[group] as? [String: Any] else { fail("tokens.json has no \(group) group") } + for name in entries.keys.sorted() { + let entry = entries[name]! + let token: Token + if let wrapped = entry as? [String: Any] { + guard let raw = wrapped["value"] else { fail("\(group).\(name) has no value") } + token = Token(group: group, name: name, value: parseValue(raw), use: wrapped["use"] as? String) + } else { + token = Token(group: group, name: name, value: parseValue(entry), use: nil) + } + tokens.append(token) + byQualified[token.qualified] = token + } +} + +var gradients: [Gradient] = [] +if let entries = json["gradients"] as? [String: Any] { + for name in entries.keys.sorted() { + guard let entry = entries[name] as? [String: Any], let start = entry["start"] as? String, + let end = entry["end"] as? String, let stops = entry["stops"] as? [[String: Any]] + else { fail("gradients.\(name) needs start, end and stops") } + gradients.append( + Gradient( + name: name, use: entry["use"] as? String, start: start, end: end, + stops: stops.map { stop in + guard let color = stop["color"] as? String, let location = stop["location"] as? NSNumber else { + fail("gradients.\(name) has a stop without color and location") + } + return Stop(color: color, location: location.doubleValue) + })) + } +} + +// MARK: - Resolve + +/// Follows aliases to the concrete token, refusing cycles and dangling names. +func resolve(_ token: Token, seen: [String] = []) -> Token { + guard case .alias(let target) = token.value else { return token } + guard !seen.contains(target) else { fail("alias cycle at \(token.qualified)") } + guard let next = byQualified[target] else { fail("\(token.qualified) aliases \(target), which does not exist") } + return resolve(next, seen: seen + [token.qualified]) +} + +func resolveColor(_ reference: String) -> String { + let value = parseValue(reference) + switch value { + case .color(let hex): return hex + case .alias(let target): + guard let token = byQualified[target] else { fail("\(reference) does not exist") } + guard case .color(let hex) = resolve(token).value else { fail("\(reference) is not a colour") } + return hex + default: fail("\(reference) is not a colour") + } +} + +func format(_ number: Double) -> String { + if number == number.rounded(), abs(number) < 1e9 { return String(Int(number)) } + return String(number) +} + +/// The value as the manifest and the docs print it. +func rendered(_ token: Token) -> String { + switch resolve(token).value { + case .color(let hex): return hex + case .number(let number): return format(number) + case .word(let word): return word + case .curve(let numbers): return numbers.map(format).joined(separator: ",") + case .alias: fatalError("unreachable") + } +} + +for token in tokens { _ = resolve(token) } // every alias must land + +// MARK: - Swift + +let weights = ["ultraLight", "thin", "light", "regular", "medium", "semibold", "bold", "heavy", "black"] + +func swiftLiteral(_ token: Token) -> (type: String, literal: String) { + switch token.value { + case .color(let hex): return ("Color", "Color(hex: 0x\(hex.dropFirst()))") + case .number(let number): + return (token.group == "motion" ? "Double" : "CGFloat", format(number)) + case .word(let word): + guard weights.contains(word) else { fail("\(token.qualified): \(word) is not a font weight") } + return ("Font.Weight", ".\(word)") + case .curve(let numbers): + let n = numbers.map(format) + return ( + "UnitCurve", + "UnitCurve.bezier(startControlPoint: UnitPoint(x: \(n[0]), y: \(n[1])), " + + "endControlPoint: UnitPoint(x: \(n[2]), y: \(n[3])))" + ) + case .alias(let target): + guard let other = byQualified[target] else { fail("\(target) does not exist") } + return (swiftLiteral(other).type, "\(typeName(other.group)).\(other.swiftName)") + } +} + +func swiftColorReference(_ reference: String) -> String { + let value = parseValue(reference) + switch value { + case .color(let hex): return "Color(hex: 0x\(hex.dropFirst()))" + case .alias(let target): + guard let other = byQualified[target] else { fail("\(target) does not exist") } + return "\(typeName(other.group)).\(other.swiftName)" + default: fail("\(reference) is not a colour") + } +} + +func docComment(_ text: String?, indent: String) -> String { + guard let text, !text.isEmpty else { return "" } + return "\(indent)/// \(text.prefix(1).uppercased() + text.dropFirst()).\n" +} + +var swift = """ + // Generated by scripts/design-tokens.swift from App/BlurtiOS/Design/tokens.json — do not edit. + // Regenerate with scripts/design-sync.sh; scripts/check.sh fails on drift. + // periphery:ignore:all + // swiftlint:disable file_length + import SwiftUI + + // swiftlint:disable type_body_length + /// The design's colours and numbers, from their one source + /// (`Design/tokens.json`): what the views draw with, what DESIGN.md's tables + /// say, and what the asset catalog holds, all from the same file. Swift names + /// follow the token names by one rule (`kb/key-modifier` → `kbKeyModifier`). + enum DesignTokens { + + """ + +for group in groups { + swift += " enum \(typeName(group)) {\n" + for token in tokens where token.group == group { + let (type, literal) = swiftLiteral(token) + let use = token.use.map { $0 } ?? (token.value.isAlias ? "`\(aliasTarget(token))`" : nil) + swift += docComment(use, indent: " ") + swift += " nonisolated static let \(token.swiftName): \(type) = \(literal)\n" + } + swift += " }\n\n" +} + +swift += " enum Gradients {\n" +for gradient in gradients { + swift += docComment(gradient.use, indent: " ") + swift += " nonisolated static let \(camel(gradient.name)) = LinearGradient(\n stops: [\n" + for stop in gradient.stops { + swift += " .init(color: \(swiftColorReference(stop.color)), location: \(format(stop.location))),\n" + } + swift += " ],\n startPoint: .\(gradient.start), endPoint: .\(gradient.end))\n" +} +swift += " }\n\n" + +swift += """ + /// Every token as `group.name: value`, the way tokens.json renders it, so a + /// test can pin the compiled values to the file (`DesignTokensTests`). + nonisolated static let manifest: [String: String] = [ + + """ +for token in tokens { + swift += " \"\(token.qualified)\": \"\(rendered(token))\",\n" +} +for gradient in gradients { + let stops = gradient.stops.map { "\(resolveColor($0.color))@\(format($0.location))" }.joined(separator: " ") + swift += " \"gradients.\(gradient.name)\": \"\(stops) \(gradient.start)>\(gradient.end)\",\n" +} +swift += " ]\n}\n// swiftlint:enable type_body_length\n" + +extension Value { + var isAlias: Bool { + if case .alias = self { return true } + return false + } +} + +func aliasTarget(_ token: Token) -> String { + if case .alias(let target) = token.value { return target } + return "" +} + +// MARK: - DESIGN.md + +func table(_ header: [String], _ rows: [[String]]) -> String { + var lines = ["| " + header.joined(separator: " | ") + " |"] + lines.append("| " + header.map { _ in "---" }.joined(separator: " | ") + " |") + for row in rows { lines.append("| " + row.joined(separator: " | ") + " |") } + return lines.joined(separator: "\n") +} + +func code(_ text: String) -> String { "`\(text)`" } + +func tokenRows(_ group: String, valueLabel: String) -> String { + let rows = tokens.filter { $0.group == group }.map { token -> [String] in + var value = code(rendered(token)) + if case .alias(let target) = token.value { value += " (" + code(target) + ")" } + return [code(token.name), value, code("\(typeName(group)).\(token.swiftName)"), token.use ?? "—"] + } + return table(["Token", valueLabel, "Swift", "Use"], rows) +} + +var blocks: [String: String] = [:] +blocks["brand"] = tokenRows("brand", valueLabel: "Value") +blocks["keyboard"] = tokenRows("keyboard", valueLabel: "Value") +blocks["metrics"] = tokenRows("metrics", valueLabel: "Points") +blocks["type"] = tokenRows("type", valueLabel: "Value") +blocks["motion"] = tokenRows("motion", valueLabel: "Value") + +let themeNames = tokens.filter { $0.group == "themes" }.map { String($0.name.split(separator: "/")[0]) } +let themeOrder = ["light", "dark"] + Set(themeNames).subtracting(["light", "dark"]).sorted() +let roles = [ + "surface", "key", "key-modifier", "legend", "legend-secondary", "signal", "popup", "field", "field-border", "notice", + "container-key", "container-key-modifier", "material", +] +blocks["themes"] = table( + [ + "Face", "Surface", "Key", "Modifier", "Legend", "Secondary", "Signal", "Pop-up", "Field", "Field border", "Notice", + "Key in container", "Modifier in container", "Host material", + ], + themeOrder.compactMap { theme -> [String]? in + let cells = roles.compactMap { role in byQualified["themes.\(theme)/\(role)"].map { code(rendered($0)) } } + guard cells.count == roles.count else { return nil } + return [code(theme)] + cells + }) + +blocks["gradients"] = gradients.map { gradient in + let stops = gradient.stops.map { "\(code(resolveColor($0.color))) at \(format($0.location))" } + .joined(separator: ", ") + return + "- \(code(gradient.name)) (\(code("Gradients.\(camel(gradient.name))")), \(gradient.start) → \(gradient.end)): \(stops)" + + (gradient.use.map { " — \($0)" } ?? "") +}.joined(separator: "\n") + +let designURL = repoRoot.appendingPathComponent(designPath) +guard var design = try? String(contentsOf: designURL, encoding: .utf8) else { fail("could not read \(designPath)") } +var replaced: Set = [] +for (name, body) in blocks { + let begin = "" + let end = "" + guard let beginRange = design.range(of: begin), + let endRange = design.range(of: end, range: beginRange.upperBound.. String { + // Xcode's own layout, byte for byte: eight spaces in front of the colour's + // keys, ten in front of its components. + func components(_ hex: String) -> String { + let digits = Array(hex.dropFirst()) + let red = String(digits[0...1]) + let green = String(digits[2...3]) + let blue = String(digits[4...5]) + return [ + " \"color-space\" : \"srgb\",", + " \"components\" : {", + " \"alpha\" : \"1.000\",", + " \"blue\" : \"0x\(blue)\",", + " \"green\" : \"0x\(green)\",", + " \"red\" : \"0x\(red)\"", + " }", + ].joined(separator: "\n") + } + return """ + { + "colors" : [ + { + "color" : { + \(components(light)) + }, + "idiom" : "universal" + }, + { + "appearances" : [ + { + "appearance" : "luminosity", + "value" : "dark" + } + ], + "color" : { + \(components(dark)) + }, + "idiom" : "universal" + } + ], + "info" : { + "author" : "xcode", + "version" : 1 + } + } + + """ +} + +let colorSets: [(String, String, String)] = [ + ("AccentColor", "keyboard.app/accent-light", "keyboard.app/accent-dark"), + ("CardFill", "keyboard.app/card-fill-light", "keyboard.app/card-fill-dark"), + ("CardBorder", "keyboard.app/card-border-light", "keyboard.app/card-border-dark"), + ("Page", "keyboard.app/page-light", "keyboard.app/page-dark"), + ("Text", "keyboard.app/text-light", "keyboard.app/text-dark"), + ("Muted", "keyboard.app/muted-light", "keyboard.app/muted-dark"), + ("CTA", "keyboard.app/cta-light", "keyboard.app/cta-dark"), + ("CTAText", "keyboard.app/cta-text-light", "keyboard.app/cta-text-dark"), +] + +// MARK: - Write + +func write(_ text: String, to relative: String) { + let url = outputRoot.appendingPathComponent(relative) + do { + try FileManager.default.createDirectory(at: url.deletingLastPathComponent(), withIntermediateDirectories: true) + try text.write(to: url, atomically: true, encoding: .utf8) + } catch { + fail("could not write \(relative): \(error)") + } +} + +write(swift, to: swiftPath) +write(design, to: designPath) +for (set, light, dark) in colorSets { + guard let lightToken = byQualified[light], let darkToken = byQualified[dark] else { + fail("\(set) needs \(light) and \(dark)") + } + write( + catalog(light: rendered(lightToken), dark: rendered(darkToken)), to: "\(catalogPath)/\(set).colorset/Contents.json") +} +print( + "design-tokens: \(tokens.count) tokens, \(gradients.count) gradients → \(swiftPath), \(designPath) " + + "(\(replaced.count) blocks), \(colorSets.count) colour sets") diff --git a/scripts/ios-check.sh b/scripts/ios-check.sh new file mode 100755 index 00000000..cbb796c6 --- /dev/null +++ b/scripts/ios-check.sh @@ -0,0 +1,91 @@ +#!/bin/bash +# The iPhone code's quality gates, in one place — what check.sh is for the Mac. +# +# scripts/ios-check.sh +# BLURT_SIM_DEVICE="iPhone 17" scripts/ios-check.sh +# +# The same bar the engine and the Mac shell meet in check.sh, applied to +# App/BlurtiOS: +# 1. BlurtiOSTests on a simulator, warnings as errors, with BlurtiOSCore's +# line-coverage gate (ios-test.sh; the TSan and ASan passes are its +# BLURT_IOS_SANITIZER runs, CI's ios-sanitizers job). +# 2. `swiftlint analyze` — unused imports — over that build's compiler log, +# as check.sh runs it over the Mac app's. +# 3. periphery over the iPhone project — unused declarations. +# 4. The probe's XCUITest bundle (BlurtiOSProbe), compiled: its own scheme, +# run only by hand (the screenshot and keyboard-flow scripts), so nothing +# else ever builds it — and it went uncompiled through a Swift 6 change. +# swift-format, `swiftlint lint` and the invariants already cover App/BlurtiOS +# from check.sh, which reads every tracked Swift file. +# +# CI's ios-build job runs this; check.sh runs it too on a Mac with an iPhone +# simulator, so a local green means the same thing on both platforms. Like +# check.sh, every step reports, so one failure doesn't hide the next. +# +# `swiftlint analyze` sees the files the build actually compiled. CI's build is +# from scratch, so it sees everything; a local incremental one may not — +# delete .build/ios-sim for a full local pass, as with check.sh's app build. +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)" +cd "$REPO_ROOT" +# shellcheck source=scripts/release-lib.sh +source "$REPO_ROOT/scripts/release-lib.sh" +# shellcheck source=scripts/ios-lib.sh +source "$REPO_ROOT/scripts/ios-lib.sh" + +require_tools --hint='scripts/bootstrap.sh installs it' swiftlint periphery xcodegen + +FAILED=() +BUILD_LOG="$(mktemp -t blurt-ios-build)" +APP_LOG="$(mktemp -t blurt-ios-build-app)" +trap 'rm -f "$BUILD_LOG" "$APP_LOG"' EXIT + +echo "==> ios-test (BlurtiOSTests + BlurtiOSCore coverage gate)" +BLURT_IOS_BUILD_LOG="$BUILD_LOG" "$REPO_ROOT/scripts/ios-test.sh" || FAILED+=("ios-test") + +echo "==> swiftlint analyze (unused imports, App/BlurtiOS)" +if [ -s "$BUILD_LOG" ]; then + # The iPhone modules' compile commands only. The build also compiles the + # engine — for iOS, where its `#if os(macOS)` files are inactive and + # SourceKit can't resolve them — and check.sh analyzes the engine from the + # Mac build, where they are live. + grep -v -- '-module-name BlurtEngine ' "$BUILD_LOG" >"$APP_LOG" || true + swiftlint analyze --strict --quiet --compiler-log-path "$APP_LOG" App/BlurtiOS || FAILED+=("swiftlint analyze") +else + echo "error: no build log to analyze — the build above did not run" >&2 + FAILED+=("swiftlint analyze") +fi + +echo "==> build-for-testing (BlurtiOSProbe)" +# The tests' destination, flags and derived data, so the build above is +# reused and only the probe's own files compile (a generic destination would +# rebuild everything for a second architecture). +UDID="$(ios_pick_device "${BLURT_SIM_DEVICE:-iPhone 18 Pro}" | cut -f1)" +if [ -n "$UDID" ]; then + xcodebuild build-for-testing -project App/BlurtiOS/BlurtiOS.xcodeproj -scheme BlurtiOSProbe \ + -destination "platform=iOS Simulator,id=$UDID" -derivedDataPath "$IOS_DERIVED" \ + "${IOS_SIM_SIGNING[@]}" -enableCodeCoverage YES -quiet || FAILED+=("build-for-testing (BlurtiOSProbe)") +else + echo "error: no iPhone simulator to build the probe for" >&2 + FAILED+=("build-for-testing (BlurtiOSProbe)") +fi + +echo "==> periphery (App/BlurtiOS)" +# Its own build, for a device: the tests' simulator index would report the +# phone's capture path (WindowedAudioSource) as unused, since the simulator +# builds SimulatorAudioSource in its place. `--retain-public`: the engine's +# public API is used by the Mac app this scan can't see, and the engine's own +# findings are excluded from the report, so only the iPhone code's dead +# declarations are reported. BlurtiOSCore's API is `package`, not public, so +# it is still scanned. +periphery scan --project App/BlurtiOS/BlurtiOS.xcodeproj --schemes BlurtiOS \ + --retain-public --retain-unused-protocol-func-params \ + --report-exclude 'Sources/**' --strict --quiet || FAILED+=("periphery") + +if [ "${#FAILED[@]}" -gt 0 ]; then + echo "error: ${#FAILED[@]} iPhone check(s) failed:" >&2 + printf ' %s\n' "${FAILED[@]}" >&2 + exit 1 +fi +echo "==> ios-check: ok" diff --git a/scripts/ios-keyboard-flows.sh b/scripts/ios-keyboard-flows.sh new file mode 100755 index 00000000..04415538 --- /dev/null +++ b/scripts/ios-keyboard-flows.sh @@ -0,0 +1,149 @@ +#!/bin/bash +# Drive the real Blurt keyboard through its flows in the simulator — the +# extension in its own process over a field in the app, with the app +# listening — for each face × mic concept × layout, and collect a screenshot +# of every step. The truth about the keyboard as used, which the gallery +# (the app's process, no host, no touches) cannot give. +# +# scripts/ios-keyboard-flows.sh # the whole matrix +# scripts/ios-keyboard-flows.sh --face dark --voice a --layout panel +# scripts/ios-keyboard-flows.sh --layout full --align right # the mic at an edge (Settings › alignment) +# +# The probe app sets the layout, mic concept and side in the App Group for +# each run; a run with --align puts the side back to the middle at the end, +# so Neil's own keyboard on this simulator never inherits a probe's setting. +# +# Prints one FLOW-OK / FLOW-FAIL line per step; screenshots land in +# .build/design/flows/--/, with a sheet.png of every +# step's keyboard side by side. Blurt must be enabled as a keyboard on the +# simulator once by hand. +set -uo pipefail + +REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)" +cd "$REPO_ROOT" || exit 1 +# shellcheck source=scripts/ios-lib.sh +source "$REPO_ROOT/scripts/ios-lib.sh" +DERIVED="$IOS_DERIVED" +FACES="light dark" +VOICES="a b c" +LAYOUTS="panel full slimBar" +ALIGN="" +OUT="$REPO_ROOT/.build/design/flows" +while [ $# -gt 0 ]; do + case "$1" in + --face) + FACES="${2:?}" + shift 2 + ;; + --voice) + VOICES="${2:?}" + shift 2 + ;; + --layout) + LAYOUTS="${2:?}" + shift 2 + ;; + --out) + OUT="${2:?}" + shift 2 + ;; + --align) + ALIGN="${2:?}" + shift 2 + ;; + *) + echo "usage: ios-keyboard-flows.sh [--face light|dark] [--voice a|b|c] [--layout slimBar|panel|full] [--align left|right] [--out DIR]" >&2 + exit 2 + ;; + esac +done + +# The runs whose XCUITest failed. The matrix carries on — every other run's +# screenshots are still worth having — and the script exits non-zero at the end. +FAILED_RUNS=() + +PICKED="$(ios_pick_device "${BLURT_SIM_DEVICE:-iPhone 18 Pro}")" +[ -n "$PICKED" ] || { + echo "ios-keyboard-flows: no iPhone simulator available" >&2 + exit 1 +} +UDID="${PICKED%% *}" +ios_wait_booted "$UDID" + +(cd "$REPO_ROOT/App/BlurtiOS" && xcodegen generate --quiet) +xcodebuild build -project "$REPO_ROOT/App/BlurtiOS/BlurtiOS.xcodeproj" -scheme BlurtiOS \ + -destination "platform=iOS Simulator,id=$UDID" -derivedDataPath "$DERIVED" "${IOS_SIM_SIGNING[@]}" -quiet +xcrun simctl install "$UDID" "$DERIVED/Build/Products/Debug-iphonesimulator/BlurtiOS.app" +xcrun simctl privacy "$UDID" grant microphone dev.alex.blurt.ios + +for face in $FACES; do + xcrun simctl ui "$UDID" appearance "$face" + for voice in $VOICES; do + for layout in $LAYOUTS; do + name="$face-$voice-$layout${ALIGN:+-$ALIGN}" + dir="$OUT/$name" + rm -rf "$dir" + mkdir -p "$dir" + echo "==> $name" + TEST_RUNNER_BLURT_PROBE_ARGS="$face $voice $layout${ALIGN:+ $ALIGN}" xcodebuild test \ + -project "$REPO_ROOT/App/BlurtiOS/BlurtiOS.xcodeproj" -scheme BlurtiOSProbe \ + -only-testing:BlurtiOSProbe/BlurtKeyboardFlows \ + -destination "platform=iOS Simulator,id=$UDID" -derivedDataPath "$DERIVED" \ + -resultBundlePath "$dir/flows.xcresult" "${IOS_SIM_SIGNING[@]}" -collect-test-diagnostics never \ + 2>&1 | grep -E "^FLOW-|XCTAssertTrue failed|Blurt never came up|error:|TEST BUILD FAILED" | grep -v "XCTAssertTrue failed" | sed "s/^/ /" + # xcodebuild's own verdict, read before any other command replaces it. + if [ "${PIPESTATUS[0]}" -ne 0 ]; then FAILED_RUNS+=("$name"); fi + xcrun xcresulttool export attachments --path "$dir/flows.xcresult" --output-path "$dir/attachments" >/dev/null 2>&1 + python3 - "$dir" <<'PY' +import json, os, shutil, sys +d = sys.argv[1] +try: + manifest = json.load(open(os.path.join(d, "attachments", "manifest.json"))) +except FileNotFoundError: + sys.exit(0) +for test in manifest: + for a in test.get("attachments", []): + name = a["suggestedHumanReadableName"] + if name.startswith("flow-"): + shutil.copy(os.path.join(d, "attachments", a["exportedFileName"]), os.path.join(d, name.split("_")[0] + ".png")) +PY + # The sheet: the bottom of every step's screenshot — the field is at + # the top, the keyboard at the bottom — captioned with the step. + captions=() + for shot in "$dir"/flow-*.png; do + [ -e "$shot" ] || continue + case "$shot" in *-crop.png) continue ;; esac + crop="${shot%.png}-crop.png" + h=$(sips -g pixelHeight "$shot" | awk '/pixelHeight/ {print $2}') + w=$(sips -g pixelWidth "$shot" | awk '/pixelWidth/ {print $2}') + ch=$((h * 45 / 100)) + # sips wants the offset plus the height strictly inside the image. + sips -c "$ch" "$w" --cropOffset $((h - ch - 1)) 0 "$shot" --out "$crop" >/dev/null + step=$(basename "$shot" .png) + captions+=("${step#flow-??-}=$crop") + done + if [ ${#captions[@]} -gt 0 ]; then + swift "$REPO_ROOT/scripts/design-diff.swift" sheet --out "$dir/sheet.png" --columns 3 --scale 0.5 \ + "${captions[@]}" >/dev/null + fi + done + done +done +xcrun simctl ui "$UDID" appearance light +echo "ios-keyboard-flows: screenshots in $OUT" + +# An --align run leaves the App Group's side where the probe set it: put it +# back through the simulator's own defaults daemon (a plist edit alone is +# not seen until cfprefsd reloads). +if [ -n "$ALIGN" ]; then + GROUP="$(xcrun simctl get_app_container "$UDID" dev.alex.blurt.ios groups 2>/dev/null | awk -F'\t' '/group.dev.alex.blurt/ {print $2}')" + if [ -n "$GROUP" ]; then + xcrun simctl spawn "$UDID" defaults write "$GROUP/Library/Preferences/group.dev.alex.blurt" micAlignment center + echo "ios-keyboard-flows: mic alignment back to the middle" + fi +fi + +if [ "${#FAILED_RUNS[@]}" -gt 0 ]; then + echo "ios-keyboard-flows: ${#FAILED_RUNS[@]} run(s) failed: ${FAILED_RUNS[*]}" >&2 + exit 1 +fi diff --git a/scripts/ios-keyboard-shot.sh b/scripts/ios-keyboard-shot.sh new file mode 100755 index 00000000..b29dad4a --- /dev/null +++ b/scripts/ios-keyboard-shot.sh @@ -0,0 +1,100 @@ +#!/bin/bash +# A screenshot of the real Blurt keyboard — the extension in its own process, +# over a text field in the app — as the gallery cannot take one: what renders +# in the app's process (the gallery, the home hero) can differ from what the +# extension may draw. The BlurtiOSProbe UI test launches the app on +# -BlurtProbeField, which sets the face, the mic concept and the layout in the +# App Group, taps the globe until Blurt's keyboard is up, and attaches the +# screen; this exports it. +# +# scripts/ios-keyboard-shot.sh # light, the shipped mic, the saved layout +# scripts/ios-keyboard-shot.sh --face dark --voice b --layout panel --out .build/design/ext/panel-b.png +# +# Blurt must be enabled as a keyboard on the simulator once by hand +# (Settings → General → Keyboard → Keyboards → Add New Keyboard → Blurt). +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)" +cd "$REPO_ROOT" +# shellcheck source=scripts/ios-lib.sh +source "$REPO_ROOT/scripts/ios-lib.sh" +DERIVED="$IOS_DERIVED" +FACE=light +VOICE="" +LAYOUT="" +OUT="" +while [ $# -gt 0 ]; do + case "$1" in + --face) + FACE="${2:?}" + shift 2 + ;; + --voice) + VOICE="${2:?}" + shift 2 + ;; + --layout) + LAYOUT="${2:?}" + shift 2 + ;; + --out) + OUT="${2:?}" + shift 2 + ;; + *) + echo "usage: ios-keyboard-shot.sh [--face light|dark] [--voice a|b|c] [--layout slimBar|panel|full] [--out FILE.png]" >&2 + exit 2 + ;; + esac +done +[ -n "$OUT" ] || OUT="$REPO_ROOT/.build/design/ext/keyboard-$FACE${VOICE:+-$VOICE}${LAYOUT:+-$LAYOUT}.png" +mkdir -p "$(dirname "$OUT")" + +PICKED="$(ios_pick_device "${BLURT_SIM_DEVICE:-iPhone 18 Pro}")" +[ -n "$PICKED" ] || { + echo "ios-keyboard-shot: no iPhone simulator available" >&2 + exit 1 +} +UDID="${PICKED%% *}" +ios_wait_booted "$UDID" +WORK="$(mktemp -d)" +trap 'rm -rf "$WORK"' EXIT + +(cd "$REPO_ROOT/App/BlurtiOS" && xcodegen generate --quiet) +# `xcodebuild test` does not reinstall a host app it thinks it has installed; +# the keyboard is inside it, so the last build is put on the simulator first. +xcodebuild build -project "$REPO_ROOT/App/BlurtiOS/BlurtiOS.xcodeproj" -scheme BlurtiOS \ + -destination "platform=iOS Simulator,id=$UDID" -derivedDataPath "$DERIVED" "${IOS_SIM_SIGNING[@]}" -quiet +xcrun simctl install "$UDID" "$DERIVED/Build/Products/Debug-iphonesimulator/BlurtiOS.app" +# xcodebuild hands the test runner only the environment variables prefixed +# TEST_RUNNER_; the test reads BLURT_PROBE_ARGS. +if ! TEST_RUNNER_BLURT_PROBE_ARGS="$FACE $VOICE $LAYOUT" xcodebuild test -project "$REPO_ROOT/App/BlurtiOS/BlurtiOS.xcodeproj" \ + -scheme BlurtiOSProbe -only-testing:BlurtiOSProbe/BlurtKeyboardShot \ + -destination "platform=iOS Simulator,id=$UDID" -derivedDataPath "$DERIVED" \ + -resultBundlePath "$WORK/shot.xcresult" "${IOS_SIM_SIGNING[@]}" -collect-test-diagnostics never -quiet; then + # Say why, from the result bundle, before it goes. + xcrun xcresulttool get test-results tests --path "$WORK/shot.xcresult" 2>/dev/null | python3 -c ' +import json, sys +def walk(node): + if isinstance(node, dict): + if node.get("nodeType") == "Failure Message": print("ios-keyboard-shot:", node.get("name")) + for child in node.get("children", []): walk(child) + elif isinstance(node, list): + for child in node: walk(child) +walk(json.load(sys.stdin).get("testNodes", [])) +' || true + exit 1 +fi +xcrun xcresulttool export attachments --path "$WORK/shot.xcresult" --output-path "$WORK/attachments" >/dev/null +python3 - "$WORK/attachments" "$OUT" <<'PY' +import json, os, shutil, sys +attachments, out = sys.argv[1], sys.argv[2] +manifest = json.load(open(os.path.join(attachments, "manifest.json"))) +for test in manifest: + for a in test.get("attachments", []): + if a["suggestedHumanReadableName"].startswith("blurt-keyboard"): + shutil.copy(os.path.join(attachments, a["exportedFileName"]), out) + print(f"ios-keyboard-shot: {out}") + sys.exit(0) +sys.exit("ios-keyboard-shot: no screenshot in the result bundle") +PY diff --git a/scripts/ios-lib.sh b/scripts/ios-lib.sh new file mode 100755 index 00000000..726d584f --- /dev/null +++ b/scripts/ios-lib.sh @@ -0,0 +1,47 @@ +#!/bin/bash +# Shared by scripts/ios-*.sh: the one simulator signing recipe, and picking a +# simulator. Sourced, not run. Needs python3 (the CLT stub on a bare Mac). +# +# Ad hoc signing (identity "-") is what the simulator accepts without a team, +# and unlike CODE_SIGNING_ALLOWED=NO it embeds the entitlements, so the App +# Group the app and keyboard share works in the simulator too. +# shellcheck disable=SC2034 # used by the scripts that source this file +IOS_SIM_SIGNING=(CODE_SIGN_STYLE=Manual CODE_SIGN_IDENTITY=- CODE_SIGNING_REQUIRED=NO DEVELOPMENT_TEAM=) + +# The iPhone build's derived data, shared so each script's build reuses the +# last one's (ios-check.sh's probe build relies on it). Needs REPO_ROOT set by +# the sourcing script; BLURT_DERIVED_DATA moves it (CI puts it in RUNNER_TEMP). +# shellcheck disable=SC2034 # used by the scripts that source this file +IOS_DERIVED="${BLURT_DERIVED_DATA:-$REPO_ROOT/.build/ios-sim}" + +# ios_pick_device NAME → prints "udidname" for the available simulator +# called NAME on the newest iOS runtime, else for whichever iPhone this Mac +# (or CI runner) has; prints nothing if there is none. +ios_pick_device() { + xcrun simctl list devices available -j | python3 -c ' +import json, re, sys +wanted = sys.argv[1] +runtimes = json.load(sys.stdin)["devices"] +def version(rt): + m = re.search(r"iOS-(\d+)-(\d+)", rt) + return (int(m.group(1)), int(m.group(2))) if m else (0, 0) +found = [] +for rt, devs in runtimes.items(): + if "iOS" not in rt: + continue + for d in devs: + if d["name"].startswith("iPhone"): + found.append((version(rt), d["name"] == wanted, d["name"], d["udid"])) +if not found: + sys.exit(0) +found.sort(key=lambda f: (f[1], f[0])) +best = found[-1] +print(best[3] + "\t" + best[2]) +' "$1" +} + +# ios_wait_booted UDID → boots the simulator if needed and waits for it. +ios_wait_booted() { + xcrun simctl boot "$1" 2>/dev/null || true + xcrun simctl bootstatus "$1" -b >/dev/null +} diff --git a/scripts/ios-record.sh b/scripts/ios-record.sh new file mode 100755 index 00000000..f9437c1c --- /dev/null +++ b/scripts/ios-record.sh @@ -0,0 +1,109 @@ +#!/bin/bash +# Record the keyboard's motion from the simulator: the gallery's `live` state +# walks a whole dictation on a clock (about 11.7 s a cycle), this records two +# cycles and cuts one clean loop, cropped to the keyboard, for the review packet +# (DESIGN.md › Reproducing it). +# +# scripts/ios-record.sh # panel, dark → .build/design/loops/panel-dark.mp4 +# scripts/ios-record.sh --layout slimBar --theme light --out ~/Desktop/blurt-review +# scripts/ios-record.sh --frames 0,0.35,0.7 # also grab stills at these seconds into the loop +# +# Needs ffmpeg (Brewfile). The crop comes from a still capture of the same +# layout taken first, so the video's frame is the keyboard's own pixels. +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)" +cd "$REPO_ROOT" +# shellcheck source=scripts/ios-lib.sh +source "$REPO_ROOT/scripts/ios-lib.sh" + +LAYOUT=panel +THEME=dark +OUT="$REPO_ROOT/.build/design/loops" +FRAMES="" +VOICE="" +CYCLE=11.7 +BUILD=() +while [ $# -gt 0 ]; do + case "$1" in + --layout) + LAYOUT="${2:?}" + shift 2 + ;; + --theme) + THEME="${2:?}" + shift 2 + ;; + --out) + OUT="${2:?}" + shift 2 + ;; + --frames) + FRAMES="${2:?}" + shift 2 + ;; + --voice) + VOICE="${2:?}" + shift 2 + ;; + --no-build) + BUILD=(--no-build) + shift + ;; + *) + echo "usage: ios-record.sh [--layout L] [--theme T] [--voice a|b|c] [--out DIR] [--frames s,s,…] [--no-build]" >&2 + exit 2 + ;; + esac +done +command -v ffmpeg >/dev/null || { + echo "ios-record: ffmpeg not installed (brew bundle)" >&2 + exit 1 +} +mkdir -p "$OUT" +name="$LAYOUT-$THEME${VOICE:+-$VOICE}" + +# A still first: it builds, boots, and gives the crop rectangle. +scripts/design-capture.sh --layout "$LAYOUT" --states idle --themes "$THEME" --out "$OUT/.still" \ + ${VOICE:+--voice "$VOICE"} ${BUILD[@]+"${BUILD[@]}"} >/dev/null +raw="$OUT/.still/$LAYOUT-idle-$THEME${VOICE:+-$VOICE}.raw.png" +read -r CROP_W CROP_H CROP_X CROP_Y < <(swift scripts/design-diff.swift crop "$raw" "$OUT/.still/crop-probe.png" \ + | sed -E 's/crop: ([0-9]+)×([0-9]+) at \(([0-9]+),([0-9]+)\).*/\1 \2 \3 \4/') +[ -n "${CROP_W:-}" ] || { + echo "ios-record: could not read the crop from $raw" >&2 + exit 1 +} + +PICKED="$(ios_pick_device "${BLURT_SIM_DEVICE:-iPhone 18 Pro}")" +UDID="${PICKED%% *}" +BUNDLE_ID=dev.alex.blurt.ios + +echo "==> recording $name (two cycles)" +xcrun simctl terminate "$UDID" "$BUNDLE_ID" 2>/dev/null || true +movie="$OUT/$name.raw.mov" +rm -f "$movie" +xcrun simctl io "$UDID" recordVideo --codec h264 --force "$movie" & +REC=$! +sleep 1 +xcrun simctl launch "$UDID" "$BUNDLE_ID" -BlurtGallery "$LAYOUT" live "$THEME" -BlurtGalleryBare \ + ${VOICE:+-BlurtGalleryVoice "$VOICE"} >/dev/null +LAUNCH_AT=1 +sleep "$(python3 -c "print(2 * $CYCLE + 3)")" +kill -INT "$REC" +wait "$REC" 2>/dev/null || true + +# Skip the launch and the first idle lead-in's start; one full cycle from +# the second idle so the loop begins and ends at rest. +start="$(python3 -c "print($LAUNCH_AT + 2.0 + $CYCLE)")" +ffmpeg -loglevel error -y -ss "$start" -t "$CYCLE" -i "$movie" \ + -vf "crop=$CROP_W:$CROP_H:$CROP_X:$CROP_Y,fps=60" -an -c:v libx264 -pix_fmt yuv420p -crf 18 \ + "$OUT/$name.mp4" +echo "loop: $OUT/$name.mp4 ($CYCLE s, ${CROP_W}×${CROP_H})" + +if [ -n "$FRAMES" ]; then + IFS=, read -ra FRAME_LIST <<<"$FRAMES" + for t in "${FRAME_LIST[@]}"; do + ffmpeg -loglevel error -y -ss "$t" -i "$OUT/$name.mp4" -frames:v 1 "$OUT/$name-${t}s.png" + echo "frame: $OUT/$name-${t}s.png" + done +fi diff --git a/scripts/ios-sim.sh b/scripts/ios-sim.sh new file mode 100755 index 00000000..29c7359c --- /dev/null +++ b/scripts/ios-sim.sh @@ -0,0 +1,87 @@ +#!/bin/bash +# Build the iPhone app and run it in the iOS Simulator: the whole loop in one +# command, for anyone with Xcode and no Apple team. Signing is ad hoc, which the +# simulator accepts, and the App Group works there without provisioning, so the +# app, the keyboard and everything between them run for real; only the phone's +# microphone, background behaviour and haptics need a device. +# +# scripts/ios-sim.sh # build, boot, install, launch +# scripts/ios-sim.sh --screenshot out.png # …then capture the screen +# scripts/ios-sim.sh --no-build … # reuse the last build (a capture loop builds once) +# BLURT_SIM_DEVICE="iPhone 17" scripts/ios-sim.sh +# BLURT_LAUNCH_ARGS="-BlurtGallery panel idle,recording" scripts/ios-sim.sh --screenshot panel.png +# BLURT_SHOT_DELAY=5 … # seconds before the screenshot (default 3) +# +# The named simulator, or whichever iPhone this Mac has. Xcode 27 replaced +# Simulator.app with Device Hub; either is opened if found (neither is, with +# only the Command Line Tools selected). In Device Hub, turn off "Always +# simulate hardware keyboard" or no on-screen keyboard ever appears. +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)" +# shellcheck source=scripts/ios-lib.sh +source "$REPO_ROOT/scripts/ios-lib.sh" +DERIVED="$IOS_DERIVED" +# PRODUCT_BUNDLE_IDENTIFIER in App/BlurtiOS/project.yml. +BUNDLE_ID=dev.alex.blurt.ios +SHOT="" +BUILD=1 +while [ $# -gt 0 ]; do + case "$1" in + --screenshot) + SHOT="${2:?usage: ios-sim.sh [--no-build] [--screenshot ]}" + shift 2 + ;; + --no-build) + BUILD=0 + shift + ;; + *) + echo "usage: ios-sim.sh [--no-build] [--screenshot ]" >&2 + exit 2 + ;; + esac +done + +APP="$DERIVED/Build/Products/Debug-iphonesimulator/BlurtiOS.app" +if [ "$BUILD" -eq 1 ]; then + echo "generate + build" + (cd "$REPO_ROOT/App/BlurtiOS" && xcodegen generate --quiet) + xcodebuild -project "$REPO_ROOT/App/BlurtiOS/BlurtiOS.xcodeproj" -scheme BlurtiOS \ + -destination 'generic/platform=iOS Simulator' -derivedDataPath "$DERIVED" \ + "${IOS_SIM_SIGNING[@]}" -quiet build +elif [ ! -d "$APP" ]; then + echo "ios-sim: --no-build but nothing built at $APP; run once without it" >&2 + exit 1 +fi + +PICKED="$(ios_pick_device "${BLURT_SIM_DEVICE:-iPhone 18 Pro}")" +if [ -z "$PICKED" ]; then + echo "ios-sim: no iPhone simulator available (xcrun simctl list devices)" >&2 + exit 1 +fi +UDID="${PICKED%% *}" +NAME="${PICKED#* }" + +echo "boot $NAME ($UDID)" +ios_wait_booted "$UDID" +DEVELOPER_DIR="$(xcode-select -p)" +for app in "$DEVELOPER_DIR/../Applications/DeviceHub.app" "$DEVELOPER_DIR/Applications/Simulator.app"; do + if [ -d "$app" ]; then + open "$app" + break + fi +done + +echo "install + launch" +xcrun simctl install "$UDID" "$APP" +xcrun simctl privacy "$UDID" grant microphone "$BUNDLE_ID" +xcrun simctl terminate "$UDID" "$BUNDLE_ID" 2>/dev/null || true +# The `+` expansion keeps an empty array from tripping `set -u` on macOS's bash 3.2. +read -ra LAUNCH_ARGS <<<"${BLURT_LAUNCH_ARGS:-}" +xcrun simctl launch "$UDID" "$BUNDLE_ID" ${LAUNCH_ARGS[@]+"${LAUNCH_ARGS[@]}"} + +if [ -n "$SHOT" ]; then + sleep "${BLURT_SHOT_DELAY:-3}" + xcrun simctl io "$UDID" screenshot "$SHOT" +fi diff --git a/scripts/ios-test.sh b/scripts/ios-test.sh new file mode 100755 index 00000000..53c9ea9c --- /dev/null +++ b/scripts/ios-test.sh @@ -0,0 +1,118 @@ +#!/bin/bash +# Run the iPhone app's unit tests (BlurtiOSTests) on a simulator. +# +# scripts/ios-test.sh # tests + the line-coverage gate +# BLURT_IOS_SANITIZER=thread scripts/ios-test.sh # tests under ThreadSanitizer +# BLURT_IOS_SANITIZER=address scripts/ios-test.sh # tests under AddressSanitizer +# BLURT_SIM_DEVICE="iPhone 17" scripts/ios-test.sh +# +# Hosted by the app, signed ad hoc, on the named simulator or whichever iPhone +# this Mac (or CI runner) has. The engine's own tests are `swift test` +# (scripts/check.sh); these cover the iPhone code's pure logic — the App Group +# contract, the keyboard's rules, term packs. The per-test log is in the +# result bundle the last line names. A failure never triggers xcodebuild's +# sysdiagnose collection (`-collect-test-diagnostics never`): that step takes +# ten minutes on this simulator and the result bundle already says what failed. +# +# The same gates the engine's suite runs under in check.sh: warnings are errors +# (project.yml), a plain run must hold BlurtiOSCore's line coverage at or +# above MIN_IOS_COVERAGE, and CI runs the suite again under each sanitizer. A +# sanitizer run skips the coverage gate — the instrumented build is a second +# pass over the same tests, not a second measurement — and builds into its own +# derived-data directory, so it never reuses (or poisons) the plain build. +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)" +# shellcheck source=scripts/release-lib.sh +source "$REPO_ROOT/scripts/release-lib.sh" +# shellcheck source=scripts/ios-lib.sh +source "$REPO_ROOT/scripts/ios-lib.sh" + +# BlurtiOSCore's line-coverage floor (percent) — the engine's own bar +# (check.sh's MIN_COVERAGE). The core is the iPhone code's logic, apart from +# its UI the way BlurtEngine is apart from the Mac shell; the views in the app +# and the keyboard are measured by eye and by the probe's screenshot flows, as +# the Mac shell is by its XCUITest suite, not by this number. Raise it as +# coverage grows; never lower it to land a change. +MIN_IOS_COVERAGE=88 +# Every core file counts: nothing in it needs a device. A file that ever does +# gets an exclusion here, with its reason, the way check.sh lists the engine's. +IOS_CORE="App/BlurtiOS/BlurtiOSCore/" + +# A plain run measures coverage; a sanitizer run instruments for its sanitizer +# instead. Exactly one, so the array is never empty. +SANITIZER="${BLURT_IOS_SANITIZER:-}" +case "$SANITIZER" in + "") TEST_FLAGS=(-enableCodeCoverage YES) ;; + thread) TEST_FLAGS=(-enableThreadSanitizer YES) ;; + address) TEST_FLAGS=(-enableAddressSanitizer YES) ;; + *) + echo "ios-test: BLURT_IOS_SANITIZER must be thread or address, not '$SANITIZER'" >&2 + exit 1 + ;; +esac +DERIVED="$IOS_DERIVED${SANITIZER:+-$SANITIZER}" + +PICKED="$(ios_pick_device "${BLURT_SIM_DEVICE:-iPhone 18 Pro}")" +if [ -z "$PICKED" ]; then + echo "ios-test: no iPhone simulator available (xcrun simctl list devices)" >&2 + exit 1 +fi +UDID="${PICKED%% *}" +echo "testing on ${PICKED#* }${SANITIZER:+ under the $SANITIZER sanitizer}" +ios_wait_booted "$UDID" + +(cd "$REPO_ROOT/App/BlurtiOS" && xcodegen generate --quiet) +STAMP="$(date +%Y%m%d-%H%M%S)" +RESULTS="$DERIVED/BlurtiOSTests-$STAMP.xcresult" +# The whole xcodebuild log, kept rather than `-quiet`ed away: ios-check.sh feeds +# its compiler invocations to `swiftlint analyze`. The terminal gets it through +# xcbeautify, as check.sh's app build does; the log path is printed at the end. +BUILD_LOG="${BLURT_IOS_BUILD_LOG:-$DERIVED/BlurtiOSTests-$STAMP.log}" +mkdir -p "$(dirname "$BUILD_LOG")" +pretty_xcodebuild +# Serial for the reason project.yml's scheme gives (the tests share one global +# App Group override); said here too so no test plan or default can undo it. +xcodebuild test -project "$REPO_ROOT/App/BlurtiOS/BlurtiOS.xcodeproj" -scheme BlurtiOS \ + -destination "platform=iOS Simulator,id=$UDID" -derivedDataPath "$DERIVED" \ + -resultBundlePath "$RESULTS" "${IOS_SIM_SIGNING[@]}" "${TEST_FLAGS[@]}" \ + -parallel-testing-enabled NO -collect-test-diagnostics never 2>&1 \ + | tee "$BUILD_LOG" | "${PRETTY[@]}" +xcrun xcresulttool get test-results summary --path "$RESULTS" 2>/dev/null | python3 -c ' +import json, sys +d = json.load(sys.stdin) +print("ios-test:", d.get("passedTests", "?"), "passed,", d.get("failedTests", "?"), "failed,", d.get("skippedTests", 0), "skipped") +' || true +echo "results: $RESULTS" +echo "build log: $BUILD_LOG" + +[ -n "$SANITIZER" ] && exit 0 + +echo "==> coverage gate (>= ${MIN_IOS_COVERAGE}% BlurtiOSCore lines)" +# The core is static, linked into the app and the keyboard both, so its files +# can appear under either binary: each is counted once, at the better figure +# (the tests run in the app's process; the keyboard's copy is never reached). +COVERAGE="$(xcrun xccov view --report --json "$RESULTS" | python3 -c ' +import json, sys +root, core = sys.argv[1], sys.argv[2] +files = {} +for target in json.load(sys.stdin)["targets"]: + for f in target["files"]: + path = f["path"] + rel = path[len(root) + 1:] if path.startswith(root + "/") else path + if not rel.startswith(core): + continue + best = files.get(rel, (0, 0)) + if f["coveredLines"] >= best[0]: + files[rel] = (f["coveredLines"], f["executableLines"]) +if not files: + sys.exit("no BlurtiOSCore sources in the coverage report") +covered = sum(c for c, _ in files.values()) +total = sum(t for _, t in files.values()) +print(round(100 * covered / total, 2)) +' "$REPO_ROOT" "$IOS_CORE")" +echo "BlurtiOSCore line coverage: ${COVERAGE}%" +if ! awk -v c="$COVERAGE" -v min="$MIN_IOS_COVERAGE" 'BEGIN{ exit (c+0 < min+0) }'; then + echo "error: coverage ${COVERAGE}% is below the ${MIN_IOS_COVERAGE}% floor" >&2 + exit 1 +fi diff --git a/scripts/ios-typecheck.sh b/scripts/ios-typecheck.sh new file mode 100755 index 00000000..f610c254 --- /dev/null +++ b/scripts/ios-typecheck.sh @@ -0,0 +1,68 @@ +#!/bin/bash +# Typecheck the iPhone app and its keyboard on a Mac that has no iOS SDK. +# +# The Command Line Tools ship no iOS SDK, but their macOS SDK carries the Mac +# Catalyst frameworks (UIKit, SwiftUI and AVFAudio built for iOS-on-Mac). That +# is the closest stand-in for iOS an Xcode-less Mac has: `os(iOS)` is true, the +# engine builds in its iOS shape, and the Swift 6 isolation errors the iPhone +# targets can hit surface here with the same wording CI's `ios-build` job +# prints. Each target is checked with the flags its project.yml sets — warnings +# are errors, and every declaration defaults to the main actor. +# +# What it cannot see: APIs that differ between Catalyst and iOS proper, code +# signing, linking, and the generated project itself. CI stays the authority; +# this is the fast local loop for the App/BlurtiOS sources. The unit tests +# (`@testable import`) need the real build: scripts/ios-test.sh. +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)" +SDK="$(xcrun --show-sdk-path)" +CATALYST="$SDK/System/iOSSupport" +if [ ! -d "$CATALYST/System/Library/Frameworks/UIKit.framework" ]; then + echo "ios-typecheck: no Mac Catalyst frameworks under $SDK" >&2 + exit 1 +fi + +WORK="$(mktemp -d)" +trap 'rm -rf "$WORK"' EXIT + +# The 18.0 is the deployment target App/BlurtiOS/project.yml sets. +TARGET=arm64-apple-ios18.0-macabi +COMMON=( + -target "$TARGET" -sdk "$SDK" + -F "$CATALYST/System/Library/Frameworks" -I "$CATALYST/usr/include" + -swift-version 6 -parse-as-library +) + +echo "BlurtEngine → $TARGET" +find "$REPO_ROOT/Sources/BlurtEngine" -name '*.swift' -print0 \ + | xargs -0 xcrun swiftc "${COMMON[@]}" -module-name BlurtEngine \ + -emit-module -emit-module-path "$WORK/BlurtEngine.swiftmodule" + +IOS="$REPO_ROOT/App/BlurtiOS" +# project.yml's SWIFT_PACKAGE_NAME, so BlurtiOSCore's `package` API resolves. +IOS_FLAGS=( + "${COMMON[@]}" -I "$WORK" -warnings-as-errors -package-name BlurtiOS + -Xfrontend -default-isolation -Xfrontend MainActor +) +TARGET_FLAGS=("${IOS_FLAGS[@]}" -typecheck) + +# The logic both targets link: emitted, not just checked, so they import it. +# Extension-safe, since the keyboard links it (APPLICATION_EXTENSION_API_ONLY). +echo "BlurtiOSCore" +xcrun swiftc "${IOS_FLAGS[@]}" -application-extension -D DEBUG -module-name BlurtiOSCore \ + -emit-module -emit-module-path "$WORK/BlurtiOSCore.swiftmodule" "$IOS"/BlurtiOSCore/*.swift + +# The app also compiles the keyboard's sources, all but its entry point +# (project.yml: the theme picker and the home screen draw the real keyboard +# and orb), and CI builds Debug, so `#if DEBUG` code is checked too. +echo "BlurtiOS" +find "$IOS/BlurtKeyboard/Sources" -name '*.swift' ! -name KeyboardViewController.swift -print0 \ + | xargs -0 xcrun swiftc "${TARGET_FLAGS[@]}" -D DEBUG -module-name BlurtiOS \ + "$IOS"/BlurtiOS/Sources/*.swift "$IOS"/Shared/*.swift + +echo "BlurtKeyboard" +xcrun swiftc "${TARGET_FLAGS[@]}" -application-extension -module-name BlurtKeyboard \ + "$IOS"/BlurtKeyboard/Sources/*.swift "$IOS"/Shared/*.swift + +echo "ios-typecheck: ok" diff --git a/scripts/mutate.sh b/scripts/mutate.sh index 7aaa5c47..aa203eba 100755 --- a/scripts/mutate.sh +++ b/scripts/mutate.sh @@ -51,7 +51,7 @@ DEFAULT_TARGETS=( "Sources/BlurtEngine/Hotkey/DictationKeyGate.swift" "Sources/BlurtEngine/Hotkey/DictationKeyRouter.swift" "Sources/BlurtEngine/Hotkey/TriggerKey.swift" - "Sources/BlurtEngine/Injection/KeyInjector+Separator.swift" + "Sources/BlurtEngine/Injection/InsertionSeparator.swift" "Sources/BlurtEngine/Pipeline/MeterBarGeometry.swift" "Sources/BlurtEngine/Pipeline/OverlayPlacement.swift" "Sources/BlurtEngine/Pipeline/PipelinePhase.swift"