Repository navigation
Blurt for iPhone: the app that listens, the keyboard that inserts - #223
Merged
Merged
Conversation
Bumps the github-actions group with 2 updates: [github/codeql-action/init](https://github.com/github/codeql-action) and [github/codeql-action/analyze](https://github.com/github/codeql-action). Updates `github/codeql-action/init` from 4.38.0 to 4.38.1 - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@b96794f...1c5b675) Updates `github/codeql-action/analyze` from 4.38.0 to 4.38.1 - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@b96794f...1c5b675) --- updated-dependencies: - dependency-name: github/codeql-action/init dependency-version: 4.38.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions - dependency-name: github/codeql-action/analyze dependency-version: 4.38.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com>
… portable The engine now declares iOS 18 beside macOS 15. The Accessibility reads, the clipboard paste and its ⌘V, the CoreAudio route reads, the TCC reset and the code-signing probe are fenced behind `#if os(macOS)`; the pipeline, the dictation client and the settings stores compile for both. Three pieces the fenced files used to own move to platform-free files, since the pipeline and the iOS host need them: `CapturedFocus` + `FocusedFieldContext` (`FocusContext.swift`), `PermissionStatus`, and the spacing rule as `InsertionSeparator` (it was `KeyInjector`'s extension; an iOS keyboard writing through `insertText` faces the same decision). Two seams change shape so an iOS host can use them. `InjectorProtocol`'s `setTargetApp(NSRunningApplication?)` becomes `setTarget(CapturedFocus?)` — a pid and a name, resolved to the process handle inside `KeyInjector` on macOS. `DictationSession`'s public initializer takes a `HostFocusCapture`, the two closures that replace the Accessibility capture where there is none (an iOS keyboard hands in `documentContextBeforeInput` as the text before the cursor). `check.yml` gains a required `ios-build` job that compiles the library for the iOS simulator, the only place a symbol that escapes its fence is caught, and the dependency and invariant guards widen from `App/Blurt` to every `App/*`. Also unwraps the double-wrapped capture failure: `MicCapture.start()` already throws a `BlurtError`, and wrapping it again read as "Audio capture failed: Audio capture failed: …". Redoes the approach of the unmerged `feat/blurtengine-ios` branch against the current engine. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`App/BlurtiOS/` beside `App/Blurt/`: a SwiftUI app plus a keyboard extension, generated by XcodeGen from their own `project.yml` (the project is not committed; CI and a checkout generate it). The shape iPhone forces: iOS lets no keyboard use the microphone, so the app listens and transcribes and the keyboard is a remote control that inserts the words. They share an App Group (`Shared/SharedState.swift`) and wake each other with Darwin notifications. The app keeps the microphone open for a listening window (`ListeningWindow`, extended by every dictation) over an always-on audio-only capture session (`WindowedAudioSource`) that converts the phone's native format to the 16 kHz mono PCM the dictation API wants (`PCMConverter`) — iOS refuses to open the mic from the background, and during a dictation the app is in the background. The engine's `DictationSession` runs unchanged over that capture, with the keyboard's text-before-cursor arriving through `HostFocusCapture` and the result leaving through `KeyboardRelayInjector` (clipboard plus the quiet "Copied" when no keyboard is around). The keyboard drives the engine's `DictationKeyGate` from the mic key's finger down/up, so a tap latches and a hold is push-to-talk as on the Mac; it inserts through `InsertionSeparator`, reads the phone's contact names for the request's key terms, and taps the haptics on the phase edges. Three layouts — slim bar, mic panel, full keyboard — over one model, chosen in the app. Sign in with AssemblyAI is a stub until the login service exists; debug builds take a pasted key. `check.yml`'s `ios-build` job now generates and builds the app for the simulator too, and SwiftLint covers the new sources. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…public `CaptureSessionRecorder.audioSettings()` moves to its own file so the recorder stays within the 400-line lint budget (the fence tipped it over). `FocusContext.swift` needed no Foundation once `pid` is spelled as the `Int32` it is. `InsertionSeparator` is public for the iOS keyboard, which periphery's Mac scan cannot see, so it carries the same ignore the other iOS-facing API does. Also adds the output-styles screen to the iPhone app: Default plus up to four named profiles over `StyleProfileStore`, with the Mac editor's caps. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d on main The keyboard and the app default every declaration to the main actor, but the App Group payloads and the constants that name them cross into the capture path and the engine's closures, which run anywhere — so they are nonisolated. The lexicon completion reads UIKit's entries where UIKit hands them over, on the main thread. The converter's one-buffer handoff uses a Mutex rather than a captured var, in case the input block is typed as Sendable. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…scripts/ios-typecheck.sh The capture output calls its delegate on the delivery queue, but with the target defaulting to the main actor the extension's conformance was inferred main-actor even on a nonisolated class — spell it nonisolated. The converter's input block is @sendable and runs synchronously before convert returns, so the buffer crosses into it unchecked. Both were caught by the new script, which typechecks the iPhone targets against the Mac Catalyst frameworks on a Mac with no iOS SDK — the same flags each project.yml target sets. CI's ios-build job stays the authority. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…n screen; periphery config A Darwin notification is held for a suspended process and delivered when it resumes, so the keyboard inside the app the user left would insert a dictation done elsewhere — every host app runs its own keyboard process. A result is now taken out of the App Group once inserted and ignored after ten seconds. The keyboard clears its presence when it leaves the screen, so a dictation that finishes after that goes to the clipboard; the app checks presence before it writes a result. The phone's word list is re-read at most hourly. Periphery: the iPhone app consumes engine API the Mac-project scan cannot see, so those declarations carry ignore comments, and periphery ≥ 3.5 reports every ignore on a referenced declaration as superfluous. That check is off, with the honest fix (a scan that sees both apps) noted in .periphery.yml. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Listening now means the window has not lapsed AND the app has checked in within fifteen seconds with the microphone really open — not interrupted by a call, not killed by iOS with the window's end still in the future. Otherwise the mic key opens the app, which reopens the window, instead of sending a press nobody answers. The keyboard drops a phase snapshot that is over (a notice older than three seconds, an in-flight state older than any dictation can be), and the app publishes idle at launch so the last run's state is gone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Without Full Access the key did nothing; now it brings the user to the app, whose checklist says what is missing. Opening the app needs no Full Access. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The generated project no longer hard-codes the Mac app's team: XcodeGen
expands ${BLURT_TEAM} from the environment, and unset it resolves to no team,
which CI (signing off) is fine with. The README walks the first phone run
from the App Store download through Developer Mode and the App Group.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two diagnostics Swift 6.4 adds and warnings-as-errors turns fatal: a weak capture inside a closure that itself captures self strongly (the Darwin observers now capture weak at the outer closure), and a Mutex captured by the audio converter's escaping-typed input block (now an unchecked handoff box, sound because the block runs synchronously before convert returns). Verified with a simulator build on Xcode 27.0; CI's Xcode 26.6 accepts both forms. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…pear AVCaptureSession carries no audio in the simulator (FigCaptureSessionSimulator fails with -12782 on start), so the listening window now takes any ListeningSource: WindowedAudioSource on a phone, SimulatorAudioSource — an Audio Queue asking for 16 kHz mono 16-bit outright — in the simulator. The per-utterance feed both push into is UtteranceFeed. The keyboard died the moment it appeared: UIKit returns the supplementary lexicon on an XPC queue, and a main-actor closure there traps. The callback is now Sendable, hands the immutable lexicon over, and reads it on the main actor. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… Xcode without the App Store Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Ink surface, the status pill (36 pt, rim, shadow, inset 12, gap 8), the orb with its sweeping ring, the engine's own meter geometry, tracked-uppercase status words — every number from the Mac's BrandOrb, OverlayView and BlurtBrand. The orb is the mic key (96 / 44 / 42 pt by layout) with a mic glyph, a stop glyph and a level-driven glow while recording. Keys are warm white on raised ink, modifiers a step darker, brightening on press. Heights follow from the rows: 60 / 216 / 252. Debug builds render any layout in any state from a launch argument (-BlurtGallery <layout> <states>), with the keyboard's sources compiled into the app for it, so a screenshot never needs Notes. App/BlurtiOS/DESIGN.md records the tokens, the state table, the layout arithmetic and the procedure. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…b-actions-c83b7ed659 ci: Bump the github-actions group with 2 updates
Blurt for iPhone: multiplatform engine, the app and its keyboard
…double-space period The keys can look exactly like the iPhone's own: a second look beside the layout, following the app being typed in between the system keyboard's light and dark palettes, with its spacing and a 1 pt drop under each key. The full layout now follows the iPhone's geometry in both looks — ten letters at one width, the middle row centred, shift and delete flanking, return two modifiers wide — with letter pop-ups on press, shift that comes on by itself where the field asks, and a quick double space that ends a sentence. Hands-free: with the toggle on, a dictation starts the moment the keyboard appears, as a synthetic tap through the engine's gate, so the next real tap stops it. The design doc records the palettes, the geometry and the rules. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Pasted, copied and error stayed on the pill until the next dictation; the Mac pill fades out after 0.8 s / 1.6 s. The keyboard now returns to its resting line after 1.2 s / 2 s. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The wordmark, the orb as the hero with the listening state under it (ring sweeping while the mic is open, glowing with the level while recording), the style chips, the recent dictations as the design's cards, the setup checklist in a card only while something is missing. Settings move to a sheet, grouped as on the Mac. The Mac's accent and card colours come in through the asset catalog, its icon and wordmark are shared from App/Blurt rather than copied. A debug launch flag opens the mic at launch so that state can be captured. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A horizontal swipe in either direction, as often as you like, flips between the mic panel and the letter keyboard; the page slides the way the finger went, two dots mark which is up, the keyboard resizes with it, and it is back to the mic each time the keyboard appears. Letter keys ignore a touch that travelled, and the mic cancels one, so a swipe never types or dictates. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…o page dots The look setting is gone: the keys sit exactly where the iPhone keyboard's do (6 between keys, 11 between rows, 3 at the edges, 5 pt corners, a 1 pt drop) in Blurt's colours, in every appearance. Themes come later, through the same palette seam; the iPhone's own palettes are recorded in DESIGN.md as the first to offer. The carousel keeps its swipe and loses its dots. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s the letter rows The status pill is gone. Voice lives in one place, the bar above the keys where the system puts its suggestions: the orb, tap or hold, with the live meter on either side while recording. State is the orb's own — dimmed when Blurt isn't ready, the ring sweeping while working, a stop glyph and a glow while recording, a check, a clipboard or an orange exclamation for a moment after — and every transition has its haptic. The bottom row is the stock one: 123 · globe · space · return. Hands-free is on by default, so a dictation is under way the moment the keyboard comes up. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
No glyph on the orb — the Mac's carries none. Tap it: the ring sweeps as the app's orb does while the mic comes up, then the circle grows sideways into a capsule holding the live wave for as long as you talk, and shrinks back with the ring sweeping while the words come. The panel's orb sits in the true centre of the space above the keys (the reserved meter row that pushed it up is gone); cancel moved to the corner so the capsule has room. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…u're in Select the misheard word (or nothing), tap the + beside the orb, and the voice bar becomes a field the keys type into, pre-filled with the selection; return or ✓ saves it to Blurt's key terms in the App Group, where the app reads it on the very next dictation, and replaces the misheard word in the text with what was typed. Key terms move from the app's own defaults to the App Group for this; Settings binds to the same place and shows the count. Also, with no setting to make: key clicks follow the phone's own keyboard setting, the return key says what the field asks (send, search, go…), number fields open on the symbols page, and the letter rows follow the phone's first language (AZERTY, QWERTZ, QWERTY). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…els as a table Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The test class is main-actor isolated so its app object can be a plain stored property (the target's default isolation made a lazy default an error). Each run now ends with a sheet.png — the keyboard from every step's screenshot, side by side and captioned — so a face × mic × layout can be judged at a glance without opening the screenshots one by one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
One percent black over the host's light material read as a keyboard a shade darker than the border around it (230 against 231, measured). The floor is painted in the face's material colour at the same one percent — nothing to see, and the touches still reach the keyboard — and repainted when the field, and so the face, changes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…cross Alternating or quick swipes jumbled the panel's flip: each page's slide was a SwiftUI insertion/removal transition whose direction was read from state, and on a reversed or interrupted swipe the outgoing page kept the direction of the flip before, so both cards travelled the same way and crossed. `PanelCarousel` lays the two pages out as a strip one page wide: the page that is up sits in the middle, the other is set down a whole page away on the side the finger moved towards, and the strip slides one page — the first leaves one way as the second arrives the other. Side by side, a page apart, they cannot cross whichever way the flips come. A flip back before the slide has settled retreats instead of starting over; a retreat cut short keeps its side; a slide that was overtaken cannot settle the strip late (`PanelCarouselState`, tested). Each page keeps its own height while the keyboard resizes under the slide. The flows test adds four pairs from one fixed screen point — left then right and left twice, each with no wait and again once settled — and checks each lands back on the panel. (An element mid-slide is no anchor for a coordinate: the second swipe of a pair once resolved against the mic as it slid out, and went the wrong way.) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Select a word in the host field and the + beside the mic becomes a chip in the field's shape holding the word; one tap adds it to the key terms as it stands, no typing, the text untouched. A check for a moment, then a quiet check: Blurt knows it. A word already in the list shows the check from the start. Hold the chip to open the field pre-filled instead (fix the spelling, replace it in the text). The selection is read on selectionDidChange and textDidChange, with Full Access only; the chip hugs the word and the pair stays centred, the element giving way to a 96 pt minimum; in the panel it grows from the corner up to 240 pt. Tokens addterm/chip-*, motion chip-hold; gallery states selected and known; the flows probe highlights the typed word and taps the chip. 85 iOS tests, 15/15 flows in the real extension. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Settings → Keyboard, under the layout and in its shape — Left alignment · Default · Right alignment with a line beneath. MicAlignment lives in the App Group and the keyboard reads it on appearance, as it does the layout. At an edge the mic key is the thing at the edge: in the bar and the slim bar the pair slides there and the + moves to the mic's inner side; the panel becomes a big voice bar — the element at the edge as wide as it shows, the + beside it, cancel in the far corner. Left and right are the phone's sides, so a right-to-left language flips the leading/trailing mapping and the mic stays under the thumb chosen. Gallery -BlurtGalleryAlign, probe word left|right, ios-keyboard-flows.sh --align; the probe resets the key terms at launch so the add-chip flow finds its word new every run, and knows an edge-aligned panel's mic is element-wide. 88 iOS tests; flows green in the real extension: full right 16/16, panel right 26/26, slim bar left 12/12. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A pair centred as a whole put the grille left of centre and read as off. Now the element is dead centre and an unseen twin of the + balances the other side, so the + hangs off the element rather than shifting it. Edge alignment unchanged. Probe: the centring check measures the mic, not the pair; selection flow double-taps the word itself and accepts either face of the chip; the run resets the key terms and the mic's side at launch (the App Group outlives a run); switchAwayAndBack drives the system globe away and back from the mic page and the keys page — the mic comes back both ways, at the panel's own height; dismissAndReturn puts the keyboard away (a new probe-dismiss button in the probe field) and brings it back — written, not yet run green. Neil's "mic gone after switching keyboards" in Messages is the carousel case the other session is reproducing there; nothing here touches PanelCarousel. Full flow 18/18 and panel flow 28/28 with the switch steps, in the probe app. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`MessagesCarouselProbe` brings Blurt up in Messages — the globe's menu, else hopping — and swipes the panel from one fixed point twelve times: left, right, left twice, right twice, and pairs with no wait between, with a screenshot and a FLOW line after each saying which page is up. Run under `xcrun simctl io <udid> recordVideo` to see the motion. Verified the strip carousel in Neil's host: every swipe lands on the page it should, the mic element drawn on each return. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Chasing "switching keyboards makes the dictation button disappear" (the panel page up, no grille). The probe field gains a dismiss button and one button per layout (what Settings does while a keyboard is up elsewhere), and the flows end with the re-appearances: the system globe away and back from the mic page and the keys page; put away and brought back from both pages; the layout changed to each other layout and back, re-shown each time; a slow swipe that rests on the mic long enough to press it before travelling; a swipe while recording. In the probe app every one brings the mic key back, full size, at the panel's own height, drawn (screenshots). The reappearance probes live in their own file; the shared helpers are no longer private. Not reproduced here, nor in the other session's Messages carousel probe. Panel flow 40/40, full flow 26/26. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s by command The review and transition passes of 2026-09-30, verified on the real extension. Layout: the mic and the + are placed by one tested rule in every layout (VoiceRowLayout): the mic dead centre or at the chosen edge, the + hanging off it on its centre line, a chip growing to the row's edge, the pair centred when the row is too narrow (the slim bar, where the chip used to draw over the grille). The panel's + leaves its corner; the cancel × mirrors it (× · mic · +). One GlyphKey (44 pt) for + / × / ✓. Panel and slim-bar keys at the full keyboard's KeyGeometry widths; the panel's mic centred between the keyboard's top and the keys; the symbols' #+= page; the top-row pop-up inside the keyboard; a tap is "ended on the key ± 8 pt". Drawing: the grille is plain SwiftUI paths, never a Canvas — a Canvas's render surface failed to present in the extension three separate ways (asynchronous; a model mutation inside Messages' layout pass; background and resume), the key there and the dots not. The trait-change registration is gone; the face is re-read in viewIsAppearing. Behaviour: after a one-tap add the chip shows ✓ for 1.2 s then gives way to the + (only that word's chip); a tap on a known-word ✓ dismisses it; opening the term field closes a live dictation (release from the +, cancel over a highlight or an unanswered press); collapsing the highlight the field opened over no longer deletes the word; host typing is claimed one character at a time; save replaces only while the highlight stands; a release during connecting is held for the first recording phase; a touch cancelled inside the mic delay presses nothing; the × only while the app is there; an in-flight phase with the app gone reads idle; notices never replay; lost Full Access resets the phase and gate; pressed states are gesture state. Protocol: every phase the app publishes names the command it last took (PhaseSnapshot.command) and the keyboard settles its gate only for the press it answers — the previous dictation's "pasted" landing after a new press used to turn the next tap into a second press. Every command names its keyboard (KeyboardCommand.keyboard) and a result goes back to the keyboard that pressed, else the clipboard. The press retry re-reads the phase, then re-puts the command. Tests: 110 iOS tests (was 88): VoiceRowLayout, the gate's answers, KeyboardModelFuzzTests (five seeds × 2000 random events with invariants after each, 200 disappear/appear recovery rounds). Probes check the mic's pixels (MicPixels), not only its frame; the Messages carousel probe checks frames and pixels after every return; the flows restore the App Group side they set. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
iPhone keyboard: iPhone look, hands-free, pop-ups, notice dwell
iPhone keyboard on the brand: one placement rule, path-drawn grille, id-tagged protocol
# Conflicts: # AGENTS.md # Sources/BlurtEngine/FocusCapture/FocusCapture+Editability.swift # Sources/BlurtEngine/FocusCapture/FocusCapture.swift # Sources/BlurtEngine/Injection/KeyInjector+SystemActions.swift # Sources/BlurtEngine/Injection/KeyInjector.swift # Sources/BlurtEngine/Injection/SystemClipboard.swift # Sources/BlurtEngine/Pipeline/DictationSession.swift # Tests/BlurtEngineTests/BrowserBundleIDTests.swift # Tests/BlurtEngineTests/KeyInjectorInsertTests.swift # Tests/BlurtEngineTests/KeyInjectorSystemActionsTests.swift # Tests/BlurtEngineTests/Stubs/InjectorTestSupport.swift # scripts/check-invariants.sh # scripts/check.sh
…aseURL Follow-ups to merging main (Sparkle, read-aloud TTS) into the iPhone work: SelectionCopy and SelectionSpeaker use the Mac's pasteboard and ⌘C, so they sit behind `#if os(macOS)` like the rest of the AppKit half; the iPhone app builds a HostIdentity without the releaseURL Sparkle removed; TextShortcutTests names FocusedFieldContext by its cross-platform top-level name; the session file back under the lint length; a grille line reflowed for swift-format. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`KeyboardPalette.keyGap` — the layouts read `KeyGeometry.gap` now — and the test rig's unused proxy in the gate suite split out earlier today. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
nbisht-aai
requested review from
alexkroman
and
a balanced review from Copilot
October 1, 2026 19:04
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The command transport, listening lifecycle, capture format, test isolation, extension rendering, and distributable resources have unresolved correctness or release blockers.
Review effort: Balanced
Findings: 3
Open (11)
What changed in this PR
Adds an iPhone host and keyboard extension around the multiplatform Blurt engine, using App Group messaging while keeping microphone capture in the app.
Changes:
- Makes engine seams portable across macOS and iOS.
- Adds the iOS app, keyboard, shared protocol, design system, and tests.
- Adds iOS build, invariant, design-sync, and simulator tooling.
| File | Description |
|---|---|
Package.swift |
Adds iOS engine support. |
Sources/BlurtEngine/Audio/* |
Adapts capture APIs and platform boundaries. |
Sources/BlurtEngine/FocusCapture/* |
Exposes portable focus-context types. |
Sources/BlurtEngine/Injection/* |
Adds portable targeting and insertion rules. |
Sources/BlurtEngine/Permissions/* |
Separates macOS-only permission behavior. |
Sources/BlurtEngine/Pipeline/* |
Adds host-supplied focus capture. |
Sources/BlurtEngine/UserAgent.swift |
Reports the active Apple platform. |
Sources/BlurtEngine/README.md |
Documents the expanded engine API. |
App/Blurt/* |
Updates macOS integration for revised seams. |
App/BlurtiOS/BlurtiOS/Sources/* |
Implements the iPhone app and audio pipeline. |
App/BlurtiOS/BlurtKeyboard/Sources/* |
Implements keyboard layouts, input, and dictation controls. |
App/BlurtiOS/Shared/* |
Defines cross-process state and design primitives. |
App/BlurtiOS/Design/* |
Adds tokens, Figma tooling, and bundled fonts. |
App/BlurtiOS/BlurtiOSTests/* |
Tests protocol, keyboard, layout, and app behavior. |
App/BlurtiOS/BlurtiOSProbe/* |
Adds simulator keyboard-flow probes. |
App/BlurtiOS/project.yml |
Configures app, extension, tests, resources, and entitlements. |
App/BlurtiOS/DESIGN.md |
Documents keyboard design decisions. |
App/BlurtiOS/README.md |
Documents iOS development and architecture. |
Tests/BlurtEngineTests/* |
Updates tests for portable types and platform guards. |
scripts/ios-*.sh |
Adds iOS build, test, capture, and flow automation. |
scripts/design-sync.sh |
Checks generated design-token consistency. |
scripts/check*.sh |
Extends repository checks and keyboard invariants. |
.github/workflows/check.yml |
Adds the required iOS CI job. |
AGENTS.md |
Documents iOS architecture and guardrails. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…ts 30 minutes `mutate.sh --list` (main's new check) named `KeyInjector+Separator.swift`, which the multiplatform work renamed to `InsertionSeparator.swift`. The ios-build job hit its 15-minute cap inside the periphery scan — the simulator build and the unit tests alone take ~12 minutes on macos-26 — so it gets the same 30 the check job has. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Collaborator
|
I'm looking at this and testing and making a few adjustments! |
- openApp: open the app with UIApplication.open(_:options:) up the responder chain. iOS 18 turned openURL: into a logged no-op, so the orb and "open Blurt" silently did nothing. - disappeared: words already being transcribed are left to finish (and land on the clipboard) instead of being cancelled. - A swipe off the orb undoes only its own press (gate.otherKeyDown): flipping the carousel mid-recording no longer cancels a latched dictation. - An unanswered press expires after BlurtShared.commandFreshnessWindow (moved from the app so both sides share it), so a killed app or a dropped stale press can't leave the gate latched and skip hands-free. - beginAddingTerm during .processing cancels only from a highlighted word; the plain + lets the words land. Each has a regression test in KeyboardModelGateTests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The iPhone code's logic now lives apart from its UI, the way BlurtEngine sits apart from the Mac shell. BlurtiOSCore is a static framework both the app and the keyboard link: the App Group contract, KeyboardModel, VoiceState, KeyboardInteraction, TermPack, KeyboardRelayInjector and friends. No SwiftUI, no views, no audio; its API is `package` (one SWIFT_PACKAGE_NAME), so periphery still sees what nothing uses. The three UI hooks the model had (palette, KeyboardLayout.height, MicAlignment's SwiftUI alignment) became shell-side extensions; a preview now hands the model a ThemeFace instead of a palette. The iPhone suite now holds the engine's gates: - ios-test.sh fails when BlurtiOSCore line coverage drops below 88% (check.sh's MIN_COVERAGE). It is 96.81%, up from 84%, with new tests for the presence heartbeat, the app's Darwin signals, notice and saved-term dwells, the release held while the mic comes up, the word list hand-off, openApp's responder walk (via a urlOpener seam), SharedStore defaults, TermPack's share file and APIKeyModel. - A new ios-sanitizers job runs the suite under TSan and ASan; `gate` requires it. - Tests are pinned serial (scheme + -parallel-testing-enabled NO): they swap a global App Group override, and an async test suspended mid-await must not see another suite's. check-invariants' no-audio-in-the-keyboard rule now covers BlurtiOSCore, which the keyboard links. ios-typecheck.sh emits the core module first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…onts The iPhone code now meets every quality gate the engine and the Mac shell do. scripts/ios-check.sh is its check.sh: BlurtiOSTests with BlurtiOSCore's coverage gate, then `swiftlint analyze` (unused imports, never run on App/BlurtiOS before) over that build's log, then periphery. CI's ios-build runs it; a local check.sh runs it too when an iPhone simulator on this Xcode's runtime exists. codeql.yml now builds the iPhone app as well, so it is in the Swift database. The first analyze pass found 17 unused imports, now gone (CGFloat and CFNotificationCenter users import CoreFoundation, which is what actually declares them). Removed the Figma plugin (Design/figma, design-figma-call.sh, the generator's tokens.js output) and the bundled brand fonts, whose licence covered only the web. BlurtType keeps its roles (mono, heading, body) on the system's monospaced, serif and default faces; the fonts token group, UIAppFonts and the registration checks go with them. Copilot review fixes: - An automatic window close (expiry, a call, Siri) now ends the dictation first: a recording is released so its words land, a mic coming up is cancelled. It used to sit in .recording over a dead feed until the 120 s auto-release, with the keyboard showing idle. - WindowedAudioSource throws when the session refuses its input or output instead of opening a window that hears nothing. - ios-keyboard-flows.sh exits non-zero when a run's XCUITest fails. - PrivacyInfo.xcprivacy for the app and the keyboard (App Group and own UserDefaults, the only required-reason API either uses). Also: - The app icon's art is full-bleed: its own transparent rounded corners were filled white by iOS, a halo that read as a broken icon mid-launch. Shared with the Mac app, which gets the same fix. - The launch screen is the app's Page colour, not blank system white. - LSSupportsOpeningDocumentsInPlace: false, matching TermPack's copy-then-delete import (and silencing Xcode's warning). - The setup card says to tap Keyboards on Blurt's Settings page — the only Settings link Apple allows. - check-invariants.sh no longer pipes the ~30 KB settled-decisions table into `grep -q`: an early match left printf writing to a closed pipe, EPIPE under pipefail, and a "missing" row on CI at random. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A fuzz rule now holds the keyboard to it: a cancel over a recording or a transcription comes only from the user — the × key, or the term field opened over a highlighted word. With the earlier bugs put back it catches the cancel on leaving the screen and the swipe off the orb; the + during processing needs a sequence too rare for random events, and keeps its deterministic test. On its first run against the fixed code it found one more: right after the keyboard released a recording (leaving the screen, opening the term field) the picture still says recording, and a swipe across the orb then sent a press and a cancel — the cancel landing on the words being transcribed. undoPress now cancels only a press that could have started a dictation (`pressSentOver`); a press over one the app was still running started nothing, since the engine drops it. Pinned by a gate test. - scripts/ios-check.sh compiles BlurtiOSProbe (build-for-testing). It has its own scheme and nothing else built it, so MicPixels.swift sat broken under Swift 6 isolation; now fixed (main-actor, as XCUIAutomation is). - The setup step names the whole path — Apps › Blurt › Keyboards — since openSettingsURLString lands on Settings' top level in the iOS 26 simulator although Blurt's page exists. README to match. - KeyboardModel's word-list hand-off moves to KeyboardModel+Lexicon.swift (the file was past SwiftLint's length limit); the fuzz test's event table to an extension, for the same reason. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- CLAUDE.md: running the iPhone app (the matching simulator runtime, ios-sim.sh, enabling the keyboard, the Mac-side microphone the simulator records through), ios-check.sh, the generated iPhone project, and which reviewers cover App/BlurtiOS. - AGENTS.md settled decisions + project-guardrails: logic stays in BlurtiOSCore (no SwiftUI there), no private Settings deep links, no bundled brand fonts; the skill also carries the never-cancel-words rule and the serial iPhone tests. - swift6-concurrency-reviewer: the iPhone's invariants (no audio in the keyboard, nonisolated payloads, system callbacks off the main actor, no cancel on a stale picture, close order, serial tests). - cleanup-reviewer: what on the iPhone is deliberate — the TEMPORARY voice candidates, gallery/probe entry points, package access, seams. - macos-hig-reviewer: Mac only. - CONTRIBUTING.md: an iPhone section; README.md: App/BlurtiOS in the map. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From a reuse / simplification / efficiency / altitude pass over this branch's work. - DictationCoordinator.windowClosingOnItsOwn decided release-or-cancel from its own copy of the phase, which trails the session and could still say .connecting over a live recording — cancelling it. It now just releases: the session's commands run in order and a release checks the session's own phase on its turn. - The unanswered press is stored whole (a KeyboardCommand), so its id and age can't disagree; KeyboardCommand.isFresh is the one freshness rule the app and the keyboard both read; appeared() and syncGate share letGoOfExpiredPress. KeyboardModel.face resolves like voiceKind does. - BlurtType.mono takes Font.Weight (its own enum mirrored it once the bundled faces went). One KeyboardRig + snapshot() for the four suites that each built their own; a no-op 200 ms test sleep goes. - scripts: one IOS_DERIVED in ios-lib.sh; require_tools and pretty_xcodebuild from release-lib.sh; one TEST_FLAGS array; no placeholder coverage exclude; `swiftlint analyze` reads only the iPhone modules' compile commands, so no stderr filter is needed; here-strings throughout check-invariants.sh; check.sh decides once whether the iPhone gates run and skips the Catalyst typecheck when they do. - CI cost: the probe build uses the tests' destination and flags, so it reuses their build (arm64 only, seconds) instead of rebuilding for a second architecture; CodeQL builds the iPhone code for arm64 only; the duplicate xcodegen step goes; xcbeautify is installed for the logs. Kept as is: periphery's own build. The tests' simulator index reports WindowedAudioSource unused, since the simulator builds SimulatorAudioSource in its place. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Blurt for iPhone: same engine as the Mac app, a different shell because iOS lets no keyboard use the microphone.
Engine (
Sources/BlurtEngine): multiplatform —platforms: [.macOS(.v15), .iOS(.v18)]. The Mac-only halves (Accessibility reads, clipboard paste, the hotkey tap, TCC, signing, the updater) sit behind#if os(macOS); the portable seams areFocusContext.swift(FocusCapture,CapturedFocus,FocusedFieldContext— public),InsertionSeparator(the leading-separator rule, also used by the keyboard),PermissionStatus,InjectorProtocol.setTarget(CapturedFocus?)(a pid instead of anNSRunningApplication), andDictationSession.HostFocusCaptureso a host can supply the press-time field context. The Mac app's behaviour is unchanged; its tests pass as before.App (
App/BlurtiOS): one screen on the brand (status, the voice element, the one green button, styles, recent dictations, a setup card), Settings, a 15-minute listening window kept alive by the audio session, key terms shared through the App Group so the keyboard can add one on the spot, contact names from the phone's lexicon, share key-term lists (.blurtterms,blurt://terms).Keyboard (
App/BlurtiOS/BlurtKeyboard): three layouts (mic bar, mic panel with a swipe carousel to a full keyboard, full keyboard) at Apple's measured key geometry; two brand faces following the field; the grille mic element (a dot lattice that is also the meter); hands-free start; one-handed mic alignment; highlight-a-word-to-add-a-key-term in one tap; haptics on the phase edges. The app and keyboard talk through the App Group with id-tagged commands and phases (every phase names the command it answers; results go back to the keyboard that pressed). NoCanvasanywhere in the extension — its render surface failed to present three different ways; the element is drawn with paths.Repo:
check.ymlgains anios-buildjob (engine, app and keyboard on the iOS simulator, the iOS unit tests, a periphery scan over the iOS project), required throughgate.check-invariants.shandcheck.shcoverApp/*, and a new invariant mechanises "the keyboard never hears anything" (no audio type compiles into the extension).DESIGN.mdis the keyboard's design document;Design/tokens.jsonis the one source for every colour and number, withscripts/design-sync.sh --checkfailing on drift. TheBlurtiOSProbeXCUITest scheme drives the real extension in the simulator and in Messages, checking the mic element's pixels, not just its frame.Not in this PR / known: sign-in with AssemblyAI (the login and key-minting service doesn't exist yet; debug builds take a pasted key); the bundle ids and App Group (
dev.alex.blurt.ios,group.dev.alex.blurt) are placeholders inherited from the Mac app and need org-owned ids before TestFlight; nothing has run on a phone yet (simulator only — the App Group needs a paid team); an open keyboard applies a changed setting on its next appearance, not live; fast two-thumb typing on the full keyboard drops the overlapping key (SwiftUI gestures are single-touch across views).Fonts — please check before merging.
App/BlurtiOS/Design/fonts/bundles the brand's three faces (Modern Gothic Mono, UN 11ST, Oceanic Text) from the website rebrand set. Their licence covers the web; embedding them in an MIT-licensed app has not been cleared with Brand Studio. If that can't be cleared, the keyboard falls back to the system faces with a small code change and the files come out.How it was tested
Locally on a Mac with Xcode 27 (this merge commit):
swift buildandswift test(engine); the Mac app builds;scripts/ios-typecheck.sh;scripts/ios-test.sh(110 iOS unit tests, including a seeded fuzz of the keyboard's state machine — five seeds × 2000 random events with invariants after each — and a 200-round disappear/appear recovery test);scripts/check-invariants.sh(13 rules, self-test green);scripts/design-sync.sh --check;swift-format lint --strict;swiftlint --strict. In the simulator:scripts/ios-keyboard-flows.shfor all three layouts and the right-edge panel (0 failures) and the Messages carousel probe three times with pixel checks (the grille drawn at appearance and after every return).scripts/check.shas a whole was not run on this exact tree (the sanitizer passes take long); CI'scheckand the newios-buildjob will.scripts/check.shpasses (or CI will, if I'm not on a Mac) — see above; CI is the authority on this treemain; the keyboard rule is newAGENTS.md(layers, settled decisions),App/BlurtiOS/README.md,App/BlurtiOS/DESIGN.md,Sources/BlurtEngine/README.md🤖 Generated with Claude Code