Add read-aloud work mode and hold-to-ask

.claude/skills/project-guardrails/SKILL.md

@@ -61,7 +61,12 @@ genuinely correct, and reaching for it means it's time to stop and ask.
   transcribes, so it is not a door back to streaming STT.
 - **No separate LLM cleanup pass.** Cleanup rides in the same dictation request,
   as `config.llm_instruction` (`CleanupInstruction`). No LLM Gateway
-  client, no `StylerProtocol`, no post-transcription styling stage.
+  client for dictation, no `StylerProtocol`, no post-transcription styling
+  stage. The one gateway client in the tree is the experimental read-aloud
+  feature's (`TTS/ReadAloudLLM.swift`, the only file `check-invariants.sh`
+  exempts from this rule): the skip-jargon listening rewrite of a selection, and
+  the spoken answer to a request about one. It never cleans up a dictation, so
+  it is not a door back to a cleanup pass.
 - **No local models / model downloads.** Transcription is a remote AssemblyAI
   call. No on-device ASR/LLM, no model cache, no download UI.
 - **There is no `config.conversation_context`.** `config.stt_prompt`

AGENTS.md

@@ -78,6 +78,10 @@ Sources/BlurtEngine/         the engine (dependency-free Swift package)
   Pipeline/                  DictationSession (actor) + phases, UI projections, geometry, log
   STT/                       AssemblyAITranscriber, Conversation/TranscriptionContext, KeytermsBoost,
                              CleanupInstruction, SyncSTTLimits
+  TTS/                       experimental read-aloud: SelectionSpeaker, AssemblyAISpeechSynthesizer
+                             (the one WebSocket), SpeechSegmenter, StreamingPCMPlayer +
+                             PCMSchedule, SelectionCopy, ReadAloudLLM (the skip-jargon rewrite
+                             and the answer to a spoken ask, the one gateway client)
 App/Blurt/
   project.yml                XcodeGen source of truth — Blurt.xcodeproj is GENERATED
   Blurt/                     the shell: App.swift scenes, AppCoordinator, Wizard/, Overlay/,
@@ -400,7 +404,7 @@ rule along with the row.
 | Pre-open the mic to shave bring-up latency (a warm/prepared recorder)          | Measured against `kAudioDevicePropertyDeviceIsRunningSomewhere`: neither building an `AVCaptureSession` nor the retired `AVAudioRecorder.prepareToRecord()` opens the device, so neither pre-pays the 180–600 ms route activation `record()` costs. A warm-recorder lifecycle bought ~15 ms and cost a device-identity check, a pin check, an expiry and a bring-up flag. `MicCapture.warmUp()` is stateless: it absorbs the process's first-touch cost and holds nothing.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
 | Add streaming STT                                                              | The dictation API returns the full (already rewritten) text in one response; the overlay shows "Transcribing…" then the full text. The one WebSocket allowed is the experimental read-aloud feature's streaming **TTS**, confined to `Sources/BlurtEngine/TTS/AssemblyAISpeechSynthesizer.swift` (the invariant gate exempts only that file).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
 | Call the dictation API in a way its docs don't describe                        | **The reference is the source of truth.** Every path, header, multipart part, `config` key, response key and error key Blurt sends or reads is one it documents. The route has been renamed repeatedly and still answers to names the docs have dropped — `prompt`, `keyterms`, `word_boost`, `language_code`, `llm: null` — so a 200 is not evidence a field is supported, only that it has not been removed yet; treat anything absent from the docs as deprecated. Undocumented behaviour carries no compatibility promise, so a feature resting on it can stop working silently: that is why `llm: null` (the only rewrite off switch there is) and the `message` error field were dropped on 2026-09-11, along with `warmUp()`'s GET at the bare host root in favour of the documented `GET /warm`. Running tighter than a documented cap is fine — `STTPrompt.characterCap` does. Measuring is still how a claim gets settled, and the docs have been wrong twice (see `DictationWireTypes` on unknown fields, `STTPrompt` on the 6000 cap), but a measurement licenses _distrusting_ a documented field, never _sending_ an undocumented one. |
-| Add a client-side LLM cleanup pass                                             | Cleanup is the dictation API's server-side rewrite, requested by `config.llm_instruction` on the same `/v1/transcribe/live` call. No LLM Gateway client, no `StylerProtocol`, no styling stage, no second request — transcription steering belongs in `STTPrompt`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+| Add a client-side LLM cleanup pass                                             | Cleanup is the dictation API's server-side rewrite, requested by `config.llm_instruction` on the same `/v1/transcribe/live` call. No LLM Gateway client, no `StylerProtocol`, no styling stage, no second request — transcription steering belongs in `STTPrompt`. The one gateway client allowed is the experimental read-aloud feature's, confined to `Sources/BlurtEngine/TTS/ReadAloudLLM.swift` (the invariant gate exempts only that file): the skip-jargon listening rewrite of a selection, and the spoken answer to a request about one, neither of which the TTS socket has a server-side equivalent for. It never cleans up a dictation; the only transcript it sees is a hand-back press's request (`.handedBack`), which is never pasted.                                                                                                                                                                                                                                                                                                                                                                                               |
 | Add local models or model downloads                                            | Transcription is a remote AssemblyAI call: no on-device ASR/LLM, no model cache, no download UI.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
 | Pin transcription to English, or set a language at all                         | Hurt non-English transcription; language is left to the model's own detection. **No `config.language_codes`** either — note the plural, which is the documented spelling, though the undocumented singular `language_code` is accepted too (a 200, not the unknown-key 400 a bogus name earns), so both are live and both are pinned absent. Detection was measured to work with neither it nor a prompt set (es/fr/de/ja clips each transcribed in their own language against the live endpoint, rewrite included), and re-measured 2026-09-11: a Spanish clip came back as Spanish with no field, with `language_codes: ["es"]`, **and** with `language_codes: ["en"]` — asking for English did not force English, so the reference's `["en"]` default does not describe this route and omission is not a bet on a default. Setting one would only take working detection away. `KeytermsWireTests` asserts the absence of both spellings.                                                                                                                                                                                                         |
 | Send `config.prompt` alongside `config.stt_prompt`                             | They are the same field under two names — the docs give `stt_prompt` as the field and note it is "Also accepted as `prompt`" — and a request carrying both is rejected before the audio is read (`provide only one of stt_prompt or prompt; they are the same field`). So this is never a compatible addition: it is a 400 on _every_ dictation. `stt_prompt` is the documented spelling and the only one to send; `prompt` is the legacy one, and the route's own inner validator still answers under it (see `STTPrompt.characterCap`), which is a reason to treat it as deprecated rather than interchangeable.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
@@ -732,7 +736,11 @@ It exposes `press()` / `release()` / `cancel()` / `cancelRecording()`, a synchro
 fire-and-forget `submit(_: Command)` mirroring those four for callback-shaped hosts (commands run in
 exact emit order — the tap wires straight into it, no per-callback `Task` spawning), and a
 `phase: PipelinePhase` (`idle | starting | recording | transcribing | injecting | failed |
-cancelled`, plus the terminal successes `pasted` / `noTarget`). `phaseStream()` yields the current
+cancelled`, plus the terminal successes `pasted` / `noTarget` / `handedBack`). `press(handingBack:
+true)` (`Command.pressHandingBack`) is the one variation: the same dictation, whose transcript ends
+the run as `.handedBack(text)` instead of being pasted or remembered. Read-aloud's "ask about the
+selection" is its only caller, and it rides the phase rather than a callback so the router sees the
+transcript and the dictation's end in one order. `phaseStream()` yields the current
 phase immediately then
 every transition, and is **multi-observer** (one continuation per call), though hosts should still
 render from one consumer and project the phase into their own state.

App/Blurt/Blurt/AppCoordinator.swift

@@ -142,6 +142,7 @@ final class AppCoordinator {
       onStart: { router.submit(.press) },
       onStop: { router.submit(.release) },
       onCancel: { router.submit(.cancel) },
+      onLatch: { router.keyLatched() },
       // Recovery-only teardown; `cancelRecording()`'s doc owns the rationale.
       onRecordingDiscarded: { router.submit(.cancelRecording) }
     )
@@ -289,8 +290,10 @@ final class AppCoordinator {
     // pipeline phases. No-op whenever the gate is already idle, which is every
     // normal flow. Asked through the router, because while a read-aloud owns the
     // gate the reset isn't this dictation's to make (see `SelectionSpeechRouting`).
+    // It gets the phase itself because an ask about a selection ends in one,
+    // `.handedBack`, carrying the transcript the router answers.
     if phase.isTerminal {
-      speechRouter?.sessionReachedTerminalPhase()
+      speechRouter?.sessionReachedTerminalPhase(phase)
     }
   }
 }

App/Blurt/Blurt/Hotkey/DictationKeyTap.swift

@@ -30,6 +30,9 @@ final class DictationKeyTap {
   private let onStart: @MainActor () -> Void
   private let onStop: @MainActor () -> Void
   private let onCancel: @MainActor () -> Void
+  /// A tap latched the recording on. Dictation ignores it (the recording just
+  /// keeps going); read-aloud's router reads it as "that was a tap, not a hold".
+  private let onLatch: @MainActor () -> Void
   /// Fired when a *state-recovery* reset (disabled-tap recovery, trigger
   /// rebinding) discards a live gate state: the key events that would have ended
   /// that dictation can no longer arrive, so the owner must end the capture —
@@ -66,11 +69,13 @@ final class DictationKeyTap {
     onStart: @escaping @MainActor () -> Void,
     onStop: @escaping @MainActor () -> Void,
     onCancel: @escaping @MainActor () -> Void,
+    onLatch: @escaping @MainActor () -> Void,
     onRecordingDiscarded: @escaping @MainActor () -> Void
   ) {
     self.onStart = onStart
     self.onStop = onStop
     self.onCancel = onCancel
+    self.onLatch = onLatch
     self.onRecordingDiscarded = onRecordingDiscarded
     // Every half of the binding comes from its store, not a hard-coded
     // `.rightCommand` / `.tapOrHold`: `fromPersisted` owns the unset default, and
@@ -238,6 +243,7 @@ final class DictationKeyTap {
     case .start: onStart()
     case .stop: onStop()
     case .cancel: onCancel()
+    case .latch: onLatch()
     case .none: break
     }
   }

App/Blurt/Blurt/Hotkey/SelectionSpeechRouter.swift

@@ -3,24 +3,29 @@ import BlurtEngine
 
 /// Sits between the trigger key and the dictation session, and turns a press
 /// over selected text into "read it aloud" when the experimental switch is on
-/// (`SelectionSpeechStore`). When the switch is off, every command goes straight
-/// to `DictationSession.submit`.
+/// (`SelectionSpeechStore`), or, holding the key with asking on
+/// (`SelectionAskStore`), into "answer what I'm saying about it". When the
+/// switch is off, every command goes straight to `DictationSession.submit`.
 ///
 /// The decisions are the engine's `SelectionSpeechRouting`. This class carries
-/// out the actions it returns: the selection read, the speech task, and the
-/// gate resets.
+/// out the actions it returns: the selection read, the hold timer, the speech
+/// task, and the gate resets.
 ///
 /// The cost: while the switch is on, a dictation press starts one AX read later.
 /// That's a few milliseconds in a native app, but can be tens to hundreds in an
 /// Electron app, and up to the AX timeout for each round trip against a hung
 /// one. Recording starts at the press, so a slow read clips the first words.
+/// An ask doesn't pay it: its recording starts at the hold, after the start
+/// chime the user waits for.
 final class SelectionSpeechRouter {
   private static let logger = HostIdentity.current.logger("SelectionSpeech")
 
   private let session: DictationSession
   private let speaker = SelectionSpeaker()
   private var routing = SelectionSpeechRouting()
   private var speech: Task<Void, Never>?
+  /// The pending `holdElapsed`, restarted by each press that may ask.
+  private var holdTimer: Task<Void, Never>?
   /// `DictationKeyTap.syncAfterTerminalPhase`: a dictation ended without a key event.
   var syncGateAfterSession: () -> Void = {}
   /// `DictationKeyTap.resetGate`: a read ended without a key event.
@@ -31,11 +36,18 @@ final class SelectionSpeechRouter {
   }
 
   func submit(_ command: DictationSession.Command) {
-    perform(routing.submit(command, readAloudEnabled: SelectionSpeechStore().isEnabled))
+    perform(
+      routing.submit(
+        command, readAloudEnabled: SelectionSpeechStore().isEnabled, askEnabled: SelectionAskStore().isEnabled))
   }
 
-  func sessionReachedTerminalPhase() {
-    perform(routing.sessionReachedTerminalPhase())
+  /// `DictationKeyTap`'s latch: the press was a tap.
+  func keyLatched() {
+    perform(routing.keyLatched())
+  }
+
+  func sessionReachedTerminalPhase(_ phase: PipelinePhase) {
+    perform(routing.sessionReachedTerminalPhase(phase))
   }
 
   private func perform(_ actions: [SelectionSpeechRouting.Action]) {
@@ -48,7 +60,15 @@ final class SelectionSpeechRouter {
           guard let self else { return }
           self.perform(self.routing.selectionResolved(selection))
         }
-      case .speak(let text): speak(text)
+      case .startHoldTimer: startHoldTimer()
+      case .speak(let text):
+        // Read per press, like the switch itself, so a Settings change applies
+        // to the next read.
+        let style = ReadAloudStyleStore().style
+        run { [speaker] in try await speaker.speak(text, style: style) }
+      case .answer(let request, let selection):
+        let style = ReadAloudStyleStore().style
+        run { [speaker] in try await speaker.answer(request, about: selection, style: style) }
       case .stopSpeech:
         speech?.cancel()
         speech = nil
@@ -58,11 +78,23 @@ final class SelectionSpeechRouter {
     }
   }
 
-  private func speak(_ text: String) {
-    let speaker = speaker
+  /// One timer at a time: a newer press cancels the older one's, so a timer
+  /// left over from a quick tap can't mark the next press as a hold.
+  private func startHoldTimer() {
+    holdTimer?.cancel()
+    holdTimer = Task { [weak self] in
+      try? await Task.sleep(for: SelectionSpeechRouting.askHoldDelay)
+      guard !Task.isCancelled, let self else { return }
+      self.holdTimer = nil
+      self.perform(self.routing.holdElapsed())
+    }
+  }
+
+  /// Runs one read or answer as the current speech task.
+  private func run(_ work: @escaping @Sendable () async throws -> Void) {
     speech = Task { [weak self] in
       do {
-        try await speaker.speak(text)
+        try await work()
       } catch is CancellationError {
       } catch {
         Self.logger.error("read-aloud failed: \(error.localizedDescription, privacy: .public)")