On-device LLM polish on a second hotkey via Apple Foundation Models (#36) - #54
Merged
Merged
Conversation
`HotkeyMonitor.start` takes `[HotkeyRole: Hotkey]` and reports
`HotkeyMonitorEvent { role, event }`. `HotkeyChordSet` owns one unchanged
`HotkeyChordTracker` per role, feeds every keyboard transition to all of
them and swallows the union; the tap monitor holds one behind its lock.
The Carbon monitor registers one hot key per chord under one handler,
with the role in the low bits of the hot key ID, and skips a chord it
cannot register on its own.
The coordinator remembers which role started the recording and lets
only that role's release or cancel end it, so two nested chords hand
over cleanly.
No behaviour change: only the dictate chord is registered. The polish
chord arrives in the next commit; #38's toggle chord is one enum case.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ranch `Settings.polishHotkey` is a second recordable chord, empty (off) by default. The coordinator registers it beside the dictate chord unless it is empty or equal to it; a press tags the cycle, publishes `willPolish` for the overlay and warms the refiner in a detached utility task while the user is still speaking. The release path gains one branch, after the pipeline and the empty guard: on a polish cycle with at least `minimumPolishWords` (4) words the state goes to `.polishing`, the injected `TranscriptRefiner` runs, and nil falls back to the processed text. On the normal path the only new work is one Bool read. The word gate sits in the coordinator so the fake-refiner tests cover it and the refiner never sees three words. `CycleTiming.polish` is nil on the normal path and the model's wall time on a polish cycle, zero when the gate skipped it, so the two paths can be logged apart. `.polishing` gets its arm in the menu bar glyph, the status line and the overlay, which keeps the pill up from release to paste on a polish cycle. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A new target, the only one that imports FoundationModels. `OnDeviceLanguageModel` is instructions plus options: availability as a value, a prewarmed session, a plain and a guided `respond`, an eight second budget, and the drain of a call that ran past it. #37 builds a second instance with its own instructions. `TranscriptPolisher` is the polish hotkey's `TranscriptRefiner` on top of it, and `PolishPostFilter` strips a leading think block and U+200B, U+200C, U+200D and U+FEFF. The lessons of the removed tidy pass (8537063), and how each is kept: - `respond` awaited on an actor's executor turned sub-second replies into timeouts: the call and the timer run in detached tasks. - The system model serialises requests: a call abandoned at the budget is parked in a Mutex and the next call drains it first. - One session per exchange: `prepare()` warms one at key-down, `refine` takes it and drops it. - Guided generation with greedy sampling: the model fills a `@Generable` `cleanedText` field; a decoding or unsupported-guide failure retries once with plain `respond(to:)` on a fresh session. Anything else (unavailable, refused, timed out, empty) returns nil and the coordinator pastes the text as dictated. One log line per call, numbers and the error's case name only, never the transcript. `pladder-cli polish <text file | -> [--instructions <file>]` runs the prompt cold and warm and prints both timings; nothing in `swift test` calls the model. The prompt is the plan's rule list tuned with that harness: three inline examples (a self-correction, a spoken list, a question with a request that must be kept), German fillers and "nein" in the lists, and the language named in the user message ("Transcript, in German:") from NLLanguageRecognizer, because the English examples otherwise pulled a German transcript into English. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`AppModel` injects `TranscriptPolisher` into the coordinator and polls the model's availability with the permissions, since Apple Intelligence can be switched in System Settings while the app runs. One new row under Push to Talk, "Dictate and polish" (German "Diktieren und überarbeiten"), with a recorder like the send key's and one warning line, in order: - empty: nothing, the key is off (the default); - equal to the push-to-talk chord: it is never registered, so say so; - Apple Intelligence off, not eligible, still downloading or otherwise unavailable: the row stays and says the key pastes the text as dictated until it is available; - a macOS shortcut owns the chord: the existing conflict sentence; - without Accessibility a chord Carbon cannot register: the key is off until the grant, there is no stand-in for it; - no modifier: the existing "can no longer be typed" sentence. A second footer line says what the key does and that it takes a second or two; the existing line is untouched. A polish cycle logs `polished release-to-paste … s: stop …, engine …, process …, polish …, paste …` under the same `timing` category; the plain line is byte for byte what it was. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
CLAUDE.md gains the `pladder-cli polish` command, a decisions row for the Dictate and polish key, the "adding a prompt" rule for `OnDeviceLanguageModel`, and the polish latency and quality risk; the post-processing row no longer reads as if Apple Intelligence were gone for good. BENCHMARKS.md shows the `polished release-to-paste` line and says the plain one is unchanged. PERFORMANCE.md, README.md (a feature bullet and a FAQ entry) and INSTALL.md say where the model is used. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The drain of a call that ran past its timeout was awaited before the next call's timer started, so a system model that never answered would have held every later polish, and its paste, with no limit. The drain now runs in the raced task: the next call still queues behind the old one, but the eight-second budget covers both, and past it the text is pasted as dictated. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This was referenced Sep 23, 2026
Owner
Author
Live test (integration build, M1, installed to /Applications)The installed binary ran with its own settings file (
Not tested live: the Carbon path (no Accessibility) and German UI strings. |
CI builds with Xcode 26 (Swift 6.3.3, macOS 26 SDK), where GenerationOptions only has `sampling:`; the macOS 27 SDK renamed it to `samplingMode:` and deprecated the old label. Pick by compiler version, so neither toolchain fails or warns. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #36.
What
A second recordable hotkey, Dictate and polish, off by default. A dictation started with it runs the usual pipeline, then Apple's on-device model (FoundationModels), then pastes. The normal hotkey's path is unchanged apart from one Bool read.
PladderRefine, the only one that imports FoundationModels.OnDeviceLanguageModelis a generic wrapper: availability, a prewarmed session, plain and guided (@Generable)respond, greedy sampling, an 8 s timeout race, and the drain of an abandoned call.TranscriptPolisherputs the cleanup prompt on top. Learn dictionary entries from the user's corrections, reviewed on device #37 reuses the wrapper with its own instructions.PladderCorestays Foundation-only. It gets aTranscriptRefinerprotocol (prepare(),refine(_:) -> String?), injected into the coordinator, plus aFakeRefinerin the tests.if willPolishbranch after the pipeline sets.polishing, awaits the refiner, and falls back to the processed text on nil. Transcripts under four words skip the model. The session is prewarmed at key-down of the polish key.<think>/<thinking>block and U+200B/U+200C/U+200D/U+FEFF.CycleTiming.polishis nil on the normal path. A polish cycle logs its ownpolished release-to-paste … polish X.XXX, …line, so the plain line the benchmark rule watches is unchanged.pladder-cli polish <file> [--instructions <file>]is the harness for tuning the prompt without dictating live.Hotkey layer (first commit, no behaviour change)
HotkeyRole { dictate, polish },HotkeyMonitorEvent { role, event }, andHotkeyMonitor.start(chords: [HotkeyRole: Hotkey], submitKey:).HotkeyChordSetowns one unchangedHotkeyChordTrackerper role. Carbon registers one hot key per chord under one handler, with the role in the low bits of the ID. #38 builds on this commit; see there.Review fix
727d3f8: the drain of an abandoned model call ran before the next call's timer started. A system model that never answered would therefore have held every later polish, and its paste, with no limit. The drain now runs inside the budget.Prompt
The plan's prompt neither resolved self-corrections nor made lists (all variants tried are in the commit body). The final prompt has explicit rules, three inline examples, and German filler and correction words. The user message names the transcript's language (
NLLanguageRecognizer), because English examples otherwise pulled German transcripts into English.pladder-cli polish, M1, warm (prepared session, the path a real key press takes):-itemsBenchmark (critical-path rule)
The code between
recordingStoppedandinsertedchanges: there is oneif willPolishafter the pipeline, andfinal/toInsertin place ofprocessed. On the normal path that is one Bool read.polishHotkeyRoutesThroughTheRefinerandnormalHotkeyNeverCallsTheRefinerprove the branch is not taken.Whole-buffer bench,
pladder-cli bench bench/fixtures, M1, macOS 27.0, six runs per fixture with the first discarded, 10 s idle before each, no thermal tags:main(be197e0)Load average 3.0 → 2.7 on
mainand 2.3 → 3.3 on the branch. Every difference is under 6 %, which is noise. The bench times the engine, which is untouched. WER is identical.Tests
PladderCoreTests: 226 (was 201): 7
HotkeyChordSetTests, 18PolishHotkeyTestswith the fake refiner (polish routes through the refiner, normal does not, short and blank skip, nil falls back, unavailable degrades, the timing stage, the overlay flag, a polish chord equal to the dictate chord is not registered). PladderRefineTests: 10 (post-filter, chunking, prompt framing). None calls the model.swift build,swift testandswift build -c releasepass at every commit.Notes for the reviewer
integration/36-37-38resolves them and adds a Delete-to-clear to this row (from Hold-or-toggle hotkey and Escape to cancel #38's recorder change) and a warning when the polish chord equals the toggle chord.🤖 Generated with Claude Code