feat: 28-language interview picker and mid-interview audio switching

[alpha5611331]

Closes #24.

Language had one member and was read by nothing. This makes it a real setting: a picker on the control bar that decides which speech model transcribes the call, what language suggestions are written in, and the language of the exported report - changeable mid-interview, not only before Start.

Since the first review this has grown two things, both driven by the backend moving its ASR to Deepgram Nova-3: the language set went from 6 to 28, and the microphone became switchable mid-interview for the same reason the language is.

Pairs with PowerInterviewAI/backend#58. Neither side needs the other to ship: the backend defaults the field, so this client works against a deployment that predates it (suggestions stay English, and so does the transcription). A code this build knows but an older backend does not is resolved back to English there rather than faked, so a client ahead of its backend degrades one session instead of breaking it.

What changed

src/main/types/language.ts, src/renderer/types/language.ts	the enum, mirrored the way SuggestionMode is, plus display metadata
src/main/store/config.store.ts	language is typed and resolved on read
src/main/services/suggestion-{live,action}.service.ts, tools.service.ts	language on all three request bodies
src/renderer/services/live-transcription.service.ts	?language= on the ASR sockets, setLanguage() to switch them, and setAudioInputDevice() / setStream() to swap the microphone
src/renderer/hooks/use-interview-language.ts	the setter that reconciles the two halves
src/renderer/hooks/use-audio-input-device.ts	the same shape for the microphone
src/renderer/components/custom/control-panel/language-group.tsx	the language control
src/renderer/components/custom/control-panel/audio-group.tsx	the microphone control, now live mid-interview

The setting has two halves that move at different speeds

Suggestions are free. Each service reads the config store as it builds its request, so the next suggestion already follows a change - nothing to reconnect, nothing to wait for.

The ASR is not. The provider takes the language as a connection parameter, so switching means tearing both channels down and re-opening them: a second or two of gap, and whatever utterance was mid-flight is orphaned. useInterviewLanguage is where those two are reconciled, and the UI reports the difference rather than papering over it - the trigger spins while the sockets come back, and the menu says what will happen before the user commits, because a two-second hole in the transcript is alarming if it arrives unannounced mid-question.

This is a property of the provider, not a choice. Deepgram has no mid-stream language change at all, so the reconnect machinery below is now load-bearing rather than incidental.

Two guards on the reconnect

Both protect against the same failure: two sockets on one channel, one of them orphaned and still relaying audio into a dead session. The existing onclose path schedules a backoff reconnect on any unexpected close, and a deliberate close for a language switch looks exactly like one.

• onclose now ignores a close from a socket that is no longer this.ws. That is the tail of a replacement, not a disconnect. (This one is a latent fix on the pre-existing reconnect path too.)
• setLanguage sets a switching flag so the backoff reconnect does not fire for the close it caused itself, and reconnects immediately instead of waiting out WS_RETRY_BASE_DELAY_MS.

connectWebSocket rebuilds the URL per attempt rather than capturing it, which is what lets a reconnect pick up the new language at all.

The first guard has two consequences worth calling out, both caught on review:

• setLanguage reports channelDisconnected itself rather than leaving it to onclose. new WebSocket assigns this.ws synchronously, so the old socket's close always arrives after its replacement exists and is correctly ignored - which would have swallowed the report too, leaving the orphaned partial gating live suggestions for the rest of the session.
• Its no-op check keys on this.ws, not on active, which start() sets only after its first connect returns. In that window a socket already exists on the old language and an active check would skip it.

A switch that cannot reconnect hands the channel back to the ordinary backoff loop rather than leaving it silent until the assistant is restarted - five failed attempts is a network or provider problem, not a permanent one - and the toast says it is still retrying. The two channels are switched with allSettled, since Promise.all leaves the second one's rejection unhandled.

The microphone is now switchable too, and it is cheaper

The audio control used to lock while the assistant ran, which made the one case it exists for unreachable. A headset that dies, is unplugged, or was the wrong device to begin with is noticed exactly when the interviewer says they cannot hear you - and the only fix was stopping the assistant, which clears the transcript and the suggestion history with it.

Nothing reconnects. The device is only what feeds the worklet; it is not a connection parameter. AudioWsStream.setStream() replaces the MediaStreamAudioSourceNode while the socket, the provider session and any utterance in flight all survive, so there is no gap and the dialog promises the opposite of what the language menu warns about.

Three things it has to hold:

• The replacement stream is acquired before anything is torn down, and the previous one stopped only after the swap succeeds. Releasing first reads as the obvious cleanup order and works every time the new device is present; on the one path that matters - unplugged, in use, permissions refused - it leaves the session with no microphone at all, mid-answer, having been asked only to change one.
• A stream that finishes opening after the session stopped is released, not left holding the device with its indicator light on, since nothing else keeps a reference to it.
• The existing AudioContext is reused. Its sampleRate is fixed at construction and convertTo16kPcm reads it, so a fresh context would resample every frame against the wrong rate - quietly, and only for users whose second device runs at a different rate than their first.
• The bail-out tests the context alone, never also the worklet node. start() creates source from this.stream and only assigns workletNode after await addModule(), so there is a real window where a context and a source exist and the node does not. Returning early there leaves source bound to the stream the caller is about to stop, and start() then wires that dead source into the graph - socket up, channel relaying silence for the rest of the session, nothing reporting it. Caught on review after the first implementation had exactly that guard.

Only ch_1 moves; ch_0 is loopback audio from the call and has no device to change.

The bug this widening exposed, and the one to read first

classifyInterviewerTurn returned Skip for every interviewer question in a non-Latin script - Japanese, Chinese, Thai, Russian, Korean, Arabic, Hindi, Hebrew. Skip drops the turn outright: no request, no card, no error. Roughly a third of the languages this PR adds produced no suggestions at all.

normalize() reduces a turn to ASCII, which is right for matching an English backchannel lexicon and wrong as a test for whether anything was said - those scripts reduce to nothing. Empty then hit the branch meant for [laugh] and (inaudible) and was read as non-speech.

The bug is as old as the classifier. Going from six Latin-script languages to 28 is what made it reachable, and it is exactly the failure the classifier's documented asymmetry exists to prevent: a question misread as filler produces nothing at all, mid-interview. It could not be seen from the English side, where that branch is correct.

The two cases are now told apart by whether any letter in any script survived the non-speech markers. If one did, the verdict is Uncertain, which defers to the backend gate - the one stage that can actually read the language, and which fails open. Skip stays for turns that really were non-speech. Non-ASCII question marks (？, ؟) fold to ? so a finished question in those scripts answers immediately rather than waiting out the settle timer; Greek's ; is deliberately left alone, since it is an ordinary semicolon everywhere else and reading it as terminal would answer English fragments.

Every English case is unchanged. test/interviewer-turn.test.mjs pins nine scripts that must not be skipped, the markers that must still be skipped, and the question marks that should answer outright.

Decisions worth arguing with

English sends no parameter at all, not language=en. The backend treats an absent language as English, so a session that never touches the picker produces byte-identical traffic to what shipped before this existed.

The setting is persisted before the switch and never rolled back on failure, for both controls. A failed reconnect that reverted the setting would leave the user with no route to the language they picked; leaving it set means stopping and starting the assistant recovers, and the toast says so. Same for the microphone, where a failed swap leaves the audio running on the previous device.

Resolution happens on the way out of the config store, not on the way in. The disk holds whatever some build wrote - a code a later release dropped, or one an older release never knew - and every consumer reads through getConfig(), so that is the single point where an unknown code can be stopped before it reaches the ASR URL and three request bodies.

28 languages, which is exactly what the backend's Deepgram Nova-3 ASR streams. It was six under the previous ASR. The rule did not change - offering one the ASR cannot hear would not degrade gracefully, it would answer a question that was never asked - only the ceiling did.

Endonym first in the menu, English name second. Someone whose interview has just switched language finds "Deutsch" faster than "German"; the English column is there for the reverse lookup. The trigger shows the code (EN, ES) next to the icon - a globe alone is only useful to someone who already knows what it is set to, which is the one question this control has to answer at a glance.

Two things the widening broke that six never exercised, both fixed here: the menu opens upward from the bottom-most control into an overflow-hidden main, so an uncapped 28-item list ran off the top of the window rather than flipping (it is now capped and scrolls); and the Arabic and Hebrew endonyms rendered into the menu's LTR flow, putting punctuation on the wrong side of the word (dir="auto").

Explicitly not in scope

Localising the app's own chrome - buttons, labels, dialogs, toasts. Separate feature, separate cost (~45 components plus every main-process error string), and not what makes the product unusable in Spanish today: an English button on a Spanish interview is an inconvenience, an English transcript of Spanish speech is a wrong answer read out loud.

Testing

pnpm lint, both tsc configs, pnpm build and pnpm test:main all clean.

test/language.test.mjs now also pins the two mirrors staying in step, which is the failure the widening made likely and which is silent in both directions: an enum member with no picker entry renders a blank trigger, and a picker entry with no enum member resolves straight back to English when picked.

test/audio-device-switch.test.mjs pins the acquire-before-release ordering and the reused AudioContext. These are source-level checks, unusually for this directory - every other test loads a built main-process module, and this is renderer code with no runtime harness. They are worth the awkwardness because the ordering is exactly what a later tidy-up breaks, with no symptom a type checker or linter can see.

Not covered: the reconnect and the swap themselves. AudioWsStream needs a real WebSocket, AudioContext and getDisplayMedia, none of which the dependency-free main-process harness reaches. Both are worth exercising by hand:

• Change language during a running session; confirm the transcript resumes on the new language, exactly one socket per channel survives, and the card that was in flight is not left pending.
• Change microphone during a running session; confirm the transcript does not gap, and that picking a device that is unplugged mid-dialog leaves the interview running on the previous one.

[gitar-bot]

Gitar is working

_Gitar

[alpha5611331]

Multilingual review pass

Reviewed the language feature end to end across this PR and PowerInterviewAI/backend#58. Eleven commits pushed here. eslint, both tsc configs, vite build and pnpm test:main are green.

Bugs found and fixed

Non-Latin questions that open on "OK" were dropped outright (780ca6e) - the worst of these. The earlier fix covered turns that normalize to nothing; a turn is not required to be in one script. OK、では次の質問です。 normalizes to exactly ok, the backchannel lexicon eats it, the core comes back empty and the question is skipped by the core.length === 0 branch without the empty-normalized branch ever running. Opening a turn on "OK" or "Yes" is ordinary in Japanese, Korean, Chinese, Russian, Greek, Arabic, Hebrew, Thai and Hindi, and Deepgram transcribes those loanwords in Latin script. Ten new cases fail without the guard.

The merge separator rewrote the whole session's history (969815a) - cleaned is rebuilt from every stored transcript on each ingest and the separator was read once per rebuild from the current config, so switching an English interview to Japanese an hour in stripped the spaces out of every block merged so far, on screen and in the transcript the next request carries. Transcript now carries the language it was transcribed in.

Two overlapping language switches leaked an open socket (d041578, e75e06c) - both switches run a connectWithRetry loop that assigns this.ws synchronously per attempt, so the older loop waking from its backoff overwrites the newer one's socket, and the newer one is then unreferenced: stop() never closes it and the Deepgram session behind it stays open for the life of the app. The picker looks like it prevents this and does not - switching is only set after the config write is awaited, so a second pick lands in the gap. Gave the channel the generation token the microphone path already had, and the same guard to both hooks.

Nothing set a text direction for Arabic or Hebrew (4d11e7a, c2aeb75) - the picker's endonyms did; the transcript, the question line and the answers did not. The visible failure is the neutrals, not the run: sentence-final punctuation takes the paragraph's direction, so the question mark lands at the wrong end, and any answer reorders at each script switch (which is every answer, since the prompts keep product names and code in Latin). Block code is pinned ltr. Separately, the docs dialog joined all 28 endonyms into one string, so Arabic and Hebrew merged into a single RTL run and printed in the opposite order to every other pair.

NO_SUGGESTION_NEEDED did not match behind a directional mark (cb8badc) - the string the backend prompts single out as failing hardest. U+200F is a format character, not whitespace, so \s left it in place and the sentinel rendered on screen as the answer. Models writing RTL routinely open a response with one.

The exported report was half-translated (ae3f321) - the summarize prompt translates the headings it writes, on the stated grounds that a half-English report looks unfinished; the client then wrapped that Spanish summary in # **Transcripts** and an | Interviewer on every turn. Five nouns per language in export-labels.ts. The report is the one artifact that leaves the machine, which is why it is the documented exception to the app chrome staying English.

Also 5fc70ac (the mirror test counted rows and measured label width, but not that each row's short is its own code - a half-edited copied row shows the wrong language on the control bar and reads as correct) and a stale format_turns comment left behind by the AssemblyAI migration.

Verified, no change needed

• All 28 codes are valid Nova-3 streaming languages per Deepgram's docs, including no and id.
• Deepgram parameter and message names all match the reference.
• {ja, zh, th} agree between transcript-join.ts and the backend's _UNSPACED_LANGUAGES.
• configStore.getConfig() resolves on the way out and both IPC handlers return through it.
• Greek's ; in FOREIGN_QUESTION_MARKS is U+037E, not ASCII - code and CLAUDE.md agree.
• The classifier cannot false-Skip in the 17 Latin-script languages: a question keeps its ? glued to its last token, which blocks full consumption.

Left alone, worth a decision

1. Nothing detects backend/client language drift. A code the client offers and the backend does not resolves silently to English - transcription and answers both, with the picker showing a check beside the language the user asked for. This is the "confident answer to a question that was never asked" failure the design is built around, and no in-repo test can catch it. The cheap fix is a protocol echo: have the ASR socket send the resolved language as its first frame, and raise the existing half-applied warning when it differs from what was requested.
2. The last frozen-but-unended words are lost on every stop and every switch. The backend's close() flush cannot reach a client that closed the socket first, which is what both paths do (see backend#58 for the corrected docstring). The stop control frame already exists server-side and breaks the relay loop with the socket still open; nothing sends it. The menu already warns for switches, but Stop drops the tail right before an export.
3. The credit timer restarts on every ASR connect, and the picker now lets a user reconnect on demand. At a 12-second interval the leak is small, but it is newly reachable from a button.
4. hero has no multilingual copy at all. Nothing there contradicts the feature today, so this is a follow-up rather than a fix - and adding public docs for an unmerged feature needs its own branch.