diff --git a/docs/primitives/a11y.md b/docs/primitives/a11y.md index d062eca..92cc3fd 100644 --- a/docs/primitives/a11y.md +++ b/docs/primitives/a11y.md @@ -2,7 +2,7 @@ ## Announcer Library -The `Announcer` library allows for relevant information to be voiced along the focus path of the application. The `Announcer` traverses the `focusPath` of the app collecting strings or promises of strings to announce to the user. The array of information is passed to a SpeechEngine which is responsible for converting the text to speech. By default we use the [speechSynthesis API](https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesis), but you can replace this by overwriting `Announcer._textToSpeech`. +The `Announcer` library allows for relevant information to be voiced along the focus path of the application. The `Announcer` traverses the `focusPath` of the app collecting strings or promises of strings to announce to the user. The array of information is passed to a speech engine which is responsible for converting the text to speech. By default we use the [speechSynthesis API](https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesis), but you can replace it with a platform TTS via [`Announcer.setSpeechEngine`](#custom-speech-engine). Note: The speechSynth api has some known problems:
https://stackoverflow.com/questions/39391502/js-speechsynthesis-problems-with-the-cancel-method
@@ -85,6 +85,141 @@ You may also use `PAUSE-#` to pause speech for # seconds before saying the next ['PAUSE-2.5', 'Hello there!']; ``` +## Custom Speech Engine + +Some TV platforms ship their own text-to-speech service rather than the Web Speech API — webOS (Luna), Tizen, or a native bridge exposed by the app shell. Pass an implementation to `Announcer.setSpeechEngine()` and every announcement is routed through it instead: + +```js +import { useAnnouncer } from '@solidtv/solid/primitives'; + +const Announcer = useAnnouncer(); + +Announcer.setSpeechEngine({ + speak: (phrase) => webOSLunaTTSSpeak(phrase), + cancel: () => webOSLunaTTSStop(), +}); +``` + +Call it with no arguments to restore the default speechSynthesis engine: + +```js +Announcer.setSpeechEngine(); +``` + +### The engine interface + +An engine only speaks a single phrase. The `Announcer` still owns everything around it — flattening strings, `PAUSE-#` delays, nested arrays / promises / functions, `append`, `cancel` and the network retry — so an integration is usually a few lines. + +```ts +interface SpeechEngine { + speak: ( + phrase: string, + options: { lang: string; voice?: string }, + ) => void | Promise; + cancel: VoidFunction; +} +``` + +| member | description | +| -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `speak` | Speaks one phrase. `options.lang` is `Announcer.lang` (or the utterance's `lang`), `options.voice` is `Announcer.voice`. Return a promise that resolves when the phrase has finished so the series waits for it; return nothing and the series moves on to the next phrase immediately. | +| `cancel` | Stops whatever is currently being spoken. Called by `Announcer.cancel()` and whenever a new announcement replaces an in-flight one. | + +Reporting completion matters for pacing. If the platform gives you a callback, wrap it: + +```js +Announcer.setSpeechEngine({ + speak: (phrase, { lang }) => + new Promise((resolve, reject) => { + webOSLunaTTS.speak(phrase, lang, { + onComplete: resolve, + onFailure: reject, + }); + }), + cancel: () => webOSLunaTTSStop(), +}); +``` + +### Errors + +Rejecting from `speak` signals a failure. The rejection's `error` property is used to classify it, mirroring the [speechSynthesis error codes](https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesisErrorEvent/error): + +| `error` value | behavior | +| ---------------------------- | -------------------------------------------------------------- | +| `'network'` | Retried up to 3 times with a growing back-off | +| `'canceled'`/`'interrupted'` | Treated as benign — the series ends quietly, nothing is logged | +| anything else | Propagates to the caller | + +```js +const error = new Error('TTS unavailable'); +error.error = 'network'; // gets retried +throw error; +``` + +### Notes + +- Setting an engine is global and takes effect immediately; it does not need to happen before `useAnnouncer()`. +- The engine is bypassed when `Announcer.aria` is `true`, since that mode writes to an `aria-live` region for the platform screen reader instead of speaking directly. +- The default engine is the only code that touches `window.speechSynthesis`, so an engine works on devices where the Web Speech API is missing entirely. + +## Device Detection (LG & Samsung) + +`Announcer.detectSpeechEngine()` tests the device and switches to that TV's own speech output. It only reads globals — it never speaks — so it is safe to call during startup, and it returns the platform it picked. + +```js +import { useAnnouncer } from '@solidtv/solid/primitives'; + +const Announcer = useAnnouncer(); +const platform = Announcer.detectSpeechEngine(); // 'webos' | 'tizen' | 'default' +``` + +| platform | device | what gets configured | +| --------- | ------------- | --------------------------------------------------------------------- | +| `webos` | LG | Installs a Luna TTS engine and sets `Announcer.aria = false` | +| `tizen` | Samsung | Sets `Announcer.aria = true` so Voice Guide reads the announcements | +| `default` | Anything else | Restores the speechSynthesis engine and leaves `Announcer.aria` as-is | + +The two platforms work in fundamentally different ways, so it is worth knowing which one you are on. + +### LG (webOS) — Luna TTS + +webOS exposes the same TTS service its Audio Guidance uses, so we can hand it strings directly. Announcements go through `luna://com.webos.service.tts`, and each phrase subscribes for feedback so the promise settles when the TV has actually finished speaking — which keeps `PAUSE-#` timing in the middle of a series accurate. + +Two things are required on device: + +1. **webOSTV.js must be loaded.** Detection is a capability test for `webOS.service.request`, not a user-agent match. If the app is on webOS but the library is missing, detection returns `default`, logs a warning, and falls back to speechSynthesis. +2. **The service must be in your app permissions.** Add it to `appinfo.json`: + +```json +{ + "requiredPermissions": ["com.webos.service.tts"] +} +``` + +Without the permission every request fails. Rather than reject on each phrase, the engine logs one warning naming the permission and stays quiet, so a config typo doesn't turn into a stream of unhandled rejections. + +To pass an `appID` (which lets `stop` target only your app's messages) build the engine yourself: + +```js +import { createWebOSEngine } from '@solidtv/solid/primitives'; + +Announcer.setSpeechEngine( + createWebOSEngine(webOS.service.request.bind(webOS.service), { + appID: 'com.example.myapp', + }), +); +``` + +### Samsung (Tizen) — Voice Guide + +**Samsung does not provide an API for an app to speak a string.** The TTS on a Samsung TV is the built-in Voice Guide screen reader, and it only reads the DOM. So there is no engine to install — `detectSpeechEngine()` instead turns on `Announcer.aria`, which writes each announcement into an `aria-live="assertive"` region for Voice Guide to pick up. + +Consequences worth planning for: + +- **Nothing is audible unless the viewer has Voice Guide turned on** in the TV's accessibility settings. That is a system setting; the app cannot enable it. Detection warns in the console when it can tell that Voice Guide is off, and you can check it yourself with `isTizenVoiceGuideEnabled()` (which returns `undefined` on models where it can't be read). +- **`Announcer.voice` is ignored**, and language comes from the `lang` attribute on the injected span rather than from the engine. +- **Timing is approximate.** The Announcer can't observe when Voice Guide finishes a phrase, so a series resolves as soon as the labels are written. `PAUSE-#` still delays, but it doesn't pace against real speech the way the webOS path does. + ## API ### Properties @@ -96,11 +231,13 @@ You may also use `PAUSE-#` to pause speech for # seconds before saying the next ### Methods -| name | args | description | -| -------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| speak | | Performs a manual announce | -|   | `announcement` | See _SpeechType_ above | -|   | `options` | Object containing one or more boolean flags:
| -| clearPrevFocus | `depth` | Clears the last known focusPath - depth can trim known focusPath | -| cancel | none | Cancels current speaking | -| setupTimers | `options` | Object containing:
| +| name | args | description | +| ------------------ | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| speak | | Performs a manual announce | +|   | `announcement` | See _SpeechType_ above | +|   | `options` | Object containing one or more boolean flags:
| +| clearPrevFocus | `depth` | Clears the last known focusPath - depth can trim known focusPath | +| cancel | none | Cancels current speaking | +| setupTimers | `options` | Object containing:
| +| setSpeechEngine | `engine` | Replaces the text-to-speech backend - see [Custom Speech Engine](#custom-speech-engine). Call with no argument to restore the speechSynthesis default. | +| detectSpeechEngine | none | Tests the device and switches to its built-in speech output, returning `'webos'`, `'tizen'` or `'default'` - see [Device Detection](#device-detection-lg--samsung). | diff --git a/src/primitives/announcer/announcer.ts b/src/primitives/announcer/announcer.ts index b0ae90f..309afbe 100644 --- a/src/primitives/announcer/announcer.ts +++ b/src/primitives/announcer/announcer.ts @@ -1,6 +1,19 @@ import type { ElementNode } from '@solidtv/solid'; import { untrack } from 'solid-js'; -import SpeechEngine, { type SeriesResult, type SpeechType } from './speech.js'; +import speakSeries, { + setSpeechEngine, + type SeriesResult, + type SpeechEngine, + type SpeechType, +} from './speech.js'; +import { + createWebOSEngine, + detectSpeechPlatform, + isTizenVoiceGuideEnabled, + isWebOS, + webOSLunaRequest, + type SpeechPlatform, +} from './platformEngines.js'; import { debounce } from '@solid-primitives/scheduled'; import { focusPath } from '../useFocusManager.js'; @@ -119,7 +132,7 @@ function textToSpeech( return; } - return (currentlySpeaking = SpeechEngine(toSpeak, aria, lang, voice)); + return (currentlySpeaking = speakSeries(toSpeak, aria, lang, voice)); } export interface Announcer { @@ -138,6 +151,27 @@ export interface Announcer { focusDebounce?: number; focusChangeTimeout?: number; }) => void; + /** + * Replace the text-to-speech backend — for platforms with their own TTS + * (webOS Luna, Tizen, a native bridge) instead of the Web Speech API. + * Call with no argument to restore the default. Has no effect while + * `Announcer.aria` is true, since that path writes to an aria live region + * rather than speaking. + */ + setSpeechEngine: (engine?: SpeechEngine | null) => void; + /** + * Detects the TV platform and switches the Announcer to that device's own + * speech output, returning what it picked. Safe to call anywhere — it reads + * globals, it doesn't speak. + * + * - LG (`webos`) — installs an engine driving the Luna TTS service, and + * clears `aria`. + * - Samsung (`tizen`) — turns `aria` on. Samsung has no speak API; its Voice + * Guide screen reader is the TTS and it reads the live region instead. + * - Anything else (`default`) — restores the Web Speech API engine and leaves + * `aria` as configured. + */ + detectSpeechEngine: () => SpeechPlatform; onFocusChange?: DebounceWithFlushFunction; refresh: (depth?: number) => void; } @@ -182,6 +216,39 @@ export const Announcer: Announcer = { Announcer.onFocusChange(untrack(() => focusPath())); } }, + setSpeechEngine: setSpeechEngine, + detectSpeechEngine: function () { + const platform = detectSpeechPlatform(); + + if (platform === 'tizen') { + // Samsung's Voice Guide is the only TTS available to an app, and it + // reads the DOM rather than accepting strings. + setSpeechEngine(); + Announcer.aria = true; + + if (isTizenVoiceGuideEnabled() === false) { + console.warn( + 'Announcer: Voice Guide is off in the TV settings, so announcements will be silent.', + ); + } + return platform; + } + + if (platform === 'webos') { + setSpeechEngine(createWebOSEngine(webOSLunaRequest()!)); + Announcer.aria = false; + return platform; + } + + if (isWebOS()) { + console.warn( + 'Announcer: webOS detected but webOS.service.request is unavailable — include webOSTV.js to use the Luna TTS engine. Falling back to speechSynthesis.', + ); + } + + setSpeechEngine(); + return platform; + }, setupTimers: function ({ focusDebounce = 400, focusChangeTimeout = fiveMinutes, diff --git a/src/primitives/announcer/platformEngines.ts b/src/primitives/announcer/platformEngines.ts new file mode 100644 index 0000000..03275ba --- /dev/null +++ b/src/primitives/announcer/platformEngines.ts @@ -0,0 +1,236 @@ +import type { SpeechEngine, SpeechError } from './speech.js'; + +/** + * The speech channel a device uses. + * + * - `webos` — LG. The Luna TTS service (`com.webos.service.tts`) speaks on our + * behalf, so we drive it directly with a {@link SpeechEngine}. + * - `tizen` — Samsung. There is no API for an app to speak a string; the + * built-in Voice Guide screen reader is the TTS, and it only reads the DOM. + * The Announcer's `aria` mode exists for exactly this, so we switch to it. + * - `default` — anything else, including desktop browsers: Web Speech API. + */ +export type SpeechPlatform = 'webos' | 'tizen' | 'default'; + +const TTS_URI = 'luna://com.webos.service.tts'; + +interface LunaResponse { + returnValue?: boolean; + msgStatus?: 'done' | 'stopped' | 'canceled' | 'error'; + msgID?: string; + errorCode?: number; + errorText?: string; +} + +interface LunaRequest { + cancel: VoidFunction; +} + +interface LunaRequestOptions { + method: string; + parameters?: Record; + subscribe?: boolean; + onSuccess?: (response: LunaResponse) => void; + onFailure?: (response: LunaResponse) => void; +} + +export type LunaRequestFn = ( + uri: string, + options: LunaRequestOptions, +) => LunaRequest; + +interface WebOSGlobal { + service?: { request?: LunaRequestFn }; +} + +interface TizenTVInfo { + getMenuValue: (key: unknown) => unknown; + TvInfoMenuKey: { VOICE_GUIDE_KEY: unknown }; +} + +interface DeviceGlobals { + webOS?: WebOSGlobal; + tizen?: object; + webapis?: { tvinfo?: TizenTVInfo }; +} + +function globals(): DeviceGlobals { + return globalThis as DeviceGlobals; +} + +function userAgent(): string { + return typeof navigator === 'undefined' ? '' : navigator.userAgent; +} + +function speechError(code: string, message: string): SpeechError { + const error: SpeechError = new Error(message); + error.error = code; + return error; +} + +/** + * The Luna call the webOS engine is built on, or undefined when it isn't + * reachable. This is a capability test rather than a user-agent match: the app + * must have loaded webOSTV.js for `webOS.service.request` to exist, and without + * it there is no way to reach the TTS service. + */ +export function webOSLunaRequest(): LunaRequestFn | undefined { + const service = globals().webOS?.service; + return typeof service?.request === 'function' + ? service.request.bind(service) + : undefined; +} + +/** + * True on LG TVs. Only tells us which device we are on — reaching the TTS + * service still needs {@link webOSLunaRequest}. + */ +export function isWebOS(): boolean { + // "Web0S" (with a zero) is what LG's TV user agent actually reports. + return /web0s|webos/i.test(userAgent()) || !!globals().webOS; +} + +/** + * True on Samsung TVs. `tizen`/`webapis` are injected by the platform; the + * user-agent check covers the window before `webapis.js` has loaded. + */ +export function isTizen(): boolean { + const { tizen, webapis } = globals(); + return !!tizen || !!webapis || /tizen/i.test(userAgent()); +} + +/** + * Whether Samsung's Voice Guide is switched on in TV settings, or undefined + * when it can't be read. Nothing an app announces through `aria` mode is + * audible while this is off — it is a system setting, not something the app + * can enable. + */ +export function isTizenVoiceGuideEnabled(): boolean | undefined { + try { + const tvinfo = globals().webapis?.tvinfo; + if (!tvinfo) { + return undefined; + } + const value = tvinfo.getMenuValue(tvinfo.TvInfoMenuKey.VOICE_GUIDE_KEY); + // Reported as the string "true"/"false" on current firmware, but older + // models have returned a boolean or 1/0. + return value === 'true' || value === true || value === 1; + } catch { + return undefined; + } +} + +/** + * A {@link SpeechEngine} backed by LG's Luna TTS service — the same engine + * that powers webOS Audio Guidance. + * + * Each phrase subscribes for feedback so the returned promise settles when the + * TV has actually finished speaking it, which keeps a `PAUSE-` in the middle of + * a series accurate. Requires webOSTV.js and the `com.webos.service.tts` + * permission in `appinfo.json`. + */ +export function createWebOSEngine( + request: LunaRequestFn, + options: { appID?: string } = {}, +): SpeechEngine { + const { appID } = options; + let warnedUnavailable = false; + + return { + speak(phrase, { lang }) { + return new Promise((resolve, reject) => { + // Held on an object so `settle` can reach the request handle without + // touching a binding that is still in its temporal dead zone if the + // service answers before `request` returns. + const pending: { handle?: LunaRequest; settled: boolean } = { + settled: false, + }; + + const settle = (finish: VoidFunction) => { + pending.settled = true; + pending.handle?.cancel(); + finish(); + }; + + pending.handle = request(TTS_URI, { + method: 'speak', + parameters: { + text: phrase, + language: lang, + // The Announcer cancels before it starts a new series, so never + // discard what is already queued for this one. + clear: false, + feedback: true, + subscribe: true, + ...(appID ? { appID } : {}), + }, + subscribe: true, + onSuccess: (response) => { + switch (response.msgStatus) { + case 'done': + settle(resolve); + break; + case 'stopped': + case 'canceled': + // Someone cancelled us — benign, and classified so the series + // ends quietly instead of retrying. + settle(() => + reject(speechError('canceled', 'webOS TTS canceled')), + ); + break; + case 'error': + settle(() => + reject( + speechError( + 'synthesis-failed', + `webOS TTS error: ${response.errorText ?? 'unknown'}`, + ), + ), + ); + break; + default: + // The initial acknowledgement carries no msgStatus. Keep the + // subscription open and wait for the real one. + break; + } + }, + onFailure: (response) => { + // The request never reached the service — most often a missing + // com.webos.service.tts permission in appinfo.json, which fails for + // every phrase. Warn once and resolve: rejecting here would turn a + // one-line config mistake into an unhandled rejection per phrase. + if (!warnedUnavailable) { + warnedUnavailable = true; + console.warn( + `Announcer: webOS TTS unavailable (${response.errorText ?? 'unknown error'}). Check that com.webos.service.tts is in the appinfo.json permissions.`, + ); + } + settle(resolve); + }, + }); + + // A response that arrived before `request` returned couldn't cancel the + // subscription, so clean it up here. + if (pending.settled) { + pending.handle.cancel(); + } + }); + }, + cancel() { + request(TTS_URI, { method: 'stop', parameters: {} }); + }, + }; +} + +/** + * Which speech channel this device wants, without changing anything. + */ +export function detectSpeechPlatform(): SpeechPlatform { + if (isTizen()) { + return 'tizen'; + } + if (webOSLunaRequest()) { + return 'webos'; + } + return 'default'; +} diff --git a/src/primitives/announcer/speech.ts b/src/primitives/announcer/speech.ts index b4ff14d..43255fe 100644 --- a/src/primitives/announcer/speech.ts +++ b/src/primitives/announcer/speech.ts @@ -12,6 +12,31 @@ export interface SeriesResult { cancel: () => void; } +export interface SpeechOptions { + lang: string; + voice?: string; +} + +/** + * Pluggable text-to-speech backend. Install one with + * `Announcer.setSpeechEngine()` to route speech through a platform API (webOS + * Luna, Tizen, a native bridge, ...) instead of the Web Speech API. The + * Announcer keeps owning the series: flattening, PAUSE- handling, append, + * cancel and nesting all still work — the engine only speaks one phrase. + */ +export interface SpeechEngine { + /** + * Speak a single phrase. Resolve when it has finished so the Announcer knows + * when to move on; resolve immediately if the platform can't report + * completion. Reject with an error carrying `error: 'network'` to be retried + * (3 attempts, backing off), or `error: 'canceled' | 'interrupted'` to end + * the series quietly. Any other rejection propagates to the caller. + */ + speak: (phrase: string, options: SpeechOptions) => void | Promise; + /** Stop whatever is currently being spoken. */ + cancel: VoidFunction; +} + // Aria label type AriaLabel = { text: string; lang: string }; const ARIA_PARENT_ID = 'aria-parent'; @@ -22,7 +47,7 @@ let ariaLabelPhrases: AriaLabel[] = []; // raw SpeechSynthesisErrorEvent so callers can classify the failure without // depending on the SpeechSynthesisErrorEvent global, which isn't defined on // every TV browser. -interface SpeechError extends Error { +export interface SpeechError extends Error { error?: string; } @@ -129,49 +154,67 @@ function createAriaElement(): HTMLDivElement | HTMLElement { } /** - * Speak a string + * The default engine — the browser's Web Speech API. * - * @param phrase Phrase to speak - * @param utterances An array which the new SpeechSynthesisUtterance instance representing this utterance will be appended - * @param lang Language to speak in * @return {Promise} Promise resolved when the utterance has finished speaking, and rejected if there's an error */ -function speak( - phrase: string, - utterances: SpeechSynthesisUtterance[], - lang = 'en-US', - voiceName?: string, -) { - const synth = window.speechSynthesis; - - return new Promise((resolve, reject) => { - let selectedVoice; - if (voiceName) { - const availableVoices = synth.getVoices(); - selectedVoice = - availableVoices.find((v) => v.name === voiceName) || availableVoices[0]; - } +const webSpeechEngine: SpeechEngine = { + speak(phrase, { lang, voice: voiceName }) { + const synth = window.speechSynthesis; + + return new Promise((resolve, reject) => { + let selectedVoice; + if (voiceName) { + const availableVoices = synth.getVoices(); + selectedVoice = + availableVoices.find((v) => v.name === voiceName) || + availableVoices[0]; + } - const utterance = new SpeechSynthesisUtterance(phrase); - utterance.lang = lang; - if (selectedVoice) { - utterance.voice = selectedVoice; - } - utterance.onend = () => { - resolve(); - }; - utterance.onerror = (e) => { - const error: SpeechError = new Error( - `Speech synthesis error: ${e.error}`, - ); - // Preserve the code so speakSeries can tell benign interruptions - // ("interrupted"/"canceled") apart from real failures ("network", etc.). - error.error = e.error; - reject(error); - }; - utterances.push(utterance); - synth.speak(utterance); - }); + const utterance = new SpeechSynthesisUtterance(phrase); + utterance.lang = lang; + if (selectedVoice) { + utterance.voice = selectedVoice; + } + utterance.onend = () => { + resolve(); + }; + utterance.onerror = (e) => { + const error: SpeechError = new Error( + `Speech synthesis error: ${e.error}`, + ); + // Preserve the code so speakSeries can tell benign interruptions + // ("interrupted"/"canceled") apart from real failures ("network", etc.). + error.error = e.error; + reject(error); + }; + synth.speak(utterance); + }); + }, + cancel() { + window.speechSynthesis?.cancel(); + }, +}; + +let speechEngine: SpeechEngine = webSpeechEngine; + +/** + * Install a custom speech engine, or pass nothing to restore the Web Speech + * API default. Prefer `Announcer.setSpeechEngine()`. + */ +export function setSpeechEngine(engine?: SpeechEngine | null): void { + speechEngine = engine ?? webSpeechEngine; +} + +/** + * Devices that ship a platform TTS instead of the Web Speech API may not + * define SpeechSynthesisUtterance at all, where a bare `instanceof` throws. + */ +function isUtterance(phrase: unknown): phrase is SpeechSynthesisUtterance { + return ( + typeof SpeechSynthesisUtterance !== 'undefined' && + phrase instanceof SpeechSynthesisUtterance + ); } /** @@ -212,12 +255,10 @@ function speakSeries( voice?: string, root = true, ): SeriesResult { - const synth = window.speechSynthesis; const remainingPhrases = flattenStrings( Array.isArray(series) ? series : [series], ); const nestedSeriesResults: SeriesResult[] = []; - const utterances: SpeechSynthesisUtterance[] = []; let active: boolean = true; const seriesChain = (async () => { @@ -245,7 +286,7 @@ function speakSeries( while (active && retriesLeft > 0) { try { if (aria) addChildrenToAriaDiv({ text: phrase, lang }); - else await speak(phrase, utterances, lang, voice); + else await speechEngine.speak(phrase, { lang, voice }); retriesLeft = 0; // Exit retry loop on success } catch (e) { retriesLeft = await handleSpeechError( @@ -255,7 +296,7 @@ function speakSeries( ); } } - } else if (phrase instanceof SpeechSynthesisUtterance) { + } else if (isUtterance(phrase)) { // Handle SpeechSynthesisUtterance objects with retry logic const totalRetries = 3; let retriesLeft = totalRetries; @@ -268,7 +309,10 @@ function speakSeries( if (text) { if (aria) addChildrenToAriaDiv({ text, lang: objectLang }); else - await speak(text, utterances, objectLang, objectVoice?.name); + await speechEngine.speak(text, { + lang: objectLang, + voice: objectVoice?.name, + }); retriesLeft = 0; // Exit retry loop on success } } catch (e) { @@ -321,7 +365,7 @@ function speakSeries( // phrases from this canceled series. ariaLabelPhrases = []; } else { - synth.cancel(); // Cancel all ongoing speech + speechEngine.cancel(); // Cancel all ongoing speech } } nestedSeriesResults.forEach((nestedSeriesResult) => { diff --git a/src/primitives/index.ts b/src/primitives/index.ts index 7bc9f5b..e44183d 100644 --- a/src/primitives/index.ts +++ b/src/primitives/index.ts @@ -33,4 +33,15 @@ export { createBlurredImage } from './utils/createBlurredImage.js'; export type * from './types.js'; export type { KeyHandler } from '../core/focusManager.js'; -export type { SpeechType } from './announcer/speech.js'; +export type { + SpeechType, + SpeechEngine, + SpeechOptions, + SpeechError, +} from './announcer/speech.js'; +export { + createWebOSEngine, + detectSpeechPlatform, + isTizenVoiceGuideEnabled, + type SpeechPlatform, +} from './announcer/platformEngines.js'; diff --git a/tests/announcer-engine.spec.ts b/tests/announcer-engine.spec.ts new file mode 100644 index 0000000..e46e61e --- /dev/null +++ b/tests/announcer-engine.spec.ts @@ -0,0 +1,103 @@ +import { describe, it, expect, afterEach } from 'vitest'; +import speak, { + setSpeechEngine, + type SpeechOptions, +} from '../src/primitives/announcer/speech.ts'; + +type Spoken = { phrase: string; options: SpeechOptions }; + +function recordingEngine() { + const spoken: Spoken[] = []; + let canceled = 0; + return { + spoken, + get canceled() { + return canceled; + }, + engine: { + speak: (phrase: string, options: SpeechOptions) => { + spoken.push({ phrase, options }); + }, + cancel: () => { + canceled++; + }, + }, + }; +} + +describe('Announcer custom speech engine', () => { + afterEach(() => { + setSpeechEngine(); + }); + + it('routes phrases through the installed engine', async () => { + const recorder = recordingEngine(); + setSpeechEngine(recorder.engine); + + await speak(['Hello', 'button'], false, 'pt-BR').series; + + expect(recorder.spoken.length).toBe(1); + expect(recorder.spoken[0]!.phrase).toContain('Hello'); + expect(recorder.spoken[0]!.phrase).toContain('button'); + expect(recorder.spoken[0]!.options.lang).toBe('pt-BR'); + }); + + it('still honors PAUSE- entries and series ordering', async () => { + const recorder = recordingEngine(); + setSpeechEngine(recorder.engine); + + await speak(['First', 'PAUSE-0', 'Second'], false).series; + + expect(recorder.spoken.map((s) => s.phrase)).toEqual(['First', 'Second']); + }); + + it('passes the configured voice through', async () => { + const recorder = recordingEngine(); + setSpeechEngine(recorder.engine); + + await speak('Solo', false, 'en-US', 'Custom Voice').series; + + expect(recorder.spoken[0]!.options.voice).toBe('Custom Voice'); + }); + + it('waits for an async engine before speaking the next phrase', async () => { + const order: string[] = []; + setSpeechEngine({ + speak: (phrase: string) => { + order.push(`start:${phrase}`); + return new Promise((resolve) => + setTimeout(() => { + order.push(`end:${phrase}`); + resolve(); + }, 10), + ); + }, + cancel: () => {}, + }); + + await speak(['One', ['Two']], false).series; + + expect(order).toEqual(['start:One', 'end:One', 'start:Two', 'end:Two']); + }); + + it('cancels through the engine', async () => { + const recorder = recordingEngine(); + setSpeechEngine(recorder.engine); + + const series = speak(['Interrupt me'], false); + series.cancel(); + await series.series; + + expect(recorder.canceled).toBe(1); + }); + + it('restores the default engine when unset', async () => { + const recorder = recordingEngine(); + setSpeechEngine(recorder.engine); + setSpeechEngine(); + + await speak(['Back to default'], true).series; + + expect(recorder.spoken.length).toBe(0); + }); +}); diff --git a/tests/announcer-platform.spec.ts b/tests/announcer-platform.spec.ts new file mode 100644 index 0000000..a9267f3 --- /dev/null +++ b/tests/announcer-platform.spec.ts @@ -0,0 +1,221 @@ +import { describe, it, expect, afterEach, vi } from 'vitest'; +import { Announcer } from '../src/primitives/announcer/announcer.ts'; +import speak, { setSpeechEngine } from '../src/primitives/announcer/speech.ts'; +import { + createWebOSEngine, + detectSpeechPlatform, + isTizenVoiceGuideEnabled, +} from '../src/primitives/announcer/platformEngines.ts'; + +type LunaHandler = (response: Record) => void; +type LunaCall = { + uri: string; + method: string; + parameters?: Record; + onSuccess?: LunaHandler; + onFailure?: LunaHandler; +}; + +/** Stand-in for webOSTV.js — records calls and lets a test drive the callbacks. */ +function lunaStub() { + const calls: LunaCall[] = []; + const canceled: number[] = []; + const request = vi.fn((uri: string, options: Omit) => { + const index = calls.push({ uri, ...options }) - 1; + return { cancel: () => canceled.push(index) }; + }); + return { calls, canceled, request }; +} + +function withGlobals(values: Record) { + const g = globalThis as Record; + for (const [key, value] of Object.entries(values)) { + g[key] = value; + } +} + +function clearGlobals(...keys: string[]) { + const g = globalThis as Record; + for (const key of keys) { + delete g[key]; + } +} + +describe('webOS Luna speech engine', () => { + afterEach(() => { + setSpeechEngine(); + }); + + it('speaks through com.webos.service.tts with a subscription', async () => { + const luna = lunaStub(); + setSpeechEngine(createWebOSEngine(luna.request)); + + const series = speak(['Hello there'], false, 'en-GB'); + await Promise.resolve(); + + expect(luna.calls.length).toBe(1); + const call = luna.calls[0]!; + expect(call.uri).toBe('luna://com.webos.service.tts'); + expect(call.method).toBe('speak'); + expect(call.parameters).toMatchObject({ + text: 'Hello there', + language: 'en-GB', + clear: false, + feedback: true, + subscribe: true, + }); + + // The initial ack carries no msgStatus and must not end the phrase. + call.onSuccess!({ returnValue: true, msgID: 'abc123456789' }); + call.onSuccess!({ msgStatus: 'done', msgID: 'abc123456789' }); + + await series.series; + expect(luna.canceled).toContain(0); + }); + + it('passes an appID when configured', async () => { + const luna = lunaStub(); + setSpeechEngine( + createWebOSEngine(luna.request, { appID: 'com.example.app' }), + ); + + const series = speak('Hi', false); + await Promise.resolve(); + luna.calls[0]!.onSuccess!({ msgStatus: 'done' }); + await series.series; + + expect(luna.calls[0]!.parameters!.appID).toBe('com.example.app'); + }); + + it('ends the series quietly when the TV reports it stopped', async () => { + const luna = lunaStub(); + setSpeechEngine(createWebOSEngine(luna.request)); + + const series = speak('One', false); + await Promise.resolve(); + luna.calls[0]!.onSuccess!({ msgStatus: 'stopped' }); + + // 'stopped'/'canceled' are classified as benign, so the series resolves + // instead of rejecting, and the phrase is not retried. + await expect(series.series).resolves.toBeUndefined(); + expect(luna.calls.filter((c) => c.method === 'speak').length).toBe(1); + }); + + it('warns once and stays quiet when the TTS service is unreachable', async () => { + const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}); + const luna = lunaStub(); + setSpeechEngine(createWebOSEngine(luna.request)); + + for (const phrase of ['First', 'Second']) { + const series = speak(phrase, false); + await Promise.resolve(); + const call = luna.calls.find((c) => c.parameters?.text === phrase)!; + call.onFailure!({ returnValue: false, errorText: 'Denied' }); + await expect(series.series).resolves.toBeUndefined(); + } + + // Both phrases failed, but a missing permission should be reported once. + expect(warn).toHaveBeenCalledTimes(1); + expect(warn.mock.calls[0]![0]).toContain('com.webos.service.tts'); + warn.mockRestore(); + }); + + it('calls stop on cancel', () => { + const luna = lunaStub(); + setSpeechEngine(createWebOSEngine(luna.request)); + + const series = speak(['Interrupt me'], false); + series.cancel(); + + expect(luna.calls.some((c) => c.method === 'stop')).toBe(true); + }); +}); + +describe('detectSpeechEngine', () => { + afterEach(() => { + clearGlobals('webOS', 'tizen', 'webapis'); + setSpeechEngine(); + Announcer.aria = false; + vi.restoreAllMocks(); + }); + + it('installs the Luna engine on LG', async () => { + const luna = lunaStub(); + withGlobals({ webOS: { service: { request: luna.request } } }); + + expect(Announcer.detectSpeechEngine()).toBe('webos'); + expect(Announcer.aria).toBe(false); + + const series = speak('Hello', false); + await Promise.resolve(); + expect(luna.calls[0]!.method).toBe('speak'); + series.cancel(); + }); + + it('switches to aria mode on Samsung', () => { + withGlobals({ tizen: {} }); + + expect(Announcer.detectSpeechEngine()).toBe('tizen'); + expect(Announcer.aria).toBe(true); + }); + + it('warns when Samsung Voice Guide is off', () => { + const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}); + withGlobals({ + tizen: {}, + webapis: { + tvinfo: { + TvInfoMenuKey: { VOICE_GUIDE_KEY: 'voiceGuide' }, + getMenuValue: () => 'false', + }, + }, + }); + + Announcer.detectSpeechEngine(); + + expect(isTizenVoiceGuideEnabled()).toBe(false); + expect(warn).toHaveBeenCalledWith(expect.stringContaining('Voice Guide')); + }); + + it('reads Voice Guide as enabled', () => { + withGlobals({ + webapis: { + tvinfo: { + TvInfoMenuKey: { VOICE_GUIDE_KEY: 'voiceGuide' }, + getMenuValue: () => 'true', + }, + }, + }); + + expect(isTizenVoiceGuideEnabled()).toBe(true); + }); + + it('reports undefined Voice Guide state when webapis throws', () => { + withGlobals({ + tizen: {}, + webapis: { + tvinfo: { + TvInfoMenuKey: { VOICE_GUIDE_KEY: 'voiceGuide' }, + getMenuValue: () => { + throw new Error('not supported'); + }, + }, + }, + }); + + expect(isTizenVoiceGuideEnabled()).toBeUndefined(); + }); + + it('falls back to the default engine elsewhere', () => { + expect(detectSpeechPlatform()).toBe('default'); + expect(Announcer.detectSpeechEngine()).toBe('default'); + }); + + it('does not install the Luna engine without webOSTV.js', () => { + const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}); + withGlobals({ webOS: {} }); + + expect(Announcer.detectSpeechEngine()).toBe('default'); + expect(warn).toHaveBeenCalledWith(expect.stringContaining('webOSTV.js')); + }); +});