diff --git a/.fallowrc.json b/.fallowrc.json index 43f4763993..f3446717d0 100644 --- a/.fallowrc.json +++ b/.fallowrc.json @@ -13,6 +13,8 @@ "packages/provider-revyl/src/plugin.ts", "packages/provider-webos/test/package-smoke.mjs", "packages/provider-webos/tsdown.config.ts", + "packages/provider-tizen/test/package-smoke.mjs", + "packages/provider-tizen/tsdown.config.ts", "vitest.mutation.config.ts", "src/sdk/index.ts", "src/sdk/io.ts", @@ -100,8 +102,9 @@ // through this workspace link to core source so typecheck needs no build; Fallow then credits // the source files rather than the package. "agent-device", - // Nothing in the root imports the webOS plugin or the tv-cdp package it bundles, but layering + // Nothing in the root imports the TV plugins or the tv-cdp package they bundle, but layering // requires the root manifest to declare every workspace package. + "@agent-device/tizen", "@agent-device/tv-cdp", "@agent-device/webos", // Oxlint resolves the shared config's JS plugins dynamically from their package names. diff --git a/examples/sdk/tsconfig.json b/examples/sdk/tsconfig.json index 0a39ef5df1..8f65072510 100644 --- a/examples/sdk/tsconfig.json +++ b/examples/sdk/tsconfig.json @@ -62,6 +62,7 @@ { "path": "../../packages/provider-revyl" }, { "path": "../../packages/provider-testmu" }, { "path": "../../packages/provider-webos" }, + { "path": "../../packages/provider-tizen" }, { "path": "../../packages/provider-webdriver" }, { "path": "../../packages/provision-kit" }, { "path": "../../packages/proxy" }, diff --git a/package.json b/package.json index 7035902303..d696defaf0 100644 --- a/package.json +++ b/package.json @@ -377,6 +377,7 @@ "yauzl": "^3.4.0", "@agent-device/testmu": "workspace:*", "@agent-device/revyl": "workspace:*", - "@agent-device/webos": "workspace:*" + "@agent-device/webos": "workspace:*", + "@agent-device/tizen": "workspace:*" } } diff --git a/packages/provider-tizen/README.md b/packages/provider-tizen/README.md new file mode 100644 index 0000000000..76feed07bb --- /dev/null +++ b/packages/provider-tizen/README.md @@ -0,0 +1,122 @@ +# @agent-device/tizen + +Drive web apps on Samsung Tizen TVs with [agent-device](https://agent-device.dev). The plugin +debug-launches the app over `sdb`, forwards its DevTools port, and drives the page over the Chrome +DevTools Protocol: remote keys, pointer, text, snapshots, screenshots and console logs. + +Status: experimental. Verified on a Samsung Smart Monitor M7 (Tizen 9.0, packaged build of a React +Native Web app): `connect`, `open`, `open --relaunch`, `snapshot -i`, `tv-remote` (directions, +select, back), `back`, `is focused`, `press`, `find ... click`, `hover`, `longpress`, `fill`, +`type`, `wait`, `scroll`, `screenshot`, `logs`, `apps`, `appstate`, `close`, `disconnect`. + +## How it plugs in + +agent-device leases only iOS, Android and HarmonyOS devices, so a Tizen TV enters as an +**Android TV device that this plugin owns** (`platform: android`, `target: tv`, id +`tizen:`). Every command on that device goes to this plugin's runtime, never to adb. + +## Requirements + +- `sdb` from the Tizen SDK of the VS Code Tizen extension, set up with its Set Up step. The plugin + finds it in the SDK that `$TIZEN_SDK` or `~/.tizen.path.config` names, or at the `sdbPath` + option; it is not on `PATH` +- The TV in Developer Mode with this host's IP set as the developer IP; see Samsung's + [Connecting the TV and SDK](https://developer.samsung.com/smarttv/develop/getting-started/using-sdk/tv-device.html) +- The app installed on the TV from a debug-enabled package +- The TV and this host on the same network + +## Install + +```sh +agent-device plugins add @agent-device/tizen +agent-device daemon stop # the next command starts a daemon with the plugin loaded +``` + +Plugin options: `sdbPath` (the `sdb` binary, when the SDK is not found) and `device` (default TV +IP or sdb serial). + +To try a local build, pack the package and install the tarball into a plugin store installation: + +```sh +pnpm --filter @agent-device/tizen pack --out /tmp/agent-device-tizen.tgz +installation=$(uuidgen) +npm install --prefix ~/.agent-device/plugins/$installation /tmp/agent-device-tizen.tgz +``` + +Then register `{"plugins": {"@agent-device/tizen": {"installation": ""}}}` in +`~/.agent-device/config.json` and run `agent-device daemon stop`. + +## Use + +```sh +agent-device daemon stop # the lease TTL is read when the daemon starts +export AGENT_DEVICE_LEASE_TTL_MS=1800000 AGENT_DEVICE_LEASE_MAX_TTL_MS=1800000 # see Known limitations +export AGENT_DEVICE_SESSION=checkout-tv # one session name per agent +agent-device connect tizen --device 192.0.2.10 # TV IP or sdb serial +agent-device open # kill, debug launch, forward, attach over CDP +agent-device wait text +agent-device tv-remote press down +agent-device snapshot -i +agent-device screenshot ./tv.png +agent-device logs start # then logs path +agent-device close +agent-device disconnect +``` + +`--device` takes the TV IP (`sdb connect :26101` runs when it is not connected yet) or an sdb +serial. Without it, the plugin uses the `device` option, then the only TV connected to `sdb`. +`connect` does not contact the TV; `open` connects it. + +The TV is a leased device, so every command from `connect` to `disconnect` names its session with +`AGENT_DEVICE_SESSION`, `--session ` or a config session. Without one, the command fails with +`REMOTE_SESSION_REQUIRED`. + +## Supported + +| Command | How | +| --- | --- | +| `open` / `close` | `sdb shell 0 was_kill`, `sdb shell 0 debug` (prints the DevTools port), `sdb forward`, then CDP / `sdb shell 0 was_kill` | +| `tv-remote press up/down/left/right/select/back`, `back` | `Input.dispatchKeyEvent` with the Samsung remote codes (Back = `XF86Back`, 10009) | +| `appstate` | the attached app while its page is visible | +| `apps` | `sdb shell 0 vd_applist`: user-installed apps with their titles | +| `logs` | console calls, exceptions and browser log entries of the attached page, one line each | +| `snapshot`, `find`, `is`, `get`, `wait` | DOM walk over `Runtime.evaluate`: visible interactive, labelled or text nodes, focus from `document.activeElement`; `-s`, `-d` and `-i` apply to the walk | +| `screenshot` | `Page.captureScreenshot` | +| `click`, `press `, `longpress`, `hover` | `Input.dispatchMouseEvent` | +| `fill` | pointer press on the field, select its contents, then `Input.insertText`, or Backspace for an empty value | +| `type` | `Input.insertText` into the focused element | +| `scroll` | `mouseWheel` at the centre of the scroll container or viewport, by the requested distance | + +Not yet: system buttons (`home`, menu, volume, channel and media keys), `network dump`, `perf`, +`install` and `uninstall`, recording, URLs and deep links in `open`, `power`. + +## Known limitations + +- **`open` is always a cold start.** `sdb shell 0 debug` launches the app and cannot attach to a + running one, so `open` kills the app first. `open` returns once the document has loaded, which + can be before the app leaves its splash screen; wait for app content (`wait text ...`) before + the first snapshot. +- **Idle leases expire after 60 s by default, and expiry ends the session.** agent-device gives a + direct-device provider lease its default TTL and beats it only while a command runs. When the + lease ends, by `disconnect` or by expiry, the plugin closes the page connection and removes the + `sdb` forward. After a longer pause, the next command fails with `Lease is not active` or + `No active session` and needs a new `connect` and `open`. To avoid it, run + `agent-device daemon stop`, then start the daemon with `AGENT_DEVICE_LEASE_TTL_MS` and + `AGENT_DEVICE_LEASE_MAX_TTL_MS` set (for example 30 min). +- **`screenshot` captures the app page, not the TV screen.** The restricted `sdb shell 0` has no + screen-capture command. When the app is in the background, the page is hidden and renders no + frames, so `screenshot` fails at once; `open ` brings the app back. While the on-screen + keyboard is open, the page viewport and so the screenshot are shorter. +- **`appstate` knows only the attached app.** The TV's REST API (`:8001/api/v2/applications/`) + reports `visible: false` for an app in front of the user, and `sdb shell 0` has no + foreground-app command. +- **Console messages from before the attach are replayed without object previews**, so objects + in them show as `Object`. Messages logged after the attach show their properties. +- **A second DevTools client on the same page can hang the session.** Close Chrome DevTools and + other inspector clients attached to the app before a session. +- **Snapshots are DOM-derived.** Labels come from `aria-label`, `alt`, `title` or own text; + rects are CSS px, which match screenshot pixels only at `devicePixelRatio` 1 (1080p apps). + The walk does not enter iframes or shadow roots. `is text` on an input compares its label, not + its value; the value is in `snapshot --raw`. +- **System overlays are outside the page.** The Samsung menu, on-screen keyboard and other system + layers never appear in `screenshot` or `snapshot`. diff --git a/packages/provider-tizen/package.json b/packages/provider-tizen/package.json new file mode 100644 index 0000000000..4534c8d9ae --- /dev/null +++ b/packages/provider-tizen/package.json @@ -0,0 +1,49 @@ +{ + "name": "@agent-device/tizen", + "version": "0.21.36-dev", + "type": "module", + "description": "Samsung Tizen TV web app plugin for agent-device.", + "files": [ + "dist" + ], + "scripts": { + "build": "tsdown --config tsdown.config.ts", + "prepack": "pnpm build" + }, + "exports": { + ".": { + "types": "./src/index.ts", + "default": "./src/index.ts" + }, + "./agent-device": { + "types": "./src/index.ts", + "default": "./src/index.ts" + }, + "./package.json": "./package.json" + }, + "devDependencies": { + "@agent-device/host-kit": "workspace:*", + "@agent-device/kernel": "workspace:*", + "@agent-device/test-kit": "workspace:*", + "@agent-device/tv-cdp": "workspace:*", + "agent-device": "workspace:*" + }, + "publishConfig": { + "exports": { + ".": { + "import": "./dist/agent-device.mjs" + }, + "./agent-device": { + "import": "./dist/agent-device.mjs" + } + }, + "access": "public" + }, + "license": "MIT", + "repository": { + "type": "git", + "url": "git+https://github.com/callstack/agent-device.git", + "directory": "packages/provider-tizen" + }, + "homepage": "https://agent-device.dev/docs/tizen" +} diff --git a/packages/provider-tizen/src/index.test.ts b/packages/provider-tizen/src/index.test.ts new file mode 100644 index 0000000000..2683a9acb7 --- /dev/null +++ b/packages/provider-tizen/src/index.test.ts @@ -0,0 +1,28 @@ +import assert from 'node:assert/strict'; +import { test } from 'vitest'; + +import tizen from './index.ts'; + +test('the plugin connection leases an Android TV device and passes on the requested TV', () => { + const { connection } = tizen(); + + assert.deepEqual(connection.resolve({ flags: { device: '192.0.2.10' } }).profile, { + leaseProvider: 'tizen', + platform: 'android', + target: 'tv', + device: '192.0.2.10', + }); + assert.deepEqual(connection.resolve({ flags: {} }).profile, { + leaseProvider: 'tizen', + platform: 'android', + target: 'tv', + }); +}); + +test('verify defers the TV and app to session open, naming the plugin default TV', async () => { + const verified = await tizen({ device: '192.0.2.10' }).connection.verify({ flags: {} }); + + assert.equal(verified.service, 'Samsung Tizen'); + assert.deepEqual(verified.device, { status: 'deferred', name: '192.0.2.10' }); + assert.deepEqual(verified.app, { status: 'deferred' }); +}); diff --git a/packages/provider-tizen/src/index.ts b/packages/provider-tizen/src/index.ts new file mode 100644 index 0000000000..d286ac19ae --- /dev/null +++ b/packages/provider-tizen/src/index.ts @@ -0,0 +1,21 @@ +import type { AgentDevicePlugin } from 'agent-device/plugins'; +import { tvPluginConnection } from '@agent-device/tv-cdp/connection'; + +import { PROVIDER, type TizenOptions } from './provider.ts'; + +function tizen(options: TizenOptions = {}) { + return { + apiVersion: 2, + provider: PROVIDER, + credentials: { variables: [] }, + connection: tvPluginConnection({ + provider: PROVIDER, + service: 'Samsung Tizen', + verificationMessage: 'Tizen TVs are reached over sdb when a session opens.', + defaultDevice: options.device, + }), + setup: async (host) => (await import('./runtime.ts')).createTizenRuntime(options, host), + } satisfies AgentDevicePlugin; +} + +export { tizen as default }; diff --git a/packages/provider-tizen/src/keys.ts b/packages/provider-tizen/src/keys.ts new file mode 100644 index 0000000000..aa524b3f05 --- /dev/null +++ b/packages/provider-tizen/src/keys.ts @@ -0,0 +1,6 @@ +import { NAVIGATION_KEYS, type RemoteKeys } from '@agent-device/tv-cdp'; + +export const TIZEN_REMOTE_KEYS: RemoteKeys = { + ...NAVIGATION_KEYS, + back: { key: 'XF86Back', code: 'XF86Back', keyCode: 10009 }, +}; diff --git a/packages/provider-tizen/src/provider.ts b/packages/provider-tizen/src/provider.ts new file mode 100644 index 0000000000..a3438685b0 --- /dev/null +++ b/packages/provider-tizen/src/provider.ts @@ -0,0 +1,8 @@ +export const PROVIDER = 'tizen'; + +export type TizenOptions = Readonly<{ + /** Path to `sdb` when it is not found in the Tizen SDK named by $TIZEN_SDK or ~/.tizen.path.config. */ + sdbPath?: string; + /** TV to drive when the lease names none: an sdb serial or an IP; defaults to the only connected TV. */ + device?: string; +}>; diff --git a/packages/provider-tizen/src/runtime.test.ts b/packages/provider-tizen/src/runtime.test.ts new file mode 100644 index 0000000000..6008195822 --- /dev/null +++ b/packages/provider-tizen/src/runtime.test.ts @@ -0,0 +1,149 @@ +import assert from 'node:assert/strict'; +import { chmodSync, readFileSync, writeFileSync } from 'node:fs'; +import path from 'node:path'; +import { AppError, isRequestCanceledError } from '@agent-device/kernel/errors'; +import { mkdtempForTestSync } from '@agent-device/test-kit'; +import { afterEach, test, vi } from 'vitest'; + +import { createTizenRuntime } from './runtime.ts'; + +const APP_ID = 'AbCdE12345.example'; + +// An sdb whose `devices` lists the serials in SDB_STATE and whose `connect` adds one unless +// SDB_UNREACHABLE is set. Every call is recorded. One script serves every test, because macOS checks each new executable on its first run. +const SDB = path.join(mkdtempForTestSync('sdb-'), 'sdb'); +writeFileSync( + SDB, + `#!/bin/sh +echo "$*" >> "$SDB_CALLS" +case "$1" in + devices) echo 'List of devices attached'; while read -r serial; do printf '%s\\tdevice\\tQE55Q80\\n' "$serial"; done < "$SDB_STATE" ;; + connect) + if [ -n "$SDB_UNREACHABLE" ]; then echo "failed to connect to $2"; exit 0; fi + echo "$2" >> "$SDB_STATE"; echo "connected to $2" ;; +esac +`, +); +chmodSync(SDB, 0o755); + +const shutdowns: (() => Promise)[] = []; +afterEach(async () => { + for (const shutdown of shutdowns.splice(0)) await shutdown(); +}); + +async function tizenRuntime(connected: readonly string[], env: Record = {}) { + const dir = mkdtempForTestSync('sdb-state-'); + const state = path.join(dir, 'connected'); + const log = path.join(dir, 'calls'); + writeFileSync(state, connected.map((serial) => `${serial}\n`).join('')); + writeFileSync(log, ''); + const { runtime } = await createTizenRuntime( + { sdbPath: SDB }, + { + env: { ...process.env, SDB_STATE: state, SDB_CALLS: log, ...env }, + createError: (code, message, details) => new AppError(code, message, details), + }, + ); + shutdowns.push(runtime.shutdown); + return { runtime, calls: () => readFileSync(log, 'utf8').trim().split('\n') }; +} + +const lease = (leaseId: string) => ({ + leaseId, + tenantId: 'tenant', + runId: 'run', + backend: 'android-instance' as const, + leaseProvider: 'tizen', + createdAt: 0, + heartbeatAt: 0, + expiresAt: Number.MAX_SAFE_INTEGER, +}); + +test('a TV named by IP is connected over sdb, keeps that name and is identified by its serial', async () => { + const { runtime, calls } = await tizenRuntime([]); + + const allocated = await runtime.leaseLifecycle.allocate(lease('one'), { + flags: { device: '192.0.2.10' }, + }); + + assert.deepEqual(calls(), ['devices', 'connect 192.0.2.10:26101', 'devices']); + assert.equal(allocated?.device.name, '192.0.2.10'); + assert.equal(allocated?.device.id, 'tizen:192.0.2.10:26101'); + assert.equal(allocated?.device.platform, 'android'); + assert.equal(allocated?.device.target, 'tv'); +}); + +test('a second lease naming the held TV by IP:port instead of IP is refused with DEVICE_IN_USE', async () => { + const { runtime } = await tizenRuntime([]); + await runtime.leaseLifecycle.allocate(lease('one'), { flags: { device: '192.0.2.10' } }); + + const second = runtime.leaseLifecycle.allocate(lease('two'), { + flags: { device: '192.0.2.10:26101' }, + }); + + await assert.rejects(second, (error: AppError) => { + assert.equal(error.code, 'DEVICE_IN_USE'); + return true; + }); +}); + +test('without --device, the only connected TV is used and no connect runs', async () => { + const { runtime, calls } = await tizenRuntime(['192.0.2.10:26101']); + + const allocated = await runtime.leaseLifecycle.allocate(lease('one')); + + assert.equal(allocated?.device.id, 'tizen:192.0.2.10:26101'); + assert.deepEqual(calls(), ['devices']); +}); + +test('without --device and with two TVs connected, allocation asks for one', async () => { + const { runtime } = await tizenRuntime(['192.0.2.10:26101', '192.0.2.11:26101']); + + const allocation = runtime.leaseLifecycle.allocate(lease('one')); + + await assert.rejects(allocation, (error: AppError) => { + assert.equal(error.code, 'DEVICE_NOT_FOUND'); + assert.deepEqual(error.details?.available, ['192.0.2.10:26101', '192.0.2.11:26101']); + return true; + }); +}); + +test('a TV that sdb cannot connect to fails as DEVICE_NOT_FOUND', async () => { + const { runtime } = await tizenRuntime([], { SDB_UNREACHABLE: '1' }); + + const allocation = runtime.leaseLifecycle.allocate(lease('one'), { + flags: { device: '192.0.2.10' }, + }); + + await assert.rejects(allocation, (error: AppError) => { + assert.equal(error.code, 'DEVICE_NOT_FOUND'); + assert.equal(error.message, 'sdb could not connect to the Tizen TV 192.0.2.10:26101'); + return true; + }); +}); + +test('Ctrl-C while open waits for the killed app stops before the debug launch', async () => { + const { runtime, calls } = await tizenRuntime(['192.0.2.10:26101']); + const allocated = await runtime.leaseLifecycle.allocate(lease('one')); + assert.ok(allocated); + const cancel = new AbortController(); + + const opening = runtime.getInteractor(allocated.device, cancel.signal)!.open(APP_ID); + await vi.waitFor(() => + assert.equal(calls().at(-1), `-s 192.0.2.10:26101 shell 0 was_kill ${APP_ID}`), + ); + cancel.abort(); + + await assert.rejects(opening, isRequestCanceledError); + assert.equal(calls().at(-1), `-s 192.0.2.10:26101 shell 0 was_kill ${APP_ID}`); +}); + +test('close kills the app on the leased TV', async () => { + const { runtime, calls } = await tizenRuntime(['192.0.2.10:26101']); + const allocated = await runtime.leaseLifecycle.allocate(lease('one')); + assert.ok(allocated); + + await runtime.getInteractor(allocated.device)!.close!(APP_ID); + + assert.equal(calls().at(-1), `-s 192.0.2.10:26101 shell 0 was_kill ${APP_ID}`); +}); diff --git a/packages/provider-tizen/src/runtime.ts b/packages/provider-tizen/src/runtime.ts new file mode 100644 index 0000000000..24fe2e5916 --- /dev/null +++ b/packages/provider-tizen/src/runtime.ts @@ -0,0 +1,57 @@ +import { createTvRuntime, type TvConnection, type TvRuntimeHost } from '@agent-device/tv-cdp'; + +import { TIZEN_REMOTE_KEYS } from './keys.ts'; +import { PROVIDER, type TizenOptions } from './provider.ts'; +import { createSdb, sdbSerial } from './sdb.ts'; +import { locateSdb, type HostEnv } from './tools.ts'; + +type TizenRuntimeHost = TvRuntimeHost & Readonly<{ env: HostEnv }>; + +async function readySerials(sdb: ReturnType) { + return (await sdb.listDevices()) + .filter((device) => device.state === 'device') + .map((device) => device.serial); +} + +export function createTizenRuntime(options: TizenOptions, host: TizenRuntimeHost) { + const sdb = createSdb(() => locateSdb(options.sdbPath, host.env), host.env); + + async function resolveSerial(requested: string | undefined) { + const connected = await readySerials(sdb); + if (!requested) { + if (connected.length === 1) return connected[0]!; + throw host.createError( + 'DEVICE_NOT_FOUND', + connected.length === 0 + ? 'No Tizen TV is connected to sdb' + : 'More than one Tizen TV is connected to sdb', + { available: connected, hint: 'Pass --device .' }, + ); + } + const serial = sdbSerial(requested); + if (connected.includes(serial)) return serial; + await sdb.connect(serial); + if ((await readySerials(sdb)).includes(serial)) return serial; + throw host.createError('DEVICE_NOT_FOUND', `sdb could not connect to the Tizen TV ${serial}`, { + available: connected, + hint: 'Check that the TV is on and in Developer Mode with this host IP set as the developer IP.', + }); + } + + async function connectTv(flagDevice: string | undefined): Promise { + const requested = flagDevice ?? options.device; + const tv = sdb.forDevice(await resolveSerial(requested)); + return { + id: tv.serial, + name: requested ?? tv.serial, + launch: async (appId, { signal }) => await tv.launch(appId, { signal }), + close: (appId, options) => tv.kill(appId, options?.signal), + listApps: (options) => tv.listApps(options?.signal), + }; + } + + return createTvRuntime( + { provider: PROVIDER, label: 'Tizen', remoteKeys: TIZEN_REMOTE_KEYS, connectTv }, + host, + ); +} diff --git a/packages/provider-tizen/src/sdb.test.ts b/packages/provider-tizen/src/sdb.test.ts new file mode 100644 index 0000000000..4aefd05cb0 --- /dev/null +++ b/packages/provider-tizen/src/sdb.test.ts @@ -0,0 +1,282 @@ +import assert from 'node:assert/strict'; +import { once } from 'node:events'; +import { chmodSync, readFileSync, writeFileSync } from 'node:fs'; +import { createServer } from 'node:http'; +import path from 'node:path'; +import { isRequestCanceledError } from '@agent-device/kernel/errors'; +import { mkdtempForTestSync } from '@agent-device/test-kit'; +import { afterEach, test, vi } from 'vitest'; + +import { createSdb, parseAppList, parseDebugPort, parseDevices, sdbSerial } from './sdb.ts'; + +const APP_ID = 'AbCdE12345.example'; +const SERIAL = '192.0.2.10:26101'; +const DEBUG_PORT = 34567; + +// Records every call. SDB_DEBUG picks how `0 debug` answers, SDB_REMOVE_FAIL makes `forward --remove` +// exit 1 with it on stderr, and SDB_FAIL does the same for the other commands. One script serves every test, because macOS checks each new +// executable on its first run. +const SDB = path.join(mkdtempForTestSync('sdb-'), 'sdb'); +writeFileSync( + SDB, + `#!/bin/sh +echo "$*" >> "$SDB_CALLS" +case "$*" in + *" debug "*) + case "$SDB_DEBUG" in + port) echo 'successfully launched pid = 4242 with debug 1 port: ${DEBUG_PORT}' ;; + hang) exec sleep 30 ;; + *) echo 'closed'; exit 1 ;; + esac ;; + *"forward --remove"*) if [ -n "$SDB_REMOVE_FAIL" ]; then echo "$SDB_REMOVE_FAIL" >&2; exit 1; fi ;; + *vd_applist*) printf '%s\\n' '--app_title =Example TV----' '--app_tizen_id =${APP_ID}----' ;; + *) if [ -n "$SDB_FAIL" ]; then echo "$SDB_FAIL" >&2; exit 1; fi ;; +esac +`, +); +chmodSync(SDB, 0o755); + +const cleanups: (() => void)[] = []; +afterEach(() => { + for (const cleanup of cleanups.splice(0)) cleanup(); +}); + +function fakeTv(env: Record = {}) { + const log = path.join(mkdtempForTestSync('sdb-calls-'), 'calls'); + writeFileSync(log, ''); + const sdb = createSdb(() => SDB, { ...process.env, SDB_CALLS: log, ...env }); + const calls = () => readFileSync(log, 'utf8').trim().split('\n').filter(Boolean); + return { sdb, tv: sdb.forDevice(SERIAL), calls }; +} + +const forwardedPort = (calls: readonly string[]) => + calls + .map((call) => call.match(new RegExp(`forward tcp:(\\d+) tcp:${DEBUG_PORT}$`))?.[1]) + .find(Boolean); + +// Serves the page list the TV's inspector exposes, on the local port sdb was asked to forward. +async function servePageOnForward(calls: () => string[]) { + const port = await vi.waitFor( + () => { + const forwarded = forwardedPort(calls()); + assert.ok(forwarded); + return Number(forwarded); + }, + { timeout: 3000 }, + ); + const server = createServer((_request, response) => { + response.end( + JSON.stringify([ + { + title: 'Example TV', + url: `file:///opt/usr/apps/${APP_ID}/index.html`, + webSocketDebuggerUrl: 'ws://localhost/devtools/page/1', + }, + ]), + ); + }); + server.listen(port); + await once(server, 'listening'); + cleanups.push(() => void server.close()); +} + +test('launch kills the app, forwards the debug port it prints and returns the page; close removes the forward', async () => { + const { tv, calls } = fakeTv({ SDB_DEBUG: 'port' }); + + const launching = tv.launch(APP_ID, { settleMs: 0 }); + await servePageOnForward(calls); + const tunnel = await launching; + let notified = false; + tunnel.onClose(() => { + notified = true; + }); + await tunnel.close(); + + assert.equal(tunnel.target.webSocketDebuggerUrl, 'ws://localhost/devtools/page/1'); + assert.equal(notified, true); + const localPort = forwardedPort(calls()); + assert.deepEqual(calls(), [ + `-s ${SERIAL} shell 0 was_kill ${APP_ID}`, + `-s ${SERIAL} shell 0 debug ${APP_ID}`, + `-s ${SERIAL} forward tcp:${localPort} tcp:${DEBUG_PORT}`, + `-s ${SERIAL} forward --remove tcp:${localPort}`, + ]); +}); + +test('a debug launch that prints no port in time says the app was probably still running', async () => { + const { tv } = fakeTv({ SDB_DEBUG: 'hang' }); + + const launching = tv.launch(APP_ID, { settleMs: 0, timeoutMs: 300 }); + + await assert.rejects(launching, (error: { message?: string; details?: { hint?: string } }) => { + assert.equal(error.message, `${APP_ID} gave no debug port in 300ms`); + assert.match(error.details?.hint ?? '', /still running/); + return true; + }); +}); + +test('a debug launch that exits without a port fails before any forward', async () => { + const { tv, calls } = fakeTv({ SDB_DEBUG: 'none' }); + + const launching = tv.launch(APP_ID, { settleMs: 0 }); + + await assert.rejects(launching, { message: `sdb debug ${APP_ID} exited without a port` }); + assert.equal(forwardedPort(calls()), undefined); +}); + +test('a cancelled launch keeps its request-canceled reason and opens no forward', async () => { + const { tv, calls } = fakeTv({ SDB_DEBUG: 'hang' }); + const cancel = new AbortController(); + + const launching = tv.launch(APP_ID, { settleMs: 0, signal: cancel.signal }); + await vi.waitFor(() => assert.ok(calls().includes(`-s ${SERIAL} shell 0 debug ${APP_ID}`))); + cancel.abort(); + + await assert.rejects(launching, isRequestCanceledError); + assert.equal(forwardedPort(calls()), undefined); +}); + +test('a debug port that shows no page within the timeout removes the forward', async () => { + const { tv, calls } = fakeTv({ SDB_DEBUG: 'port' }); + + const launching = tv.launch(APP_ID, { settleMs: 0, timeoutMs: 600 }); + + await assert.rejects(launching, { + message: `${APP_ID} opened debug port ${DEBUG_PORT} but showed no page within 600ms`, + }); + assert.match(calls().at(-1) ?? '', /forward --remove tcp:\d+$/); +}); + +test('a launch cancelled while waiting for the page removes the forward and stays cancelled', async () => { + const { tv, calls } = fakeTv({ SDB_DEBUG: 'port' }); + const cancel = new AbortController(); + + const launching = tv.launch(APP_ID, { settleMs: 0, signal: cancel.signal }); + await vi.waitFor(() => assert.ok(forwardedPort(calls()))); + cancel.abort(); + + await assert.rejects(launching, isRequestCanceledError); + assert.equal(calls().at(-1), `-s ${SERIAL} forward --remove tcp:${forwardedPort(calls())}`); +}); + +test('a failed launch whose forward cannot be removed keeps its error and names the removal failure', async () => { + const { tv } = fakeTv({ SDB_DEBUG: 'port', SDB_REMOVE_FAIL: 'error: listener not found' }); + + const launching = tv.launch(APP_ID, { settleMs: 0, timeoutMs: 300 }); + + await assert.rejects( + launching, + (error: { message?: string; details?: Record }) => { + assert.match(error.message ?? '', /showed no page within 300ms$/); + assert.match(String(error.details?.tunnelRemoval), /error: listener not found/); + return true; + }, + ); +}); + +test('listApps reads the installed apps from vd_applist on the TV', async () => { + const { tv, calls } = fakeTv(); + + const apps = await tv.listApps(); + + assert.deepEqual(apps, [{ id: APP_ID, name: 'Example TV' }]); + assert.deepEqual(calls(), [`-s ${SERIAL} shell 0 vd_applist`]); +}); + +test('a forward sdb refuses fails with its stderr and runs no removal', async () => { + const { tv, calls } = fakeTv({ SDB_DEBUG: 'port', SDB_FAIL: 'error: cannot bind listener' }); + + const launching = tv.launch(APP_ID, { settleMs: 0 }); + + await assert.rejects( + launching, + (error: { message?: string; details?: Record }) => { + assert.match( + error.message ?? '', + /forward tcp:\d+ tcp:34567 failed: error: cannot bind listener$/, + ); + assert.equal(error.details?.tunnelRemoval, undefined); + return true; + }, + ); + assert.equal( + calls().some((call) => call.includes('--remove')), + false, + ); +}); + +test('a failing sdb command reports its stderr in a COMMAND_FAILED message', async () => { + const { sdb } = fakeTv({ SDB_FAIL: `failed to connect to ${SERIAL}` }); + + const connecting = sdb.connect(SERIAL); + + await assert.rejects(connecting, { + code: 'COMMAND_FAILED', + message: `sdb connect ${SERIAL} failed: failed to connect to ${SERIAL}`, + }); +}); + +test('an sdb missing at the configured path fails as TOOL_MISSING with the SDK hint', async () => { + const sdb = createSdb(() => path.join(mkdtempForTestSync('sdb-'), 'sdb'), process.env); + + const listing = sdb.listDevices(); + + await assert.rejects(listing, (error: { code?: string; details?: { hint?: string } }) => { + assert.equal(error.code, 'TOOL_MISSING'); + assert.match(error.details?.hint ?? '', /sdbPath/); + return true; + }); +}); + +test('parseDevices reads serial, state and model from sdb devices', () => { + const output = [ + 'List of devices attached ', + '192.0.2.10:26101 \tdevice \tQE55Q80', + '192.0.2.11:26101 \toffline \t', + '', + ].join('\n'); + + const devices = parseDevices(output); + + assert.deepEqual(devices, [ + { serial: '192.0.2.10:26101', state: 'device', model: 'QE55Q80' }, + { serial: '192.0.2.11:26101', state: 'offline', model: '' }, + ]); +}); + +test('parseAppList reads each user app id with its title from vd_applist', () => { + const output = [ + '', + 'get app list...', + 'apps count: 2', + '---------------------------------------------------------------------------------------------', + '--------------app_id =3201000000001-------------', + '--------------app_title =Example TV: News, Sports-------------', + '--------------app_tizen_id =AbCdE12345.example-------------', + '---------------------------------------------------------------------------------------------', + '--------------app_id =FgHiJ67890.dist-------------', + '--------------app_title =-------------', + '--------------app_tizen_id =FgHiJ67890.Untitled-------------', + ].join('\n'); + + const apps = parseAppList(output); + + assert.deepEqual(apps, [ + { id: 'AbCdE12345.example', name: 'Example TV: News, Sports' }, + { id: 'FgHiJ67890.Untitled', name: 'FgHiJ67890.Untitled' }, + ]); +}); + +test('parseDebugPort finds the port a debug launch prints, and nothing in other output', () => { + assert.equal( + parseDebugPort('... successfully launched pid = 4242 with debug 1\nport: 37145\n'), + 37145, + ); + assert.equal(parseDebugPort('closed'), undefined); +}); + +test('a bare TV IP gets the sdb port, and a full serial is kept', () => { + assert.equal(sdbSerial('192.0.2.10'), '192.0.2.10:26101'); + assert.equal(sdbSerial('192.0.2.10:26101'), '192.0.2.10:26101'); + assert.equal(sdbSerial('emulator-26101'), 'emulator-26101'); +}); diff --git a/packages/provider-tizen/src/sdb.ts b/packages/provider-tizen/src/sdb.ts new file mode 100644 index 0000000000..b05f56bba7 --- /dev/null +++ b/packages/provider-tizen/src/sdb.ts @@ -0,0 +1,222 @@ +import net from 'node:net'; + +import { isCommandTimeoutError, runCmd, runCmdStreaming } from '@agent-device/host-kit/command'; +import { Deadline, pollUntilSettled, sleep } from '@agent-device/host-kit/retry'; +import { AppError, asAppError, isRequestCanceledError } from '@agent-device/kernel/errors'; +import { listTargets, withTunnelRemoval, type PageTunnel } from '@agent-device/tv-cdp'; + +import { sdbError, type HostEnv } from './tools.ts'; + +const SDB_PORT = 26101; +// was_kill returns before the app process is gone; a debug launch that races it lands on the +// still-running app and blocks. +const KILL_SETTLE_MS = 2000; +const REMOVE_TIMEOUT_MS = 5000; + +export type SdbDevice = Readonly<{ + serial: string; + state: string; + model: string; +}>; + +export function parseDevices(output: string): SdbDevice[] { + return output + .split('\n') + .slice(1) + .map((line) => line.trim().split(/\s+/)) + .filter(([serial, state]) => serial && state) + .map(([serial, state, model]) => ({ serial: serial!, state: state!, model: model ?? '' })); +} + +type TizenApp = Readonly<{ id: string; name: string }>; + +// `0 vd_applist` prints one block per user app, each field as `--app_title =Name----`. +export function parseAppList(output: string): TizenApp[] { + const apps: TizenApp[] = []; + let title = ''; + for (const line of output.split('\n')) { + const field = line.match(/^-+([a-z_]+)\s*=(.*?)-+\s*$/); + if (!field) continue; + const [, key, raw = ''] = field; + const value = raw.trim(); + if (key === 'app_id') title = ''; + else if (key === 'app_title') title = value; + else if (key === 'app_tizen_id' && value) apps.push({ id: value, name: title || value }); + } + return apps; +} + +export const parseDebugPort = (output: string) => { + const port = Number(output.match(/port:\s*(\d+)/)?.[1]); + return port || undefined; +}; + +/** `192.0.2.10` and `192.0.2.10:26101` name the same TV; anything else is taken as a serial. */ +export function sdbSerial(device: string): string { + return /^[\d.]+$/.test(device) ? `${device}:${SDB_PORT}` : device; +} + +async function freeLocalPort(): Promise { + const server = net.createServer(); + await new Promise((resolve, reject) => { + server.once('error', reject); + server.listen(0, '127.0.0.1', resolve); + }); + const { port } = server.address() as net.AddressInfo; + await new Promise((resolve) => server.close(resolve)); + return port; +} + +type LaunchOptions = Readonly<{ signal?: AbortSignal; timeoutMs?: number; settleMs?: number }>; + +/** Removes the forward, then rethrows the error that ended the launch, naming a failed removal. */ +async function failAfter(remove: () => Promise, launchError: unknown): Promise { + try { + await remove(); + } catch (error) { + throw withTunnelRemoval(launchError, error); + } + throw launchError; +} + +export function createSdb(bin: () => string, env: HostEnv) { + const sdb = async ( + args: string[], + options: { signal?: AbortSignal; timeoutMs?: number } = {}, + ) => { + const command = bin(); + try { + return (await runCmd(command, args, { env, timeoutMs: 60_000, ...options })).stdout; + } catch (error) { + throw sdbError(`sdb ${args.filter((arg) => arg !== '-s').join(' ')}`, error); + } + }; + + const listDevices = async () => parseDevices(await sdb(['devices'])); + + const forDevice = (serial: string) => { + const shell0 = (args: string[], signal?: AbortSignal) => + sdb(['-s', serial, 'shell', '0', ...args], { signal }); + + // `0 debug` is a launch, not an attach: against a running app it blocks forever with no + // output, so the caller kills the app first and this bounds the wait. + async function debugLaunch(appId: string, signal: AbortSignal | undefined, timeoutMs: number) { + const found = new AbortController(); + let output = ''; + let port: number | undefined; + const read = (chunk: string) => { + output += chunk; + port ??= parseDebugPort(output); + if (port) found.abort(); + }; + try { + await runCmdStreaming(bin(), ['-s', serial, 'shell', '0', 'debug', appId], { + env, + timeoutMs, + signal: signal ? AbortSignal.any([signal, found.signal]) : found.signal, + onStdoutChunk: read, + onStderrChunk: read, + }); + } catch (error) { + if (port) return port; + if (isCommandTimeoutError(error)) { + throw new AppError('COMMAND_FAILED', `${appId} gave no debug port in ${timeoutMs}ms`, { + ...(output.trim() ? { output: output.trim() } : {}), + hint: output.trim() + ? `Check that ${appId} is installed with a debug-enabled package.` + : `${appId} was probably still running. Close it on the TV and retry.`, + }); + } + // An app that is not installed makes sdb print `closed` and exit with a failure. + if (asAppError(error).details?.processExitError !== true) { + throw sdbError(`sdb debug ${appId}`, error); + } + } + if (port) return port; + throw new AppError('COMMAND_FAILED', `sdb debug ${appId} exited without a port`, { + output: output.trim(), + hint: `Check that ${appId} is installed on the TV.`, + }); + } + + // A forward lives in the sdb server, not in a child process, so it ends only when removed; a TV + // that drops off the network shows up as the page socket closing. + async function forward(remotePort: number, signal: AbortSignal | undefined) { + const localPort = await freeLocalPort(); + // Removal also runs after a cancel, so it has its own short deadline instead of the request's. + const remove = async () => { + await sdb(['-s', serial, 'forward', '--remove', `tcp:${localPort}`], { + timeoutMs: REMOVE_TIMEOUT_MS, + }); + }; + try { + await sdb(['-s', serial, 'forward', `tcp:${localPort}`, `tcp:${remotePort}`], { signal }); + } catch (error) { + // sdb refused the forward when it exited with a failure; a cancel or timeout may have + // stopped it after the server made one. + if (asAppError(error).details?.processExitError === true) throw error; + return await failAfter(remove, error); + } + const listeners: (() => void)[] = []; + let removal: Promise | undefined; + const close = () => { + if (removal) return removal; + removal = remove(); + for (const listener of listeners) listener(); + return removal; + }; + return { + localPort, + onClose: (listener: () => void) => listeners.push(listener), + close, + }; + } + + return { + serial, + + async kill(appId: string, signal?: AbortSignal) { + await shell0(['was_kill', appId], signal); + }, + + // A stale forward answers for a dead port, so every launch opens a fresh one. + async launch(appId: string, options: LaunchOptions = {}): Promise { + const { signal, timeoutMs = 60_000, settleMs = KILL_SETTLE_MS } = options; + await shell0(['was_kill', appId], signal).catch((error: unknown) => { + if (isRequestCanceledError(error)) throw error; + }); + await sleep(settleMs, signal); + const remotePort = await debugLaunch(appId, signal, timeoutMs); + const tunnel = await forward(remotePort, signal); + const { value: target } = await pollUntilSettled({ + deadline: Deadline.fromTimeoutMs(timeoutMs), + intervalMs: 500, + signal, + settled: (found) => found !== undefined, + probe: async () => (await listTargets(tunnel.localPort, signal).catch(() => []))[0], + }).catch(async (error: unknown) => await failAfter(tunnel.close, error)); + if (target) return { target, onClose: tunnel.onClose, close: tunnel.close }; + return await failAfter( + tunnel.close, + new AppError( + 'COMMAND_FAILED', + `${appId} opened debug port ${remotePort} but showed no page within ${timeoutMs}ms`, + { hint: `Retry open ${appId}.` }, + ), + ); + }, + + async listApps(signal?: AbortSignal) { + return parseAppList(await shell0(['vd_applist'], signal)); + }, + }; + }; + + return { + listDevices, + async connect(serial: string) { + await sdb(['connect', serial], { timeoutMs: 30_000 }); + }, + forDevice, + }; +} diff --git a/packages/provider-tizen/src/tools.test.ts b/packages/provider-tizen/src/tools.test.ts new file mode 100644 index 0000000000..53af687d27 --- /dev/null +++ b/packages/provider-tizen/src/tools.test.ts @@ -0,0 +1,47 @@ +import assert from 'node:assert/strict'; +import { mkdirSync, writeFileSync } from 'node:fs'; +import path from 'node:path'; +import { mkdtempForTestSync } from '@agent-device/test-kit'; +import { test } from 'vitest'; + +import { locateSdb } from './tools.ts'; + +function extensionHome() { + const home = mkdtempForTestSync('tizen-home-'); + const platform = path.join(home, '.tizen-extension-platform'); + const tools = path.join(platform, 'server/sdktools/data/tools'); + mkdirSync(tools, { recursive: true }); + writeFileSync(path.join(tools, 'sdb'), ''); + writeFileSync(path.join(home, '.tizen.path.config'), JSON.stringify({ url: platform })); + return { home, tools }; +} + +test('sdb is found in the SDK that ~/.tizen.path.config names', () => { + const { home, tools } = extensionHome(); + + assert.equal(locateSdb(undefined, { HOME: home }), path.join(tools, 'sdb')); +}); + +test('TIZEN_SDK wins over the extension config, and an explicit path wins over both', () => { + const extension = extensionHome(); + const sdk = mkdtempForTestSync('tizen-sdk-'); + mkdirSync(path.join(sdk, 'tools')); + writeFileSync(path.join(sdk, 'tools/sdb'), ''); + const env = { HOME: extension.home, TIZEN_SDK: sdk }; + + assert.equal(locateSdb(undefined, env), path.join(sdk, 'tools/sdb')); + assert.equal(locateSdb('/opt/sdb', env), '/opt/sdb'); +}); + +test('without an SDK, sdb fails as TOOL_MISSING with a hint naming sdbPath', () => { + const home = mkdtempForTestSync('tizen-empty-'); + + assert.throws( + () => locateSdb(undefined, { HOME: home }), + (error: { code?: string; details?: { hint?: string } }) => { + assert.equal(error.code, 'TOOL_MISSING'); + assert.match(error.details?.hint ?? '', /sdbPath/); + return true; + }, + ); +}); diff --git a/packages/provider-tizen/src/tools.ts b/packages/provider-tizen/src/tools.ts new file mode 100644 index 0000000000..48f7419fe4 --- /dev/null +++ b/packages/provider-tizen/src/tools.ts @@ -0,0 +1,58 @@ +import { existsSync, readFileSync } from 'node:fs'; +import { homedir } from 'node:os'; +import path from 'node:path'; + +import { AppError, asAppError } from '@agent-device/kernel/errors'; + +export type HostEnv = Readonly>; + +const SDK_HINT = + 'Install the VS Code Tizen extension and run its Set Up step, set TIZEN_SDK to the SDK data directory, or set the plugin option sdbPath.'; +const CONNECT_HINT = + 'Check that the TV is on, in Developer Mode with this host IP set, and reachable with sdb connect .'; + +// sdb ships only inside the VS Code Tizen extension's SDK, never on PATH. The extension records +// the SDK root in ~/.tizen.path.config. +function sdkDataDirs(env: HostEnv): string[] { + const dirs = env.TIZEN_SDK ? [env.TIZEN_SDK] : []; + const config = path.join(env.HOME ?? homedir(), '.tizen.path.config'); + if (existsSync(config)) { + try { + const { url } = JSON.parse(readFileSync(config, 'utf8')) as { url?: unknown }; + if (typeof url === 'string') dirs.push(path.join(url, 'server/sdktools/data')); + } catch { + // An unreadable config is the same as no config: the SDK is not found. + } + } + return dirs; +} + +export function locateSdb(explicit: string | undefined, env: HostEnv): string { + if (explicit) return explicit; + const found = sdkDataDirs(env) + .map((dir) => path.join(dir, 'tools', process.platform === 'win32' ? 'sdb.exe' : 'sdb')) + .find((candidate) => existsSync(candidate)); + if (!found) { + throw new AppError('TOOL_MISSING', 'sdb was not found in a Tizen SDK', { hint: SDK_HINT }); + } + return found; +} + +export function sdbError(name: string, error: unknown): AppError { + const failure = asAppError(error, 'COMMAND_FAILED'); + if (failure.code === 'TOOL_MISSING') { + return new AppError('TOOL_MISSING', 'sdb is not installed at the configured path', { + hint: SDK_HINT, + }); + } + if (failure.details?.processExitError !== true) return failure; + const output = (key: string) => { + const value = failure.details?.[key]; + return typeof value === 'string' ? value.trim() : ''; + }; + const detail = output('stderr') || output('stdout'); + return new AppError('COMMAND_FAILED', `${name} failed${detail ? `: ${detail}` : ''}`, { + hint: CONNECT_HINT, + processExitError: true, + }); +} diff --git a/packages/provider-tizen/test/package-smoke.mjs b/packages/provider-tizen/test/package-smoke.mjs new file mode 100644 index 0000000000..8457e0d648 --- /dev/null +++ b/packages/provider-tizen/test/package-smoke.mjs @@ -0,0 +1,2 @@ +export const provider = 'tizen'; +export const args = ['--device', '192.0.2.10']; diff --git a/packages/provider-tizen/tsconfig.json b/packages/provider-tizen/tsconfig.json new file mode 100644 index 0000000000..7399c8aba4 --- /dev/null +++ b/packages/provider-tizen/tsconfig.json @@ -0,0 +1,30 @@ +{ + "extends": "../../tsconfig.json", + "compilerOptions": { + "composite": true, + "noEmit": false, + "emitDeclarationOnly": true, + "declaration": true, + "declarationDir": "./dist-types", + "tsBuildInfoFile": "./dist-types/index.tsbuildinfo", + "rootDir": ".", + "paths": { + "agent-device/plugins": ["./node_modules/agent-device/src/sdk/plugins.ts"] + } + }, + "include": ["src"], + "references": [ + { + "path": "../host-kit" + }, + { + "path": "../kernel" + }, + { + "path": "../test-kit" + }, + { + "path": "../tv-cdp" + } + ] +} diff --git a/packages/provider-tizen/tsdown.config.ts b/packages/provider-tizen/tsdown.config.ts new file mode 100644 index 0000000000..75de008ad3 --- /dev/null +++ b/packages/provider-tizen/tsdown.config.ts @@ -0,0 +1,4 @@ +import { TV_PLUGIN_BUNDLE } from '@agent-device/tv-cdp/bundle'; +import { defineConfig } from 'tsdown'; + +export default defineConfig(TV_PLUGIN_BUNDLE); diff --git a/packages/provider-webos/README.md b/packages/provider-webos/README.md index bb5ab85466..2864e9963e 100644 --- a/packages/provider-webos/README.md +++ b/packages/provider-webos/README.md @@ -13,7 +13,8 @@ app): `connect`, `open`, `snapshot -i`, `snapshot -s`, `tv-remote` (directions, agent-device leases only iOS, Android and HarmonyOS devices, so a webOS TV enters as an **Android TV device that this plugin owns** (`platform: android`, `target: tv`, id -`webos:`). Every command on that device goes to this plugin's runtime, never to adb. +`webos::`, or `webos:` for a USB entry). Every command on +that device goes to this plugin's runtime, never to adb. ## Requirements diff --git a/packages/provider-webos/src/ares.test.ts b/packages/provider-webos/src/ares.test.ts index 21312b2fc2..44c3e4543b 100644 --- a/packages/provider-webos/src/ares.test.ts +++ b/packages/provider-webos/src/ares.test.ts @@ -19,17 +19,31 @@ function fakeAresBin(outputs: Record) { return bin; } -test('listDevices returns only TV profiles with their IP and default flag', async () => { +test('listDevices returns only TV profiles, each at its ip:port, or at its name for a USB entry', async () => { const bin = fakeAresBin({ 'ares-setup-device': JSON.stringify([ - { profile: 'tv', name: 'living-room', default: true, deviceinfo: { ip: '192.0.2.10' } }, - { profile: 'ose', name: 'signage', default: false, deviceinfo: { ip: '192.0.2.20' } }, + { + profile: 'tv', + name: 'living-room', + default: true, + deviceinfo: { ip: '192.0.2.10', port: '9922' }, + }, + { + profile: 'ose', + name: 'signage', + default: false, + deviceinfo: { ip: '192.0.2.20', port: '9922' }, + }, + { profile: 'tv', name: 'usb-tv', default: false, deviceinfo: { uid: 'usb-1' } }, ]), }); const devices = await createAres(bin, process.env).listDevices(); - assert.deepEqual(devices, [{ name: 'living-room', isDefault: true }]); + assert.deepEqual(devices, [ + { name: 'living-room', endpoint: '192.0.2.10:9922', isDefault: true }, + { name: 'usb-tv', endpoint: 'usb-tv', isDefault: false }, + ]); }); test('listApps reads each app id with its title from ares-install --listfull', async () => { diff --git a/packages/provider-webos/src/ares.ts b/packages/provider-webos/src/ares.ts index 26b78535d6..87ee7d7649 100644 --- a/packages/provider-webos/src/ares.ts +++ b/packages/provider-webos/src/ares.ts @@ -7,6 +7,8 @@ import { listTargets, type PageTunnel } from '@agent-device/tv-cdp'; export type AresDevice = Readonly<{ name: string; + /** The TV endpoint the entry reaches, the same for every entry that names that TV. */ + endpoint: string; isDefault: boolean; }>; @@ -14,6 +16,8 @@ type AresSetupEntry = { profile: string; name: string; default: boolean; + // ares lists an SSH entry with its IP and port, and a USB (novacom) entry with neither. + deviceinfo: { ip: string; port: string } | { ip?: undefined }; }; export type HostEnv = Readonly>; @@ -40,8 +44,8 @@ export function createAres(bin: string | undefined, hostEnv: HostEnv) { await ares('ares-launch', ['-d', device, '--close', appId], signal); }, - async listApps(): Promise<{ id: string; name: string }[]> { - return parseAppList(await ares('ares-install', ['-d', device, '--listfull'])); + async listApps(signal?: AbortSignal): Promise<{ id: string; name: string }[]> { + return parseAppList(await ares('ares-install', ['-d', device, '--listfull'], signal)); }, // ares-inspect is a long-lived ssh tunnel that prints its local inspector port once it is up. @@ -68,7 +72,23 @@ export function createAres(bin: string | undefined, hostEnv: HostEnv) { }); child.stdout?.on('data', collect); child.stderr?.on('data', collect); - const close = () => child.kill(); + const exited = new Promise((resolve) => { + if (child.exitCode !== null || child.signalCode !== null) resolve(); + else { + child.once('exit', () => resolve()); + child.once('error', () => resolve()); + } + }); + const close = async () => { + child.kill('SIGTERM'); + const forceKill = setTimeout(() => child.kill('SIGKILL'), 5_000); + forceKill.unref(); + try { + await exited; + } finally { + clearTimeout(forceKill); + } + }; const appTarget = async (port: number) => { try { const targets = await listTargets(port); @@ -94,15 +114,15 @@ export function createAres(bin: string | undefined, hostEnv: HostEnv) { const port = Number(output.match(/localhost:(\d+)/)?.[1]); return port ? await appTarget(port) : undefined; }, - }).catch((error: unknown) => { - close(); + }).catch(async (error: unknown) => { + await close(); throw error; }); if (target) { collecting = false; return { target, close, onClose: (listener) => child.once('exit', listener) }; } - close(); + await close(); const lastError = inspectorError ? `; the inspector last failed with: ${errorMessage(inspectorError)}` : ''; @@ -121,6 +141,7 @@ export function createAres(bin: string | undefined, hostEnv: HostEnv) { .filter((device) => device.profile === 'tv') .map((device) => ({ name: device.name, + endpoint: aresEndpoint(device), isDefault: device.default, })); }, @@ -128,6 +149,11 @@ export function createAres(bin: string | undefined, hostEnv: HostEnv) { }; } +// Two entries can name one TV, so the TV is the address the entry reaches, not the entry. +function aresEndpoint({ name, deviceinfo }: AresSetupEntry) { + return deviceinfo.ip ? `${deviceinfo.ip}:${deviceinfo.port}` : name; +} + function aresError(name: string, error: unknown): AppError { const failure = asAppError(error, 'COMMAND_FAILED'); if (failure.code === 'TOOL_MISSING') { diff --git a/packages/provider-webos/src/runtime.test.ts b/packages/provider-webos/src/runtime.test.ts index 90593fa436..521ee6c68e 100644 --- a/packages/provider-webos/src/runtime.test.ts +++ b/packages/provider-webos/src/runtime.test.ts @@ -27,14 +27,20 @@ function lease(leaseId: string) { }; } -// Two registered TVs, bedroom the ares default. Every ares call is recorded, and ares-inspect fails +// Two registered TVs, bedroom the ares default, and lounge a second entry for the living-room TV. Every ares call is recorded, and ares-inspect fails // so open stops right after the launch. One script answers for every ares CLI through symlinks, // because macOS checks each new executable on its first run. const ARES_BIN = mkdtempForTestSync('ares-'); const CALLS = path.join(ARES_BIN, 'calls.log'); const TVS = [ - { profile: 'tv', name: 'living-room', default: false }, - { profile: 'tv', name: 'bedroom', default: true }, + { + profile: 'tv', + name: 'living-room', + default: false, + deviceinfo: { ip: '192.0.2.10', port: '9922' }, + }, + { profile: 'tv', name: 'bedroom', default: true, deviceinfo: { ip: '192.0.2.11', port: '9922' } }, + { profile: 'tv', name: 'lounge', default: false, deviceinfo: { ip: '192.0.2.10', port: '9922' } }, ]; writeFileSync( path.join(ARES_BIN, 'ares'), @@ -75,7 +81,7 @@ test('connect leases the ares default TV when neither the flag nor the options n const allocated = await runtime.leaseLifecycle.allocate(lease('lease-a')); - assert.equal(allocated?.device.id, 'webos:bedroom'); + assert.equal(allocated?.device.id, 'webos:192.0.2.11:9922'); }); test('connect leases the TV the --device flag names over the plugin default', async () => { @@ -85,7 +91,20 @@ test('connect leases the TV the --device flag names over the plugin default', as flags: { device: 'living-room' }, }); - assert.equal(allocated?.device.id, 'webos:living-room'); + assert.equal(allocated?.device.id, 'webos:192.0.2.10:9922'); + assert.equal(allocated?.device.name, 'living-room'); +}); + +test('a second ares entry for the held TV is refused with DEVICE_IN_USE', async () => { + const runtime = await webosRuntime(); + await runtime.leaseLifecycle.allocate(lease('lease-a'), { flags: { device: 'living-room' } }); + + const second = runtime.leaseLifecycle.allocate(lease('lease-b'), { flags: { device: 'lounge' } }); + + await assert.rejects( + second, + (error: unknown) => error instanceof AppError && error.code === 'DEVICE_IN_USE', + ); }); test('connect to a TV ares does not know fails as DEVICE_NOT_FOUND and lists the known TVs', async () => { @@ -97,7 +116,8 @@ test('connect to a TV ares does not know fails as DEVICE_NOT_FOUND and lists the error instanceof AppError && error.code === 'DEVICE_NOT_FOUND' && error.message === 'No webOS TV named attic in ares-setup-device' && - JSON.stringify(error.details?.available) === JSON.stringify(['living-room', 'bedroom']), + JSON.stringify(error.details?.available) === + JSON.stringify(['living-room', 'bedroom', 'lounge']), ); }); diff --git a/packages/provider-webos/src/runtime.ts b/packages/provider-webos/src/runtime.ts index 3e255421cb..53ca044d52 100644 --- a/packages/provider-webos/src/runtime.ts +++ b/packages/provider-webos/src/runtime.ts @@ -27,14 +27,15 @@ export function createWebosRuntime(options: WebosOptions, host: WebosRuntimeHost } const tv = ares.forDevice(device.name); return { + id: device.endpoint, name: device.name, launch: async (appId, { relaunch, signal }) => { if (relaunch) await tv.close(appId, signal).catch(() => {}); await tv.launch(appId, signal); return await tv.inspect(appId, signal); }, - close: (appId) => tv.close(appId), - listApps: tv.listApps, + close: (appId, options) => tv.close(appId, options?.signal), + listApps: (options) => tv.listApps(options?.signal), }; } diff --git a/packages/tv-cdp/src/cdp.ts b/packages/tv-cdp/src/cdp.ts index 8e38502c29..423dbabd2c 100644 --- a/packages/tv-cdp/src/cdp.ts +++ b/packages/tv-cdp/src/cdp.ts @@ -1,4 +1,4 @@ -import { AppError } from '@agent-device/kernel/errors'; +import { AppError, asAppError, errorMessage } from '@agent-device/kernel/errors'; import { createReplyWaiters } from './replies.ts'; @@ -28,9 +28,21 @@ type CdpMessage = { error?: { message: string; code: number }; }; -export async function listTargets(port: number): Promise { +/** Keeps the error that ended an operation and names a page tunnel removal that failed with it. */ +export function withTunnelRemoval(primary: unknown, removal: unknown): AppError { + const failure = asAppError(primary, 'COMMAND_FAILED'); + return new AppError( + failure.code, + failure.message, + { ...failure.details, tunnelRemoval: errorMessage(removal) }, + failure, + ); +} + +export async function listTargets(port: number, signal?: AbortSignal): Promise { + const timeout = AbortSignal.timeout(5000); const response = await fetch(`http://localhost:${port}/json`, { - signal: AbortSignal.timeout(5000), + signal: signal ? AbortSignal.any([signal, timeout]) : timeout, }); if (!response.ok) { throw new AppError( diff --git a/packages/tv-cdp/src/index.ts b/packages/tv-cdp/src/index.ts index e290209291..68a60fc14b 100644 --- a/packages/tv-cdp/src/index.ts +++ b/packages/tv-cdp/src/index.ts @@ -1,7 +1,7 @@ import type { TvRuntimeHost } from './runtime.ts'; import type { TvPlatform } from './tv.ts'; -export { listTargets } from './cdp.ts'; +export { listTargets, withTunnelRemoval } from './cdp.ts'; export { NAVIGATION_KEYS, type RemoteKeys } from './keys.ts'; export type { TvRuntimeHost } from './runtime.ts'; export type { PageTunnel, TvConnection } from './tv.ts'; diff --git a/packages/tv-cdp/src/page-interactor.ts b/packages/tv-cdp/src/page-interactor.ts index b824ace87d..f9f28cf2f8 100644 --- a/packages/tv-cdp/src/page-interactor.ts +++ b/packages/tv-cdp/src/page-interactor.ts @@ -13,7 +13,7 @@ import { Deadline, pollUntilSettled } from '@agent-device/host-kit/retry'; import { AppError } from '@agent-device/kernel/errors'; import type { RawSnapshotNode } from '@agent-device/kernel/snapshot'; -import { connectCdp, listTargets } from './cdp.ts'; +import { connectCdp, listTargets, withTunnelRemoval } from './cdp.ts'; import { keyEvents, supportedRemoteButtons, type KeyDefinition, type RemoteKeys } from './keys.ts'; import { createPageLog, type PageLog } from './page-log.ts'; import { SNAPSHOT_EXPRESSION, type SnapshotCapture } from './snapshot.ts'; @@ -41,8 +41,9 @@ async function attachPage( timeoutMs = 30_000, ) { if (session.released) { - tunnel.close(); - throw interruptedOpen(session, appId); + session.tunnel = tunnel; + detachPage(session); + return await failAfterTeardown(session, interruptedOpen(session, appId), signal); } // Teardown closes only what the session holds, so the tunnel goes on it before the first await. session.tunnel = tunnel; @@ -79,9 +80,13 @@ async function attachPage( }); } } catch (error) { - if (!attached()) throw interruptedOpen(session, appId); - detachPage(session); - throw error; + const interrupted = !attached(); + if (!interrupted) detachPage(session); + return await failAfterTeardown( + session, + interrupted ? interruptedOpen(session, appId) : error, + signal, + ); } watchTarget(session, tunnel); } @@ -121,12 +126,66 @@ function appLog(session: TvSession, appId: string): PageLog { return created; } +// A page can drop with no request to report a failed tunnel removal to, so the session keeps the +// first failure until a close or release can. export function detachPage(session: TvSession) { - session.page?.close(); - session.tunnel?.close(); + const { page, tunnel } = session; session.page = undefined; session.tunnel = undefined; session.appId = undefined; + page?.close(); + if (!tunnel) return; + const previous = session.teardown; + const removal = tunnel.close().then( + () => undefined, + (error: unknown) => error, + ); + session.teardown = Promise.all([previous, removal]).then( + ([earlier, latest]) => earlier ?? latest, + ); +} + +/** + * Waits for every tunnel the session closed and resolves true, or resolves false as soon as the + * request is cancelled; a removal still running keeps its outcome for a later close or release. + */ +async function settleTeardown(session: TvSession, signal: AbortSignal | undefined) { + const { teardown } = session; + if (!teardown) return true; + if (!signal) { + await teardown; + return true; + } + if (signal.aborted) return false; + return await new Promise((resolve) => { + const cancelled = () => resolve(false); + signal.addEventListener('abort', cancelled, { once: true }); + void teardown.then(() => { + signal.removeEventListener('abort', cancelled); + resolve(true); + }); + }); +} + +/** Waits for every tunnel the session closed, and throws the first failed removal once. */ +export async function finishTeardown(session: TvSession) { + const failure = await session.teardown; + session.teardown = undefined; + if (failure) throw failure; +} + +/** Throws the error that ended the operation, naming a tunnel removal that failed in time with it. */ +async function failAfterTeardown( + session: TvSession, + primary: unknown, + signal: AbortSignal | undefined, +): Promise { + if (await settleTeardown(session, signal)) { + const failure = await session.teardown; + session.teardown = undefined; + if (failure) throw withTunnelRemoval(primary, failure); + } + throw primary; } export function createPageInteractor( @@ -192,6 +251,7 @@ export function createPageInteractor( return { async open(app, options) { detachPage(session); + await settleTeardown(session, signal); const tunnel = await tv.launch(app, { relaunch: options?.terminateRunningApp === true, signal, @@ -202,7 +262,12 @@ export function createPageInteractor( openDevice: unsupported('Opening the device without an app'), async close(app) { detachPage(session); - await tv.close(app); + try { + await tv.close(app, { signal }); + } catch (error) { + return await failAfterTeardown(session, error, signal); + } + await finishTeardown(session); }, async tap(x, y) { await press(x, y, 100); diff --git a/packages/tv-cdp/src/runtime.test.ts b/packages/tv-cdp/src/runtime.test.ts index 4268353394..d971172926 100644 --- a/packages/tv-cdp/src/runtime.test.ts +++ b/packages/tv-cdp/src/runtime.test.ts @@ -172,35 +172,66 @@ async function fakePage() { return page; } -async function tvRuntime(onLaunch: () => Promise = async () => undefined) { +type FakeTvOptions = Readonly<{ + onConnect?: () => Promise; + onLaunch?: () => Promise; + removal?: () => Promise; + appClose?: () => Promise; +}>; + +// One physical TV that answers to any requested name, as an IP and IP:port reach the same Tizen TV. +async function tvRuntime(options: FakeTvOptions = {}) { + const { + onConnect = async () => undefined, + onLaunch = async () => undefined, + removal = async () => undefined, + appClose = async () => undefined, + } = options; const page = await fakePage(); const tunnel = { closed: 0, onClose: [] as (() => void)[] }; - const tv: TvConnection = { - name: 'living-room', + const signals = { + close: [] as (AbortSignal | undefined)[], + listApps: [] as (AbortSignal | undefined)[], + }; + const tv = (requested: string | undefined): TvConnection => ({ + id: 'tv-1', + name: requested ?? 'living-room', launch: async (appId) => { await onLaunch(); return { target: { title: appId, url: `http://tv/${appId}/`, webSocketDebuggerUrl: page.url }, onClose: (listener) => tunnel.onClose.push(listener), - close: () => void tunnel.closed++, + close: async () => { + tunnel.closed++; + await removal(); + }, }; }, - close: async () => undefined, - listApps: async () => [{ id: APP_ID, name: 'Example' }], - }; + close: async (_appId, closeOptions) => { + signals.close.push(closeOptions?.signal); + await appClose(); + }, + listApps: async (listOptions) => { + signals.listApps.push(listOptions?.signal); + return [{ id: APP_ID, name: 'Example' }]; + }, + }); const { runtime, platformModule } = createTvRuntime( { provider: 'webos', label: 'webOS', remoteKeys: NAVIGATION_KEYS, - connectTv: async () => tv, + connectTv: async (requested) => { + await onConnect(); + return tv(requested); + }, }, { createError: (code, message, details) => new AppError(code, message, details) }, ); cleanups.push(runtime.shutdown); const allocated = await runtime.leaseLifecycle.allocate(lease('lease-a')); assert.ok(allocated); - return { runtime, platformModule, device: allocated.device, page, tunnel }; + return { runtime, platformModule, device: allocated.device, page, tunnel, signals }; } async function openedTv() { @@ -221,6 +252,128 @@ test('a second lease on a held TV is refused with DEVICE_IN_USE', async () => { ); }); +test('a lease that names the held TV by another alias is refused with DEVICE_IN_USE', async () => { + const { runtime, device } = await tvRuntime(); + + const second = runtime.leaseLifecycle.allocate(lease('lease-b'), { + flags: { device: '192.0.2.10:26101' }, + }); + + await assert.rejects(second, (error: unknown) => { + assert.ok(error instanceof AppError); + assert.equal(error.code, 'DEVICE_IN_USE'); + return true; + }); + assert.equal(device.id, 'webos:tv-1'); + assert.equal(device.name, 'living-room'); +}); + +test('the leased device keeps the requested name and resolves to the TV session by its identity', async () => { + const { runtime } = await tvRuntime(); + await runtime.leaseLifecycle.release(lease('lease-a')); + + const allocated = await runtime.leaseLifecycle.allocate(lease('lease-b'), { + flags: { device: '192.0.2.10' }, + }); + + assert.equal(allocated?.device.name, '192.0.2.10'); + assert.equal(allocated?.device.id, 'webos:tv-1'); + assert.ok(runtime.getInteractor(allocated.device)); +}); + +test('a tunnel that cannot be removed fails close with the removal error', async () => { + const { runtime, device } = await tvRuntime({ + removal: async () => { + throw new AppError('COMMAND_FAILED', 'forward --remove failed'); + }, + }); + await runtime.getInteractor(device)!.open(APP_ID); + + await assert.rejects(runtime.getInteractor(device)!.close!(APP_ID), { + message: 'forward --remove failed', + }); +}); + +test('a close that fails with its tunnel removal keeps its own error and names the removal', async () => { + const { runtime, device } = await tvRuntime({ + removal: async () => { + throw new AppError('COMMAND_FAILED', 'forward --remove failed'); + }, + appClose: async () => { + throw new AppError('COMMAND_FAILED', 'was_kill failed'); + }, + }); + await runtime.getInteractor(device)!.open(APP_ID); + + await assert.rejects(runtime.getInteractor(device)!.close!(APP_ID), { + message: 'was_kill failed', + details: { tunnelRemoval: 'forward --remove failed' }, + }); +}); + +test('a cancelled open stops waiting for the previous tunnel removal', async () => { + const pending: (() => void)[] = []; + const { runtime, device } = await tvRuntime({ + removal: () => new Promise((resolve) => pending.push(() => resolve())), + }); + await runtime.getInteractor(device)!.open(APP_ID); + const cancel = new AbortController(); + cancel.abort(); + + const reopening = runtime.getInteractor(device, cancel.signal)!.open(APP_ID); + + await assert.rejects(reopening, isRequestCanceledError); + assert.ok(pending.length > 0); + for (const finish of pending) finish(); +}); + +test('a runtime that is shutting down refuses new leases', async () => { + const { runtime } = await tvRuntime(); + + await runtime.shutdown(); + + await assert.rejects(runtime.leaseLifecycle.allocate(lease('lease-b')), { + code: 'COMMAND_FAILED', + message: 'The webOS runtime is shutting down', + }); +}); + +test('a lease whose TV connects after shutdown starts is refused and never listed', async () => { + let connecting = Promise.resolve(); + const { runtime } = await tvRuntime({ onConnect: () => connecting }); + let connected = () => {}; + connecting = new Promise((resolve) => { + connected = resolve; + }); + const allocating = runtime.leaseLifecycle.allocate(lease('lease-b')); + + await runtime.shutdown(); + connected(); + + await assert.rejects(allocating, { + code: 'COMMAND_FAILED', + message: 'The webOS runtime is shutting down', + }); + assert.equal( + await runtime.deviceInventoryProvider({ leaseProvider: 'webos', leaseId: 'lease-b' }), + null, + ); +}); + +test('a tunnel lost in the background that cannot be removed fails the lease release', async () => { + const { runtime, device, tunnel } = await tvRuntime({ + removal: async () => { + throw new AppError('COMMAND_FAILED', 'forward --remove failed'); + }, + }); + await runtime.getInteractor(device)!.open(APP_ID); + for (const listener of tunnel.onClose) listener(); + + await assert.rejects(runtime.leaseLifecycle.release(lease('lease-a')), { + message: 'forward --remove failed', + }); +}); + test('a cancelled open closes the tunnel it opened', async () => { const { runtime, device, tunnel } = await tvRuntime(); const cancel = new AbortController(); @@ -272,7 +425,7 @@ test('a tunnel that goes down detaches the page', async () => { test('a lease released while open is in flight leaves no tunnel behind', async () => { let releaseLease = async (): Promise => undefined; - const { runtime, device, tunnel } = await tvRuntime(() => releaseLease()); + const { runtime, device, tunnel } = await tvRuntime({ onLaunch: () => releaseLease() }); releaseLease = () => runtime.leaseLifecycle.release(lease('lease-a')); await assert.rejects(runtime.getInteractor(device)!.open(APP_ID), /lease ended/); @@ -489,6 +642,16 @@ test('the platform binding lists the TV apps and reports the attached app while assert.deepEqual(await binding.operations.appState?.(), { package: undefined }); }); +test('close and apps hand the request signal to the TV', async () => { + const { binding, device, runtime, signals } = await boundTv(); + + await binding.operations.listApps!({ device, filter: 'user-installed' }); + await runtime.getInteractor(device, scope.signal)!.close!(APP_ID); + + assert.deepEqual(signals.listApps, [scope.signal]); + assert.deepEqual(signals.close, [scope.signal]); +}); + test('the platform binding asks for connect once the lease that held the TV is gone', async () => { const { binding, device, runtime } = await boundTv(); diff --git a/packages/tv-cdp/src/runtime.ts b/packages/tv-cdp/src/runtime.ts index 84001544a4..70192552a4 100644 --- a/packages/tv-cdp/src/runtime.ts +++ b/packages/tv-cdp/src/runtime.ts @@ -56,7 +56,7 @@ import { AppError } from '@agent-device/kernel/errors'; import { appLogDescriptorCodec } from './app-log-descriptor.ts'; import { supportedRemoteButtons } from './keys.ts'; -import { createPageInteractor, detachPage } from './page-interactor.ts'; +import { createPageInteractor, detachPage, finishTeardown } from './page-interactor.ts'; import type { TvPlatform, TvSession } from './tv.ts'; const INSTANCE = 'local'; @@ -68,13 +68,15 @@ export type TvRuntimeHost = Readonly<{ createError(code: string, message: string, details?: Record): Error; }>; +type LeasedTv = Readonly<{ id: string; name: string }>; + // agent-device has a fixed platform list and leases only ios/android/harmonyos devices, so a TV // enters as an Android TV device that its provider owns end to end. -function tvDevice(platform: TvPlatform, name: string): DeviceInfo { +function tvDevice(platform: TvPlatform, tv: LeasedTv): DeviceInfo { return { platform: 'android', - id: `${platform.provider}:${name}`, - name, + id: `${platform.provider}:${tv.id}`, + name: tv.name, kind: 'device', target: 'tv', booted: true, @@ -175,50 +177,63 @@ export function createTvRuntime(platform: TvPlatform, host: TvRuntimeHost) { const { provider, label } = platform; const devicePrefix = `${provider}:`; const sessions = new Map(); - const leases = new Map(); + const leases = new Map(); const owner = providerRuntimeOwner(provider, INSTANCE); const ownsDevice = (device: DeviceInfo) => device.id.startsWith(devicePrefix); - const tvName = (device: DeviceInfo) => device.id.slice(devicePrefix.length); + const tvId = (device: DeviceInfo) => device.id.slice(devicePrefix.length); const sessionFor = (device: DeviceInfo) => - ownsDevice(device) ? sessions.get(tvName(device)) : undefined; + ownsDevice(device) ? sessions.get(tvId(device)) : undefined; const getInteractor = (device: DeviceInfo, signal?: AbortSignal): Interactor | undefined => { const session = sessionFor(device); return session && createPageInteractor(session, platform.remoteKeys, signal); }; + let shuttingDown = false; + const refuseWhileShuttingDown = () => { + if (shuttingDown) { + throw host.createError('COMMAND_FAILED', `The ${label} runtime is shutting down`); + } + }; + async function allocate(lease: DeviceLease, context?: LeaseLifecycleContext) { if (lease.leaseProvider !== provider) return undefined; + refuseWhileShuttingDown(); const leased = leases.get(lease.leaseId); if (leased) return { device: tvDevice(platform, leased) }; const flagDevice = context?.flags?.device; const tv = await platform.connectTv(typeof flagDevice === 'string' ? flagDevice : undefined); - const holder = [...leases].find(([id, name]) => name === tv.name && id !== lease.leaseId); + refuseWhileShuttingDown(); + const holder = [...leases].find( + ([leaseId, held]) => held.id === tv.id && leaseId !== lease.leaseId, + ); if (holder) { - throw host.createError('DEVICE_IN_USE', `${label} TV ${tv.name} is held by another lease`, { + throw host.createError('DEVICE_IN_USE', `${label} TV ${tv.id} is held by another lease`, { hint: 'Disconnect the session that holds the TV, or wait for its lease to expire.', }); } - if (!sessions.has(tv.name)) sessions.set(tv.name, { tv, label, logs: new Map() }); - leases.set(lease.leaseId, tv.name); - return { device: tvDevice(platform, tv.name) }; + if (!sessions.has(tv.id)) sessions.set(tv.id, { tv, label, logs: new Map() }); + const entry = { id: tv.id, name: tv.name }; + leases.set(lease.leaseId, entry); + return { device: tvDevice(platform, entry) }; } - function releaseLease(leaseId: string) { - const name = leases.get(leaseId); - if (!name) return false; + async function releaseLease(leaseId: string) { + const leased = leases.get(leaseId); + if (!leased) return false; leases.delete(leaseId); - const session = sessions.get(name); - if (session && ![...leases.values()].includes(name)) { + const session = sessions.get(leased.id); + if (session && ![...leases.values()].some((held) => held.id === leased.id)) { session.released = true; detachPage(session); - sessions.delete(name); + sessions.delete(leased.id); + await finishTeardown(session); } return true; } async function release(lease: DeviceLease) { - return releaseLease(lease.leaseId) ? {} : undefined; + return (await releaseLease(lease.leaseId)) ? {} : undefined; } const runtime = { @@ -226,13 +241,14 @@ export function createTvRuntime(platform: TvPlatform, host: TvRuntimeHost) { leaseLifecycle: { allocate, release }, deviceInventoryProvider: async (request: { leaseProvider?: string; leaseId?: string }) => { if (request.leaseProvider !== provider || !request.leaseId) return null; - const name = leases.get(request.leaseId); - return name ? [tvDevice(platform, name)] : null; + const leased = leases.get(request.leaseId); + return leased ? [tvDevice(platform, leased)] : null; }, ownsDevice, getInteractor, shutdown: async () => { - for (const leaseId of leases.keys()) releaseLease(leaseId); + shuttingDown = true; + await Promise.allSettled([...leases.keys()].map(releaseLease)); }, }; @@ -240,7 +256,7 @@ export function createTvRuntime(platform: TvPlatform, host: TvRuntimeHost) { platform, owner, ownsDevice, - tvName, + tvId, getInteractor, sessionFor, }; @@ -257,13 +273,13 @@ type TvRuntimeContext = Readonly<{ platform: TvPlatform; owner: Owner; ownsDevice(device: DeviceInfo): boolean; - tvName(device: DeviceInfo): string; + tvId(device: DeviceInfo): string; getInteractor(device: DeviceInfo, signal?: AbortSignal): Interactor | undefined; sessionFor(device: DeviceInfo): TvSession | undefined; }>; function createTvPlatformRuntime( - { platform, owner, ownsDevice, tvName, getInteractor, sessionFor }: TvRuntimeContext, + { platform, owner, ownsDevice, tvId, getInteractor, sessionFor }: TvRuntimeContext, host: PlatformRuntimeHost, ): PlatformRuntimeOwner { const { provider, label } = platform; @@ -289,7 +305,7 @@ function createTvPlatformRuntime( if (!session) { throw new AppError( 'COMMAND_FAILED', - `${label} TV ${tvName(device)} has no active session`, + `${label} TV ${tvId(device)} has no active session`, { hint: `Run connect ${provider}, then open .`, }, @@ -318,11 +334,11 @@ function createTvPlatformRuntime( }, describe: (_reader, target) => ({ transport, - tv: tvName(device), + tv: tvId(device), appBundleId: target.appBundleId, outputPath: target.outputPath, }), - matchesDevice: (descriptor) => descriptor.tv === tvName(device), + matchesDevice: (descriptor) => descriptor.tv === tvId(device), reconnect: async (descriptor) => { const session = sessionFor(device); return session?.page && session.appId === descriptor.appBundleId @@ -336,7 +352,7 @@ function createTvPlatformRuntime( binding: bindProviderApplicationLifecycleInteractor({ device, signal, resolveInteractor }), }); const appOperations: Pick = { - listApps: async () => await requireSession().tv.listApps(), + listApps: async () => await requireSession().tv.listApps({ signal }), appState: async () => ({ package: await visibleAttachedApp(requireSession()) }), }; return Object.freeze({ diff --git a/packages/tv-cdp/src/tv.ts b/packages/tv-cdp/src/tv.ts index c2ed8f70df..4052bd66a7 100644 --- a/packages/tv-cdp/src/tv.ts +++ b/packages/tv-cdp/src/tv.ts @@ -6,18 +6,24 @@ export type PageTunnel = Readonly<{ target: CdpTarget; /** Runs when the tunnel ends, including when the TV sleeps or the network drops. */ onClose(listener: () => void): void; - close(): void; + /** Resolves once the vendor side of the tunnel is gone, and rejects when it could not be removed. */ + close(): Promise; }>; export type InstalledApp = Readonly<{ id: string; name: string }>; +type Cancellable = Readonly<{ signal?: AbortSignal }>; + /** One TV as its vendor tooling reaches it. */ export type TvConnection = Readonly<{ + /** The one identity of the physical TV, the same for every alias that reaches it. */ + id: string; + /** The name the lease asked for, which the daemon selects the device by. */ name: string; /** Starts the app and opens a tunnel to its page target. */ - launch(appId: string, options: { relaunch: boolean; signal?: AbortSignal }): Promise; - close(appId: string): Promise; - listApps(): Promise; + launch(appId: string, options: Cancellable & { relaunch: boolean }): Promise; + close(appId: string, options?: Cancellable): Promise; + listApps(options?: Cancellable): Promise; }>; export type TvPlatform = Readonly<{ @@ -36,5 +42,7 @@ export type TvSession = { appId?: string; released?: boolean; tunnel?: PageTunnel; + /** Resolves to the first tunnel removal that failed since the last close or release. */ + teardown?: Promise; page?: CdpSession; }; diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 56f982c475..82d3229a93 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -133,6 +133,9 @@ importers: '@agent-device/testmu': specifier: workspace:* version: link:packages/provider-testmu + '@agent-device/tizen': + specifier: workspace:* + version: link:packages/provider-tizen '@agent-device/tv-cdp': specifier: workspace:* version: link:packages/tv-cdp @@ -907,6 +910,24 @@ importers: specifier: workspace:* version: link:../.. + packages/provider-tizen: + devDependencies: + '@agent-device/host-kit': + specifier: workspace:* + version: link:../host-kit + '@agent-device/kernel': + specifier: workspace:* + version: link:../kernel + '@agent-device/test-kit': + specifier: workspace:* + version: link:../test-kit + '@agent-device/tv-cdp': + specifier: workspace:* + version: link:../tv-cdp + agent-device: + specifier: workspace:* + version: link:../.. + packages/provider-webdriver: dependencies: '@agent-device/capture-kit': diff --git a/scripts/check-provider-plugin.mjs b/scripts/check-provider-plugin.mjs index db02a751c7..513aecad63 100644 --- a/scripts/check-provider-plugin.mjs +++ b/scripts/check-provider-plugin.mjs @@ -89,6 +89,7 @@ try { ...fixture.environment?.(endpoint), AGENT_DEVICE_HOME: home, AGENT_DEVICE_NO_UPDATE_NOTIFIER: '1', + AGENT_DEVICE_SESSION: 'provider-smoke', }; const preload = path.join(scratch, 'fetch-fixture.mjs'); await fs.writeFile( diff --git a/scripts/layering/model.ts b/scripts/layering/model.ts index 4dd816b966..184f2decdf 100644 --- a/scripts/layering/model.ts +++ b/scripts/layering/model.ts @@ -125,6 +125,7 @@ export const UNRANKED_ZONES: ReadonlySet = new Set([ 'provider-revyl', 'provider-testmu', 'provider-webos', + 'provider-tizen', 'tv-cdp', 'proxy', 'react-native', diff --git a/src/commands/provider-plugin-catalog.ts b/src/commands/provider-plugin-catalog.ts index 8d5dfb5162..03e8a282e5 100644 --- a/src/commands/provider-plugin-catalog.ts +++ b/src/commands/provider-plugin-catalog.ts @@ -8,6 +8,7 @@ const PROVIDER_PLUGIN_CATALOG: readonly ProviderPluginDeclaration[] = [ { provider: 'revyl', package: '@agent-device/revyl', label: 'Revyl' }, { provider: 'testmu', package: '@agent-device/testmu', label: 'TestMu AI' }, { provider: 'webos', package: '@agent-device/webos', label: 'LG webOS' }, + { provider: 'tizen', package: '@agent-device/tizen', label: 'Samsung Tizen' }, ]; export function getProviderPluginDeclaration( diff --git a/src/commands/schema/cli-help-tv.ts b/src/commands/schema/cli-help-tv.ts index eee1aaf45f..4191358e59 100644 --- a/src/commands/schema/cli-help-tv.ts +++ b/src/commands/schema/cli-help-tv.ts @@ -1,6 +1,7 @@ -export const TV_PLUGIN_FLOWS = `LG webOS TV flow (web apps, TV in Developer Mode): +export const TV_PLUGIN_FLOWS = `LG webOS and Samsung Tizen TV flows (web apps, TV in Developer Mode): export AGENT_DEVICE_SESSION=checkout-tv agent-device connect webos --device living-room agent-device open com.example.app - Add the plugin with plugins add @agent-device/webos. After open, drive the app with tv-remote, snapshot, close and disconnect. Prerequisites: the webOS CLI (npm install -g @webos-tools/cli) and the TV in Developer Mode, registered with ares-setup-device. --device names an ares-setup-device entry; connect does not contact the TV, open reads it. The TV enters as an Android TV device with id webos:. The page is driven over the Chrome DevTools Protocol through the ares-inspect tunnel. + Add the plugin with plugins add @agent-device/webos. After open, drive the app with tv-remote, snapshot, close and disconnect. Prerequisites: the webOS CLI (npm install -g @webos-tools/cli) and the TV in Developer Mode, registered with ares-setup-device. --device names an ares-setup-device entry; connect does not contact the TV, open reads it. The TV enters as an Android TV device with id webos:: from that entry, or webos: for a USB entry. The page is driven over the Chrome DevTools Protocol through the ares-inspect tunnel. + Samsung Tizen: agent-device connect tizen --device 192.0.2.10, with the plugin from plugins add @agent-device/tizen, then the same commands. Prerequisites: sdb from the VS Code Tizen extension's SDK, the TV in Developer Mode with this host as its developer IP, and the app installed from a debug-enabled package. --device takes the TV IP or an sdb serial; connect does not contact the TV, open runs sdb connect when needed. The TV enters as an Android TV device with id tizen:. open kills the app and debug-launches it, then drives the page over the Chrome DevTools Protocol through an sdb forward. `; diff --git a/tsconfig.typecheck.json b/tsconfig.typecheck.json index f6d45eb9d2..8380463560 100644 --- a/tsconfig.typecheck.json +++ b/tsconfig.typecheck.json @@ -33,6 +33,7 @@ { "path": "packages/provider-revyl" }, { "path": "packages/provider-testmu" }, { "path": "packages/provider-webos" }, + { "path": "packages/provider-tizen" }, { "path": "packages/provider-webdriver" }, { "path": "packages/provision-kit" }, { "path": "packages/proxy" }, diff --git a/website/astro.config.mjs b/website/astro.config.mjs index 64dc526986..02b62b24d5 100644 --- a/website/astro.config.mjs +++ b/website/astro.config.mjs @@ -115,6 +115,7 @@ export default defineConfig({ 'limrun', 'revyl', 'webos', + 'tizen', ]), { label: 'Reference', diff --git a/website/docs/docs/commands.md b/website/docs/docs/commands.md index 2d0864ea38..b98c67c6f1 100644 --- a/website/docs/docs/commands.md +++ b/website/docs/docs/commands.md @@ -325,6 +325,7 @@ vega virtual-device stop - Vega OS: use `--platform vega --target tv` with Vega component IDs such as `com.example.app.main`, and `--serial VirtualDevice` to select the VVD explicitly. Commands go through the SDK-matched Vega CLI and VDA. The VVD supports app open/close, `back`, `home`, and the eight navigation buttons listed above, with exact holds. - Some Fire TV devices ship with Vega OS instead of Fire OS. Vega OS is a different OS and is supported only in the Vega Virtual Device: app inventory, snapshot, screenshot, selectors, install, touch/text/gesture, logs, and performance report unsupported for Vega targets, and physical Vega OS devices are not supported. Use the VVD display as visual truth. - LG webOS TVs run through the optional `@agent-device/webos` plugin, which drives web apps over the Chrome DevTools Protocol. See [LG webOS](/docs/webos). +- Samsung Tizen TVs run through the optional `@agent-device/tizen` plugin, which drives web apps over the Chrome DevTools Protocol. See [Samsung Tizen](/docs/tizen). - On Android TV and tvOS, use `screenshot --overlay-refs` for visual focus evidence, especially when focus metadata is missing or transient. ## Desktop targets diff --git a/website/docs/docs/plugins.md b/website/docs/docs/plugins.md index 89d000d290..04983ea1d8 100644 --- a/website/docs/docs/plugins.md +++ b/website/docs/docs/plugins.md @@ -2,7 +2,7 @@ title: Provider plugins --- -Provider plugins add device providers that agent-device does not bundle, such as [TestMu AI](/docs/testmu) and [LG webOS](/docs/webos). Installing one requires npm on your host: +Provider plugins add device providers that agent-device does not bundle, such as [TestMu AI](/docs/testmu), [LG webOS](/docs/webos), and [Samsung Tizen](/docs/tizen). Installing one requires npm on your host: ```bash agent-device plugins add @example/agent-device-provider diff --git a/website/docs/docs/tizen.md b/website/docs/docs/tizen.md new file mode 100644 index 0000000000..63861eb8db --- /dev/null +++ b/website/docs/docs/tizen.md @@ -0,0 +1,109 @@ +--- +title: Samsung Tizen +description: Drive web apps on Samsung Tizen TVs with agent-device through sdb and the Chrome DevTools Protocol. +--- + +Use the `@agent-device/tizen` plugin to drive web apps on Samsung Tizen TVs. The plugin +debug-launches the app over `sdb`, forwards its DevTools port, and drives the page over the Chrome +DevTools Protocol (CDP): remote keys, pointer, text, snapshots, screenshots, and console logs. + +The plugin is experimental. The commands it admits are listed in +[How commands map to the TV](#how-commands-map-to-the-tv). + +## Requirements + +- `sdb` from the Tizen SDK of the VS Code Tizen extension, set up with its Set Up step. The plugin + finds it in the SDK that `$TIZEN_SDK` or `~/.tizen.path.config` names. `sdb` is not on `PATH`; + to point at it directly, set the `sdbPath` plugin option. +- The TV in Developer Mode with this host's IP set as the developer IP. To set this up, follow + Samsung's + [Connecting the TV and SDK](https://developer.samsung.com/smarttv/develop/getting-started/using-sdk/tv-device.html). +- The app installed on the TV from a debug-enabled package. +- The TV and this host on the same network. + +## Install the plugin + +```bash +agent-device plugins add @agent-device/tizen +``` + +The plugin is an optional npm package installed under `AGENT_DEVICE_HOME`. +After you add or update it, close open sessions and run `agent-device daemon stop` +with the state directory you use; the next device command loads the plugin. + +Plugin options, set under `plugins["@agent-device/tizen"].options` in `config.json`: + +| Option | Meaning | +| --- | --- | +| `sdbPath` | Path to `sdb`, when the Tizen SDK is not found. | +| `device` | TV IP or sdb serial to drive when `connect` names none. Defaults to the only TV connected to `sdb`. | + +## Connect and run a session + +```bash +export AGENT_DEVICE_SESSION=checkout-tv +agent-device connect tizen --device 192.0.2.10 +agent-device open com.example.app +agent-device wait text "Home" +agent-device tv-remote press down +agent-device snapshot -i +agent-device screenshot ./tv.png +agent-device logs start +agent-device close com.example.app +agent-device disconnect +``` + +The TV is a leased device, so every command from `connect` to `disconnect` names its session with +`AGENT_DEVICE_SESSION`, `--session `, or a config session. Without one, the command fails with +`REMOTE_SESSION_REQUIRED`. + +`--device` takes the TV IP or an sdb serial. When the TV is not connected to `sdb` yet, the +plugin runs `sdb connect :26101`. Without `--device`, the plugin uses the `device` option, +then the only TV connected to `sdb`. `connect` does not contact the TV; `open` connects it. + +agent-device leases only iOS, Android, and HarmonyOS devices, so a Tizen TV enters as an Android +TV device that the plugin owns (`platform: android`, `target: tv`, id `tizen:`). +Every command on that device goes to the plugin, never to adb. + +## How commands map to the TV + +| Command | How | +| --- | --- | +| `open` / `close` | `sdb shell 0 was_kill`, `sdb shell 0 debug`, and `sdb forward`, then CDP / `sdb shell 0 was_kill` | +| `tv-remote press up/down/left/right/select/back`, `back` | `Input.dispatchKeyEvent` with the Samsung remote codes | +| `appstate` | The attached app while its page is visible | +| `apps` | `sdb shell 0 vd_applist` | +| `logs` | Console calls, exceptions, and browser log entries of the attached page, one line each | +| `snapshot`, `find`, `is`, `get`, `wait` | A DOM walk over `Runtime.evaluate`: visible interactive, labelled, or text nodes, with focus from `document.activeElement`. `-s`, `-d`, and `-i` apply to the walk. | +| `screenshot` | `Page.captureScreenshot` | +| `click`, `press `, `longpress`, `hover` | `Input.dispatchMouseEvent` | +| `fill` | Pointer press on the field, select its contents, then `Input.insertText`, or Backspace for an empty value | +| `type` | `Input.insertText` into the focused element | +| `scroll` | `mouseWheel` at the centre of the scroll container or the viewport, by the requested distance | + +Not supported yet: system buttons (`home`, menu, volume, channel, and media keys), `network dump`, +`perf`, `install` and `uninstall`, recording, URLs and deep links in `open`, and `power`. + +## Known limitations + +- `open` is always a cold start: `sdb shell 0 debug` cannot attach to a running app, so `open` + kills it first. `open` can return before the app leaves its splash screen; wait for app content + before the first snapshot. +- Idle leases expire after 60 seconds by default, and expiry ends the session. When the lease + ends, by `disconnect` or by expiry, the plugin closes the page connection and removes the `sdb` + forward. The next command then fails with `Lease is not active` or `No active session` and + needs a new `connect` and `open`. To keep longer pauses alive, run `agent-device daemon stop`, + then start the daemon with `AGENT_DEVICE_LEASE_TTL_MS` and `AGENT_DEVICE_LEASE_MAX_TTL_MS` set, + for example `1800000`. +- `screenshot` captures the app page, not the TV screen. When the app is in the background, the + page renders no frames, so `screenshot` fails; `open ` brings the app back. While the + on-screen keyboard is open, the page viewport and the screenshot are shorter. System layers such + as the Samsung menu never appear in `screenshot` or `snapshot`. +- `appstate` knows only the attached app. The TV reports no foreground app that the plugin can + read. +- Console messages logged before the attach are replayed without object previews. +- A second DevTools client on the same page can hang the session. Close Chrome DevTools and other + inspector clients attached to the app first. +- Snapshots are read from the DOM. Labels come from `aria-label`, `alt`, `title`, or own text. + Rects are CSS pixels, which match screenshot pixels only at `devicePixelRatio` 1. The walk does + not enter iframes or shadow roots. diff --git a/website/docs/docs/webos.md b/website/docs/docs/webos.md index b33235a9f8..713ffdf9e1 100644 --- a/website/docs/docs/webos.md +++ b/website/docs/docs/webos.md @@ -61,8 +61,9 @@ then the ares default device. `connect` does not contact the TV; the plugin read `ares-setup-device` when `open` starts the session. agent-device leases only iOS, Android, and HarmonyOS devices, so a webOS TV enters as an Android -TV device that the plugin owns (`platform: android`, `target: tv`, id `webos:`). -Every command on that device goes to the plugin, never to adb. +TV device that the plugin owns (`platform: android`, `target: tv`, id `webos::`, +or `webos:` for a USB entry). Every command on that device goes to the +plugin, never to adb. ## How commands map to the TV