Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 96 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,102 @@ broker, template, warm pool, and `SANDBOX_ENV_*` values in the LexVoice reposito
This frontend repository only runs the Next.js UI. It does not create, release,
or warm sandbox sessions.

### Generic control and Orin deployment boundary

The current Generic session path is:

```text
Browser
-> agent-starter-react
-> LiveKit Room
-> Agent Dispatch
-> LexVoice Generic Agent
-> lex-reflex /start
-> LiveKit room_audio and room_video_raw
-> LexVoice Video Processor
-> LiveKit room_video
-> Browser
```

The Browser displays the UI, starts and stops the session through the existing
Next.js session flow, connects to LiveKit, and uses the existing Agent Dispatch.
Neither the Browser nor Next.js knows the Jetson IP, connects to Jetson port
`8013`, receives a Jetson heartbeat, or maintains an Endpoint Lease. Do not add
a public Jetson address variable such as `NEXT_PUBLIC_JETSON_IP`, and do not put
the Jetson IP in Next.js runtime configuration or page responses.

The Jetson address remains in the existing LexVoice Generic environment as
`EDGE_MEDIA_URL`. The LexVoice Generic Agent owns calls to lex-reflex `/start`
and `/stop`. lex-reflex publishes the raw `room_audio` and `room_video_raw`
tracks; the LexVoice Video Processor consumes the raw video and publishes
`room_video` for the Browser. Device registration, endpoint discovery, and
Endpoint Lease design are deferred until the cloud platform is integrated. The
archived `codex/endpoint-connectivity-probe` PR remains a reference for that
future governance work, not part of the current runtime architecture.

#### Open the cloud frontend from Orin

An Orin Firefox or Chromium browser can open the frontend through a private
cloud address such as `http://10.2.77.108:4003`. This only establishes Browser
access to the cloud UI; it does not require lex-reflex to know the UI address,
Jetson-to-Next.js heartbeat or IP reporting, or Browser access to lex-reflex.

Bind Next.js to all cloud-side interfaces rather than only localhost:

```bash
# Development
pnpm dev --hostname 0.0.0.0 --port 4003

# Production
pnpm build
pnpm start --hostname 0.0.0.0 --port 4003
```

Allow the Orin private network to reach cloud TCP port `4003`. The Orin must
also be able to reach the configured LiveKit address and its required WSS, TCP,
and UDP ports.

Verify the cloud listener locally:

```bash
curl --noproxy '*' --connect-timeout 5 -I http://127.0.0.1:4003/
```

Verify the route, port, and home page from Orin:

```bash
ip route get 10.2.77.108
nc -vz 10.2.77.108 4003
curl --noproxy '*' --connect-timeout 5 -I http://10.2.77.108:4003/
```

The local and Orin HTTP checks should return `200`. Also request a JavaScript
or CSS asset that actually appears in the returned HTML; this confirms that the
page is not the only reachable resource:

```bash
FRONTEND_ORIGIN=http://10.2.77.108:4003
FRONTEND_HTML="$(curl --noproxy '*' --connect-timeout 5 --fail --silent --show-error "$FRONTEND_ORIGIN/")"
FRONTEND_ASSET="$(printf '%s' "$FRONTEND_HTML" | grep -Eo '/_next/static/[^" ]+\.(js|css)' | head -n 1)"
test -n "$FRONTEND_ASSET"
curl --noproxy '*' --connect-timeout 5 --fail --silent --show-error \
--dump-header - --output /dev/null "$FRONTEND_ORIGIN$FRONTEND_ASSET"
```

The asset request should return `200` with a Content-Type matching the selected
JavaScript or CSS resource.

For manual acceptance, open `http://10.2.77.108:4003` in Orin Firefox or
Chromium and confirm the complete page, JavaScript, CSS, Start and Stop controls
load without a blank screen or indefinite loading state. Start must join the
LiveKit Room and dispatch the LexVoice Generic Agent; LexVoice then starts
lex-reflex, which publishes `room_audio` and `room_video_raw`, and the Video
Processor publishes `room_video`. Stop must clean up the Agent session, cause
LexVoice to stop lex-reflex, release the media devices, and leave the Browser
ready to start again. In browser developer tools, confirm there is no Jetson IP
input and no request to `10.2.2.199:8013`. Next.js must not store the Jetson IP,
and Jetson must not send a heartbeat to Next.js.

For standalone frontend development, install dependencies and run the dev
server directly:

Expand Down
17 changes: 17 additions & 0 deletions components/app/session-provider.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { APP_CONFIG_DEFAULTS, type AppConfig } from '@/app-config';
import type { BrowserSourceClient } from '@/hooks/useBrowserSourceClient';
import { useRoom } from '@/hooks/useRoom';
import { SelectedVideoTrackProvider } from '@/hooks/useSelectedVideoTrack';
import { ensureBrowserRandomUuid } from '@/lib/browser-runtime-compat';

const DEFAULT_BROWSER_SOURCE_CLIENT: BrowserSourceClient = {
enabled: false,
Expand Down Expand Up @@ -43,6 +44,22 @@ interface SessionProviderProps {
}

export const SessionProvider = ({ appConfig, children }: SessionProviderProps) => {
const compatibility = ensureBrowserRandomUuid();
if (!compatibility.ok) {
return (
<main className="grid min-h-svh place-content-center p-6">
<div role="alert" className="max-w-lg space-y-2 text-center">
<h1 className="text-xl font-semibold">Browser compatibility error</h1>
<p>{compatibility.message}</p>
</div>
</main>
);
}

return <CompatibleSessionProvider appConfig={appConfig}>{children}</CompatibleSessionProvider>;
};

const CompatibleSessionProvider = ({ appConfig, children }: SessionProviderProps) => {
const {
room,
isSessionActive,
Expand Down
23 changes: 22 additions & 1 deletion lib/browser-room-session.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ let fallbackSessionId: string | null = null;

export function getVoiceSessionId(
storage: Pick<Storage, 'getItem' | 'setItem'> | null | undefined = getSessionStorage(),
createSessionId: () => string = () => crypto.randomUUID()
createSessionId: () => string = createBrowserRandomUuid
) {
if (storage) {
try {
Expand Down Expand Up @@ -50,6 +50,27 @@ export function resetVoiceSessionId(
export const getBrowserRoomSessionId = getVoiceSessionId;
export const resetBrowserRoomSessionId = resetVoiceSessionId;

export function createBrowserRandomUuid(
cryptoProvider: Pick<Crypto, 'getRandomValues'> & Partial<Pick<Crypto, 'randomUUID'>> = crypto
) {
if (typeof cryptoProvider.randomUUID === 'function') {
return cryptoProvider.randomUUID();
}

return createBrowserRandomUuidFromRandomValues(cryptoProvider);
}

export function createBrowserRandomUuidFromRandomValues(
cryptoProvider: Pick<Crypto, 'getRandomValues'>
) {
const bytes = cryptoProvider.getRandomValues(new Uint8Array(16));
bytes[6] = (bytes[6] & 0x0f) | 0x40;
bytes[8] = (bytes[8] & 0x3f) | 0x80;

const hex = Array.from(bytes, (byte) => byte.toString(16).padStart(2, '0'));
return `${hex.slice(0, 4).join('')}-${hex.slice(4, 6).join('')}-${hex.slice(6, 8).join('')}-${hex.slice(8, 10).join('')}-${hex.slice(10).join('')}`;
}

function getSessionStorage() {
if (typeof window === 'undefined') {
return null;
Expand Down
45 changes: 45 additions & 0 deletions lib/browser-runtime-compat.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
import { createBrowserRandomUuidFromRandomValues } from './browser-room-session';

type BrowserCryptoProvider = Pick<Crypto, 'getRandomValues'> & Partial<Pick<Crypto, 'randomUUID'>>;

export type BrowserRandomUuidStatus =
| { ok: true; installed: boolean }
| { ok: false; message: string };

export function ensureBrowserRandomUuid(
cryptoProvider: BrowserCryptoProvider | undefined = globalThis.crypto
): BrowserRandomUuidStatus {
if (!cryptoProvider || typeof cryptoProvider.getRandomValues !== 'function') {
return {
ok: false,
message: 'This browser does not provide the secure random values required by randomUUID.',
};
}

if (typeof cryptoProvider.randomUUID === 'function') {
return { ok: true, installed: false };
}

try {
Object.defineProperty(cryptoProvider, 'randomUUID', {
configurable: true,
enumerable: false,
writable: false,
value: () => createBrowserRandomUuidFromRandomValues(cryptoProvider),
});
} catch {
return {
ok: false,
message: 'This browser could not install the required randomUUID compatibility support.',
};
}

if (typeof cryptoProvider.randomUUID !== 'function') {
return {
ok: false,
message: 'This browser could not install the required randomUUID compatibility support.',
};
}

return { ok: true, installed: true };
}
68 changes: 68 additions & 0 deletions tests/browser-room-session.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,74 @@ function createMemoryStorage() {
};
}

async function withCryptoProvider(provider, callback) {
const descriptor = Object.getOwnPropertyDescriptor(globalThis, 'crypto');
Object.defineProperty(globalThis, 'crypto', {
configurable: true,
value: provider,
});
try {
return await callback();
} finally {
if (descriptor) {
Object.defineProperty(globalThis, 'crypto', descriptor);
} else {
delete globalThis.crypto;
}
}
}

test('creates an RFC 4122 v4 browser room id when native randomUUID is unavailable', async () => {
const { createBrowserRandomUuid, isValidConnectionRoomId } = await loadBrowserRoomSessionModule();

const sessionId = await withCryptoProvider(
{
getRandomValues(bytes) {
bytes.set(Array.from({ length: 16 }, (_, index) => index));
return bytes;
},
},
() => createBrowserRandomUuid()
);

assert.equal(sessionId, '00010203-0405-4607-8809-0a0b0c0d0e0f');
assert.equal(sessionId[14], '4');
assert.match(sessionId[19], /[89ab]/);
assert.equal(isValidConnectionRoomId(sessionId), true);
});

test('prefers native randomUUID for a new browser room id', async () => {
const { createBrowserRandomUuid } = await loadBrowserRoomSessionModule();
const nativeSessionId = '33333333-4444-4555-8666-777777777777';

const sessionId = await withCryptoProvider(
{
randomUUID() {
return nativeSessionId;
},
getRandomValues() {
assert.fail('getRandomValues must not run when native randomUUID is callable');
},
},
() => createBrowserRandomUuid()
);

assert.equal(sessionId, nativeSessionId);
});

test('reuses a valid stored browser room id without creating another', async () => {
const { getBrowserRoomSessionId } = await loadBrowserRoomSessionModule();
const storage = createMemoryStorage();
const storedSessionId = '44444444-5555-4666-8777-888888888888';
storage.setItem('lexvoice.session_id.v1', storedSessionId);

const sessionId = getBrowserRoomSessionId(storage, () => {
assert.fail('stored session id should be reused');
});

assert.equal(sessionId, storedSessionId);
});

test('resetting browser room session rotates the next room id', async () => {
const { getBrowserRoomSessionId, resetBrowserRoomSessionId } =
await loadBrowserRoomSessionModule();
Expand Down
67 changes: 67 additions & 0 deletions tests/browser-runtime-compat.test.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
import assert from 'node:assert/strict';
import { readFile } from 'node:fs/promises';
import test from 'node:test';
import * as browserCompat from '../lib/browser-runtime-compat.ts';

test('installs an RFC 4122 v4 randomUUID when the browser crypto API omits it', () => {
const cryptoProvider = {
getRandomValues(bytes) {
bytes.set(Array.from({ length: 16 }, (_, index) => index));
return bytes;
},
};

const result = browserCompat.ensureBrowserRandomUuid(cryptoProvider);

assert.deepEqual(result, { ok: true, installed: true });
assert.equal(cryptoProvider.randomUUID(), '00010203-0405-4607-8809-0a0b0c0d0e0f');
assert.equal(cryptoProvider.randomUUID()[14], '4');
assert.match(cryptoProvider.randomUUID()[19], /[89ab]/);
});

test('preserves the native randomUUID implementation', () => {
const nativeRandomUuid = () => '33333333-4444-4555-8666-777777777777';
const cryptoProvider = {
randomUUID: nativeRandomUuid,
getRandomValues() {
assert.fail('getRandomValues must not run when native randomUUID exists');
},
};

const result = browserCompat.ensureBrowserRandomUuid(cryptoProvider);

assert.deepEqual(result, { ok: true, installed: false });
assert.equal(cryptoProvider.randomUUID, nativeRandomUuid);
});

test('fails safely when the browser crypto object rejects the polyfill', () => {
const cryptoProvider = Object.preventExtensions({
getRandomValues(bytes) {
return bytes;
},
});

const result = browserCompat.ensureBrowserRandomUuid(cryptoProvider);

assert.equal(result.ok, false);
assert.match(result.message, /randomUUID/i);
assert.equal('randomUUID' in cryptoProvider, false);
});

test('SessionProvider exposes an explicit compatibility error before creating a Room', async () => {
const source = await readFile('components/app/session-provider.tsx', 'utf8');

assert.match(source, /ensureBrowserRandomUuid\(/);
assert.match(source, /role="alert"/);
assert.match(source, /Browser compatibility error/);
});

test('AgentControlBar retains the high-level useChat send path', async () => {
const source = await readFile(
'components/livekit/agent-control-bar/agent-control-bar.tsx',
'utf8'
);

assert.match(source, /await send\(message\)/);
assert.doesNotMatch(source, /streamText|sendBrowserChatMessage/);
});
Loading