Skip to content

Expose the voice-agent latency critical path - #674

Merged
thesyncim merged 4 commits into
acceleratefrom
codex/voice-latency-tracing
Sep 28, 2026
Merged

thesyncim merged 4 commits into
acceleratefrom
codex/voice-latency-tracing

Conversation

@thesyncim

@thesyncim thesyncim commented Sep 28, 2026 •

Copy link
Copy Markdown
Member

The existing roundtrip_ms clock started after transcript cadence settled, hiding roughly 350 ms of caller wait. It also gave no way to distinguish turn-decision time from reply generation and speech synthesis.

This PR traces the path from the last transcript revision to the first audio chunk published by the Go edge. The call timeline API exposes its stage timings and every flow, reply, and subagent model attempt with turn ID, TTFT, and full duration. Model calls annotate the stages they overlap, so their durations are not added twice. Call setup is measured separately: the Go edge logs signaling and each ICE peer connection, while Python reports join duration in its agent event and OpenTelemetry metric.

Console timing is opt-in: stream.Accelerated(log_latency=True) prints per-model lines and an ASCII DAG for each turn. It is quiet by default; the simple_voice_ai example enables it with VOICE_LATENCY_DAG=1. The router records the metrics regardless of this logging flag.

A spoken turn in the local test call printed:

voice latency DAG turn=turn-1790592989844605000 | speech end -> first audio ~3,334 ms
  [speech end]
       |
       v
  [STT 171 ms]
       |
       v
  [cadence 350 ms]
       |
       v
  [decision 1,063 ms]
       +-- flow openai/gpt-6-luna: TTFT 275 ms, full 497 ms
       |
       v
  [first speakable text 824 ms]
       +-- reply openai/gpt-6-luna: TTFT 243 ms, full 493 ms
       |
       v
  [TTS handoff 22 ms]
       |
       v
  [first audio 904 ms]

speech_end_to_audio_ms adds provider STT processing to the edge roundtrip; it excludes network transit and browser playback. The example also drops a memory filter that prevented it from starting without a memory provider. The OpenAPI change is reflected in the generated Go, Python, and JavaScript clients.

Checks: Changed Go packages, the Go SDK, all 29 accelerated-agent tests, JavaScript typecheck, Swift client audit, and git diff --check pass. The accelerate CI baseline was restored and merged in #676. Full CI is running on this rebased branch.

@thesyncim
thesyncim force-pushed the codex/voice-latency-tracing branch from 65cd601 to 92f6125 Compare September 28, 2026 10:47
@thesyncim thesyncim changed the title Trace call connection and voice turn latency Expose the voice-agent latency critical path Sep 28, 2026
@thesyncim
thesyncim force-pushed the codex/voice-latency-tracing branch from ffa719c to 20b6337 Compare September 28, 2026 11:51
@thesyncim
thesyncim merged commit 85b8039 into accelerate Sep 28, 2026
12 checks passed
@thesyncim
thesyncim deleted the codex/voice-latency-tracing branch September 28, 2026 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants