Add voice tracing - #48633
Conversation
Add fail-open connection and callback telemetry plus application-declared target turns. Harden propagation, transport outcome classification, and the application-owned Voice sample lifecycle.
Classify WebSocket Going Away consistently as a clean close and preserve source-aware outcomes for unfinished turns in the basic Voice sample.
|
Thank you for your contribution knit (@knitvoger)! We will review the pull request and get back to you soon. |
|
Azure Pipelines: Successfully started running 1 pipeline(s). 9 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR introduces content-free OpenTelemetry tracing/metrics for the typed Voice WebSocket relay, adds application-declared “target turn” spans for model/tool work, and tightens W3C propagation handling (allowlisted baggage, tracestate validation, and close-code semantics).
Changes:
- Added
SessionTerminationandSession.start_target_turn(...)withTargetTurn.activate()/TargetTurn.complete(...)to let apps explicitly model target-decision traces. - Implemented Voice connection/callback OpenTelemetry spans + aggregate duration/propagation-failure metrics while filtering/sanitizing propagated context.
- Updated samples and expanded test coverage around telemetry behavior, termination classification, and transport failure ordering.
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_voice_host.py | Adds W3C extraction filtering/validation, connection+callback spans, termination committing, and default observability configuration behavior. |
| sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_session.py | Adds SessionTermination, exposes Session.termination, and wires connection context + target-turn creation. |
| sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_turn.py | New TargetTurn API for application-owned target-decision tracing and duration metrics. |
| sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_tracing.py | New package-local tracing + metrics helpers (_SpanScope, duration histograms, propagation failure counter). |
| sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/init.py | Exposes new public surface: SessionTermination, TargetTurn* symbols. |
| sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/_constants.py | Adds “going away” close code constant and normal-close-code set. |
| sdk/agentserver/azure-ai-agentserver-invocations/samples/basic_voice_agent/basic_voice_agent.py | Updates sample to declare/complete target turns, cap concurrency, bound output retention, and classify outcomes. |
| sdk/agentserver/azure-ai-agentserver-invocations/samples/basic_voice_agent/README.md | Documents target-turn tracing completion and sample outcome mapping. |
| sdk/agentserver/azure-ai-agentserver-invocations/README.md | Updates Voice docs with tracing model, propagation constraints, and target-turn usage. |
| sdk/agentserver/azure-ai-agentserver-invocations/tests/voice/test_voice_transport_findings.py | Updates propagation expectations and adds tracestate size-limit validation test. |
| sdk/agentserver/azure-ai-agentserver-invocations/tests/voice/test_voice_tracing.py | New contract tests for spans/metrics, sanitization, termination semantics, and failure modes. |
| sdk/agentserver/azure-ai-agentserver-invocations/tests/voice/test_voice_session.py | Updates session slot expectations for new fields. |
| sdk/agentserver/azure-ai-agentserver-invocations/tests/voice/test_basic_voice_agent.py | Extends sample tests for outcome mapping, cancellation, and completion semantics. |
| sdk/agentserver/azure-ai-agentserver-invocations/pyproject.toml | Promotes opentelemetry-api to runtime dependency and bumps dev OTel versions. |
| sdk/agentserver/azure-ai-agentserver-invocations/dev_requirements.txt | Updates OTel API/SDK version constraints. |
| sdk/agentserver/azure-ai-agentserver-invocations/_version.py | Bumps package version to 1.1.0b2. |
| sdk/agentserver/azure-ai-agentserver-invocations/api.md | Updates API surface documentation for new tracing/termination APIs. |
| sdk/agentserver/azure-ai-agentserver-invocations/api.metadata.yml | Updates API metadata hash and recorded python version. |
| sdk/agentserver/azure-ai-agentserver-invocations/CHANGELOG.md | Adds 1.1.0b2 release notes describing new tracing/termination behavior. |
| .chronus/changes/wujin-voice-tracing-v2-2026-8-17.md | Adds chronus change entry for the feature. |
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 20 out of 20 changed files in this pull request and generated no new comments.
Suppressed comments (4)
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_turn.py:97
TargetTurn.complete(..., output_item_count: int | None = None)is typed as optional, but_validate_completioneffectively makesoutput_item_countmandatory for at leastRESPONSE(must be provided and >= 1) andNONE(must be explicitly 0;Nonecurrently fails). Either makeoutput_item_countrequired in the public signature (and docs/api.md) or adjust validation to treatoutput_item_count=Noneas0forNONEand keep the clearer error forRESPONSE.
if normalized_outcome is TargetTurnOutcome.RESPONSE:
if validated_response_id is None or validated_output_count is None or validated_output_count < 1:
raise ValueError("response outcome requires a response_id and at least one output item")
elif normalized_outcome is TargetTurnOutcome.NONE:
if validated_response_id is not None or validated_output_count != 0:
raise ValueError("none outcome requires no response_id and zero output items")
elif validated_output_count is not None and validated_output_count > 0 and validated_response_id is None:
raise ValueError("positive output_item_count requires a response_id")
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_tracing.py:136
- This relies on
token.varbeing present and stable, which is an internal detail of how the OTel context implementation currently stores its token. Prefer using the public API (otel_context.detach(token)) and store only what detach needs (the token), to reduce the chance of breakage across OpenTelemetry versions.
def _attach_context(context: Any) -> tuple[Any, Any] | None:
if context is None:
return None
previous = _current_context()
if previous is None:
return None
try:
return otel_context.attach(context), previous
except BaseException: # pylint: disable=broad-exception-caught
return None
def _reset_context(attachment: tuple[Any, Any] | None) -> None:
if attachment is None:
return
token, previous = attachment
try:
token.var.reset(token)
return
except BaseException: # pylint: disable=broad-exception-caught
pass
try:
token.var.set(previous)
except BaseException: # pylint: disable=broad-exception-caught
pass
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_voice_host.py:316
- Returning
Nonehere disables all connection/callback tracing even when only the propagator failed to construct. Consider failing open by usingbase_contextas the extracted context when_VOICE_TRACE_PROPAGATORisNone(still record the propagation failure), so local spans/metrics can remain available even if remote extraction is unavailable.
base_context = _new_voice_context()
if base_context is None or _VOICE_TRACE_PROPAGATOR is None:
_record_propagation_failure("extraction_error")
return None
sdk/agentserver/azure-ai-agentserver-invocations/tests/voice/test_voice_tracing.py:69
- This fixture mutates global OpenTelemetry state (potentially replacing the tracer provider) and never restores the prior provider, which can cause order-dependent failures when running the full test suite. Prefer saving the previous provider and restoring it in teardown (e.g., via a
yieldfixture), or isolate these tests by using monkeypatching aroundtrace.set_tracer_provider/metrics.set_meter_provider.
@pytest.fixture
def spans():
"""Capture spans without replacing a provider another test installed."""
global _PROVIDER, _EXPORTER
if _PROVIDER is None:
existing = trace.get_tracer_provider()
if hasattr(existing, "add_span_processor"):
_PROVIDER = existing
else:
_PROVIDER = TracerProvider()
trace.set_tracer_provider(_PROVIDER)
_EXPORTER = InMemorySpanExporter()
_PROVIDER.add_span_processor(SimpleSpanProcessor(_EXPORTER))
_EXPORTER.clear()
return _PROVIDER, _EXPORTER
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 20 out of 20 changed files in this pull request and generated no new comments.
Suppressed comments (6)
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_turn.py:305
Span.set_status(...)is being called withStatusCode.ERROR. In the OpenTelemetry Python API,set_statustypically expects aStatusobject (e.g.,Status(StatusCode.ERROR)) rather than the enum value. As written, this call can be ignored (due to the broad exception handler) and you’ll silently lose error status on target-turn spans. Consider constructing/passing the properStatustype so error spans are correctly marked.
span.set_status(otel_trace.StatusCode.ERROR)
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_tracing.py:271
- Same
Span.set_status(...)issue as in_turn.py: passingStatusCode.ERRORmay be rejected by the OTel API and (because exceptions are swallowed) will result in spans not being marked as errored. Use the OpenTelemetryStatusobject (or the API’s supported status type) so callback/connection spans reliably surface error status.
self._span.set_attribute("error.type", error_type)
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_tracing.py:275
- Same
Span.set_status(...)issue as in_turn.py: passingStatusCode.ERRORmay be rejected by the OTel API and (because exceptions are swallowed) will result in spans not being marked as errored. Use the OpenTelemetryStatusobject (or the API’s supported status type) so callback/connection spans reliably surface error status.
self._span.set_status(otel_trace.StatusCode.ERROR)
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_voice_host.py:332
- When multiple
traceparentheaders are present, the code flags the request asinvalidbut still passes the duplicatetraceparentvalues into the propagator. Depending on the propagator’s behavior, one of the duplicates could still be accepted, which undermines the ‘invalid’ classification and can allow header-spoofing/ambiguity. Consider sanitizingtrace_headersin thelen(traceparents) != 1case (e.g., drop alltraceparentheaders before extraction, or only keep a single canonical one while still recordinginvalid) so extraction behavior matches the security intent.
traceparents = [value for name, value in raw_headers if name.lower() == b"traceparent"]
if not traceparents:
failure = "missing"
elif len(traceparents) != 1:
failure = "invalid"
trace_headers = raw_headers
valid_tracestate = _has_valid_tracestate(raw_headers)
if not valid_tracestate:
failure = "invalid"
trace_headers = [(name, value) for name, value in raw_headers if name.lower() != b"tracestate"]
context = _VOICE_TRACE_PROPAGATOR.extract(
carrier=trace_headers,
context=base_context,
getter=_VOICE_HEADER_GETTER,
)
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_voice_host.py:164
_configure_voice_observabilityacceptsenable_sensitive_databut explicitly ignores it, always deriving the value from the environment variable. If callers (orInvocationAgentServerHost) passenable_sensitive_data=True, it will have no effect, which is surprising for an argument in a public configuration path. Consider either honoring the explicit parameter (e.g., treat it as an opt-in that overrides/env-ORs with the env var) or removing the parameter from this adapter if it’s intentionally unsupported for Voice.
def _configure_voice_observability(
*,
connection_string: str | None = None,
log_level: str | None = None,
enable_sensitive_data: bool = False,
) -> None:
del enable_sensitive_data
configured = os.environ.get("OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT", "false")
try:
_CORE_CONFIGURE_OBSERVABILITY(
connection_string=connection_string,
log_level=log_level,
enable_sensitive_data=configured.strip().lower() in {"1", "true"},
)
except BaseException: # pylint: disable=broad-exception-caught
pass
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_tracing.py:136
- Context reset is implemented via
token.var.reset(token)(and aset(previous)fallback). OpenTelemetry’s context API providesotel_context.detach(token)for detaching an attachment token, which avoids relying on internalcontextvars.Tokenshape/behavior and is the standard pairing forattach. Consider switching_reset_contextto callotel_context.detach(token)(and only falling back to the lower-level reset/set approach if needed) to reduce the risk of context leaks across versions.
def _attach_context(context: Any) -> tuple[Any, Any] | None:
if context is None:
return None
previous = _current_context()
if previous is None:
return None
try:
return otel_context.attach(context), previous
except BaseException: # pylint: disable=broad-exception-caught
return None
def _reset_context(attachment: tuple[Any, Any] | None) -> None:
if attachment is None:
return
token, previous = attachment
try:
token.var.reset(token)
return
except BaseException: # pylint: disable=broad-exception-caught
pass
try:
token.var.set(previous)
except BaseException: # pylint: disable=broad-exception-caught
pass
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 20 out of 20 changed files in this pull request and generated 2 comments.
Suppressed comments (2)
sdk/agentserver/azure-ai-agentserver-invocations/pyproject.toml:1
- The minimum OpenTelemetry versions are pinned to
>=1.43.0for bothopentelemetry-apiandopentelemetry-sdk. With my dependency knowledge cutoff (Aug 2025),1.43.0is not a known released version and could cause installation failures in environments that don’t have that release available. Consider lowering the minimum to the oldest released version that provides the required APIs, or using a bounded range aligned with the repo’s tested OpenTelemetry versions.
[project]
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_voice_host.py:171
_configure_voice_observabilitycurrently swallowsBaseException, which includesSystemExitandKeyboardInterrupt. That can interfere with process shutdown behavior and makes failures harder to diagnose. Prefer catchingExceptionhere (and optionally logging a content-free warning) so only expected runtime failures are suppressed while critical process-control exceptions still propagate.
except BaseException: # pylint: disable=broad-exception-caught
pass
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 20 out of 20 changed files in this pull request and generated no new comments.
Suppressed comments (2)
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_tracing.py:136
otel_context.attach()returns an opaque token whose internal shape is not part of the public API; accessingtoken.varrelies on OpenTelemetry internals and can break across OTel implementations/versions. Prefer storing the attach token and using the publicopentelemetry.context.detach(token)to reset, instead of manipulatingContextVarinternals.
def _attach_context(context: Any) -> tuple[Any, Any] | None:
if context is None:
return None
previous = _current_context()
if previous is None:
return None
try:
return otel_context.attach(context), previous
except BaseException: # pylint: disable=broad-exception-caught
return None
def _reset_context(attachment: tuple[Any, Any] | None) -> None:
if attachment is None:
return
token, previous = attachment
try:
token.var.reset(token)
return
except BaseException: # pylint: disable=broad-exception-caught
pass
try:
token.var.set(previous)
except BaseException: # pylint: disable=broad-exception-caught
pass
sdk/agentserver/azure-ai-agentserver-invocations/azure/ai/agentserver/invocations/voice/_tracing.py:233
- The attribute key
"azure.ai.agentserver.invocations_ws.close_code"is duplicated as a raw string here, while the codebase also definesInvocationsWSConstants.ATTR_SPAN_CLOSE_CODE. Using the shared constant (or a single module-level constant for span attribute keys) reduces the risk of drift/typos between places that set/read these attributes.
def complete_connection(self, outcome: str, close_code: int) -> None:
if self._completed:
return
self._completed = True
self._set_attributes(
{
"azure.ai.agentserver.invocations_ws.close_code": close_code,
"bridge.outcome": outcome,
}
)
Description
Adds content-free OpenTelemetry observability to the typed Voice WebSocket relay, including connection and callback spans, duration and propagation-failure metrics, and application-owned
TargetTurnspans for model and tool work. It also adds allowlisted W3C context propagation and explicit session termination outcomes.Updates Voice close-code classification, lifecycle handling, samples, documentation, and API metadata, with expanded tests covering tracing, terminal behavior, cleanup, and transport failures.
Spec: voice_live_bridge/spec.md · Design: hosted_text_agent_and_voice_live_bridge.md
All SDK Contribution checklist:
General Guidelines and Best Practices
Testing Guidelines