[rosetta-livekit-bot]

Summary

• add opt-in record: { inputDelta: true } telemetry for llm_node and provider llm_request spans
• chain each delta to the last committed generation with reconstructable attributes and span links
• share chat-item fingerprints with context equivalence, preserve GenAI message boundaries, and avoid fallback wrapper duplication
• give running-tool placeholders stable, distinct IDs

Port of livekit/agents#7605.

Validation

• npx --yes node@24 /usr/local/bin/pnpm test agents --silent (180 files, 3043 passed, 5 skipped)
• pnpm build:agents
• pnpm --filter @livekit/agents typecheck
• pnpm --filter @livekit/agents throws:check
• pnpm --filter @livekit/agents api:check
• focused ESLint and Prettier checks on the ported files

The repository-wide pnpm lint and pnpm format:check commands were also run. They remain blocked by existing formatting/lint findings outside this port (295 lint findings and 42 files reported by Prettier).

cue-cli runtime driving was not available in this environment: the binary is not installed and uvx cue-cli cannot resolve a registry package. Equivalent runtime behavior is covered by the new real AgentSession span tests, including scheduling/commit and disabled-option paths. There is no target infrastructure or behavior gap.

Source diff coverage

• livekit-agents/livekit/agents/llm/chat_context.py: adapted to agents/src/llm/chat_context.ts; TypeScript chat-item shapes share essential-field equality and media-safe fingerprints.
• livekit-agents/livekit/agents/llm/llm.py: adapted to agents/src/llm/llm.ts; provider request spans use the async-local delta scope and fallback wrapper spans omit duplicate input only in delta mode.
• livekit-agents/livekit/agents/telemetry/__init__.py: adapted to agents/src/telemetry/index.ts; exports the TypeScript inputDelta module through the telemetry barrel.
• livekit-agents/livekit/agents/telemetry/gen_ai.py: adapted to agents/src/telemetry/gen_ai.ts; canonical instruction splitting, in-place system messages, and message layout use JS GenAI types.
• livekit-agents/livekit/agents/telemetry/input_delta.py: adapted as new agents/src/telemetry/input_delta.ts; uses AsyncLocalStorage and OpenTelemetry JS span/link APIs.
• livekit-agents/livekit/agents/telemetry/trace_types.py: ported to agents/src/telemetry/trace_types.ts.
• livekit-agents/livekit/agents/voice/agent_activity.py: adapted to agents/src/voice/agent_activity.ts; each activity owns a tracker, inference tasks inherit the scope, and scheduled speech commits it.
• livekit-agents/livekit/agents/voice/agent_session.py: adapted to agents/src/voice/agent_session.ts; the public option uses idiomatic inputDelta and remains off by default.
• livekit-agents/livekit/agents/voice/generation.py: adapted to agents/src/voice/generation.ts; node spans record deltas and running-tool placeholders use stable IDs.
• tests/test_chat_ctx.py: adapted to agents/src/llm/chat_context.test.ts using Vitest.
• tests/test_input_delta.py: adapted to new agents/src/telemetry/input_delta.test.ts and agents/src/voice/input_delta_session.test.ts; tracker, reconstruction, option-off, fallback, and real session lifecycle scenarios are covered.
• tests/test_running_tool_placeholders.py: ported to agents/src/voice/running_tool_placeholders.test.ts.
• tests/test_trace_types_pii.py: ported to agents/src/telemetry/trace_types.test.ts.

No source files were not applicable. Target-only recording-option fixtures, the generated API report, and a minor changeset for @livekit/agents were updated as required adaptations.

---

Ported from livekit/agents#7605

Original PR description

Summary

At each turn, the LLM spans record all of the conversation and all of the system instructions. Thus, the size of the spans increases at each turn of a call.

This change adds an option. When you set the option, the spans record only the input that changed. The model always receives all of the input. The option changes only the telemetry.

await session.start(agent, record={"input_delta": True})

The option is off by default. When the option is off, the spans do not change.

Design

The module telemetry/input_delta.py contains the full description. This is a summary.

Terms

• Record: the list of entries that a span records. On an llm_node span, the record is lk.pii.chat_ctx, with one entry for each chat item. On an llm_request span, the record is gen_ai.input.messages, with one entry for each GenAI message.
• Instructions: the instructions message of the agent. On an llm_request span, the instructions are in gen_ai.system_instructions, not in the record. All other system messages are entries of the record, at their position. This change to gen_ai.system_instructions also applies when the option is off.
• Commit: a generation is committed when its speech is scheduled. A discarded preemptive generation is not committed.
• Parent: the span of the last committed generation that has the same span name.

Procedure for each span

1. The tracker calculates a key for each item of the record: the item ID and a fingerprint of the content.
2. It compares the keys with the keys of the parent, in order. It finds the longest shared prefix.
3. The span records only the entries after this prefix.
4. lk.input.dropped_from_base gives the number of parent entries after this prefix.
5. On an llm_request span, if the instructions are the same as in the parent, the span does not record gen_ai.system_instructions.
6. If the span shares no entries and no instructions with the parent, the span records all of the input.
7. When the generation is committed, the keys of the span replace the keys of the parent.

Rebuild

1. Make the full record of the parent (lk.input.base_span_id). Use this procedure again if the parent also has a parent.
2. Remove the last lk.input.dropped_from_base entries.
3. Add the entries of this span.

If an llm_request span with a parent has no gen_ai.system_instructions, use the instructions of the parent.

Rules

• The comparison does not use the role of an item. A system message that stays at its position is shared. The framework adds the expressive guide again at the end of each reply. Thus, the old guide is dropped, and the new guide is recorded.
• An edited, removed or moved item stops the shared prefix at that item. The span records the entries from that item. It does not record all of the input.
• In gen_ai.input.messages, consecutive tool calls of an assistant turn are one message, also when a skipped item (for example, a config update) is between them. The prefix never stops inside such a message.
• On an llm_node span, the instructions are the first entry of lk.pii.chat_ctx. Thus, a change of the instructions causes a full record of lk.pii.chat_ctx.
• The fingerprint does not include media data. It uses the image ID and the audio transcript.
• Each AgentActivity has its own tracker. After a handoff, the first span records all of the input.
• The span of the FallbackAdapter wrapper does not record the input. Only the span of the provider records the input.

Example (expressive mode)

Generation	Input sent to the model	Record of llm_node	lk.input.dropped_from_base
Turn 1	[I, u1, G]	[I, u1, G] (full)	none
Turn 2	[I, u1, a1, u2, G]	[a1, u2, G]	1
Turn 3, tool call	[I, …, u2, a2, u3, G]	[a2, u3, G]	1
Turn 3, tool reply	[I, …, u3, G, fc, fo]	[fc, fo]	0

Attributes on a delta span: lk.input.delta, lk.input.base_span_id (and a span link to the parent), lk.input.dropped_from_base.

Relation to #7539

This change replaces the delta mode of #7539. It keeps these items from #7539: content fingerprints, no input on delegating spans, and the test scenarios.

This change is different from #7539 in these items:

• Each delta span identifies its parent span and the number of dropped entries. Thus, you can make the full input again.
• Only committed generations change the baseline. Preemptive generations and other LLM calls do not change the baseline.
• The span does not record the system instructions again when they do not change.
• The option applies to one session. It does not apply to all of the process.
• lk.pii.chat_ctx stays on the span as a delta. #7539 removes it.

Refactor

Each ChatItem type now has the base class _ChatItemBase. This base class has no fields. Each type declares its essential fields one time in _essential_fields(). ChatContext.is_equivalent compares these fields, and _fingerprint() hashes them. The operation of is_equivalent does not change. New tests in tests/test_chat_ctx.py examine it.

Tests

tests/test_input_delta.py contains unit tests for the tracker. It also contains session tests: two turns with a tool step, the option off, and a fallback.

[devin-ai-integration]

Note

Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.

Devin Review found 6 potential issues.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

[changeset-bot]

🦋 Changeset detected

Latest commit: abed250

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 39 packages

Name	Type
@livekit/agents	Patch
@livekit/agents-plugin-anam	Patch
@livekit/agents-plugin-anthropic	Patch
@livekit/agents-plugin-assemblyai	Patch
@livekit/agents-plugin-azure	Patch
@livekit/agents-plugin-baseten	Patch
@livekit/agents-plugin-bey	Patch
@livekit/agents-plugin-cartesia	Patch
@livekit/agents-plugin-cerebras	Patch
@livekit/agents-plugin-deepgram	Patch
@livekit/agents-plugin-did	Patch
@livekit/agents-plugin-elevenlabs	Patch
@livekit/agents-plugin-fishaudio	Patch
@livekit/agents-plugin-google	Patch
@livekit/agents-plugin-hume	Patch
@livekit/agents-plugin-inworld	Patch
@livekit/agents-plugin-krisp	Patch
@livekit/agents-plugin-lemonslice	Patch
@livekit/agents-plugin-liveavatar	Patch
@livekit/agents-plugin-livekit	Patch
@livekit/agents-plugin-meta	Patch
@livekit/agents-plugin-minimax	Patch
@livekit/agents-plugin-mistral	Patch
@livekit/agents-plugin-mistralai	Patch
@livekit/agents-plugin-neuphonic	Patch
@livekit/agents-plugin-openai	Patch
@livekit/agents-plugin-perplexity	Patch
@livekit/agents-plugin-phonic	Patch
@livekit/agents-plugin-protoface	Patch
@livekit/agents-plugin-resemble	Patch
@livekit/agents-plugin-rime	Patch
@livekit/agents-plugin-runway	Patch
@livekit/agents-plugin-sarvam	Patch
@livekit/agents-plugin-silero	Patch
@livekit/agents-plugin-soniox	Patch
@livekit/agents-plugin-tavus	Patch
@livekit/agents-plugins-test	Patch
@livekit/agents-plugin-trugen	Patch
@livekit/agents-plugin-xai	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[devin-ai-integration]

Devin Review found 3 new potential issues.