Vercel AI: reasoning tokens never reach `invoke_agent` parents, and streamed parents report candidate-only output

[RulaKhaled]

Follow-up to #23433, which fixes Gemini reasoning tokens on model-call (gen_ai.generate_content) spans. Two gaps are deliberately out of scope there and need a decision.

Background

Per the conventions, gen_ai.usage.output_tokens is reasoning-inclusive and gen_ai.usage.reasoning.output_tokens is a subset of it. On ai v4/v5 with @ai-sdk/google, the SDK's outputTokens is candidate-only — reasoning ("thoughts") is exposed only through providerMetadata.google.usageMetadata. #23433 recomputes output on model-call spans from candidatesTokenCount + thoughtsTokenCount.

It cannot do the same on gen_ai.invoke_agent parents: their ai.usage.* is totalUsage summed across every step, while ai.response.providerMetadata describes the last step only. So #23433 gates output_tokens, total_tokens and reasoning.output_tokens off parents entirely.

Gap 1 — streamed parents report candidate-only output

With spanStreamingIntegration enabled, each span is emitted independently, so there is no pass that can re-derive a parent from its children. Measured on v10 with #23433 applied:

parent	static (event processor)	streamed
single-step	output 101, total 115 ✅	output 1, total 15 ⚠️
multi-step	output 201, total 1101 ✅	output 21, total 921 ⚠️

The static path is correct because applyAccumulatedTokens overwrites the parent by summing its (already corrected) children. The streamed path has no equivalent.

Gap 2 — reasoning.output_tokens never reaches parents

accumulateTokensForParent / applyAccumulatedTokens sum input and output but never reasoning, so unlike output/total there is no fallback once it is gated. A parent therefore carries no reasoning count on either path. Writing the ungated value would be wrong rather than merely absent: on a two-step call it reports the last step's reasoning as the whole call's (e.g. 100 where the true aggregate is 80 + 100 = 180).

Options to investigate

1. Sum reasoning in accumulateTokensForParent / applyAccumulatedTokens. Gives parents a true aggregate reasoning count and closes Gap 2 on the static path. Does not help the streamed path, which has no accumulation. Touches shared token plumbing that every provider flows through, so it needs its own testing.

2. Read the SDK's own reasoning tokens instead of deriving them from providerMetadata. ai v6+ exposes ai.usage.outputTokenDetails.reasoningTokens (and the deprecated ai.usage.reasoningTokens) directly on the span, sourced from usage.outputTokens.reasoning, and v6 outputTokens is already total = text + reasoning. Nothing in the SDK reads either today. This is provider-agnostic, works on both paths, and would cover Gemini, OpenAI and Anthropic at once — but only for SDK versions that emit it.

Also worth folding in

• On v10, the OpenAI branch of getProviderMetadataAttributes writes gen_ai.usage.output_tokens.reasoning, which @sentry/conventions marks deprecated in favour of gen_ai.usage.reasoning.output_tokens. It has the same last-step-only issue on parents.
• On develop, the Vercel AI OTel path was removed in ref(server-utils): Remove addVercelAiProcessors and Vercel AI OTel span processing #23384. enrichSpanOnEnd in vercel-ai-dc-subscriber.ts reads input/output/total but never outputTokenDetails.reasoningTokens, so gen_ai.usage.reasoning.output_tokens is currently only ever populated for OpenAI. Option 2 likely applies there more cleanly than on v10.

[linear-code]

JS-3557