Skip to content

fix(ai): preserve Vercel AI Gateway cost and routing metadata - #63320

Open
Divyanshu-kumar14 wants to merge 3 commits into
anomalyco:devfrom
Divyanshu-kumar14:fix-gateway-cost
Open

Divyanshu-kumar14 wants to merge 3 commits into
anomalyco:devfrom
Divyanshu-kumar14:fix-gateway-cost

Conversation

@Divyanshu-kumar14

@Divyanshu-kumar14 Divyanshu-kumar14 commented Nov 11, 2026 •

Copy link
Copy Markdown

Issue for this PR

Closes #63293

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Vercel AI Gateway responses carry billed cost and routing under provider_metadata.gateway, but sessions persisted without it: gatewayProtocol attached metadata only to finish while core reads step-finish, wrote it under top-level gateway while core reads providerMetadata["vercel-ai-gateway"], and never decoded the string gateway.cost into Usage.cost, so SessionUsage.record fell back to catalog pricing. The fix tracks the request's providerMetadataKey, strictly parses non-negative finite cost values (empty, whitespace, negative, and non-finite values still fall back to catalog pricing), and emits merged metadata on both step-finish and finish. A separate commit redacts assistant.native in sanitized exports because it now persists Gateway routing identifiers; it can be dropped independently.

How did you verify your code works?

Reproduced against a mock /v1/messages endpoint returning string cost: "0.0207155": opencode session export now records the billed cost and native.gateway, and --sanitize redacts it. The core Gateway suite covers the Messages alias and the empty-cost catalog fallback end-to-end, while protocol-level cases live in the AI provider suite. @opencode/ai (2,098 pass / 28 skip / 0 fail), core gateway and session suites (520 pass / 0 fail across 10 randomized runs), bun run check (35/35), bun run typecheck:tsc (33/33), and oxlint are clean.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Added by the opencode bot (AI-generated).

Related: #60309

Public API: step-finish events from gateway routes now carry gateway metadata and usage.cost, assistant messages store native.gateway, and sanitized exports redact assistant native (backwards compatible)

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix works. I ran the #63293 reproduction from source against a mock gateway. On the base, the assistant message stores the catalog estimate (0.0021) and no gateway metadata. With this pull request it stores the billed 0.0207155 and native.gateway with routing.finalProvider and generationId. --sanitize redacts it. The ai and core tests closest to the change pass, and tsc --noEmit reports no errors in packages/ai or in the changed core files.

The approach is sound:

  • Core's native() keeps only providerMetadata[providerMetadataKey], so writing the metadata under both keys stores just one copy, and top-level gateway keeps working for existing @opencode/ai callers.
  • Treating a gateway cost of "0" as a real free request matches the Usage.cost docs.
  • Redacting assistant native in sanitized exports matches how compaction.native is already handled.

Public API: all additive, and existing callers keep working.

  • step-finish events from gateway routes now carry providerMetadata.gateway and providerMetadata[providerMetadataKey].gateway, and usage.cost is filled from the gateway cost when the response has no numeric cost.
  • Stored assistant messages and session exports gain native.gateway.
  • session export --sanitize now redacts assistant native, which used to pass through unchanged.

One request: session-runner-vercel-gateway.test.ts is large for what it checks (see the inline comment).

Related: #60309

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commits address the earlier test-size point. The core gateway test is down to two cases, the billed Messages cost with routing and the empty-cost catalog fallback, and the runner layer setup is now a shared test/lib/session-runner-harness.ts that session-runner-recorded.test.ts also uses. The source is unchanged since the last review, so the fix still holds. The core gateway, recorded-runner and session-create tests pass, and the core test type check is clean. Nothing new to flag. The public API impact is unchanged from the earlier review.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the last review this was only rebased onto newer dev. The only change to the pull request's own commits is the claude: true field added to the instruction-discovery mock in the shared test harness, matching dev. The fix code is unchanged, the gateway tests in ai and core pass, and the core test type check is clean. Nothing new to flag. The public API impact is unchanged from the earlier review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Vercel AI Gateway billed cost and serving provider are dropped from session messages

1 participant