Repository navigation
fix(ai): preserve Vercel AI Gateway cost and routing metadata - #63320
Divyanshu-kumar14 wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
The fix works. I ran the #63293 reproduction from source against a mock gateway. On the base, the assistant message stores the catalog estimate (0.0021) and no gateway metadata. With this pull request it stores the billed 0.0207155 and native.gateway with routing.finalProvider and generationId. --sanitize redacts it. The ai and core tests closest to the change pass, and tsc --noEmit reports no errors in packages/ai or in the changed core files.
The approach is sound:
- Core's
native()keeps onlyproviderMetadata[providerMetadataKey], so writing the metadata under both keys stores just one copy, and top-levelgatewaykeeps working for existing@opencode/aicallers. - Treating a gateway cost of
"0"as a real free request matches theUsage.costdocs. - Redacting assistant
nativein sanitized exports matches howcompaction.nativeis already handled.
Public API: all additive, and existing callers keep working.
step-finishevents from gateway routes now carryproviderMetadata.gatewayandproviderMetadata[providerMetadataKey].gateway, andusage.costis filled from the gateway cost when the response has no numeric cost.- Stored assistant messages and session exports gain
native.gateway. session export --sanitizenow redacts assistantnative, which used to pass through unchanged.
One request: session-runner-vercel-gateway.test.ts is large for what it checks (see the inline comment).
Related: #60309
f0ad39d to
666fd69
Compare
There was a problem hiding this comment.
The new commits address the earlier test-size point. The core gateway test is down to two cases, the billed Messages cost with routing and the empty-cost catalog fallback, and the runner layer setup is now a shared test/lib/session-runner-harness.ts that session-runner-recorded.test.ts also uses. The source is unchanged since the last review, so the fix still holds. The core gateway, recorded-runner and session-create tests pass, and the core test type check is clean. Nothing new to flag. The public API impact is unchanged from the earlier review.
666fd69 to
d88be62
Compare
There was a problem hiding this comment.
Since the last review this was only rebased onto newer dev. The only change to the pull request's own commits is the claude: true field added to the instruction-discovery mock in the shared test harness, matching dev. The fix code is unchanged, the gateway tests in ai and core pass, and the core test type check is clean. Nothing new to flag. The public API impact is unchanged from the earlier review.
Issue for this PR
Closes #63293
Type of change
What does this PR do?
Vercel AI Gateway responses carry billed cost and routing under
provider_metadata.gateway, but sessions persisted without it:gatewayProtocolattached metadata only tofinishwhile core readsstep-finish, wrote it under top-levelgatewaywhile core readsproviderMetadata["vercel-ai-gateway"], and never decoded the stringgateway.costintoUsage.cost, soSessionUsage.recordfell back to catalog pricing. The fix tracks the request'sproviderMetadataKey, strictly parses non-negative finite cost values (empty, whitespace, negative, and non-finite values still fall back to catalog pricing), and emits merged metadata on bothstep-finishandfinish. A separate commit redactsassistant.nativein sanitized exports because it now persists Gateway routing identifiers; it can be dropped independently.How did you verify your code works?
Reproduced against a mock
/v1/messagesendpoint returning stringcost: "0.0207155":opencode session exportnow records the billed cost andnative.gateway, and--sanitizeredacts it. The core Gateway suite covers the Messages alias and the empty-cost catalog fallback end-to-end, while protocol-level cases live in the AI provider suite.@opencode/ai(2,098 pass / 28 skip / 0 fail), core gateway and session suites (520 pass / 0 fail across 10 randomized runs),bun run check(35/35),bun run typecheck:tsc(33/33), and oxlint are clean.Checklist
Added by the opencode bot (AI-generated).
Related: #60309
Public API: step-finish events from gateway routes now carry gateway metadata and usage.cost, assistant messages store native.gateway, and sanitized exports redact assistant native (backwards compatible)