Skip to content

Add cached token fields to TokenUsage and parse OpenAI-compatible cache usage - #259

Open
yaman-694 wants to merge 2 commits into
WordPress:trunkfrom
yaman-694:add/cached-tokens-token-usage
Open

Add cached token fields to TokenUsage and parse OpenAI-compatible cache usage#259
yaman-694 wants to merge 2 commits into
WordPress:trunkfrom
yaman-694:add/cached-tokens-token-usage

Conversation

@yaman-694

Copy link
Copy Markdown

Summary

Adds nullable cachedTokens and cacheCreationTokens fields to the TokenUsage DTO and parses them in AbstractOpenAiCompatibleTextGenerationModel, so prompt-cache usage reported by OpenAI-compatible providers is no longer discarded.

OpenAI returns usage.prompt_tokens_details.cached_tokens (and, more recently, cache_write_tokens) on every text-generation response for cache-eligible models, but the base class previously read only the three top-level usage fields and then stripped usage from additionalData, making cache hit rate unobservable to callers. Prompt caching is a major cost lever (50-75% input discount on hit), and without this data downstream applications cannot measure whether their prompt structure is actually cache-friendly.

What changed

  • src/Results/DTO/TokenUsage.php: new nullable cachedTokens and cacheCreationTokens properties following the existing thoughtTokens pattern (constants, optional constructor params, getters, JSON schema entries, conditional toArray() emission, fromArray() support). Semantics are documented in the class docblock: cached tokens are a subset of prompt tokens, not additive.
  • src/Providers/OpenAiCompatibleImplementation/AbstractOpenAiCompatibleTextGenerationModel.php: the usage parser now reads prompt_tokens_details.cached_tokens into cachedTokens and prompt_tokens_details.cache_write_tokens into cacheCreationTokens. The UsageData PHPStan shape is extended accordingly. Field names were verified against the current OpenAI API spec.
  • Both fields stay null when a provider does not supply them, so "not reported" is distinguishable from a real 0.

Backward compatibility

  • The new constructor parameters are optional with null defaults, so every existing new TokenUsage(...) call site behaves identically.
  • toArray() emits the new keys only when non-null, so serialized output is unchanged for existing consumers, and the JSON schema required list is untouched.
  • The naming is provider-agnostic and also fits Anthropic's cache_read_input_tokens / cache_creation_input_tokens for when a first-class Anthropic implementation surfaces them.
  • The fields are nullable from the start, consistent with the direction proposed in Improve handling of unprovided usage statistics #158.

Testing

  • 10 new unit tests: DTO construction and getters, toArray()/fromArray() round trips (including key omission when unset), JSON schema coverage, and parser tests with prompt_tokens_details present and absent.
  • composer lint (PHPCS including PHPCompatibility 7.4- and PHPStan) and composer test:unit pass (1110 tests).
  • Verified end to end with a minimal OpenAI-compatible provider pointed at a mocked chat/completions endpoint: both values surface through PromptBuilder -> generateTextResult() -> getTokenUsage(), and both return null when the provider omits the details object.

Issue

Fixes #236

Note: the wordpress/openai-ai-provider package (now on the Responses API) has a parallel gap in its own parser, where input_tokens_details.cached_tokens is dropped. Happy to file a follow-up issue in that repo.

@github-actions

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: yaman-694 <yamanzyan@git.wordpress.org>
Co-authored-by: chubes4 <extrachill@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@codecov

codecov Bot commented Jul 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.23%. Comparing base (052a468) to head (969377a).

Additional details and impacted files
@@             Coverage Diff              @@
##              trunk     #259      +/-   ##
============================================
+ Coverage     88.16%   88.23%   +0.07%     
- Complexity     1217     1221       +4     
============================================
  Files            61       61              
  Lines          3945     3969      +24     
============================================
+ Hits           3478     3502      +24     
  Misses          467      467              
Flag Coverage Δ
unit 88.23% <100.00%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Surface prompt_tokens_details.cached_tokens (and parallel cache fields) in TokenUsage

1 participant