Add cached token fields to TokenUsage and parse OpenAI-compatible cache usage - #259
Add cached token fields to TokenUsage and parse OpenAI-compatible cache usage#259yaman-694 wants to merge 2 commits into
Conversation
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## trunk #259 +/- ##
============================================
+ Coverage 88.16% 88.23% +0.07%
- Complexity 1217 1221 +4
============================================
Files 61 61
Lines 3945 3969 +24
============================================
+ Hits 3478 3502 +24
Misses 467 467
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Summary
Adds nullable
cachedTokensandcacheCreationTokensfields to theTokenUsageDTO and parses them inAbstractOpenAiCompatibleTextGenerationModel, so prompt-cache usage reported by OpenAI-compatible providers is no longer discarded.OpenAI returns
usage.prompt_tokens_details.cached_tokens(and, more recently,cache_write_tokens) on every text-generation response for cache-eligible models, but the base class previously read only the three top-levelusagefields and then strippedusagefromadditionalData, making cache hit rate unobservable to callers. Prompt caching is a major cost lever (50-75% input discount on hit), and without this data downstream applications cannot measure whether their prompt structure is actually cache-friendly.What changed
src/Results/DTO/TokenUsage.php: new nullablecachedTokensandcacheCreationTokensproperties following the existingthoughtTokenspattern (constants, optional constructor params, getters, JSON schema entries, conditionaltoArray()emission,fromArray()support). Semantics are documented in the class docblock: cached tokens are a subset of prompt tokens, not additive.src/Providers/OpenAiCompatibleImplementation/AbstractOpenAiCompatibleTextGenerationModel.php: the usage parser now readsprompt_tokens_details.cached_tokensintocachedTokensandprompt_tokens_details.cache_write_tokensintocacheCreationTokens. TheUsageDataPHPStan shape is extended accordingly. Field names were verified against the current OpenAI API spec.nullwhen a provider does not supply them, so "not reported" is distinguishable from a real0.Backward compatibility
nulldefaults, so every existingnew TokenUsage(...)call site behaves identically.toArray()emits the new keys only when non-null, so serialized output is unchanged for existing consumers, and the JSON schemarequiredlist is untouched.cache_read_input_tokens/cache_creation_input_tokensfor when a first-class Anthropic implementation surfaces them.Testing
toArray()/fromArray()round trips (including key omission when unset), JSON schema coverage, and parser tests withprompt_tokens_detailspresent and absent.composer lint(PHPCS including PHPCompatibility 7.4- and PHPStan) andcomposer test:unitpass (1110 tests).chat/completionsendpoint: both values surface throughPromptBuilder -> generateTextResult() -> getTokenUsage(), and both returnnullwhen the provider omits the details object.Issue
Fixes #236
Note: the
wordpress/openai-ai-providerpackage (now on the Responses API) has a parallel gap in its own parser, whereinput_tokens_details.cached_tokensis dropped. Happy to file a follow-up issue in that repo.