Give llm-cost-based-ratelimit test templates the fields each scenario prices on - #3441
Conversation
…provider templates
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📝 WalkthroughWalkthroughChangesThe LLM cost-based rate-limit scenarios now define token and response-model mappings for provider templates. Provider-specific fields cover caching, reasoning, service tiers, web search, inference metadata, and Bedrock path-parameter model extraction. LLM cost mappings
Priority: ⬇️ Low Estimated code review effort: 3 (Moderate) | ~20 minutes Change: Bug fix Merge Risk: ⚪ Minimal · up to The provider cost mappings are ready to merge based on the available evidence. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Description checkExplanation The description explains the problem, implementation, expected behavior, test coverage, and related pull request. It does not follow the required template and omits explicit sections for user stories, documentation, security checks, samples, and test environment. Resolution Rewrite the description using the repository template. Add the missing sections and provide the required details, including documentation impact, security-check answers, sample information, and test environments. Label the existing testing information under Unit tests and Integration tests.
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
The LLM provider templates in
llm-cost-based-ratelimit.featuredeclared onlydisplayName. The llm-cost policy now reads token locations from the route's provider template, so a template with no extraction fields yields no model and no token counts, the cost resolves to zero, the budget never drains, and the scenarios asserting a 429 on the third request never reach it.Each template now declares the fields its own scenario exercises, and nothing more. All 24 carry
promptTokens,completionTokensandresponseModel. Beyond that: the caching scenarios addcachedTokens, the Anthropic cache scenarios add the cache write fields andcacheAccounting: additive, the service tier scenarios addserviceTier, the reasoning and thinking scenarios addreasoningTokens, and the web search and geo/speed scenarios add theproviderFieldstheir calculator reads. Bedrock reads its model from the URL viapathParam.Budgets and asserted status codes are unchanged. Every scenario was priced through the policy against the same mock payload the suite sends, and each result matches the cost documented in that scenario's comment, so each budget remains exactly two requests' worth.
Pairs with the llm-cost v1.2.0 release in wso2/gateway-controllers#304, which is the change that makes extraction template-driven.