Skip to content

Give llm-cost-based-ratelimit test templates the fields each scenario prices on - #3441

Merged
renuka-fernando merged 1 commit into
wso2:mainfrom
Irash-Perera:fix/it-llm-cost-template-fields
Sep 17, 2026
Merged

renuka-fernando merged 1 commit into
wso2:mainfrom
Irash-Perera:fix/it-llm-cost-template-fields

Conversation

@Irash-Perera

Copy link
Copy Markdown
Contributor

The LLM provider templates in llm-cost-based-ratelimit.feature declared only displayName. The llm-cost policy now reads token locations from the route's provider template, so a template with no extraction fields yields no model and no token counts, the cost resolves to zero, the budget never drains, and the scenarios asserting a 429 on the third request never reach it.

Each template now declares the fields its own scenario exercises, and nothing more. All 24 carry promptTokens, completionTokens and responseModel. Beyond that: the caching scenarios add cachedTokens, the Anthropic cache scenarios add the cache write fields and cacheAccounting: additive, the service tier scenarios add serviceTier, the reasoning and thinking scenarios add reasoningTokens, and the web search and geo/speed scenarios add the providerFields their calculator reads. Bedrock reads its model from the URL via pathParam.

Budgets and asserted status codes are unchanged. Every scenario was priced through the policy against the same mock payload the suite sends, and each result matches the cost documented in that scenario's comment, so each budget remains exactly two requests' worth.

Pairs with the llm-cost v1.2.0 release in wso2/gateway-controllers#304, which is the change that makes extraction template-driven.

@coderabbitai

coderabbitai Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 6eb3c618-b315-4a86-8a93-81b46fd16638

📥 Commits

Reviewing files that changed from the base of the PR and between 0ddcda1 and 1a0a5d6.

📒 Files selected for processing (1)
  • gateway/it/features/llm-cost-based-ratelimit.feature

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

Changes

The LLM cost-based rate-limit scenarios now define token and response-model mappings for provider templates. Provider-specific fields cover caching, reasoning, service tiers, web search, inference metadata, and Bedrock path-parameter model extraction.

LLM cost mappings

Layer / File(s) Summary
Baseline token and model mappings
gateway/it/features/llm-cost-based-ratelimit.feature
Generic and provider templates now map prompt tokens, completion tokens, and response models.
Provider-specific usage fields
gateway/it/features/llm-cost-based-ratelimit.feature
Gemini, Anthropic, and OpenAI templates now map cached tokens, reasoning tokens, service tiers, cache accounting, and provider fields.
Bedrock model extraction
gateway/it/features/llm-cost-based-ratelimit.feature
The Bedrock template maps usage tokens from the payload and the response model from a path parameter regex.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 1a0a5

The provider cost mappings are ready to merge based on the available evidence.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description explains the problem, implementation, expected behavior, test coverage, and related pull request. It does not follow the required template and omits explicit sections for user stories,… Rewrite the description using the repository template. Add the missing sections and provide the required details, including documentation impact, security-check answers, sample information, and test environments. Label the existing testing …
✅ Passed checks (4 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: adding scenario-specific fields to the LLM cost-based rate-limit test templates.
Full details: Description check

Explanation

The description explains the problem, implementation, expected behavior, test coverage, and related pull request. It does not follow the required template and omits explicit sections for user stories, documentation, security checks, samples, and test environment.

Resolution

Rewrite the description using the repository template. Add the missing sections and provide the required details, including documentation impact, security-check answers, sample information, and test environments. Label the existing testing information under Unit tests and Integration tests.

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@Irash-Perera Irash-Perera changed the title Declare token extraction fields in cost-based ratelimit test templates Give llm-cost-based-ratelimit test templates the fields each scenario prices on Sep 15, 2026
@renuka-fernando
renuka-fernando merged commit 8edea7c into wso2:main Sep 17, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants