Add OrcaRouter-Adaptive submission to leaderboard#43
Open
ZhenghuaBao wants to merge 2 commits into
Open
Conversation
Mirrors the upstream RouterArena submission merged in PR #104 (predictions), #108 (README), and #110 (link). LinUCB contextual bandit over a 10-model pool; Arena 72.08, Accuracy 75.54, Cost/1K $1.00, Robustness 22.62. The three optimality columns and latency are nulled — the submitted prediction file did not include for_optimality entries, same convention as GPT-5 and Azure-Model-Router. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Author
|
Hi @yl231, Thank you for maintaining RouterArena. I would like to request that this PR be reviewed and merged so that our work can be displayed on the official RouterArena website. Thanks for your time and consideration. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the OrcaRouter-Adaptive entry to the leaderboard data, mirroring the upstream RouterArena submission merged in PR #104 (predictions), PR #108 (README rank), and PR #110 (website link).
Submission
Bot-reported metrics
The three optimality columns and latency are dashed — the submitted prediction file did not include
for_optimality=Trueentries, same convention as GPT-5 and Azure-Model-Router. May be filled in a follow-up submission.Files touched
src/data/routerMetrics/leaderboard.json— new row, sorted by Arena Score (between Sqwish 75.27 and R2-Router 71.6)src/data/routers.json— new metadata entrysrc/data/routers.yaml— new metadata entryNo category_scores.json change —
routerData.tsauto-synthesizes the entry from the leaderboard row.Test plan
python -c "json.load(...)"on both edited JSON files — parses clean