Skip to content

feat(models): keep a Bedrock-routed model in its own row - #1448

Open
ulises-c wants to merge 1 commit into
getagentseal:mainfrom
ulises-c:feat/model-route-bedrock
Open

ulises-c wants to merge 1 commit into
getagentseal:mainfrom
ulises-c:feat/model-route-bedrock

Conversation

@ulises-c

@ulises-c ulises-c commented Sep 15, 2026

Copy link
Copy Markdown

Summary

Closes #1449 — the pilot of a three-step series toward answering "how much metered API spend am I incurring on top of my subscriptions?": this PR (Bedrock, from the model id) → #1450 (OpenRouter and routes read from the provider's own column, each route carrying metered / subscription, plus the list of routes to add next) → #1451 (--route / --billing filters, payload fields, a "By billing" grouping beside "By tool").

  • A model billed through AWS Bedrock is reported as its own row — Haiku 4.5 (Bedrock), GPT-5.6 Luna (Bedrock) — instead of surfacing as a raw id or, if aliased, merging into the direct-API row. Same tokens, different invoice; the row now says which.
  • getModelRoute(id) in src/models.ts recognises the Bedrock id shapes tools actually write: bare anthropic.claude-…-v1:0, us./eu./global. inference-profile prefixes, Bedrock's OpenAI ids (openai.gpt-5.6-luna, -1:0 version spelling), :300k context tags, arn:aws:bedrock:… and LiteLLM bedrock/ wrappers. getShortModelName appends the route label. Every Bedrock spelling of one model folds to one row; direct ids are byte-for-byte unchanged; pricing still runs on the full id, so cross-region profiles keep their uplifted rate.
  • looksLikeLocalModel no longer reads Bedrock's -v1:0 as an Ollama tag, so an unpriced Bedrock model reaches Unpriced instead of being treated as free local inference.
  • Bedrock is the first route; the hook is where OpenRouter / Vertex / Azure spellings belong once a provider is shown to record them (Add OpenRouter as a billing route and carry metered/subscription on each call: read the endpoint from the provider's own column, not the model id (with a list of routes to follow) #1450).

Why a separate row: the route decides whether the spend is metered

The motivating case: Claude Code and Codex on fixed-fee subscriptions (Claude Max, ChatGPT), plus API usage on top — Claude through Bedrock, models through OpenRouter. The subscription is a sunk monthly cost; the API usage is the variable part, and it is the number worth watching. Today CodeBurn cannot separate them: codeburn plan treats a whole harness as under the plan, proxy-path marks a whole project, and Hermes' cost_status = included is honoured per session (hermes.ts:483) but is not a dimension anywhere. A Bedrock call is always metered, so keeping its row apart is the first cut a user can actually read the variable spend off.

On the mechanism, versus #1241 (#1239): that PR folded OpenCode's minimax/MiniMax-M3 and MiniMaxAI/MiniMax-M3 into one row with the split kept as rawModels — right there, since both were the same metered door at the same price. Bedrock vs direct is a different billing relationship, and LiteLLM carries it as distinct SKUs (a us. inference profile prices ~10% above the bare id), so it falls under #1053's rule ("distinct SKUs that only share a label stay two rows"). The display-suffix mechanism is #1031's (grok-4.5-buildGrok 4.5 (build)): ids and pricing untouched. Because #1241 routes Trend through getShortModelName, desktop and codeburn models agree on the Bedrock rows without further work.

The suffix is the only slot the row key offers today (modelBreakdown has been keyed on getShortModelName since v0.1.0). Making the route — and the metered / subscription mode it implies — a real, filterable field is #1450 / #1451, following the store-at-parse / apply-at-read pattern #890 lays out.

Display-only: model rows are keyed at aggregation time from raw ids the caches already hold, so no cache version moves. Confined to src/models.ts; the Semgrep hot-path rule (src/providers/, src/parser.ts) does not apply and the change has no bracket-assigns regardless (ran it locally: 0 findings).

Testing

  • I have tested this locally against real data (not just unit tests)
  • npm test passes — main (3a1dac2): 4110 passed / 5 skipped; branch: 4126 passed / 5 skipped, +16 (15 in tests/model-routes.test.ts, 1 in tests/menubar-json.test.ts for the topModels contract), zero new failures
  • npm run build succeeds; also npx tsc --noEmit, npm run test:locks (36 passed) and npm run verify:upgrade (PASSED, 1 skipped) — the same steps tests.yml / upgrade-path.yml run

Real data, this machine, both providers that record Bedrock ids today:

Claude Code — one session run with CLAUDE_CODE_USE_BEDROCK=1 ANTHROPIC_MODEL=us.anthropic.claude-haiku-4-5-20251001-v1:0 (transcript records anthropic.claude-haiku-4-5-20251001-v1:0), alongside direct-API sessions of the same model. codeburn models --provider claude:

│ Claude     │ Haiku 4.5             │ Conversation (100%)   │     20 │    134 │  177.3K │  $0.187 │
│ Claude     │ Haiku 4.5 (Bedrock)   │ Exploration (100%)    │     10 │    102 │   23.0K │  $0.029 │

Hermes Agent — sessions on --provider bedrock (state.db model = openai.gpt-5.6-luna, billing_provider = bedrock) next to direct Anthropic sessions. codeburn models --provider hermes:

│ Hermes Agent   │ Fable 5.1                │ Coding (44%)   │ 253.5K │   1.5M │  217.0M │ $302.66 │
│ Hermes Agent   │ GPT-5.6 Sol (Bedrock)    │ Coding (92%)   │    244 │  80.8K │   15.8M │  $18.27 │
│ Hermes Agent   │ GPT-5.6 Luna (Bedrock)   │ Coding (61%)   │     54 │   2.4K │  455.2K │  $0.039 │

Before this change the Bedrock rows read anthropic.claude-haiku-4-5-20251001-v1:0 / openai.gpt-5.6-luna verbatim.

Not covered: Vertex @date ids (still stripped and merged with direct — documented in docs/providers/claude.md), and routes whose model id does not change (OpenRouter), which need the provider's own route column rather than id shape (#1450). Filtering / GUI: #1451.

Affiliation: none. I use Bedrock through an employer AWS account and wanted to see the split.

The same model billed through a different door is a different invoice, and
the tools that ran it record a different id per door: Claude Code with
CLAUDE_CODE_USE_BEDROCK=1 writes `anthropic.claude-haiku-4-5-20251001-v1:0`,
Hermes on its bedrock provider writes `anthropic.claude-fable-5-1` /
`openai.gpt-5.6-luna`, where a direct session writes `claude-fable-5-1`.
Reports key model rows by display name (parser.ts modelBreakdown,
menubar-json topModels, usage-aggregator), so those ids either surfaced raw
or, had they been aliased to the friendly name, would have merged with the
direct row and hidden which account the spend landed on.

`getModelRoute(id)` recognises the Bedrock shapes -- bare foundation-model
ids, `us.`/`eu.`/`global.` inference-profile prefixes, the `-vN:M` version
(and Bedrock's `-1:0` spelling on its OpenAI ids), `:300k` context tags,
`arn:aws:bedrock:...` wrappers and LiteLLM's `bedrock/` prefix -- and returns
`{ id, label, baseModel }`. `getShortModelName` renders a routed id as
"<base short name> (Bedrock)", so every Bedrock spelling of one model shares
a row, the direct row is unchanged, and pricing still runs on the full id (a
cross-region profile keeps its own rate). Vendor segments come from a fixed
table so dotted version numbers (`gpt-4.1-mini`, `glm-4.7`) are never
mistaken for a route; a segment that does not name its brand
(`deepseek.r1`) is re-joined with the vendor so it still finds its short name.

`looksLikeLocalModel` now consults the route first: `-v1:0` is a version, not
an Ollama tag, so an unpriced Bedrock model reaches Unpriced instead of being
treated as free local inference.

Display-only. Model rows are keyed at aggregation time from raw ids that the
session and daily caches already hold, so no cache version moves. A user
alias on the full Bedrock id still wins over the route.

Bedrock is the first route; OpenRouter, Vertex and Azure spellings belong in
the same hook once a provider is shown to record them (getagentseal#1450).

Closes getagentseal#1449
@ulises-c
ulises-c force-pushed the feat/model-route-bedrock branch from fbe3049 to 4c5591a Compare September 15, 2026 22:09
@ulises-c ulises-c changed the title Report a Bedrock-routed model as its own row, separate from the direct-API row feat(models): keep a Bedrock-routed model in its own row Sep 15, 2026
@ulises-c
ulises-c marked this pull request as ready for review September 15, 2026 22:32
@ulises-c
ulises-c requested a review from iamtoruk as a code owner September 15, 2026 22:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

models: a Bedrock-routed model renders as its raw id, and aliasing it would merge it into the direct-API row

1 participant