Skip to content

Production evidence: Anthropic cache bust bug causes $14.82 waste across 3 sessions (22.5% bust rate) #40790

Closed as not planned

Description

@Arrowyi

Summary

Production data from OpenCode 1.18.9 shows the Anthropic prompt cache bust bug (Issue #24841) is still active and causing significant financial waste.

Key findings:

Estimated waste: $14.82 (39% of total cost) could have been avoided if caching worked correctly.


Evidence

Session 1: ses_03563ecbdffe92x5KFrsYJRWnN ($18.66)

Title: Creating AGENTS.md for repository
Date: 2026-08-04
Messages: 43 (38 tool-calls, 2 stop)
Total tokens: 80 input / 48,028 output / 6,148,780 cache read / 2,301,161 cache write

Cost breakdown:

Component Tokens Rate Cost %
Input 80 $5.00/M $0.00 0%
Output 48,028 $25.00/M $1.20 6.4%
Cache Read 6,148,780 $0.50/M $3.07 16.5%
Cache Write 2,301,161 $6.25/M $14.38 77.1%

Cache bust pattern (all after tool-calls):

msg#  cache_read  cache_write  cost    prev_finish  flag
1     0           84,195       $0.55   (start)      CACHE RESET
21    0           233,523      $1.47   tool-calls   CACHE RESET
29    0           255,794      $1.61   tool-calls   CACHE RESET
30    0           256,289      $1.61   tool-calls   CACHE RESET
34    0           259,313      $1.63   tool-calls   CACHE RESET
35    0           259,498      $1.63   tool-calls   CACHE RESET
36    0           259,734      $1.63   tool-calls   CACHE RESET
37    0           260,018      $1.63   tool-calls   CACHE RESET
38    0           260,207      $1.63   tool-calls   CACHE RESET

Consecutive cache busts (#34-38): 5 in a row, wasting $8.15

Expected cost if cache worked: $5.84
Actual cost: $18.66
Waste: $12.82 (69%)


Session 2: ses_03861eca0ffeuobD3dLHqYq99p ($11.50)

Title: LLM速度对比与Agent配置优化建议
Date: 2026-08-03
Messages: 73 (67 tool-calls, 3 stop)
Cache busts: 2
Cache write: 523,712 tokens × $6.25/M = $3.27 (28.5% of cost)


Session 3: ses_0381fd091ffe4KS4TxsnTcsOFW ($7.51)

Title: Claude Code Max订阅与API token费用对比
Date: 2026-08-03
Messages: 45 (35 tool-calls, 5 stop)
Cache busts: 2
Cache write: 498,810 tokens × $6.25/M = $3.12 (41.5% of cost)


Comparison: DeepSeek V4 Pro (no cache write fee)

Session: ses_02b3612c6ffeOFnJw6eQrlTKW0 (2026-08-06)
Cost: $0.03
Cache write: 0 (DeepSeek does not charge for cache writes)
Cache read: grows steadily from 2K to 89K (no busts)

This demonstrates that the issue is specific to Anthropic's cache write pricing combined with the cache bust bug.


Root Cause

Matches Issue #24841 exactly:

  1. OpenCode's prompt loop reloads all messages from DB via filterCompactedEffect() at the start of every iteration
  2. Tool parts transition from pending → completed with output text between API calls
  3. toModelMessages() serializes the conversation with different bytes
  4. Anthropic's prompt cache sees different content → cache invalidated from that position forward
  5. Entire conversation context (~250K tokens) re-written at cache-write pricing ($6.25/M)

Verified: All 9 cache busts in the $18.66 session occurred immediately after tool-calls finish type.


New Evidence Not in Previous Reports

  1. Consecutive cache busts: 5 in a row (messages Codex mini #34-38), each costing $1.63. This suggests the cache is being invalidated on every single turn during heavy tool-use loops.

  2. Cache bust rate: 22.5% (9/40 messages) in a real production session with only 43 messages.

  3. Cost per bust: $1.46-$1.63 (250K tokens), lower than the $3.50 (560K) reported in Prompt loop DB reload breaks Anthropic cache after tool calls (63% of spend) #24841, but still significant at scale.

  4. Production cost data: Real money wasted ($14.82 across 3 sessions) with exact per-message cost breakdown from OpenCode's SQLite database.


Database Queries

Query 1: Session-level cache stats

SELECT 
  id, 
  cost, 
  tokens_input, 
  tokens_output, 
  tokens_cache_read, 
  tokens_cache_write 
FROM session 
WHERE id = 'ses_03563ecbdffe92x5KFrsYJRWnN';

Query 2: Message-level cache pattern

SELECT 
  json_extract(data, '$.finish') as finish,
  json_extract(data, '$.tokens.cache.read') as cache_r,
  json_extract(data, '$.tokens.cache.write') as cache_w,
  json_extract(data, '$.cost') as cost
FROM message 
WHERE session_id = 'ses_03563ecbdffe92x5KFrsYJRWnN' 
  AND json_extract(data, '$.role') = 'assistant'
ORDER BY json_extract(data, '$.time.created');

Query 3: Verify cache busts after tool-calls

WITH msg_list AS (
  SELECT 
    json_extract(data, '$.finish') as finish,
    json_extract(data, '$.tokens.cache.read') as cache_r,
    json_extract(data, '$.tokens.cache.write') as cache_w,
    ROW_NUMBER() OVER (ORDER BY json_extract(data, '$.time.created')) as rn
  FROM message 
  WHERE session_id = 'ses_03563ecbdffe92x5KFrsYJRWnN' 
    AND json_extract(data, '$.role') = 'assistant'
)
SELECT 
  m1.rn,
  m1.cache_r,
  m1.cache_w,
  m2.finish as prev_finish
FROM msg_list m1
LEFT JOIN msg_list m2 ON m1.rn = m2.rn + 1
WHERE m1.cache_r = 0 AND m1.cache_w > 0;

Impact

  • Financial: $14.82 wasted across 3 sessions (39% of total Opus 5 spend)
  • User trust: Users expect prompt caching to work as advertised; silent cache busts at 12.5x read cost are unacceptable
  • Model choice: Forces users to avoid Anthropic models on Zen for tool-heavy workloads, limiting model selection

Request

  1. Reopen Issue Prompt loop DB reload breaks Anthropic cache after tool calls (63% of spend) #24841 or create a new issue with this evidence
  2. Prioritize PR fix(session): cache messages across prompt loop to preserve prompt cache byte-identity #36852 (or equivalent fix) to cache messages across prompt loop iterations
  3. Add cache bust detection/logging so users can see when this is happening
  4. Consider a cost warning when cache bust rate exceeds a threshold

Environment

  • OpenCode: 1.18.9
  • OS: macOS (darwin)
  • Provider: OpenCode Zen (opencode/claude-opus-5)
  • Variant: max
  • Agent: Sisyphus - ultraworker (via oh-my-openagent)

Additional Data

Full session data available upon request (can export from ~/.local/share/opencode/opencode.db). Per-message cost breakdown available. Can provide additional sessions if needed.

Related: #24841 #25366 #31525 #20110 #36852

Activity

  1. Arrowyi commented on Aug 14, 2026

    @Arrowyi
    Author

    Adding new evidence that was not in the original report and addressing the byte-identity point raised in #31525.

    Consecutive cache busts (not previously reported)

    The most expensive session (ses_03563ecbdffe92x5KFrsYJRWnN, $18.66, 43 messages) shows 5 consecutive cache busts across messages #34-38. Each turn re-wrote ~260K tokens at cache-write pricing ($6.25/M), costing $1.63 per turn:

    msg#  cache_read  cache_write  cost    prev_finish
    34    0           259,313      $1.63   tool-calls
    35    0           259,498      $1.63   tool-calls
    36    0           259,734      $1.63   tool-calls
    37    0           260,018      $1.63   tool-calls
    38    0           260,207      $1.63   tool-calls
    

    Five consecutive turns, $8.15, zero cache reads. This is not a one-off invalidation; the cache is being re-written on every turn during a tool-use loop.

    Full bust pattern (all 9 after tool-calls)

    msg# cache_write cost prev_finish
    1 84,195 $0.55 (start)
    21 233,523 $1.47 tool-calls
    29 255,794 $1.61 tool-calls
    30 256,289 $1.61 tool-calls
    34-38 259-260K $1.63 x5 tool-calls (x5)

    9 of 9 busts immediately follow a tool-calls finish. The correlation is deterministic, not incidental.

    On the byte-identity framing (#31525)

    I understand the objection that Anthropic caching is keyed on content, not object identity. The data here does not depend on that framing. The observable fact is that cache_read = 0 and cache_write > 0 on turns immediately after tool-calls, and cache_write scales with full context size (~260K tokens), not with the incremental delta. Whatever the exact mechanism, the effect is that the entire conversation is re-serialized into the cache on each tool-turn, and Anthropic charges cache-write rates for it.

    Reproducible from the database

    No API introspection required. Anyone can verify from ~/.local/share/opencode/opencode.db:

    WITH msg_list AS (
      SELECT
        json_extract(data, '$.finish') as finish,
        json_extract(data, '$.tokens.cache.read') as cache_r,
        json_extract(data, '$.tokens.cache.write') as cache_w,
        json_extract(data, '$.cost') as cost,
        ROW_NUMBER() OVER (ORDER BY json_extract(data, '$.time.created')) as rn
      FROM message
      WHERE session_id = 'ses_03563ecbdffe92x5KFrsYJRWnN'
        AND json_extract(data, '$.role') = 'assistant'
    )
    SELECT m1.rn, m1.cache_r, m1.cache_w, m1.cost, m2.finish as prev_finish
    FROM msg_list m1
    LEFT JOIN msg_list m2 ON m1.rn = m2.rn + 1
    WHERE m1.cache_r = 0 AND m1.cache_w > 0;

    Scope and limitations

    This is production evidence from OpenCode 1.18.9 with opencode/claude-opus-5 (max), running the Sisyphus ultraworker agent. Total waste across 3 sessions is $14.82 (39% of spend). I have since switched these workloads to DeepSeek, whose sessions show steady cache-read growth and zero cache-write charges, which isolates the issue to Anthropic's cache-write pricing combined with this behavior.

    I am not asking for a specific implementation, only that this be looked at with the deterministic 9/9 tool-call correlation and the 5-consecutive-bust sequence on record. If a fix lands in #14743 or elsewhere, happy to re-run the same workload and provide before/after numbers.

  2. cn4750 commented on Sep 3, 2026

    @cn4750

    I also see this issue.

  3. rijnhard commented on Sep 7, 2026

    @rijnhard

    This was actually included in a published article. https://thenewstack.io/agent-harness-token-costs/

    the actual source: https://www.eishanlawrence.com/blog/harness-efficiency-bench

    Basically using OpenCode at the moment can have a severe financial impact.

  4. thdxr commented on Oct 11, 2026

    @thdxr
    Member

    Thanks for the report! OpenCode 2.0 is now out and development is focused on V2, so we're closing issues filed against OpenCode 1. Please upgrade (opencode upgrade --major, details: https://anoma.ly/notes/opencode-2-0), and if this is still relevant on 2.0, reopen this issue or open a new one with your 2.0 version.

    — from 𝕺𝖕𝖊𝖓𝕮𝖔𝖉𝖊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions