Skip to content

ladder: the ledger scenario corpus fails intermittently with "database is locked" — observed once, not reproduced #658

Description

@Yaraslaut

Found during the landing sweep for PR #654, whose Application ladder leg went red. Filed rather than folded: the failure has nothing to do with that branch, and an unrecorded intermittent failure is one nobody finds again on purpose.

The finding

One scenario in the ledger corpus failed on a SQLite database is locked error while the rest of the corpus and all 1031 ladder tests passed.

Verification status: observed once in CI, not reproduced, and not investigated

From Application ladder, job 106204287825 of run 35557586945 (PR #654 at 0644d6cf, 2026-09-21T04:20:01Z):

ledger: 16 file(s) against ws://127.0.0.1:35615
  ok   step 34 (line 137) [books] use books
  ok   step 35 (line 139) [books] do SetCategory accountId=$spend categoryId=$loser ruleId=$rule ruleVersion=7
FAIL step 35 (line 140): do SetCategory accountId=$spend categoryId=$loser ruleId=$rule ruleVersion=7
  expected: @kind == ok
  actual:   @kind == err
  client:   books (modelId=9761190600845593476)
  reply:    err message="HY000 (5) - [SQLite]database is locked (5)"

.../scripts/scenario/scenarios/ledger/update-rule-version-conflict.scenario: 38 steps, 56 assertions, 1 failure
  FAIL update-rule-version-conflict.scenario

FAILED: ledger

What I did NOT do: reproduce it locally, re-run the leg, inspect the scenario, read the ledger server's SQLite configuration, or check whether update-rule-version-conflict.scenario is unusual in its concurrency. This is a log reading, nothing more.

Why it is worth a ticket despite the weak evidence

  1. It is not PR core: stop a cancelled control call reaching the wrapped backend, and pin parkIfInFrame's unreachable double-claim arm (fixes #636, fixes #648) #654's fault, and the next person will assume it is. The whole ladder test suite passed — 100% tests passed, 0 tests failed out of 1031 — and the branch touches SynchronousBackendAdapter and parkIfInFrame, neither of which the ledger scenario server goes near. A red Application ladder on an unrelated PR costs that PR a 49-minute cycle and a diagnosis, which is exactly what it cost here.
  2. database is locked is a contention signature, not a logic error. SQLite returns SQLITE_BUSY when another connection holds a write lock past the busy timeout. That points at the corpus runner's concurrency or the server's busy-timeout configuration rather than at the scenario's assertions.
  3. Step 35 appears twice in the excerpt — once ok at line 139 and once FAIL at line 140. Whether that is a retry, two adjacent steps sharing a number, or a display artefact of the runner, I do not know, and it may be the most informative detail here.

What would resolve it

  • Establish whether the ledger scenario server sets a SQLite busy timeout at all, and what the corpus runner's concurrency is. If there is no busy timeout, this is a configuration defect with a one-line fix and the intermittency is explained.
  • If a busy timeout exists and is adequate, find what holds the write lock — the scenario is named update-rule-version-conflict, so it is plausibly designed to produce contention, which would make this a scenario that races its own premise.

What would change the verdict

  • Close as invalid if SQLITE_BUSY here is the deliberate, asserted outcome of a conflict scenario and the corpus expected err — i.e. if the expected: @kind == ok line means the scenario file itself is wrong rather than the run.
  • Raise in priority if it recurs on another unrelated PR. One observation is a flake; two is a tax on every PR that touches the ladder path filter.

Related

🤖 Generated with Claude Code

https://claude.ai/code/session_01VptDWG2fKr2vBnLSJcgzgW

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: ladderSubsystem: ladderarea: offlineSubsystem: offlinebugSomething isn't workingtriage: rescopeReal problem, wrong framing; rewrite before building

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions