Ordered window starts two full-source reads after a signalling refetch with a NaN boundary

Summary

An ordered live query issues two full-source reads after one signalling refetch. The law in the ordered-work oracle allows one. The window is descending with nulls: first, an anti-joined marker filters it, and one source row has a NaN rank, so the bounded boundary cannot be expressed.

The failure is on main. #2194 does not cause it. It changes the oracle's generator, so the seed reproduces only with the oracle file from #2194.

Law

The ordered-work oracle (packages/db/tests/query/ordered-work-oracle.property.test.ts) checks this rule: after a signalling refetch, the loader starts at most one full-source request. #2194 makes the law skip a bounded read that ends on a null or NaN rank, because ARCHITECTURE.md sends that boundary to the full-source fallback. Here the loader starts two full-source requests, not one.

Reproduce

Check out main (tested at 501b481).
Copy packages/db/tests/query/ordered-work-oracle.property.test.ts and packages/db/tests/oracle-config.ts from the ordered-loader-ledger branch (fix(db): keep overlapping ordered requests loading and release what they supersede #2194).
From packages/db, run:

TANSTACK_DB_ORACLE_SEED=-1142653059 npx vitest run --coverage.enabled=false \
  tests/query/ordered-work-oracle.property.test.ts \
  -t "keeps rows and per-change work bounded"

Result:

AssertionError: full-source requests after a signalling refetch at step 2: expected 2 to be 1

Counterexample:

{"path":"on-demand","provider":"refetch-signal","latency":"immediate","cancellation":"cooperative","peer":false,"filter":"anti-joined","via":"marker","indexed":false,"direction":"desc","nulls":"first","terms":1,"limit":2,"offset":0,"initial":[{"id":0,"rank":0,"eligible":true},{"id":2,"rank":NaN,"eligible":true},{"id":1,"rank":0,"eligible":false}],"history":[{"type":"delete-visible"},{"type":"move","offset":0,"limit":1}]}

Open question

The cause is not known yet. There are two possible causes:

The exemption in the law is too narrow for a NaN rank at the start of a descending, nulls: first order.
The loader starts a second full-source read that it does not need.

A fix must decide which one, with ARCHITECTURE.md as the authority, before it changes the law or the loader.

This issue was written by Isaac.

fix(db): hold publication for a demand that fails behind a sibling's …

Two more generated traces fail on main with a similar shape. Each is an ordered window on the on-demand path, filtered through a marker, that goes through a replay. They may share a cause with this issue. That is a hypothesis from the shapes. No ablation has run yet.

Trace 1: two full-source reads after a signalling refetch, ascending order

This is the same law as this issue, without a NaN rank. It failed on main at 7538afd. Replay it from packages/db:

TANSTACK_DB_ORACLE_SEED=-1572621508 \
TANSTACK_DB_ORACLE_PATH=7:2:3:3:3:3:3:4:4 \
TANSTACK_DB_ORACLE_PROPERTY=ordered-work.eager-indexed-window \
npx vitest run --coverage.enabled=false \
  tests/query/ordered-work-oracle.property.test.ts \
  -t "keeps rows and per-change work bounded"

AssertionError: full-source requests after a signalling refetch at step 1: expected 2 to be +0

{"path":"on-demand","provider":"refetch-signal","latency":"immediate","cancellation":"cooperative","peer":false,"filter":"anti-joined","via":"marker","indexed":false,"direction":"asc","nulls":"first","terms":1,"limit":1,"offset":0,"initial":[],"history":[{"type":"upsert","id":0,"rank":0,"eligible":false},{"type":"refetch"}]}

Trace 2: a window keeps its old rows after a truncate replay

The window shrinks from 3 rows to 1 while its source is held, and then the source truncates. The settled window keeps the 3 rows. It failed on main at 7e46f81. Replay it from packages/db:

TANSTACK_DB_ORACLE_SEED=-104272808 \
TANSTACK_DB_ORACLE_PATH=293:3:7:11:8:8:8 \
TANSTACK_DB_ORACLE_PROPERTY=replay.ordered-window-faults \
npx vitest run --coverage.enabled=false \
  tests/query/load-subset-replay-refinement-oracle.test.ts \
  -t "load and release faults"

AssertionError: settled window: expected [ 10, 20, 30 ] to deeply equal [ 10 ]

[[{"type":"hold"},{"type":"window","offset":0,"limit":3},{"type":"truncateMarkers"},{"type":"window","offset":0,"limit":1},{"type":"truncate","armAfterStart":false}],{"direction":"asc","offset":0,"limit":1,"ties":false,"peer":false,"joined":false,"markerMode":"on-demand","includes":true}]

What the three traces share

	This issue	Trace 1	Trace 2
Path	on-demand	on-demand	on-demand
Filter	anti-join through a marker	anti-join through a marker	on-demand markers, with includes
Replay	signalling refetch	signalling refetch	truncate replay
Symptom	an extra full-source read	an extra full-source read	the window keeps its old size

One possible single cause: the replay path re-establishes the window from request state that the replay made obsolete. The next step is an ablation. Replay each trace without the marker filter, and trace 2 without includes. If all three pass without the marker, the cause is likely in the marker-filtered replay path. #2191 is probably separate: it concerns failure bookkeeping, and its model leaves out truncate replay.

Both new traces came up in random campaigns on PR #2263. That PR does not cause them: each fails on main.

This comment was written by Isaac.

test(db): keep the two traces reported on #2221 as named replays

All three traces pass on PR #2231 (branch codex/issue-2224). Two separate changes fix them, so they do not share one marker-filter cause, and the ablation is not necessary.

Trace	Fails on `main`	Fix on #2231
This issue (seed -1142653059)	yes	`fe48cbe54`: a short unindexed prefix grows and does not read the full source
Trace 1 (seed -1572621508)	yes, `expected 2 to be +0`	`fe48cbe54`, the same cause
Trace 2 (seed -104272808)	yes, `expected [ 10, 20, 30 ] to deeply equal [ 10 ]`	`ce1299ded`: a window move that waits behind a replay applies after a newer truncate discards that replay

Evidence

This issue and trace 1: with the loader from before fe48cbe54, each counterexample fails with the message above. With the change, the loader reads no full source after the refetch. Of the two possible causes in this issue, the second one applies: the loader started a full-source read that it did not need.
Trace 2: bisection over the PR's commits. The trace fails at 26bff5f4f and passes at ce1299ded. The older code dropped the deferred one-row move together with the discarded replay, so the window kept three rows.
Each trace is now a named replay in its oracle (5b4c304d4). The ordered-work oracle holds this issue and trace 1. The load-subset replay oracle holds trace 2.

#2191 is a different issue. Its model does not include truncate replay, and #2231 does not change it.

#2231 closes this issue.

This comment was written by Isaac.

Merge remote-tracking branch 'origin/main' into codex/issue-2224

fix(db): keep a live query running when a lazy demand fails during a …

Navigation Menu

Repository navigation

Ordered window starts two full-source reads after a signalling refetch with a NaN boundary #2221

Summary

Law

Reproduce

Open question

Trace 1: two full-source reads after a signalling refetch, ascending order

Trace 2: a window keeps its old rows after a truncate replay

What the three traces share

Evidence

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Ordered window starts two full-source reads after a signalling refetch with a NaN boundary #2221

Description

Summary

Law

Reproduce

Open question

Activity

KyleAMathews commented on Oct 27, 2026

Trace 1: two full-source reads after a signalling refetch, ascending order

Trace 2: a window keeps its old rows after a truncate replay

What the three traces share

KyleAMathews commented on Oct 27, 2026

Evidence

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions