fix(db): keep a live query running when a lazy demand fails during a truncate replay

.changeset/keep-live-query-through-failed-replay.md

@@ -0,0 +1,11 @@
+---
+'@tanstack/db': patch
+---
+
+A query that joins and includes the same source has two subscriptions to it. Their truncate replays now share one barrier, so neither publishes while the other still loads. When a lazy demand of either fails while a failed replay of the source holds the query's results, the query keeps its last complete result until a later replay of that source succeeds. Before, retiring the other replay's failed demand could publish the window without that demand's rows.
+
+Some sources refetch and then report that their pages shifted. An ordered window over such a source no longer skips a row tied with the refetched first page's new end. When the report shows that the page's end moved, the window repairs its first page. It no longer trusts a row that an older tie request installed.
+
+A window move made while a truncate replay holds a live query's results now applies after that replay, even when a newer truncate discards it. Before, the move could reject, and an older move's window stayed. A reload of the query or of a source still ends the move.
+
+An ordered, limited query over a source without an order index can read too few rows to fill its window. A join that drops rows can cause this. It no longer reads the whole source then. It reads a prefix twice as long instead. It grows again while that adds rows, so every request stays bounded.

docs/contributing/glossary.md

@@ -131,7 +131,7 @@ earlier node alone.
 | demand | The logical need for source data. Demand may outlive or replace physical work. | Request, transport, or row ownership. |
 | logical subset owner | One subscription claim that keeps a subset demand active. | Physical acquisition. |
 | request data | The immutable `LoadSubsetOptions` and attached signal passed to an adapter. | Demand or established coverage. |
-| acquisition attempt | One invocation that asks a sync adapter to start physical work. A synchronous throw ends the attempt before acceptance. If cleanup invalidates the captured sync run after the adapter returns but before activation, core cancels the tentative acquisition without retaining its lease; cleanup owns the adapter resources. | Physical acquisition or transport. |
+| acquisition attempt | One invocation that asks a sync adapter to start physical work. A synchronous throw is a rejection: the attempt fails, keeps its owner, and the owner's release unloads it. If cleanup invalidates the captured sync run after the adapter returns but before activation, core cancels the tentative acquisition without retaining its lease; cleanup owns the adapter resources. | Physical acquisition or transport. |
 | physical acquisition | Request-scoped work accepted by a sync adapter for a demand. | Acquisition attempt, logical demand, or transport. |
 | lease | An ownership token that requires a matching release. TanStack DB leases do not expire on a timer. Always qualify which resource the lease owns. | Demand, request, or data coverage. |
 | acquisition lease | The release obligation created when an adapter accepts a physical acquisition. | The acquisition attempt itself. |
@@ -221,7 +221,7 @@ Use nouns for state and verbs for transitions:
 - A demand **becomes active** or **retires**.
 - Core **starts an acquisition attempt**. If the adapter accepts it, the attempt
   **establishes a physical acquisition** and its **acquisition lease**. A
-  synchronous throw **fails the attempt** before either is established. Cleanup
+  synchronous throw **fails the attempt** as a rejection does. Cleanup
   may instead invalidate a returned attempt before activation; core then
   **cancels the tentative acquisition** without retaining its lease.
 - A logical owner **retires**. Core **releases an acquisition lease**. The

packages/browser-db-sqlite-persistence/tests/multi-tab-electric-oracle.test.ts

@@ -1459,4 +1459,34 @@ describe(`multi-tab Electric over one persisted database`, () => {
       ),
     60_000,
   )
+
+  // Known gap, pinned from the random campaign (seed -1660880085, path
+  // 55:0:1:4:4:4:1:1:1:1:3:4:4:4:4): row 1 reaches v2 on the server and both
+  // tabs catch up, but after tab B stops, tab A catches up, and tab A stops,
+  // the durable rows lose row 1. Law 2 expects every open tab and the durable
+  // rows to show the head. This reproduces on main and is tracked with #2281.
+  // It replays through the same driver and checks with the known-gap detector
+  // off. Remove `fails` when the durable row survives the stops.
+  it.fails(
+    `keeps a durable row after both tabs catch up to it and then stop`,
+    () =>
+      runGeneratedHistory(
+        {
+          tabCount: 2,
+          steps: [
+            { kind: `server change`, key: 1, pick: 0 },
+            { kind: `catch up`, tab: 0, pick: 0 },
+            { kind: `server change`, key: 1, pick: 0 },
+            { kind: `catch up`, tab: 0, pick: 0 },
+            { kind: `catch up`, tab: 1, pick: 1 },
+            { kind: `catch up`, tab: 0, pick: 0 },
+            { kind: `stop`, tab: 1, pick: 0 },
+            { kind: `catch up`, tab: 0, pick: 0 },
+            { kind: `stop`, tab: 0, pick: 0 },
+          ],
+        },
+        () => undefined,
+      ),
+    60_000,
+  )
 })

packages/db-sqlite-persistence-core/tests/sync-metadata-session-persisted.test.ts

@@ -1209,6 +1209,29 @@ describe(`sync metadata session oracle through the persisted wrapper: peer repla
   it.each(peerFullReplays)(`replays under full rows: %s`, (_name, history) =>
     assertPersistedHistory(history, persistedPeerFullLane),
   )
+  // Known gap, pinned from the persisted-peers-full random campaign (seed
+  // 654774679, shrunk to this prefix): an outer transaction inserts over a
+  // bare seed, an inner one re-inserts the row and commits, and a peer's full
+  // reload then leaves row 1 with the cached seed's metadata, where the model
+  // expects none. This reproduces on main and is tracked with #2281. It
+  // replays through the same model, driver, and comparison. Remove `fails`
+  // when the reload stops restoring that metadata.
+  it.fails(
+    `replays under full rows: a reload after an inner re-insert over a seed clears the seed's metadata`,
+    () =>
+      assertPersistedHistory(
+        [
+          { type: `begin` },
+          { type: `seed`, key: 1, narrow: false, bare: true },
+          { type: `insertOverSeed`, key: 1, metadata: false },
+          { type: `begin` },
+          { type: `reinsert`, key: 1, metadata: false },
+          { type: `commit` },
+          { type: `peer`, key: 2, notice: `reload` },
+        ],
+        persistedPeerFullLane,
+      ),
+  )
   it.each(peerOnDemandReplays)(`replays on demand: %s`, (_name, history) =>
     assertPersistedHistory(history, persistedPeerOnDemandLane),
   )

packages/db/mangle-cache.json

@@ -371,7 +371,6 @@
   "repairNeedsReplacement": "gK",
   "coversSource": "gL",
   "standing": "gM",
-  "windowIsInsideOwnPrefix": "gN",
   "debt": "gO",
   "releaseOwned": "gP",
   "moveWork": "ae",
@@ -437,5 +436,8 @@
   "views": "hc",
   "show": "hd",
   "getWritableSyncTransaction": "ar",
-  "mergesOverRow": "b_"
+  "mergesOverRow": "b_",
+  "awaitsSiblingReplays": "gN",
+  "grownPrefixRows": "gZ",
+  "ownPrefixEnd": "he"
 }

packages/db/src/collection/subscription.ts

@@ -22,6 +22,7 @@ import type {
   CollectionDependent,
   LoadSubsetOptions,
   LoadSubsetRequestResult,
+  SubscribeChangesOptions,
   Subscription,
   SubscriptionEvents,
   SubscriptionLoadSubsetErrorEvent,
@@ -93,14 +94,9 @@ type CollectionSubscriptionOptions = {
   owner?: CollectionDependent
 }
 
-type TruncateReplayPublicationControl = Readonly<{
-  // `retired`: a reload retired the sync run and no truncate came.
-  start: (retired: boolean) => void
-  succeed: () => void
-  // The replay could not load every active demand again; a live query
-  // reports each failure on its node, in failure order.
-  fail?: (errors: Array<Error>) => void
-}>
+type TruncateReplayPublicationControl = NonNullable<
+  SubscribeChangesOptions[`truncateReplayPublication`]
+>
 
 type TruncatePublicationState = {
   loadedInitialState: boolean
@@ -737,6 +733,12 @@ export class CollectionSubscription
       if (activeFailures.length > 0) {
         this.abandonTruncateReplay(replayState, activeFailures)
       } else {
+        if (
+          this.awaitsSiblingReplays(replayState, () =>
+            this.checkTruncateReplayComplete(replayState),
+          )
+        )
+          return
         this.flushTruncateReplay(replayState)
       }
     } finally {
@@ -852,13 +854,9 @@ export class CollectionSubscription
   }
 
   private setReadyIfIdle(): void {
-    const replayState = this.truncateReplayState
-    const hasPendingReplayWork =
-      replayState &&
-      (replayState.pendingSetups > 0 || replayState.pending.size > 0)
     if (
       this.pendingLoadSubsetParticipants.size === 0 &&
-      !hasPendingReplayWork
+      !this.hasTruncateReplayWork
     ) {
       this.setStatus(`ready`)
     }
@@ -874,6 +872,53 @@ export class CollectionSubscription
     this.stalePublishedRows.clear()
   }
 
+  /**
+   * A lazy demand that failed outside this subscription's replay, while a
+   * failed replay of another of the graph's subscriptions to this collection
+   * holds publication, fails this subscription's replacement too. It keeps
+   * the last publication until every demand load that joins this replay
+   * loads, a later replay succeeds, or this subscription's last demand
+   * retires. The failed demand's retry joins it, so it cannot publish
+   * without that demand's rows.
+   */
+  failReplacement(error: Error): void {
+    if (this.truncateReplayState) return
+    const replayState = this.createTruncateReplayState(
+      this.collection._sync.getSyncRunGeneration(),
+      () => new Map(),
+    )
+    replayState.pendingSetups = 0
+    replayState.failed = true
+    replayState.completion.reject(error)
+    this.truncateReplayState = replayState
+    this.truncateReplacementPending = true
+  }
+
+  /**
+   * A graph's subscriptions to one collection share a barrier. While another
+   * one's replay still loads, this replay stays open, so demand this one
+   * starts meanwhile joins it; it finishes once those replays settle.
+   */
+  private awaitsSiblingReplays(
+    replayState: TruncateReplayState,
+    finish: () => void,
+  ): boolean {
+    const siblings = this.options.truncateReplayPublication?.siblingReplays()
+    void siblings?.then(() => {
+      if (this.truncateReplayState === replayState) finish()
+    })
+    return siblings !== undefined
+  }
+
+  /** Whether this subscription's replay still waits on its own work. */
+  public get hasTruncateReplayWork(): boolean {
+    const replayState = this.truncateReplayState
+    return (
+      replayState !== undefined &&
+      (replayState.pendingSetups > 0 || replayState.pending.size > 0)
+    )
+  }
+
   /** Drop the replay without publishing; an unfinished wait rejects as aborted. */
   private discardTruncateReplay(): void {
     const replayState = this.truncateReplayState
@@ -1508,6 +1553,8 @@ export class CollectionSubscription
       this.flushTruncateReplay(replayState)
       return
     }
+    if (this.awaitsSiblingReplays(replayState, () => this.retireEmptyReplay()))
+      return
     this.discardTruncateReplay()
     this.stalePublishedRows = new Map(this.publishedRows)
     this.restorePublishedSnapshotTracking()

packages/db/src/query/live/ARCHITECTURE.md

@@ -750,14 +750,18 @@ old constant, even after its first load settles: deduplication and query state
 may retain its identity. Request data uses stable data properties, not stateful
 getters. The signal and subscription references do not change, but their
 lifecycle remains live. Cancellation and release are not data mutations.
+A truncate replay asks again for each retained demand with that demand's
+request data, so an adapter sees the same `where` expression it received
+before the truncate.
 The immutable-demand boundary matrix checks direct and deferred sync startup,
 adapter return and asynchronous settlement, cancellation, and release identity.
 
 A Collection subscription installs each logical subset owner before it calls
 the source adapter. Reentrant release during `loadSubset` therefore retires the
 logical owner at once, but physical release waits until the adapter returns and
 proves that it established an acquisition. A synchronous `loadSubset` throw
-rolls the tentative owner back without calling `unloadSubset`. Logical demand
+is a rejection: the load failed, it keeps its owner, and the owner's release
+unloads it as it does any failed load. Logical demand
 retires even when `unloadSubset` fails. Each physical acquisition gets one release
 attempt, marked before calling adapter or error-listener code. Reentrant and
 repeated teardown cannot repeat it. Other acquisitions still receive cleanup,
@@ -905,7 +909,10 @@ applied settlement. A failed or obsolete replacement cannot retire established
 coverage, and a retry requests the same union again. A partial shrink may keep
 one intersecting acquisition without replacement. This avoids repeatedly
 loading the growing prefix while allowing a changing bounded window to return
-to one acquisition.
+to one acquisition. The controller starts new coverage before it releases an
+acquisition that no longer intersects current demand. A demand change
+therefore never leaves its subscription without logical demand partway
+through, which would end that subscription's truncate replay.
 
 Every sync `commit()` returns an applied receipt: `true` when that
 transaction's writes and events are already visible, or a promise when the
@@ -1021,7 +1028,18 @@ when its window is inside that page: every window row sorts at or before the
 `offset + limit`-th source row, and the next source row sorts strictly after
 it in the leading term. The refetched page holds every row from the start
 through its end, and any row it left out sorts after that end, so the window
-is proven. A window that needs a row past the page, such as one whose page
+is proven. The loader checks the next source row through an installed row,
+which is evidence only when the page read it, or when the loader read the tie
+of the page's current end: the refetch re-read that tie, so every row that ties
+with the end is installed. A row that another request installed proves
+nothing. Neither does a tie that the loader read for an older end, because a
+refetch can move the end earlier, so a row the provider never sent can tie
+with the new end. When the end moved, the loader repairs. (Reading only the
+new end's tie would also complete the proof, since the refetched page holds
+every row before the end, and saves one request; the bounded repair is kept
+because that read cost shipped bytes for one saved request per such
+refetch.) A window that needs a row
+past the page, such as one whose page
 holds rows a joined filter rejects, still repairs. So does any loader while
 its source holds accepted sync rows back, as behind a persisting optimistic
 transaction: the visible rows then predate the refetch. Core reissues the
@@ -1189,7 +1207,10 @@ join-key update, or a root change, and for Collection and Effect consumers. The
 finite root requests; this contract does not imply that it can evaluate the
 relation filter remotely.
 Finite continuation still requires an order index; an underfilled unindexed
-prefix retains the established full-source fallback.
+prefix grows. Each growth reads a prefix twice as long as the last one, so
+every read stays bounded and none reads the full source. The prefix grows
+while growing still adds rows to the source. When the last growth added
+none, the source holds no more rows and the window stays short.
 
 A custom string comparator is a local ordering contract. If any resolved
 source order term uses `stringSort: 'custom'`, bounded and unbounded queries
@@ -1330,7 +1351,16 @@ late rejection from an abandoned sync run cannot reset the replacement sync
 run's requested window.
 A window move started during an active source replay waits for that replay and
 applies only after its replacement is complete. A failed replay rejects the
-move without advancing the reported window. Replay completion callbacks carry
+move without advancing the reported window. A replay that a newer truncate or
+the retirement of its last demand discards did not fail, even though its
+completion rejects with `AbortError`: the move waits for any replay that still
+holds publication and then applies. If a replay it waits on fails, the move
+rejects with that replay's own error, its earliest failure that still decides
+it (maintainer decision). When a replay it waits on is discarded and
+the query's or a source's sync run changed meanwhile, as a reload changes it,
+the move rejects with that replay's `AbortError`. The move compares the sync
+runs only when a replay it waits on rejects, so a reload of another source
+whose replay it does not wait on leaves the move to apply. Replay completion callbacks carry
 their sync-run identity and become no-ops after cleanup or restart.
 Cleanup rejects the replay barrier, and therefore every window move waiting on
 it, with `AbortError`; no waiter may outlive the discarded subscription.
@@ -1454,6 +1484,19 @@ normalized once by the subscription.
 The `loadSubset:error` event, `lastSubsetError`, and any window move waiting on
 that replay expose the same `Error` object.
 
+A live query's subscriptions to one collection, such as a join and an include
+of the same source, share one publication barrier. None of their replays
+publishes, and none whose last demand retired closes, while another of them
+still has replay work, so demand that any of them starts meanwhile joins its
+own open replay. While a failed replay of any of them holds publication, a
+lazy demand of any of them that fails after initial-query readiness fails its
+own subscription's replacement too, so retiring the other replay's failed
+demand cannot publish the window without this demand's rows. It holds until
+every demand load that joins it loads, a later replay of the source succeeds,
+or the subscription's last demand retires. The query retries the failed demand
+at once and the retry joins the hold, so the hold ends only after the failed
+demand's rows load.
+
 ### Mutation boundaries and initial readiness
 
 A transaction `mutationFn` must not start or await collection or live-query