diff --git a/.changeset/sync-metadata-session-reads.md b/.changeset/sync-metadata-session-reads.md new file mode 100644 index 0000000000..d0d355e49d --- /dev/null +++ b/.changeset/sync-metadata-session-reads.md @@ -0,0 +1,9 @@ +--- +'@tanstack/db': patch +'@tanstack/db-sqlite-persistence-core': patch +'@tanstack/query-db-collection': patch +--- + +Fix sync metadata reads that missed committed sync writes while an optimistic transaction held them. A sync source now reads its own writes in commit order: `metadata.row.get`, `metadata.collection.get`, and `list()` include committed transactions that have not applied yet, then the open transactions. A transaction begun inside an open one is independent, and a hold no longer changes commit order. A source insert over a different source row is a duplicate even when a seed or a hold is present. + +Persisted collections treat cached rows as seeds, so a later `DbClient` seed can replace them, also after another tab's notice makes the collection reload from storage. A reload from another tab's notice keeps a seed accepted during a pending transaction, keeps row metadata of rows that do not exist, and keeps a seed's row metadata unless the notice names it. An update over a `DbClient` seed now stores the whole row the collection shows, so a reload or restart keeps the seed's fields. With `rowUpdateMode: 'full'`, an update now replaces its stored row, so a reload or restart no longer brings back fields the update dropped. A reload deletes rows that storage lacks also while a pending transaction holds them. After a notice that requires a full reload, a later notice that names its rows keeps a seed's row metadata again. A schema reset from another tab clears every row and its metadata; in on-demand mode it keeps the `DbClient` seeds the collection holds and still replays demand. Rows that storage holds after a reset keep their metadata, and an equal re-insert of one keeps it too. A commit whose signal aborts before the persisted wrapper applies it no longer takes a `DbClient` seed's place. An insert in the persisted wrapper replaces the row it lands on and writes an `insert` mutation to storage. An equal re-insert keeps the row's metadata without writing it, and the wrapper no longer slows down quadratically with the size of a transaction. diff --git a/docs/contributing/glossary.md b/docs/contributing/glossary.md index 6f484efac5..824ae70abd 100644 --- a/docs/contributing/glossary.md +++ b/docs/contributing/glossary.md @@ -51,6 +51,7 @@ production queues, caches, or semantic helpers merely to share their names. | truncate replay | An authoritative source replacement after a truncate, inside the current sync run. The Collection subscription remains active; any dependent live-query graph remains active behind its publication barrier. | Restart, reload, retry, or repair. | | source snapshot | The source rows established by one snapshot operation within its declared predicate and window. It proves full-source state only when the provider says that scope is authoritative and exhausted. | Public snapshot or proof of full-source coverage. | | public snapshot | The last coherent set of rows exposed to reads, events, and downstream queries. | Private replacement or source snapshot. | +| hydration seed | A row, with optional row metadata, that `DbClient.hydrate`, `applyCollectionChunk`, or `initialData` supplies to a Collection from outside its sync source. It is provisional base state: it supplies only an object (row data or row metadata) that no surviving source write has written, and a seed that arrives after such a write is ignored for that object. A seed is judged when it arrives: one that lands after a committed source write stays ignored for that object even if a persisted Collection later abandons the write before applying it. Among seeds of an unwritten row, the latest wins whole. | A sync transaction, a source write, persisted rows, or a live-query snapshot. | | exposed authoritative base | The applied source rows available before a Collection adds optimistic mutations. Committed sync writes queued for reconciliation are not part of this base. `collection.base` reads it without starting sync. | Visible Collection state or a pending sync queue. | | private replacement | Source or graph state withheld while an authoritative replay or repair is incomplete. | Public snapshot. | | publication | The boundary that makes one coherent result observable to reads, events, and downstream queries. | Provider return, request settlement, or sync commit by itself. | diff --git a/docs/contributing/oracle-coverage.md b/docs/contributing/oracle-coverage.md index c34f023efc..afab073a85 100644 --- a/docs/contributing/oracle-coverage.md +++ b/docs/contributing/oracle-coverage.md @@ -1488,7 +1488,7 @@ generic visitor behavior outside the named collectors. | React paced-mutation render cuts | [render-cut oracle](https://github.com/TanStack/db/blob/main/packages/react-db/tests/usePacedMutations-render-cuts-oracle.test.tsx), [PR #2170 review](oracle-reviews/pr-2170-react-paced-render-cuts.md) | `usePacedMutations` publishes callbacks only from committed renders and does so before descendant layout effects, while callback-only commits preserve mutate identity. An independent committed-document model crosses ordinary and StrictMode schedules with mutation before or after a suspended candidate, mutation after the candidate commits, and delayed or immediate mutation from a descendant layout effect. It compares ordered callback traces and public rows at the mutation cut, the trace after persistence, and mutate identity at the commit cut. Render-time publication, layout-effect-only publication, and split insertion/layout publication mutants fail at the intended comparisons. A commit between transaction admission and persistence, arbitrary React schedules, server delivery, and other frameworks remain outside this receiving owner. | | Optimistic state | [same-key composition](https://github.com/TanStack/db/blob/main/packages/db/tests/optimistic-composition-oracle.test.ts), [history model](https://github.com/TanStack/db/blob/main/packages/db/tests/optimistic-history-oracle.ts), [generated histories](https://github.com/TanStack/db/blob/main/packages/db/tests/optimistic-transaction-oracle.property.test.ts), [outcomes](https://github.com/TanStack/db/blob/main/packages/db/tests/optimistic-history-outcomes-oracle.test.ts), [publication](https://github.com/TanStack/db/blob/main/packages/db/tests/optimistic-history-publication-oracle.test.ts) | Optimistic state drops at settlement and held sync transactions publish with it ([settlement-drop review](oracle-reviews/2026-10-03-settlement-drop.md)); whole-row active overlays, metadata and prior-value events. A raw subscriber without initial state checks that each truncate batch inserts a key once ([2026-10-02 review](oracle-reviews/2026-10-02-truncate-raw-subscriber.md)). Never rebase a pending snapshot merely to simplify the model. Generated histories also write source batches inside mutation handlers, before they return; the Collection must own the request by then ([handler source batch review](oracle-reviews/2026-10-01-handler-source-batch.md)). The same-key transaction lane also authors a delete followed by a reinsert, which must be visible optimistically and net to the truth-table request ([state mutation round 2](oracle-reviews/2026-10-02-state-mutation-round-2.md)). Source rows are written as `insert` for a key the source lacks, and batches may delete keys the source holds ([state-stack mutation gaps](oracle-reviews/2026-10-02-state-stack-mutant-gaps.md)). Batches may also delete a key the source never held. That delete leaves an active request in place ([absent-key source delete review](oracle-reviews/2026-10-02-absent-key-source-delete.md)). A separate partial campaign keeps the default partial row update mode, where source updates may omit a field and merge into the source row, not into an optimistic row ([round 3 review](oracle-reviews/2026-10-05-state-mutation-round-3.md)). An aborted open batch restores the driver's own source rows as well as its keys, so a later partial update still omits the field ([aborted-batch review](oracle-reviews/2026-10-08-partial-update-aborted-batch.md)). A failed synchronous manual `mutate` callback is now checked after same-key update, distinct insert, and same-key delete, with and without earlier intent, plus two two-Collection cases, one with a restoration listener failure. The throw cut compares public rows and pending payload; the later commit checks only surviving intent. A restoration listener error retains the callback error as the aggregate cause and does not stop other Collection restoration. These witnesses allow transient notifications during the callback and do not cover nested throwing callbacks or async work after the synchronous scope. After every step the driver also checks, by transaction identity, that the Collection tracks exactly its unsettled transactions, so a failed or rolled-back transaction leaves the map ([failed transaction retention review](oracle-reviews/failed-transaction-retention.md)). The [transaction ownership oracle](https://github.com/TanStack/db/blob/main/packages/db/tests/transaction-ownership-oracle.property.test.ts) owns release across two Collections: merged-away pairs, conflicting rollbacks with a throwing subscriber, asynchronous success and failure settlements with a throwing subscriber, rollbacks from a `truncate` listener during a sync commit, a reused transaction id, and a `mutate()` callback that throws, checked in the same call stack and after promises flush. It also checks the settlement error shape and that a settled transaction's set of tracking Collections is empty; its mutations still name their Collection, so neither owner claims the Collection is collectable. Focused [restoration ownership](https://github.com/TanStack/db/blob/main/packages/offline-transactions/tests/restoration-ownership.test.ts) witnesses check that repeated offline `mutate()` calls add to one transaction, that a completed restoration settles `isPersisted`, and that a restoration one Collection cannot track leaves no Collection tracking it. Its grammar also has direct writes (`insert`, `update` and `delete` outside a transaction), optionally with a throwing subscriber that may have been sent the rows first; it captures each direct write's own transaction and requires it to fail when the admission throws and to complete otherwise, counts handler calls (none for a rejected write), and keeps a mirror subscriber on each Collection that must match the rows after every step. A write whose admission throws rolls back only its own transaction and rethrows. The [change-event history oracle](https://github.com/TanStack/db/blob/main/packages/db/tests/change-event-history-oracle.test.ts) generates local-only direct writes that meet a throwing subscriber, registered after the mirror, and requires the call to rethrow with its transaction completed and the rows following the model. Focused witnesses cover a local-only subscriber that writes the key it was told about, and a subscriber that throws a value that is not an Error; whether a throwing subscriber may stop a later subscriber is not decided. Open (approved limits): a direct write made inside another write's change subscriber reports its subscriber failure to the outer write; on a local-only Collection a direct write whose subscriber throws completes on the fast path but rolls back on the transaction path, and the change-event oracle generates only the fast path; a direct write admitted from inside a sync-commit listener, where the rollback's recompute is deferred, is not generated; the generated ownership histories read every receipt themselves, so only a focused witness (rolling back an empty manual transaction) checks that an unread receipt reaches no `unhandledrejection`; two Collection instances with one id, a Collection cleaned up before its transaction settles, and offline restoration have focused witnesses only; no generated history combines a held completed row with a second Collection. | | Drafts and native values | [proxy](https://github.com/TanStack/db/blob/main/packages/db/tests/proxy-oracle.test.ts), [detachment](https://github.com/TanStack/db/blob/main/packages/db/tests/proxy-detachment-contract-oracle.test.ts), [iteration](https://github.com/TanStack/db/blob/main/packages/db/tests/proxy-iteration-contract-oracle.test.ts), [revert](https://github.com/TanStack/db/blob/main/packages/db/tests/proxy-revert-oracle.property.test.ts), [native methods](https://github.com/TanStack/db/blob/main/packages/db/tests/proxy-native-methods-oracle.property.test.ts) | Native-operation controls, exact patches and actual stored rows; alias/cycle/adversarial-key histories. Set and Map reorder histories check draft patches and stored iteration order after replacement and clear-and-readd; RegExp replacement/reversion histories check `lastIndex`. General native-mutator and symbol-write support is not established by a plain-object oracle. The revert owner generates assignment, revert, delete, nested, symbol-key, and `for...of` histories and checks `getChanges()` and the draft against an independent draft-equality model: ordered Map and Set contents (including Sets inside Map values), RegExp `lastIndex`, array holes, and symbol keys inside nested values. Map and Set mutators and top-level symbol writes are outside its grammar; array mutators appear only in the mixed block below. Its round-trip grammar also replaces a field with a new object and restores the original value through a nested write. The law requires every changed key, and permits a written key whose final value equals its original to be reported with that value: a false positive repeats the stored value, and a false negative loses a write ([review](oracle-reviews/proxy-revert-replaced-object.md)). A further revert block mixes array `push`, `pop`, and `reverse` with assignments, a nested object replacement, and a retained handle, and checks that a mutator through a replaced handle changes nothing; pinned witnesses require a native write through a Map or Set value, a typed-array view, or a frozen draft to be reported, and a nested revert inside a Map or Set to keep the container's other write; Map and Set mutators are not mixed with assignments, and an equal-valued assignment keeps a retained handle attached (declared). The proxy owner's stored-function law checks, against a native row, that a function stored as data reads back as stored by every read path and that a stored method sees the draft as `this`, so its writes are tracked. The detachment owner pins which key classes a draft copies: enumerable string keys and every symbol key. The revert owner also generates typed arrays (including a subclass that does not forward its constructor argument), URLs, and a class with only private state as written values, with a property that writes two of them over one field. The native-methods owner generates rows (holes, `undefined`, duplicates, nested arrays, empty rows, typed `-0` and `NaN`) and calls every method of `Array.prototype` and `TypedArray.prototype`, in fixed and random campaigns with replay; it compares the result, whether the method returned the array itself, the identity of each original object, and the published row with native values, and a new built-in method fails until it has arguments. The proxy owner's `defineProperty`, stored-function, and frozen-draft laws are bounded tables against native rows. Its `defineProperty` law compares result, value, descriptor, and changes, for values and accessors. The iteration owner checks that array iterators see a push, removal, index write, or length cut made while they are open. Open: a draft reads a class instance of the original row as a plain object, so private-field getters read `undefined`, and a built-in method called through `this` on a nested Map or Set draft rejects the Proxy receiver; a mutator that changes nothing publishes an equal value; writing back a keyless class instance of the original row is a change, and `DataView` setters are untracked ([draft proxy review](oracle-reviews/code-weight-draft-proxy.md)). | -| Query DB and observer | [ownership](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/ownership-lifecycle.oracle.test.ts), [persisted direct-write window](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/persisted-direct-write-window.oracle.test.ts) ([review](oracle-reviews/issue-2029-direct-write-lock-window.md)), [load lifecycle](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/load-subset-lifecycle.test.ts) (fixed tests; no oracle owner (tracked in #2264)), [observer histories](https://github.com/TanStack/db/blob/main/packages/db/tests/live-query-observer-history-oracle.property.test.ts), [retained snapshots](https://github.com/TanStack/db/blob/main/packages/db/tests/live-query-observer-snapshot-oracle.property.test.ts) ([review](oracle-reviews/pr-2043-retained-observer-snapshots.md)) | Real QueryClient boundary, applied-settlement barriers for existing and cached demand, mutation-handler Collection identity, terminal fetch-record lifecycle, and a per-listener eligibility ledger, not a duplicate dispatch queue. The bounded handler grammar crosses insert, update, delete, parameter and mutation-alias access, refetch, and clearError. A held clearError retry retains the prior public error through initial, background, and Collection application failures; success clears it, failure advances the count, including a repeated error at clock time zero. An intermediate Query retry attempt does not recount the previous background error; a held final attempt can succeed or fail. A held application after successful Query fetch keeps the error visible until the applied result. A mocked persisted-baseline scan holds an older success while a newer terminal Query failure arrives; the newer public error and count survive application, including when the Error object and clock tick repeat. Native SQLite persistence, concurrent retries across multiple tracked Queries, mutation-handler fetch-boundary settlement, and deferred result application remain outside this witness. The mutation overlap grammar crosses both start orders. Check reentry, peer survival, FIFO and disposal independently of final rows. A retained observer snapshot shows its capture-time rows on a later first read of `data` or `state`. Direct-write witnesses now distinguish an idle pending eager cache seed (success with no new freshness timestamp) from an in-flight pending request, and require update, delete, and upsert to reach both eager Collections through their shared physical Query, in both acquisition orders. They check both public Collections and the Query cache immediately after each write. A held older shared fetch also crosses B's release and reacquisition, then compares both public rows at refetch and load settlement; mixed select wrappers and three or more co-owners remain outside this witness. The load-lifecycle tests also restore an intermediate checkpoint that keeps an explicit subset refetch pending until a post-write request applies, and rejects a no-cancel mutant at the replacement-request assertion. | +| Query DB and observer | [ownership](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/ownership-lifecycle.oracle.test.ts), [persisted direct-write window](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/persisted-direct-write-window.oracle.test.ts) ([review](oracle-reviews/issue-2029-direct-write-lock-window.md)), [load lifecycle](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/load-subset-lifecycle.test.ts) (fixed tests; no oracle owner (tracked in #2264)), [observer histories](https://github.com/TanStack/db/blob/main/packages/db/tests/live-query-observer-history-oracle.property.test.ts), [retained snapshots](https://github.com/TanStack/db/blob/main/packages/db/tests/live-query-observer-snapshot-oracle.property.test.ts) ([review](oracle-reviews/pr-2043-retained-observer-snapshots.md)) | Real QueryClient boundary, applied-settlement barriers for existing and cached demand, mutation-handler Collection identity, terminal fetch-record lifecycle, and a per-listener eligibility ledger, not a duplicate dispatch queue. The bounded handler grammar crosses insert, update, delete, parameter and mutation-alias access, refetch, and clearError. A held clearError retry retains the prior public error through initial, background, and Collection application failures; success clears it, failure advances the count, including a repeated error at clock time zero. An intermediate Query retry attempt does not recount the previous background error; a held final attempt can succeed or fail. A held application after successful Query fetch keeps the error visible until the applied result. A mocked persisted-baseline scan holds an older success while a newer terminal Query failure arrives; the newer public error and count survive application, including when the Error object and clock tick repeat. Native SQLite persistence, concurrent retries across multiple tracked Queries, mutation-handler fetch-boundary settlement, and deferred result application remain outside this witness. The mutation overlap grammar crosses both start orders. Check reentry, peer survival, FIFO and disposal independently of final rows. A retained observer snapshot shows its capture-time rows on a later first read of `data` or `state`. Direct-write witnesses now distinguish an idle pending eager cache seed (success with no new freshness timestamp) from an in-flight pending request, and require update, delete, and upsert to reach both eager Collections through their shared physical Query, in both acquisition orders. They check both public Collections and the Query cache immediately after each write. A held older shared fetch also crosses B's release and reacquisition, then compares both public rows at refetch and load settlement; mixed select wrappers and three or more co-owners remain outside this witness. The load-lifecycle tests also restore an intermediate checkpoint that keeps an explicit subset refetch pending until a post-write request applies, and rejects a no-cancel mutant at the replacement-request assertion. . Two sync-metadata session witnesses fail on `30175be48`: a claim that inserts a row keeps applied non-owner metadata, and a claim during a held retention cleanup publishes the cleaned owner (see Sync metadata session)| | Observer hydration under terminal statuses | `packages/db/tests/live-query-observer-hydration-oracle.property.test.ts` | ARCHITECTURE.md law 16 and the glossary "crash": while a hydration seed is active, a Collection `error` or `crashed` shows in the snapshot, attached or detached, and is announced; an attached observer that sees the failure ends the seed, so cleanup then shows `cleaned-up` and live rows, not the stale failure or the seed. Decision (not a law-16 claim): `cleaned-up` and restart under an active seed keep the seed as `ready`, because cleanup is not a failure and the next sync run's readiness hands off. Generated histories cross both modes, attach order, live writes, `markReady()`, `fail()`, `crash()`, cleanup, and restart, with declared reach; replays pin the crash, fail-then-cleanup, and crash-before-attach traces. Limits: rows under a failure status are not claimed; the source is a plain controlled Collection, not a live-query graph or an on-demand subset; streamed client errors, handoff timing, and framework adapters stay with the focused observer and adapter tests; the property is not in the `oracle-config.ts` path-replay registry. | | Query DB SSR dehydration | `packages/query-db-collection/tests/ssr-dehydration-oracle.test.ts`, [review record](oracle-reviews/issue-1950-ssr-dehydration.md) | A plain Query cache control, a live on-demand compound predicate, a signal-only request, and an ordered cursor with a custom comparator all retain their row through QueryClient dehydration, reconstruction of Seroval's emitted stream payload, and Query Core hydration from that payload. The on-demand query functions still receive the original IR, comparator, and signal; serialized metadata retains enumerable user metadata but excludes request options. The original implementation fails the compound and signal cases at the stream checkpoint. A serializable stream-only request-options leak passes the former chunk-count check but fails the current payload assertion. This owner covers successful Query cache entries at the installed Query Core 5.90.20 and Seroval 1.5.0 versions. A TanStack Start integration owner still needs the reporter's Router 1.171.33 and Seroval 1.6.8 path, browser Collection resume/refetch, and request cancellation across SSR. Query subset and pagination oracles remain owners of predicate/order/cursor meaning and supported value types. The issue's `shouldDehydrateQuery` workaround replaces Query Core's success-only default and admits pending/error queries; any documented workaround must preserve that default filter. | | Query DB ordered refetch | `packages/query-db-collection/tests/refetch-ordered-prefix-oracle.test.ts` | Issue #2136: after preload, and after `utils.refetch()` or a Query invalidation settles, an on-demand Query Collection behind a LEFT-joined, ordered, limited live query must show the window over the current backend rows. The design under test: query-db reports `invalidateLoadedSubsets()` when an ordered page changes keys and either has a later page or ends earlier in the order than before, and each ordered loader runs its bounded prefix repair. Its model is a pure top-k of marked backend rows. A fixed and a random campaign generate rounds of marker flips (which serve continuation pages), backend changes while the provider lags, a second live query (`observe`) whose own demand can load a page's new row first, optional Query invalidation with further changes, and a refetch, across one or two order terms, a numeric or locale-collated string order, limits 1–3, offsets 0–1, and a static or function Query key. With an unchanged backend, a repeated refetch must serve no more rows, start at most one first-page request, and report nothing. A refetch or invalidation must never start a new full-source request. A coverage law runs at the same checkpoints: every backend row from the start of the order through the window's last row (every row, when the window is short) must be in the source Collection with its current values, so a latent gap fails before a marker exposes it; a named replay of the issue trace with the moved row unmarked passes the window law and fails only the coverage law when Query DB never reports. The grammar can hold an optimistic update across a refetch, with the checkpoint moved to its release, and a premise-directed campaign under a hold (other rows observed, a row moved ahead of the page's end, a new row further ahead, a write to the last-sorting observed row) requires a report during a hold in every case and fails a skip that reads rows held back behind the transaction; the general campaigns did not. The general grammar holds optimistic deletes as well as updates. A named replay of a refetch during a held delete left the window short after release, also with this PR's report and overlap changes reverted, until the fix that resumes a continuation after a synchronous answer; no window is checked during a hold, where #2148 and #2169 remain open. Reports (`loadedSubsets:invalidated`) are bounded: a step reports at most once, and only when some ordered request was answered again with different rows; named replays require no report when a one-page window, or a window's last page, only loses a row, and a sync-boundary replay requires a report when a page ending on a row without an order value has its end move earlier (on the earlier gate, which read that page as empty, it reported nothing), and another requires a report when the provider returns a one-page window's rows in reverse order (the earlier gate took the last array row as the page's end). When a page's end moves earlier, Query DB reports, and a loader whose window lies inside its own first page must skip the repair: a named replay of an insert into a top-N list bounds the refetch's requests by its Queries, and a filtered one-page window whose page end moves past an unseen row must still repair (found by the random campaign; a loader that always skips fails it, the f9 tie replay, and the ordered-work oracle's signalling replays). The skip also refuses while the source holds accepted sync rows back: a replay refetches while an optimistic update persists, so the report arrives before the refetched rows are visible, and a skip that reads those older rows fails it. The ordered loader state oracle checks that a prefix read issued before an ordering invalidation, which settles after the bounded repair, does not retire the repair's prefix; the repair-first order failed before that fix. A `loadSubset` call that acquires a released, invalidated page can report, because Query then refetches that page on its own. A failed repair shows only the last window or the right one. Named replays cover the issue trace after a refetch and after an invalidation, with a page-capping provider, with another live query loading the page's new row first, with an empty later page, with a page whose end moves earlier while a tie request holds its old last row, a second refetch after an invalidation, a repair failure, and a provider that honors neither cursor nor offset. Never signalling, the old overlap-to-full-source rule, ignoring empty later pages, reporting on any page change, removing the repair loop guard, and reporting once per page each fail it by assertion. A replay makes a page's application fail and checks the window shows its last result or the right one, then the next refetch restores it; recording a page before its rows are accepted cannot be distinguished, because the failed attempt reports on its own. Open: the report laws bound reports but do not require each one; a page whose end moves earlier reports even with unique order values, where no row can be missing; it does not check whether a repair waits for in-flight work; the generated campaigns use an uncapped cursor provider only (an offset-only provider and a page-capping provider each hit a loader defect that reproduces on main: #2182, #2183); the generated campaigns do not check the window after a marker flip (a named replay checks the trace where the loader stopped refilling after a joined-side removal, which the repair-retirement fix below closed); a failed repair still recovers through the loader's full-source retry; interval, focus, and reconnect refetches share the invalidation path but are not driven; NaN order values have no executable witness; overlapping `refetch: true` acquisitions on one Query hash are counted, so the first to finish does not clear the loop guard for the other, but no witness separates that from a plain set; a Query refetch that joins a running `refetch: true` acquisition on the same hash does not report, and no witness reaches it (#2185); follower tabs of a multi-tab persisted collection never receive the report (#2174). The offset-only and page-capping loader defects have `it.fails` witnesses in the oracle. A finished bounded repair used to discard its boundary when its retirement made Query DB remove a row the loader had sent; the loader then restarted paging with a smaller first page that could not retire the repair's prefix, so every later refetch read both. A named replay found by the random campaign (at four times the run count) and an ordered loader state witness fail on that, and the fix also closed the joined-side refill defect. At four times the run count, the random campaign found a short window that stopped paging before the last row of the order: a report's repair released continuation Queries, and Query DB answered their reacquisition synchronously from its cache, so the graph turn inside each request asked for more while the loader was still requesting and nothing resumed the staged continuation. Only the coverage law failed. A named replay and an ordered loader state witness (a synchronous answer whose writes the graph already processed must continue the chain) fail on that, and pass with the fix. The loader side of the report also has a second owner: the ordered-work oracle's `refetch-signal` provider re-runs each served request alone and reports when any answer changed, across that oracle's generated plans, and a signalling refetch must start no full-source request unless a bounded page in that step ended on a null or NaN rank, which takes the documented full-source fallback (a random seed found that case). Never signalling and the old overlap-to-full-source rule each fail it. It excludes a peer live query, which can keep a short window after a refetch (#2155). The persisted-collection oracle checks that a source's report reaches the Collection through the managed cache-generation wrapper and that a retired source session's report is dropped. Remaining work for this class is tracked in #2186. | @@ -3778,6 +3778,92 @@ an answer, and a Collection that synced to zero rows before (it waits like an empty restore) are outside it. Hooks read Collection status only, so the framework conformance suite's status scenarios cover the binding side. +## Sync metadata session + +[Issue #2220](https://github.com/TanStack/db/issues/2220) reports that a +sync-side metadata read misses an earlier accepted transaction's write while a +persisting optimistic transaction holds it. The law is borrowed from standard +theory (maintainer decisions of 2026-10-22 and 2026-10-23, recorded in +[the review record](oracle-reviews/2026-10-22-sync-metadata-session.md)): one +sync run is a session with read-your-writes, own reads, and monotonic writes +in commit order. Every sync transaction is independent, including one begun +inside another; accepted transactions apply in commit order, and a hold +cannot change that order. An outer insert that lands on a different source +row in commit order makes its transaction invalid, and an insert written over +such a row is rejected as a duplicate, whatever holds or seeds there are. Consumers see whole +transactions. Hydration seeds and persisted cache rows are older than every +source write, and a later seed replaces either until the source writes the +row or its metadata; an equal re-insert keeps metadata without writing it. +In the persisted wrapper an insert replaces whatever row it lands on. A +peer's notice of a write in this run counts as a source write, and writes +during a storage read follow the same rules, the last write for a key +winning. When a stored row's author is unknown, a seed this tab holds stays +unless storage changed it, and other stored rows count as source writes +(decisions 13 and 15). Reloads keep metadata of a missing row (decision 16), +a write over a seed stores the row this tab shows (decision 17), and a +reload after a named notice keeps a seed's metadata (decision 18). + +| Owner | Production paths | Observations and checkpoint | +|---|---|---| +| [sync metadata session oracle](https://github.com/TanStack/db/blob/main/packages/db/tests/sync-metadata-session-oracle.ts) (`flat`, `nested`, `seeds`, `nested-seeds`, `seeds-full`, `nested-conflicts` lanes) | `DbClient` Collection, core hold, `DbClient.hydrate` seeds | `metadata.row.get`, `metadata.collection.get`, `list()`, `list(prefix)`, `collection.base`, each commit receipt's state, thrown errors; after every action and after settlement | +| [persisted driver](https://github.com/TanStack/db/blob/main/packages/db-sqlite-persistence-core/tests/sync-metadata-session-persisted.test.ts) (`persisted-flat`, `persisted-nested`, `persisted-seeds`, `persisted-cached`, `persisted-conflicts`, `persisted-nested-seeds`, `persisted-reads`, `persisted-reads-seeds`, `persisted-peers`, `persisted-peers-full`, `persisted-peers-on-demand`) | `persistedCollectionOptions` over an in-memory adapter, store hold, core hold, and a storage read held in flight; seeds through a `DbClient`, alone and crossed with nesting and a row cached in storage at startup; a peer's notices, eager, under full row updates, and on demand | the same source reads after every action; `collection.base` and receipts after settlement; in the peer lanes, the stored rows after settlement | +| paired properties and boundary witnesses in the core driver file | production against itself: holds removed, row 2's actions removed, or seeds performed first; stale run; write without a transaction; cleanup with a held seed | source reads (and row 1's view) at each shared checkpoint, or every observation after settlement for seed order; thrown errors; unhandled rejections | +| [row metadata composition oracle](https://github.com/TanStack/db/blob/main/packages/db/tests/collection-row-metadata-composition-oracle.test.ts) held-window and nested lanes | Collection, core hold, a transaction begun inside an open one, `DbClient.hydrate` | implicit row-metadata rules during a hold, and for an outer transaction whose inserts are reclassified after an inner write applies first, with and without a hold | +| [hydration authority oracle](https://github.com/TanStack/db/blob/main/packages/db/tests/db-client-hydration-authority-oracle.test.ts) | `DbClient` initialData, hydration chunks, and an adapter sync transaction (inserts, updates, deletes, truncates, and metadata-only writes) in full and partial row modes | public rows, the synced store, and row metadata through `client.dehydrate()` after each action; D5 per object | +| [Query ownership lifecycle](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/ownership-lifecycle.oracle.test.ts) receiving witnesses | Query DB claims, releases, and retention cleanup over core and `persistedCollectionOptions` | published owners and non-owner row metadata | +| receiving expectations in the [persisted oracle](https://github.com/TanStack/db/blob/main/packages/db-sqlite-persistence-core/tests/persisted-oracle.test.ts), the Electric oracle, the Browser coordinator oracle, and the Electron IPC tests | persisted source inserts over a hydrated baseline; the durable mutation a source insert writes | public and durable rows; persisted mutation types | + +On `30175be48`, 51 of 186 tests fail in the session, composition, and +retention files: held reads, commit order and outer writes after a held inner +commit, every seed rule, and the paired properties. The persisted lanes fail +the held reads through core, reads of an enclosing open transaction, an +insert's implicit clear, equal re-inserts, inserts over seeds and cached rows +under partial row updates, and late seeds beside a store-held transaction. +Six Query witnesses fail, including two that only dropping Query's `??` +metadata fallback repairs. The review record's evidence block lists reach +counts, mutants, and GREEN runs. + +Two comparisons are narrowed on purpose, each recorded with its authority in +the review record: + +- Under partial row updates the seed-order property drops `reinsert` from + both runs: decision "b" lets a seed moved first change a partial update's + merge base, so a re-insert planned from the arrival-order row is no longer + equal. The named witnesses `partialUpdateThenLateSeed` and + `seedUnderOpenReinsert` cover that interaction. +- Key independence skips histories in which the model invalidates a + transaction: invalidation removes the whole transaction, row 1's writes + included (decision 7). + +The `persisted-peers` lane plays a second tab's source through the +persistence coordinator: named row, metadata-only, delete, full-reload, and +reset notices, crossed with seeds, the cached row, nesting, and holds. A +reload that a notice causes is a storage read: its truncate replaces rows +without truncating source authority or row metadata, an unchanged row keeps +its status, and a seed's metadata survives a reload after a named notice +(decisions 9, 11, and 15 to 18). A reset is a source truncate, and every +later notice in its term reloads storage; on demand the reset keeps the +seeds this tab holds (decisions 19 and 20). After settlement the stored rows +must equal the model's, which a restart would read. +`persisted-peers-full` repeats the lane under full row updates, where an +update replaces its row in storage too; `persisted-peers-on-demand` repeats +it on demand with a demand for every row. A reset over metadata of a missing +row is too rare in generated histories to declare; a replay pins it. A reset +may rewrite its rows, which its reload then applies as source writes. +`persisted-reads-seeds` crosses read holds with seeds and the cached row and +aborts a buffered commit's signal, which abandons it at the read's end +(decision 21); a seed that arrived after it stays judged against it +(decision 22). Dependent suffixes of an abandoned transaction belong to the +abort-graph oracle. The peer is the driver, not a second +Collection, so the claim stops at the controlled coordinator; a second +production driver with two real coordinators is the open receiving +witness (issue #2272). Missed notices and notices over the targeted key limit are not +generated; the persisted oracle's grids cover them. Two held-cleanup +Query retention witnesses race on CI; the maintainer deferred the race until +a failure reports the timer trace these witnesses now print, and the review +record's "Open" section holds the evidence. Restart and resume reads belong to the +SQLite resume-snapshot and Electric resume owners; this oracle's claim ends +with one sync run. ## Deferred contracts and evidence The maintainer assigned offline policy work to diff --git a/docs/contributing/oracle-reviews/2026-10-22-sync-metadata-session.md b/docs/contributing/oracle-reviews/2026-10-22-sync-metadata-session.md new file mode 100644 index 0000000000..036de3f8d9 --- /dev/null +++ b/docs/contributing/oracle-reviews/2026-10-22-sync-metadata-session.md @@ -0,0 +1,1100 @@ +# Sync metadata session: borrowed laws and independent transactions + +Base revision: `30175be48` (`main`). Opened by +[issue #2220](https://github.com/TanStack/db/issues/2220). Every lane below +is RED on the base revision. The same pull request repairs production; see +"Production repair" for the evidence block. + +## Decisions + +The maintainer made five decisions on 2026-10-22; the fifth replaced an +encoding the author had extended without asking. Three more followed on +2026-10-23; the sixth withdrew the second: + +1. **Borrow standard theory.** "this is a clear case where we borrow laws"; + "Generally we want to follow standard theory here". One sync run is a + session. +2. **Closed nesting (withdrawn by decision 6).** "closed nesting is what the + api suggests and what we're partially enforcing so let's go with that". A + sync transaction begun while another is open was to be a closed-nested + child of the innermost open one. +3. **Read your writes in its standard form.** A read includes the source's last + surviving write: it returns that write or a write ordered after it. +4. **Seeds are older than every source write.** A hydration seed never + supersedes a source write to the same object, whatever their arrival order. +5. **D5: a source insert replaces a seed, in either order.** The first K4 + question asked about a seed arriving while the source's open transaction + had inserted the same row. The maintainer answered "yeah seeds are older". + The author then extended that answer, without asking, to seeds arriving + after the insert committed, and encoded it as timestamp order: the + source's insert of a seeded row was an update, so the seed's metadata and + fields survived it. A fresh adversarial reviewer showed that this reading + went beyond the question asked, broke the monotonic-reads transfer (a read + changed from absent to the seed's metadata with no new source write), and + adopted a partial merge of stale fields that review R9 of 2026-10-08 had + called speculative. The author put two arrival-independent readings to the + maintainer: A, a seed as a base layer that the source's insert merges + over; and B, a source insert that names the whole row and its metadata + and so replaces any seed of that row. The maintainer asked how a seed could + arrive after the insert. The realistic case is a `HydrationBoundary` that + renders on a later navigation, while the collection already syncs the row. + The maintainer chose "yup, b". + + Under B a seed supplies only objects that no surviving source write has + written. A source insert replaces a seeded row and its metadata without a + duplicate-key error; an update merges over it; an insert equal to a row + only a seed supplied is an insert, not an equal re-insert. + `docs/guides/ssr.md` ("Initial Data") is revised to say so. + + A third review round found that D5 implied two answers when a seed + arrives after a partial source update of the same row: seeds beneath + field by field (arrival-independent; a late seed can drop a field under + a live row), or ignored once the source has written the row (a partial + update merges over the row as it stood). The maintainer chose the second + ("b"). A seed that arrives after a surviving source write to an object is + ignored for that object; only a partial update's merge base depends on + arrival order. Metadata is unaffected, because metadata writes name the + whole object. The seed-order property therefore compares consumers' rows + only under full row updates. The glossary gains "hydration seed". +6. **Independent transactions (2026-10-23).** A first closed-nesting repair + passed every oracle in this record and broke eight existing tests in three + files: `collection-truncate.test.ts`, `collection-sync-reentrancy-oracle.test.ts`, + and `includes-collection-oracle.property.test.ts`. Each relies on a + transaction begun inside an open one committing and applying on its own. + Handler write-backs, change listeners, and live-query facades share the + sync API with the source, and the API cannot tell their `begin` from a + source's nested one. Under closed nesting a handler's confirmation would + merge into the source's open transaction, and a handler that awaited its + receipt could deadlock. The author put the conflict to the maintainer, with + B (independent transactions) recommended. The maintainer answered: "yeah B + is fine -- I don't have any real use case for nested transactions to begin + with and you're right that to add this properly we'd need clear transaction + ids to support the nesting". + + Every sync transaction is therefore independent. `write`, `truncate`, + `commit`, and abort act on the innermost open transaction. Accepted + transactions apply in commit order, and a hold delays application without + changing that order. A read orders the session's writes by commit: accepted + transactions in commit order, then the open transactions in begin order. An + abort discards only its own transaction's writes. An inner transaction + still reads the open writes of the transactions it began inside; the + author's proposal said so, and the maintainer accepted it. +7. **Cover every nested history (2026-10-23).** The first encoding of + decision 6 excluded an inner row write or truncate that conflicts with an + enclosing open transaction's row writes. The maintainer asked to "cover + everything with the grammar". The model now judges those histories. An + open transaction's inserts follow the rows before them in commit order; + a commit fixes their kinds. An outer insert that lands on a different row + the source wrote makes its transaction invalid, as a duplicate-key insert + would. No read includes an invalid transaction, it ignores later writes, + and its commit rejects. Two rules came from existing authority, not from + the maintainer. Later writes cannot revive an invalid transaction, a + truncate included: core's own comment says so ("later writes cannot + revive it"), and core's `truncate` contradicted it. A seed accepted after + an insert was written cannot change the insert's kind, because a seed is + older than every source write (D4). +8. **Cached rows are seeds, and wrapper inserts replace (2026-10-23).** The + persisted wrapper wrote every source insert to core and storage as an + update. The SQLite persistence plan (`PERSISTNCE-PLAN-SQLITE-ONLY.md`, + 2026-02-10, §D) gives the reason: a remote snapshot that overlaps the + local cache must not raise `DuplicateKeySyncError`. Under partial row + updates the update also merged a cached row's omitted fields under the + insert. No document argued for that merge; the test that pins it + (#2037) calls it pre-existing behavior. The maintainer chose option C: a + cached row is a snapshot older than every source write, like a hydration + seed, so a source insert replaces it and its metadata. While the change + was being implemented, three existing persisted owners showed that the + wrapper's inserts also replace rows the source itself wrote earlier in the + run (the hydration FIFO and hydration-failure tests, and the persisted + optimistic-history oracle). The maintainer then chose C1: in the wrapper, + an insert replaces whatever row it lands on, never raises a duplicate-key + error, and never invalidates its transaction. Its metadata still follows + the 2026-10-05 rules: it clears unless it is an equal re-insert of a row + the source wrote. Core keeps duplicates and invalidation. +9. **Cached rows are seeds in core too (2026-10-26).** The prep-PR review + found that a later `DbClient` seed could not replace a persisted cache + row: the wrapper loaded cached rows into core as ordinary adapter work, so + core counted them as source rows. The maintainer confirmed that cached + rows are seeds there too ("yeah they're seeds too"). A cache load + happens at preload, so a `DbClient` seed arrives after it and wins, as a + later seed does. The same review confirmed the metadata law that a kept + value is not a write: an equal re-insert keeps metadata as an update + does, so the latest seed still governs it until the source writes it. +10. **A merged write can carry a later seed into an insert's kind + (2026-10-26).** Decision 7 said a seed accepted after an insert was + written cannot change the insert's kind. Decision "b" lets that seed + change a partial update's merge base. A random persisted campaign found + the two meeting: an outer equal re-insert, then a later seed, then an + inner partial update that merges over the seed and commits ahead. Core + and the wrapper re-judged the re-insert against the row the inner update + left, which differs, so core invalidated the outer transaction and the + wrapper cleared its metadata; the model kept the re-insert. The + maintainer chose production's reading ("2"): an insert's kind follows the + row it lands on. A later seed alone does not change it, and neither does + an open transaction's own write over the seed, but a transaction that + commits after the seed and merges over it does. +11. **A peer's write in this run is a source write (2026-10-26).** A + second high review found that rows a peer's source wrote in this run, + which reach this tab through storage, were treated as seeds or skipped + behind a `DbClient` seed. The maintainer adopted the recommendation that + follows D4: such a row and its metadata are source-written for this + session. It replaces a `DbClient` seed this tab holds, a later seed does + not supersede it, and an equal re-insert of it keeps metadata. The rule + of `origin/main` that "DbClient hydration seeds remain source-owned until + confirmed by the source" now yields to a peer's notice of a write in + this run; rows from a previous run stay seeds (decision 9). Eighteen + persisted-oracle cases that held the seed against a peer's targeted + notice were rewritten to this rule. Asked about them, the maintainer + answered "LWW is a good general heuristic here": among source writes, + the last one wins, and a seed is older than every source write (D4). +12. **A source transaction may stay open across a storage read + (2026-10-26).** The new read-hold lane found that a commit threw "a + persisted sync transaction cannot cross a hydration cycle" when an + on-demand storage read started while the transaction was open. The + maintainer chose to support it: a transaction binds to a storage read + when it commits, not when it first writes, and writes during a read + follow the same metadata rules as other writes: the last write for a key + wins, so an insert without metadata clears an explicit write before it + and a cached row's metadata. The wrapper's earlier rules, that such an + insert kept an explicit write before it (on every path) and kept the + metadata of rows the read supplied, and that a commit crossing a read + threw, are retired; ten persisted-oracle cases were rewritten. +13. **Unknown authorship keeps what this tab holds (2026-10-26).** After a + full-reload notice, a missed notice, or a truncate, this tab cannot tell + which stored rows a source wrote in this run. The third high review + asked which way that should cut. The maintainer first chose to count + every such row as written, then, shown that a full reload would then + delete or roll back every held `DbClient` seed that storage lacks or + holds older (the persisted oracle's `keeps a DbClient hydration seed + across … peer notice` cases), chose the rule the code already followed: + a seed this tab holds stays, and a stored row this tab did not hold + counts as a source write, so a later seed cannot replace it. A peer's + write to a key this tab holds as a seed stays hidden until a notice + names the key or the source writes it. Decision 15 extends the rule to + every stored row. +14. **A commit fixes its inserts' kinds as it read them (2026-10-26).** The + fourth high review asked whether an inner transaction's insert, judged + against an enclosing open transaction's write it read, should be judged + again against the row it lands on when it commits ahead. The maintainer + kept the model's rule: a commit fixes its inserts' kinds in the order as + the transaction read it, so an inner equal re-insert of the enclosing + row keeps metadata and replaces the row it lands on. `30175be48` already + let an inner insert over an enclosing open truncate replace the row it + lands on. Decision 10's "an insert's kind follows the row it lands on" + concerns the inserts of transactions still open. +15. **Under unknown writes, only a changed row is a source write + (2026-10-26).** Decision 13 left the code's two paths disagreeing about + stored rows when this tab cannot tell what a source wrote: a reload + counted every stored row and its truncate as source writes, so later + seeds were ignored for the rest of the run, while a validation read left + unchanged rows alone. The maintainer chose the second: a stored row that + differs from the one this tab holds was written; an unchanged seed stays + one; a dropped row is a source's delete; and a reload's truncate is not a + source truncate. A reset still is. +16. **Row metadata of a missing row stays until a source deletes the row + (2026-10-27).** The peer lane found `begin rowSet(2) commit + peer(1,row)`: SQLite stores row metadata on its row, so metadata a source + writes for a row that does not exist is not durable, and a reload + deleted it. Whether a notice reloaded decided whether it survived. The + maintainer chose option C: this tab keeps such metadata for the rest of + the run unless a source deletes the row; a restart loses it, as it was + never durable. Three persisted-oracle cases that pinned the deletion + (`… orphan row metadata after a no-demand peer reload`, twice, and the + orphan check of `does not republish unchanged rows for a repeated + reconciliation notice`) were rewritten. +17. **A write over a `DbClient` seed stores the row this tab shows + (2026-10-27).** `seed(2) begin upsert(2) commit peer(1,row)`: seeds never + reach storage, so an update merged over a seed stored only its own + fields, and a reload or restart dropped the seed's fields. The + maintainer chose option B: the first update a transaction makes to a row + a `DbClient` seed holds stores the seed's row merged with the update. +18. **A reload after a named notice keeps a seed's metadata (2026-10-27).** + `seed(2,narrow) begin upsert(2) commit seed(2) peer(1,row)`: the update + kept the first seed's metadata, and a later seed replaced it in this tab + (decision 9), but storage cannot follow a seed, so a reload put the + first seed's metadata back. The maintainer chose option B: metadata no + source wrote in this run stays as this tab shows it across reloads, and + decision 17 stores only the row. Asked whether that holds after a + full-reload notice too, where two `main` cases expect stored metadata to + win, the maintainer chose the scoped form, B′: decision 18 governs + reloads after notices that name their keys; when this run's writes are + unknown, stored metadata that differs is a source write (decision 15). +19. **A reset's treatment of held seeds depends on the sync mode + (2026-10-27).** M4 found that an on-demand reset beside a held + `DbClient` seed skipped its truncate, so metadata of a missing row and + rows a hold kept from `base` survived it. Asked which rule governs held + seeds, the maintainer chose the mode-specific rule after a corrected + premise (eager resets already truncate first): eager, a reset is a plain + source truncate, seeds included; on demand, it keeps held seeds' rows and + metadata until the replayed demand replaces them, so one publication + replaces them without an empty view. +20. **An on-demand reset is a source truncate that keeps held seeds + (2026-10-27).** The new lane then found a seed of row 2 that arrived + after an on-demand reset beside a held seed of row 1: production + applied it, because the reset deleted only rows it saw, while with no + held seed the reset truncated and rejected the same seed. A reset that + skipped its truncate also skipped the demand replay that the truncate + drives. The maintainer chose a source truncate that exempts held seed + keys: they keep their row, metadata, seed status, and seed authority; + every other key is cleared and closed to later seeds; demand replays. + A first answer ("seen objects only") was withdrawn after a corrected + premise. + +21. **A persisted Collection accepts a source transaction when it hands it + to core (2026-10-28).** H2 needed a late abort: a commit whose signal + aborts while the wrapper still buffers it behind a storage read. + `SyncConfig.commit` says that aborting after acceptance has no effect. + The maintainer first chose that text, but it failed 16 cases of the + abort-graph law ("abort graphs preserve independent siblings and reject + dependent suffixes") and its witnesses, which the question had not + named. Asked again, the maintainer kept the abort-graph law: the + wrapper's acceptance is its handoff to core, so a signal aborted before + then abandons the transaction, its receipt rejects, and source reads + include it until then. +22. **A seed is judged when it arrives (2026-10-28).** The reads-and-seeds + lane found a seed that arrives after a buffered write to its row, which + is then abandoned. The glossary's "surviving source write" would revive + the seed; production ignored it for good. The maintainer chose judging at + arrival, for less code in a tiny window: the seed stays ignored for that + object. The glossary's hydration-seed entry says so. + +## Laws and sources + +| Law | Source | Transfer to a sync run | +|---|---|---| +| Read your writes | Terry et al. 1994, §3.1 | Every read includes every accepted write of the run and every write of its open transactions, ordered by commit: accepted transactions in commit order, then open transactions in begin order | +| Monotonic writes; one current value | Terry §3.4; Moss 1981, p. 61 | A transaction's writes take effect at its commit. The last write in commit order, issue order within a transaction, is the value, for reads and for what applies | +| Monotonic reads | Terry §3.2 | The writes a read includes only grow, except at an abort. When an inner transaction commits, an enclosing open transaction's write to the same object is ordered after it, so a read can change without a new write | +| Own reads | Adya 1999, §3.1.2 | A transaction reads its own writes; the innermost open transaction's writes come last in the order | +| Aborted reads | Adya §3.1.2, G1a | Not guaranteed across open transactions: an inner transaction reads an enclosing transaction's open writes and can commit before that transaction aborts (decision 6). No read after an abort includes the aborted writes | +| Invalid transactions | decision 7; core's invalidation comment | A transaction whose insert lands on a different source row, in commit order, is invalid: no read includes it, later writes are ignored, and its commit rejects. Where inserts replace (the persisted wrapper, decision 8), the insert replaces that row instead | +| Termination order | the sync API | `commit` and abort act on the innermost open transaction; the API has no transaction handle | +| Monotonic atomic view | Bailis et al. 2014 (HAT), Def. 27 | Once a consumer observes any effect of a transaction, it observes all of its effects thereafter, across both stores. The comparison is stronger: `collection.base` must equal the model at every checkpoint | +| Applied receipt | `SyncConfig.commit` docs; glossary "applied receipt" | A receipt settles when its transaction applies | +| Sticky availability | HAT §5.1.3 | The persisted wrapper and core form one session | +| Thomas's write rule | Thomas 1979; Bernstein, Hadzilacos, and Goodman 1987, §4.2 | A seed's snapshot is older than every source write, so it is ignored for any object a surviving source write has written, in either arrival order | + +Writes-follow-reads and PRAM follow from read-your-writes, monotonic reads, +and monotonic writes under commit-order apply; the oracle does not check them +separately. Two transfers are excluded. Session guarantees are per session, so consumers +(`collection.base`) keep the applied view. Adya allows version order to differ +from commit order; TanStack DB chooses commit order. + +The research survey read the primary texts locally and verified 13 +load-bearing quotes against the text or page images. The Thomas's-write-rule +row names the textbook rule; those texts were not read for this record. Moss is the only primary +source on nesting. Lynch, Merritt, Weihl, and Fekete's formal treatment was +not accessible. + +## Executable owner + +`packages/db/tests/sync-metadata-session-oracle.ts` holds the contract, an +event-log model, the history grammar, the driver contract, and the +comparison. Drivers: + +- `sync-metadata-session-oracle.property.test.ts`: a `DbClient` Collection; + lanes `flat`, `nested` (depth ≤ 3), `seeds`, `nested-seeds`, `seeds-full` + (`rowUpdateMode: 'full'`), and `nested-conflicts`, which favours row + writes to one row inside nested transactions so that inner writes land + ahead of enclosing ones and transactions become invalid. +- `db-sqlite-persistence-core/tests/sync-metadata-session-persisted.test.ts`: + `persistedCollectionOptions` over an in-memory adapter, with store and core + holds; lanes `persisted-flat`, `persisted-nested`, `persisted-seeds`, + `persisted-cached` (row 1 is in storage when the run starts; the model + holds it as a seed accepted before any action), `persisted-conflicts`, + `persisted-nested-seeds`, and `persisted-reads`, whose read hold keeps an + on-demand storage read in flight while the source writes and commits + (decision 12). The last crosses `DbClient` seeds with nesting + and the cached row, with the conflict lane's weighting; its + `keepsMetadata` flag favours writes that keep metadata over writes that + clear it, without excluding any action, so a later seed often meets + metadata that an accepted re-insert kept. Every persisted lane declares + the one model difference of decision 8: an insert over a different row + replaces it. + +The row-metadata composition oracle keeps its base nested lane, which +already judged independent transactions: a transaction begun inside T1 +applies first, and T1's inserts are reclassified against the rows it leaves. +This change adds a `held` variant of that lane (a hold cannot change commit +order) and applies D5 to its seed variant. Its held-window lane, added +earlier for this issue, stays. + +The grammar generates every nested history (decision 7). The key +independence property skips histories in which the model invalidates a +transaction: invalidation is per transaction, so a row 2 conflict also +removes row 1's writes. That is atomicity, not a cross-key leak. + +## RED evidence on `30175be48` + +| Witness | Law | Observation | +|---|---|---| +| `hold, begin, collectionSet(cursor), commit, begin` | Read your writes | cursor reads absent in the open transaction | +| `hold, begin, rowSet(1), collectionSet(gc:a), commit, begin` | Read your writes | row and prefix list miss the held writes | +| `hold, begin, rowSet(1), begin, rowSet(1), commit, commit, release` | Commit order | the held inner commit stays after the open outer transaction: the read returns the inner write, and the release applies the outer write first, so the inner write wins | +| `hold, begin, begin, rowSet(1), commit, rowSet(2)` | Termination order | the outer transaction's write throws `SyncTransactionAlreadyCommittedWriteError` | +| `begin, rowSet(1), commit, seed(1)` | Seeds are older | the seed's metadata replaces the source's metadata-only write | +| persisted: `hold(core), begin, upsert(1), commit, begin, commit, begin, collectionSet(cursor), commit` | Read your writes | row 1's metadata reads absent; the store-hold neighbour passes | +| persisted: `hold(core), begin, collectionSet(cursor), commit, begin, upsert(1), commit` | Read your writes | the cursor reads absent once the wrapper has handed both transactions to core; the store-hold neighbour passes | +| persisted: `begin, rowSet(2), upsert(2)` | Own reads | the insert's implicit clear is not visible in the open transaction | +| `begin, upsert(1), seed(1), commit` | A source insert replaces a seed (D5) | production turns the open insert into an update over the late seed, so the seed's metadata survives the insert's clear and its other field survives in the row | +| `seed(1), begin, reinsert(1)` | A source insert replaces a seed (D5) | core treats the insert as an equal re-insert and keeps the seed's metadata; the persisted wrapper clears it, as D5 requires | +| composition nested lane, `immediate` and `held` | D5 seed variant; commit order under a hold | both variants fail; the held variant also applies the open outer transaction before the inner one | +| retention: `keeps an accepted insert when it commits ahead of an open truncate (held: true)` | Commit order | the inner insert applies after the outer truncate, so row 1 survives | + +Witnesses written for closed nesting and then revised for decision 6 pass on +`30175be48`, which already applied independent transactions in commit order +when nothing was held: `outerAbortKeepsInnerCommit` and +`innerTruncateAppliesAtItsCommit`. The run against the base production files +(`sync.ts`, `state.ts`, `client.ts` from `30175be48`, with this branch's +tests) fails 51 of 186 tests in the session, composition, and retention +files. + +## Calibration + +- A temporary commit-order reader (walk every pending transaction newest + first, stop at a truncate for rows, merge `list()` oldest first) made the + core flat lane and both issue witnesses GREEN in three runs. The same edit + left the nested and seed lanes RED, as their laws require more than a reader + change. The edit was reverted. +- With that reader, a mutant that ignores the truncate stop fails the + composition held-window lane's truncated variant, and a `list()` that merges + only the newest transaction fails both held-window variants. +- A bounded exhaustive model check (outside the repository) found that + readers which omit ancestors or read only the newest pending transaction + violate the laws. It also judged today's independent nested apply against + closed nesting; decision 6 superseded that part. +- The campaign reach test requires each lane to reach its declared states; + "Production repair" lists the counts. + +- A mutant gap hunt against the commit-order reader baseline, in a separate + worktree: walking row reads oldest first, ignoring the row truncate stop, + and merging `list()` newest first fail at the comparison. Walking + collection reads oldest first fails both flat campaigns but survives the + named witnesses and the composition held-window lanes. Returning a delete + write's missing value is equivalent: a delete carries no value. +- Adding the receipt observation kept the core flat lane GREEN with the + commit-order reader in three runs. +- Seeds. Under the first K4 encoding (reading A), `begin, upsert(1), + seed(1), commit` passed on production, and a mutant that derived the + implicit metadata write from the original `insert` failed it. Under D5 + the expectation reverses, and that witness is RED. While encoding A, the + model briefly judged the insert's row presence after its own row write, + which turned the wrapper's implicit-clear witness falsely GREEN. Under D5 + the clear is an ordinary write again, and the witness is RED. +- Paired formulations. Key independence and seed order fail on `30175be48` + without consulting the model's reads. + - Key independence, seeds lane: `hold, seed(1), seed(2), begin, cancel`. + Row 1's held seed reads absent once row 2's seed follows; without + `seed(2)` it reads as seeded. This is the issue's newest-transaction read + in a cross-key form. + - Seed order: a seed that arrives after a source's metadata write replaces + it, while the same calls with the seed first keep the source's write. + Under reading A, a reviewer found that a reordered run could throw on an + equal re-insert, a false positive. Under D5 the insert replaces the seed + in both orders, so the re-insert stays equal, and a throw would be a D5 + violation. + - Calibration: with the commit-order reader, flat key independence passes. + A mutant reader that stops at the newest transaction touching any row's + metadata fails it on both flat campaigns. Nested key independence fails + on the known closed-nesting throw. No correct production exists yet to + calibrate seed order GREEN; with every seed already first, its two runs + perform identical calls. +- Range (ORC-004). Two edge cases from outside the design work were added. + - Collection keys `gc:` (equal to the prefix) and `gc` (a proper prefix of + it) needed only new parameter values. With the commit-order reader, the + core flat lanes and both issue witnesses stayed GREEN in three runs. The + probe exposed one oracle artifact: the model listed keys in declaration + order while the drivers sort them, which only matched for the old key set. + The model now sorts. + - A seed that carries a field the source never writes exposed a missing + rule: under `rowUpdateMode: 'partial'`, a source update merges over the + row before it, a seed included. The model now merges updates; an insert + replaces the row (D5). `updateOverSeed` checks the merge and + `updateOverSeedFullRows` checks replacement under a `seeds-full` lane; + both pass on production. + +## Production repair + +The repair keeps the existing structures and changes these rules. + +- **Reads.** `metadata.row.get`, `metadata.collection.get`, and `list()` + walk every pending transaction, newest first, then applied state; a + truncate hides older row metadata. The persisted wrapper reads every open + transaction, innermost first, then its staged transactions, then core. +- **Commit order.** `acceptSyncTransaction` places a committed transaction + after every accepted one and ahead of every open one; seeds use the same + method. An inner commit during a hold therefore no longer leaves the queue's + last transaction committed, so the outer transaction can still write and + commit, and a release applies in commit order. A replay keeps an accepted + transaction's operation kinds: an inner insert written over an outer open + truncate replaces the row it lands on, as `30175be48` already did when + nothing was held. An open transaction's inserts still follow the rows before + it and can make the transaction invalid. +- **Seeds (D4, D5).** Row data and row metadata are judged separately against + committed adapter work, including a wrapping sync's staged committed + transactions. A seed replaces earlier seeds whole. A source insert over a + row only a seed supplied is an insert; over a seeded row the source also + wrote, it stays the re-insert the source wrote, so a later seed beneath an + open update cannot make it a duplicate. An insert names its whole row even + as an equal re-insert. A held seed's receipt has a rejection handler. +- **Wrapper metadata.** An insert without metadata stages a clear unless it is + an equal re-insert of a row the source wrote; the last write for a key + wins. +- **Query.** `getRowMetadata` no longer falls back to applied metadata, which + read a staged clear as "no value". +- **Invalid transactions (decision 7).** Core's readers skip a transaction a + replay invalidated, and `write` and `truncate` ignore it; `truncate` no + longer clears the invalidation. Its metadata writes need no check: no read + includes it, and it never applies (a mutant that lets them through is + equivalent). +- **Seed authority across a hold.** The replay projection records which keys + accepted source transactions wrote. An accepted source write makes a row + the source's, as its application does, so a seed no longer exempts that + row while the transaction is held. Before, an inner commit during a hold + kept the outer transaction valid, and the same history without the hold + invalidated it. +- **Wrapper inserts (decisions 8 and 9).** A source insert replaces the row + it lands on. The wrapper judges its metadata when the source writes it and + hands it to core as an update that names the whole row + (`originalSyncType: 'insert'`, core's own form for an equal re-insert), so + core replaces the row and only the wrapper's metadata writes change its + metadata: an insert that clears staged a delete, and an equal re-insert + keeps the value without writing it. Before, the wrapper wrote the kept + value back as explicit metadata, which made the source its owner, and a + delete then the insert over a different row. In storage it is an `insert` + mutation, which the SQLite adapter upserts. + The wrapper records the keys its source transactions handed to core in this + run; any other row came from the cache or a seed, so an insert equal to it + still clears metadata. After an inner commit, the wrapper re-judges each + enclosing transaction's implicit metadata clears on the keys the inner + transaction wrote, as core's replay does, and an inner re-insert keeps its + metadata when it lands where the row is absent. The wrapper keeps, per key, + every insert without metadata since the last write that named the key's + metadata, and re-judges them all: if any no longer re-inserts the row it + lands on, the key's metadata clears; otherwise the write before them + stands. An update without metadata keeps that judgment open, because the + inserts before it still decide the row's metadata. +- **Duplicate inserts.** A seed exempts a source insert from the duplicate + check only over a row that only seeds supplied; over a row the source + wrote, it is judged like any insert, whatever holds or seeds there are. In + the replay, a seed beneath an open insert of a row the source also wrote + keeps that insert's kind. +- **Cached rows (decisions 9 and 11).** A storage read marks a row it + applies as a seed unless core holds it as a source row or a source wrote + it in this run, which the runtime records from this tab's handoffs and + peers' notices (a truncate or a full-reload notice makes every stored row + count as written). The read stays one transaction; core keeps seed status + only for its seed keys, and every other write of the read, a validation + delete included, is adapter work. A `DbClient` seed holds against stored + rows only until a source writes its key in this run. +- **Writes during a storage read (decision 12).** They follow the ordinary + metadata rules, and a transaction binds to a read when it commits. + +### Evidence block + +- **Law:** read your writes, monotonic writes in commit order, own reads, + monotonic atomic view, applied receipts, invalid transactions, and D4/D5 + seed authority, as stated above, with the wrapper difference of decision + 8. Authority: decisions 1 and 3–8. +- **Reach:** declared with `createOracleReach` in each fixed campaign + (ORC-015); a campaign fails below its minimum. Core lanes: 11, or 5 for + states that need a seed and a source write on one row, or a write after an + invalidation. Persisted lanes: 5. The prep-PR round added the states + `duplicate source insert` (and its hold and seeded forms) and `seed under + a kept re-insert`, and raised the seeds and nested-conflicts campaigns to + 150 and 450 histories; every fixed campaign meets its minimums. Counts in + the fixed campaigns before that round: + + | State | flat | nested | seeds | nested-seeds | seeds-full | nested-conflicts | + |---|---|---|---|---|---|---| + | read with two held transactions | 18 | 37 | 16 | 20 | 12 | 65 | + | open transaction over a held transaction | 51 | 61 | 36 | 47 | 30 | 136 | + | truncate during a hold | 30 | 31 | 22 | 24 | 15 | 69 | + | nested commit | | 59 | | 52 | | 158 | + | nested commit during a hold | | 43 | | 36 | | 107 | + | abort keeps an inner commit | | 11 | | 9 | | 24 | + | inner write to an enclosing row | | 7 | | 10 | | 92 | + | transaction invalidated | | 2 | | | | 28 | + | commit of an invalid transaction | | 2 | | | | 15 | + | write to an invalid transaction | | 1 | | | | 16 | + | seed under a source insert | | | 37 | 27 | 24 | | + | seed under an open update | | | 10 | 5 | 7 | | + + `nested-conflicts` runs 300 fixed histories; the others run 100. Persisted + lanes: `persisted-conflicts` (150 histories) reaches an insert that + replaces a different row 18 times and an inner write to an enclosing row + 50 times; `persisted-cached` (80) reaches a source insert over the cached + row 25 times and a source update over it 14 times. +- **RED:** + - `30175be48` production with this branch's tests, as listed under "RED + evidence": 51 of 186 tests fail in the session, composition, and + retention files. + - Decision 7, on the production before its repair: 7 failures in the + session oracle (reads include an invalid transaction's staged metadata + and a collection write made after its invalidation; a hold changes + whether a seed exempts a row). Two more were model and grammar faults, + fixed before the production change: key independence compared histories + that invalidate a transaction, and the model kept a transaction's row + writes from before its own truncate. + - Decision 8, on the wrapper before its repair: the `persisted-cached` lane + fails because a source insert merges over the cached row and keeps its + `seeded` field; the persisted lanes fail on nested conflicts. +- **GREEN:** `packages/db`: 286 files, 13,917 tests pass; the session oracle + file passes 78 of 78 twice. `packages/db-sqlite-persistence-core`: 33 + files, 1,732 tests pass (2 skipped, 2 todo); the persisted session file + passes three runs in a row. Query DB, node, browser, and Electric suites + pass with their receiving expectations updated for decision 8. +- **Mutant:** each fails the oracle. + - Commit in place (no commit-order move): 17 failures, including + `commitOrderDuringHold`, `outerContinuesAfterInnerDuringHold`, nested hold + invariance, and the nested campaigns. + - Replay reclassifies accepted transactions and throws on a duplicate, as + `30175be48` did: both variants of the retention witness. A variant that + marks the accepted transaction invalid without throwing changes only the + internal projection and survives; no public checkpoint observes it. + - Seed exemption only for seed-only rows: `seedUnderOpenReinsert`. The fixed + campaigns do not catch it; random campaigns did, twice, before the witness + existed. + - An equal re-insert that merges under partial row updates instead of + replacing its row: `seedUnderOpenReinsert`. + - Readers that include an invalid transaction: the nested and conflict + campaigns. A truncate that revives an invalid transaction: the nested and + conflict campaigns and the conflict lane's reach. + - Wrapper, earlier rules: reading only the innermost open transaction; + keeping an earlier explicit write over an insert's clear; clearing on + every insert; hiding staged transactions from seed authority. Each fails + at least one persisted lane. + - Wrapper, decision 8: an insert that merges over a held row (12 persisted + hydration cases and the hydration FIFO test); storage that receives + inserts as updates (the sync-config and hydration cases); a cached row + counted as source-written (`persisted-cached`); enclosing implicit clears + left unjudged (`persisted-conflicts`). An inner re-insert that loses its + kept metadata survives every campaign; the pinned witness `keeps an inner + re-insert's metadata when it commits ahead of the row it re-inserted` + kills it. + - Decision 9 (prep-PR round): the original handoff on `743a6ce14` fails + all four new witnesses and `persisted-nested-seeds`; writing the kept + value only when core would not keep it fails the inner re-insert + witness; loading cached rows as source rows fails the cached-row witness + and `persisted-nested-seeds`; a core that ignores a wrapper's + `originalSyncType` merges over seeded rows and fails `persisted-seeds` + and `persisted-cached`; dropping the insert's judgment at a later update + fails `rejudges an enclosing re-insert that a later update kept`; re-judging + only a key's last insert fails the two chain witnesses. +- **Limits:** under partial row updates the seed-order property drops + `reinsert` from both runs, because decision "b" lets a moved seed change a + re-insert's planned row. Key independence skips histories that invalidate + a transaction. The `persisted-reads` and `persisted-peers` lanes' adapter + keeps durable rows, so storage reads and reloads within a run are + exercised there; the other persisted lanes' cached row is readable only + until the run's first durable commit. Since the eighth round, a peer is a + grammar dimension of `persisted-peers`, played by the driver through the + coordinator (see "Eighth round"). The error an invalid commit rejects + with, and restart and resume, stay outside this oracle. + +### Code weight + +Final head `d1166fe6f`, against the merge-base `4c39add29`: `app` grows +232 bytes gzip, `@tanstack/db` 523, `db-sqlite-persistence-core` 1,331, and +`query-db-collection` shrinks 21. Each fix commit since the eighth round, +measured against its parent: + +| Commit | `app` | `db` | persistence | +|---|---|---|---| +| `d3847dbb6` notice-carried reload | 0 | 0 | −6 | +| `e4fb7a71d` decisions 19 and 20 (reset truncate, kept seeds) | +76 | +134 | +76 | +| `3b1973276` per-key unknown writes | 0 | 0 | +151 | +| `18f7df562` drop read knowledge from unknown writes | 0 | 0 | −30 | +| `4b6081698` H1, H2, H5, H6, decisions 21 and 22 | +28 | −15 | −31 | + +The rest of the change from the eighth round's figures comes from the merges +of `origin/main`. The `app` growth since the eighth round is the source +truncate that a reset opens (`sourceTruncate`, `truncatesSourceKey`) and +the H1 reader check that uses it. The H5 and H6 dead-code removals paid for +the H2 move of `noteRunWrites` after the commit. + +Eighth round, against `7e46f81ad`: `app` grows 125 bytes, 20 fewer than +before, because `_syncedRowMetadataKeysOutsideBase` left the Collection +class; the staging check for preserved keys and the metadata reader's check +for a status-keeping truncate cost less than it saved. Persistence reaches ++1,206 (204 more): the stored-row status rule, whole-row reload writes, the +seed bases that decision 17 stores, and the held-row checks. + +`pnpm size:compare` against `427a7c31e` (`origin/main` after the second +merge): the typical `app` scenario grows 126 bytes gzip. The second high +review's repairs cost `app` 9 bytes: the internal whole-row marker and +readers that skip invalidated transactions in place. The marker is a bare +`Symbol()`, because its `Symbol.for` description string alone cost 31 +bytes and the wrapper imports the symbol from `@tanstack/db`. Seed status +became a key set on every seed transaction (a `DbClient` chunk passes its +keys), so per-key storage reads cost no extra check. `@tanstack/db` as a +whole package grows 375, `db-sqlite-persistence-core` 849, and +`query-db-collection` shrinks 14. In persistence, the second high review +went from 876 to 699 because removing the operation walk and the +hydration-deferred metadata path paid for its rules; the third added 150 +for the per-object record of writes in this run, judging open inserts +again after internal writes, and the `begin` guard. `app` did not change +in the third. + +The seventh round: against `6c1ca8f8f`, `app` grew 124 bytes, two fewer +than before, because the projection's accepted-source key set became a local +of its rebuild, and persistence 933 (84 more: the internal apply's written +keys and the first reload repair). Against `b8d8ee07c`, after two merges, +the pushed head measures `app` +135 and decision 15 adds 10 more (+145): +core's truncate keeps authority and seed status for a transaction that +marks seed keys. Persistence reaches +1002 (69 for decision 15: the shared +stored-seed rule and the dropped rows' deletes). A first repair that +validated reloaded rows in place, with no core change, failed the three +exact-shape witnesses, because only a truncate republishes rows that change +events consider equal. + +Earlier rounds: the private-member mangle cache now +covers the branch's new private methods. Removing Query's owner supplement +saved 9 bytes, and tracking peer writes and judging enclosing inserts again +after an abandoned transaction added 91 to persistence. + +The prep-PR round changed these numbers. Decision 9 costs core 12 minified +bytes (a wrapper's `originalSyncType` passes through `write`), paid for by +the state rebuild that `reset()` now reuses; `app` stayed at +111. The +duplicate-insert repair (B, C) adds 14 bytes to `app`. H first added 19 +bytes as a delete-and-return pair of set updates; merging the two +`DbClient` key sets into one map cut that to 6. In persistence, decision 9 +removed the kept-value lookup and the delete-then-insert handoff and added +the cached seed marking and the per-key chain of implicit inserts (+83); +the per-transaction staged rows that make a write-time read linear (D) add +the rest. Each addition has a witness or mutant that fails without it. + +- Core, paid by every app: the readers' loops over every pending + transaction and their skip of invalid transactions, commit-order + acceptance, the source-key sets that tell a seed-only row from a + source-written one, and the insert-replaces-its-row rule. +- `DbClient` only: per-object seed authority, including a wrapping sync's + staged transactions. It lives in `client.ts`, so an app without `DbClient` + does not ship it. +- Persistence only: the staged-row fold, the re-insert judgment, the record + of implicit metadata clears and their re-judgment after an inner commit, + the source-written key set, and the insert handoff to core and storage. + Each has a mutant that fails a persisted lane or witness. + +Refactors made to pay for it: the seed authority lookups moved out of the +state class into `client.ts` (`app` +240 to +111); the seed acceptance method +became the single acceptance method for commits and seeds; the two seed +authority lookups became one loop; the base invariant throw for an accepted +duplicate is gone; guards on an invalid transaction's metadata writes were +removed once a mutant showed them unobservable; decision 8 removed the +wrapper's invalidation and duplicate-key paths. Two refactors were tried and +reverted because they grew gzip: one shared reader for `row.get` and +`collection.get` (gzip compresses the two loops better than the helper), and +moving the wrapper's deferred-key predicate into core (it moved bytes from +persistence into every app). + +## Review ledger + +Three loss-audit scanners and three oracle reviewers examined this change. + +| Finding | Disposition | +|---|---| +| G1a was cited as permitting child reads | Fixed: the laws table cites relative commit and delayed commit | +| Monotonic atomic view dropped "thereafter" | Fixed: stated, with the stronger per-checkpoint comparison | +| Writes-follow-reads and PRAM uncited | Fixed: stated as consequences, not checked separately | +| Receipt checkpoint for a nested child undefined | Fixed: receipts are an observation; the law follows from the existing receipt contract | +| Adjacent form F2 (nested truncate during a hold) not pinned | Fixed: witness added | +| Adjacent form F3 (consumer sees a seed, then the source) not pinned | Fixed by D5: the order is legal, and `seedUnderOpenInsert` checks it | +| Source and consumer seed layers differ without explanation | Fixed: model prose explains the two layers | +| Reach test does not ablate generator dimensions | Fixed: see "Reach ablation" below | +| Derived checks (hold invisibility, unrelated-action invariance) and the bounded model check live outside the repository | Fixed: three production-against-production properties (hold invariance, key independence, seed order); the bounded model check stays outside | +| Mutant "collection reads oldest first survives" | Refuted on re-run: both flat campaigns kill it | +| The issue's held-cursor history passed through the wrapper | Checkpoint artifact: actions ran back to back, so transactions stayed in the wrapper's staging. With continuations drained between actions (the receipt `flush`), the core-hold variant fails | +| `list()` presence of a key set to `undefined` | Fixed: core lanes store `undefined`; `get` reads it as absent and `list()` keeps it | +| Equal re-inserts not generated | Fixed: `reinsert` action | +| Reach ablation | Fixed: removing each axis removes its regime | +| No second formulation in the repository | Fixed: hold invariance compares production with and without holds | +| Nesting and seeds through the wrapper not driven | Fixed: `persisted-nested` and `persisted-seeds` lanes | +| Stale runs and writes without a transaction | Fixed: boundary witnesses | +| Restart and resume reads | Owned elsewhere: SQLite resume-snapshot and Electric resume owners | +| Truncate across several keys | Covered: the session model has two row keys; truncate clears both | + +## Second review round + +Three fresh reviewers examined the changes made after the first round: an +adversarial review of the seed laws, a mutant gap hunt in a separate +worktree, and a tension scan of the Query witnesses. + +| Finding | Disposition | +|---|---| +| The K4 encoding extended the maintainer's answer to committed inserts, broke the monotonic-reads transfer, and adopted the partial merge that R9 called speculative | Fixed by decision D5 (reading B), asked of and chosen by the maintainer | +| `runSeedOrderInvariance` could throw on an equal re-insert after reordering | Fixed by D5: an insert replaces a seed in both orders, so the re-insert stays equal | +| Seed-order invariance cannot choose between arrival-independent readings | Accepted: the witnesses and D5 carry that choice; the property checks arrival independence only | +| The hydration-authority owner runs only full row updates and no metadata | Fixed: it crosses full and partial modes, adapter inserts and updates with and without metadata, and seeds carrying metadata and an extra field, `initialData` rows with an extra field, and a metadata-only source write; it reads metadata through `client.dehydrate()`. After the third review round, 19 of its 52 grid cells and one pinned production case fail on `30175be48`: an insert keeps a seed's metadata or merges its field (including over `initialData`); a late seed's metadata is dropped after an update that wrote none; a later seed overwrites a metadata-only source write; a second seed merges over the first; a metadata-less later seed keeps the earlier seed's metadata | +| Seed-to-seed merge versus replacement is not distinguished | Fixed: narrow seeds omit the extra field; `laterSeedReplacesSeed` fails on `30175be48`, where a second seed merges over the first | +| Reader mutants for a missing truncate stop, `list()` ignoring a pending delete, and prefix matching that is too loose or too strict died only in generated campaigns | Fixed: the named witnesses `openTruncateHidesHeldRow`, `openDeleteHidesHeldListEntry`, and `prefixBoundary` each kill their mutant and pass with the commit-order reader | +| A full-mode update merged only in the accepted projection survived: no checkpoint observes the accepted row | Fixed: an upsert that names metadata carries a field `m`, so row shapes differ; `fullUpdateThenEqualInsertDuringHold` kills the mutant with a duplicate-key throw at the equal re-insert, and production passes | +| A merge keyed on the original sync type, at the projection only, died only in one random run | Covered by the same witness's observation (the projection's row); not separately re-run | +| The Query retention witness did not separate "drop the fallback" from a full repair, and its prose blamed the fallback | Fixed: prose corrected; a peer-owner witness keeps the row, so the claim is an update and the stale owner comes from the claim's read of applied owners | +| A held removal of a hydrated row, then a claim, should clear the seed's metadata | Fixed: witness added; it fails on production and passes with the fallback dropped | +| Query witnesses read `_state.syncedMetadata` | Fixed where a `DbClient` exists: they read `client.dehydrate()`; the others say why they keep the internal read | +| Mutant: Query owners ignore accepted in-memory edges (`rowToQueries` supplement in `getOwnersFromMetadata`) | On production the new witness `publishes a held owner beside a later claim of its row` fails before the mutant matters. With the commit-order reader and the `??` fallback dropped, the mutant passes all 402 ownership tests. Probably equivalent there: each application that adds an edge also stages the owner set, and the reader then finds it. Open: a history with an in-memory edge but no staged owner write would separate them; none was found. If none exists, the repair may delete the supplement | +| Mutant: Query retires owner tracking before staging the cleanup | Equivalent by source inspection: no read occurs between the early retirement and the commit | +| Mutant: Query never retires owner tracking at cleanup | Killed by a sibling test (`keeps a released row and fails loudly when its release commit fails`), not by the retention witness | + +## Third review round + +Three fresh reviewers examined the D5 encoding, the Query and hydration +tests, and a mutant campaign in a separate worktree. + +| Finding | Disposition | +|---|---| +| D5 implied two answers for a seed that arrives after a partial source update | Fixed by the maintainer's choice "b" (decision 5); the session model places seeds by arrival, and seed order compares rows only under full updates | +| A later seed without metadata keeps the earlier seed's metadata; seeds always carried metadata | Fixed: bare seeds; `bareLaterSeedClearsMetadata` fails on `30175be48` | +| No fresh source insert over a seed-only row | Fixed: `insertOverSeed` action; `insertOverSeedReplaces` fails on `30175be48` | +| Seeds were never crossed with closed nesting | Fixed: `nested-seeds` lane | +| A late seed after a partial update was unwitnessed | Fixed: `partialUpdateThenLateSeed` fails on `30175be48` (the late seed's metadata is dropped) | +| A late seed erases Query's owner edges on a hydrated row Query claimed by a metadata-only write | Fixed: `ignores a late hydration seed for a row whose metadata a claim has published` fails on `30175be48` | +| An adapter insert over an `initialData` row merges its field; the hydration grid could not see it | Fixed: `initialData` rows carry a field and stay present in insert-first histories | +| The hydration grid had no metadata-only source write, and its reference cleared metadata on an equal re-insert of a source row | Fixed: the action is added and the clear is scoped to seed-only rows | +| Query tests matched the opaque owner-key encoding | Fixed: owner counts plus public-behaviour identity siblings (release and hydrated-release checks) | +| Stale counts, missing "hydration seed" glossary term, SSR omissions | Fixed | +| Mutants against a scratch D5 seed implementation: equal insert over a seed stays an update; replace only applied seeds; replace only pending seeds; metadata authority from row writes only; seed replacement in full mode only | Killed (by `equalInsertOverSeed`; seed-lane properties; hydration insert cells and `seedUnderOpenInsert`; `seedUnderMetadataOnlyWrite`; `laterSeedReplacesSeed` and partial hydration cells) | +| Mutant: Query owner supplement deleted | Survives with or without the fallback; probably equivalent, as recorded above | +| Mutant: Query fallback removed | Repairs exactly the fallback witnesses with no new failures | +| The persisted equal-insert witness is insensitive to core seed mutants | Accepted: it checks the wrapper's own clearing; core seeds are judged by the core lanes | +| A scratch D5 seed implementation still leaves seed lanes RED on histories with deletes, releases, and truncates between seeds | Open: diagnose during the production fix | + +## Fourth review round (prep-PR) + +A code review, a simplification pass, and 10× random campaigns examined the +branch before its pull request. Items B to H were then taken through one +plan of oracle changes before any repair: one grammar action for B and C, a +late-abort probe for E, a scaling experiment for D, and a churn count for H. + +| Finding | Disposition | +|---|---| +| A: a seed's metadata returns through the wrapper after an accepted equal re-insert (`persisted-seeds`, random seed -387838471) | Fixed: the wrapper hands inserts to core as whole-row updates (decision 9). Witness `lets a later seed replace metadata that an accepted re-insert kept` fails on `743a6ce14` | +| A sibling: an inner re-insert of an enclosing row commits ahead onto a seeded row; core classifies it `insert` and the kept value again claims the metadata | Fixed by the same change. Witness `lets a later seed replace metadata that an inner re-insert kept`; it alone kills the narrow fix that writes the kept value only when core would not keep it | +| A gap: no persisted lane crossed seeds with nesting or with the cached row | Fixed: `persisted-nested-seeds`, with reach `seed under a kept re-insert` (minimum 5) | +| Cached row versus a later `DbClient` seed: the seed could not replace the cached row or its metadata | Fixed (decision 9). Witness `lets a later seed replace a cached row's metadata` | +| An update without metadata after an enclosing insert dropped the insert's re-judgment, so an inner commit could not make it clear metadata (found by the 10× campaign) | Fixed. Witness `rejudges an enclosing re-insert that a later update kept` | +| The wrapper re-judged only a key's last insert without metadata against the value saved before it, so an earlier insert whose kind an inner commit changed was missed, in either direction (found by a second 10× campaign) | Fixed: each key keeps its chain of such inserts. Witnesses `rejudges every enclosing re-insert of a key, not only the last` and `lets an enclosing clear become an equal re-insert after an inner commit` | +| B: a hold hides a duplicate source insert (`acceptSyncTransaction` refreshes the projection only when a transaction moves) | Fixed. The grammar never wrote a different insert over a row the source wrote; the `conflictingInsert` action does, and the model expects core's duplicate-key error (decision 8 keeps duplicates in core). On `743a6ce14` plus decision 9, 26 core tests fail, every seeded lane among them, including hold invariance. Core's classifier now exempts only a row that only seeds supplied; a seed beneath an open insert keeps that insert's kind in the replay instead | +| C: a seed hides a duplicate insert in the same or an enclosing transaction | Fixed by the same change and caught by the same action (`seed(1) begin upsert(1) conflictingInsert(1)` was accepted) | +| D: the wrapper's staged-row fold is O(N²) per transaction | Fixed. 4000 inserts in one transaction took 48 ms against 0.8 ms on `origin/main`. Each staged transaction now keeps its row per key, so a write-time read folds one entry per transaction; 16000 inserts take 6 ms. A mutant that drops partial updates' merge base fails three witnesses and four persisted campaigns. No oracle measures work; the scaling run is the evidence | +| E: an aborted staged transaction leaves enclosing implicit metadata judged against it | Refuted for a store-held transaction: `commit` says that aborting after acceptance has no effect, and its inner transaction still applied. Confirmed and fixed for a transaction buffered behind a storage read, which an abort abandons before acceptance: when a deferred transaction leaves without reaching the Collection, the wrapper judges the enclosing inserts again. Witness `judges enclosing inserts again when a buffered inner transaction is abandoned` fails without it. A durable failure ends the run with a terminal error, outside this law | +| A storage read marked every row the Collection did not hold as a seed, including a row a peer's source wrote in this run and that this tab loads for a new demand | Fixed: the runtime records the keys that peer notices name as written in this run, and stops marking seeds after a notice or catch-up that needs a full reload, because it names no keys. Witnesses `keeps a row another tab wrote this run when a later seed arrives` (named keys, full reload) fail on `e4b562448`; a mutant that ignores a full reload fails the second | +| Query owner supplement (`rowToQueries` in `getOwnersFromMetadata`) | Removed. Each edge Query adds stages its owner set in the same sync transaction, and Query rolls edges back when that transaction fails, so the supplement only covered owner metadata that held transactions hid from reads. The commit-order reader makes it redundant; all 1,486 Query tests pass without it | +| F: Query's owner read lost its applied-metadata fallback for a sync without `metadata` | No action: core always supplies `metadata`, and no wrapper in the repository drops it | +| G: state internals are public for `client.ts` | Accepted design: the seed authority lookups moved to `client.ts` so that an app without `DbClient` ships 130 fewer gzip bytes | +| A later seed merged into an outer re-insert's row by an inner update that commits ahead (random seed 50521469; also fails in core on `743a6ce14`) | Decision 10: the model now counts such a seed in the insert's kind. Witnesses `lateSeedUnderMergedUpdate` and `heldLateSeedUnderMergedUpdate` (core) and `lets an inner update merged over a later seed change an enclosing re-insert` (persisted). The held witness kills a core that keeps the insert's kind after the accepted inner write; without a hold the inner write applies and clears the seed mark, so that mutant is invisible there | +| H: `appliedAdapterMetadataKeys` grows with every deleted key | Fixed: the deleted-key and metadata-key sets became one map, `appliedAdapterKeys`, from each key whose metadata an applied adapter write fixed to whether that write deleted the row. After 5000 keys were inserted and deleted it holds 5000 entries instead of 10000 across two sets; the deleted keys remain, as D4 requires. The hydration authority grid gains `delete,update` cells, which a mutant that loses the returning key's authority fails. `sourceWrittenKeys` in the wrapper is unchanged | +## Fifth review round (second high review) + +A high review of `ad966ec12` raised ten items. The plan of oracle changes +came first: per-key storage-read witnesses (peer metadata, a peer delete, a +held seed, an equal re-insert), a read hold in the persisted grammar with +the `persisted-reads` lane, a public-message witness, and scaling runs. + +| Finding | Disposition | +|---|---| +| R1: a storage read marked as seeds for its whole transaction lets a later seed replace a peer's metadata | Fixed. The read stays one transaction; core keeps seed status only for the keys it lists (`preserveHydrationSeedKeys` may be a key set). Witness `keeps a row another tab wrote this run…` now checks metadata; it failed on `749079331` and passes with the `origin/main` wrapper | +| R1b: a DbClient seed held against a peer's write in this run | Fixed by decision 11; witness `treats a row a peer wrote this run as a source row (replaces a held seed)` | +| R2: validation deletes inside a seed transaction lost delete authority | Fixed by the same per-key rule; witness `keeps a row a peer deleted absent when a later seed arrives` | +| R3: a committed enclosing transaction stays judged against an abandoned buffered write | Refuted on the same path: the outer commit threw the hydration-cycle error, and after decision 12 the inner transaction either flushes or is dropped before the outer one commits | +| R4: writes during a storage read used older metadata rules | Fixed by decision 12: one write path. Witness `clears an explicit metadata write with a later insert during a storage read` and the `persisted-reads` lane (reach `write during a storage read`) fail on `749079331` and on `origin/main` | +| R5: `prependHydrationRows` did not shift partial-update indexes read by the staged-row walk | Resolved by removing the walk: each insert keeps its transaction's row from just before it | +| R6: core honoured `originalSyncType` from any public write | Fixed: the wrapper marks a whole-row update with the internal `syncWriteNamesWholeRow` symbol. Witness `merges a public partial update whatever fields the message carries` | +| R7: seed keys entered `sourceKeys` while staged | Refuted as unreachable: no caller classifies a seed-only row then | +| R8: two sets of keys written in this run | Fixed: the runtime keeps one record, fed by this tab's handoffs and peers' notices. Witness `treats a row a peer wrote this run as a source row (keeps metadata on an equal re-insert)` | +| R9: re-judgment was quadratic | Fixed by the same change as R5: 8000 implicit inserts re-judge in 2.5 ms (2000 took 18 ms). A mutant that judges against the transaction's whole row fails two witnesses and three campaigns | +| R10: readers copied the pending queue on every read | Fixed: they skip invalidated transactions in place | +| S1 (sibling): a commit threw when an on-demand read started during its transaction | Fixed by decision 12; the `persisted-reads` lane found it | + +## Sixth review round (third high review) + +A high review of `ff80a31c3` raised nine items. The oracle changes came +first: a peer-route witness grid (metadata-only notice, a missed notice, a +peer row during an open insert) and stored rows for the `persisted-reads` +lane's read hold, with reach `storage read of a stored row`. + +| Finding | Disposition | +|---|---| +| H1: a peer's metadata-only notice did not count as a source write | Fixed: the runtime records row keys and metadata keys apart (D5 judges them separately). Witness `treats a peer's write in this run as a source write (metadata-only notice over a held seed)` failed on `ff80a31c3` | +| H2: the gap-recovery fallback and term-fence reloads left the run's writes known | Fixed: both mark them unknown. Witness `… (write whose notice was missed)` failed on `ff80a31c3` | +| H3: rows a peer's notice or a storage read landed ahead of an open wrapper insert did not judge it again | Fixed: every internal apply judges the open transactions' inserts again. Witness `… (peer row during an open insert)` failed on `ff80a31c3` | +| H4: a held seed ignores the unknown-writes state that seed marking honours | Accepted design (decision 13): under unknown authorship, what this tab holds wins. The persisted oracle's seed-across-peer-notice cases with absent, older, and equal durable data witness it; a trial that let unknown rows replace held seeds failed eight of them | +| H5: the read lane's storage read returned no rows | Fixed in the oracle: the holding adapter keeps durable rows; the lane reaches `storage read of a stored row` and passes | +| H6: the storage read assumed `begin` opened the transaction it marks | Fixed: it marks seeds only when `begin` opened one | +| H7: the staged-row fold copied the queues on every read | Fixed: it iterates them in place | +| H8: re-judgment scanned every implicit insert on each inner commit | Fixed: it looks up the inner transaction's keys | +| H9: `syncWriteNamesWholeRow` was exported without `@internal` | Fixed | + +## Seventh review round (fourth high review) + +A high review of `e4b81558b` raised ten items. The oracle change came first: +a reload-route grid in the persisted oracle, `keeps previous-run rows seeds +across a reload a peer's notice causes`, which crosses the three reloads a +named-key notice can cause (no `DbClient` seed held, so the reload +truncates core; a held seed, so it rewrites rows in place; more keys than a +targeted update allows) with a full-reload neighbour, and observes a later +seed of a previous run's row, of the row the notice named, and of a key +storage never held. + +| Finding | Disposition | +|---|---| +| F1: a follower merges a targeted insert's row over its own and keeps a removed field | Refuted on the same path: a follower that shows the key reloads it from storage, and `follows durable same-key order for targeted insert replacement` (active and retired demand) passes | +| F2: an inner insert is judged against the enclosing open transaction's row, though it commits ahead onto another | Accepted design (decision 14). Production and the model agree (`nested`, `nested-conflicts`); a probe of the reported trace keeps metadata with no error. The mutant that re-judges accepted inserts, as `30175be48`'s replay did, fails both retention witnesses (fourth round) | +| F3: reloads a peer's notice causes rewrote previous-run rows as source writes, and the reload's truncate counted as a source truncate | Fixed (decisions 9, 11, and 15). A reload marks its transaction's seed keys: rows no source wrote, and under unknown writes rows storage left unchanged. Core keeps authority and seed status across that transaction's truncate, and the reload deletes each dropped row as a source write. The truncate stays, because it publishes each durable row's exact shape (three exact-shape witnesses need it). The grid failed three of four variants on `5013e5aba`; a later seed of a never-stored key was ignored after a truncating reload. Decision 15 added a full-reload variant followed by a new demand's storage read and a row the peer deleted; both full-reload variants failed on `44ef0c2d0`. Mutants: core treating the truncate as a source truncate (four variants fail), no unchanged-seed exemption under unknown writes (both full-reload variants), and no per-key delete of a dropped row (four variants). An earlier repair that validated rows in place lost the exact-shape publication | +| F4: every internal apply re-judged every open implicit insert | Fixed: the runtime passes the keys the internal apply's transaction wrote (every key after a truncate). Ten storage reads beside 4000 open inserts re-judged 40000 times, now none. A mutant that re-judges no keys fails `… (peer row during an open insert)`; no generated lane catches it | +| F5: staging adds a storage read's seed keys to the source-key set that a rebuild excludes | Refuted as unobservable, as R7: a mutant that always rebuilds at acceptance changes no persisted session lane, persisted-oracle case, or core session lane; the wrapper's only classifier call skips keys no source wrote in this run | +| F6: metadata reads scan every pending transaction | Accepted cost of read-your-writes: 10000 reads take 0.7 ms with no held transaction and 4.2 ms with 100 | +| F7: the run's written-key sets grow, and the internal-apply listener outlives cleanup | No action: the sets keep deleted keys because D4 and decision 11 need them, as H; cleanup empties the arrays the listener reads, and the next run replaces it | +| F8: a redundant deleted-key check, and a stored `acceptedSourceKeys` only the rebuild reads | Fixed | +| F9: state internals are public for `client.ts` | Duplicate of round 4's G (accepted design); `any` follows `client.ts`'s own `AnyCollection` convention | +| F10: seed keys are marked before the read's commit | Refuted: the read's begin, writes, and commit run synchronously, its commit takes no signal, a retired run's `begin` opens no transaction to mark (H6), and an accepted transaction always applies | + +## Eighth round (the peer dimension) + +After the fourth high review, the maintainer asked for the next oracle +improvement before another review: a peer in the persisted session grammar. +Every real finding since the fifth round had sat on a peer or reload route +that only pinned grids covered. + +- **Law.** A peer's write in this run is a source write, accepted when this + tab applies its notice, after every accepted transaction and beneath the + open ones (decision 11). A full-reload notice keeps a held `DbClient` seed + (decision 13) and makes stored rows and metadata that differ source + writes (decisions 15 and 18). Reloads keep metadata of a missing row + (decision 16); a write over a seed stores the row this tab shows + (decision 17). +- **Grammar and model.** The `peer` action sends one of four notices: a + named row with its metadata, metadata alone, a delete, or a full reload. + It is legal only when no store hold or read hold is active, so every + accepted transaction of this tab is durable and the notice is the next in + the stream. The model tells cached rows from `DbClient` seeds and keeps + what a reload restores per row: storage's row and metadata, which each of + this tab's commits sets to what this tab shows (decisions 16 to 18). +- **Driver.** A `SingleProcessCoordinator` that delivers the `tx:committed` + shapes the Browser and Electron coordinators send; the holding adapter + keeps rows and collection metadata. +- **Lane and reach.** `persisted-peers` (seeds, the cached row, nesting to + depth 2, store holds, 150 fixed histories). A 150-history sample with the + fixed seed reaches each named notice kind 44 to 69 times, a named write + over a held seed 42, a full reload over a held seed 19, a peer write during + a hold 46, and during an open insert 17; the declared minimum is 5. A + declared state for a full reload that reveals a hidden peer write was + dropped: a write of the row by this tab stores its row over the peer's + (decision 17), so only a named notice reveals it, which `named peer write + over a held seed` covers. + +| Finding | Disposition | +|---|---| +| P1: a reload's held-seed set came from `base`, so it truncated over a seed accepted during a hold (`hold seed(2) peer(1,row)`) | Fixed: held seeds come from core's hydrated keys | +| P2: a named notice for a row only held transactions wrote merged the peer's whole row over this tab's held row (`hold begin upsert(2) commit peer(2,row)`) | Fixed: a row in `base` counts as ambiguous, and so does a held row that the notice would change, so the notice reloads. The first fix checked held rows only and regressed `begin remove(1) hold commit peer(1,row)`; the lane found it at once. The second counted every held row and broke the real-host reload-class oracle's law (e), "a commit the tab already knows never makes it reread rows", on its Browser and Electron drivers: the notices it rerouted carried exactly the held row. A third version reloaded only when the notice's merge changed the held row; a peer's whole row that kept the held `v` and dropped its other fields then kept them. The grammar's row notice now has a variant that keeps this tab's `v` (`peer(…,row,keeps)`), and the replay `a notice whose row keeps a held row's value still replaces it` failed on that version. The final check reloads unless the notice carries exactly the held row, and the reload-class oracle passes on both drivers. Generated histories reach that premise too rarely to declare it (none in 300); the replay holds it | +| An in-place reload merged a stored row over a held seed (`hold seed(2) seed(1,narrow) peer(2,row)`) | Fixed: reloads write each stored row whole | +| A reload deleted again a row a held delete removed, dropping metadata written after it | Fixed: a dropped row is deleted only while core holds it, judged before the reload's truncate | +| F5 (seventh round, refuted as unobservable): staging counted a reload's preserved keys as source writes | Reachable after all: `seed(2,bare) begin reinsert(1) hold peer(2,reload)` judged the open re-insert equal. Fixed in core: staging skips preserved keys, as the rebuild does | +| L1, L2, and the later-seed metadata case | Decisions 16, 17, and 18 (B′) | +| A reload counted unchanged rows' writes as source writes, so metadata a seed governs lost that status | Fixed: a reload's write of an unchanged row keeps its status. A truncate in a storage read that keeps statuses keeps row metadata in core, and `metadata.row.get` reads through it; `_syncedRowMetadataKeysOutsideBase` left core | +| A named row notice did not mark its metadata as written in this run, so decision 18 kept a held seed's metadata over the peer's | Fixed: a peer's row comes with its metadata (decision 11) | +| Driver: the holding adapter dropped collection metadata, and cleared it on truncate | Fixed in the test adapter | + +Evidence: + +- **RED.** Each shrunk history above failed on the production before its + fix; the survey of 2,000 generated histories before any fix found P1 in + 22 and P2 in 9. +- **GREEN.** Four surveys of 1,000 generated histories (seeds 77, 91, 105, + 133) find no mismatch; the fixed and random `persisted-peers` campaigns + and the shrunk histories, kept as replays through the same model, pass. +- **Mutants.** P1 reverted fails both peer campaigns; P2 reverted fails the + fixed one; B′ collapsed to B fails seven cases, two `main` full-reload + cases among them; decision 11's metadata keys removed fails `treats a row + a peer wrote this run as a source row (replaces a held seed)`. Three + mutants survive every campaign and fail only their replay: no seed base + (decision 17), storage metadata always winning (decision 18), and a + dropped-row delete without the held check. The campaigns reach their + premises too rarely; the replays keep them. In core, staging that counts + preserved keys (F5) fails its replay; a status-keeping truncate that + clears row metadata fails nine cases, both peer campaigns among them; and + a metadata reader that stops at such a truncate fails two `replaces a + hydrated seed and metadata with a peer's targeted …` cases. +- **Limits.** The peer is the driver; no second Collection or real + coordinator delivers its notices, so the claim stops at the controlled + coordinator (ORC-014). The next receiving witness is a second production + driver of this oracle with two coordinators over a real channel + (issue #2272). Missed + notices and notices over the targeted key limit are not generated; the + persisted oracle's grids cover them. The adapter is in memory, and restart + stays outside this oracle. + +## Ninth round (medium review) + +A medium review of `2156c9c57` raised ten items. Planned together, they +asked for three oracle changes before any production edit: a peer lane +under full row updates (M1), a reset notice (M4), and a Query storage-scan +witness (M10). + +- **Law.** A durable write stores the row this tab shows, so storage after + settlement equals the model's storage, which a restart reads. Under full + row updates an update replaces its row there too. A reset is a source + truncate; eager it takes held seeds, on demand it keeps them with their + objects' authority (decisions 19 and 20). A notice in a reset's term may + predate the reset, so this tab reloads storage for it (the wrapper's + reset fence). +- **Grammar and model.** The peer notice gains `reset`, which empties + storage. The model's truncate may keep a set of keys, and a reset on + demand keeps the `DbClient` seeds it holds. After a reset every notice + reloads storage. New lanes: `persisted-peers-full` and + `persisted-peers-on-demand`, the second with a demand for every row. +- **Observation.** The peer lanes now compare the holding adapter's stored + rows with the model's after settlement. Before, storage was seen only + when a reload read it back. +- **Reach.** The full-row lane declares `accepted update drops a field`; + the on-demand lane declares `reset over a held seed` with the peer + states. A reset over metadata of a missing row reached 1 in 150 + histories, so a replay holds it instead of a declared state. + +| Finding | Disposition | +|---|---| +| M1: decision 17's seed merge ignored `rowUpdateMode: 'full'` | Confirmed and wider: the wrapper stored every sync update as a merge, so under full rows a reload or restart brought back fields an update dropped, with or without a seed (`main` too). Fixed: only a partial update merges in storage; every other write replaces its row | +| M2: seed marks added at a read's begin | Refuted, as F10 | +| M3: seed metadata applies over a rejected row | Accepted: row and metadata are separate objects (D5) | +| M4: a reset beside a held seed kept orphan metadata | Confirmed and wider: the reset also kept rows a hold kept from `base` and skipped the demand replay its truncate drives; a reload deleted only `base` rows that storage lacked. Fixed: a reset is a source truncate that keeps held seeds (decisions 19 and 20), and a reload deletes every accepted row that storage lacks | +| M5: a status-keeping truncate keeps flags of dropped keys | Refuted: a dropped key is outside the read's preserved keys, so its delete is a source write and clears its flags (the grid's `gone` row) | +| M6: an accepted inner equal insert keeps its kind | Accepted (decision 14) | +| M7, M8 | Duplicates of F9 and F7, accepted earlier | +| M9: the commit wrapper reads the newest pending transaction on every commit | Harmless: `withInternalApply` resets the key set before each use | +| M10: Query's storage-scan owners omit accepted edges | Open; see "Open" | +| Lane: a reset kept a held seed but cleared the authority a peer's metadata write had fixed over it, so a later seed's metadata replaced the peer's | Fixed: the reset's truncate keeps the authority entries of kept keys | +| Survey: a source row write after an on-demand reset closed the kept seed's metadata to a later seed | Fixed: a kept key stays exempt for its metadata; a row write fixes row authority through the applied row | +| Survey: after a full-reload notice, a named notice that the wrapper turns into a reload erased a seed's metadata of another row (`peer(1,reload) seed(2) begin upsert(2) commit peer(1,row,keeps)`, also on `2156c9c57`) | Fixed: a notice that names no keys makes writes unknown per key, not for the rest of the run. A key is known again once this tab's source writes it, and a storage read records each row it finds changed as this run's source write. This tab's own truncate still makes every later write unknown. Ending the unknown state with the reload instead failed two `persisted-oracle` witnesses: an on-demand reload with no demand reads no rows, so a row it never read must still beat a later seed. Marking keys known before the read wrote them failed both oracles widely. Mutants: not recording changed rows fails the fixed campaigns of all three peer lanes; no knowledge from this tab's writes fails the family's replay; knowledge kept across a later full-reload notice fails 16 cases. Knowledge from a read's rows too survived every case and was removed | + +Evidence: + +- **RED.** On `2156c9c57`'s production, ten cases fail: the five new + replays and the fixed and random campaigns of both new lanes. A + replay of the kept-authority case failed on the first M4 repair. +- **GREEN.** Every persisted session lane and replay passes at the + configured runs. Two surveys at four times the runs found two families, + one on the reset path and one that `2156c9c57` already had; both are + fixed and pinned, and two more surveys at four times the runs pass. +- **Mutants.** Killed: an update over a seed merging under full rows; no + seed base for partial updates (now by both partial campaigns, through the + storage check); every update replacing; a reset that keeps no seeds; no + rewrite of a kept seed's row; reload deletes from `base` only; + authority that ignores kept keys; a truncate that clears kept metadata. + For the scoped unknown writes, the committed sticky flag fails the + family's replay, and not recording changed rows fails the fixed + campaigns of all three peer lanes. Survived: a session reader that stops + at the reset's truncate for kept keys. A truncating transaction applies at once, so no read sees it + pending; the reader change was removed. +- **Limits.** The reset's term fence is exercised only through the + driver's single-term peer. The full-row and on-demand lanes cover the + peer grammar, not the other lanes' holds and reads. + +## Tenth round (high review) + +A high review of `18f7df562` raised ten items (H1 to H10). + +- **Law.** A reset's own reload reads metadata through its open truncate + only for kept seeds. What storage holds after a reset is this run's + source writes. A transaction the wrapper abandons before handoff wrote + nothing, so a seed beneath it holds (decision 21); a seed that arrives + after it stays judged against it (decision 22). +- **Grammar and model.** A reset may rewrite its rows: the resetting tab + writes storage's rows back, and the reload applies them after its + truncate. A new `abandon` action aborts the signal of the newest + outermost transaction committed during a read hold; only the read's end + may follow, which abandons it. A transaction begun over a buffered one + reads it and becomes a dependent successor, which the abort-graph oracle + owns, so `abandon` needs none. New lane `persisted-reads-seeds` crosses + read holds with seeds and the cached row, loaded by a demand for every + row. +- **Reach.** `reset that rewrites its rows` and `abort after commit` are + declared. Abandoning the writer of a held seed's row stays below five in + 150 histories; a replay pins it. + +| Finding | Disposition | +|---|---| +| H1: the session reader read through a reset's open truncate, so the reset skipped writing stored metadata equal to what it was about to delete | Confirmed. RED: the rewriting reset's replay and the on-demand campaigns. Fixed: the reader stops at a source truncate for keys it does not keep. Round 9 had removed this as unreachable; the wrapper reads inside its own open reset | +| Sibling: an equal re-insert of a row a reset wrote back cleared its metadata | Fixed: a reset records the rows it restores as this run's source writes | +| Sibling (model): the model kept decision 16's metadata of a missing row through a rewriting reset | Fixed in the model: only rows are durable | +| H2: run writes were recorded before the handoff and kept when it abandoned the transaction | Confirmed. RED: the abandon replay. Fixed: only a transaction handed to core with a live signal is recorded | +| H3: `truncate` ignores an invalid open transaction | Accepted design (decision 7); a mutant that revives it fails the nested and conflict campaigns | +| H4: a read's seed marks and run-written rows stay if its transaction is discarded | First half refuted earlier (F10). Second half true but harmless: under unknown writes a changed stored row is this run's source write whether or not the read applies (decision 15) | +| H5: the targeted-invalidation seed checks are always false | Confirmed by source: the notice's keys are recorded first. Removed | +| H6: the hydration binding's cross-cycle throw and its assert are unreachable | Confirmed by source. Removed with their `try`; the assert at the handoff stays | +| H7 | Duplicate of G (accepted design) | +| H8 | Duplicate of F7; the newer known-key set is cleared at each full-reload notice | +| H9: `appliedAdapterKeys` keeps an entry for nearly every synced key while a `DbClient` tracks authority | Accepted: per-object seed authority needs it (D4, D5); the map exists only under `DbClient` tracking | +| H10: the session reader walks every pending transaction | Observation, no performance law: pending transactions accumulate only while an optimistic transaction holds sync | +| Survey: a seed after a buffered write that is then abandoned | Decision 22; the model records each seed's losses before abandonment | + +Evidence: + +- **RED.** On `18f7df562`: H1's replay and the on-demand campaigns; the + re-insert sibling's replay; H2's replay. +- **GREEN.** Every persisted session lane and replay; three searches of + `persisted-reads-seeds` at four times the runs. +- **Limits.** The abandon action covers one buffered transaction with no + successor; dependent suffixes belong to the abort-graph oracle. +- **Files.** Under `oracle/no-unit-test` (#2278), the session model module + declares its check, model, and grammar. Its fixed boundary tests moved to + `packages/db/tests/sync-metadata-session.test.ts`, and the persisted fixed + witnesses to + `packages/db-sqlite-persistence-core/tests/persisted-sync-metadata-session.test.ts`. + +## Open + +- **Held-cleanup retention witnesses race on CI.** `does not publish a + retention-cleaned owner through a claim made during the cleanup hold` and + `hydrated clients drop a held-cleanup row once its peer and claimant + release it` (Query ownership lifecycle) sometimes see no expiry + transaction staged within 5 seconds of the unload while an optimistic + blocker holds sync. They failed on CI at `1c7624710` and `9ec84d81c` and + passed on the rerun of `9ec84d81c`, at `c065ab5ec`, and at `bc8f5f3d2`; + locally they pass five times in five at load average 43. The witnesses + without a hold pass in under 120 ms. Refuted: a Node timer firing before + the entry's wall-clock deadline (a timer set after a 50 ms busy task fired + 12 ms later), and a wait on a held receipt (no step of the expiry awaits + one). The witnesses now trace each timer Query DB sets during the test + (where it was set, whether it fired, what cleared it) and each Query + fetch, and a failed retention wait prints that trace. With it, the + witnesses passed on CI at `543d29066` and in four local runs of CI's + collections command. Deferred by the maintainer on 2026-10-27: the pull + request merges with the race open ("it'll eventually fail at which point + we can fix it"). Resume when a failure reports its trace; the trace + names the step to fix, in production if a valid expiry was cancelled, or + in the witness if only its timing raced. + +- **Query storage-scan owners (M10).** `getOwnersFromMetadata` no longer + adds this tab's accepted edges, which matters only where it reads a + storage scan: `loadPersistedBaselineForQuery` for a retained Query whose + known rows are not all in the Collection. Three witnesses (a held scan + beside another Query's claim; a retained Query with a stored row, and with + a row moved out of its predicate) each took the in-memory baseline, which + includes the claim. The scan and every storage hydration hold the apply + lock, so a claim during a held scan waits. Not excluded: a claim + committed during a storage read, which the wrapper buffers (decision 12), + with a retained Query's scan before the buffer flushes. That history is + the missing witness; no production change without its RED. + +Nothing else in this bug class is known to be open. The limits in the +evidence block still apply. + +## Oracle guide requirements + +| Requirement | Outcome | +|---|---| +| ORC-001 | Met: decisions and sources above; limits in the module's opening prose. | +| ORC-002 | Met: the model folds an event log; production scans queued write sets. | +| ORC-003 | Met: contract, model, grammar, driver contract, and comparison are separate sections with prose. | +| ORC-004 | Met: witnesses are reconstructed by the grammar; the ablation test shows each axis is needed for its regime; range probes added prefix-boundary keys (new values only) and a seed field (a missing merge rule, now modelled); exclusions (`remove` or `reinsert` of an absent row) are stated. | +| ORC-005 | Met: real sync API, `DbClient.hydrate`, and the persisted wrapper; observations at each checkpoint. | +| ORC-006 | Met: production on `30175be48` and the temporary reader mutants. | +| ORC-007 | Met: fixed and random campaigns per lane with replay properties registered in `oracle-config.ts`. | +| ORC-008 | Met: frames, accepted queue, seeds, and hold each change an observation (see the witnesses). | +| ORC-009 | Met: frames are sync transactions, each independent (decision 6); the model's serialization order is the order a read and an application use. | +| ORC-010 | Met: the runner preserves the mismatch and aggregates a cleanup failure. | +| ORC-011 | Met: three properties compare production with itself, without the model's reads: hold invariance (holds removed), key independence (row 2's actions removed), and seed order (seeds performed first, decisions D4 and D5). A shared fault in the model's reads cannot pass them. | +| ORC-012 | This record. | +| ORC-013 | Met for hold kind (store-hold neighbour passes, core hold fails on `30175be48`) and nesting (inner commits during a hold fail on `30175be48`; outer aborts and unheld inner truncates pass there). | +| ORC-014 | Met: the persisted lanes receive the flat, nested, and seed laws through the wrapper. Restart and resume belong to other owners. | diff --git a/docs/guides/ssr.md b/docs/guides/ssr.md index 94509a3763..e188b392c7 100644 --- a/docs/guides/ssr.md +++ b/docs/guides/ssr.md @@ -645,8 +645,18 @@ is: 3. hydrated rows Fresh adapter sync is authoritative over all three. Hydrated and initial rows -are provisional base state, so the adapter's first insert for the same key is -reconciled as an update instead of raising a duplicate-key error. +are provisional base state. The adapter's first insert for the same key +replaces the provisional row and its row metadata instead of raising a +duplicate-key error; an insert without metadata clears the seed's metadata. +An adapter update merges over the provisional row like any update. A hydrated +row that arrives after the adapter has written its key, or while the adapter's +transaction for it is open, does not change what the adapter wrote: it only +supplies a row or row metadata that the adapter has not written. Row data and +row metadata count separately, so a later hydrated row can still supply +metadata for a key whose row the adapter only updated. After an adapter delete +or truncate, a later hydrated row does not bring the key back. Among hydrated +rows for a key the adapter has not written, the latest one wins, including its +row metadata or the lack of it. Hydrated rows and `initialData` never mark adapter sync as ready by themselves. The adapter still owns readiness through its sync lifecycle. diff --git a/packages/browser-db-sqlite-persistence/tests/per-collection-coordinator.test.ts b/packages/browser-db-sqlite-persistence/tests/per-collection-coordinator.test.ts index 411657bef0..d5ab71d7c4 100644 --- a/packages/browser-db-sqlite-persistence/tests/per-collection-coordinator.test.ts +++ b/packages/browser-db-sqlite-persistence/tests/per-collection-coordinator.test.ts @@ -1651,9 +1651,11 @@ describe(`sync-ingested write ownership oracle`, () => { expect(electedAdapter.applied).toHaveLength(1) expect(electedAdapter.applied[0]).toMatchObject({ truncate: true, + // A source insert is persisted as an insert, which replaces any + // cached row (decision C1 of 2026-10-23). mutations: [ { - type: `update`, + type: `insert`, key: `rich`, value: { id: `rich`, title: `through Browser owner` }, }, diff --git a/packages/db-sqlite-persistence-core/src/persisted.ts b/packages/db-sqlite-persistence-core/src/persisted.ts index 3859ea985d..239a64af79 100644 --- a/packages/db-sqlite-persistence-core/src/persisted.ts +++ b/packages/db-sqlite-persistence-core/src/persisted.ts @@ -6,10 +6,12 @@ import { SyncTransactionAbortedError, collectionOptionsClaim, compileSingleRowExpression, + deepEquals, equalPersistedSnapshotValues, getLoadSubsetDemandKey, reportUncaught, safeRandomUUID, + syncWriteNamesWholeRow, toBooleanPredicate, whenSyncAccepted, withAcceptedReceipt, @@ -1269,13 +1271,19 @@ function toMetadataMutations( type BufferedSyncTransaction = { operations: Array> partialUpdateOperationIndexes: Set + // Operations written as inserts. An insert names the whole row, so it + // replaces the row it lands on, in the Collection and in storage. + insertOperations: Set> rowMetadataWrites: Map collectionMetadataWrites: Map - deferredHydrationMetadataDeleteKeys: Set - hydrationContext?: { suppliedRowKeys: Set } hydrationSequence?: number truncate: boolean internal: boolean + /** + * For each key whose row a DbClient seed held when the transaction reached + * the Collection, that row. Storage lacks it (decision 17). + */ + seedBases?: Map lifecycleGeneration: number admittedStorageCollectionId?: string expectedResumeGenerationOwner?: symbol @@ -1289,6 +1297,10 @@ type BufferedSyncTransaction = { rejectApplied?: (error: unknown) => void } +// A transaction's row for one key: a whole row, or the merged fields of +// partial updates to apply over the row before the transaction. +type StagedRow = { present: boolean; value?: T; partial: boolean } + type OpenSyncTransaction = Omit< BufferedSyncTransaction, `applyToCollection` | `prependHydrationRows` | `shouldFailStopOnAbort` @@ -1305,6 +1317,20 @@ type OpenSyncTransaction = Omit< // Every outcome of the transaction releases its admission waiters. publicationAdmissionWaiters?: Set<() => void> terminalFailure?: { error: unknown } + // Each key's row as this transaction leaves it: a whole row, or the merged + // fields of partial updates to apply over the row before the transaction. + stagedRows: Map> + // Each key's inserts without metadata since the last write that named its + // metadata, and that write. Their kinds, and so whether they clear, can + // change when an inner transaction commits first. + implicitMetadata: Map< + TKey, + { + // Each insert's value and the transaction's row just before it. + inserts: Array<{ value: T; before?: StagedRow }> + previous?: MetadataWrite + } + > // Set just before the transaction is handed to the Collection; from then // on core holds its keys itself. handedToCollection?: boolean @@ -1567,7 +1593,7 @@ class PersistedCollectionRuntime< > { private readonly applyMutex = new ApplyMutex() private readonly activeSubsets = new Map() - private activeHydrationContext: { suppliedRowKeys: Set } | undefined + private activeHydrationContext: object | undefined private hydrationSequence = 0 private waitForReservedCommit: WaitForReservedCommit | undefined private abandonReservedCommit: (() => void) | undefined @@ -1629,6 +1655,27 @@ class PersistedCollectionRuntime< private resumeGenerationOwner = Symbol(`persisted resume generation owner`) private lifecycleGeneration = 0 private internalApplyDepth = 0 + // Keys a source wrote in this sync run: this tab's, as the wrapper hands + // its transactions to the Collection, and peers', from their committed + // notices. A stored row with such a key is a source row, not a previous + // run's cache (decisions 9 and 11). A source truncate names no keys, so + // after one every stored row may be this run's. A notice that requires a + // full reload names none either, so every stored row may be a peer's write + // until this tab's source writes the key; a storage read records each row + // it finds changed as a source's write. + private runWrittenKeys = new Set() + // Row metadata is a separate object (D5): keys whose metadata a source + // wrote in this run, a delete included. + private runWrittenMetadataKeys = new Set() + private runWritesUnknown = false + private peerWritesUnknown = false + private keysKnownSincePeerWrites = new Set() + // Judges the open source transactions' inserts again after an internal + // write lands rows ahead of them, on the keys it wrote (every key after a + // truncate, as undefined). + private internalApplyListener: + ((keys: Iterable | undefined) => void) | undefined + private internalApplyKeys: Set | undefined private appliedReceiptSequence = 0 private syncErrorReported = false private reportedSyncError: unknown @@ -1917,7 +1964,15 @@ class PersistedCollectionRuntime< this.syncControls = { ...syncControls, commit: commit - ? (signal) => this.trackAppliedReceipt(commit(signal)) + ? (signal) => { + const transaction = + this.collection?._state.pendingSyncedTransactions.at(-1) + if (transaction?.truncate) this.internalApplyKeys = undefined + else + for (const operation of transaction?.operations ?? []) + this.internalApplyKeys?.add(operation.key as TKey) + return this.trackAppliedReceipt(commit(signal)) + } : null, } } @@ -2152,10 +2207,6 @@ class PersistedCollectionRuntime< return this.hydratingGeneration === this.lifecycleGeneration } - getActiveHydrationContext(): { suppliedRowKeys: Set } | undefined { - return this.activeHydrationContext - } - getHydrationSequence(): number { return this.hydrationSequence } @@ -2244,6 +2295,11 @@ class PersistedCollectionRuntime< const previousCollection = this.collection const previousControls = this.syncControls this.setSyncControls(syncControls) + this.runWrittenKeys.clear() + this.runWrittenMetadataKeys.clear() + this.runWritesUnknown = false + this.peerWritesUnknown = false + this.keysKnownSincePeerWrites.clear() if (this.collection === collection) { this.boundCollection ??= collection return @@ -3415,12 +3471,173 @@ class PersistedCollectionRuntime< this.resumeGenerationOwner = Symbol(`persisted resume generation owner`) } + /** Whether a source may have written this key in this sync run. */ + isRunWritten(key: TKey): boolean { + return this.writesUnknown(key) || this.runWrittenKeys.has(key) + } + + /** Whether a source may have written this key without naming it. */ + private writesUnknown(key: TKey): boolean { + return ( + this.runWritesUnknown || + (this.peerWritesUnknown && !this.keysKnownSincePeerWrites.has(key)) + ) + } + + /** Records keys whose rows and metadata this tab's source wrote. */ + noteRunWrites( + keys: Iterable, + metadataKeys: Iterable, + truncated: boolean, + ): void { + if (truncated) this.runWritesUnknown = true + for (const key of keys) { + this.runWrittenKeys.add(key) + this.keysKnownSincePeerWrites.add(key) + } + for (const key of metadataKeys) { + this.runWrittenMetadataKeys.add(key) + this.keysKnownSincePeerWrites.add(key) + } + } + + setInternalApplyListener( + listener: ((keys: Iterable | undefined) => void) | undefined, + ): void { + this.internalApplyListener = listener + } + + /** + * Whether a DbClient seed holds this key against stored rows. A source + * write in this run supersedes the seed (decision 11). + */ + holdsHydrationSeed(key: TKey): boolean { + return ( + this.collection?._hasHydratedKey(key) === true && + !this.runWrittenKeys.has(key) + ) + } + + /** + * How a row read from storage meets this tab. A row and metadata equal to + * what this tab holds are no write: the row keeps its status, a seed + * staying a seed (decision 15). Otherwise the row is a previous run's + * cache, a seed older than every source write (decision 9), unless the + * Collection holds it as a source row, a source wrote it in this run + * (decision 11), or this run's writes are unknown (decisions 13 and 15). + */ + private storedStatus(row: { + key: TKey + value: T + metadata?: unknown + }): `seed` | `kept` | `written` { + const state = this.collection?._state + if (!state) return `written` + const seeded = state.hydrationSeedKeys.has(row.key) + if ( + equalPersistedSnapshotValues( + state.getAcceptedSyncedRow(row.key), + row.value, + ) && + (this.keepsSeedMetadata(row.key) || + equalPersistedSnapshotValues( + this.syncControls.metadata?.row.get(row.key), + row.metadata, + )) + ) + return seeded ? `seed` : `kept` + return !this.writesUnknown(row.key) && + !this.runWrittenKeys.has(row.key) && + (seeded || !state.syncedData.has(row.key)) + ? `seed` + : `written` + } + + /** + * Opens a storage read's transaction. Core keeps the status of each key it + * lists, a seed's included, so a later seed can still replace a seed, and + * records adapter authority for every other write. Without a list, the + * read is adapter work. A `begin` that opened no transaction, as in a + * retired sync run, marks nothing. + */ + private beginStorageRead( + statuses: ReadonlyMap | undefined, + sourceTruncate = false, + ): void { + const state = this.collection?._state + const queued = state?.pendingSyncedTransactions.length ?? 0 + this.syncControls.begin?.() + const pending = + state && state.pendingSyncedTransactions.length > queued + ? state.pendingSyncedTransactions.at(-1) + : undefined + if (!pending || !statuses) return + pending.preserveHydrationSeedKeys = new Set(statuses.keys()) + pending.sourceTruncate = sourceTruncate + for (const [key, status] of statuses) + if (status === `seed`) state!.hydrationSeedKeys.add(key) + } + + /** + * Writes a stored row over the row this tab holds. A stored row is whole, + * so it replaces that row. Its metadata replaces this tab's unless this + * tab held the row and keeps its metadata as a seed's (decision 18). + */ + private writeStoredRow( + row: { key: TKey; value: T; metadata?: unknown }, + keepsSeedMetadata: boolean, + ): void { + this.syncControls.write?.({ + type: `update`, + value: row.value, + [syncWriteNamesWholeRow]: true, + } as { type: `update`; value: T }) + if (!keepsSeedMetadata || !this.keepsSeedMetadata(row.key)) + this.writeStoredMetadata(row.key, row.metadata) + } + + /** + * Whether a reload keeps this tab's metadata for the key: no source wrote + * it in this run, so it is a seed's (decision 18). When this run's writes + * are unknown, stored metadata that differs is a source write instead + * (decision 15). + */ + private keepsSeedMetadata(key: TKey): boolean { + return !this.writesUnknown(key) && !this.runWrittenMetadataKeys.has(key) + } + + private writeStoredMetadata(key: TKey, metadata: unknown): void { + const rowMetadata = this.syncControls.metadata?.row + if (equalPersistedSnapshotValues(rowMetadata?.get(key), metadata)) return + if (metadata === undefined) rowMetadata?.delete(key) + else rowMetadata?.set(key, metadata) + } + + /** + * The keys among stored rows whose status a storage read keeps. Under + * unknown writes every other row is a source's write (decision 15), which + * later reads must judge as this run's. + */ + private keptStatuses( + rows: ReadonlyArray<{ key: TKey; value: T; metadata?: unknown }>, + ): Map { + const statuses = new Map() + for (const row of rows) { + const status = this.storedStatus(row) + if (status !== `written`) statuses.set(row.key, status) + else if (this.writesUnknown(row.key)) this.runWrittenKeys.add(row.key) + } + return statuses + } + private withInternalApply(task: () => TResult): TResult { - this.internalApplyDepth++ + if (this.internalApplyDepth++ === 0) this.internalApplyKeys = new Set() try { return task() } finally { this.internalApplyDepth-- + if (this.internalApplyDepth === 0) + this.internalApplyListener?.(this.internalApplyKeys) } } @@ -3499,7 +3716,7 @@ class PersistedCollectionRuntime< let boundRowVersion: number | undefined let replayFailure: { reason: unknown } | undefined const resetSequence = this.resetSequence - const hydrationContext = { suppliedRowKeys: new Set() } + const hydrationContext = {} this.activeHydrationContext = hydrationContext try { this.throwIfTerminal() @@ -3558,9 +3775,6 @@ class PersistedCollectionRuntime< !config.bindKeySetEvidence || this.persistedKeySetEvidence?.status !== `incompatible` ) { - for (const row of rows) { - hydrationContext.suppliedRowKeys.add(row.key) - } const applied = this.applyRowsToCollection(rows, validationRows) await whenSyncAccepted(applied) if (boundRowVersion !== undefined) { @@ -3672,12 +3886,11 @@ class PersistedCollectionRuntime< transactionPresence.set(key, recovered) return recovered } - const present = - this.collection?._hasHydratedKey(key) === true - ? `present` - : snapshotInvalidatedByTruncate - ? `absent` - : `unknown` + const present = this.holdsHydrationSeed(key) + ? `present` + : snapshotInvalidatedByTruncate + ? `absent` + : `unknown` transactionPresence.set(key, present) return present } @@ -3782,81 +3995,43 @@ class PersistedCollectionRuntime< return true } - return this.withInternalApply(() => { - this.syncControls.begin?.() + const base = this.collection?.base + const readRows = rows.filter( + (row) => + !(validationRows && base?.has(row.key)) && + !this.holdsHydrationSeed(row.key), + ) + // An accepted row a hold keeps from consumers is this tab's too. + const held = new Set( + readRows + .filter( + (row) => + base?.has(row.key) || + this.collection?._state.getAcceptedSyncedRow(row.key) !== undefined, + ) + .map((row) => row.key), + ) - // A full durable snapshot validates already public keys. Only the - // requested subset may add keys to the public Collection. + return this.withInternalApply(() => { + // One read publishes once. + this.beginStorageRead(this.keptStatuses(readRows)) + // A full durable snapshot validates already public keys. Only + // the requested subset may add keys to the public Collection. if (validationRows) { const durableRows = new Map(validationRows.map((row) => [row.key, row])) - for (const key of this.collection?.base.keys() ?? []) { - if (this.collection?._hasHydratedKey(key)) continue + for (const key of base?.keys() ?? []) { + if (this.holdsHydrationSeed(key)) continue const durable = durableRows.get(key) - if (!durable) { - this.syncControls.write?.({ type: `delete`, key }) - } else { - const valueChanged = !equalPersistedSnapshotValues( - this.collection?.base.get(key), - durable.value, - ) - const metadataChanged = !equalPersistedSnapshotValues( - this.syncControls.metadata?.row.get(key), - durable.metadata, - ) - if ( - valueChanged || - (metadataChanged && durable.metadata !== undefined) - ) { - // A persisted read supplies a complete row. In partial-update - // mode, deleting it within this sync transaction removes fields - // that are absent from the durable value before replacement. - if (valueChanged) { - this.syncControls.write?.({ type: `delete`, key }) - } - this.syncControls.write?.({ - type: `update`, - value: durable.value, - metadata: durable.metadata as - Record | undefined, - }) - } - if (metadataChanged && durable.metadata === undefined) { - this.syncControls.metadata?.row.delete(key) - } - } - } - for (const key of this.collection?._syncedRowMetadataKeysOutsideBase() ?? - []) { - this.syncControls.metadata?.row.delete(key) + if (!durable) this.syncControls.write?.({ type: `delete`, key }) + else if (!equalPersistedSnapshotValues(base?.get(key), durable.value)) + this.writeStoredRow(durable, true) + else if (!this.keepsSeedMetadata(key)) + this.writeStoredMetadata(key, durable.metadata) } } - - for (const row of rows) { - if ( - (validationRows && this.collection?.base.has(row.key)) || - this.collection?._hasHydratedKey(row.key) - ) { - continue - } - if ( - this.collection?.base.has(row.key) && - !equalPersistedSnapshotValues( - this.collection.base.get(row.key), - row.value, - ) - ) { - this.syncControls.write?.({ type: `delete`, key: row.key }) - } - this.syncControls.write?.({ - type: `update`, - value: row.value, - metadata: row.metadata as Record | undefined, - }) - } - + for (const row of readRows) this.writeStoredRow(row, held.has(row.key)) if (collectionMetadata) this.applyCollectionMetadataSnapshot(collectionMetadata) - return this.syncControls.commit?.() ?? true }) } @@ -3940,49 +4115,80 @@ class PersistedCollectionRuntime< } const base = this.collection?.base + // A seed accepted during a hold holds its row before it is public. const hydratedKeys = new Set( - Array.from(base?.keys() ?? []).filter( - (key) => this.collection?._hasHydratedKey(key) === true, + Array.from(this.collection?._state.hydratedKeys ?? []).filter((key) => + this.holdsHydrationSeed(key), ), ) const rowKeys = new Set(rows.map((row) => row.key)) - // A truncate replays the Collection's demand. A follower routes that - // demand to its leader, whose source loads and commits again, so a - // follower replaces its rows in place, except after a reset, which owes - // the replay. + // A truncate replays the Collection's demand and publishes each durable + // row's exact shape. A follower routes that demand to its leader, whose + // source loads and commits again, so a follower replaces its rows in + // place, except after a reset, which owes the replay. A reset's truncate + // keeps the seeds this tab holds (decision 20). const followsLeader = this.canRouteRemoteDemandThroughCoordinator() && !this.persistence.coordinator.isLeader(this.storageCollectionId) - const truncate = hydratedKeys.size === 0 && (replayDemand || !followsLeader) + const truncate = replayDemand || (hydratedKeys.size === 0 && !followsLeader) + + // An accepted row a hold keeps from consumers is this tab's too. + const state = this.collection?._state + const held = new Set( + rows + .filter( + (row) => + base?.has(row.key) || + state?.getAcceptedSyncedRow(row.key) !== undefined, + ) + .map((row) => row.key), + ) + + // Each accepted row that storage lacks is a source's delete, a row a + // hold keeps from consumers included. + const dropped = Array.from( + state?.acceptedSyncedEntries() ?? [], + ([key]) => key, + ).filter((key) => !hydratedKeys.has(key) && !rowKeys.has(key)) + // A reset's truncate clears a kept seed's row, which it writes back. + const seedRows = replayDemand + ? Array.from(hydratedKeys, (key) => state?.getAcceptedSyncedRow(key)) + : [] + // A reset truncated everything before it, so what storage holds after it + // a source wrote in this run. + if (replayDemand) { + const restored = rows + .filter((row) => !hydratedKeys.has(row.key)) + .map((row) => row.key) + this.noteRunWrites(restored, restored, false) + } return this.withInternalApply(() => { - this.syncControls.begin?.() - if (truncate) { - this.syncControls.truncate?.() - } else { - for (const key of base?.keys() ?? []) { - if (!hydratedKeys.has(key) && !rowKeys.has(key)) { - this.syncControls.write?.({ type: `delete`, key }) - } - } - for (const key of this.collection?._syncedRowMetadataKeysOutsideBase() ?? - []) { - this.syncControls.metadata?.row.delete(key) - } - } + // A reload is a storage read, not a source write, so its truncate + // replaces rows without truncating source authority, and a row it + // finds unchanged keeps its status (decisions 9, 11, and 15). A reset + // is a source truncate of all but the seeds this tab holds. + // A reload's truncate keeps row metadata in core, so metadata of a row + // that storage lacks stays until a source deletes the row (decision 16). + const statuses = replayDemand + ? new Map(Array.from(hydratedKeys, (key) => [key, `seed` as const])) + : this.keptStatuses(rows.filter((row) => !hydratedKeys.has(row.key))) + this.beginStorageRead(statuses, replayDemand) + if (truncate) this.syncControls.truncate?.() + for (const key of dropped) + this.syncControls.write?.({ type: `delete`, key }) + for (const value of seedRows) + if (value !== undefined) + this.syncControls.write?.({ + type: `update`, + value, + [syncWriteNamesWholeRow]: true, + } as { type: `update`; value: T }) for (const row of rows) { if (hydratedKeys.has(row.key)) continue - if (!truncate && base?.has(row.key)) { - // Remove stale fields before applying a complete persisted row in - // the default partial-update mode, without truncating seed keys. - this.syncControls.write?.({ type: `delete`, key: row.key }) - } - this.syncControls.write?.({ - type: `update`, - value: row.value, - metadata: row.metadata as Record | undefined, - }) + // A reset's truncate clears metadata; a reload's keeps it. + this.writeStoredRow(row, !replayDemand && held.has(row.key)) } this.applyCollectionMetadataSnapshot(collectionMetadata) @@ -4309,6 +4515,36 @@ class PersistedCollectionRuntime< transaction: BufferedSyncTransaction, streamPosition: { term: number; seq: number; rowVersion: number }, ): PersistedTx { + // Only a partial update merges into its row; every other write replaces + // it. Storage lacks a DbClient seed's row, so a partial update that merges + // over it stores the row this tab shows (decision 17). The seed's metadata + // stays a seed's (decision 18). + const written = new Set() + const mutations = transaction.operations.map((operation, index) => { + const key = operation.key + const partial = transaction.partialUpdateOperationIndexes.has(index) + const seedBase = + partial && !written.has(key) + ? transaction.seedBases?.get(key) + : undefined + written.add(key) + return operation.type === `update` + ? { + type: + partial && seedBase === undefined + ? (`update` as const) + : (`insert` as const), + key: operation.key, + value: (seedBase === undefined + ? operation.value + : { ...seedBase, ...operation.value }) as Record, + } + : { + type: `delete` as const, + key: operation.key, + value: operation.value as Record, + } + }) return { txId: safeRandomUUID(), term: streamPosition.term, @@ -4316,19 +4552,7 @@ class PersistedCollectionRuntime< rowVersion: streamPosition.rowVersion, cacheGenerationClaimId: this.cacheGenerationClaim?.claimId, truncate: transaction.truncate, - mutations: transaction.operations.map((operation) => - operation.type === `update` - ? { - type: `update` as const, - key: operation.key, - value: operation.value as Record, - } - : { - type: `delete` as const, - key: operation.key, - value: operation.value as Record, - }, - ), + mutations, rowMetadataMutations: toMetadataMutations(transaction.rowMetadataWrites), collectionMetadataMutations: toMetadataMutations( transaction.collectionMetadataWrites, @@ -4896,7 +5120,7 @@ class PersistedCollectionRuntime< // This payload could have been committed before the schema reset. // The current durable snapshot is the only safe row authority. await this.runInHydrationScope((adapter) => - this.reloadActiveSubsetsUnsafe(adapter), + this.reloadUnknownWritesUnsafe(adapter), ) return } @@ -4995,15 +5219,8 @@ class PersistedCollectionRuntime< // it moves the stream position. A sync transaction that begins later // straddles it; one that ends before then never crosses it. this.throwIfReservedCommitOpen() - // Every reload in this recovery continues its hydration cycle. - const reloadInCycle = (adapter: HydrationPersistenceAdapter) => - this.reloadActiveSubsetsUnsafe( - adapter, - /* skipUnchangedReplace */ false, - /* restoreResetBaseline */ false, - /* replayDemand */ false, - /* continuesHydrationCycle */ true, - ) + // Every reload in this recovery continues its hydration cycle, under + // unknown writes: the missed notices named no keys. if (this.persistence.coordinator.pullSince && this.latestRowVersion >= 0) { let pullResponse: PullSinceResponse | undefined try { @@ -5040,7 +5257,9 @@ class PersistedCollectionRuntime< pullResponse.latestRowVersion, ) if (pullResponse.requiresFullReload) { - await this.runInHydrationScope((adapter) => reloadInCycle(adapter)) + await this.runInHydrationScope((adapter) => + this.reloadUnknownWritesUnsafe(adapter, false, true), + ) this.publicRowVersion = Math.max( this.publicRowVersion, pullResponse.latestRowVersion, @@ -5049,7 +5268,9 @@ class PersistedCollectionRuntime< } const deltas = pullResponse.deltas if (!deltas) { - await this.runInHydrationScope((adapter) => reloadInCycle(adapter)) + await this.runInHydrationScope((adapter) => + this.reloadUnknownWritesUnsafe(adapter, false, true), + ) this.publicRowVersion = Math.max( this.publicRowVersion, pullResponse.latestRowVersion, @@ -5092,8 +5313,10 @@ class PersistedCollectionRuntime< if (lifecycleGeneration !== this.lifecycleGeneration) return false // A gap is not a schema reset. Replace the durable snapshot in one // publication so readers never see an empty intermediate Collection. + // The missed notices named no keys, so every stored row may be this + // run's. await this.runInHydrationScope((adapter) => - this.reloadActiveSubsetsUnsafe(adapter), + this.reloadUnknownWritesUnsafe(adapter), ) this.publicRowVersion = Math.max( this.publicRowVersion, @@ -5214,7 +5437,24 @@ class PersistedCollectionRuntime< adapter: HydrationPersistenceAdapter, continuesHydrationCycle = false, ): Promise { + if (!txCommitted.requiresFullReload) { + for (const key of [ + ...txCommitted.changedRows.map((row) => row.key), + ...txCommitted.deletedKeys, + ]) + this.runWrittenKeys.add(key as TKey) + // A peer's row comes with its metadata (decision 11). + for (const key of [ + ...txCommitted.changedRows.map((row) => row.key), + ...txCommitted.deletedKeys, + ...(txCommitted.rowMetadataMutations ?? []).map( + (mutation) => mutation.key, + ), + ]) + this.runWrittenMetadataKeys.add(key as TKey) + } if (this.scopedRecovery) { + if (txCommitted.requiresFullReload) this.runWritesUnknown = true this.refreshScopedSubsets() return } @@ -5232,7 +5472,14 @@ class PersistedCollectionRuntime< ) if (txCommitted.requiresFullReload) { - await reloadActiveSubsets(txCommitted.reconciliationReload === true) + await this.runInHydrationScope( + (scopedAdapter) => + this.reloadUnknownWritesUnsafe( + scopedAdapter, + txCommitted.reconciliationReload === true, + ), + adapter, + ) return } @@ -5259,15 +5506,20 @@ class PersistedCollectionRuntime< const hasChangedAndDeletedKey = txCommitted.deletedKeys.some((key) => changedKeys.has(key), ) - const hasChangedPublicKey = txCommitted.changedRows.some((row) => - this.collection?.base.has(row.key as TKey), - ) + // A row that held transactions write is ambiguous in the same way while + // consumers still see the earlier row, unless the notice carries exactly + // that row: a commit this tab already knows rereads nothing. + const hasChangedPublicKey = txCommitted.changedRows.some((row) => { + if (this.collection?.base.has(row.key as TKey)) return true + const held = this.collection?._state.getAcceptedSyncedRow(row.key as TKey) + return ( + held !== undefined && !equalPersistedSnapshotValues(held, row.value) + ) + }) const hasUnseenDemandedKey = this.activeSubsets.size > 0 && txCommitted.changedRows.some( - (row) => - !this.collection?.base.has(row.key as TKey) && - !this.collection?._hasHydratedKey(row.key as TKey), + (row) => !this.collection?.base.has(row.key as TKey), ) if ( hasRepeatedChange || @@ -5289,6 +5541,10 @@ class PersistedCollectionRuntime< await reloadActiveSubsets() } + /** + * Applies a notice that names its keys. Its keys are this run's source + * writes, over any seed they land on (decision 11). + */ private async applyTargetedInvalidationUnsafe( txCommitted: TxCommitted & { requiresFullReload: false }, ): Promise { @@ -5304,7 +5560,6 @@ class PersistedCollectionRuntime< value: newValue, } of txCommitted.changedRows) { const key = changedKey as TKey - if (this.collection?._hasHydratedKey(key)) continue const previous = this.collection?.base.get(key) if (previous === undefined && this.activeSubsets.size === 0) continue const completeValue = { @@ -5326,17 +5581,10 @@ class PersistedCollectionRuntime< } } - for (const deletedKey of txCommitted.deletedKeys) { - const key = deletedKey as TKey - if (!this.collection?._hasHydratedKey(key)) { - this.syncControls.write?.({ type: `delete`, key }) - } - } + for (const deletedKey of txCommitted.deletedKeys) + this.syncControls.write?.({ type: `delete`, key: deletedKey as TKey }) txCommitted.rowMetadataMutations?.forEach((mutation) => { - // A DbClient hydration seed owns both its row and row metadata until - // an accepted source transaction confirms the key. - if (this.collection?._hasHydratedKey(mutation.key as TKey)) return applyMetadataWrite( this.syncControls.metadata?.row, mutation.key as TKey, @@ -5357,6 +5605,27 @@ class PersistedCollectionRuntime< await whenSyncAccepted(applied) } + /** + * Reloads storage after notices that named no keys: any stored row or + * metadata of a key this tab has not written since may be a source's + * write (decisions 13 and 15). + */ + private async reloadUnknownWritesUnsafe( + adapter: HydrationPersistenceAdapter, + skipUnchangedReplace = false, + continuesHydrationCycle = false, + ): Promise { + this.peerWritesUnknown = true + this.keysKnownSincePeerWrites.clear() + await this.reloadActiveSubsetsUnsafe( + adapter, + skipUnchangedReplace, + /* restoreResetBaseline */ false, + /* replayDemand */ false, + continuesHydrationCycle, + ) + } + private async reloadActiveSubsetsUnsafe( adapter: HydrationPersistenceAdapter, skipUnchangedReplace = false, @@ -5392,13 +5661,13 @@ class PersistedCollectionRuntime< : Array.from(this.collection?.base ?? []) .filter( ([key, row]) => - !this.collection?._hasHydratedKey(key) && + !this.holdsHydrationSeed(key) && !matchesAnySubset(row, evaluators), ) .map(([key]) => key) this.hydratedDemands.clear() - const hydrationContext = { suppliedRowKeys: new Set() } + const hydrationContext = {} this.activeHydrationContext = hydrationContext this.hydratingGeneration = lifecycleGeneration try { @@ -5473,10 +5742,6 @@ class PersistedCollectionRuntime< ) return - for (const key of mergedRows.keys()) { - hydrationContext.suppliedRowKeys.add(key) - } - if (activeSubsetOptions.length === 0) { // Retired demand does not hide already public rows. Validate only // those keys; do not publish unrelated durable rows without demand. @@ -5492,30 +5757,17 @@ class PersistedCollectionRuntime< ), ) } else if ( - skipUnchangedReplace && - this.matchesCollectionSnapshot(mergedRows, collectionMetadata) + // Equal public rows do not republish. + !skipUnchangedReplace || + !this.matchesCollectionSnapshot(mergedRows, collectionMetadata) ) { - // A full replacement would also clear metadata for missing rows. - // Clear only those entries so equal public rows do not republish. - const orphanKeys = - this.collection?._syncedRowMetadataKeysOutsideBase() ?? [] - if (orphanKeys.length > 0) { - const applied = this.withInternalApply(() => { - this.syncControls.begin?.() - for (const key of orphanKeys) { - this.syncControls.metadata?.row.delete(key) - } - return this.syncControls.commit?.() ?? true - }) - await whenSyncAccepted(applied) - } - } else { + const rows = Array.from(mergedRows, ([key, row]) => ({ + key, + value: row.value, + metadata: row.metadata, + })) const applied = this.replaceCollectionSnapshot( - Array.from(mergedRows.entries()).map(([key, row]) => ({ - key, - value: row.value, - metadata: row.metadata, - })), + rows, collectionMetadata, replayDemand, ) @@ -5823,19 +6075,10 @@ function createWrappedSyncConfig< const bindToCurrentHydration = ( transaction: OpenSyncTransaction, ) => { + // A transaction binds to a storage read when it commits (decision 12). if (transaction.internal || !runtime.isHydratingNow()) return - const hydrationSequence = runtime.getHydrationSequence() - if ( - transaction.hydrationSequence !== undefined && - transaction.hydrationSequence !== hydrationSequence - ) { - throw new InvalidPersistedCollectionConfigError( - `a persisted sync transaction cannot cross a hydration cycle`, - ) - } transaction.queuedBecauseHydrating = true - transaction.hydrationSequence = hydrationSequence - transaction.hydrationContext = runtime.getActiveHydrationContext() + transaction.hydrationSequence = runtime.getHydrationSequence() } const assertHydrationSequenceCurrent = ( transaction: OpenSyncTransaction, @@ -5864,18 +6107,29 @@ function createWrappedSyncConfig< if (index === -1) return pendingPublicationTransactions.splice(index, 1) // A deferred transaction that never reaches the Collection releases - // the optimistic rows it held. - if (!transaction.handedToCollection) + // the optimistic rows it held, and enclosing inserts judged against + // its writes are judged again without them. + if (!transaction.handedToCollection) { params.collection._state.recomputeOptimisticState(false) + rejudgeEnclosing( + transaction.truncate ? undefined : transaction.operationKeys, + ) + } } // Committed source transactions not yet handed to the Collection. Core - // holds settled optimistic rows on their keys until they apply. - params.collection._state.setDeferredSyncKeys((key) => - pendingPublicationTransactions.some( - (transaction) => - !transaction.handedToCollection && - transaction.operationKeys.has(key), - ), + // holds settled optimistic rows on their keys until they apply, and a + // late hydration seed cannot supersede them. + params.collection._state.setDeferredSyncKeys( + (key) => + pendingPublicationTransactions.some( + (transaction) => + !transaction.handedToCollection && + transaction.operationKeys.has(key), + ), + () => + pendingPublicationTransactions.filter( + (transaction) => !transaction.handedToCollection, + ), ) // A source receipt is accepted once its rows are durable, and resolves // when core makes them visible. @@ -5899,9 +6153,99 @@ function createWrappedSyncConfig< } const metadataValue = (write: MetadataWrite) => write.type === `delete` ? undefined : write.value - // A metadata read sees the open transaction's write, then the newest - // staged one, then the Collection's. A truncate hides older row - // metadata. Reading a staged write makes its transaction a dependency. + // The row that the source's staged writes leave, in commit order then + // the open transactions, or undefined when none writes the key. Each + // transaction keeps its row per key, so the read folds one entry per + // transaction. `before` stops at one transaction, using the row it had + // left just before one of its inserts. + const stagedRow = ( + key: TKey, + before?: { + transaction: OpenSyncTransaction + row?: StagedRow + }, + ): { present: boolean; value?: T } | undefined => { + let row: { present: boolean; value?: T } | undefined + for (const transactions of [ + pendingPublicationTransactions, + transactionStack, + ]) + for (const transaction of transactions) { + if (transaction.truncate) row = { present: false } + const staged = + transaction === before?.transaction + ? before.row + : transaction.stagedRows.get(key) + if (staged) + row = staged.partial + ? { + present: true, + value: { + ...(row === undefined + ? params.collection._state.getAcceptedSyncedRow(key) + : row.value), + ...staged.value, + } as T, + } + : staged + if (transaction === before?.transaction) return row + } + return row + } + // A transaction begun inside an open one commits ahead of it, so an + // enclosing insert of a key it wrote can land on a different row, or + // on no row. An insert replaces the row it lands on (decision C1), so + // the enclosing transaction stays valid, but whether its insert clears + // metadata follows the row it now lands on, as in core. + // Only the keys that landed ahead are judged again; without keys, as + // after a truncate, every key is. + const rejudgeEnclosing = (written?: Iterable) => { + for (const transaction of transactionStack) { + const keys = written ?? transaction.implicitMetadata.keys() + // An insert without metadata clears it unless it now re-inserts; + // when every one re-inserts, the write before them stands. + for (const key of keys) { + const implicit = transaction.implicitMetadata.get(key) + if (!implicit) continue + const clears = implicit.inserts.some( + (insert) => + !isEqualReinsert(key, insert.value, { + transaction, + row: insert.before, + }), + ) + if (clears) + transaction.rowMetadataWrites.set(key, { type: `delete` }) + else if (implicit.previous) + transaction.rowMetadataWrites.set(key, implicit.previous) + else transaction.rowMetadataWrites.delete(key) + } + } + } + // An insert equal to a row a source wrote in this run, this tab's or a + // peer's, keeps its metadata (the row-metadata rules of 2026-10-05 and + // decision 11); over a cached or seeded row it is an insert. Without a + // staged write, core classifies the insert against its accepted rows. + const isEqualReinsert = ( + key: TKey, + value: T, + before?: { + transaction: OpenSyncTransaction + row?: StagedRow + }, + ): boolean => { + const staged = stagedRow(key, before) + if (staged) return staged.present && deepEquals(staged.value, value) + return ( + runtime.isRunWritten(key) && + params.collection._state.classifyPendingSyncedInsert(key, value) === + `update` + ) + } + // A metadata read sees the open transactions' writes, innermost first, + // then the newest staged one, then the Collection's. A truncate hides + // older row metadata. Reading a staged write makes its transaction a + // dependency. const readStagedMetadata = ( key: K, getMap: ( @@ -5911,10 +6255,12 @@ function createWrappedSyncConfig< stored: () => unknown, ): unknown => { if (startupState.cleanedUp) return undefined - const openTransaction = getOpenTransaction() - const openWrite = openTransaction && getMap(openTransaction).get(key) - if (openWrite) return metadataValue(openWrite) - if (truncates && openTransaction?.truncate) return undefined + for (let index = transactionStack.length - 1; index >= 0; index--) { + const transaction = transactionStack[index]! + const write = getMap(transaction).get(key) + if (write) return metadataValue(write) + if (truncates && transaction.truncate) return undefined + } for ( let index = pendingPublicationTransactions.length - 1; index >= 0; @@ -6043,30 +6389,40 @@ function createWrappedSyncConfig< } try { assertHydrationSequenceCurrent(transaction) - for (const key of transaction.deferredHydrationMetadataDeleteKeys) { - if ( - !transaction.hydrationContext?.suppliedRowKeys.has(key) && - !transaction.rowMetadataWrites.has(key) - ) { - transaction.rowMetadataWrites.set(key, { type: `delete` }) - } - } - transaction.deferredHydrationMetadataDeleteKeys.clear() params.begin(transaction.beginOptions) if (transaction.truncateOptions) params.truncate(transaction.truncateOptions) - for (const operation of transaction.operations) { + if (transaction.seedBases) { + for (const key of transaction.operationKeys) + if (runtime.holdsHydrationSeed(key)) + transaction.seedBases.set( + key, + params.collection._state.getAcceptedSyncedRow(key), + ) + } + transaction.operations.forEach((operation) => { if (operation.type === `delete`) { params.write({ type: `delete`, key: operation.key }) - } else { - params.write({ - type: `update`, - value: operation.value, - metadata: operation.metadata, - }) + return } - } + // An insert names the whole row, so it replaces the row it lands + // on: a seeded or cached row, which is older than every source + // write, or a row an enclosing open transaction hid when it was + // written and this insert committed ahead of. The wrapper judged + // its metadata when the source wrote it: an insert that clears + // metadata staged a delete, and an equal re-insert keeps it + // without writing it. Core applies it as an update that names the + // whole row, so only those writes change its metadata. + params.write({ + type: `update`, + value: operation.value, + metadata: operation.metadata, + ...(transaction.insertOperations.has(operation) + ? { [syncWriteNamesWholeRow]: true } + : {}), + } as ChangeMessageOrDeleteKeyMessage) + }) if (params.metadata) { for (const [key, write] of transaction.rowMetadataWrites) { applyMetadataWrite(params.metadata.row, key, write) @@ -6080,6 +6436,14 @@ function createWrappedSyncConfig< // aborted; only then do core's own holds cover its keys. transaction.handedToCollection = signal?.aborted !== true const applied = params.commit(signal) + // Only a transaction core received is a source write of this run; + // an abandoned one wrote nothing. + if (!transaction.internal && transaction.handedToCollection) + runtime.noteRunWrites( + transaction.operationKeys, + transaction.rowMetadataWrites.keys(), + transaction.truncate, + ) transaction.applicationReceipt = applied // Core accepted the transaction, so it is no longer staged here; // the receipt resolves when it is visible. @@ -6103,6 +6467,9 @@ function createWrappedSyncConfig< fail: params.fail, }, ) + // Rows a storage read or a peer's notice lands in the Collection come + // ahead of the open source transactions (decisions 7, 10, and 11). + runtime.setInternalApplyListener(rejudgeEnclosing) const findReservedTransaction = () => transactionStack.find( @@ -6260,29 +6627,20 @@ function createWrappedSyncConfig< const transaction: OpenSyncTransaction = { operations: [], partialUpdateOperationIndexes: new Set(), + insertOperations: new Set(), + implicitMetadata: new Map(), + stagedRows: new Map(), rowMetadataWrites: new Map(), collectionMetadataWrites: new Map(), - deferredHydrationMetadataDeleteKeys: new Set(), truncate: options?.snapshot === true, beginOptions: options, internal, lifecycleGeneration: runtime.getLifecycleGeneration(), operationKeys: new Set(), hasDependentSuccessor: false, - queuedBecauseHydrating: - terminalFailure === undefined && - !internal && - runtime.isHydratingNow(), - hydrationSequence: - terminalFailure === undefined && !internal - ? runtime.getHydrationSequence() - : undefined, - hydrationContext: - terminalFailure === undefined && - !internal && - runtime.isHydratingNow() - ? runtime.getActiveHydrationContext() - : undefined, + // A transaction binds to a storage read when it commits + // (decision 12), so it may stay open across one. + queuedBecauseHydrating: false, ...(terminalFailure === undefined ? {} : { terminalFailure }), } transactionStack.push(transaction) @@ -6300,8 +6658,10 @@ function createWrappedSyncConfig< if (!openTransaction) { throw new NoPendingSyncTransactionWriteError() } - bindToCurrentHydration(openTransaction) const normalization = runtime.normalizeSyncWriteMessage(message) + const reinserts = + message.type === `insert` && + isEqualReinsert(normalization.operation.key, message.value) if ( message.type === `update` && @@ -6322,8 +6682,39 @@ function createWrappedSyncConfig< } } + if (message.type === `insert` && !openTransaction.internal) + openTransaction.insertOperations.add(normalization.operation) openTransaction.operations.push(normalization.operation) openTransaction.operationKeys.add(normalization.operation.key) + const staged = openTransaction.stagedRows.get( + normalization.operation.key, + ) + openTransaction.stagedRows.set( + normalization.operation.key, + normalization.operation.type === `delete` + ? { present: false, partial: false } + : openTransaction.partialUpdateOperationIndexes.has( + openTransaction.operations.length - 1, + ) + ? { + present: true, + value: { + ...staged?.value, + ...normalization.operation.value, + }, + partial: staged?.partial ?? true, + } + : { + present: true, + value: normalization.operation.value, + partial: false, + }, + ) + // A write that names the row's metadata ends the inserts that + // decided it. An update without metadata keeps it, and an insert + // without metadata joins those inserts. + if (message.type === `delete` || message.metadata !== undefined) + openTransaction.implicitMetadata.delete(normalization.operation.key) if (normalization.operation.type === `delete`) { openTransaction.rowMetadataWrites.set(normalization.operation.key, { type: `delete`, @@ -6332,25 +6723,19 @@ function createWrappedSyncConfig< message.type === `insert` && normalization.operation.metadata === undefined ) { - // Reset stale metadata for a fresh insert, but don't clobber an - // explicit metadata write already queued for this key in the same - // transaction (e.g. query reconcile stamps owners, then inserts). - if (openTransaction.queuedBecauseHydrating) { - openTransaction.deferredHydrationMetadataDeleteKeys.add( - normalization.operation.key, - ) - } else if ( - !openTransaction.rowMetadataWrites.has( - normalization.operation.key, - ) - ) { - openTransaction.rowMetadataWrites.set( - normalization.operation.key, - { - type: `delete`, - }, - ) - } + // An insert names the row's whole metadata, so one without + // metadata clears it; the last write for a key wins. + const key = normalization.operation.key + const implicit = openTransaction.implicitMetadata.get(key) + const insert = { value: message.value, before: staged } + if (implicit) implicit.inserts.push(insert) + else + openTransaction.implicitMetadata.set(key, { + inserts: [insert], + previous: openTransaction.rowMetadataWrites.get(key), + }) + if (!reinserts) + openTransaction.rowMetadataWrites.set(key, { type: `delete` }) } else if (normalization.operation.metadata !== undefined) { openTransaction.rowMetadataWrites.set(normalization.operation.key, { type: `set`, @@ -6370,6 +6755,7 @@ function createWrappedSyncConfig< () => params.metadata!.row.get(key), ), set: (key: TKey, value: unknown) => { + getOpenTransaction()?.implicitMetadata.delete(key) updateMetadataWrite( key, { type: `set`, value }, @@ -6378,6 +6764,7 @@ function createWrappedSyncConfig< ) }, delete: (key: TKey) => { + getOpenTransaction()?.implicitMetadata.delete(key) updateMetadataWrite( key, { type: `delete` }, @@ -6435,12 +6822,11 @@ function createWrappedSyncConfig< for (const owner of new Set(pendingOwners.values())) { markPendingMetadataDependency(owner) } - const openTransaction = getOpenTransaction() - if (openTransaction) { + for (const transaction of transactionStack) { for (const [ key, metadataWrite, - ] of openTransaction.collectionMetadataWrites) { + ] of transaction.collectionMetadataWrites) { applyMetadataWrite(merged, key, metadataWrite) } } @@ -6467,8 +6853,10 @@ function createWrappedSyncConfig< openTransaction.operations = [] openTransaction.operationKeys.clear() openTransaction.partialUpdateOperationIndexes.clear() + openTransaction.insertOperations.clear() + openTransaction.implicitMetadata.clear() + openTransaction.stagedRows.clear() openTransaction.rowMetadataWrites.clear() - openTransaction.deferredHydrationMetadataDeleteKeys.clear() // Intentionally preserve collectionMetadataWrites across truncate. // Callers (for example electric resume/reset handling) may stage // collection-scoped metadata before truncating row data, and those @@ -6506,25 +6894,19 @@ function createWrappedSyncConfig< const error = new SyncTransactionAbortedError() return abortPublicationTransaction(openTransaction, error) } - try { - bindToCurrentHydration(openTransaction) - assertHydrationSequenceCurrent(openTransaction) - } catch (error) { - settlePublicationAdmissionWaiters(openTransaction) - throw error - } + bindToCurrentHydration(openTransaction) const transaction = { operations: openTransaction.operations, partialUpdateOperationIndexes: openTransaction.partialUpdateOperationIndexes, + insertOperations: openTransaction.insertOperations, rowMetadataWrites: openTransaction.rowMetadataWrites, collectionMetadataWrites: openTransaction.collectionMetadataWrites, - deferredHydrationMetadataDeleteKeys: - openTransaction.deferredHydrationMetadataDeleteKeys, - hydrationContext: openTransaction.hydrationContext, hydrationSequence: openTransaction.hydrationSequence, truncate: openTransaction.truncate, internal: false, + // The handoff fills it; the durable step reads it. + seedBases: (openTransaction.seedBases = new Map()), lifecycleGeneration: openTransaction.lifecycleGeneration, expectedResumeGenerationOwner: openTransaction.expectedResumeGenerationOwner, @@ -6543,7 +6925,16 @@ function createWrappedSyncConfig< openTransaction.operations = baselineOperations.concat( openTransaction.operations, ) - for (const { key, metadata } of rows) { + for (const { key, value, metadata } of rows) { + // The baseline row comes first, under the transaction's own + // writes of the key. + const staged = openTransaction.stagedRows.get(key) + if (!staged || staged.partial) + openTransaction.stagedRows.set(key, { + present: true, + value: { ...value, ...staged?.value }, + partial: false, + }) if ( metadata !== undefined && !openTransaction.rowMetadataWrites.has(key) @@ -6561,6 +6952,11 @@ function createWrappedSyncConfig< } openTransaction.signal = signal pendingPublicationTransactions.push(openTransaction) + rejudgeEnclosing( + openTransaction.truncate + ? undefined + : openTransaction.operationKeys, + ) if ( openTransaction.queuedBecauseHydrating && runtime.isHydratingNow() diff --git a/packages/db-sqlite-persistence-core/tests/persisted-demand.test.ts b/packages/db-sqlite-persistence-core/tests/persisted-demand.test.ts index 6327bdafd4..157e49e1db 100644 --- a/packages/db-sqlite-persistence-core/tests/persisted-demand.test.ts +++ b/packages/db-sqlite-persistence-core/tests/persisted-demand.test.ts @@ -805,7 +805,11 @@ describe(`persistedCollectionOptions`, () => { } }) - it(`rejects a transaction begun before hydration at its write cut`, async () => { + // Decision 12: a transaction begun before a storage read may write and + // commit while it runs. When the read fails, its partial update publishes + // over the recovered durable baseline, as one begun during the read does + // (the next test). + it(`commits a transaction begun before a failing storage read`, async () => { const requestFailure = new Error(`incremental subset rejected`) const baseline: Todo = { id: `shared`, @@ -849,32 +853,26 @@ describe(`persistedCollectionOptions`, () => { void load.catch(() => undefined) await loadEntered.promise - expect(() => - sourceParams.write({ - type: `update`, - value: { id: baseline.id, title: `Source update` }, - }), - ).toThrow(`cannot cross a hydration cycle`) - expect(() => sourceParams.commit()).toThrow( - `cannot cross a hydration cycle`, - ) + sourceParams.write({ + type: `update`, + value: { id: baseline.id, title: `Source update` }, + }) + const receipt = Promise.resolve(sourceParams.commit()) rejectLoad.resolve() await expect(load).rejects.toBe(requestFailure) + await receipt + const expected: Todo = { ...baseline, title: `Source update` } expect({ status: collection.status, publicError: collection._lifecycle.getSyncError(), publicRow: stripVirtualProps(collection.get(baseline.id)), durableRow: adapter.rows.get(baseline.id), - recoverySnapshotCalls: adapter.loadResumeSnapshotCalls.filter( - ({ includeRows }) => includeRows === true, - ).length, }).toEqual({ status: `ready`, publicError: undefined, - publicRow: undefined, - durableRow: baseline, - recoverySnapshotCalls: 0, + publicRow: expected, + durableRow: expected, }) } catch (error) { hasPrimaryFailure = true diff --git a/packages/db-sqlite-persistence-core/tests/persisted-oracle.test.ts b/packages/db-sqlite-persistence-core/tests/persisted-oracle.test.ts index 2953a91c78..92cd8050fb 100644 --- a/packages/db-sqlite-persistence-core/tests/persisted-oracle.test.ts +++ b/packages/db-sqlite-persistence-core/tests/persisted-oracle.test.ts @@ -2298,11 +2298,14 @@ describeUnlessOracleReplay(`persistedCollectionOptions`, () => { limit: index + 1, })) // Model complete source obligations independently of production queues. - // Hydration establishes the baseline first; source patches retain detail. - // Fresh inserts reset metadata, while updates keep baseline ownership. - // Row values retain the pre-existing quiescent wrapper semantics: source - // inserts are upserts, and partial mode retains omitted baseline fields. - // Metadata reset does not imply replacement of the complete row value. + // Hydration establishes the baseline first. A cached baseline row is a + // snapshot older than every source write, like a hydration seed + // (maintainer decision C1 of 2026-10-23, recorded in + // docs/contributing/oracle-reviews/2026-10-22-sync-metadata-session.md): + // a source insert names the whole row, so it replaces the baseline row + // and resets its metadata, while a partial update merges over it and + // keeps baseline ownership. A later partial update merges over the row + // the first source write left. const sourceTitles = hasSuccessor ? [`first source value`, `second source value`] : [`first source value`] @@ -2312,7 +2315,10 @@ describeUnlessOracleReplay(`persistedCollectionOptions`, () => { { type: `write`, transactionId: title, - row: { id: `shared`, title, detail: `retained baseline` }, + row: + type === `insert` + ? { id: `shared`, title } + : { id: `shared`, title, detail: `retained baseline` }, }, { type: `commit`, transactionId: title }, ]), @@ -2868,15 +2874,17 @@ describeUnlessOracleReplay(`persistedCollectionOptions`, () => { }, ) - // The same targeted path must respect DbClient hydration authority. A seed - // key may also exist in older durable storage. Active complete and filtered - // demand, and retired demand, cannot let a peer delta replace that - // source-owned value. The filtered case makes the durable row leave the - // predicate while the hydrated seed still matches it. The ordinary row - // beside it proves the notice was processed, so a - // subject that simply drops all targeted messages cannot pass. The model is - // the authored durable snapshot with the protected seed overlaid; public - // rows and change keys are checked after the targeted commit publishes. + // A peer's notice reports a source write in this run, and the last write + // wins: it replaces a DbClient seed this tab holds, row and metadata + // (decision 11 in docs/contributing/oracle-reviews/ + // 2026-10-22-sync-metadata-session.md). Before that decision this test held + // the seed against the peer. Active complete and filtered demand, and + // retired demand, each publish the peer's durable result for the seed key + // as for any public row. The filtered case makes the durable row leave the + // predicate. The ordinary row beside it proves the notice was processed. + // The model is the authored durable snapshot; public rows, the seed key's + // metadata, and change keys are checked after the targeted commit + // publishes. it.each( ([`active demand`, `active filter`, `retired demand`] as const).flatMap( (route) => @@ -2890,7 +2898,7 @@ describeUnlessOracleReplay(`persistedCollectionOptions`, () => { ), ), )( - `preserves a hydrated seed and metadata during a targeted $mutation with $route ($metadataAction)`, + `replaces a hydrated seed and metadata with a peer's targeted $mutation with $route ($metadataAction)`, async ({ route, mutation, metadataAction }) => { const ordinary = route === `active filter` @@ -3106,24 +3114,27 @@ describeUnlessOracleReplay(`persistedCollectionOptions`, () => { await flushAsyncWork() const expected = route === `active filter` - ? new Map([ - ...expectedTargetedSnapshot(before, authoredDurable, `Keep`), - [seed.id, seed] as const, - ]) + ? expectedTargetedSnapshot(before, authoredDurable, `Keep`) : expectedScopedSnapshot( before, authoredDurable, - new Map([[seed.id, seed]]), + new Map(), route === `active demand`, ) expectExactSnapshot(publicRows(), expected) - expect(sourceParams.metadata?.row.get(seed.id)).toEqual(seedMetadata) + // Row metadata lives with a public row, so a seed key that the peer + // deleted or moved out of demand keeps none. + expect(sourceParams.metadata?.row.get(seed.id)).toEqual( + metadataAction === `set` && expected.has(seed.id) + ? { owner: `peer durable` } + : undefined, + ) expect(publishedSnapshots).toHaveLength(1) for (const snapshot of publishedSnapshots) expectExactSnapshot(snapshot, expected) expect(publishedOrdinaryValues).toEqual([durableOrdinary]) expect(publishedKeys).toContain(ordinary.id) - expect(publishedKeys).not.toContain(seed.id) + expect(publishedKeys).toContain(seed.id) } catch (error) { hasPrimaryFailure = true throw error diff --git a/packages/db-sqlite-persistence-core/tests/persisted-reconciliation.test.ts b/packages/db-sqlite-persistence-core/tests/persisted-reconciliation.test.ts index b7c979ab63..2a0ed75606 100644 --- a/packages/db-sqlite-persistence-core/tests/persisted-reconciliation.test.ts +++ b/packages/db-sqlite-persistence-core/tests/persisted-reconciliation.test.ts @@ -198,10 +198,11 @@ it(`does not request a rotated in-flight subset twice`, async () => { // checks the persisted wrapper and mounted dependent graph, not SQLite or a // browser transport. The notification's explicit flag reaches the equal-seq // branch that ordinary tx deduplication would skip. An orphan row-metadata -// entry is not part of the loaded row snapshot; an unchanged-row shortcut -// must still remove it, as an ordinary full replacement does. The held subset -// read exposes the pending cut; the final comparison waits for the public -// metadata cleanup and checks that neither Collection published a row event. +// entry is not part of the loaded row snapshot; this tab keeps it until a +// source deletes the row (decision 16 in docs/contributing/oracle-reviews/ +// 2026-10-22-sync-metadata-session.md; this test cleared it before). The held +// subset read exposes the pending cut; the final comparison follows the +// read's return and checks that neither Collection published a row event. it(`does not republish unchanged rows for a repeated reconciliation notice`, async () => { const adapter = createRecordingAdapter([{ id: `crossing`, title: `settled` }]) const coordinator = createCoordinatorHarness() @@ -244,10 +245,13 @@ it(`does not republish unchanged rows for a repeated reconciliation notice`, asy }) try { const loadSubset = adapter.loadSubset + const reloadReturned = createDeferred() adapter.loadSubset = async (...args) => { reloadEntered.resolve() await releaseReload.promise - return loadSubset(...args) + const rows = await loadSubset(...args) + reloadReturned.resolve() + return rows } coordinator.emit( { @@ -273,8 +277,9 @@ it(`does not republish unchanged rows for a repeated reconciliation notice`, asy liveEvents: [], }) releaseReload.resolve() - await vi.waitFor(() => - expect(source.metadata?.row.get(`ghost`)).toBeUndefined(), + await atPersistedOracleCheckpoint( + reloadReturned.promise, + `reconciliation reload returned`, ) await flushAsyncWork() expect({ @@ -290,7 +295,7 @@ it(`does not republish unchanged rows for a repeated reconciliation notice`, asy liveEvents: [], sourceIds: [`crossing`], liveIds: [`crossing`], - orphanMetadata: undefined, + orphanMetadata: { stale: true }, status: `ready`, liveStatus: `ready`, }) diff --git a/packages/db-sqlite-persistence-core/tests/persisted-reload-metadata.test.ts b/packages/db-sqlite-persistence-core/tests/persisted-reload-metadata.test.ts index 311b058a1d..9279d5c4a2 100644 --- a/packages/db-sqlite-persistence-core/tests/persisted-reload-metadata.test.ts +++ b/packages/db-sqlite-persistence-core/tests/persisted-reload-metadata.test.ts @@ -264,7 +264,11 @@ describe(`persistedCollectionOptions`, () => { await readyPromise }) - it(`preserves metadata only for rows supplied by the owning hydration`, async () => { + // A row the storage read supplies is a previous run's cache, a seed + // (decision 9). A source insert without metadata names the whole row, so + // it clears that row's metadata as it does for any seed (decision 12: a + // write during a storage read follows the same rules). + it(`clears a cached row's metadata when the source inserts it during the owning hydration`, async () => { const adapter = createRecordingAdapter([ { id: `hydrated`, title: `Persisted baseline` }, ]) @@ -338,9 +342,12 @@ describe(`persistedCollectionOptions`, () => { }).toEqual({ publicHydrated: { id: `hydrated`, title: `Source replacement` }, publicNew: { id: `new`, title: `Genuinely new row` }, - hydratedMetadata: { source: `persisted` }, + hydratedMetadata: undefined, newMetadata: undefined, - durableMetadataMutations: [{ type: `delete`, key: `new` }], + durableMetadataMutations: [ + { type: `delete`, key: `hydrated` }, + { type: `delete`, key: `new` }, + ], }) } catch (error) { hasPrimaryFailure = true @@ -480,8 +487,9 @@ describe(`persistedCollectionOptions`, () => { Promise.allSettled(receipts), `source begun during hydration settled`, ) - // Accepted source writes follow the baseline; metadata survival belongs - // to the rows supplied by the original owning hydration, not later loads. + // Accepted source writes follow the baseline. The source's insert + // without metadata names the whole row, so it clears the cached row's + // metadata whichever load supplied it (decisions 9 and 12). expect({ outcomes, loads, @@ -500,7 +508,7 @@ describe(`persistedCollectionOptions`, () => { status: `ready`, publicRow: { ...baseline, title: `source` }, durableRow: { ...baseline, title: `source` }, - metadata: ownerSuppliesRow ? { owner: `persisted` } : undefined, + metadata: undefined, }) } catch (error) { hasPrimaryFailure = true @@ -977,14 +985,16 @@ describe(`persistedCollectionOptions`, () => { } }) - // A full-reload notice makes row metadata for absent public keys obsolete, - // even when no subset demand remains. The reference snapshot has zero orphan - // metadata keys; a public row's presence changes whether the wrapper reads - // a durable row snapshot, but not that judgment. The sync metadata API is - // the observable boundary at the settled notice checkpoint. A source - // metadata-only transaction establishes the orphan through the real path. + // Row metadata of a key with no row is not durable, but this tab keeps it + // until a source deletes the row (decision 16 in docs/contributing/ + // oracle-reviews/2026-10-22-sync-metadata-session.md; a full-reload notice + // cleared it before). That holds when no subset demand remains: a public + // row's presence changes whether the wrapper reads a durable row snapshot, + // not that judgment. The sync metadata API is the observable boundary at + // the settled notice checkpoint. A source metadata-only transaction + // establishes the orphan through the real path. it.each([false, true])( - `clears orphan row metadata after a no-demand peer reload (public row=%s)`, + `keeps orphan row metadata after a no-demand peer reload (public row=%s)`, async (hasPublicRow) => { const row = { id: `visible`, title: `Visible row` } const adapter = createRecordingAdapter(hasPublicRow ? [row] : []) @@ -1047,7 +1057,7 @@ describe(`persistedCollectionOptions`, () => { orphan: source.metadata?.row.get(`ghost`), publicRow: stripVirtualProps(collection.get(row.id)), }).toEqual({ - orphan: undefined, + orphan: { obsolete: true }, publicRow: hasPublicRow ? row : undefined, }) } catch (error) { @@ -1934,7 +1944,11 @@ describe(`persistedCollectionOptions`, () => { }, ) - it(`rejects a source transaction that crosses a hydration cycle`, async () => { + // A source transaction binds to a storage read when it commits, so it may + // stay open across one (decision 12). Its insert is the last write to the + // row: it follows the read's row, and without metadata it clears the + // cached row's metadata. + it(`commits a source transaction that stays open across a hydration cycle`, async () => { const adapter = createRecordingAdapter() const firstEntered = createEventGate() const releaseFirst = createEventGate() @@ -1995,11 +2009,10 @@ describe(`persistedCollectionOptions`, () => { const second = Promise.resolve(collection._sync.loadSubset(secondOptions)) await secondEntered.promise - expect(() => sourceParams.commit()).toThrow( - `cannot cross a hydration cycle`, - ) + const receipt = Promise.resolve(sourceParams.commit()) releaseSecond.resolve() await second + await receipt expect({ publicRow: stripVirtualProps(collection.get(`shared`)), @@ -2007,9 +2020,9 @@ describe(`persistedCollectionOptions`, () => { durableMetadata: adapter.rowMetadata.get(`shared`), status: collection.status, }).toEqual({ - publicRow: { id: `shared`, title: `second hydration` }, - durableRow: { id: `shared`, title: `second hydration` }, - durableMetadata: { owner: `second hydration` }, + publicRow: { id: `shared`, title: `source value` }, + durableRow: { id: `shared`, title: `source value` }, + durableMetadata: undefined, status: `ready`, }) } catch (error) { @@ -2027,7 +2040,9 @@ describe(`persistedCollectionOptions`, () => { } }) - it(`rejects a source transaction begun before hydration and committed after it`, async () => { + // Decision 12: a transaction begun before a storage read and committed + // after it applies. Its insert is the last write to the row. + it(`commits a source transaction begun before hydration and committed after it`, async () => { const adapter = createRecordingAdapter() const hydrationEntered = createEventGate() const releaseHydration = createEventGate() @@ -2077,18 +2092,16 @@ describe(`persistedCollectionOptions`, () => { releaseHydration.resolve() await hydration - expect(() => sourceParams.commit()).toThrow( - `cannot cross a hydration cycle`, - ) + await Promise.resolve(sourceParams.commit()) expect({ publicRow: stripVirtualProps(collection.get(`shared`)), durableRow: adapter.rows.get(`shared`), durableMetadata: adapter.rowMetadata.get(`shared`), status: collection.status, }).toEqual({ - publicRow: { id: `shared`, title: `hydrated value` }, - durableRow: { id: `shared`, title: `hydrated value` }, - durableMetadata: { owner: `hydration` }, + publicRow: { id: `shared`, title: `source value` }, + durableRow: { id: `shared`, title: `source value` }, + durableMetadata: undefined, status: `ready`, }) } catch (error) { @@ -2104,7 +2117,11 @@ describe(`persistedCollectionOptions`, () => { } }) - it(`preserves full-reload metadata only for rows supplied by that reload`, async () => { + // A row the storage read supplies is a previous run's cache, a seed + // (decision 9). A source insert without metadata names the whole row, so + // it clears that row's metadata as it does for any seed (decision 12: a + // write during a storage read follows the same rules). + it(`clears a cached row's metadata when the source inserts it during a full reload`, async () => { const adapter = createRecordingAdapter() adapter.rowMetadata.set(`hydrated`, { source: `persisted` }) adapter.rowMetadata.set(`new`, { source: `orphaned` }) @@ -2221,9 +2238,12 @@ describe(`persistedCollectionOptions`, () => { }).toEqual({ publicHydrated: { id: `hydrated`, title: `Source replacement` }, publicNew: { id: `new`, title: `Genuinely new row` }, - hydratedMetadata: { source: `persisted` }, + hydratedMetadata: undefined, newMetadata: undefined, - durableMetadataMutations: [{ type: `delete`, key: `new` }], + durableMetadataMutations: [ + { type: `delete`, key: `hydrated` }, + { type: `delete`, key: `new` }, + ], }) } catch (error) { hasPrimaryFailure = true diff --git a/packages/db-sqlite-persistence-core/tests/persisted-source-receipts.test.ts b/packages/db-sqlite-persistence-core/tests/persisted-source-receipts.test.ts index 4419cb988e..531ab20bbd 100644 --- a/packages/db-sqlite-persistence-core/tests/persisted-source-receipts.test.ts +++ b/packages/db-sqlite-persistence-core/tests/persisted-source-receipts.test.ts @@ -3101,7 +3101,12 @@ describe(`persistedCollectionOptions`, () => { } }) - it(`preserves row metadata set before a metadata-less insert in the same sync transaction`, async () => { + // The last write for a key wins (decision 12 and the row-metadata rules of + // 2026-10-05): an insert without metadata names the row's whole metadata, + // so it clears an explicit write before it in the same transaction, as it + // does outside hydration. Query writes a row before it stamps the row's + // owners, so its owners survive. + it(`clears row metadata set before a metadata-less insert in the same sync transaction`, async () => { const adapter = createRecordingAdapter() const ownership = { queryCollection: { owners: [`gc:q1`] } } const sync: SyncConfig = { @@ -3134,8 +3139,8 @@ describe(`persistedCollectionOptions`, () => { await collection.stateWhenReady() await flushAsyncWork() - expect(adapter.rowMetadata.get(`remote-1`)).toEqual(ownership) - expect(collection._state.syncedMetadata.get(`remote-1`)).toEqual(ownership) + expect(adapter.rowMetadata.get(`remote-1`)).toBeUndefined() + expect(collection._state.syncedMetadata.get(`remote-1`)).toBeUndefined() }) it(`resets stale row metadata for a metadata-less insert with no queued metadata`, async () => { diff --git a/packages/db-sqlite-persistence-core/tests/persisted-startup.test.ts b/packages/db-sqlite-persistence-core/tests/persisted-startup.test.ts index 2543ce9925..3a12db7129 100644 --- a/packages/db-sqlite-persistence-core/tests/persisted-startup.test.ts +++ b/packages/db-sqlite-persistence-core/tests/persisted-startup.test.ts @@ -303,7 +303,9 @@ describe(`persistedCollectionOptions`, () => { getKey: (row: Todo) => row.id, persistence: { adapter }, } - const options = + // Each branch creates its Collection from one options type, so the + // call never resolves overloads against a union. + const syncOptions = mode === `sync-present` ? persistedCollectionOptions({ ...base, @@ -314,8 +316,14 @@ describe(`persistedCollectionOptions`, () => { }, }, }) - : persistedCollectionOptions(base) - const first = createCollection(options) + : undefined + const localOptions = syncOptions + ? undefined + : persistedCollectionOptions(base) + const options = syncOptions ?? localOptions! + const first = syncOptions + ? createCollection(syncOptions) + : createCollection(localOptions!) const second = createCollection({ id: `unclaimed-${mode}`, getKey: (row: Todo) => row.id, @@ -444,7 +452,9 @@ describe(`persistedCollectionOptions`, () => { getKey: (row: Todo) => row.id, persistence: { adapter }, } - const options = + // Each branch creates its Collection from one options type, so the + // call never resolves overloads against a union. + const syncOptions = mode === `sync-present` ? persistedCollectionOptions({ ...base, @@ -455,8 +465,14 @@ describe(`persistedCollectionOptions`, () => { }, }, }) - : persistedCollectionOptions(base) - const first = createCollection(options) + : undefined + const localOptions = syncOptions + ? undefined + : persistedCollectionOptions(base) + const options = syncOptions ?? localOptions! + const first = syncOptions + ? createCollection(syncOptions) + : createCollection(localOptions!) const second = createCollection({ id: `post-cleanup-copy-${mode}`, getKey: (row: Todo) => row.id, diff --git a/packages/db-sqlite-persistence-core/tests/persisted-subset.test.ts b/packages/db-sqlite-persistence-core/tests/persisted-subset.test.ts index 5d89167586..fb9c8d0389 100644 --- a/packages/db-sqlite-persistence-core/tests/persisted-subset.test.ts +++ b/packages/db-sqlite-persistence-core/tests/persisted-subset.test.ts @@ -1140,8 +1140,11 @@ describe(`persistedCollectionOptions`, () => { title: `From remote`, }) expect(adapter.applyCommittedTxCalls).toHaveLength(1) + // A source insert names the whole row, so storage replaces any cached row + // instead of merging over it (decision C1 of 2026-10-23); the adapter's + // insert is an upsert, so cache and snapshot overlap stays safe. expect(adapter.applyCommittedTxCalls[0]?.tx.mutations[0]?.type).toBe( - `update`, + `insert`, ) }) }) diff --git a/packages/db-sqlite-persistence-core/tests/persisted-sync-metadata-session.test.ts b/packages/db-sqlite-persistence-core/tests/persisted-sync-metadata-session.test.ts new file mode 100644 index 0000000000..ea66b21212 --- /dev/null +++ b/packages/db-sqlite-persistence-core/tests/persisted-sync-metadata-session.test.ts @@ -0,0 +1,741 @@ +import { expect, it, vi } from 'vitest' +import { DbClient, collectionOptions, createCollection } from '@tanstack/db' +import { persistedCollectionOptions } from '../src' +import { + atPersistedOracleCheckpoint, + createCoordinatorHarness, + createDeferred, + createRecordingAdapter, + flushAsyncWork, +} from './persisted-support' +import type { Todo, TodoSyncParams } from './persisted-support' + +// Fixed witnesses for the sync metadata session through the persisted +// wrapper. The session law is owned by sync-metadata-session-oracle.ts and +// its persisted driver, sync-metadata-session-persisted.test.ts. + +// A row another tab's source wrote in this run reaches this tab when it loads +// a new demand from storage. It is a source row, not a persisted cache row, +// so a later DbClient hydration seed cannot replace it (D4; decision 9 marks +// only previous runs' rows as seeds). Recorded in +// docs/contributing/oracle-reviews/2026-10-22-sync-metadata-session.md. +it.each([`named keys`, `a full reload`] as const)( + `keeps a row another tab wrote this run when a later seed arrives (%s)`, + async (notice) => { + const adapter = createRecordingAdapter([]) + const coordinator = createCoordinatorHarness() + const collectionId = `remote-write-then-seed` + const client = new DbClient() + let source!: TodoSyncParams + const collection = client.collection( + collectionOptions( + persistedCollectionOptions({ + id: collectionId, + getKey: (row) => row.id, + syncMode: `on-demand`, + sync: { + sync: (params) => { + source = params + params.markReady() + return { loadSubset: () => true } + }, + }, + persistence: { adapter, coordinator }, + }), + ), + ) + try { + await atPersistedOracleCheckpoint(collection.preload(), `source ready`) + adapter.rows.set(`remote`, { id: `remote`, title: `written by a peer` }) + adapter.rowMetadata.set(`remote`, { peer: true }) + coordinator.emit( + { + type: `tx:committed`, + term: 1, + seq: 1, + txId: `peer-write`, + latestRowVersion: 1, + ...(notice === `named keys` + ? { + requiresFullReload: false as const, + changedRows: [ + { + key: `remote`, + value: { id: `remote`, title: `written by a peer` }, + }, + ], + deletedKeys: [], + } + : { requiresFullReload: true as const }), + }, + `peer`, + collectionId, + ) + await flushAsyncWork() + await atPersistedOracleCheckpoint( + Promise.resolve(collection._sync.loadSubset({})), + `demand loaded`, + ) + await vi.waitFor(() => + expect(collection.get(`remote`)?.title).toBe(`written by a peer`), + ) + client.hydrate({ + collections: [ + { + collectionId, + rows: [ + { + key: `remote`, + value: { id: `remote`, title: `stale seed` }, + metadata: { seeded: true }, + }, + ], + }, + ], + }) + await flushAsyncWork() + // The peer wrote the row and its metadata, so the seed supplies neither. + expect({ + title: collection.get(`remote`)?.title, + metadata: source.metadata?.row.get(`remote`), + }).toEqual({ title: `written by a peer`, metadata: { peer: true } }) + } finally { + await client.cleanup() + } + }, +) + +// A source transaction committed while a storage read runs waits, before +// its durable step, until the read ends. Aborting its signal then abandons it +// (the commit contract: an abort before acceptance). An enclosing open +// transaction's inserts were judged against it, so they are judged again +// without it: here the enclosing insert clears metadata once more. +it(`judges enclosing inserts again when a buffered inner transaction is abandoned`, async () => { + const adapter = createRecordingAdapter([]) + let source!: TodoSyncParams + const collection = createCollection( + persistedCollectionOptions({ + id: `abandoned-buffered-inner`, + getKey: (row) => row.id, + syncMode: `on-demand`, + sync: { + sync: (params) => { + source = params + params.markReady() + return { loadSubset: () => true } + }, + }, + persistence: { adapter }, + }), + ) + await atPersistedOracleCheckpoint(collection.preload(), `source ready`) + source.begin() + source.write({ + type: `insert`, + value: { id: `1`, title: `first` }, + metadata: { m: 0 }, + }) + await atPersistedOracleCheckpoint( + Promise.resolve(source.commit()), + `first row applied`, + ) + source.begin() + source.write({ type: `insert`, value: { id: `1`, title: `second` } }) + expect(source.metadata?.row.get(`1`)).toBeUndefined() + const gate = createDeferred() + const loadSubset = adapter.loadSubset + adapter.loadSubset = async (...args) => { + await gate.promise + return loadSubset(...args) + } + const loading = Promise.resolve(collection._sync.loadSubset({ limit: 5 })) + await flushAsyncWork() + source.begin() + source.write({ type: `insert`, value: { id: `1`, title: `second` } }) + const controller = new AbortController() + const receipt = source.commit(controller.signal) + // The inner equal re-insert commits ahead, so the enclosing insert keeps. + expect(source.metadata?.row.get(`1`)).toEqual({ m: 0 }) + controller.abort() + gate.resolve() + await loading + await expect(Promise.resolve(receipt)).rejects.toMatchObject({ + name: `AbortError`, + }) + await flushAsyncWork() + expect(source.metadata?.row.get(`1`)).toBeUndefined() +}) + +// A peer that deletes a row ends it for this session too. When this tab's +// storage read validates its public keys and finds the row gone, the delete +// is a source delete, so a later DbClient hydration seed cannot bring the row +// back (D4). Recorded in +// docs/contributing/oracle-reviews/2026-10-22-sync-metadata-session.md. +it(`keeps a row a peer deleted absent when a later seed arrives`, async () => { + const adapter = createRecordingAdapter([{ id: `cached`, title: `cached` }]) + const coordinator = createCoordinatorHarness() + const collectionId = `peer-delete-then-seed` + const client = new DbClient() + const collection = client.collection( + collectionOptions( + persistedCollectionOptions({ + id: collectionId, + getKey: (row) => row.id, + syncMode: `on-demand`, + sync: { + sync: (params) => { + params.markReady() + return { loadSubset: () => true } + }, + }, + persistence: { adapter, coordinator }, + }), + ), + ) + try { + await atPersistedOracleCheckpoint(collection.preload(), `source ready`) + const demand = {} + await atPersistedOracleCheckpoint( + Promise.resolve(collection._sync.loadSubset(demand)), + `cached row loaded`, + ) + expect(collection.get(`cached`)?.title).toBe(`cached`) + collection._sync.unloadSubset(demand) + adapter.rows.delete(`cached`) + coordinator.emit( + { + type: `tx:committed`, + term: 1, + seq: 1, + txId: `peer-delete`, + latestRowVersion: 1, + requiresFullReload: true, + }, + `peer`, + collectionId, + ) + await vi.waitFor(() => expect(collection.has(`cached`)).toBe(false)) + client.hydrate({ + collections: [ + { + collectionId, + rows: [ + { key: `cached`, value: { id: `cached`, title: `stale seed` } }, + ], + }, + ], + }) + await flushAsyncWork() + expect(collection.has(`cached`)).toBe(false) + } finally { + await client.cleanup() + } +}) + +// Writes while a storage read is in flight follow the same metadata rules as +// other writes: the last write for a key wins, so an insert without metadata +// clears an explicit write before it, and the source reads that clear +// (read-your-writes). Recorded in +// docs/contributing/oracle-reviews/2026-10-22-sync-metadata-session.md. +it(`clears an explicit metadata write with a later insert during a storage read`, async () => { + const adapter = createRecordingAdapter([]) + let source!: TodoSyncParams + const collection = createCollection( + persistedCollectionOptions({ + id: `insert-clears-during-read`, + getKey: (row) => row.id, + syncMode: `on-demand`, + sync: { + sync: (params) => { + source = params + params.markReady() + return { loadSubset: () => true } + }, + }, + persistence: { adapter }, + }), + ) + await atPersistedOracleCheckpoint(collection.preload(), `source ready`) + source.begin() + source.write({ + type: `insert`, + value: { id: `1`, title: `first` }, + metadata: { m: 0 }, + }) + await atPersistedOracleCheckpoint( + Promise.resolve(source.commit()), + `first row applied`, + ) + const gate = createDeferred() + const loadSubset = adapter.loadSubset + adapter.loadSubset = async (...args) => { + await gate.promise + return loadSubset(...args) + } + const loading = Promise.resolve(collection._sync.loadSubset({ limit: 5 })) + await flushAsyncWork() + source.begin() + source.metadata?.row.set(`1`, { explicit: true }) + source.write({ type: `insert`, value: { id: `1`, title: `second` } }) + expect(source.metadata?.row.get(`1`)).toBeUndefined() + const receipt = source.commit() + gate.resolve() + await loading + await receipt + await flushAsyncWork() + expect(source.metadata?.row.get(`1`)).toBeUndefined() +}) + +// A peer's write in this run replaces a DbClient seed this tab already holds +// (decision 11), and a later seed does not bring the old seed back. An equal +// re-insert of the peer's row by this tab's source keeps its metadata, as an +// equal re-insert of a source row does. Recorded in +// docs/contributing/oracle-reviews/2026-10-22-sync-metadata-session.md. +it.each([ + `replaces a held seed`, + `keeps metadata on an equal re-insert`, +] as const)( + `treats a row a peer wrote this run as a source row (%s)`, + async (check) => { + const adapter = createRecordingAdapter([]) + const coordinator = createCoordinatorHarness() + const collectionId = `peer-row-is-source-${check.replace(/ /g, `-`)}` + const client = new DbClient() + let source!: TodoSyncParams + const collection = client.collection( + collectionOptions( + persistedCollectionOptions({ + id: collectionId, + getKey: (row) => row.id, + syncMode: `on-demand`, + sync: { + sync: (params) => { + source = params + params.markReady() + return { loadSubset: () => true } + }, + }, + persistence: { adapter, coordinator }, + }), + ), + ) + try { + await atPersistedOracleCheckpoint(collection.preload(), `source ready`) + if (check === `replaces a held seed`) + client.hydrate({ + collections: [ + { + collectionId, + rows: [ + { + key: `remote`, + value: { id: `remote`, title: `earlier seed` }, + metadata: { seed: 0 }, + }, + ], + }, + ], + }) + await flushAsyncWork() + adapter.rows.set(`remote`, { id: `remote`, title: `written by a peer` }) + adapter.rowMetadata.set(`remote`, { peer: true }) + coordinator.emit( + { + type: `tx:committed`, + term: 1, + seq: 1, + txId: `peer-write`, + latestRowVersion: 1, + requiresFullReload: false, + changedRows: [ + { + key: `remote`, + value: { id: `remote`, title: `written by a peer` }, + }, + ], + deletedKeys: [], + }, + `peer`, + collectionId, + ) + await flushAsyncWork() + await atPersistedOracleCheckpoint( + Promise.resolve(collection._sync.loadSubset({})), + `demand loaded`, + ) + await vi.waitFor(() => + expect(collection.get(`remote`)?.title).toBe(`written by a peer`), + ) + if (check === `replaces a held seed`) { + client.hydrate({ + collections: [ + { + collectionId, + rows: [ + { + key: `remote`, + value: { id: `remote`, title: `stale seed` }, + metadata: { seeded: true }, + }, + ], + }, + ], + }) + } else { + source.begin() + source.write({ + type: `insert`, + value: { id: `remote`, title: `written by a peer` }, + }) + await atPersistedOracleCheckpoint( + Promise.resolve(source.commit()), + `equal re-insert applied`, + ) + } + await flushAsyncWork() + expect({ + title: collection.get(`remote`)?.title, + metadata: source.metadata?.row.get(`remote`), + }).toEqual({ title: `written by a peer`, metadata: { peer: true } }) + } finally { + await client.cleanup() + } + }, +) + +// Peer writes in this run, per object and per route (decision 11). A peer's +// notice can name only row metadata; a peer's write can reach this tab only +// through a reload after a gap the catch-up cannot fill; and a peer's row can +// arrive while this tab's source has an open insert of the same key. +// Recorded in docs/contributing/oracle-reviews/ +// 2026-10-22-sync-metadata-session.md. +it.each([ + `metadata-only notice over a held seed`, + `write whose notice was missed`, + `peer row during an open insert`, +] as const)( + `treats a peer's write in this run as a source write (%s)`, + async (variant) => { + const adapter = createRecordingAdapter([]) + const coordinator = createCoordinatorHarness() + const collectionId = `peer-route-${variant.replace(/ /g, `-`)}` + const client = new DbClient() + let source!: TodoSyncParams + const collection = client.collection( + collectionOptions( + persistedCollectionOptions({ + id: collectionId, + getKey: (row) => row.id, + syncMode: `on-demand`, + sync: { + sync: (params) => { + source = params + params.markReady() + return { loadSubset: () => true } + }, + }, + persistence: { adapter, coordinator }, + }), + ), + ) + const seed = (title: string, metadata: Record) => + client.hydrate({ + collections: [ + { + collectionId, + rows: [{ key: `k`, value: { id: `k`, title }, metadata }], + }, + ], + }) + const observe = () => ({ + title: collection.get(`k`)?.title, + metadata: source.metadata?.row.get(`k`), + }) + try { + await atPersistedOracleCheckpoint(collection.preload(), `source ready`) + if (variant === `metadata-only notice over a held seed`) { + seed(`held seed`, { seed: 0 }) + await flushAsyncWork() + adapter.rowMetadata.set(`k`, { peer: true }) + coordinator.emit( + { + type: `tx:committed`, + term: 1, + seq: 1, + txId: `peer-metadata`, + latestRowVersion: 1, + requiresFullReload: false, + changedRows: [], + deletedKeys: [], + rowMetadataMutations: [ + { type: `set`, key: `k`, value: { peer: true } }, + ], + }, + `peer`, + collectionId, + ) + await flushAsyncWork() + seed(`late seed`, { seeded: true }) + await flushAsyncWork() + // The peer wrote only metadata, so the row stays a seed's and the + // later seed wins it; the peer's metadata stays. + expect(observe()).toEqual({ + title: `late seed`, + metadata: { peer: true }, + }) + } else if (variant === `write whose notice was missed`) { + coordinator.setPullSinceResponse({ + type: `rpc:pullSince:res`, + rpcId: `pull-failed`, + ok: false, + error: `unavailable`, + } as never) + // The notice for the peer's write of `k` never arrives. The next + // notice names another key and reveals the gap. + adapter.rows.set(`k`, { id: `k`, title: `written by a peer` }) + adapter.rowMetadata.set(`k`, { peer: true }) + adapter.rows.set(`other`, { id: `other`, title: `later peer write` }) + coordinator.emit( + { + type: `tx:committed`, + term: 1, + seq: 3, + txId: `peer-after-gap`, + latestRowVersion: 3, + requiresFullReload: false, + changedRows: [ + { + key: `other`, + value: { id: `other`, title: `later peer write` }, + }, + ], + deletedKeys: [], + }, + `peer`, + collectionId, + ) + await flushAsyncWork() + await atPersistedOracleCheckpoint( + Promise.resolve(collection._sync.loadSubset({})), + `demand loaded`, + ) + await vi.waitFor(() => + expect(collection.get(`k`)?.title).toBe(`written by a peer`), + ) + seed(`stale seed`, { seeded: true }) + await flushAsyncWork() + expect(observe()).toEqual({ + title: `written by a peer`, + metadata: { peer: true }, + }) + } else { + await atPersistedOracleCheckpoint( + Promise.resolve(collection._sync.loadSubset({})), + `demand loaded`, + ) + source.begin() + source.write({ type: `insert`, value: { id: `k`, title: `mine` } }) + await atPersistedOracleCheckpoint( + Promise.resolve(source.commit()), + `first row applied`, + ) + source.begin() + // An equal re-insert of this tab's row, judged before the peer writes. + source.write({ type: `insert`, value: { id: `k`, title: `mine` } }) + adapter.rows.set(`k`, { id: `k`, title: `written by a peer` }) + adapter.rowMetadata.set(`k`, { peer: true }) + coordinator.emit( + { + type: `tx:committed`, + term: 1, + seq: 2, + txId: `peer-during-open-insert`, + latestRowVersion: 2, + requiresFullReload: false, + changedRows: [ + { key: `k`, value: { id: `k`, title: `written by a peer` } }, + ], + deletedKeys: [], + rowMetadataMutations: [ + { type: `set`, key: `k`, value: { peer: true } }, + ], + }, + `peer`, + collectionId, + ) + await flushAsyncWork() + // The peer's row lands ahead of the open insert, which now changes + // the row it lands on, so the insert clears metadata (decisions 7, + // 10, and 11); the source reads its own open clear. + expect(observe()).toEqual({ + title: `written by a peer`, + metadata: undefined, + }) + await atPersistedOracleCheckpoint( + Promise.resolve(source.commit()), + `insert applied`, + ) + await flushAsyncWork() + expect(observe()).toEqual({ title: `mine`, metadata: undefined }) + } + } finally { + await client.cleanup() + } + }, +) + +// A peer's notice can make this tab reload its demand from storage instead of +// applying the notice's rows: when it names a row this tab shows, names a +// key no demand has seen, or names more keys than a targeted update allows. +// A reload is how this tab reads storage again; it is not a source write. A +// stored row from a previous run that no source wrote in this run stays a +// seed across it, so a later `DbClient` seed still replaces it (decision 9), +// while a row the notice names is a source write that the seed cannot +// replace (decision 11). A key that storage never held is no source's +// write either, so a seed supplies it. After a full-reload notice this tab +// cannot tell which stored rows a source wrote, so a stored row counts as a +// source write only if it differs from the row this tab holds (decision +// 15): the peer's row does, the unchanged previous-run row stays a seed, and +// a later storage read for a new demand keeps that status. A row the peer +// deleted is a source's delete, so a later seed cannot bring it back (D4). +// Each variant +// reaches a different branch: no `DbClient` seed held, a held seed, a notice +// over the key limit, and unknown writes. Recorded in +// docs/contributing/oracle-reviews/2026-10-22-sync-metadata-session.md. +it.each([ + `notice naming a shown row`, + `notice naming a shown row while a seed is held`, + `notice over the targeted key limit`, + `full-reload notice`, + `full-reload notice, then a new demand reads the rows`, +] as const)( + `keeps previous-run rows seeds across a reload a peer's notice causes (%s)`, + async (variant) => { + const adapter = createRecordingAdapter([ + { id: `cached`, title: `previous run` }, + { id: `shown`, title: `previous run` }, + { id: `gone`, title: `previous run` }, + ]) + adapter.rowMetadata.set(`cached`, { cached: true }) + adapter.rowMetadata.set(`shown`, { cached: true }) + const coordinator = createCoordinatorHarness() + const collectionId = `reload-route-${variant.replace(/ /g, `-`)}` + const client = new DbClient() + let source!: TodoSyncParams + const collection = client.collection( + collectionOptions( + persistedCollectionOptions({ + id: collectionId, + getKey: (row) => row.id, + syncMode: `on-demand`, + sync: { + sync: (params) => { + source = params + params.markReady() + return { loadSubset: () => true } + }, + }, + persistence: { adapter, coordinator }, + }), + ), + ) + const seed = (key: string) => + client.hydrate({ + collections: [ + { + collectionId, + rows: [ + { + key, + value: { id: key, title: `later seed` }, + metadata: { seeded: true }, + }, + ], + }, + ], + }) + const observe = (key: string) => ({ + title: collection.get(key)?.title, + metadata: source.metadata?.row.get(key), + }) + try { + await atPersistedOracleCheckpoint(collection.preload(), `source ready`) + await atPersistedOracleCheckpoint( + Promise.resolve(collection._sync.loadSubset({})), + `demand loaded`, + ) + await vi.waitFor(() => + expect(collection.get(`cached`)?.title).toBe(`previous run`), + ) + if (variant === `notice naming a shown row while a seed is held`) { + seed(`held`) + await flushAsyncWork() + } + adapter.rows.set(`shown`, { id: `shown`, title: `written by a peer` }) + adapter.rowMetadata.set(`shown`, { peer: true }) + adapter.rows.delete(`gone`) + const unseen = Array.from({ length: 129 }, (_, index) => ({ + key: `unseen-${index}`, + value: { id: `unseen-${index}`, title: `written by a peer` }, + })) + if (variant === `notice over the targeted key limit`) + for (const row of unseen) adapter.rows.set(row.key, row.value) + coordinator.emit( + { + type: `tx:committed`, + term: 1, + seq: 1, + txId: `peer-write`, + latestRowVersion: 1, + ...(variant.startsWith(`full-reload notice`) + ? { requiresFullReload: true as const } + : { + requiresFullReload: false as const, + changedRows: [ + { + key: `shown`, + value: { id: `shown`, title: `written by a peer` }, + }, + ...(variant === `notice over the targeted key limit` + ? unseen + : []), + ], + deletedKeys: [`gone`], + rowMetadataMutations: [ + { type: `set` as const, key: `shown`, value: { peer: true } }, + ], + }), + }, + `peer`, + collectionId, + ) + await vi.waitFor(() => + expect(collection.get(`shown`)?.title).toBe(`written by a peer`), + ) + await flushAsyncWork() + if (variant === `full-reload notice, then a new demand reads the rows`) + await atPersistedOracleCheckpoint( + Promise.resolve(collection._sync.loadSubset({ limit: 10 })), + `new demand loaded`, + ) + seed(`cached`) + seed(`shown`) + seed(`fresh`) + seed(`gone`) + await flushAsyncWork() + expect({ + cached: observe(`cached`), + shown: observe(`shown`), + fresh: observe(`fresh`), + gone: observe(`gone`), + }).toEqual({ + cached: { title: `later seed`, metadata: { seeded: true } }, + shown: { title: `written by a peer`, metadata: { peer: true } }, + fresh: { title: `later seed`, metadata: { seeded: true } }, + gone: { title: undefined, metadata: undefined }, + }) + } finally { + await client.cleanup() + } + }, +) diff --git a/packages/db-sqlite-persistence-core/tests/sync-metadata-session-persisted.test.ts b/packages/db-sqlite-persistence-core/tests/sync-metadata-session-persisted.test.ts new file mode 100644 index 0000000000..bef1b39f32 --- /dev/null +++ b/packages/db-sqlite-persistence-core/tests/sync-metadata-session-persisted.test.ts @@ -0,0 +1,1265 @@ +import { fc, test as fcTest } from '@fast-check/vitest' +import { describe, expect, it } from 'vitest' +import { DbClient, collectionOptions, createCollection } from '@tanstack/db' +import { + absentMarker, + cachedMetadata, + cachedRow, + describeMismatch, + historyRegimes, + persistedCachedLane, + persistedConflictLane, + persistedFlatLane, + persistedNestedLane, + persistedNestedSeedLane, + persistedPeerFullLane, + persistedPeerLane, + persistedPeerOnDemandLane, + persistedReadLane, + persistedReadSeedLane, + persistedSeedLane, + runSessionHistory, + sessionCollectionKeys, + sessionHistoryArbitrary, +} from '../../db/tests/sync-metadata-session-oracle' +import { + oraclePropertyOptions, + oracleRuns, + readOracleRunConfig, +} from '../../db/tests/oracle-config' +import { createOracleReach } from '../../db/tests/oracle-reach' +import { SingleProcessCoordinator, persistedCollectionOptions } from '../src' +import type { CollectionConfig, SyncConfig } from '@tanstack/db' +import type { + PeerNotice, + ReceiptState, + SessionAction, + SessionDriver, + SessionLane, +} from '../../db/tests/sync-metadata-session-oracle' +import type { PersistedTx, PersistenceAdapter, ProtocolEnvelope } from '../src' + +/** + * The persisted wrapper is a second production path for the sync metadata + * session oracle. Its laws, model, grammar, and comparison are the oracle's + * own (`packages/db/tests/sync-metadata-session-oracle.ts`). The source and + * the wrapper are one session: a source read through the wrapper must give + * what the same read gives on a bare Collection. + * + * Production path: `persistedCollectionOptions` over an in-memory adapter. + * The source sync function receives the wrapper's sync API. A `hold` holds + * one kind of delay. A store hold makes the adapter's next durable write + * wait, so transactions committed after it stay deferred in the wrapper. A + * core hold is a persisting optimistic insert of a fresh fixture key, so + * transactions the wrapper has handed to core stay held there. `release` + * ends either. + * + * Checkpoints: source reads after every action; `collection.base` only at the + * final checkpoint, after every hold ends and the wrapper's continuations + * drain. The wrapper hands rows to core after its durable step, so consumers + * see them a turn or more later even with no hold. In the peer lanes the + * holding adapter's stored rows are compared at the final checkpoint too. + * + * Lanes: `persisted-flat`, `persisted-nested`, `persisted-seeds`, + * `persisted-cached` (row 1 is in storage when the run starts), + * `persisted-conflicts` (inner writes that land ahead of enclosing ones), and + * `persisted-nested-seeds`, which crosses seeds with nesting and the cached + * row, `persisted-reads`, whose read hold keeps an on-demand storage + * read in flight, `persisted-reads-seeds`, which adds seeds and the cached + * row and may abort a buffered commit's signal, and `persisted-peers`, where another tab's source writes + * storage and sends its notices through the persistence coordinator, with + * `persisted-peers-full` under full row updates and + * `persisted-peers-on-demand`, which loads on demand with a demand for every + * row active from the start. Seeds + * use a `DbClient` that owns the persisted Collection. Restart and resume + * reads are outside these lanes. + * + * A peer is a second writer the driver plays. It writes the stored row and + * metadata directly, takes the next stream position after this tab's + * durable commits, and emits the `tx:committed` notice that the Browser and + * Electron coordinators send: changed rows with their metadata, metadata + * alone, a deleted key, or a notice that requires a full reload. A reset + * empties storage, starts the next reset epoch, and sends + * `collection:reset`. The model + * makes a peer action legal only when no store hold or read hold is active, + * so every accepted transaction of this tab is durable and the notice is the + * next one in the stream. + */ + +type Row = { id: number; v: unknown } +type SyncActions = Parameters[`sync`]>[0] + +/** + * An in-memory adapter that can hold its next durable step. With `cached`, + * its reads supply row 1 from a previous run's storage until this run's + * first durable commit; the oracle observes one run. With `stores`, it keeps + * every durable row and returns them to reads, so a storage read during the + * run hands this tab's own rows back to the wrapper. + */ +function createHoldingAdapter(cached = false, stores = false) { + let holdNext = false + let releaseStore: (() => void) | undefined + let latest = { term: 0, seq: 0, rowVersion: 0 } + let resetEpoch = 0 + let cachedRows = cached + ? [{ key: 1, value: cachedRow, metadata: cachedMetadata }] + : [] + const stored = new Map( + stores && cached + ? [[1, { value: cachedRow as Row, metadata: cachedMetadata }]] + : [], + ) + const storedRows = () => + stores ? Array.from(stored, ([key, row]) => ({ key, ...row })) : cachedRows + // A reload restores collection metadata from storage, so storage keeps it. + const storedCollectionMetadata = new Map() + const collectionMetadata = () => + Array.from(storedCollectionMetadata, ([key, value]) => ({ key, value })) + const adapter: PersistenceAdapter = { + loadSubset: () => Promise.resolve(storedRows()), + loadResumeSnapshot: () => + Promise.resolve({ + rows: storedRows(), + keySet: { status: `consistent` }, + collectionMetadata: collectionMetadata(), + latestTerm: latest.term, + latestSeq: latest.seq, + latestRowVersion: latest.rowVersion, + resetEpoch, + }), + loadCollectionMetadata: () => Promise.resolve(collectionMetadata()), + applyCommittedTx: async (_target, tx: PersistedTx) => { + cachedRows = [] + if (holdNext) { + holdNext = false + await new Promise((resolve) => { + releaseStore = resolve + }) + } + latest = { term: tx.term, seq: tx.seq, rowVersion: tx.rowVersion } + if (!stores) return + if (tx.truncate) stored.clear() + for (const mutation of tx.mutations) { + const key = mutation.key as number + if (mutation.type === `delete`) stored.delete(key) + else + stored.set(key, { + value: (mutation.type === `update` + ? { ...stored.get(key)?.value, ...mutation.value } + : mutation.value) as Row, + metadata: stored.get(key)?.metadata, + }) + } + for (const mutation of tx.rowMetadataMutations ?? []) { + const row = stored.get(mutation.key as number) + if (row) + row.metadata = mutation.type === `delete` ? undefined : mutation.value + } + for (const mutation of tx.collectionMetadataMutations ?? []) + if (mutation.type === `delete`) + storedCollectionMetadata.delete(mutation.key) + else storedCollectionMetadata.set(mutation.key, mutation.value) + }, + ensureIndex: () => Promise.resolve(), + } + return { + adapter, + /** A peer's durable write at the next stream position. */ + peerWrite: ( + key: number, + notice: PeerNotice, + row: Row, + metadata: unknown, + rewrites = false, + ): typeof latest => { + if (notice === `reset`) { + // A resetting tab that writes its rows back leaves storage as it was. + if (!rewrites) { + stored.clear() + storedCollectionMetadata.clear() + } + resetEpoch++ + latest = { term: latest.term + 1, seq: 0, rowVersion: 0 } + return latest + } + if (notice === `delete`) stored.delete(key) + else if (notice === `metadata`) stored.get(key)!.metadata = metadata + else stored.set(key, { value: row, metadata }) + latest = { + term: Math.max(latest.term, 1), + seq: latest.seq + 1, + rowVersion: latest.rowVersion + 1, + } + return latest + }, + resetEpoch: () => resetEpoch, + /** The row storage holds for a key, if any. */ + storedRow: (key: number): Row | undefined => stored.get(key)?.value, + holdNextStore: () => { + holdNext = true + }, + releaseStore: () => { + holdNext = false + releaseStore?.() + releaseStore = undefined + }, + } +} + +const drainTurns = async () => { + for (let turn = 0; turn < 20; turn++) + await new Promise((resolve) => setTimeout(resolve, 0)) +} + +let run = 0 + +async function createPersistedDriver( + withClient = false, + cached = false, + reads = false, + peers = false, + fullRows = false, + onDemand = false, +): Promise { + const store = createHoldingAdapter(cached, reads || peers) + // The local coordinator persists this tab's commits through the adapter; + // `deliver` hands it a peer's notice. + let deliver: ((message: ProtocolEnvelope) => void) | undefined + const coordinator = peers + ? Object.assign(new SingleProcessCoordinator(), { + subscribe: ( + _collectionId: string, + onMessage: (message: ProtocolEnvelope) => void, + ) => { + deliver = onMessage + return () => { + deliver = undefined + } + }, + }) + : undefined + const id = `sync-metadata-session-persisted-${run++}` + let sync!: SyncActions + let resolveMutation: (() => void) | undefined + const client = withClient ? new DbClient() : undefined + const options = persistedCollectionOptions({ + id, + getKey: (row) => row.id, + // A read hold needs an on-demand load to keep in flight. + ...(reads || onDemand ? { syncMode: `on-demand` as const } : {}), + sync: { + ...(fullRows ? { rowUpdateMode: `full` as const } : {}), + sync: (actions) => { + sync = actions + actions.begin() + actions.commit() + actions.markReady() + return reads || onDemand ? { loadSubset: () => true } : undefined + }, + }, + onInsert: () => + new Promise((resolve) => { + resolveMutation = resolve + }), + persistence: { adapter: store.adapter, coordinator }, + } as CollectionConfig & { + persistence: { adapter: PersistenceAdapter } + }) + const collection = client + ? client.collection( + collectionOptions(options as CollectionConfig), + ) + : createCollection(options) + await collection.preload() + // Every row is demanded, so a reload or reset reads them all. + if (onDemand) await collection._sync.loadSubset({}) + await drainTurns() + + let depth = 0 + let holdCount = 0 + // Signals of the outermost transactions committed during a read hold. + let readCommits: Array = [] + let blocker: { isPersisted: { promise: Promise } } | undefined + const receipts: Array> = [] + const rejections: Array = [] + const receiptStates: Array = [] + const track = (receipt: true | Promise, expectAbort: boolean) => { + if (receipt === true) return + receipts.push( + receipt.then( + () => { + if (expectAbort) rejections.push(`cancelled transaction applied`) + }, + (error: unknown) => { + if ( + !expectAbort || + !(error instanceof Error) || + error.name !== `AbortError` + ) + rejections.push( + error instanceof Error + ? `${error.name}: ${error.message}` + : String(error), + ) + }, + ), + ) + } + const abort = () => { + const controller = new AbortController() + controller.abort() + track(sync.commit(controller.signal), true) + depth-- + } + const metadata = () => sync.metadata! + const read = (value: unknown) => (value === undefined ? absentMarker : value) + let readGate: { resolve: () => void } | undefined + let reading: Promise | undefined + const release = async () => { + readGate?.resolve() + readGate = undefined + readCommits = [] + await reading + reading = undefined + store.releaseStore() + const pending = blocker + blocker = undefined + resolveMutation?.() + resolveMutation = undefined + await pending?.isPersisted.promise + } + const asMetadata = (value: unknown) => + value === undefined ? {} : { metadata: value as Record } + + return { + consumerCheckpoint: `final`, + begin: () => { + sync.begin() + depth++ + }, + commit: () => { + const index = receiptStates.length + const controller = new AbortController() + if (readGate && depth === 1) readCommits.push(controller) + const receipt = sync.commit(controller.signal) + depth-- + if (receipt === true) { + receiptStates.push(`applied`) + return + } + receiptStates.push(`pending`) + receipts.push( + receipt.then( + () => { + receiptStates[index] = `applied` + }, + () => { + receiptStates[index] = `rejected` + }, + ), + ) + }, + cancel: abort, + abandon: () => readCommits.pop()!.abort(), + rowSet: (key, value) => metadata().row.set(key, value), + rowDelete: (key) => metadata().row.delete(key), + insert: (_key, value, rowMetadata) => + sync.write({ + type: `insert`, + value: value as Row, + ...asMetadata(rowMetadata), + }), + update: (_key, value, rowMetadata) => + sync.write({ + type: `update`, + value: value as Row, + ...asMetadata(rowMetadata), + }), + delete: (key) => sync.write({ type: `delete`, key }), + truncate: () => sync.truncate(), + collectionSet: (key, value) => metadata().collection.set(key, value), + collectionDelete: (key) => metadata().collection.delete(key), + hold: (holdStore, holdRead) => { + if (holdRead) { + let open!: () => void + const gate = new Promise((resolve) => { + open = resolve + }) + readGate = { resolve: open } + const loadSubset = store.adapter.loadSubset + store.adapter.loadSubset = async (...args) => { + await gate + store.adapter.loadSubset = loadSubset + return loadSubset(...args) + } + reading = Promise.resolve( + collection._sync.loadSubset({ limit: 100 + holdCount++ }), + ).catch(() => undefined) + } else if (holdStore) store.holdNextStore() + else blocker = collection.insert({ id: 1000 + holdCount++, v: 0 }) + }, + release, + seed: (key, row, rowMetadata) => + client!.hydrate({ + collections: [ + { + collectionId: id, + rows: [ + { + key, + value: row as Row, + metadata: rowMetadata as Record, + }, + ], + }, + ], + }), + peer: async (key, notice, row, metadata, rewrites) => { + const position = store.peerWrite( + key, + notice, + row as Row, + metadata, + rewrites, + ) + const named = + notice === `row` + ? { + changedRows: [{ key, value: row as Record }], + deletedKeys: [], + rowMetadataMutations: [ + { type: `set` as const, key, value: metadata }, + ], + } + : notice === `metadata` + ? { + changedRows: [], + deletedKeys: [], + rowMetadataMutations: [ + { type: `set` as const, key, value: metadata }, + ], + } + : { changedRows: [], deletedKeys: [key] } + deliver?.({ + v: 1, + dbName: `sync-metadata-session`, + collectionId: id, + senderId: `peer-tab`, + ts: Date.now(), + payload: + notice === `reset` + ? { + type: `collection:reset`, + schemaVersion: 2, + resetEpoch: store.resetEpoch(), + } + : { + type: `tx:committed`, + term: position.term, + seq: position.seq, + txId: `peer-${position.seq}`, + latestRowVersion: position.rowVersion, + ...(notice === `reload` + ? { requiresFullReload: true } + : { requiresFullReload: false, ...named }), + }, + }) + // The wrapper handles a notice under its apply lock and may reload + // storage first. A notice it never handled leaves the peer's write + // invisible, which the comparison reports, because every peer action + // the model allows changes a read except a full reload that a held + // seed keeps (decision 13). + await drainTurns() + }, + storedRows: () => ({ + 1: read(store.storedRow(1)), + 2: read(store.storedRow(2)), + }), + observe: () => ({ + rowMetadata: { + 1: read(metadata().row.get(1)), + 2: read(metadata().row.get(2)), + }, + collection: { + ...Object.fromEntries( + sessionCollectionKeys.map((key) => [ + key, + read(metadata().collection.get(key)), + ]), + ), + }, + list: sorted(metadata().collection.list()), + prefixList: sorted(metadata().collection.list(`gc:`)), + base: { + 1: read(collection.base.get(1)), + 2: read(collection.base.get(2)), + }, + receipts: [...receiptStates], + }), + flush: () => new Promise((resolve) => setTimeout(resolve, 0)), + settle: async () => { + await release() + while (depth > 0) abort() + await drainTurns() + await Promise.all(receipts) + await drainTurns() + return rejections + }, + cleanup: () => collection.cleanup(), + } +} + +function sorted( + entries: ReadonlyArray<{ key: string; value: unknown }>, +): Array<[string, unknown]> { + return [...entries] + .map(({ key, value }): [string, unknown] => [key, value]) + .sort(([a], [b]) => a.localeCompare(b)) +} + +async function assertPersistedHistory( + history: ReadonlyArray, + lane: SessionLane = persistedFlatLane, +): Promise { + const { mismatch } = await runSessionHistory(lane, history, () => + createPersistedDriver( + lane.seeds, + lane.cached === true, + lane.readHold === true, + lane.peers === true, + lane.fullRows === true, + lane.onDemand === true, + ), + ) + if (mismatch) throw new Error(describeMismatch(history, mismatch)) +} + +const property = `sync-metadata-session.persisted-flat` +const fixedSeed = 2_220_011 +const requestedReplayProperty = readOracleRunConfig().replayProperty + +describe(`sync metadata session oracle through the persisted wrapper`, () => { + it.each([false, true])( + `keeps a held cursor visible after a later commit (store hold: %s)`, + async (holdStore) => { + await assertPersistedHistory([ + { type: `hold`, store: holdStore }, + { type: `begin` }, + { type: `collectionSet`, key: `cursor` }, + { type: `commit` }, + { type: `begin` }, + { type: `upsert`, key: 1, metadata: false }, + { type: `commit` }, + ]) + }, + ) + + // The reported path: once the wrapper has handed transactions to core and + // core holds them, a read after a third transaction loses the earliest + // write. A store hold keeps them in the wrapper's staging instead; that + // neighbour passes on 30175be48. + it.each([false, true])( + `keeps a held write visible after a third transaction (store hold: %s)`, + async (holdStore) => { + await assertPersistedHistory([ + { type: `hold`, store: holdStore }, + { type: `begin` }, + { type: `upsert`, key: 1, metadata: true }, + { type: `commit` }, + { type: `begin` }, + { type: `commit` }, + { type: `begin` }, + { type: `collectionSet`, key: `cursor` }, + { type: `commit` }, + ]) + }, + ) + + // A source insert equal to a seeded row is still an insert: the source never + // wrote the row, so it names the row's metadata and clears the seed's. + it(`clears a seed's metadata when the source inserts an equal row`, async () => { + await assertPersistedHistory( + [ + { type: `seed`, key: 1 }, + { type: `begin` }, + { type: `reinsert`, key: 1, metadata: false }, + ], + persistedSeedLane, + ) + }) + + // An equal re-insert keeps metadata as an update does, without writing it, + // so the latest seed still governs the row's metadata after it commits. + it(`lets a later seed replace metadata that an accepted re-insert kept`, async () => { + await assertPersistedHistory( + [ + { type: `begin` }, + { type: `seed`, key: 1, bare: true }, + { type: `upsert`, key: 1, metadata: false }, + { type: `reinsert`, key: 1, metadata: false }, + { type: `seed`, key: 1 }, + { type: `commit` }, + { type: `seed`, key: 1, bare: true }, + ], + persistedSeedLane, + ) + }) + + // The same holds when the re-insert commits ahead of the enclosing + // transaction whose row it re-inserted and lands on a seeded row. + it(`lets a later seed replace metadata that an inner re-insert kept`, async () => { + await assertPersistedHistory( + [ + { type: `begin` }, + { type: `seed`, key: 2 }, + { type: `upsert`, key: 2, metadata: false }, + { type: `begin` }, + { type: `reinsert`, key: 2, metadata: false }, + { type: `commit` }, + { type: `seed`, key: 2, bare: true }, + ], + persistedNestedSeedLane, + ) + }) + + // A persisted cache row is a seed (decision C), so a later DbClient seed + // replaces its metadata until the source writes the metadata. + it(`lets a later seed replace a cached row's metadata`, async () => { + await assertPersistedHistory( + [ + { type: `begin` }, + { type: `upsert`, key: 1, metadata: false }, + { type: `seed`, key: 1, bare: true }, + ], + persistedNestedSeedLane, + ) + }) + + // An update without metadata keeps metadata, so the insert before it still + // decides the row's metadata. When an inner transaction commits ahead and + // changes the row that insert lands on, the insert clears the metadata. + it(`rejudges an enclosing re-insert that a later update kept`, async () => { + await assertPersistedHistory( + [ + { type: `begin` }, + { type: `upsert`, key: 1, metadata: false }, + { type: `reinsert`, key: 1, metadata: false }, + { type: `upsert`, key: 1, metadata: false }, + { type: `begin` }, + { type: `upsert`, key: 1, metadata: true }, + { type: `commit` }, + ], + persistedNestedSeedLane, + ) + }) + + // Every insert without metadata in an enclosing transaction is re-judged + // when an inner one commits ahead: an earlier re-insert that now lands on a + // different row clears metadata, and a later equal re-insert keeps that. + it(`rejudges every enclosing re-insert of a key, not only the last`, async () => { + await assertPersistedHistory( + [ + { type: `begin` }, + { type: `insertOverSeed`, key: 1, metadata: false }, + { type: `commit` }, + { type: `begin` }, + { type: `reinsert`, key: 1, metadata: false }, + { type: `reinsert`, key: 1, metadata: false }, + { type: `begin` }, + { type: `upsert`, key: 1, metadata: true }, + { type: `commit` }, + ], + persistedNestedSeedLane, + ) + }) + + // An enclosing insert that cleared metadata can become an equal re-insert + // of the row an inner transaction committed ahead; it then keeps the + // metadata a seed supplied beneath. + it(`lets an enclosing clear become an equal re-insert after an inner commit`, async () => { + await assertPersistedHistory( + [ + { type: `begin` }, + { type: `insertOverSeed`, key: 1, metadata: false }, + { type: `reinsert`, key: 1, metadata: false }, + { type: `begin` }, + { type: `reinsert`, key: 1, metadata: false }, + { type: `seed`, key: 1 }, + { type: `commit` }, + ], + persistedNestedSeedLane, + ) + }) + + // A later seed beneath an open re-insert does not change its kind, but an + // inner update that merges over that seed and commits ahead does: the + // re-insert then lands on a different row and clears metadata (decision 10). + it(`lets an inner update merged over a later seed change an enclosing re-insert`, async () => { + await assertPersistedHistory( + [ + { type: `begin` }, + { type: `upsert`, key: 1, metadata: true }, + { type: `reinsert`, key: 1, metadata: false }, + { type: `begin` }, + { type: `seed`, key: 1, bare: true }, + { type: `upsert`, key: 1, metadata: false }, + { type: `commit` }, + ], + persistedNestedSeedLane, + ) + }) + + // An inner transaction reads the open writes of the transactions it began + // inside: one session reads all its open writes. + it(`reads an enclosing transaction's write inside a nested transaction`, async () => { + await assertPersistedHistory( + [{ type: `begin` }, { type: `rowSet`, key: 1 }, { type: `begin` }], + persistedNestedLane, + ) + }) + + // An inner transaction's equal re-insert keeps the row's metadata. When it + // commits ahead of the outer transaction whose row it re-inserted, it lands + // where that row is absent; it is still the re-insert it was written as. + it(`keeps an inner re-insert's metadata when it commits ahead of the row it re-inserted`, async () => { + await assertPersistedHistory( + [ + { type: `begin` }, + { type: `rowSet`, key: 1 }, + { type: `commit` }, + { type: `begin` }, + { type: `upsert`, key: 1, metadata: false }, + { type: `begin` }, + { type: `reinsert`, key: 1, metadata: false }, + { type: `commit` }, + ], + persistedConflictLane, + ) + }) + + // An insert without metadata clears the row's metadata, even inside the + // transaction that set it (2026-10-05 rules). + it(`reads an insert's implicit metadata clear in the open transaction`, async () => { + await assertPersistedHistory([ + { type: `begin` }, + { type: `rowSet`, key: 2 }, + { type: `upsert`, key: 2, metadata: false }, + ]) + }) + + it(`reaches held and open-over-held reads in the fixed campaign`, () => { + const counts = new Map() + for (const history of fc.sample( + sessionHistoryArbitrary(persistedFlatLane), + { + numRuns: oracleRuns(50), + seed: fixedSeed, + }, + )) + for (const regime of historyRegimes(persistedFlatLane, history)) + counts.set(regime, (counts.get(regime) ?? 0) + 1) + for (const regime of [ + `read with two held transactions`, + `open transaction over a held transaction`, + `store hold`, + `core hold`, + ]) + expect(counts.get(regime) ?? 0, regime).toBeGreaterThan(5) + }) + + if (requestedReplayProperty === undefined) { + fcTest.prop([sessionHistoryArbitrary(persistedFlatLane)], { + numRuns: oracleRuns(50), + seed: fixedSeed, + })( + `keeps the session law on flat histories (fixed)`, + assertPersistedHistory, + 120_000, + ) + fcTest.prop( + [sessionHistoryArbitrary(persistedFlatLane)], + oraclePropertyOptions(50, property), + )( + `keeps the session law on flat histories (random)`, + assertPersistedHistory, + 120_000, + ) + } else if (requestedReplayProperty === property) { + fcTest.prop( + [sessionHistoryArbitrary(persistedFlatLane)], + oraclePropertyOptions(50, property), + )( + `replays persisted flat session histories`, + assertPersistedHistory, + 120_000, + ) + } +}) + +/** + * Each lane's fixed campaign and the states it must reach (ORC-015). The + * conflict lane runs more histories: an insert over a different row needs an + * outer insert, an inner write to the same row, and the inner commit. + */ +const extraLanes: Array<{ + lane: SessionLane + property: string + seed: number + runs: number + reach: Array +}> = [ + { + lane: persistedNestedLane, + property: `sync-metadata-session.persisted-nested`, + seed: 2_220_012, + runs: 50, + reach: [`nested commit`, `nested commit during a hold`, `store hold`], + }, + { + lane: persistedSeedLane, + property: `sync-metadata-session.persisted-seeds`, + seed: 2_220_013, + runs: 50, + reach: [`seed`, `store hold`], + }, + { + lane: persistedCachedLane, + property: `sync-metadata-session.persisted-cached`, + seed: 2_220_015, + runs: 80, + reach: [ + `source insert over a seeded row`, + `source update over a seeded row`, + `store hold`, + ], + }, + { + lane: persistedConflictLane, + property: `sync-metadata-session.persisted-conflicts`, + seed: 2_220_014, + runs: 150, + reach: [ + `inner write to an enclosing row`, + `insert replaces a different row`, + `duplicate source insert`, + `store hold`, + ], + }, + // Writes while a storage read is in flight follow the same laws: the + // wrapper buffers their transactions until the read ends. + { + lane: persistedReadLane, + property: `sync-metadata-session.persisted-reads`, + seed: 2_220_017, + runs: 150, + reach: [ + `write during a storage read`, + `storage read of a stored row`, + `nested commit`, + `store hold`, + ], + }, + // A transaction committed during a read and abandoned before the wrapper + // hands it to core never wrote its rows, so a seed beneath it still holds + // them (decision 21). Abandoning a seeded row's writer is rare here; a + // replay pins it. + { + lane: persistedReadSeedLane, + property: `sync-metadata-session.persisted-reads-seeds`, + seed: 2_220_021, + runs: 150, + reach: [`write during a storage read`, `abort after commit`, `seed`], + }, + // Another tab's source writes reach this tab through its notices: a named + // write is a source write that lands beneath open transactions and over + // seeds (decision 11); a full reload keeps a held seed and makes every + // changed stored row a source write (decisions 13 and 15). + { + lane: persistedPeerLane, + property: `sync-metadata-session.persisted-peers`, + seed: 2_220_018, + runs: 150, + reach: [ + `peer row notice`, + `peer metadata notice`, + `peer delete notice`, + `peer reload notice`, + `named peer write over a held seed`, + `full reload over a held seed`, + `peer write during an open insert`, + `peer write during a hold`, + ], + }, + // The peer lane under full row updates: an update replaces its row, so + // storage must not keep a seed's fields beneath it (decision 17). + { + lane: persistedPeerFullLane, + property: `sync-metadata-session.persisted-peers-full`, + seed: 2_220_019, + runs: 150, + reach: [ + `peer row notice`, + `peer metadata notice`, + `peer delete notice`, + `peer reload notice`, + `named peer write over a held seed`, + `full reload over a held seed`, + `peer write during an open insert`, + `peer write during a hold`, + `accepted update drops a field`, + ], + }, + // The peer lane on demand: a reset is a source truncate that keeps the + // seeds this tab holds (decisions 19 and 20). Metadata of a missing row is + // rare here; a replay pins a reset over it. + { + lane: persistedPeerOnDemandLane, + property: `sync-metadata-session.persisted-peers-on-demand`, + seed: 2_220_020, + runs: 150, + reach: [ + `peer row notice`, + `peer reload notice`, + `peer reset notice`, + `named peer write over a held seed`, + `full reload over a held seed`, + `reset over a held seed`, + `reset that rewrites its rows`, + `peer write during a hold`, + ], + }, + // A seed's metadata still governs a row whose metadata an accepted + // re-insert kept, here also when the re-insert committed ahead of the + // enclosing transaction whose row it re-inserted. + { + lane: persistedNestedSeedLane, + property: `sync-metadata-session.persisted-nested-seeds`, + seed: 2_220_016, + runs: 300, + reach: [ + `seed under a kept re-insert`, + `inner write to an enclosing row`, + `insert replaces a different row`, + `duplicate source insert`, + `store hold`, + ], + }, +] + +/** + * Shrunk peer-lane counterexamples, replayed through the same model, driver, + * and comparison as the campaigns. Each reached a defect that the fixed and + * random campaigns rarely generate: a held seed that a reload must keep, a + * row that held transactions wrote, metadata of a missing row, a write over a + * seed that must store the seed's row, a seed's metadata that a later seed + * replaced, a row a held delete already removed, and a full reload's staged + * writes beneath an open re-insert, and a notice whose row keeps a held + * row's value while dropping its other fields: the campaigns almost never + * hold an absent row's insert when such a notice arrives. Recorded in + * docs/contributing/oracle-reviews/2026-10-22-sync-metadata-session.md. + */ +const peerReplays: Array<[string, Array]> = [ + [ + `a notice whose row keeps a held row's value still replaces it`, + [ + { type: `hold` }, + { type: `begin` }, + { type: `upsert`, key: 2, metadata: true }, + { type: `commit` }, + { type: `peer`, key: 2, notice: `row`, keepsValue: true }, + ], + ], + [ + `a reload keeps a seed accepted during a hold`, + [ + { type: `hold` }, + { type: `seed`, key: 2 }, + { type: `peer`, key: 1, notice: `row` }, + ], + ], + [ + `a notice for a row held transactions wrote reloads it`, + [ + { type: `hold` }, + { type: `begin` }, + { type: `upsert`, key: 2, metadata: true }, + { type: `commit` }, + { type: `peer`, key: 2, notice: `row` }, + ], + ], + [ + `a reload keeps metadata of a missing row`, + [ + { type: `begin` }, + { type: `rowSet`, key: 2 }, + { type: `commit` }, + { type: `peer`, key: 1, notice: `row` }, + ], + ], + [ + `a write over a seed stores the seed's row`, + [ + { type: `seed`, key: 1, narrow: true, bare: true }, + { type: `begin` }, + { type: `seed`, key: 2 }, + { type: `upsert`, key: 1, metadata: true }, + { type: `commit` }, + { type: `peer`, key: 2, notice: `row` }, + ], + ], + [ + `a reload after a named notice keeps a later seed's metadata`, + [ + { type: `seed`, key: 2, narrow: true }, + { type: `begin` }, + { type: `upsert`, key: 2, metadata: false }, + { type: `commit` }, + { type: `seed`, key: 2 }, + { type: `peer`, key: 1, notice: `row` }, + ], + ], + [ + `a reload does not delete a row a held delete removed`, + [ + { type: `peer`, key: 2, notice: `row` }, + { type: `hold` }, + { type: `begin` }, + { type: `peer`, key: 2, notice: `delete` }, + { type: `rowSet`, key: 2 }, + { type: `commit` }, + { type: `peer`, key: 1, notice: `reload` }, + ], + ], + [ + `a reload after a reset deletes a held row that storage lacks`, + [ + { type: `peer`, key: 1, notice: `reset` }, + { type: `begin` }, + { type: `upsert`, key: 2, metadata: true }, + { type: `hold`, store: false }, + { type: `commit` }, + { type: `peer`, key: 2, notice: `delete` }, + ], + ], + [ + `a named notice after a full reload keeps another row's seed metadata`, + [ + { type: `peer`, key: 1, notice: `reload` }, + { type: `seed`, key: 2 }, + { type: `begin` }, + { type: `upsert`, key: 2, metadata: false }, + { type: `commit` }, + { type: `peer`, key: 1, notice: `row`, keepsValue: true }, + ], + ], + [ + `an equal re-insert of a row a reset wrote back keeps its metadata`, + [ + { type: `peer`, key: 1, notice: `reset`, rewrites: true }, + { type: `begin` }, + { type: `reinsert`, key: 1, metadata: false }, + ], + ], + [ + `a held reload's writes stay beneath an open re-insert`, + [ + { type: `seed`, key: 2, bare: true }, + { type: `begin` }, + { type: `reinsert`, key: 1, metadata: false }, + { type: `hold` }, + { type: `peer`, key: 2, notice: `reload` }, + ], + ], +] + +// Under full row updates an update replaces its row, so storage keeps no +// field the update dropped: neither a seed's (decision 17 stores the row this +// tab shows) nor the stored row's own. +const peerFullReplays: Array<[string, Array]> = [ + [ + `a full update over a seed stores only the update`, + [ + { type: `seed`, key: 1 }, + { type: `begin` }, + { type: `upsert`, key: 1, metadata: false }, + { type: `commit` }, + { type: `peer`, key: 2, notice: `reload` }, + ], + ], + [ + `a full update drops a field the stored row had`, + [ + { type: `begin` }, + { type: `upsert`, key: 1, metadata: true }, + { type: `commit` }, + { type: `begin` }, + { type: `upsert`, key: 1, metadata: false }, + { type: `commit` }, + { type: `peer`, key: 2, notice: `reload` }, + ], + ], +] + +// On demand, a reset keeps a held seed and truncates every other source +// object, held rows and metadata of a missing row included (decisions 19 +// and 20). A notice after the reset reloads storage (the reset fence). +const peerOnDemandReplays: Array<[string, Array]> = [ + [ + `a reset whose storage keeps a row keeps the row's stored metadata`, + [ + { type: `begin` }, + { type: `upsert`, key: 2, metadata: true }, + { type: `commit` }, + { type: `peer`, key: 1, notice: `reset`, rewrites: true }, + ], + ], + [ + `a reset beside a held seed clears metadata of a missing row`, + [ + { type: `seed`, key: 1 }, + { type: `begin` }, + { type: `rowSet`, key: 2 }, + { type: `commit` }, + { type: `peer`, key: 1, notice: `reset` }, + ], + ], + [ + `a reset keeps a held seed's source-written metadata from a later seed`, + [ + { type: `peer`, key: 1, notice: `metadata` }, + { type: `seed`, key: 1, bare: true }, + { type: `peer`, key: 1, notice: `reset` }, + { type: `seed`, key: 1, bare: true }, + ], + ], + [ + `a source row write keeps a kept seed's metadata open to a later seed`, + [ + { type: `begin` }, + { type: `seed`, key: 1, bare: true }, + { type: `peer`, key: 1, notice: `reset` }, + { type: `upsert`, key: 1, metadata: false }, + { type: `commit` }, + { type: `hold`, store: false }, + { type: `begin` }, + { type: `seed`, key: 1 }, + ], + ], + [ + `a reset beside a held seed deletes a held row`, + [ + { type: `seed`, key: 1 }, + { type: `hold`, store: false }, + { type: `begin` }, + { type: `upsert`, key: 2, metadata: true }, + { type: `commit` }, + { type: `peer`, key: 1, notice: `reset` }, + ], + ], +] + +// A transaction the wrapper buffers during a read and that is abandoned +// before it is handed to core wrote nothing, so the seed beneath it holds. A +// seed that arrives after it was judged then and stays ignored (decision 22). +const readSeedReplays: Array<[string, Array]> = [ + [ + `a seed that arrives after a buffered write stays ignored when it is abandoned (metadata)`, + [ + { type: `begin` }, + { type: `hold`, read: true }, + { type: `rowSet`, key: 2 }, + { type: `commit` }, + { type: `seed`, key: 2 }, + { type: `abandon` }, + { type: `release` }, + ], + ], + [ + `a seed that arrives after a buffered write stays ignored when it is abandoned (row)`, + [ + { type: `hold`, read: true }, + { type: `begin` }, + { type: `upsert`, key: 2, metadata: false }, + { type: `commit` }, + { type: `seed`, key: 2 }, + { type: `abandon` }, + { type: `release` }, + ], + ], + [ + `an abandoned commit leaves a seed held against a later read`, + [ + { type: `seed`, key: 1 }, + { type: `hold`, read: true }, + { type: `begin` }, + { type: `upsert`, key: 1, metadata: false }, + { type: `commit` }, + { type: `abandon` }, + { type: `release` }, + { type: `hold`, read: true }, + { type: `release` }, + ], + ], +] + +describe(`sync metadata session oracle through the persisted wrapper: peer replays`, () => { + it.each(peerReplays)(`replays: %s`, (_name, history) => + assertPersistedHistory(history, persistedPeerLane), + ) + it.each(peerFullReplays)(`replays under full rows: %s`, (_name, history) => + assertPersistedHistory(history, persistedPeerFullLane), + ) + it.each(peerOnDemandReplays)(`replays on demand: %s`, (_name, history) => + assertPersistedHistory(history, persistedPeerOnDemandLane), + ) + it.each(readSeedReplays)( + `replays with reads and seeds: %s`, + (_name, history) => assertPersistedHistory(history, persistedReadSeedLane), + ) +}) + +describe(`sync metadata session oracle through the persisted wrapper: nesting and seeds`, () => { + for (const { + lane, + property: lanePropertyName, + seed, + runs, + reach: states, + } of extraLanes) { + const runLane = (history: Array) => + assertPersistedHistory(history, lane) + if (requestedReplayProperty === undefined) { + const reach = createOracleReach(`${lanePropertyName} (fixed)`, states, { + minimum: 5, + }) + fcTest.prop([sessionHistoryArbitrary(lane)], { + numRuns: oracleRuns(runs), + seed, + })( + `keeps the session law on ${lane.name} histories (fixed)`, + (history) => { + reach.beginHistory() + for (const regime of historyRegimes(lane, history)) + if (states.includes(regime)) reach.reach(regime) + return runLane(history) + }, + 240_000, + ) + it(`reaches the declared states in the ${lane.name} fixed campaign`, () => + reach.assertReached()) + fcTest.prop( + [sessionHistoryArbitrary(lane)], + oraclePropertyOptions(50, lanePropertyName), + )( + `keeps the session law on ${lane.name} histories (random)`, + runLane, + 120_000, + ) + } else if (requestedReplayProperty === lanePropertyName) { + fcTest.prop( + [sessionHistoryArbitrary(lane)], + oraclePropertyOptions(50, lanePropertyName), + )(`replays ${lane.name} session histories`, runLane, 120_000) + } + } +}) diff --git a/packages/db/mangle-cache.json b/packages/db/mangle-cache.json index d332dc950a..abd9bfcb0c 100644 --- a/packages/db/mangle-cache.json +++ b/packages/db/mangle-cache.json @@ -20,7 +20,6 @@ "initialLoadComplete": "ao", "needsOrderingRepair": "ap", "pendingLoadSubsetParticipants": "aq", - "appliedAdapterDeletedKeys": "ar", "finishOrderedPrefixRepair": "au", "pendingLoadSubsetPromises": "aw", "valueMap": "ax", @@ -98,7 +97,6 @@ "_clone": "bW", "requestAndObserve": "bY", "syncCallbackEpoch": "bZ", - "hasAppliedAdapterTruncate": "b_", "inFlightLeaseHolders": "b$", "releaseInFlightLease": "b0", "sentToD2RowsBySource": "b3", @@ -437,5 +435,7 @@ "recordOutcome": "ha", "endedProblem": "hb", "views": "hc", - "show": "hd" + "show": "hd", + "getWritableSyncTransaction": "ar", + "mergesOverRow": "b_" } diff --git a/packages/db/src/client.ts b/packages/db/src/client.ts index f0a8dec2de..1f0122053f 100644 --- a/packages/db/src/client.ts +++ b/packages/db/src/client.ts @@ -18,6 +18,7 @@ import { resumeDeferredLiveQueryCollections, } from './live-query-options.js' import { codedMessage, devBuild } from './error-message.js' +import { truncatesSourceKey } from './collection/state.js' import type { LiveQueryObserverClientHost } from './live-query-observer-client.js' import type { StandardSchemaV1 } from '@standard-schema/spec' import type { Collection } from './collection/index.js' @@ -36,7 +37,11 @@ import type { DeferredLiveQueryCollections, LiveQueryOptions, } from './live-query-options.js' -import type { PendingMetadataWrite } from './collection/state.js' +import type { + CollectionStateManager, + PendingMetadataWrite, + StagedSyncTransaction, +} from './collection/state.js' const collectionConfigFactory: unique symbol = Symbol.for( `@tanstack/db.collectionConfig.factory`, @@ -345,6 +350,61 @@ export function isCollectionOptions( return hasCollectionOptionsBrand(value) } +/** + * A late hydration seed cannot supersede committed adapter work. Row data + * and row metadata are separate objects, so each has its own lookup. A + * wrapping sync's committed transactions that it has not handed to this + * Collection yet count as committed adapter work. + */ +function createAdapterAuthorityLookups( + state: CollectionStateManager, +): { + rows: (key: TKey) => boolean + metadata: (key: TKey) => boolean +} { + const rowKeys = new Set() + const metadataKeys = new Set() + // The latest source truncate, with the held seeds a schema reset kept. + let truncate: StagedSyncTransaction | undefined = + state.hasAppliedAdapterTruncate + ? { + truncate: true, + operations: [], + rowMetadataWrites: new Map(), + preserveHydrationSeedKeys: state.adapterTruncateKeptKeys, + sourceTruncate: true, + } + : undefined + for (const transaction of [ + ...state.pendingSyncedTransactions.filter((pending) => pending.committed), + ...state.getStagedSyncTransactions(), + ]) { + if ( + transaction.truncate && + (!transaction.preserveHydrationSeedKeys || transaction.sourceTruncate) + ) + truncate = transaction + for (const operation of transaction.operations) + if (!transaction.preserveHydrationSeedKeys?.has(operation.key as TKey)) + rowKeys.add(operation.key as TKey) + for (const key of transaction.rowMetadataWrites.keys()) + if (!transaction.preserveHydrationSeedKeys?.has(key)) + metadataKeys.add(key) + } + return { + rows: (key) => + (truncate !== undefined && truncatesSourceKey(truncate, key)) || + rowKeys.has(key) || + state.appliedAdapterKeys?.get(key) === true || + (state.syncedData.has(key) && !state.hydrationSeedKeys.has(key)), + // A key whose row an applied adapter write deleted is in the map too. + metadata: (key) => + (truncate !== undefined && truncatesSourceKey(truncate, key)) || + metadataKeys.has(key) || + state.appliedAdapterKeys?.has(key) === true, + } +} + export class DbClient { private collectionsById = new Map() private pendingHydration = new Map>() @@ -1122,23 +1182,23 @@ export class DbClient { }, seedKind?: `initialData` | `hydration`, ): void { - const hasAuthoritativeAdapterWork = - collection._state.createAdapterAuthorityLookup() - const hasProjectedSyncedKey = collection._state.createSyncedKeyLookup() - const rows = chunk.rows.flatMap((row) => { + // A hydration seed supplies only the row data and row metadata that no + // committed adapter write has written; each object is judged alone. + const hydrating = seedKind === `hydration` + const authority = createAdapterAuthorityLookups(collection._state) + const seeds = chunk.rows.map((row) => { const value = collection.validateData(row.value, `insert`) - const key = collection.config.getKey(value) - const isAdapterAuthoritative = - seedKind === `hydration` && hasAuthoritativeAdapterWork(key) - - return isAdapterAuthoritative ? [] : [{ ...row, key, value }] + return { ...row, key: collection.config.getKey(value), value } }) + const rows = seeds.filter((row) => !hydrating || !authority.rows(row.key)) const rowMetadataWrites = new Map() - for (const row of rows) { - if (row.metadata !== undefined) { + for (const row of seeds) { + if (hydrating && authority.metadata(row.key)) continue + // The latest seed wins whole, so one without metadata clears it. + if (row.metadata !== undefined) rowMetadataWrites.set(row.key, { type: `set`, value: row.metadata }) - } + else if (hydrating) rowMetadataWrites.set(row.key, { type: `delete` }) } if (seedKind) { @@ -1150,29 +1210,32 @@ export class DbClient { } } - if (rows.length > 0) { - collection._state.acceptSeedTransaction({ + if (rows.length > 0 || rowMetadataWrites.size > 0) { + const applied = createDeferred() + // Nothing awaits a seed's receipt; cleanup may still reject it. + void applied.promise.catch(() => undefined) + collection._state.acceptSyncTransaction({ committed: true, applicationStarted: false, layoutChanged: false, + // A seed replaces any earlier seed of its row whole. operations: rows.map((row) => ({ - type: hasProjectedSyncedKey(row.key) - ? (`update` as const) - : (`insert` as const), + type: `insert` as const, key: row.key, value: row.value, })), rowMetadataWrites, - // Hydrated metadata follows every row it describes. - explicitRowMetadataWrites: new Map( - [...rowMetadataWrites].map(([key, write]) => [ - key, - { position: rows.length, write }, - ]), - ), + // An accepted transaction's metadata writes are never rebuilt. + explicitRowMetadataWrites: new Map(), collectionMetadataWrites: new Map(), - applied: createDeferred(), - preserveHydrationSeedKeys: seedKind !== undefined, + applied, + // A seed chunk keeps its keys' seed status. + preserveHydrationSeedKeys: seedKind + ? new Set([ + ...rows.map((row) => row.key), + ...rowMetadataWrites.keys(), + ]) + : undefined, }) } diff --git a/packages/db/src/collection/index.ts b/packages/db/src/collection/index.ts index 2f9bf64ad7..65b9879cbd 100644 --- a/packages/db/src/collection/index.ts +++ b/packages/db/src/collection/index.ts @@ -740,15 +740,6 @@ export class CollectionImpl< return this._state.syncedData } - /** @internal Applied metadata keys without a source row. */ - public _syncedRowMetadataKeysOutsideBase(): Array { - const keys: Array = [] - for (const key of this._state.syncedMetadata.keys()) { - if (!this._state.syncedData.has(key)) keys.push(key) - } - return keys - } - /** * Get the current value for a key (virtual derived state) */ diff --git a/packages/db/src/collection/state.ts b/packages/db/src/collection/state.ts index dc3fecc5cb..4de9b42c2e 100644 --- a/packages/db/src/collection/state.ts +++ b/packages/db/src/collection/state.ts @@ -45,6 +45,8 @@ type PendingSyncedKeyState = { type PendingSyncedProjection = { states: Map> truncated: boolean + /** Keys a queued source operation writes; seeds do not count. */ + sourceKeys: Set } type PendingInsertDisposition = `insert` | `update` | `duplicate` @@ -70,7 +72,17 @@ interface PendingSyncedTransaction< collectionMetadataWrites: Map /** Resolves after application and rejects if canceled before application. */ applied: Deferred - preserveHydrationSeedKeys?: boolean + /** + * Keys whose writes keep their hydration-seed status and fix no adapter + * authority: a DbClient seed chunk's keys, or a wrapping sync's storage + * read's seed rows. + */ + preserveHydrationSeedKeys?: ReadonlySet + /** + * Set on a truncate that keeps `preserveHydrationSeedKeys`: a schema reset + * on demand truncates the source's state of every other key (decision 20). + */ + sourceTruncate?: boolean /** Builds the error for an open transaction whose insert a replay invalidates. */ duplicateKeyError?: (key: TKey) => DuplicateKeySyncError /** @@ -80,6 +92,27 @@ interface PendingSyncedTransaction< invalidationError?: Error } +/** The parts of a wrapping sync's staged transaction that seed authority reads. */ +export type StagedSyncTransaction = { + truncate?: boolean + operations: ReadonlyArray<{ key: unknown }> + rowMetadataWrites: ReadonlyMap + preserveHydrationSeedKeys?: ReadonlySet + sourceTruncate?: boolean +} + +/** Whether a sync transaction truncates the source's state of this key. */ +export function truncatesSourceKey( + transaction: StagedSyncTransaction, + key: TKey, +): boolean { + const kept = transaction.preserveHydrationSeedKeys + return ( + transaction.truncate === true && + (!kept || (transaction.sourceTruncate === true && !kept.has(key))) + ) +} + export type PendingMetadataWrite = { type: `set`; value: unknown } | { type: `delete` } @@ -123,17 +156,23 @@ export class CollectionStateManager< private pendingSyncedProjection: PendingSyncedProjection = { states: new Map(), truncated: false, + sourceKeys: new Set(), } public syncedData: SortedMap public syncedMetadata = new Map() public syncedCollectionMetadata = new Map() public hydrationSeedKeys = new Set() public hydratedKeys = new Set() - // DbClient may receive a stale hydration chunk after an adapter delete. - // Only DbClient collections retain these absences until cleanup because + // DbClient may receive a stale hydration chunk after adapter work. Each key + // whose row metadata an applied adapter write fixed maps to whether that + // write deleted the row: row metadata is a separate object, closed to later + // seeds even when the adapter never wrote the row, and a delete also closes + // the row. Only DbClient collections retain these until cleanup because // hydration has no completion boundary. - private appliedAdapterDeletedKeys?: Set - private hasAppliedAdapterTruncate = false + public appliedAdapterKeys?: Map + public hasAppliedAdapterTruncate = false + /** Held seeds that the applied adapter truncate kept (decision 20). */ + public adapterTruncateKeptKeys = new Set() // Optimistic state tracking - make public for testing public optimisticUpserts = new Map() @@ -147,6 +186,9 @@ export class CollectionStateManager< // transaction for the key that it has not handed to this Collection yet. // Such a transaction holds settled optimistic rows like a queued one. private isDeferredSyncKey: (key: TKey) => boolean = () => false + // Those committed transactions themselves, for hydration seed authority. + public getStagedSyncTransactions: () => Array> = + () => [] public heldOptimisticRows = new Map< TKey, { owner: Transaction; row?: TOutput } @@ -525,8 +567,12 @@ export class CollectionStateManager< * Lets a wrapping sync report which keys have committed sync transactions * it has not handed to this Collection yet. */ - public setDeferredSyncKeys(isDeferred: (key: TKey) => boolean): void { + public setDeferredSyncKeys( + isDeferred: (key: TKey) => boolean, + getStaged: () => Array> = () => [], + ): void { this.isDeferredSyncKey = isDeferred + this.getStagedSyncTransactions = getStaged } /** @@ -780,27 +826,7 @@ export class CollectionStateManager< } enableHydrationAuthorityTracking(): void { - this.appliedAdapterDeletedKeys ??= new Set() - } - - /** A late hydration seed cannot supersede committed adapter work. */ - createAdapterAuthorityLookup(): (key: TKey) => boolean { - const queuedKeys = new Set() - let queuedTruncate = false - for (const transaction of this.pendingSyncedTransactions) { - if (!transaction.committed || transaction.preserveHydrationSeedKeys) - continue - if (transaction.truncate) queuedTruncate = true - for (const operation of transaction.operations) { - queuedKeys.add(operation.key as TKey) - } - } - return (key) => - this.hasAppliedAdapterTruncate || - queuedTruncate || - queuedKeys.has(key) || - this.appliedAdapterDeletedKeys?.has(key) === true || - (this.syncedData.has(key) && !this.hydrationSeedKeys.has(key)) + this.appliedAdapterKeys ??= new Map() } private getProjectedSyncedKeyState( @@ -827,19 +853,23 @@ export class CollectionStateManager< } /** - * Accept a committed seed transaction, such as a DbClient hydration chunk, - * ahead of any still-open source transaction, so that source's `commit()` - * still targets its own writes. + * Accept a committed transaction in commit order: after every accepted + * transaction and ahead of every open one. A transaction begun inside an + * open one is independent, so when it commits first it also applies, and + * the source reads it, beneath that open transaction. A seed, such as a + * DbClient hydration chunk, is accepted the same way, so an open source + * transaction's `commit()` still targets its own writes. */ - acceptSeedTransaction( + acceptSyncTransaction( transaction: PendingSyncedTransaction, ): void { - const open = this.pendingSyncedTransactions.findIndex( - (pending) => !pending.committed, - ) - if (open === -1) this.pendingSyncedTransactions.push(transaction) - else this.pendingSyncedTransactions.splice(open, 0, transaction) - this.refreshPendingSyncedProjection() + const queue = this.pendingSyncedTransactions + const index = queue.indexOf(transaction) + if (index !== -1) queue.splice(index, 1) + const open = queue.findIndex((pending) => !pending.committed) + const position = open === -1 ? queue.length : open + queue.splice(position, 0, transaction) + if (position !== index) this.refreshPendingSyncedProjection() this.commitPendingTransactions() } @@ -889,15 +919,29 @@ export class CollectionStateManager< ): PendingInsertDisposition { const current = this.getProjectedSyncedKeyState(projection, key) if (!current.exists) return `insert` - if ( - (current.value !== undefined && deepEquals(current.value, value)) || - this.hydrationSeedKeys.has(key) - ) { + // A seed is older than every source write, so an insert replaces a row + // that only a seed supplied. Over a row the source wrote, it is judged + // like any insert, holds and seeds notwithstanding. + if (this.hydrationSeedKeys.has(key) && !projection.sourceKeys.has(key)) + return `insert` + if (current.value !== undefined && deepEquals(current.value, value)) { return `update` } return `duplicate` } + /** + * A partial update merges over the row before it. An insert names the + * whole row, even as an equal re-insert, and replaces it. + */ + private mergesOverRow(operation: PendingSyncOperation) { + return ( + operation.type === `update` && + operation.originalSyncType !== `insert` && + (this.config.sync.rowUpdateMode || `partial`) === `partial` + ) + } + /** Classify an adapter insert against retained and queued source state. */ classifyPendingSyncedInsert( key: TKey, @@ -915,10 +959,9 @@ export class CollectionStateManager< operation: PendingSyncOperation, ): void { const key = operation.key as TKey - const rowUpdateMode = this.config.sync.rowUpdateMode || `partial` if (operation.type === `delete`) { projection.states.set(key, { exists: false, value: undefined }) - } else if (operation.type === `update` && rowUpdateMode === `partial`) { + } else if (this.mergesOverRow(operation)) { const current = this.getProjectedSyncedKeyState(projection, key) projection.states.set(key, { exists: true, @@ -930,11 +973,17 @@ export class CollectionStateManager< } } - /** Extend the queued projection after admitting one sync operation. */ + /** + * Extend the queued projection after admitting one sync operation. A key + * its transaction keeps as a seed's is no source write, as in the rebuild. + */ stagePendingSyncOperation( operation: PendingSyncOperation, + preserved?: boolean, ): void { this.applyPendingSyncOperation(this.pendingSyncedProjection, operation) + if (!preserved) + this.pendingSyncedProjection.sourceKeys.add(operation.key as TKey) } private rebuildAutomaticRowMetadataWrites( @@ -957,22 +1006,32 @@ export class CollectionStateManager< /** * Rebuild the queued projection after truncate, application, or an - * accepted seed changes queue history. An open transaction can become + * accepted transaction changes queue history. An accepted transaction keeps + * the operation kinds it was accepted with, even when it commits ahead of + * an open transaction it began inside: its insert names the whole row. An + * open transaction's inserts follow the rows before it, so it can become * invalid when a later transaction begun inside it commits first; it then - * holds an invalidation error that its commit returns. Only the open last - * transaction can be canceled, so a replay that invalidates a committed - * transaction is an invariant failure. + * holds an invalidation error that its commit returns. */ private rebuildPendingSyncedProjection(): void { const projection: PendingSyncedProjection = { states: new Map(), truncated: false, + sourceKeys: new Set(), } + // The source keys that an accepted transaction writes. + const acceptedSourceKeys = new Set() for (const transaction of this.pendingSyncedTransactions) { if (transaction.invalidationError !== undefined) continue - const statesBeforeTransaction = new Map(projection.states) - const truncatedBeforeTransaction = projection.truncated + // Only an open transaction can become invalid and need this rollback. + const before = transaction.committed + ? undefined + : { + states: new Map(projection.states), + truncated: projection.truncated, + sourceKeys: new Set(projection.sourceKeys), + } if (transaction.truncate) { projection.states = new Map() projection.truncated = true @@ -981,9 +1040,19 @@ export class CollectionStateManager< let invalidKey: TKey | undefined for (const operation of transaction.operations) { const key = operation.key as TKey + // A seed beneath an open insert of a row the source also wrote can + // change that row, but not the insert's kind: it stays what the + // source wrote. An accepted source write makes the row the source's, + // as its application does. if ( + !transaction.committed && operation.originalSyncType === `insert` && - operation.type !== `delete` + operation.type !== `delete` && + !( + this.hydrationSeedKeys.has(key) && + projection.sourceKeys.has(key) && + !acceptedSourceKeys.has(key) + ) ) { const disposition = this.classifyProjectedInsert( projection, @@ -997,20 +1066,20 @@ export class CollectionStateManager< operation.type = disposition } this.applyPendingSyncOperation(projection, operation) + if (!transaction.preserveHydrationSeedKeys?.has(key)) { + projection.sourceKeys.add(key) + if (transaction.committed) acceptedSourceKeys.add(key) + } } if (invalidKey !== undefined) { - if (transaction.committed) - throw new SyncQueueInvariantError( - `replay made an accepted insert a duplicate`, - ) - projection.states = statesBeforeTransaction - projection.truncated = truncatedBeforeTransaction + Object.assign(projection, before) transaction.invalidationError = transaction.duplicateKeyError?.(invalidKey) ?? new SyncTransactionAbortedError() continue } - this.rebuildAutomaticRowMetadataWrites(transaction) + if (!transaction.committed) + this.rebuildAutomaticRowMetadataWrites(transaction) } this.pendingSyncedProjection = projection @@ -1162,7 +1231,6 @@ export class CollectionStateManager< }) } } - const rowUpdateMode = this.config.sync.rowUpdateMode || `partial` for (const transaction of committedSyncedTransactions) { // Handle truncate operations first if (transaction.truncate) { @@ -1172,15 +1240,28 @@ export class CollectionStateManager< // Keep active local mutations for later source transactions on keys // this truncate does not write. Completed attribution is one-use. this.syncedData.clear() - this.syncedMetadata.clear() - this.hydrationSeedKeys.clear() - this.hydratedKeys.clear() - if ( - !transaction.preserveHydrationSeedKeys && - this.appliedAdapterDeletedKeys - ) { - this.hasAppliedAdapterTruncate = true - this.appliedAdapterDeletedKeys.clear() + // A wrapping sync's storage read that keeps seed keys replaces + // rows without truncating the source's authority over them or + // their metadata. A schema reset on demand truncates all but its + // kept seeds (decision 20). + const kept = transaction.sourceTruncate + ? transaction.preserveHydrationSeedKeys + : undefined + if (!transaction.preserveHydrationSeedKeys || kept) { + // A kept seed keeps the authority a source write fixed over + // one of its objects too. + for (const keys of [ + this.syncedMetadata, + this.hydrationSeedKeys, + this.hydratedKeys, + this.appliedAdapterKeys ?? new Set(), + ]) + for (const key of keys.keys()) + if (!kept?.has(key)) keys.delete(key) + if (this.appliedAdapterKeys) { + this.hasAppliedAdapterTruncate = true + this.adapterTruncateKeptKeys = new Set(kept) + } } this.virtualPropsCache.clear() this.rowOrigins.clear() @@ -1228,7 +1309,7 @@ export class CollectionStateManager< } else { this.syncedData.set( key, - operation.type === `update` && rowUpdateMode === `partial` + this.mergesOverRow(operation) ? { ...this.syncedData.get(key), ...operation.value } : operation.value, deferOrder, @@ -1240,16 +1321,24 @@ export class CollectionStateManager< this.pendingLocalChanges.delete(key) this.pendingLocalOrigins.delete(key) if (!this.isDeferredSyncKey(key)) this.heldOptimisticRows.delete(key) - if (!transaction.preserveHydrationSeedKeys) { + if (!transaction.preserveHydrationSeedKeys?.has(key)) { this.hydrationSeedKeys.delete(key) this.hydratedKeys.delete(key) + // A write that brings a deleted row back keeps the metadata the + // delete cleared. if (operation.type === `delete`) - this.appliedAdapterDeletedKeys?.add(key) - else this.appliedAdapterDeletedKeys?.delete(key) + this.appliedAdapterKeys?.set(key, true) + else if (this.appliedAdapterKeys?.get(key)) + this.appliedAdapterKeys.set(key, false) } } for (const [key, metadataWrite] of transaction.rowMetadataWrites) { + if ( + !transaction.preserveHydrationSeedKeys?.has(key) && + !this.appliedAdapterKeys?.has(key) + ) + this.appliedAdapterKeys?.set(key, false) if (metadataWrite.type === `delete`) { this.syncedMetadata.delete(key) continue @@ -1567,14 +1656,15 @@ export class CollectionStateManager< this.heldOptimisticRows.clear() this.hydrationSeedKeys.clear() this.hydratedKeys.clear() - this.appliedAdapterDeletedKeys?.clear() + this.appliedAdapterKeys?.clear() this.hasAppliedAdapterTruncate = false + this.adapterTruncateKeptKeys.clear() this.clearOriginTrackingState() this.isLocalOnly = false this.localOnlyDirectWrite = undefined this.size = 0 this.pendingSyncedTransactions = [] - this.pendingSyncedProjection = { states: new Map(), truncated: false } + this.refreshPendingSyncedProjection() this.preSyncVisibleState.clear() this.preSyncVirtualState.clear() this.recentlySyncedKeys.clear() diff --git a/packages/db/src/collection/sync.ts b/packages/db/src/collection/sync.ts index 1d191b7940..46aaf04865 100644 --- a/packages/db/src/collection/sync.ts +++ b/packages/db/src/collection/sync.ts @@ -21,7 +21,7 @@ import { isPromiseLike } from '../utils/type-guards' import { reportUncaught } from '../utils/callbacks' import { LIVE_QUERY_INTERNAL } from '../query/live/internal.js' import { isFailedStatus } from '../live-query-primitives.js' -import { automaticRowMetadataWrite } from './state' +import { automaticRowMetadataWrite, truncatesSourceKey } from './state' import type { StandardSchemaV1 } from '@standard-schema/spec' import type { ChangeMessageOrDeleteKeyMessage, @@ -41,6 +41,15 @@ import type { CollectionEventsManager } from './events.js' import type { LiveQueryCollectionUtils } from '../query/live/collection-config-builder.js' import type { Deferred } from '../deferred' +/** + * Internal marker on a sync write: an update that names the whole row, as an + * insert does. The persisted wrapper hands an insert it already judged this + * way. It is not part of the public sync API. + * + * @internal + */ +export const syncWriteNamesWholeRow: unique symbol = Symbol() + type DeferredLoadSubset = { options: LoadSubsetOptions deferred: Deferred @@ -306,10 +315,8 @@ export class CollectionSyncManager< >, ) => { if (!isCurrentSync()) return - const pendingTransaction = this.getActivePendingSyncTransaction() - // A replay can invalidate an open transaction between writes. Its - // commit receipt owns that failure; later writes cannot revive it. - if (pendingTransaction.invalidationError !== undefined) return + const pendingTransaction = this.getWritableSyncTransaction() + if (!pendingTransaction) return const key = `key` in messageWithOptionalKey @@ -341,19 +348,31 @@ export class CollectionSyncManager< this.checkReusedRow(key, messageType, messageWithOptionalKey) } + // An insert names the whole row, so it replaces the row even as + // an update. A wrapping sync that judged its insert already marks + // it as such an update: its metadata then changes only through + // the wrapper's own writes. const message = { ...messageWithOptionalKey, type: messageType, key, originalSyncType: - messageWithOptionalKey.type === `insert` + messageWithOptionalKey.type === `insert` || + ( + messageWithOptionalKey as { + [syncWriteNamesWholeRow]?: true + } + )[syncWriteNamesWholeRow] ? (`insert` as const) : undefined, } as OptimisticChangeMessage & { originalSyncType?: `insert` } pendingTransaction.operations.push(message) - this.state.stagePendingSyncOperation(message) + this.state.stagePendingSyncOperation( + message, + pendingTransaction.preserveHydrationSeedKeys?.has(key), + ) const metadataWrite = automaticRowMetadataWrite(message) if (metadataWrite) { @@ -389,7 +408,7 @@ export class CollectionSyncManager< // accepts it now; the receipt resolves when it is visible. One // held by a persisting optimistic transaction becomes visible when // that transaction settles. - this.state.commitPendingTransactions() + this.state.acceptSyncTransaction(pendingTransaction) if (!pendingTransaction.applied.isPending()) return true return withAcceptedReceipt(pendingTransaction.applied.promise, true) }, @@ -409,13 +428,13 @@ export class CollectionSyncManager< }, truncate: (options) => { if (!isCurrentSync()) return - const pendingTransaction = this.getActivePendingSyncTransaction() + const pendingTransaction = this.getWritableSyncTransaction() + if (!pendingTransaction) return // Clear all operations from the current transaction pendingTransaction.operations = [] pendingTransaction.rowMetadataWrites.clear() pendingTransaction.explicitRowMetadataWrites.clear() - pendingTransaction.invalidationError = undefined // Intentionally preserve collectionMetadataWrites across truncate. // Collection-scoped metadata (for example persisted resume/reset // state) can be staged before truncate and should commit atomically @@ -570,6 +589,20 @@ export class CollectionSyncManager< return pendingTransaction } + /** + * The open transaction that takes a row write or truncate. A replay can + * invalidate an open transaction between writes. Its commit receipt owns + * that failure; later writes, a truncate included, cannot revive it, so + * they are ignored. Its metadata writes need no check: no read includes an + * invalid transaction, and it never applies. + */ + private getWritableSyncTransaction() { + const pendingTransaction = this.getActivePendingSyncTransaction() + return pendingTransaction.invalidationError === undefined + ? pendingTransaction + : undefined + } + /** Record an explicit row metadata write and its place among the operations. */ private writeExplicitRowMetadata( key: TKey, @@ -591,15 +624,18 @@ export class CollectionSyncManager< row: { get: (key) => { if (!isCurrentSync()) return undefined - const pendingTransaction = this.state.pendingSyncedTransactions.at(-1) - const pendingWrite = pendingTransaction?.rowMetadataWrites.get(key) - if (pendingWrite) { - return pendingWrite.type === `delete` - ? undefined - : pendingWrite.value - } - if (pendingTransaction?.truncate) { - return undefined + // A source reads its own session: every accepted or open sync + // transaction that no invalidation ended, newest first, then + // applied state. A truncate hides older row metadata, except a + // wrapping sync's storage read, which keeps it, and a schema reset's + // kept seeds (decision 20). The wrapper reads its own open reset. + const pending = this.state.pendingSyncedTransactions + for (let index = pending.length - 1; index >= 0; index--) { + const transaction = pending[index]! + if (transaction.invalidationError !== undefined) continue + const write = transaction.rowMetadataWrites.get(key) + if (write) return write.type === `delete` ? undefined : write.value + if (truncatesSourceKey(transaction, key)) return undefined } return this.state.syncedMetadata.get(key) }, @@ -615,45 +651,41 @@ export class CollectionSyncManager< collection: { get: (key) => { if (!isCurrentSync()) return undefined - const pendingTransaction = this.state.pendingSyncedTransactions.at(-1) - const pendingWrite = - pendingTransaction?.collectionMetadataWrites.get(key) - if (pendingWrite) { - return pendingWrite.type === `delete` - ? undefined - : pendingWrite.value + // Newest first, like row metadata. A truncate keeps collection + // metadata. + const pending = this.state.pendingSyncedTransactions + for (let index = pending.length - 1; index >= 0; index--) { + const transaction = pending[index]! + if (transaction.invalidationError !== undefined) continue + const write = transaction.collectionMetadataWrites.get(key) + if (write) return write.type === `delete` ? undefined : write.value } return this.state.syncedCollectionMetadata.get(key) }, set: (key, value) => { if (!isCurrentSync()) return - const pendingTransaction = this.getActivePendingSyncTransaction() - pendingTransaction.collectionMetadataWrites.set(key, { - type: `set`, - value, - }) + this.getActivePendingSyncTransaction().collectionMetadataWrites.set( + key, + { type: `set`, value }, + ) }, delete: (key) => { if (!isCurrentSync()) return - const pendingTransaction = this.getActivePendingSyncTransaction() - pendingTransaction.collectionMetadataWrites.set(key, { - type: `delete`, - }) + this.getActivePendingSyncTransaction().collectionMetadataWrites.set( + key, + { type: `delete` }, + ) }, list: (prefix) => { if (!isCurrentSync()) return [] + // Every pending transaction applies over applied state, oldest + // first, so the newest write for each key wins. const merged = new Map(this.state.syncedCollectionMetadata) - const pendingTransaction = this.state.pendingSyncedTransactions.at(-1) - if (pendingTransaction) { - for (const [ - key, - pendingWrite, - ] of pendingTransaction.collectionMetadataWrites) { - if (pendingWrite.type === `delete`) { - merged.delete(key) - } else { - merged.set(key, pendingWrite.value) - } + for (const transaction of this.state.pendingSyncedTransactions) { + if (transaction.invalidationError !== undefined) continue + for (const [key, write] of transaction.collectionMetadataWrites) { + if (write.type === `delete`) merged.delete(key) + else merged.set(key, write.value) } } diff --git a/packages/db/src/index.ts b/packages/db/src/index.ts index 0f4922e3e8..d4c78fbf59 100644 --- a/packages/db/src/index.ts +++ b/packages/db/src/index.ts @@ -9,6 +9,8 @@ export * from './client.js' export { withCollectionConfigFactory } from './client.js' /** @internal First-party adapter ownership marker. */ export { collectionOptionsClaim } from './collection-options.js' +/** @internal */ +export { syncWriteNamesWholeRow } from './collection/sync.js' export * from './types' export * from './proxy' export * from './query/index.js' diff --git a/packages/db/tests/collection-row-metadata-composition-oracle.test.ts b/packages/db/tests/collection-row-metadata-composition-oracle.test.ts index 33b9b85f69..762c1bd571 100644 --- a/packages/db/tests/collection-row-metadata-composition-oracle.test.ts +++ b/packages/db/tests/collection-row-metadata-composition-oracle.test.ts @@ -29,9 +29,30 @@ import type { SyncConfig } from '../src/types.js' * lanes: immediate, held, rebuilt, and nested. The nested lane applies a * nested transaction or a hydration seed before the open transaction. When a * rebuild changes an insert into an equal re-insert, or the reverse, the rule - * follows the write that applies. The model judges the reclassified sequence. It does not judge collection metadata, metadata-only - * publication batches, or other keys. The collection-metadata publication - * oracle owns those. + * follows the write that applies. The model judges the reclassified sequence. + * It does not judge metadata-only publication batches or other keys. The + * collection-metadata publication oracle owns publication batches; the sync + * metadata session oracle owns reads across transactions, seeds, and the + * consumer view. + * + * ## What does the source read before a held transaction applies? + * + * An accepted sync transaction always applies, in commit order (glossary, + * "accepted sync transaction"). A sync-side metadata read therefore returns + * the value that the source's accepted writes and its open transaction + * determine. Whether a persisting optimistic transaction still holds those + * writes does not change the read. The persisted wrapper states the same rule + * for its staged writes: the open transaction's write, then the newest staged + * one, then the stored value, with a truncate hiding older row metadata. + * `collection.base` is a different observation: it is the applied view and + * excludes queued writes. + * + * The held-window lane checks this rule for `metadata.row.get`, + * `metadata.collection.get`, and `metadata.collection.list`. The history's + * explicit `set` and `unset` writes also write one collection key. A later + * accepted transaction writes another key. A read sees both before and after + * the hold ends. The lane does not judge a read between writes of one open + * transaction, nested transactions, or wrapper-deferred transactions. */ type Row = { id: number; v: number } @@ -180,11 +201,22 @@ const rebuiltHistories = starts.flatMap((start) => { * * A nested transaction can also call `metadata.row.set` after its row write. * + * A transaction begun inside T1 is independent and commits first, so it + * applies first: sync transactions apply in commit order (maintainer + * decision of 2026-10-23, recorded in + * `docs/contributing/oracle-reviews/2026-10-22-sync-metadata-session.md`). * T1 applies after the nested write, so the Collection reclassifies each of * T1's inserts against the rows that the nested write leaves. An insert of a * present key is an equal re-insert, and an insert of an absent key is an * insert. Every row write in this lane writes one value, so a reclassified - * insert is never a duplicate. + * insert is never a duplicate. The `held` variant holds both transactions + * behind a persisting optimistic update until T1 commits; a hold delays + * application but cannot change commit order, so it expects the same value. + * + * A seed is older than every source write (decisions D4 and D5 of + * 2026-10-22). It sits beneath T1, and T1's first insert names the whole + * row, so it is an insert that replaces the seeded row, not an equal + * re-insert of it. * * A history is legal when T1's writes are legal both where T1 writes them and * where they apply. An insert is legal in either place. An update or a delete @@ -262,7 +294,11 @@ const reclassify = ( present = true return { kind, metadata: write.metadata } }) -/** The model's expected value: T1's writes fold over the nested write's. */ +/** + * The model's expected value: T1's writes fold over the nested write's. A + * nested insert leaves the row present for T1's inserts; a seed does not, + * because T1's first insert replaces it. + */ function expectedNestedMetadata(history: NestedHistory): unknown { const metadata = history.nestedSet ? nestedSetMetadata @@ -271,7 +307,7 @@ function expectedNestedMetadata(history: NestedHistory): unknown { : nestedMetadata return foldMetadata( metadata, - reclassify(history.nested !== `delete`, history.sequence), + reclassify(history.nested === `insert`, history.sequence), ) } const describeNested = ({ @@ -444,17 +480,186 @@ async function observeMetadata( } } +/** + * The held-window model folds collection metadata separately. An explicit + * `set` names the cursor's value and an `unset` deletes it. Row writes and + * truncates leave it alone, because collection metadata is not row state. + * T2 always writes the `later` key, so a correct `list()` contains it. + */ +const cursorKey = `cursor` +const laterKey = `later` +const laterMetadata = { m: `later` } +type HeldOpen = `empty` | `truncated` +type HeldReads = { + row: unknown + cursor: unknown + list: Array<{ key: string; value: unknown }> +} + +function expectedCursor(start: Start, sequence: ReadonlyArray) { + let current: unknown = + start === `present-with-metadata` ? startMetadata : undefined + sequence.forEach((write, step) => { + if (write.kind === `set`) current = writeMetadata(step) + else if (write.kind === `unset`) current = undefined + }) + return current +} + +function expectedHeldReads( + start: Start, + sequence: ReadonlyArray, + open: HeldOpen | `applied`, +): HeldReads { + const cursor = expectedCursor(start, sequence) + return { + // An open truncate hides every older row metadata value. + row: open === `truncated` ? undefined : expectedMetadata(start, sequence), + cursor, + list: [ + ...(cursor === undefined ? [] : [{ key: cursorKey, value: cursor }]), + { key: laterKey, value: laterMetadata }, + ], + } +} + +const readHeld = (sync: SyncActions): HeldReads => ({ + row: sync.metadata!.row.get(1), + cursor: sync.metadata!.collection.get(cursorKey), + // List order is not part of the contract. + list: [...sync.metadata!.collection.list()].sort((a, b) => + a.key.localeCompare(b.key), + ), +}) + +/** + * The held-window driver holds the history's transaction T1 behind a + * persisting optimistic update. It then commits T2, which updates key 2 and + * sets `later`. T1 is held unless it truncates, and T2 is always held. The + * driver reads at three checkpoints: + * + * - `committed`: after T2's commit returns, with no open transaction. + * - `open`: inside an open T3 that is empty or truncates. + * - `applied`: after T3 is canceled and the hold ends. + * + * T3 is canceled with an aborted signal, so the applied checkpoint expects + * T1 and T2 only. + */ +async function observeHeldReads( + start: Start, + sequence: ReadonlyArray, + open: HeldOpen, + id: string, +): Promise> { + let sync!: SyncActions + let release!: () => void + const collection = createCollection({ + id, + getKey: (row) => row.id, + startSync: true, + sync: { + sync: (actions) => { + sync = actions + actions.begin() + actions.write({ type: `insert`, value: { id: 2, v: 0 } }) + if (start !== `absent`) { + actions.write({ + type: `insert`, + value: { id: 1, v: 0 }, + ...(start === `present-with-metadata` + ? { metadata: startMetadata } + : {}), + }) + } + if (start === `present-with-metadata`) + actions.metadata!.collection.set(cursorKey, startMetadata) + actions.commit() + actions.markReady() + }, + }, + onUpdate: () => new Promise((resolve) => (release = resolve)), + }) + try { + await collection.stateWhenReady() + const blocker = collection.update(2, (draft) => { + draft.v = 9 + }) + let held: number | undefined = start === `absent` ? undefined : 0 + sync.begin() + sequence.forEach((write, step) => { + const metadata = writeMetadata(step) + if (write.kind === `insert` || write.kind === `update`) held = step + 1 + else if (write.kind === `delete` || write.kind === `truncate`) + held = undefined + if (write.kind === `set`) { + sync.metadata!.row.set(1, metadata) + sync.metadata!.collection.set(cursorKey, metadata) + } else if (write.kind === `unset`) { + sync.metadata!.row.delete(1) + sync.metadata!.collection.delete(cursorKey) + } else if (write.kind === `truncate`) sync.truncate() + else if (write.kind === `delete`) + sync.write({ + type: `delete`, + key: 1, + ...(write.metadata ? { metadata } : {}), + }) + else if (write.kind === `reinsert`) + sync.write({ + type: `insert`, + value: { id: 1, v: held! }, + ...(write.metadata ? { metadata } : {}), + }) + else + sync.write({ + type: write.kind, + value: { id: 1, v: step + 1 }, + ...(write.metadata ? { metadata } : {}), + }) + }) + const firstReceipt = sync.commit() + if (sequence.some((write) => write.kind === `truncate`)) + expect(firstReceipt).toBe(true) + else expect(firstReceipt).not.toBe(true) + sync.begin() + sync.write({ type: `update`, value: { id: 2, v: 100 } }) + sync.metadata!.collection.set(laterKey, laterMetadata) + const laterReceipt = sync.commit() + expect(laterReceipt).not.toBe(true) + const committed = readHeld(sync) + + sync.begin() + if (open === `truncated`) sync.truncate() + const openReads = readHeld(sync) + const abort = new AbortController() + abort.abort() + await expect(sync.commit(abort.signal)).rejects.toThrow() + + release() + await blocker.isPersisted.promise + await firstReceipt + await laterReceipt + return { committed, open: openReads, applied: readHeld(sync) } + } finally { + await collection.cleanup() + } +} + /** * The nested driver writes T1 through a real Collection's sync API, then * applies the nested write and commits both. A seed goes through * `DbClient.hydrate`, so that history uses a Collection that a `DbClient` - * owns. The driver reads `metadata.row.get` after T1 applies. + * owns. When `held`, a persisting optimistic update of key 2 holds both + * transactions until T1 has committed. The driver reads `metadata.row.get` + * after T1 applies. */ async function observeNested( { start, nested, nestedSet, sequence }: NestedHistory, id: string, + held = false, ): Promise { let sync!: SyncActions + let release!: () => void const row = { id: 1, v: 0 } const config = { id, @@ -463,6 +668,7 @@ async function observeNested( sync: (actions: SyncActions) => { sync = actions actions.begin() + actions.write({ type: `insert`, value: { id: 2, v: 0 } }) if (start !== `absent`) actions.write({ type: `insert`, @@ -475,6 +681,7 @@ async function observeNested( actions.markReady() }, }, + onUpdate: () => new Promise((resolve) => (release = resolve)), } const client = new DbClient() const collection = @@ -483,6 +690,11 @@ async function observeNested( : createCollection({ ...config, startSync: true }) try { await collection.stateWhenReady() + const blocker = held + ? collection.update(2, (draft) => { + draft.v = 9 + }) + : undefined sync.begin() sequence.forEach((write, step) => { const metadata = writeMetadata(step) @@ -520,6 +732,10 @@ async function observeNested( nestedReceipt = sync.commit() } const receipt = sync.commit() + if (blocker) { + release() + await blocker.isPersisted.promise + } await nestedReceipt await receipt return sync.metadata!.row.get(1) @@ -627,11 +843,20 @@ describe(`row metadata composition oracle`, () => { expect( expectedNestedMetadata({ start: `absent`, - nested: `seed`, + nested: `insert`, nestedSet: false, sequence: [{ kind: `set` }, { kind: `insert`, metadata: false }], }), ).toEqual(writeMetadata(0)) + // T1's insert replaces the seeded row, so it clears T1's earlier set. + expect( + expectedNestedMetadata({ + start: `absent`, + nested: `seed`, + nestedSet: false, + sequence: [{ kind: `set` }, { kind: `insert`, metadata: false }], + }), + ).toBeUndefined() // T1 applies after the nested set, so T1's later write wins. expect( expectedNestedMetadata({ @@ -643,10 +868,10 @@ describe(`row metadata composition oracle`, () => { ).toEqual(writeMetadata(0)) }) - it( - `matches last-write-wins metadata after a nested write`, + it.each([`immediate`, `held`] as const)( + `matches last-write-wins metadata after a nested write (%s)`, { timeout: 120_000 }, - async () => { + async (path) => { const mismatches: Array = [] for (const [index, history] of nestedHistories.entries()) { const label = describeNested(history) @@ -654,7 +879,8 @@ describe(`row metadata composition oracle`, () => { try { actual = await observeNested( history, - `row-metadata-composition-nested-${index}`, + `row-metadata-composition-nested-${path}-${index}`, + path === `held`, ) } catch (error) { mismatches.push(`${label} threw ${String(error)}`) @@ -671,6 +897,63 @@ describe(`row metadata composition oracle`, () => { }, ) + it(`rejects named wrong answers in the held-window model`, () => { + // Truncate keeps collection metadata, and an earlier set survives T2. + expect( + expectedHeldReads( + `present-with-metadata`, + [{ kind: `set` }, { kind: `truncate` }], + `empty`, + ), + ).toEqual({ + row: undefined, + cursor: writeMetadata(0), + list: [ + { key: cursorKey, value: writeMetadata(0) }, + { key: laterKey, value: laterMetadata }, + ], + }) + // An open truncate hides row metadata, not the cursor. + expect( + expectedHeldReads(`absent`, [{ kind: `set` }], `truncated`), + ).toMatchObject({ row: undefined, cursor: writeMetadata(0) }) + }) + + it.each([`empty`, `truncated`] as const)( + `reads accepted metadata during a hold, then in an open transaction that is %s`, + { timeout: 120_000 }, + async (open) => { + const mismatches: Array = [] + for (const [index, { start, sequence }] of histories.entries()) { + const label = `${start}: ${sequence.map(describeWrite).join(`, `)}` + let actual: Awaited> + try { + actual = await observeHeldReads( + start, + sequence, + open, + `row-metadata-composition-held-window-${open}-${index}`, + ) + } catch (error) { + mismatches.push(`${label} threw ${String(error)}`) + continue + } + for (const checkpoint of [`committed`, `open`, `applied`] as const) { + const expected = expectedHeldReads( + start, + sequence, + checkpoint === `open` ? open : `applied`, + ) + if (JSON.stringify(actual[checkpoint]) !== JSON.stringify(expected)) + mismatches.push( + `${label} at ${checkpoint} read ${JSON.stringify(actual[checkpoint])}, expected ${JSON.stringify(expected)}`, + ) + } + } + expect(mismatches).toEqual([]) + }, + ) + it.each([`immediate`, `held`, `rebuilt`] as const)( `matches last-write-wins metadata on the %s path`, { timeout: 120_000 }, diff --git a/packages/db/tests/collection-state-retention.test.ts b/packages/db/tests/collection-state-retention.test.ts index 8a6b7a993c..0bb5a15e67 100644 --- a/packages/db/tests/collection-state-retention.test.ts +++ b/packages/db/tests/collection-state-retention.test.ts @@ -11,11 +11,9 @@ import { expect, it } from 'vitest' import { createCollection } from '../src/collection/index.js' import { DuplicateKeySyncError, - SyncQueueInvariantError, SyncTransactionAlreadyCommittedWriteError, } from '../src/errors.js' import { BTreeIndex } from '../src/indexes/btree-index.js' -import { createDeferred } from '../src/deferred.js' import { createTransaction } from '../src/transactions.js' import { whenSyncAccepted } from '../src/sync-receipt.js' import { @@ -581,104 +579,62 @@ it.each([ }, ) -// No public path can make a committed queued transaction invalid on replay: -// only the open last transaction can be canceled. A replay that finds one is -// an invariant failure, not a recoverable rejection. -it(`throws an invariant error when replay finds an invalid committed transaction`, async () => { - // Current callers cannot create this queue state. Exercise the internal - // recovery boundary so a future refresh caller cannot strand its receipt. - const collection = createCollection({ - getKey: (row) => row.id, - startSync: true, - sync: { - sync: ({ begin, write, commit, markReady }) => { - begin() - write({ type: `insert`, value: { id: 1, value: 0 } }) - commit() - markReady() - }, - }, - }) - const applied = createDeferred() - void applied.promise.catch(() => undefined) - const pending = { - committed: true, - applicationStarted: false, - layoutChanged: false, - operations: [ - { - type: `insert` as const, - originalSyncType: `insert` as const, - key: 1, - value: { id: 1, value: 2 }, - }, - ], - rowMetadataWrites: new Map(), - explicitRowMetadataWrites: new Map(), - collectionMetadataWrites: new Map(), - applied, - } - const validApplied = createDeferred() - void validApplied.promise.catch(() => undefined) - const valid = { - ...pending, - operations: [ - { - ...pending.operations[0]!, - key: 2, - value: { id: 2, value: 2 }, +// A transaction begun inside an open one is independent and applies in +// commit order (maintainer decision B of 2026-10-23, recorded in +// docs/contributing/oracle-reviews/2026-10-22-sync-metadata-session.md). When +// it commits first, it is accepted ahead of the open transaction, and replay +// keeps the operation kinds it was accepted with. Here the inner insert of +// row 1 was written over the open transaction's truncate. In commit order it +// lands over the applied row 1, and it still names the whole row: replay must +// not make it a duplicate, with or without a hold. +it.each([false, true])( + `keeps an accepted insert when it commits ahead of an open truncate (held: %s)`, + async (held) => { + let sync!: Parameters[`sync`]>[0] + let release!: () => void + const collection = createCollection({ + getKey: (row) => row.id, + startSync: true, + sync: { + sync: (actions) => { + sync = actions + actions.begin() + actions.write({ type: `insert`, value: { id: 1, value: 0 } }) + actions.write({ type: `insert`, value: { id: 2, value: 0 } }) + actions.commit() + actions.markReady() + }, }, - ], - applied: validApplied, - } - - let primaryFailure: unknown - let hasPrimaryFailure = false - try { - await collection.stateWhenReady() - collection._state.pendingSyncedTransactions.push(valid, pending) - expect(() => collection._state.refreshPendingSyncedProjection()).toThrow( - SyncQueueInvariantError, - ) - expect(collection.get(1)?.value).toBe(0) - } catch (error) { - primaryFailure = error - hasPrimaryFailure = true - } - - const cleanupFailures: Array = [] - try { - for (const transaction of [pending, valid]) { - const index = - collection._state.pendingSyncedTransactions.indexOf(transaction) - if (index !== -1) - collection._state.pendingSyncedTransactions.splice(index, 1) + onUpdate: () => new Promise((resolve) => (release = resolve)), + }) + try { + await collection.stateWhenReady() + const blocker = held + ? collection.update(2, (draft) => { + draft.value = 9 + }) + : undefined + sync.begin() + sync.truncate() + sync.write({ type: `insert`, value: { id: 3, value: 1 } }) + sync.begin() + sync.write({ type: `insert`, value: { id: 1, value: 2 } }) + const inner = sync.commit() + expect(collection._state.syncedData.get(1)?.value).toBe(held ? 0 : 2) + // The truncate applies at once, after the inner transaction. + expect(sync.commit()).toBe(true) + await expect(Promise.resolve(inner)).resolves.not.toThrow() + if (blocker) { + release() + await blocker.isPersisted.promise + } + expect([...collection._state.syncedData.keys()]).toEqual([3]) + expect(collection._state.pendingSyncedTransactions).toHaveLength(0) + } finally { + await collection.cleanup() } - if (applied.isPending()) applied.resolve() - if (validApplied.isPending()) validApplied.resolve() - collection._state.refreshPendingSyncedProjection() - } catch (error) { - cleanupFailures.push(error) - } - try { - await collection.cleanup() - } catch (error) { - cleanupFailures.push(error) - } - if (hasPrimaryFailure && cleanupFailures.length > 0) { - throw new AggregateError( - [primaryFailure, ...cleanupFailures], - `Projection refresh and cleanup both failed`, - { cause: primaryFailure }, - ) - } - if (hasPrimaryFailure) throw primaryFailure - if (cleanupFailures.length > 0) - throw new AggregateError( - cleanupFailures, - `Projection refresh cleanup failed`, - ) -}) + }, +) it.each([ { initial: 0, updates: [1] }, diff --git a/packages/db/tests/db-client-hydration-authority-oracle.test.ts b/packages/db/tests/db-client-hydration-authority-oracle.test.ts index 1b6cffcced..0e411cbef1 100644 --- a/packages/db/tests/db-client-hydration-authority-oracle.test.ts +++ b/packages/db/tests/db-client-hydration-authority-oracle.test.ts @@ -7,147 +7,303 @@ import type { ChangeMessage } from '../src/types' /** * # Can a hydration seed supersede applied adapter work? * - * A hydration chunk seeds rows that the sync adapter has not established. An - * adapter insert, update, or delete is authoritative for its key, even when an - * older hydration chunk arrives later. An adapter truncate is authoritative - * for the whole source snapshot. A separate, untouched key may still receive - * hydration rows. The "Initial Data" contract in docs/guides/ssr.md makes - * hydration provisional and fresh adapter sync authoritative. + * A hydration seed sits beneath every source (adapter) write. It supplies only + * the objects that no surviving adapter write has written, whatever the order + * of arrival. Decision D5 in + * docs/contributing/oracle-reviews/2026-10-22-sync-metadata-session.md and the + * "Initial Data" contract in docs/guides/ssr.md state this law. Row data and + * row metadata are separate objects, and each has its own written flag: * - * The reference scans a declarative history for each key: the last adapter - * decision affecting that key wins; without one, the latest seed wins. It does - * not use Collection's queued projection or hydration classifier. The bounded - * grammar crosses four adapter decisions with hydration before, after, or on - * both sides of their applied receipt. It also crosses delete/reinsert, - * insert/delete, and truncate/reinsert histories. The driver uses DbClient initialData, - * a real sync transaction, and applyCollectionChunk. At each completed action - * it compares the complete retained source, public rows, and change-message - * mirror with the reference. Each callback also checks mirror/public agreement. - * A late chunk after delete rejects resurrection of that key; its untouched - * peer rejects whole-chunk suppression. A late chunk after truncate rejects - * resurrection of any key. One model adapter decision represents a sync - * transaction only after its applied receipt fulfills. + * - An adapter insert names the whole row and its metadata. Over a row that + * only a seed supplied, it replaces the row without a duplicate-key error, + * and an insert without metadata clears the seed's metadata. An insert equal + * to a row the source itself wrote is an equal re-insert and keeps metadata. + * - An adapter update merges over the row before it, a seed included, under + * `partial` mode, and replaces the row under `full` mode. It fixes the row + * data, and it fixes the metadata only when it names metadata. + * - An adapter metadata write sets a row's metadata and writes no row data. It + * fixes the metadata, so a later seed cannot supply it. A later seed still + * supplies the row data, because the source never wrote the row. + * - An adapter delete removes its key, and an adapter truncate removes the + * whole snapshot. Each fixes the affected objects as absent, and a later seed + * does not resurrect a removed key. + * - Clarification (b): once a surviving source write has written a row's data, + * a later seed is ignored for that row's data. A partial update merges over + * the row as it stood when the update applied, a seed included, so only a + * partial update's merge base depends on arrival order. Metadata follows the + * same rule per object: a later seed's metadata is ignored once the source has + * written that row's metadata, explicitly, by an insert's implicit clear, or + * by a delete or truncate. A seed still supplies metadata when the source + * wrote only the row's data, for example through an update without metadata. + * - Among seeds of an object that no surviving source write has written, the + * latest seed wins whole. A later seed without metadata leaves the metadata + * absent. `initialData` is the first seed and carries no metadata. * - * This owner covers one or two immediately applied adapter transactions, three - * keys, and at most two hydration chunks. The state-retention oracle and focused - * DbClient tests own queued transactions and cancellation. This test does not - * establish provider transport order or a later authoritative hydration epoch. + * The reference folds the declared history forward, key by key. Each object + * (row data, row metadata) has a written flag. A source write sets the flag and + * fixes the object; a seed fills only objects whose flag is clear. The + * reference does not use the queued projection or the hydration classifier. + * + * The grammar crosses seven adapter decisions in both `full` and `partial` row + * update modes: insert and update, each with and without metadata, a + * metadata-only write, delete, and truncate. Hydration arrives before, after, + * or on both sides of the adapter's applied receipt. That gives a grid of 56 + * cells (2 modes times 28): 7 single decisions times 3 hydration orders, and 7 + * two-decision histories (delete/reinsert, delete/update, insert/delete, + * truncate/reinsert) after hydration. Each chunk row carries metadata and a field that no adapter + * write names, `seeded`. The early chunk has `seeded`; the late chunk does not, + * so a partial merge of the late seed over the early one would keep the stale + * field. `initialData` gives its target `seeded: 'initial'`, so an adapter + * insert over an `initialData` row is observable in both modes. + * + * The driver uses DbClient initialData, a real sync transaction, and + * applyCollectionChunk. At each completed action it compares the synced store, + * the public rows, the change-message mirror, and the row metadata that + * `client.dehydrate()` reports with the reference. Each callback also checks + * mirror/public agreement. The change-message mirror carries no metadata, so + * metadata is observed only through dehydration. + * + * Pinned reference cases (not run against production) cover the equal + * re-insert, clarification (b) in both arrival orders, a metadata-less latest + * seed, and a metadata-only write held against a later seed. The reference + * throws on an insert that differs from a row the source wrote; the grammar + * never produces that history. + * + * Limits: this owner covers one or two immediately applied adapter + * transactions, all of them on the target key, three keys, and at most two + * hydration chunks. A source insert over a source-written row appears only in + * the pinned reference cases. `initialData` carries no metadata. The + * state-retention oracle and focused DbClient tests own queued transactions and + * cancellation. This test does not establish provider transport order or a + * later authoritative hydration epoch. */ -type Row = { id: string; name: string } -type AdapterDecision = `insert` | `update` | `delete` | `truncate` -type HydrationOrder = `before` | `after` | `both` +type Row = { id: string; name: string; seeded?: string } +type Metadata = { source: string } +type RowUpdateMode = `partial` | `full` +type SeedRow = { value: Row; metadata?: Metadata } type SyncActions = Parameters[`sync`]>[0] +type HydrationOrder = `before` | `after` | `both` +type HydrationStage = `before` | `after` + +/** Each adapter decision names its operation and whether it names metadata. */ +const adapterDecisions = { + insert: { op: `insert`, metadata: false }, + 'insert+metadata': { op: `insert`, metadata: true }, + update: { op: `update`, metadata: false }, + 'update+metadata': { op: `update`, metadata: true }, + metadata: { op: `metadata`, metadata: true }, + delete: { op: `delete`, metadata: false }, + truncate: { op: `truncate`, metadata: false }, +} as const +type AdapterDecision = keyof typeof adapterDecisions +type AdapterAction = + | { kind: `adapter`; op: `insert` | `update`; row: Row; metadata?: Metadata } + | { kind: `adapter`; op: `metadata`; metadata: Metadata } + | { kind: `adapter`; op: `delete` } + | { kind: `adapter`; op: `truncate` } type HistoryAction = - | { kind: `hydrate`; rows: ReadonlyArray } - | { kind: `adapter`; decision: `insert` | `update`; row: Row } - | { kind: `adapter`; decision: `delete` | `truncate` } + { kind: `hydrate`; rows: ReadonlyArray } | AdapterAction const target = `1` const peer = `2` const newKey = `3` -function initialRows(decision: AdapterDecision): Array { - const rows: Array = [{ id: peer, name: `initial-peer` }] - if (decision !== `insert`) rows.push({ id: target, name: `initial-target` }) - return rows -} +/** + * `initialData` is the first seed. It carries no metadata, and the target's + * `seeded` field is one that no adapter write names. + */ +const initialRows: ReadonlyArray = [ + { value: { id: peer, name: `initial-peer` } }, + { value: { id: target, name: `initial-target`, seeded: `initial` } }, +] -function hydrationRows(stage: `before` | `after`): Array { +/** + * The early chunk carries `seeded`, a field no adapter write names. The late + * chunk omits it, so a merge of the late chunk over the early one is visible. + */ +function hydrationRows(stage: HydrationStage): Array { + const seeded: { seeded?: string } = + stage === `before` ? { seeded: stage } : {} return [ { id: target, name: `${stage}-target` }, { id: peer, name: `${stage}-peer` }, { id: newKey, name: `${stage}-new` }, - ] + ].map((value) => ({ + value: { ...value, ...seeded }, + metadata: { source: `${stage}-hydration` }, + })) } -function adapterAction(decision: AdapterDecision): HistoryAction { - if (decision === `insert` || decision === `update`) - return { - kind: `adapter`, - decision, - row: { id: target, name: `adapter-${decision}` }, - } +function adapterAction(decision: AdapterDecision): AdapterAction { + const { op, metadata } = adapterDecisions[decision] + if (op === `delete` || op === `truncate`) return { kind: `adapter`, op } + if (op === `metadata`) + return { kind: `adapter`, op, metadata: { source: `adapter-metadata` } } return { kind: `adapter`, - decision, + op, + row: { id: target, name: `adapter-${op}` }, + ...(metadata ? { metadata: { source: `adapter-${op}` } } : {}), } } -/** Recompute each key's strongest decision from the declared history. */ +type ObservedRow = { id: string; row: Row; metadata: Metadata | undefined } +type Slot = { + row: Row | undefined + rowWritten: boolean + metadata: Metadata | undefined + metadataWritten: boolean +} + +const sameRow = (left: Row, right: Row): boolean => + left.id === right.id && + left.name === right.name && + left.seeded === right.seeded + +/** + * Forward fold, key by key. A source write sets the written flag of each object + * it touches. An insert names the row and its metadata: over a row that only a + * seed supplied it replaces both, and an insert without metadata clears the + * metadata. Over a row the source wrote, an equal insert keeps metadata unless + * it names some. An update merges over the current row under `partial` mode, + * and fixes metadata only when it names it. A metadata write fixes metadata + * alone. A delete fixes both objects as absent. A truncate fixes every key, + * including keys that a later seed names. A seed fills each object whose flag + * is clear, so a later seed replaces an earlier seed's object whole. + */ function referenceRows( - initial: ReadonlyArray, + initial: ReadonlyArray, history: ReadonlyArray, -): Array { - const keys = new Set([ - ...initial.map((row) => row.id), - ...history.flatMap((action) => - action.kind === `hydrate` - ? action.rows.map((row) => row.id) - : action.decision === `insert` || action.decision === `update` - ? [action.row.id] - : action.decision === `delete` - ? [target] - : [], - ), - ]) - const rows: Array = [] - - for (const key of keys) { - let decidedByAdapter = false - for (let index = history.length - 1; index >= 0; index--) { - const action = history[index]! - if (action.kind !== `adapter`) continue - if (action.decision === `truncate`) { - decidedByAdapter = true - break - } - if (action.decision === `delete`) { - if (key !== target) continue - decidedByAdapter = true - break - } - if (`row` in action && key === action.row.id) { - decidedByAdapter = true - rows.push(action.row) - break + rowUpdateMode: RowUpdateMode, +): Array { + const slots = new Map() + let truncated = false + const slotOf = (key: string): Slot => { + let slot = slots.get(key) + if (!slot) { + slot = { + row: undefined, + rowWritten: truncated, + metadata: undefined, + metadataWritten: truncated, } + slots.set(key, slot) } - if (decidedByAdapter) continue - - for (let index = history.length - 1; index >= -1; index--) { - const action = history[index] - const seedRows = - index === -1 ? initial : action?.kind === `hydrate` ? action.rows : [] - const seed = seedRows.find((row) => row.id === key) - if (seed) { - rows.push(seed) - break - } + return slot + } + const fixMetadata = (slot: Slot, metadata: Metadata | undefined): void => { + slot.metadata = metadata + slot.metadataWritten = true + } + const clear = (slot: Slot): void => { + slot.row = undefined + slot.rowWritten = true + fixMetadata(slot, undefined) + } + const insertRow = (slot: Slot, row: Row, metadata?: Metadata): void => { + if (slot.rowWritten && slot.row !== undefined) { + if (!sameRow(slot.row, row)) + throw new Error(`insert over a source-written row must re-insert it`) + if (metadata !== undefined) fixMetadata(slot, metadata) + return } + slot.row = row + slot.rowWritten = true + fixMetadata(slot, metadata) } - return rows.sort((left, right) => left.id.localeCompare(right.id)) + const seed = (rows: ReadonlyArray): void => { + for (const { value, metadata } of rows) { + const slot = slotOf(value.id) + if (!slot.rowWritten) slot.row = value + if (!slot.metadataWritten) slot.metadata = metadata + } + } + const write = (action: AdapterAction): void => { + if (action.op === `truncate`) { + truncated = true + for (const slot of slots.values()) clear(slot) + return + } + if (action.op === `delete`) { + clear(slotOf(target)) + return + } + if (action.op === `metadata`) { + fixMetadata(slotOf(target), action.metadata) + return + } + const slot = slotOf(action.row.id) + if (action.op === `insert`) { + insertRow(slot, action.row, action.metadata) + return + } + slot.row = + rowUpdateMode === `partial` ? { ...slot.row, ...action.row } : action.row + slot.rowWritten = true + if (action.metadata !== undefined) fixMetadata(slot, action.metadata) + } + + seed(initial) + for (const action of history) { + if (action.kind === `hydrate`) seed(action.rows) + else write(action) + } + return [...slots] + .flatMap(([id, slot]) => + slot.row === undefined + ? [] + : [{ id, row: slot.row, metadata: slot.metadata }], + ) + .sort((left, right) => left.id.localeCompare(right.id)) +} + +/** Row data fields only; collection virtual props such as `$synced` are not data. */ +function rowData(row: Row): Row { + return { id: row.id, name: row.name, seeded: row.seeded } } function observedRows(rows: Iterable): Array { return [...rows] - .map((row) => ({ id: row.id, name: row.name })) + .map(rowData) .sort((left, right) => left.id.localeCompare(right.id)) } -function applyAdapterDecision( +/** Row data and metadata as `client.dehydrate()` reports them. */ +function dehydratedRows( + client: DbClient, + collectionId: string, +): Array { + const chunk = client + .dehydrate() + .collections.find((item) => item.collectionId === collectionId) + return (chunk?.rows ?? []) + .map((row) => ({ + id: String(row.key), + row: rowData(row.value as Row), + metadata: row.metadata as Metadata | undefined, + })) + .sort((left, right) => left.id.localeCompare(right.id)) +} + +function applyAdapterAction( sync: SyncActions, - decision: AdapterDecision, + action: AdapterAction, ): ReturnType { sync.begin() - switch (decision) { + switch (action.op) { case `insert`: case `update`: sync.write({ - type: decision, - value: { id: target, name: `adapter-${decision}` }, + type: action.op, + value: action.row, + ...(action.metadata === undefined ? {} : { metadata: action.metadata }), }) break + case `metadata`: + if (!sync.metadata) throw new Error(`sync metadata API is unavailable`) + sync.metadata.row.set(target, action.metadata) + break case `delete`: sync.write({ type: `delete`, key: target }) break @@ -161,24 +317,27 @@ function applyAdapterDecision( async function runHistory( decisions: ReadonlyArray, order: HydrationOrder, + rowUpdateMode: RowUpdateMode, + seedRows: (stage: HydrationStage) => ReadonlyArray = hydrationRows, ): Promise { let sync!: SyncActions - const initial = initialRows(decisions[0]!) const client = new DbClient() const descriptor = collectionOptions({ - id: `hydration-authority-${decisions.join(`-`)}-${order}`, + id: `hydration-authority-${rowUpdateMode}-${decisions.join(`-`)}-${order}`, getKey: (row) => row.id, sync: { - rowUpdateMode: `full`, + rowUpdateMode, sync: (actions) => { sync = actions actions.markReady() }, }, }) - const collection = client.collection(descriptor, { initialData: initial }) + const collection = client.collection(descriptor, { + initialData: initialRows.map((seed) => seed.value), + }) const history: Array = [] - const mirror = new Map(initial.map((row) => [row.id, row])) + const mirror = new Map(initialRows.map((seed) => [seed.value.id, seed.value])) const callbackSnapshots: Array<{ mirror: Array publicRows: Array @@ -194,11 +353,7 @@ async function runHistory( (changes: Array>) => { for (const change of changes) { if (change.type === `delete`) mirror.delete(change.key) - else - mirror.set(change.key, { - id: change.value.id, - name: change.value.name, - }) + else mirror.set(change.key, rowData(change.value)) } callbackSnapshots.push({ mirror: observedRows(mirror.values()), @@ -209,20 +364,26 @@ async function runHistory( ) const check = () => { - const expected = referenceRows(initial, history) + const expected = referenceRows(initialRows, history, rowUpdateMode) + const expectedRows = expected.map((entry) => entry.row) expect(observedRows(collection._state.syncedData.values())).toEqual( - expected, + expectedRows, ) - expect(observedRows(collection.values())).toEqual(expected) - expect(observedRows(mirror.values())).toEqual(expected) + expect(observedRows(collection.values())).toEqual(expectedRows) + expect(observedRows(mirror.values())).toEqual(expectedRows) + expect(dehydratedRows(client, descriptor.id)).toEqual(expected) } check() - const hydrate = (stage: `before` | `after`) => { - const rows = hydrationRows(stage) + const hydrate = (stage: HydrationStage) => { + const rows = seedRows(stage) client.applyCollectionChunk({ collectionId: descriptor.id, - rows: rows.map((row) => ({ key: row.id, value: row })), + rows: rows.map(({ value, metadata }) => ({ + key: value.id, + value, + metadata, + })), }) history.push({ kind: `hydrate`, rows }) check() @@ -230,9 +391,9 @@ async function runHistory( if (order !== `after`) hydrate(`before`) for (const decision of decisions) { - const receipt = applyAdapterDecision(sync, decision) - await receipt - history.push(adapterAction(decision)) + const action = adapterAction(decision) + await applyAdapterAction(sync, action) + history.push(action) check() } if (order !== `before`) hydrate(`after`) @@ -271,54 +432,305 @@ async function runHistory( declareOracle({ check: [runHistory], model: [referenceRows] }) describe(`DbClient hydration authority oracle`, () => { - const cases: Array<{ + type Case = { decisions: ReadonlyArray label: string order: HydrationOrder - }> = ([`insert`, `update`, `delete`, `truncate`] as const).flatMap( - (decision) => - ([`before`, `after`, `both`] as const).map((order) => ({ - decisions: [decision], - label: decision, - order, - })), - ) - cases.push( - ...([`delete,insert`, `insert,delete`, `truncate,insert`] as const).map( - (label) => ({ + rowUpdateMode: RowUpdateMode + } + const orders: ReadonlyArray = [`before`, `after`, `both`] + const singles: ReadonlyArray = [ + `insert`, + `insert+metadata`, + `update`, + `update+metadata`, + `metadata`, + `delete`, + `truncate`, + ] + // A row an update brings back after a delete keeps the metadata the delete + // cleared: neither write leaves the metadata for a later seed. + const pairs = [ + `delete,insert`, + `delete,insert+metadata`, + `delete,update`, + `delete,update+metadata`, + `insert,delete`, + `truncate,insert`, + `truncate,insert+metadata`, + ] + const cases: Array = ([`full`, `partial`] as const).flatMap( + (rowUpdateMode) => [ + ...singles.flatMap((decision) => + orders.map((order) => ({ + decisions: [decision], + label: decision, + order, + rowUpdateMode, + })), + ), + ...pairs.map((label) => ({ decisions: label.split(`,`) as Array, label, order: `after` as const, - }), - ), + rowUpdateMode, + })), + ], ) + // Seeds for the metadata law. Seed A carries `seeded` and metadata; seed B + // carries neither `seeded` nor metadata unless a test says otherwise. + const seedA: SeedRow = { + value: { id: target, name: `seed-a`, seeded: `x` }, + metadata: { source: `seed-a` }, + } + const seedB: SeedRow = { + value: { id: target, name: `seed-b` }, + metadata: { source: `seed-b` }, + } + const targetOf = ( + rows: ReadonlyArray, + ): ObservedRow | undefined => rows.find((entry) => entry.id === target) + const hydrate = (rows: ReadonlyArray) => ({ + kind: `hydrate` as const, + rows, + }) + it(`reaches applied delete and truncate with late hydration`, () => { expect(cases).toContainEqual({ decisions: [`delete`], label: `delete`, order: `after`, + rowUpdateMode: `partial`, }) expect(cases).toContainEqual({ decisions: [`truncate`], label: `truncate`, order: `after`, + rowUpdateMode: `partial`, }) expect( - referenceRows(initialRows(`delete`), [ - adapterAction(`delete`), - { kind: `hydrate`, rows: hydrationRows(`after`) }, - ]), + referenceRows( + initialRows, + [adapterAction(`delete`), hydrate(hydrationRows(`after`))], + `partial`, + ), ).toEqual([ - { id: peer, name: `after-peer` }, - { id: newKey, name: `after-new` }, + { + id: peer, + row: { id: peer, name: `after-peer` }, + metadata: { source: `after-hydration` }, + }, + { + id: newKey, + row: { id: newKey, name: `after-new` }, + metadata: { source: `after-hydration` }, + }, ]) }) + it(`reference folds the D5 rules over pinned histories`, () => { + const before = hydrate(hydrationRows(`before`)) + const after = hydrate(hydrationRows(`after`)) + + // An insert replaces the seed's row and clears its metadata. + expect( + targetOf( + referenceRows( + initialRows, + [before, adapterAction(`insert`)], + `partial`, + ), + ), + ).toEqual({ + id: target, + row: { id: target, name: `adapter-insert` }, + metadata: undefined, + }) + // A partial update merges over the seed and keeps its unnamed field and metadata. + expect( + targetOf( + referenceRows( + initialRows, + [before, adapterAction(`update`)], + `partial`, + ), + ), + ).toEqual({ + id: target, + row: { id: target, name: `adapter-update`, seeded: `before` }, + metadata: { source: `before-hydration` }, + }) + // An update that does not name metadata leaves the metadata open to a later seed. + expect( + targetOf( + referenceRows(initialRows, [adapterAction(`update`), after], `partial`), + ), + ).toEqual({ + id: target, + row: { id: target, name: `adapter-update`, seeded: `initial` }, + metadata: { source: `after-hydration` }, + }) + // An update that names metadata fixes it against a later seed. + expect( + targetOf( + referenceRows( + initialRows, + [adapterAction(`update+metadata`), after], + `partial`, + ), + ), + ).toEqual({ + id: target, + row: { id: target, name: `adapter-update`, seeded: `initial` }, + metadata: { source: `adapter-update` }, + }) + // A metadata-only write fixes the metadata and leaves the row to a later seed. + expect( + targetOf( + referenceRows( + initialRows, + [adapterAction(`metadata`), after], + `partial`, + ), + ), + ).toEqual({ + id: target, + row: { id: target, name: `after-target` }, + metadata: { source: `adapter-metadata` }, + }) + // The latest seed wins for a key no adapter write has written. + expect( + referenceRows(initialRows, [before, after], `partial`).find( + (entry) => entry.id === peer, + ), + ).toEqual({ + id: peer, + row: { id: peer, name: `after-peer` }, + metadata: { source: `after-hydration` }, + }) + }) + + it(`reference keeps the metadata of an equal re-insert of a source-written row`, () => { + expect( + targetOf( + referenceRows( + [seedA], + [adapterAction(`insert+metadata`), adapterAction(`insert`)], + `partial`, + ), + ), + ).toEqual({ + id: target, + row: { id: target, name: `adapter-insert` }, + metadata: { source: `adapter-insert` }, + }) + // The clear applies only over seed-only rows, so an unequal re-insert is + // outside the grammar and the reference refuses it. + expect(() => + referenceRows( + [seedA], + [ + adapterAction(`insert`), + { kind: `adapter`, op: `insert`, row: { id: target, name: `other` } }, + ], + `partial`, + ), + ).toThrow(`insert over a source-written row must re-insert it`) + }) + + it(`reference lets seed B supply metadata when a partial update wrote only row data`, () => { + // Seed A's `seeded` survives the partial merge; B supplies only the metadata. + expect( + targetOf( + referenceRows( + [seedA], + [adapterAction(`update`), hydrate([seedB])], + `partial`, + ), + ), + ).toEqual({ + id: target, + row: { id: target, name: `adapter-update`, seeded: `x` }, + metadata: { source: `seed-b` }, + }) + // The update's metadata is the source's, so B's metadata is ignored. + expect( + targetOf( + referenceRows( + [seedA], + [adapterAction(`update+metadata`), hydrate([seedB])], + `partial`, + ), + ), + ).toEqual({ + id: target, + row: { id: target, name: `adapter-update`, seeded: `x` }, + metadata: { source: `adapter-update` }, + }) + }) + + it(`reference merges a partial update over the later seed when both seeds arrive first`, () => { + // Seed B replaces A's row whole, so the update's merge base has no `seeded`. + expect( + targetOf( + referenceRows( + [seedA], + [hydrate([seedB]), adapterAction(`update`)], + `partial`, + ), + ), + ).toEqual({ + id: target, + row: { id: target, name: `adapter-update` }, + metadata: { source: `seed-b` }, + }) + }) + + it(`reference leaves metadata absent when the latest seed names none`, () => { + expect( + targetOf( + referenceRows([seedA], [hydrate([{ value: seedB.value }])], `partial`), + ), + ).toEqual({ + id: target, + row: { id: target, name: `seed-b` }, + metadata: undefined, + }) + }) + + it(`reference holds a metadata-only write against a later seed`, () => { + expect( + targetOf( + referenceRows( + [seedA], + [adapterAction(`metadata`), hydrate([seedB])], + `partial`, + ), + ), + ).toEqual({ + id: target, + row: { id: target, name: `seed-b` }, + metadata: { source: `adapter-metadata` }, + }) + }) + it.each(cases)( - `keeps $label authoritative with hydration $order its receipt`, - async ({ decisions, order }) => runHistory(decisions, order), + `keeps $label authoritative with hydration $order its receipt in $rowUpdateMode mode`, + async ({ decisions, order, rowUpdateMode }) => + runHistory(decisions, order, rowUpdateMode), ) + + // Both seeds carry the same row fields, so the row check cannot tell a whole + // replacement from a merge; only the metadata distinguishes them. + it(`keeps metadata absent when the latest seed names none, in production`, () => { + const row = { id: target, name: `seed`, seeded: `initial` } + return runHistory([], `both`, `partial`, (stage) => + stage === `before` + ? [{ value: row, metadata: { source: `early-seed` } }] + : [{ value: row }], + ) + }) }) /** diff --git a/packages/db/tests/oracle-config.ts b/packages/db/tests/oracle-config.ts index 907038e6e5..8e6fdac1a7 100644 --- a/packages/db/tests/oracle-config.ts +++ b/packages/db/tests/oracle-config.ts @@ -201,6 +201,37 @@ const staticOracleProperties = [ `derived-publication.membership-work`, `collection-publication.metadata-cancellation`, `collection-publication.metadata-only`, + `sync-metadata-session.flat`, + `sync-metadata-session.nested`, + `sync-metadata-session.seeds`, + `sync-metadata-session.seeds-full`, + `sync-metadata-session.persisted-flat`, + `sync-metadata-session.persisted-nested`, + `sync-metadata-session.persisted-seeds`, + `sync-metadata-session.flat.hold-invariance`, + `sync-metadata-session.nested.hold-invariance`, + `sync-metadata-session.seeds.hold-invariance`, + `sync-metadata-session.seeds-full.hold-invariance`, + `sync-metadata-session.flat.key-independence`, + `sync-metadata-session.nested.key-independence`, + `sync-metadata-session.seeds.key-independence`, + `sync-metadata-session.seeds.seed-order`, + `sync-metadata-session.nested-seeds`, + `sync-metadata-session.nested-seeds.hold-invariance`, + `sync-metadata-session.nested-seeds.key-independence`, + `sync-metadata-session.nested-seeds.seed-order`, + `sync-metadata-session.seeds-full.seed-order`, + `sync-metadata-session.nested-conflicts`, + `sync-metadata-session.persisted-conflicts`, + `sync-metadata-session.persisted-cached`, + `sync-metadata-session.persisted-nested-seeds`, + `sync-metadata-session.persisted-reads`, + `sync-metadata-session.persisted-reads-seeds`, + `sync-metadata-session.persisted-peers`, + `sync-metadata-session.persisted-peers-full`, + `sync-metadata-session.persisted-peers-on-demand`, + `sync-metadata-session.nested-conflicts.hold-invariance`, + `sync-metadata-session.nested-conflicts.key-independence`, `d2-source.exact-retractions`, `d2-source.changed-restart`, `d2-source.disjoint-commutation`, diff --git a/packages/db/tests/oracle-replay-manifest.ts b/packages/db/tests/oracle-replay-manifest.ts index 873a1092ef..fd46c89917 100644 --- a/packages/db/tests/oracle-replay-manifest.ts +++ b/packages/db/tests/oracle-replay-manifest.ts @@ -610,6 +610,24 @@ const ownerGroups: ReadonlyArray< `settled-history`, ], [`db/tests/query/compiler/evaluators-oracle.test.ts`, `evaluators`, `like`], + [ + `db/tests/sync-metadata-session-oracle.property.test.ts`, + `sync-metadata-session`, + [ + `flat nested seeds seeds-full nested-seeds nested-conflicts`, + `flat.hold-invariance nested.hold-invariance seeds.hold-invariance`, + `seeds-full.hold-invariance nested-seeds.hold-invariance`, + `nested-conflicts.hold-invariance flat.key-independence`, + `nested.key-independence seeds.key-independence`, + `nested-seeds.key-independence nested-conflicts.key-independence`, + `seeds.seed-order nested-seeds.seed-order seeds-full.seed-order`, + ].join(` `), + ], + [ + `db-sqlite-persistence-core/tests/sync-metadata-session-persisted.test.ts`, + `sync-metadata-session`, + `persisted-flat persisted-nested persisted-seeds persisted-conflicts persisted-cached persisted-nested-seeds persisted-reads persisted-reads-seeds persisted-peers persisted-peers-full persisted-peers-on-demand`, + ], ] const owners = new Map() diff --git a/packages/db/tests/sync-metadata-session-oracle.property.test.ts b/packages/db/tests/sync-metadata-session-oracle.property.test.ts new file mode 100644 index 0000000000..9745fdb2fa --- /dev/null +++ b/packages/db/tests/sync-metadata-session-oracle.property.test.ts @@ -0,0 +1,1008 @@ +import { fc, test as fcTest } from '@fast-check/vitest' +import { describe, expect, it } from 'vitest' +import { DbClient, collectionOptions } from '../src/client.js' +import { DuplicateKeySyncError } from '../src/errors.js' +import { createOracleReach } from './oracle-reach.js' +import { + oraclePropertyOptions, + oracleReplayTest, + oracleRuns, +} from './oracle-config.js' +import { + absentMarker, + describeMismatch, + flatLane, + historyRegimes, + modelCheckpoints, + nestedConflictLane, + nestedLane, + nestedSeedLane, + runHoldInvariance, + runKeyIndependence, + runSeedOrderInvariance, + runSessionHistory, + seedFullLane, + seedLane, + sessionCollectionKeys, + sessionHistoryArbitrary, +} from './sync-metadata-session-oracle.js' +import type { + ReceiptState, + SessionAction, + SessionDriver, + SessionLane, + SessionModel, +} from './sync-metadata-session-oracle.js' +import type { SyncConfig } from '../src/types.js' + +/** + * Core driver for the sync metadata session oracle. The contract, model, + * grammar, and comparison live in `sync-metadata-session-oracle.ts`. + * + * The production path is a `DbClient` Collection with a real sync function. + * Each action calls the sync API that the sync function received. The hold is + * a persisting optimistic insert of a fresh fixture key (1000 and up), which + * no source action or truncate names; `release` resolves its mutation + * function. A seed is `DbClient.hydrate` with one row and its + * metadata. Cancel is `commit(signal)` with an aborted signal. + * + * Checkpoint: after each action returns, the driver reads `metadata.row.get` + * for rows 1 and 2, `metadata.collection.get` for each collection key, + * `metadata.collection.list()` and `list('gc:')` sorted by key, and + * `collection.base` for rows 1 and 2. A final checkpoint follows release of + * any hold, cancellation of any open transaction, and every receipt. + */ + +type Row = { id: number; v: unknown } +type SyncActions = Parameters[`sync`]>[0] + +let collectionCount = 0 + +async function createCoreDriver(fullRows = false): Promise { + const id = `sync-metadata-session-${collectionCount++}` + let sync!: SyncActions + let resolveMutation: (() => void) | undefined + const client = new DbClient() + const collection = client.collection( + collectionOptions({ + id, + getKey: (row) => row.id, + sync: { + rowUpdateMode: fullRows ? `full` : `partial`, + sync: (actions) => { + sync = actions + actions.begin() + actions.commit() + actions.markReady() + }, + }, + onInsert: () => + new Promise((resolve) => { + resolveMutation = resolve + }), + }), + ) + await collection.preload() + + let depth = 0 + let holdCount = 0 + let blocker: { isPersisted: { promise: Promise } } | undefined + const receipts: Array> = [] + const rejections: Array = [] + const receiptStates: Array = [] + const track = (receipt: true | Promise, expectAbort: boolean) => { + if (receipt === true) return + receipts.push( + receipt.then( + () => { + if (expectAbort) rejections.push(`cancelled transaction applied`) + }, + (error: unknown) => { + if ( + !expectAbort || + !(error instanceof Error) || + error.name !== `AbortError` + ) + rejections.push( + error instanceof Error + ? `${error.name}: ${error.message}` + : String(error), + ) + }, + ), + ) + } + const abort = () => { + const controller = new AbortController() + controller.abort() + track(sync.commit(controller.signal), true) + depth-- + } + const metadata = () => sync.metadata! + const read = (value: unknown) => (value === undefined ? absentMarker : value) + const release = async () => { + const pending = blocker + blocker = undefined + resolveMutation?.() + resolveMutation = undefined + await pending?.isPersisted.promise + } + + return { + begin: () => { + sync.begin() + depth++ + }, + commit: () => { + const index = receiptStates.length + const receipt = sync.commit() + depth-- + if (receipt === true) { + receiptStates.push(`applied`) + return + } + receiptStates.push(`pending`) + receipts.push( + receipt.then( + () => { + receiptStates[index] = `applied` + }, + () => { + receiptStates[index] = `rejected` + }, + ), + ) + }, + cancel: abort, + rowSet: (key, value) => metadata().row.set(key, value), + rowDelete: (key) => metadata().row.delete(key), + insert: (_key, value, rowMetadata) => + sync.write({ + type: `insert`, + value: value as Row, + ...(rowMetadata === undefined + ? {} + : { metadata: rowMetadata as Record }), + }), + update: (_key, value, rowMetadata) => + sync.write({ + type: `update`, + value: value as Row, + ...(rowMetadata === undefined + ? {} + : { metadata: rowMetadata as Record }), + }), + delete: (key) => sync.write({ type: `delete`, key }), + truncate: () => sync.truncate(), + collectionSet: (key, value) => metadata().collection.set(key, value), + collectionDelete: (key) => metadata().collection.delete(key), + hold: () => { + // Core lanes never generate a store hold. A fresh fixture key each time: no source action or truncate names it. + blocker = collection.insert({ id: 1000 + holdCount++, v: 0 }) + }, + release, + isDuplicateKeyError: (error) => error instanceof DuplicateKeySyncError, + seed: (key, row, rowMetadata) => + client.hydrate({ + collections: [ + { + collectionId: id, + rows: [ + { + key, + value: row as Row, + metadata: rowMetadata as Record, + }, + ], + }, + ], + }), + observe: () => ({ + rowMetadata: { + 1: read(metadata().row.get(1)), + 2: read(metadata().row.get(2)), + }, + collection: { + ...Object.fromEntries( + sessionCollectionKeys.map((key) => [ + key, + read(metadata().collection.get(key)), + ]), + ), + }, + list: sortedEntries(metadata().collection.list()), + prefixList: sortedEntries(metadata().collection.list(`gc:`)), + base: { + 1: read(collection.base.get(1)), + 2: read(collection.base.get(2)), + }, + receipts: [...receiptStates], + }), + flush: () => new Promise((resolve) => setTimeout(resolve, 0)), + settle: async () => { + await release() + while (depth > 0) abort() + await Promise.all(receipts) + return rejections + }, + cleanup: () => collection.cleanup(), + } +} + +function sortedEntries( + entries: ReadonlyArray<{ key: string; value: unknown }>, +): Array<[string, unknown]> { + return [...entries] + .map(({ key, value }): [string, unknown] => [key, value]) + .sort(([a], [b]) => a.localeCompare(b)) +} + +async function assertSessionHistory( + lane: SessionLane, + history: ReadonlyArray, +): Promise { + const { mismatch } = await runSessionHistory(lane, history, () => + createCoreDriver(lane.fullRows), + ) + if (mismatch) throw new Error(describeMismatch(history, mismatch)) +} + +// --------------------------------------------------------------------------- +// Named witnesses +// --------------------------------------------------------------------------- + +const a = ( + type: SessionAction[`type`], + key?: number | string, + metadata?: boolean, +) => + ({ + type, + ...(key === undefined ? {} : { key }), + ...(metadata === undefined ? {} : { metadata }), + }) as SessionAction + +/** + * Each witness reaches one law's premise. The model expectations below are + * derived by hand from the laws, so they also check the model against the + * wrong answers that production gives on `30175be48`. + */ +const witnesses = { + // A later seed beneath an open re-insert does not change its kind, but a + // source update that merges over that seed and commits ahead does: the + // re-insert then lands on a different row (decision 10). + lateSeedUnderMergedUpdate: { + lane: nestedSeedLane, + history: [ + a(`seed`, 1), + a(`begin`), + a(`upsert`, 1, true), + a(`reinsert`, 1, false), + a(`begin`), + { type: `seed`, key: 1, bare: true }, + a(`upsert`, 1, false), + a(`commit`), + ] as Array, + }, + // The same under a hold: the inner commit stays accepted but unapplied, so + // the seed still marks the row. + heldLateSeedUnderMergedUpdate: { + lane: nestedSeedLane, + history: [ + a(`hold`), + a(`seed`, 1), + a(`begin`), + a(`upsert`, 1, true), + a(`reinsert`, 1, false), + a(`begin`), + { type: `seed`, key: 1, bare: true }, + a(`upsert`, 1, false), + a(`commit`), + ] as Array, + }, + // Issue #2220: a held earlier write must stay visible after a later commit. + heldCursor: { + lane: flatLane, + history: [ + a(`hold`), + a(`begin`), + { type: `collectionSet`, key: `cursor` }, + a(`commit`), + a(`begin`), + a(`upsert`, 1, false), + a(`commit`), + ] as Array, + }, + // The open transaction overlays every held one, not only the newest. + openOverHeld: { + lane: flatLane, + history: [ + a(`hold`), + a(`begin`), + a(`rowSet`, 1), + { type: `collectionSet`, key: `gc:a` }, + a(`commit`), + a(`begin`), + ] as Array, + }, + // Independent transactions: an outer abort keeps an inner commit. + outerAbortKeepsInnerCommit: { + lane: nestedLane, + history: [ + a(`begin`), + a(`begin`), + a(`upsert`, 1, true), + a(`commit`), + a(`cancel`), + ] as Array, + }, + // Commit order: the outer transaction commits after the inner one, so its + // earlier write wins, and a hold does not change that order. + commitOrderDuringHold: { + lane: nestedLane, + history: [ + a(`hold`), + a(`begin`), + a(`rowSet`, 1), + a(`begin`), + a(`rowSet`, 1), + a(`commit`), + a(`commit`), + a(`release`), + ] as Array, + }, + // During a hold, the outer transaction can still write and commit after an + // inner one commits. + outerContinuesAfterInnerDuringHold: { + lane: nestedLane, + history: [ + a(`hold`), + a(`begin`), + a(`begin`), + a(`rowSet`, 1), + a(`commit`), + a(`rowSet`, 2), + a(`commit`), + ] as Array, + }, + // An inner truncate applies at its own commit, at once despite a hold. + innerTruncateAppliesAtItsCommit: { + lane: nestedLane, + history: [ + a(`begin`), + a(`upsert`, 1, true), + a(`commit`), + a(`hold`), + a(`begin`), + a(`begin`), + a(`truncate`), + a(`commit`), + a(`commit`), + ] as Array, + }, + // A seed that arrives under an open update changes the update's merge base, + // but never makes the source's equal re-insert a duplicate, and the + // re-insert still replaces the row: the late seed's extra field goes. + seedUnderOpenReinsert: { + lane: seedLane, + history: [ + a(`begin`), + { type: `seed`, key: 1, narrow: true, bare: true }, + a(`upsert`, 1, false), + a(`reinsert`, 1, false), + { type: `seed`, key: 1, bare: true }, + a(`commit`), + ] as Array, + }, + // A seed never supersedes a source metadata write on a row the source did not write. + seedUnderMetadataOnlyWrite: { + lane: seedLane, + history: [ + a(`begin`), + a(`rowSet`, 1), + a(`commit`), + a(`seed`, 1), + ] as Array, + }, + // A seed arriving under the source's open insert sits beneath it: the + // insert replaces the seeded row and clears its metadata. Consumers see the + // seed until the insert's transaction applies. + seedUnderOpenInsert: { + lane: seedLane, + history: [ + a(`begin`), + a(`upsert`, 1, false), + a(`seed`, 1), + a(`commit`), + ] as Array, + }, + // The same history with the insert committed before the seed arrives: the + // outcome does not depend on arrival order. + seedAfterCommittedInsert: { + lane: seedLane, + history: [ + a(`begin`), + a(`upsert`, 1, false), + a(`commit`), + a(`seed`, 1), + ] as Array, + }, + // A source insert equal to a seeded row is still an insert: the source never + // wrote the row, so the insert names its metadata and clears the seed's. + equalInsertOverSeed: { + lane: seedLane, + history: [ + a(`seed`, 1), + a(`begin`), + a(`reinsert`, 1, false), + ] as Array, + }, + // The latest seed of a row no source write has written wins whole, so a + // narrow seed drops the earlier seed's other field. + laterSeedReplacesSeed: { + lane: seedLane, + history: [ + a(`seed`, 1), + { type: `seed`, key: 1, narrow: true }, + ] as Array, + }, + // Once the source has updated a seeded row, a later seed is ignored for the + // row's data; the update wrote no metadata, so the late seed supplies it. + partialUpdateThenLateSeed: { + lane: seedLane, + history: [ + a(`seed`, 1), + a(`begin`), + a(`upsert`, 1, false), + a(`commit`), + { type: `seed`, key: 1, narrow: true }, + ] as Array, + }, + // The latest seed wins whole: a later seed without metadata leaves none. + bareLaterSeedClearsMetadata: { + lane: seedLane, + history: [ + a(`seed`, 1), + { type: `seed`, key: 1, bare: true }, + ] as Array, + }, + // A fresh source insert over a row only a seed supplied replaces the row + // and, naming no metadata, clears the seed's. + insertOverSeedReplaces: { + lane: seedLane, + history: [ + a(`seed`, 1), + a(`begin`), + a(`insertOverSeed`, 1, false), + a(`commit`), + ] as Array, + }, + // A source update merges over a seeded row under partial row updates, so the + // seed's other field and its metadata stay. + updateOverSeed: { + lane: seedLane, + history: [ + a(`seed`, 1), + a(`begin`), + a(`upsert`, 1, false), + a(`commit`), + ] as Array, + }, + // Under full row updates the update replaces the seeded row; its metadata + // stays, because the update names none. + updateOverSeedFullRows: { + lane: seedFullLane, + history: [ + a(`seed`, 1), + a(`begin`), + a(`upsert`, 1, false), + a(`commit`), + ] as Array, + }, + // An open truncate hides a held row write: a row read stops at the truncate. + openTruncateHidesHeldRow: { + lane: flatLane, + history: [ + a(`hold`), + a(`begin`), + a(`rowSet`, 1), + a(`commit`), + a(`begin`), + a(`truncate`), + ] as Array, + }, + // An open delete hides a held collection entry from `list()`. + openDeleteHidesHeldListEntry: { + lane: flatLane, + history: [ + a(`hold`), + a(`begin`), + a(`collectionSet`, `gc:a`), + a(`commit`), + a(`begin`), + a(`collectionDelete`, `gc:a`), + ] as Array, + }, + // `list('gc:')` includes the key equal to the prefix and excludes `gc`. + prefixBoundary: { + lane: flatLane, + history: [ + a(`begin`), + a(`collectionSet`, `gc`), + a(`collectionSet`, `gc:`), + ] as Array, + }, + // Under full row updates a held update replaces the row, so a later insert + // of the updated row is an equal re-insert, not a duplicate. A classifier + // that merged the held update would keep the inserted row's field `m`. + fullUpdateThenEqualInsertDuringHold: { + lane: seedFullLane, + history: [ + a(`begin`), + a(`upsert`, 1, true), + a(`commit`), + a(`hold`), + a(`begin`), + a(`upsert`, 1, false), + a(`commit`), + a(`begin`), + a(`reinsert`, 1, false), + ] as Array, + }, +} satisfies Record }> + +/** + * Each lane's replay names: its campaign, and the paired formulations that + * apply to it (key independence without full row updates, seed order with + * seeds). + */ +const lanes = [ + { + lane: flatLane, + property: `sync-metadata-session.flat`, + paired: { + 'hold-invariance': `sync-metadata-session.flat.hold-invariance`, + 'key-independence': `sync-metadata-session.flat.key-independence`, + }, + seed: 2_220_001, + }, + { + lane: nestedLane, + property: `sync-metadata-session.nested`, + paired: { + 'hold-invariance': `sync-metadata-session.nested.hold-invariance`, + 'key-independence': `sync-metadata-session.nested.key-independence`, + }, + seed: 2_220_002, + }, + { + lane: seedLane, + property: `sync-metadata-session.seeds`, + paired: { + 'hold-invariance': `sync-metadata-session.seeds.hold-invariance`, + 'key-independence': `sync-metadata-session.seeds.key-independence`, + 'seed-order': `sync-metadata-session.seeds.seed-order`, + }, + seed: 2_220_003, + // A seed beneath an open update needs both on one row, and duplicate + // inserts share the lane's weight, so it runs more fixed histories. + runs: 150, + }, + { + lane: seedFullLane, + property: `sync-metadata-session.seeds-full`, + paired: { + 'hold-invariance': `sync-metadata-session.seeds-full.hold-invariance`, + 'seed-order': `sync-metadata-session.seeds-full.seed-order`, + }, + seed: 2_220_004, + }, + { + lane: nestedSeedLane, + property: `sync-metadata-session.nested-seeds`, + paired: { + 'hold-invariance': `sync-metadata-session.nested-seeds.hold-invariance`, + 'key-independence': `sync-metadata-session.nested-seeds.key-independence`, + 'seed-order': `sync-metadata-session.nested-seeds.seed-order`, + }, + seed: 2_220_005, + }, + { + lane: nestedConflictLane, + property: `sync-metadata-session.nested-conflicts`, + paired: { + 'hold-invariance': `sync-metadata-session.nested-conflicts.hold-invariance`, + 'key-independence': `sync-metadata-session.nested-conflicts.key-independence`, + }, + seed: 2_220_006, + // Invalidation needs an outer insert, an inner write to the same row, + // and the inner commit, so this lane runs more fixed histories. + runs: 450, + }, +] as const + +/** + * Declared reach (ORC-015): the states each lane's laws need, and how many + * fixed-campaign histories must enter each. A seed over a source insert or an + * open update needs both on one row, and a write to an invalid transaction + * needs an invalidation first, so those minimums are lower. + */ +const common = 11 +const rare = 5 +const declaredReach: Record> = { + flat: [ + [`read with two held transactions`, common], + [`open transaction over a held transaction`, common], + [`truncate during a hold`, common], + [`duplicate source insert`, rare], + ], + nested: [ + [`nested commit`, common], + [`nested commit during a hold`, common], + [`abort keeps an inner commit`, common], + [`open transaction over a held transaction`, common], + ], + seeds: [ + [`seed`, common], + [`seed under a source insert`, rare], + [`seed under an open update`, rare], + [`open transaction over a held transaction`, common], + [`duplicate source insert over a seeded row`, rare], + [`duplicate source insert during a hold`, rare], + ], + 'nested-seeds': [ + [`seed`, common], + [`nested commit`, common], + [`inner write to an enclosing row`, rare], + [`seed under a source insert`, rare], + [`seed under an open update`, rare], + [`duplicate source insert over a seeded row`, rare], + ], + 'seeds-full': [ + [`seed`, common], + [`seed under a source insert`, rare], + [`open transaction over a held transaction`, common], + ], + 'nested-conflicts': [ + [`inner write to an enclosing row`, common], + [`nested commit during a hold`, common], + [`transaction invalidated`, common], + [`commit of an invalid transaction`, common], + [`write to an invalid transaction`, rare], + ], +} + +/** One reach counter per minimum, for a lane's fixed campaign. */ +function createLaneReach(lane: SessionLane, property: string) { + const states = declaredReach[lane.name]! + const counters = [...new Set(states.map(([, minimum]) => minimum))].map( + (minimum) => { + const declared = states + .filter(([, stateMinimum]) => stateMinimum === minimum) + .map(([state]) => state) + return { + declared: new Set(declared), + reach: createOracleReach(`${property} (fixed)`, declared, { minimum }), + } + }, + ) + return { + record(history: ReadonlyArray): void { + const regimes = historyRegimes(lane, history) + for (const { declared, reach } of counters) { + reach.beginHistory() + for (const regime of regimes) + if (declared.has(regime)) reach.reach(regime) + } + }, + assertReached(): void { + for (const { reach } of counters) reach.assertReached() + }, + } +} + +describe(`sync metadata session oracle`, () => { + it(`derives the named witnesses from the laws`, async () => { + type Observation = ReturnType + const at = async (name: keyof typeof witnesses, step: number) => { + const { lane, history } = witnesses[name] + return (await modelCheckpoints(lane, history)).get(step) as Observation + } + // Wrong answer on 30175be48: the held cursor reads as absent. + const held = await at(`heldCursor`, 6) + expect(held.collection.cursor).toEqual({ s: 2 }) + expect(held.base[1]).toBe(absentMarker) + // Wrong answer: the open transaction hides every held write. + const open = await at(`openOverHeld`, 5) + expect(open.rowMetadata[1]).toEqual({ s: 2 }) + expect(open.prefixList).toEqual([[`gc:a`, { s: 3 }]]) + // Wrong answer under closed nesting: the outer abort undoes the inner + // commit, and its receipt waits for the outer commit. + const innerCommitted = await at(`outerAbortKeepsInnerCommit`, 3) + const insertedRow = { id: 1, v: { s: 2 }, m: { s: 2 } } + expect(innerCommitted.rowMetadata[1]).toEqual({ s: 2 }) + expect(innerCommitted.base[1]).toEqual(insertedRow) + expect(innerCommitted.receipts).toEqual([`applied`]) + const outerAborted = await at(`outerAbortKeepsInnerCommit`, 4) + expect(outerAborted.rowMetadata[1]).toEqual({ s: 2 }) + expect(outerAborted.base[1]).toEqual(insertedRow) + // Wrong answers: issue order (the inner write wins), and on 30175be48 a + // held inner commit read over the open outer write and applied after it. + expect((await at(`commitOrderDuringHold`, 4)).rowMetadata[1]).toEqual({ + s: 4, + }) + expect((await at(`commitOrderDuringHold`, 5)).rowMetadata[1]).toEqual({ + s: 2, + }) + const released = await at(`commitOrderDuringHold`, 7) + expect(released.rowMetadata[1]).toEqual({ s: 2 }) + expect(released.receipts).toEqual([`applied`, `applied`]) + // Wrong answer on 30175be48: the outer transaction cannot write after an + // inner one commits during a hold. + const continued = await at(`outerContinuesAfterInnerDuringHold`, 6) + expect(continued.rowMetadata).toEqual({ 1: { s: 3 }, 2: { s: 5 } }) + expect(continued.receipts).toEqual([`pending`, `pending`]) + // Wrong answer under closed nesting: the inner truncate waits for the + // outer commit. + const innerTruncated = await at(`innerTruncateAppliesAtItsCommit`, 7) + expect(innerTruncated.base[1]).toBe(absentMarker) + expect(innerTruncated.rowMetadata[1]).toBe(absentMarker) + expect(innerTruncated.receipts).toEqual([`applied`, `applied`]) + expect((await at(`innerTruncateAppliesAtItsCommit`, 8)).receipts).toEqual([ + `applied`, + `applied`, + `pending`, + ]) + // Wrong answers: replay makes the re-insert a duplicate and invalidates + // the source's transaction, so only the late seed remains; or the + // re-insert merges and keeps the late seed's field. + const reinserted = await at(`seedUnderOpenReinsert`, 5) + expect(reinserted.base[1]).toEqual({ id: 1, v: { s: 2 } }) + expect(reinserted.receipts).toEqual([`applied`]) + // Wrong answer: the seed's metadata replaces the source's metadata write. + const seeded = await at(`seedUnderMetadataOnlyWrite`, 3) + expect(seeded.rowMetadata[1]).toEqual({ s: 1 }) + expect(seeded.base[1]).toEqual({ id: 1, v: { s: 3 }, seeded: { s: 3 } }) + // Wrong answer on 30175be48: the open insert becomes an update over the + // late seed, so the seed's metadata and other field survive it. + const underOpen = await at(`seedUnderOpenInsert`, 2) + expect(underOpen.rowMetadata[1]).toBe(absentMarker) + expect(underOpen.base[1]).toEqual({ id: 1, v: { s: 2 }, seeded: { s: 2 } }) + const openCommitted = await at(`seedUnderOpenInsert`, 3) + expect(openCommitted.rowMetadata[1]).toBe(absentMarker) + expect(openCommitted.base[1]).toEqual({ id: 1, v: { s: 1 } }) + const afterCommit = await at(`seedAfterCommittedInsert`, 3) + expect(afterCommit.rowMetadata[1]).toBe(absentMarker) + expect(afterCommit.base[1]).toEqual({ id: 1, v: { s: 1 } }) + // Wrong answer on 30175be48: core counts it as an equal re-insert. + const equalInsert = await at(`equalInsertOverSeed`, 2) + expect(equalInsert.rowMetadata[1]).toBe(absentMarker) + const truncated = await at(`openTruncateHidesHeldRow`, 5) + expect(truncated.rowMetadata[1]).toBe(absentMarker) + const deleted = await at(`openDeleteHidesHeldListEntry`, 5) + expect(deleted.list).toEqual([]) + expect(deleted.prefixList).toEqual([]) + const boundary = await at(`prefixBoundary`, 2) + expect(boundary.prefixList).toEqual([[`gc:`, { s: 2 }]]) + expect(boundary.list).toEqual([ + [`gc`, { s: 1 }], + [`gc:`, { s: 2 }], + ]) + const equalAfterFullUpdate = await at( + `fullUpdateThenEqualInsertDuringHold`, + 8, + ) + expect(equalAfterFullUpdate.rowMetadata[1]).toEqual({ s: 1 }) + expect(equalAfterFullUpdate.base[1]).toEqual({ + id: 1, + v: { s: 1 }, + m: { s: 1 }, + }) + const laterSeed = await at(`laterSeedReplacesSeed`, 1) + expect(laterSeed.rowMetadata[1]).toEqual({ s: 1 }) + expect(laterSeed.base[1]).toEqual({ id: 1, v: { s: 1 } }) + // Wrong answer on 30175be48: the late seed is dropped with its metadata. + const lateSeed = await at(`partialUpdateThenLateSeed`, 4) + expect(lateSeed.rowMetadata[1]).toEqual({ s: 4 }) + expect(lateSeed.base[1]).toEqual({ id: 1, v: { s: 2 }, seeded: { s: 0 } }) + // Wrong answer on 30175be48: the earlier seed's metadata survives. + const bare = await at(`bareLaterSeedClearsMetadata`, 1) + expect(bare.rowMetadata[1]).toBe(absentMarker) + expect(bare.base[1]).toEqual({ id: 1, v: { s: 1 }, seeded: { s: 1 } }) + // Wrong answer on 30175be48: the insert merges over the seed as an update. + const replacedSeed = await at(`insertOverSeedReplaces`, 3) + expect(replacedSeed.rowMetadata[1]).toBe(absentMarker) + expect(replacedSeed.base[1]).toEqual({ id: 1, v: { s: 2 } }) + const merged = await at(`updateOverSeed`, 3) + expect(merged.rowMetadata[1]).toEqual({ s: 0 }) + expect(merged.base[1]).toEqual({ id: 1, v: { s: 2 }, seeded: { s: 0 } }) + const replaced = await at(`updateOverSeedFullRows`, 3) + expect(replaced.rowMetadata[1]).toEqual({ s: 0 }) + expect(replaced.base[1]).toEqual({ id: 1, v: { s: 2 } }) + }) + + it.each(Object.entries(witnesses))( + `keeps the session law on witness %s`, + async (_name, { lane, history }) => { + await assertSessionHistory(lane, history) + }, + ) +}) + +// --------------------------------------------------------------------------- +// Campaigns +// --------------------------------------------------------------------------- + +/** + * Grammar ablation: each generated axis exists for a regime. Removing the + * axis must remove its regime from the fixed campaign, or the axis adds + * nothing the laws need. + */ +describe(`sync metadata session grammar ablation`, () => { + const ablations: Array<{ + lane: SessionLane + drop: (action: SessionAction) => boolean + regime: string + }> = [ + { + lane: flatLane, + drop: (x) => x.type === `hold`, + regime: `read with two held transactions`, + }, + { + lane: flatLane, + drop: (x) => x.type === `truncate`, + regime: `truncate during a hold`, + }, + { + lane: flatLane, + drop: (x) => x.type === `reinsert`, + regime: `equal re-insert`, + }, + { + lane: flatLane, + drop: (x) => x.type === `collectionSet` && x.undefinedValue === true, + regime: `undefined collection value`, + }, + { + lane: nestedLane, + drop: (x) => x.type === `cancel`, + regime: `abort keeps an inner commit`, + }, + { + lane: { ...nestedLane, maxDepth: 1 }, + drop: () => false, + regime: `nested commit`, + }, + { lane: seedLane, drop: (x) => x.type === `seed`, regime: `seed` }, + ] + it.each(ablations.map((ablation) => [ablation.regime, ablation] as const))( + `removing the axis for %s removes that regime`, + (_regime, { lane, drop, regime }) => { + const count = (filter: (action: SessionAction) => boolean) => + fc + .sample(sessionHistoryArbitrary(lane), { + numRuns: 200, + seed: 2_220_100, + }) + .filter((history) => + historyRegimes(lane, history.filter(filter)).has(regime), + ).length + const fullLane = + lane.maxDepth === 1 && regime === `nested commit` ? nestedLane : lane + const full = fc + .sample(sessionHistoryArbitrary(fullLane), { + numRuns: 200, + seed: 2_220_100, + }) + .filter((history) => + historyRegimes(fullLane, history).has(regime), + ).length + expect(full).toBeGreaterThan(10) + expect(count((action) => !drop(action))).toBe(0) + }, + ) +}) + +/** + * Second formulation: production against itself, with and without holds. + * It does not consult the model's reads, so it cannot share a fault in them. + */ +describe(`sync metadata session hold invariance`, () => { + for (const { lane, paired, seed } of lanes) { + const invariance = paired[`hold-invariance`] + const run = async (history: Array) => { + const mismatch = await runHoldInvariance(lane, history, () => + createCoreDriver(lane.fullRows), + ) + if (mismatch) throw new Error(describeMismatch(history, mismatch)) + } + oracleReplayTest(fcTest).prop([sessionHistoryArbitrary(lane)], { + numRuns: oracleRuns(50), + seed: seed + 100, + })( + `keeps ${lane.name} source reads independent of holds (fixed)`, + run, + 60_000, + ) + // A path replay of this property runs here. + oracleReplayTest(fcTest, invariance).prop( + [sessionHistoryArbitrary(lane)], + oraclePropertyOptions(50, invariance), + )( + `keeps ${lane.name} source reads independent of holds (random)`, + run, + 60_000, + ) + } +}) + +/** + * Two more production-against-production formulations. Key independence + * removes every action on row 2; seed order performs every seed first. + */ +const pairedFormulations = [ + { + suffix: `key-independence`, + title: `keeps row 1 independent of row 2`, + run: runKeyIndependence, + lanes: lanes.filter((entry) => `key-independence` in entry.paired), + }, + { + suffix: `seed-order`, + title: `keeps outcomes independent of seed arrival`, + run: runSeedOrderInvariance, + lanes: lanes.filter((entry) => `seed-order` in entry.paired), + }, +] as const + +describe(`sync metadata session paired formulations`, () => { + for (const formulation of pairedFormulations) + for (const { lane, paired, seed } of formulation.lanes) { + const name = (paired as Record)[formulation.suffix]! + const run = async (history: Array) => { + const mismatch = await formulation.run(lane, history, () => + createCoreDriver(lane.fullRows), + ) + if (mismatch) throw new Error(describeMismatch(history, mismatch)) + } + oracleReplayTest(fcTest).prop([sessionHistoryArbitrary(lane)], { + numRuns: oracleRuns(50), + seed: seed + 200, + })(`${formulation.title} on ${lane.name} histories (fixed)`, run, 60_000) + // A path replay of this property runs here. + oracleReplayTest(fcTest, name).prop( + [sessionHistoryArbitrary(lane)], + oraclePropertyOptions(50, name), + )(`${formulation.title} on ${lane.name} histories (random)`, run, 60_000) + } +}) + +describe(`sync metadata session campaigns`, () => { + for (const entry of lanes) { + const { lane, property, seed } = entry + const run = (history: Array) => + assertSessionHistory(lane, history) + const reach = createLaneReach(lane, property) + oracleReplayTest(fcTest).prop([sessionHistoryArbitrary(lane)], { + numRuns: oracleRuns(`runs` in entry ? entry.runs : 100), + seed, + })( + `keeps the session law on ${lane.name} histories (fixed)`, + (history) => { + reach.record(history) + return run(history) + }, + 120_000, + ) + oracleReplayTest(it)( + `reaches the declared states in the ${lane.name} fixed campaign`, + () => reach.assertReached(), + ) + // A path replay of this property runs here. + oracleReplayTest(fcTest, property).prop( + [sessionHistoryArbitrary(lane)], + oraclePropertyOptions(100, property), + )(`keeps the session law on ${lane.name} histories (random)`, run, 60_000) + } +}) diff --git a/packages/db/tests/sync-metadata-session-oracle.ts b/packages/db/tests/sync-metadata-session-oracle.ts new file mode 100644 index 0000000000..6e7772325a --- /dev/null +++ b/packages/db/tests/sync-metadata-session-oracle.ts @@ -0,0 +1,2529 @@ +import { fc } from '@fast-check/vitest' +import { declareOracle } from './oracle-declaration.js' + +/** + * # What does a sync source read back, and what do consumers see? + * + * A sync function writes rows and metadata through `begin`, `write`, + * `metadata.row`, `metadata.collection`, `truncate`, and `commit`. This oracle + * judges what the same sync run reads back through `metadata.row.get`, + * `metadata.collection.get`, and `metadata.collection.list`, and which rows + * consumers see through `collection.base`. + * + * ## The borrowed laws + * + * The contract borrows standard consistency theory. One sync run is one + * session. Every sync transaction is an independent transaction, including + * one begun while another is open. The API has no transaction handle, so + * `write`, `truncate`, `commit`, and abort act on the innermost open + * transaction. + * + * - Commit order. A transaction's writes take effect together at its commit, + * so accepted transactions apply in commit order, each transaction's + * writes in issue order. Each key holds one current value, the last write + * in that order (Terry et al. 1994, §3.4, monotonic writes, with commit as + * a transaction's write time). + * - Read your writes (Terry §3.1) and own reads (Adya 1999, §3.1.2): a read + * includes every accepted write of the session and every write of its + * open transactions, in serialization order: accepted transactions in + * commit order, then the open transactions in begin order. The innermost + * transaction's own writes therefore win. An inner transaction commits + * before the transactions it began inside, so once it commits, their open + * writes are ordered after its writes. A read can then change from the + * inner write to an outer one without a new write. The set of writes it + * includes still only grows (monotonic reads, Terry §3.2), except at an abort. + * - A held accepted transaction is still the session's write. A persisting + * optimistic transaction delays when consumers see it, never what the + * source reads. + * - Aborts: an abort discards only the aborted transaction's writes. A + * transaction committed inside it stays accepted. An inner transaction + * reads the open writes of the transactions it began inside, so an outer + * abort can leave a committed transaction that read an aborted write + * (Adya's G1a). The maintainer accepted these reads with independent + * transactions; the oracle does not judge what an inner transaction wrote + * because of them. + * - Atomic visibility (Bailis et al. 2014, monotonic atomic view): consumers + * see each transaction all at once. + * - Applied receipts (`SyncConfig.commit`): a receipt settles when its + * transaction applies. + * - Seeds come from another writer. A hydration seed is a snapshot older than + * every source write (Thomas's write rule). Row data and row metadata are + * separate objects. Once a committed source write has written an object + * (an insert, update, delete, or truncate for row data; an explicit write, + * an insert's implicit clear, a delete, or a truncate for metadata), a + * seed that arrives later is ignored for that object. A seed that arrives + * while the source's transaction is open sits beneath that transaction, + * as the maintainer's answer on an open insert ("seeds are older") put it. A source insert + * names the whole row and its metadata: it replaces a seeded row, without + * a duplicate-key error, and an insert without metadata clears the seed's + * metadata. A source update merges over the row as it stands when the + * update is written, a seed included, so only a partial update's merge + * base depends on arrival order. An insert equal to a row the source itself + * wrote is an equal re-insert and keeps its metadata; over a row that only + * a seed supplied it is an insert. Among seeds of a row that no source + * write has written, the latest seed wins whole, metadata included + * (`db-client-hydration-authority-oracle.test.ts`). + * + * Authority: maintainer decisions of 2026-10-22 to borrow standard theory and + * to place seeds beneath source writes (D4, D5, and its clarification "b"); + * the decision of 2026-10-23 to keep sync transactions independent, which + * withdrew closed nesting (D2) because handler and listener write-backs + * share the sync API with the source; the glossary + * entries for accepted sync transactions and the exposed authoritative base; + * and the row-metadata write rules of 2026-10-05 for implicit metadata. + * + * ## Limits + * + * The oracle does not judge `list()` order, live queries, change events, or + * the error an invalid transaction's commit rejects with. Stale runs and + * writes without an open transaction are fixed boundary witnesses, not + * generated histories. Only the `seeds-full` and `persisted-peers-full` + * lanes use full row updates. The `persisted-peers` lanes add a second tab's + * source through its notices; no other lane has a peer, and a peer's write + * reaches this tab only at a point where the wrapper has no durable step or + * storage read in flight. Only the peer lanes model storage, so only they + * compare the stored rows, and only after settlement: a restart reads them, + * but this oracle's claim ends with one sync run. A + * source update of an absent row is not a legal history (the change-message + * protocol), so it is not generated; it arises only when an inner + * transaction's update commits ahead of the enclosing write it relied on. + */ + +// --------------------------------------------------------------------------- +// History grammar +// --------------------------------------------------------------------------- + +/** + * Two source rows and five collection metadata keys separate per-key, + * cross-key, and prefix behaviour. Two keys sit under the `gc:` prefix; `gc:` + * equals the prefix, and `gc` is a proper prefix of it, so `list('gc:')` must + * include the first and exclude the second. The hold is a persisting + * optimistic insert of a fixture key that source actions never name. + */ +export const sessionRowKeys = [1, 2] as const +export const sessionCollectionKeys = [ + `cursor`, + `gc:a`, + `gc:b`, + `gc:`, + `gc`, +] as const +export const sessionPrefix = `gc:` +export type SessionRowKey = (typeof sessionRowKeys)[number] +export type SessionCollectionKey = (typeof sessionCollectionKeys)[number] + +export type SessionAction = + | { type: `begin` } + | { type: `commit` } + | { type: `cancel` } + /** + * Aborts the signal of the newest outermost transaction committed during + * the read hold, which the wrapper still buffers. The wrapper accepts a + * transaction when it hands it to core, so the transaction is abandoned + * there (decision 21). + */ + | { type: `abandon` } + | { type: `rowSet`; key: SessionRowKey } + | { type: `rowDelete`; key: SessionRowKey } + | { type: `upsert`; key: SessionRowKey; metadata: boolean } + | { type: `reinsert`; key: SessionRowKey; metadata: boolean } + | { type: `remove`; key: SessionRowKey } + | { type: `truncate` } + | { + type: `collectionSet` + key: SessionCollectionKey + undefinedValue?: boolean + } + | { type: `collectionDelete`; key: SessionCollectionKey } + | { type: `hold`; store?: boolean; read?: boolean } + | { type: `release` } + | { type: `seed`; key: SessionRowKey; narrow?: boolean; bare?: boolean } + | { type: `insertOverSeed`; key: SessionRowKey; metadata: boolean } + | { type: `conflictingInsert`; key: SessionRowKey; metadata: boolean } + | { + type: `peer` + key: SessionRowKey + notice: PeerNotice + /** + * A row notice whose row keeps the `v` this tab's accepted row has and + * drops its other fields, so merging it over that row changes nothing + * while the whole row replaces it. + */ + keepsValue?: boolean + /** + * A reset after which the resetting tab writes back the rows and + * metadata storage held, so the reset's reload finds what this tab + * holds. + */ + rewrites?: boolean + } + +/** + * How another tab's source write reaches this tab: a notice that names the + * row with its metadata, its metadata alone, or its delete, a notice that + * names no keys and makes this tab reload every row from storage, or a + * reset of storage (`collection:reset`). Model-only term: a peer is another + * tab; its notice is the `tx:committed` or `collection:reset` message. + */ +export type PeerNotice = `row` | `metadata` | `delete` | `reload` | `reset` + +/** + * A lane fixes which actions the grammar may choose and the nesting depth. + * `flat` histories are the issue's domain: no transaction begins while one + * is open. `nested` histories begin transactions inside open ones. `seeds` + * add a second writer; `seeds-full` repeats it under full row updates, and + * `nested-seeds` crosses seeds with nested transactions. Each lane is its own + * replay property. + */ +export type SessionLane = { + name: string + maxDepth: number + seeds: boolean + /** + * Whether `hold` may instead hold a wrapping sync's durable step. The model + * treats both kinds alike: neither changes what the source reads. + */ + storeHold?: boolean + /** + * Whether `hold` may instead keep a wrapping sync's storage read in + * flight. The wrapper buffers a transaction committed then until the read + * ends. Like the other holds, it changes nothing the source reads. + */ + readHold?: boolean + /** + * Whether `collectionSet` may store `undefined`. `get` cannot tell such a + * key from an absent one, but `list()` includes it. The persisted lane + * excludes it: stored collection metadata must be bindable. + */ + undefinedValues?: boolean + /** + * Whether the Collection uses `rowUpdateMode: 'full'`. An update then + * replaces the row instead of merging over it, so a source insert of a + * seeded row drops the seed's other fields. Other lanes use `'partial'`. + */ + fullRows?: boolean + /** + * Whether the grammar favours row writes to row 1 inside nested + * transactions, so an inner write often lands ahead of an enclosing write + * to the same row and an outer transaction becomes invalid. + */ + conflicts?: boolean + /** + * Whether row 1 is in the persisted cache when the run starts. A cached + * row is a snapshot older than every source write, so the model holds it + * as a seed accepted before any action (decision of 2026-10-23). + */ + cached?: boolean + /** + * Whether an insert over a different row replaces it, as the persisted + * wrapper's inserts do (maintainer decision C1 of 2026-10-23; the SQLite + * persistence plan's duplicate-key safety). Core instead makes the + * transaction invalid. The insert still clears metadata, as an insert. + */ + insertsReplace?: boolean + /** + * Whether the Collection loads on demand, with a demand for every row + * active from the start. A reset then keeps a held `DbClient` seed until + * the replayed demand replaces it (decision 19). + */ + onDemand?: boolean + /** + * Whether the grammar favours row writes that keep metadata: upserts and + * re-inserts without metadata, over metadata writes, removes, and inserts + * over a seeded row, which clear it. No action is excluded. A row's + * metadata then often stays a seed's after the source wrote the row and + * an equal re-insert kept it. + */ + keepsMetadata?: boolean + /** + * Whether another tab's source writes the rows too. Its notices reach this + * tab through the persisted coordinator (decisions 11, 13, and 15). + */ + peers?: boolean +} + +export const flatLane: SessionLane = { + name: `flat`, + maxDepth: 1, + seeds: false, + undefinedValues: true, +} +export const nestedLane: SessionLane = { + name: `nested`, + maxDepth: 3, + seeds: false, + undefinedValues: true, +} +export const seedLane: SessionLane = { + name: `seeds`, + maxDepth: 1, + seeds: true, + undefinedValues: true, +} +export const nestedSeedLane: SessionLane = { + name: `nested-seeds`, + maxDepth: 3, + seeds: true, + undefinedValues: true, +} +export const nestedConflictLane: SessionLane = { + name: `nested-conflicts`, + maxDepth: 3, + seeds: false, + undefinedValues: true, + conflicts: true, +} +export const seedFullLane: SessionLane = { + name: `seeds-full`, + maxDepth: 1, + seeds: true, + undefinedValues: true, + fullRows: true, +} +export const persistedFlatLane: SessionLane = { + name: `persisted-flat`, + maxDepth: 1, + seeds: false, + storeHold: true, + insertsReplace: true, +} +export const persistedNestedLane: SessionLane = { + name: `persisted-nested`, + maxDepth: 3, + seeds: false, + storeHold: true, + insertsReplace: true, +} +export const persistedConflictLane: SessionLane = { + name: `persisted-conflicts`, + maxDepth: 3, + seeds: false, + storeHold: true, + insertsReplace: true, + conflicts: true, +} +export const persistedCachedLane: SessionLane = { + name: `persisted-cached`, + maxDepth: 2, + seeds: false, + storeHold: true, + insertsReplace: true, + cached: true, +} +export const persistedSeedLane: SessionLane = { + name: `persisted-seeds`, + maxDepth: 1, + seeds: true, + storeHold: true, + insertsReplace: true, +} +/** + * Seeds crossed with nested transactions through the persisted wrapper, with + * the conflict lane's weighting: an inner re-insert of an enclosing row can + * commit first and land on a seeded row. The cached row and the favoured + * metadata-keeping writes let a row's metadata stay a seed's after the + * source wrote the row, so a later seed can replace it. + */ +/** + * Source writes while a storage read is in flight, nested, through the + * persisted wrapper: a read hold keeps an on-demand load open, so the wrapper + * buffers transactions committed during it. + */ +export const persistedReadLane: SessionLane = { + name: `persisted-reads`, + maxDepth: 2, + seeds: false, + storeHold: true, + readHold: true, + insertsReplace: true, +} +/** + * The read lane with `DbClient` seeds and the cached row, which a demand for + * every row loads at the start: a transaction the wrapper buffers during a + * read may have its signal aborted after it commits, which abandons it. + */ +export const persistedReadSeedLane: SessionLane = { + ...persistedReadLane, + name: `persisted-reads-seeds`, + seeds: true, + cached: true, + onDemand: true, +} +export const persistedNestedSeedLane: SessionLane = { + name: `persisted-nested-seeds`, + maxDepth: 2, + seeds: true, + storeHold: true, + insertsReplace: true, + conflicts: true, + cached: true, + keepsMetadata: true, +} +/** + * Another tab's source writes beside this tab's source, its seeds, the cached + * row, and nested transactions, through the persisted wrapper. + */ +export const persistedPeerLane: SessionLane = { + name: `persisted-peers`, + maxDepth: 2, + seeds: true, + storeHold: true, + insertsReplace: true, + cached: true, + peers: true, +} +/** + * The same under full row updates: an update replaces its row, so storage + * keeps no field the update dropped, neither the stored row's nor a seed's. + */ +export const persistedPeerFullLane: SessionLane = { + ...persistedPeerLane, + name: `persisted-peers-full`, + fullRows: true, +} +/** + * The same on demand, with a demand for every row: a reset keeps a held + * `DbClient` seed and deletes every other row (decision 19). + */ +export const persistedPeerOnDemandLane: SessionLane = { + ...persistedPeerLane, + name: `persisted-peers-on-demand`, + onDemand: true, +} + +const anyRowKey = fc.constantFrom(...sessionRowKeys) + +/** The cached row and metadata a `cached` lane starts with. */ +export const cachedRow = { id: 1, v: `cached`, seeded: `cached` } +export const cachedMetadata = { cached: true } + +/** A model at the start of a run on this lane. */ +export function createSessionModel(lane: SessionLane): SessionModel { + const model = new SessionModel( + lane.fullRows, + lane.insertsReplace === true, + lane.onDemand === true, + ) + if (lane.cached === true) + model.seed(1, cachedRow, { present: true, value: cachedMetadata }, true) + return model +} +const collectionKeyArbitrary = fc.constantFrom(...sessionCollectionKeys) + +/** + * Weights favour transaction boundaries and holds, because the laws concern + * what happens across commits: most generated writes then land inside an open + * transaction, and most histories commit more than once during a hold. + */ +export function sessionActionArbitrary( + lane: SessionLane, +): fc.Arbitrary { + // The conflict lane multiplies row writes and names row 1 three times as + // often as row 2. + const rows = lane.conflicts === true ? 3 : 1 + const rowKeyArbitrary = + lane.conflicts === true + ? fc.constantFrom(1, 1, 1, 2) + : anyRowKey + const weighted = ( + weight: number, + arbitrary: fc.Arbitrary, + ): fc.WeightedArbitrary => ({ weight, arbitrary }) + const metadataArbitrary = + lane.keepsMetadata === true + ? fc.constantFrom(false, false, false, false, false, false, false, true) + : fc.boolean() + const actions: Array> = [ + weighted(4, fc.constant({ type: `begin` as const })), + weighted(4, fc.constant({ type: `commit` as const })), + weighted(1, fc.constant({ type: `cancel` as const })), + weighted( + lane.keepsMetadata === true ? 1 : 2, + rowKeyArbitrary.map((key) => ({ type: `rowSet` as const, key })), + ), + weighted( + 1, + rowKeyArbitrary.map((key) => ({ type: `rowDelete` as const, key })), + ), + weighted( + lane.keepsMetadata === true ? 4 * rows : 2 * rows, + fc + .tuple(rowKeyArbitrary, metadataArbitrary) + .map(([key, metadata]) => ({ type: `upsert` as const, key, metadata })), + ), + weighted( + rows, + fc.tuple(rowKeyArbitrary, metadataArbitrary).map(([key, metadata]) => ({ + type: `reinsert` as const, + key, + metadata, + })), + ), + weighted( + lane.keepsMetadata === true ? 1 : rows, + rowKeyArbitrary.map((key) => ({ type: `remove` as const, key })), + ), + weighted(1, fc.constant({ type: `truncate` as const })), + // The conflict lane leaves collection metadata to the other lanes. + ...(lane.conflicts === true + ? [] + : [ + weighted( + 2, + fc + .tuple(collectionKeyArbitrary, fc.integer({ min: 0, max: 5 })) + .map(([key, roll]) => ({ + type: `collectionSet` as const, + key, + ...(lane.undefinedValues && roll === 0 + ? { undefinedValue: true } + : {}), + })), + ), + weighted( + 1, + collectionKeyArbitrary.map((key) => ({ + type: `collectionDelete` as const, + key, + })), + ), + ]), + weighted( + 3, + lane.readHold === true + ? fc.constantFrom(`core`, `store`, `read`).map((kind) => ({ + type: `hold` as const, + store: kind === `store`, + read: kind === `read`, + })) + : lane.storeHold + ? fc.boolean().map((store) => ({ type: `hold` as const, store })) + : fc.constant({ type: `hold` as const }), + ), + weighted(1, fc.constant({ type: `release` as const })), + ...(lane.readHold === true + ? // Legal only just after a commit during a read, so it weighs more. + [weighted(8, fc.constant({ type: `abandon` as const }))] + : []), + ] + if (lane.seeds || lane.cached === true) + actions.push( + weighted( + lane.keepsMetadata === true ? 1 : 2, + fc.tuple(rowKeyArbitrary, fc.boolean()).map(([key, metadata]) => ({ + type: `insertOverSeed` as const, + key, + metadata, + })), + ), + ) + if (lane.seeds) + actions.push( + weighted( + 4, + fc + .tuple(rowKeyArbitrary, fc.boolean(), fc.integer({ min: 0, max: 3 })) + .map(([key, narrow, roll]) => ({ + type: `seed` as const, + key, + narrow, + ...(roll === 0 ? { bare: true } : {}), + })), + ), + ) + if (lane.peers === true) + actions.push( + weighted( + 4, + fc + .tuple( + rowKeyArbitrary, + fc.constantFrom( + `row`, + `metadata`, + `delete`, + `reload`, + `reset`, + ), + fc.integer({ min: 0, max: 2 }), + ) + .map(([key, notice, roll]) => ({ + type: `peer` as const, + key, + notice, + ...(notice === `row` && roll === 0 ? { keepsValue: true } : {}), + ...(notice === `reset` && roll === 0 ? { rewrites: true } : {}), + })), + ), + ) + // A fresh insert over a row the source wrote names a different row. Core + // rejects it as a duplicate whatever holds or seeds there are; where + // inserts replace, it replaces that row. + actions.push( + weighted( + 1, + fc.tuple(rowKeyArbitrary, metadataArbitrary).map(([key, metadata]) => ({ + type: `conflictingInsert` as const, + key, + metadata, + })), + ), + ) + return fc.oneof(...actions) +} + +export function sessionHistoryArbitrary( + lane: SessionLane, +): fc.Arbitrary> { + return fc.array(sessionActionArbitrary(lane), { + minLength: 12, + maxLength: 48, + size: `medium`, + }) +} + +/** + * The regimes that a legal history reaches, read from the model as it + * replays. The campaign reach test requires each law's premise. + */ +export function historyRegimes( + lane: SessionLane, + history: ReadonlyArray, +): Set { + const model = createSessionModel(lane) + const regimes = new Set() + let reading = false + // Under full row updates an update can drop a field its row had; the + // settled storage check then reads what storage kept of that row. The drop + // counts once an outermost commit accepts it. + let pendingDrop = false + for (const [step, action] of history.entries()) { + if (!isLegal(lane, model, action)) continue + if (action.type === `hold` && action.read === true) reading = true + if (action.type === `release`) reading = false + if (reading && writesSource(action) && model.depth > 0) + regimes.add(`write during a storage read`) + // A read hold over rows the source already committed reads them back. + if ( + action.type === `hold` && + action.read === true && + sessionRowKeys.some((key) => model.sourceWroteRow(key)) + ) + regimes.add(`storage read of a stored row`) + const before = { + held: model.heldTransactions, + depth: model.depth, + invalidations: model.invalidations, + replacedConflicts: model.replacedConflicts, + invalid: model.depth > 0 && model.innermostInvalid, + seedOnlyRow: + `key` in action && + typeof action.key === `number` && + model.depth > 0 && + !model.innermostInvalid && + model.sourceRow(action.key).present && + !model.sourceWroteRow(action.key), + heldSeed: action.type === `peer` && model.holdsDbClientSeed(action.key), + anyHeldSeed: sessionRowKeys.some((key) => model.holdsDbClientSeed(key)), + openInsert: action.type === `peer` && model.hasOpenInsert(action.key), + writesEnclosingRow: + model.depth >= 2 && + (action.type === `upsert` || + action.type === `reinsert` || + action.type === `remove` || + action.type === `insertOverSeed` || + action.type === `conflictingInsert`) && + model.enclosingWroteRow(action.key), + } + const prior = + action.type === `upsert` ? model.sourceRow(action.key) : absent + void planAction(model, action, { s: step })(modelOnly) + const next = action.type === `upsert` ? model.sourceRow(action.key) : absent + if ( + action.type === `upsert` && + prior.present && + next.present && + Object.keys(prior.value as object).some( + (field) => !(field in (next.value as object)), + ) + ) + pendingDrop = true + if (model.depth === 0) { + if (action.type === `commit` && pendingDrop) + regimes.add(`accepted update drops a field`) + pendingDrop = false + } + if (before.held >= 2) regimes.add(`read with two held transactions`) + if (model.depth > 0 && model.heldTransactions >= 1) + regimes.add(`open transaction over a held transaction`) + if (action.type === `commit` && before.depth >= 2) { + regimes.add(`nested commit`) + if (model.isHolding) regimes.add(`nested commit during a hold`) + } + if (action.type === `cancel` && model.lastCancelKeptInnerCommit) + regimes.add(`abort keeps an inner commit`) + if ( + before.seedOnlyRow && + (action.type === `insertOverSeed` || action.type === `reinsert`) + ) + regimes.add(`source insert over a seeded row`) + if (before.seedOnlyRow && action.type === `upsert`) + regimes.add(`source update over a seeded row`) + if (action.type === `conflictingInsert`) { + regimes.add(`duplicate source insert`) + if (model.isHolding) regimes.add(`duplicate source insert during a hold`) + if (model.hasSeed(action.key)) + regimes.add(`duplicate source insert over a seeded row`) + } + if (model.invalidations > before.invalidations) + regimes.add(`transaction invalidated`) + if (model.replacedConflicts > before.replacedConflicts) + regimes.add(`insert replaces a different row`) + if (before.writesEnclosingRow) + regimes.add(`inner write to an enclosing row`) + if (before.invalid && action.type === `commit`) + regimes.add(`commit of an invalid transaction`) + if (before.invalid && writesSource(action)) + regimes.add(`write to an invalid transaction`) + if (action.type === `truncate` && model.isHolding) + regimes.add(`truncate during a hold`) + if (action.type === `seed`) { + regimes.add(`seed`) + if (model.hasSourceInsert(action.key)) + regimes.add(`seed under a source insert`) + if (model.hasOpenRowUpdate(action.key)) + regimes.add(`seed under an open update`) + if (model.keptByAcceptedReinsert(action.key)) + regimes.add(`seed under a kept re-insert`) + if (action.bare === true) regimes.add(`seed without metadata`) + } + if (action.type === `peer`) { + regimes.add(`peer ${action.notice} notice`) + if (before.heldSeed) + regimes.add( + action.notice === `reload` + ? `full reload over a held seed` + : `named peer write over a held seed`, + ) + if (before.openInsert) regimes.add(`peer write during an open insert`) + if (action.notice === `reset` && before.anyHeldSeed) + regimes.add(`reset over a held seed`) + if (action.rewrites === true) regimes.add(`reset that rewrites its rows`) + if (model.isHolding) regimes.add(`peer write during a hold`) + } + if (action.type === `reinsert`) regimes.add(`equal re-insert`) + if (action.type === `collectionSet` && action.undefinedValue === true) + regimes.add(`undefined collection value`) + if (action.type === `cancel`) regimes.add(`cancel`) + if (action.type === `abandon`) regimes.add(`abort after commit`) + if (action.type === `hold`) + regimes.add(action.store === true ? `store hold` : `core hold`) + } + return regimes +} + +/** Whether an action writes through the open transaction. */ +function writesSource(action: SessionAction): boolean { + return ![ + `begin`, + `commit`, + `cancel`, + `abandon`, + `hold`, + `release`, + `seed`, + `peer`, + ].includes(action.type) +} + +// --------------------------------------------------------------------------- +// Independent model +// --------------------------------------------------------------------------- + +/** + * The model keeps an event log rather than transaction write sets. Every + * source write appends one event naming its transaction frame; a frame + * records only its status. A read places the session's writes in + * serialization order and folds them, so the expected value follows + * directly from the laws: accepted transactions and seeds in the order they + * were accepted, then the open transactions in begin order, each + * transaction's events in issue order. The last write to an object wins; + * a seed counts only beneath every source write to its object. Production + * instead scans queued transaction write sets. The two structures differ on + * purpose. + * + * An insert's kind follows the row before it in that order (the row-metadata + * rules of 2026-10-05 and decision D5): over an absent row, or a row that + * only seeds supplied, it is an insert, and one without metadata clears the + * row's metadata; over an equal row the source wrote it is an equal + * re-insert and keeps it. A seed is older than every source write, so a seed + * accepted after the insert was written does not by itself change its kind, + * nor does an open transaction's write that merges over it. A transaction + * that commits after the seed, ordered before the insert, can merge over it, + * and the insert then lands on the row that write left (decision 10). An open transaction's kinds follow the order as it changes; a commit + * fixes them in the order as the transaction read it, before it moves ahead + * of the transactions it began inside (decision 14). + * When an inner transaction commits first, an enclosing transaction's insert + * can land on a different row that the source wrote. That transaction is then + * invalid, as a duplicate-key insert would be: no read includes its writes, + * it ignores later writes, a truncate included, and its commit rejects. + * Lanes whose inserts replace (the persisted wrapper) keep the transaction, + * and the insert replaces that row and clears its metadata. + * + * The source and consumers read different layers of the same order. The + * source reads every accepted transaction and seed, applied or held, and its + * open transactions, because accepted work always applies and the source's + * session includes it. Consumers read only applied transactions and applied + * seeds, because a hold or an open transaction delays what they see. + */ +type ObjectStore = `row` | `rowMetadata` | `collection` +type ModelValue = { present: false } | { present: true; value: unknown } +const absent: ModelValue = { present: false } + +type InsertKind = `insert` | `reinsert` | `duplicate` +type ModelWrite = { + frame: number + /** When the write was issued, to tell it from a later seed. */ + seq: number + truncate: false + store: ObjectStore + key: string | number + value: ModelValue + /** + * How a row write meets the row before it. Under partial row updates an + * `update` merges its fields over that row, a seed included. An + * `insert`, and every write under full row updates, replaces it. + */ + rowWrite?: `insert` | `update` + /** An insert without metadata, which clears it unless it re-inserts. */ + clears?: boolean + /** An insert's kind, fixed when its transaction commits. */ + kind?: InsertKind +} +/** + * A truncate; a schema reset on demand keeps the rows and metadata of the + * `DbClient` seeds it holds (decision 20). + */ +type ModelTruncate = { + frame: number + seq: number + truncate: true + keeps?: ReadonlySet +} +type ModelEvent = ModelTruncate | ModelWrite + +/** Whether an event truncates this row's data and metadata. */ +function truncatesRow(event: ModelEvent, key: string | number): boolean { + return event.truncate && event.keeps?.has(key) !== true +} + +type FrameStatus = `open` | `invalid` | `accepted` | `applied` | `aborted` +type ModelFrame = { + status: FrameStatus + /** Whether a transaction begun inside this one committed while it was open. */ + innerCommitted: boolean +} +type ModelSeed = { + key: SessionRowKey + row: unknown + metadata: ModelValue + applied: boolean + seq: number + /** + * A previous run's stored row (decision 9) rather than a `DbClient` seed. + * Both are seeds; only a `DbClient` seed holds its row when a notice names + * no keys, and only a stored row is in storage. + */ + cached: boolean + /** + * The objects a source write had written when the seed arrived. A seed is + * judged then (decision 22), so a write abandoned later does not revive it. + */ + lost?: { row: boolean; rowMetadata: boolean } +} +/** One step of the serialization order: a source write or a seed. */ +type ModelEntry = { event: ModelEvent } | { seed: ModelSeed } + +export type SessionObservation = { + rowMetadata: Record + collection: Record + list: Array<[string, unknown]> + prefixList: Array<[string, unknown]> + base: Record + /** + * Each `commit()` receipt by commit order: `applied` once its writes are + * visible, `rejected` once they are abandoned, otherwise `pending`. A + * receipt that `commit()` returned as `true` counts as `applied`. + */ + receipts: Array +} + +export type ReceiptState = `pending` | `applied` | `rejected` + +/** Observations encode absence as this marker so `undefined` stays visible. */ +export const absentMarker = `` + +export class SessionModel { + /** + * Under full row updates an update replaces the row. On demand, a reset + * keeps a held `DbClient` seed (decision 19). + */ + constructor( + private readonly fullRows = false, + private readonly insertsReplace = false, + private readonly resetKeepsSeeds = false, + ) {} + + /** Whether an insert over a different source row replaces it. */ + get insertsReplaceRows(): boolean { + return this.insertsReplace + } + + /** Whether a seed of this row arrived in the session. */ + hasSeed(key: SessionRowKey): boolean { + return this.sessionEntries().some( + (entry) => `seed` in entry && entry.seed.key === key, + ) + } + + private events: Array = [] + private frames: Array = [] + private openStack: Array = [] + /** Accepted transactions and seeds, in the order they were accepted. */ + private accepted: Array<{ frame: number } | { seed: ModelSeed }> = [] + private holding = false + private reading = false + /** Outermost transactions committed during the read hold, oldest first. */ + private committedDuringRead: Array = [] + /** A transaction whose signal aborted; the read's end abandons it. */ + private abandoning: number | undefined + /** Whether a reset happened; later notices then reload storage. */ + private fenced = false + private noticesBlocked = false + /** + * What a reload restores for each row: storage's row and metadata, which + * the peer's notices name. Each of this tab's transactions stores the row + * that this tab shows once it commits, fields a `DbClient` seed supplied + * included (decision 17), and the metadata it wrote. Row metadata of a row + * that storage lacks is not durable, but a reload keeps it in this tab + * until a source deletes the row (decision 16), so it stands here too. + * Metadata no source wrote in this run is a seed's, and a reload after a + * notice that names its keys keeps it as this tab shows it (decision 18). + */ + private storage = new Map< + SessionRowKey, + { row: ModelValue; metadata: ModelValue } + >() + + get depth(): number { + return this.openStack.length + } + get isHolding(): boolean { + return this.holding + } + /** Accepted transactions that have not applied. */ + get heldTransactions(): number { + return this.accepted.filter( + (item) => + `frame` in item && this.frames[item.frame]!.status !== `applied`, + ).length + } + lastCancelKeptInnerCommit = false + /** How many transactions have become invalid. */ + invalidations = 0 + /** Where inserts replace: checks that found an insert over a different row. */ + replacedConflicts = 0 + /** The frame each `commit()` closed, in commit order. */ + private committed: Array = [] + private seq = 0 + + begin(): void { + this.frames.push({ status: `open`, innerCommitted: false }) + this.openStack.push(this.frames.length - 1) + } + + /** Whether the innermost open transaction is invalid. */ + get innermostInvalid(): boolean { + return this.frames[this.innermost()]?.status === `invalid` + } + + write(store: ObjectStore, key: string | number, value: ModelValue): void { + if (this.innermostInvalid) return + this.events.push({ + frame: this.innermost(), + seq: this.seq++, + truncate: false, + store, + key, + value, + }) + } + + /** + * A source insert or update of a row's data. An insert names the row's + * whole metadata, so `clears` marks one written without metadata. + */ + writeRow( + key: SessionRowKey, + row: unknown, + rowWrite: `insert` | `update`, + clears = false, + ) { + if (this.innermostInvalid) return + this.events.push({ + frame: this.innermost(), + seq: this.seq++, + truncate: false, + store: `row`, + key, + value: { present: true, value: row }, + rowWrite, + ...(rowWrite === `insert` && clears ? { clears } : {}), + }) + } + + /** Whether the row is present from source writes alone, ignoring seeds. */ + sourceWroteRow(key: SessionRowKey): boolean { + const entries = this.sessionEntries().filter((entry) => `event` in entry) + return this.fold(`row`, key, entries).present + } + + /** + * Whether a valid transaction that the innermost one began inside has + * written this row's data, or truncated. Committing first puts the inner + * write ahead of that one. + */ + enclosingWroteRow(key: SessionRowKey): boolean { + const enclosing = new Set( + this.openStack + .slice(0, -1) + .filter((frame) => this.frames[frame]!.status === `open`), + ) + return this.events.some( + (event) => + enclosing.has(event.frame) && + (event.truncate + ? truncatesRow(event, key) + : event.store === `row` && event.key === key), + ) + } + + /** + * A truncate discards its transaction's earlier row data and row metadata + * writes; collection metadata writes stay. + */ + truncate(): void { + if (this.innermostInvalid) return + const frame = this.innermost() + this.events = this.events.filter( + (event) => + event.frame !== frame || event.truncate || event.store === `collection`, + ) + this.events.push({ frame, seq: this.seq++, truncate: true }) + } + + /** The row as the source sees it; the driver classifies inserts by it. */ + sourceRow(key: SessionRowKey): ModelValue { + return this.fold(`row`, key, this.sessionEntries()) + } + + /** + * Each transaction is independent. A commit accepts it, after every + * transaction accepted before it and beneath every open one. It applies + * at once unless a hold is active, and a transaction that truncates + * applies at once regardless, with everything accepted before it. + */ + commit(): void { + const frame = this.innermost() + this.committed.push(frame) + if (this.frames[frame]!.status === `invalid`) { + this.openStack.pop() + return + } + // A commit fixes the kinds its inserts have in the order as it stands. + const entries = this.sessionEntries() + entries.forEach((entry, index) => { + const event = `event` in entry ? entry.event : undefined + if (event && event.frame === frame && isInsert(event)) + event.kind = this.insertKind(entries, index) + }) + this.openStack.pop() + for (const enclosing of this.openStack) + this.frames[enclosing]!.innerCommitted = true + this.frames[frame]!.status = `accepted` + this.accepted.push({ frame }) + if (this.reading && this.openStack.length === 0 && !this.truncates(frame)) + this.committedDuringRead.push(frame) + this.store(frame) + if (!this.holding || this.truncates(frame)) this.applyAccepted() + this.invalidateOpen() + } + + /** + * A committed transaction stores, for every row it wrote or truncated, the + * row and metadata this tab now shows; for a row whose metadata alone it + * wrote, that metadata. + */ + private store(frame: number): void { + const truncates = this.truncates(frame) + const entries = this.acceptedEntries() + for (const key of sessionRowKeys) { + const writes = this.events.filter( + (event) => + event.frame === frame && !event.truncate && event.key === key, + ) + const writesRow = + truncates || + writes.some((event) => !event.truncate && event.store === `row`) + // An insert names its metadata; an update without metadata keeps it + // and stores none (decision 18). + const writesMetadata = + truncates || + writes.some( + (event) => + !event.truncate && + (event.store === `rowMetadata` || + event.rowWrite === `insert` || + !event.value.present), + ) + if (!writesRow && !writesMetadata) continue + const previous = this.storage.get(key) ?? { + row: absent, + metadata: absent, + } + this.storage.set(key, { + row: writesRow ? this.fold(`row`, key, entries) : previous.row, + metadata: writesMetadata + ? this.fold(`rowMetadata`, key, entries) + : previous.metadata, + }) + } + } + + /** An abort discards only the innermost open transaction's own writes. */ + cancel(): void { + const frame = this.openStack.pop()! + this.frames[frame]!.status = `aborted` + this.lastCancelKeptInnerCommit = this.frames[frame]!.innerCommitted + } + + hold(blocksNotices = false, reads = false): void { + this.holding = true + this.noticesBlocked = blocksNotices + this.reading = reads + } + + /** + * Whether the newest transaction is outermost, committed during the read + * hold, and has no transaction begun after it. A transaction open over a + * buffered one reads its writes, which makes it a dependent successor that + * the abort-graph oracle owns. + */ + get canAbandon(): boolean { + return ( + this.abandoning === undefined && + this.depth === 0 && + this.committedDuringRead.at(-1) === this.frames.length - 1 + ) + } + + /** Whether only the read's end may come next, which abandons a commit. */ + get awaitsAbandonment(): boolean { + return this.abandoning !== undefined + } + + /** + * The wrapper accepts a source transaction when it hands it to core, so an + * abort of the signal while a read still buffers it abandons it when the + * read ends; until then the source reads it (decision 21). A seed that + * arrived meanwhile stays judged against it (decision 22). Only the read's + * end may follow, so no later transaction depends on it here; the + * abort-graph oracle owns dependent suffixes. + */ + abandon(): void { + this.abandoning = this.committedDuringRead.pop() + } + + release(): void { + const frame = this.abandoning + if (frame !== undefined) { + // Its writes never apply. A seed that arrived after it was judged + // then, so it keeps every object it had lost (decision 22). + const entries = this.acceptedEntries() + entries.forEach((entry, index) => { + if (`seed` in entry) + entry.seed.lost = { + row: + entry.seed.lost?.row === true || + this.sourceWroteBefore(`row`, entry.seed.key, entries, index), + rowMetadata: + entry.seed.lost?.rowMetadata === true || + this.sourceWroteBefore( + `rowMetadata`, + entry.seed.key, + entries, + index, + ), + } + }) + this.frames[frame]!.status = `aborted` + this.accepted = this.accepted.filter( + (item) => !(`frame` in item && item.frame === frame), + ) + this.abandoning = undefined + } + this.holding = false + this.noticesBlocked = false + this.reading = false + this.committedDuringRead = [] + this.applyAccepted() + } + + /** + * Whether a store hold or a read hold is active. The wrapper handles a + * peer's notice after its durable steps and storage reads, so a notice is + * delivered only when neither is in flight. + */ + get blocksPeerNotices(): boolean { + return this.noticesBlocked + } + + /** + * A peer's write is a source write of this run (decision 11). A notice that + * names its keys lands it after every accepted transaction and beneath the + * open ones, as an inner commit does, so open inserts are judged against + * it. A notice that names no keys makes this tab reload storage: a + * `DbClient` seed this tab holds keeps its row (decision 13), and every + * other row or metadata that storage holds differently from this tab + * becomes a source write of the stored value (decision 15). A reset empties + * storage; every notice after it reloads storage (the reset fence). + */ + peer( + key: SessionRowKey, + notice: PeerNotice, + value: unknown, + rowValue: unknown = value, + rewrites = false, + ): void { + const row: ModelValue = { present: true, value: { id: key, v: rowValue } } + const metadata: ModelValue = { present: true, value: { peer: value } } + if (notice === `reset`) { + this.reset(rewrites) + return + } + // The peer's write reaches storage before its notice. + const previous = this.storage.get(key) ?? { + row: absent, + metadata: absent, + } + this.storage.set( + key, + notice === `metadata` + ? { row: previous.row, metadata } + : notice === `delete` + ? { row: absent, metadata: absent } + : { row, metadata }, + ) + // After a reset, a notice in the reset's term may predate it, so this + // tab reloads storage instead (the reset fence); the peer keeps one term. + if (notice !== `reload` && !this.fenced) { + this.acceptPeerFrame( + notice === `row` + ? [ + [`row`, key, row], + [`rowMetadata`, key, metadata], + ] + : notice === `metadata` + ? [[`rowMetadata`, key, metadata]] + : [ + [`row`, key, absent], + [`rowMetadata`, key, absent], + ], + ) + return + } + // A held seed keeps the peer's write from this tab until a notice names + // the row; a write of the row by this tab stores this tab's row over it + // (decision 17). + const current = this.acceptedEntries() + const writes: Array<[ObjectStore, SessionRowKey, ModelValue]> = [] + for (const other of sessionRowKeys) { + if (this.holdsDbClientSeed(other)) continue + const { row: storedRow, metadata: storedMetadata } = this.storage.get( + other, + ) ?? { row: absent, metadata: absent } + // This tab cannot tell what a source wrote, so stored metadata that + // differs is a source write too (decision 15). Only after a notice that + // names its keys does a reload keep a seed's metadata (decision 18). + const rowChanged = + canonical(storedRow) !== canonical(this.fold(`row`, other, current)) + const metadataChanged = + canonical(storedMetadata) !== + canonical(this.fold(`rowMetadata`, other, current)) + if (!rowChanged && !metadataChanged) continue + if (rowChanged) writes.push([`row`, other, storedRow]) + if (metadataChanged) writes.push([`rowMetadata`, other, storedMetadata]) + } + if (writes.length > 0) this.acceptPeerFrame(writes) + } + + /** + * Another tab empties storage and resets it. This tab reloads the empty + * storage, a source truncate (decision 15) of its rows, their metadata, and + * its collection metadata, accepted like a peer's write. Eager, the truncate + * takes a held `DbClient` seed too. On demand, a held seed keeps its row + * and metadata until the replayed demand replaces it (decision 19), so the + * reset instead deletes each other row and row metadata this tab has + * accepted; a later seed of a row it never saw still applies (decision 20). + */ + private reset(rewrites: boolean): void { + // Only rows are durable; metadata of a row storage lacks stood here only + // for this tab (decision 16), and a reset truncates it. + for (const key of sessionRowKeys) + if (!rewrites || !this.storedRow(key).present) this.storage.delete(key) + this.fenced = true + const keeps = new Set( + this.resetKeepsSeeds + ? sessionRowKeys.filter((key) => this.holdsDbClientSeed(key)) + : [], + ) + // Rows a resetting tab wrote back are stored again, so the reset's + // reload writes them after its truncate, as source writes. + const restored = rewrites + ? sessionRowKeys + .filter((key) => !keeps.has(key) && this.storedRow(key).present) + .flatMap((key): Array<[ObjectStore, SessionRowKey, ModelValue]> => [ + [`row`, key, this.storage.get(key)!.row], + [`rowMetadata`, key, this.storage.get(key)!.metadata], + ]) + : [] + this.acceptPeerFrame( + rewrites + ? restored + : sessionCollectionKeys.map((key) => [`collection`, key, absent]), + keeps, + ) + } + + private acceptPeerFrame( + writes: ReadonlyArray<[ObjectStore, string | number, ModelValue]>, + truncateKeeps?: ReadonlySet, + ): void { + this.frames.push({ status: `accepted`, innerCommitted: false }) + const frame = this.frames.length - 1 + const truncates = truncateKeeps !== undefined + if (truncates) + this.events.push({ + frame, + seq: this.seq++, + truncate: true, + keeps: truncateKeeps, + }) + for (const [store, key, value] of writes) + this.events.push({ + frame, + seq: this.seq++, + truncate: false, + store, + key, + value, + // A peer's row is whole: this tab reads it from storage or from the + // notice, which carries the stored row. + ...(store === `row` && value.present + ? { rowWrite: `insert` as const, kind: `insert` as const } + : {}), + }) + this.accepted.push({ frame }) + // A transaction that truncates applies at once, as a commit's does. + if (!this.holding || truncates) this.applyAccepted() + this.invalidateOpen() + } + + /** + * Whether a `DbClient` seed holds this row: one was accepted and no source + * write in this run, this tab's or a peer's named one, has written the + * row or truncated. A seed is older than every source write (D4). + */ + holdsDbClientSeed(key: SessionRowKey): boolean { + const entries = this.acceptedEntries() + return ( + entries.some( + (entry) => + `seed` in entry && + !entry.seed.cached && + entry.seed.key === key && + entry.seed.lost?.row !== true, + ) && + !entries.some( + (entry) => + `event` in entry && + (entry.event.truncate + ? truncatesRow(entry.event, key) + : entry.event.store === `row` && entry.event.key === key), + ) + ) + } + + /** Whether an open transaction has inserted this row. */ + hasOpenInsert(key: SessionRowKey): boolean { + const open = new Set(this.openStack) + return this.events.some( + (event) => open.has(event.frame) && isInsert(event) && event.key === key, + ) + } + + /** The `v` of the row this tab has accepted, if the row has one. */ + acceptedValue(key: SessionRowKey): { value: unknown } | undefined { + const row = this.fold(`row`, key, this.acceptedEntries()) + return row.present && + typeof row.value === `object` && + row.value !== null && + `v` in row.value + ? { value: (row.value as { v: unknown }).v } + : undefined + } + + /** The row storage holds; peer actions that name a stored row need one. */ + storedRow(key: SessionRowKey): ModelValue { + return this.storage.get(key)?.row ?? absent + } + + /** Each row as storage holds it, which a restart would read. */ + storedRows(): Record { + return Object.fromEntries( + sessionRowKeys.map((key) => [key, encode(this.storedRow(key))]), + ) + } + + /** + * A seed arrives after every accepted transaction but beneath the open + * transactions, whose writes have not yet committed. + */ + seed( + key: SessionRowKey, + row: unknown, + metadata: ModelValue, + cached = false, + ): void { + this.accepted.push({ + seed: { key, row, metadata, applied: false, seq: this.seq++, cached }, + }) + if (cached) + this.storage.set(key, { row: { present: true, value: row }, metadata }) + if (!this.holding) this.applyAccepted() + this.invalidateOpen() + } + + /** + * An open transaction whose insert now lands on a different row that the + * source wrote is invalid, and stays invalid. Where inserts replace, the + * transaction stays valid and the insert replaces that row. + */ + private invalidateOpen(): void { + for (const frame of this.openStack) { + if (this.frames[frame]!.status !== `open`) continue + if (this.insertsReplace) { + const entries = this.sessionEntries() + if ( + entries.some( + (entry, index) => + `event` in entry && + entry.event.frame === frame && + isInsert(entry.event) && + this.insertKind(entries, index) === `duplicate`, + ) + ) + this.replacedConflicts++ + continue + } + const entries = this.sessionEntries() + const duplicates = entries.some( + (entry, index) => + `event` in entry && + entry.event.frame === frame && + isInsert(entry.event) && + this.insertKind(entries, index) === `duplicate`, + ) + if (duplicates) { + this.frames[frame]!.status = `invalid` + this.invalidations++ + } + } + } + + /** Whether an open transaction has updated this row. */ + hasOpenRowUpdate(key: SessionRowKey): boolean { + const open = new Set(this.openStack) + return this.events.some( + (event) => + open.has(event.frame) && + !event.truncate && + event.rowWrite === `update` && + event.key === key, + ) + } + + /** Whether a surviving source insert wrote this row. */ + hasSourceInsert(key: SessionRowKey): boolean { + return this.sessionEntries().some( + (entry) => + `event` in entry && + !entry.event.truncate && + entry.event.rowWrite === `insert` && + entry.event.key === key, + ) + } + + /** + * Whether an accepted source insert kept this row's metadata, and no source + * write has named it since: an equal re-insert keeps metadata as an update + * does, without writing it. A seed's metadata then still governs the row. + */ + keptByAcceptedReinsert(key: SessionRowKey): boolean { + const entries = this.sessionEntries() + return ( + !this.sourceWroteBefore(`rowMetadata`, key, entries, entries.length) && + entries.some( + (entry) => + `event` in entry && + isInsert(entry.event) && + entry.event.key === key && + this.frames[entry.event.frame]!.status !== `open`, + ) + ) + } + + observe(): SessionObservation { + const session = this.sessionEntries() + const consumers = this.consumerEntries() + const rowMetadata: Record = {} + const base: Record = {} + for (const key of sessionRowKeys) { + rowMetadata[key] = encode(this.fold(`rowMetadata`, key, session)) + base[key] = encode(this.fold(`row`, key, consumers)) + } + const collection: Record = {} + const list: Array<[string, unknown]> = [] + for (const key of sessionCollectionKeys) { + const value = this.fold(`collection`, key, session) + // `get` returns undefined for a stored undefined; `list()` still has it. + collection[key] = + value.present && value.value !== undefined ? value.value : absentMarker + if (value.present) list.push([key, value.value]) + } + // `list()` order is not judged; drivers and the model both sort by key. + list.sort(([a], [b]) => a.localeCompare(b)) + return { + rowMetadata, + collection, + list, + prefixList: list.filter(([key]) => key.startsWith(sessionPrefix)), + base, + // A receipt settles when its transaction applies (`SyncConfig.commit`), + // and an invalid transaction's commit rejects. + receipts: this.committed.map((frame) => { + const status = this.frames[frame]!.status + return status === `applied` + ? `applied` + : status === `invalid` || status === `aborted` + ? `rejected` + : `pending` + }), + } + } + + private innermost(): number { + return this.openStack.at(-1)! + } + + private truncates(frame: number): boolean { + return this.events.some((event) => event.truncate && event.frame === frame) + } + + private applyAccepted(): void { + for (const item of this.accepted) { + if (`frame` in item) this.frames[item.frame]!.status = `applied` + else item.seed.applied = true + } + } + + private frameEntries(frame: number): Array { + return this.events + .filter((event) => event.frame === frame) + .map((event) => ({ event })) + } + + /** The source's serialization order: accepted work, then open transactions. */ + private sessionEntries(): Array { + return [ + ...this.acceptedEntries(), + ...this.openStack + .filter((frame) => this.frames[frame]!.status === `open`) + .flatMap((frame) => this.frameEntries(frame)), + ] + } + + /** Accepted transactions and seeds that this tab sees, in order. */ + private acceptedEntries(): Array { + return this.accepted.flatMap((item) => + `frame` in item ? this.frameEntries(item.frame) : [item], + ) + } + + /** Consumers' order: applied transactions and seeds only. */ + private consumerEntries(): Array { + return this.accepted.flatMap((item) => + `frame` in item + ? this.frames[item.frame]!.status === `applied` + ? this.frameEntries(item.frame) + : [] + : item.seed.applied + ? [item] + : [], + ) + } + + /** + * Folds one object newest first. A seed counts only if no source write to + * the object, or truncate, comes before it in the order. + */ + private fold( + store: ObjectStore, + key: string | number, + entries: ReadonlyArray, + end = entries.length, + ): ModelValue { + for (let index = end - 1; index >= 0; index--) { + const entry = entries[index]! + if (`seed` in entry) { + const seed = entry.seed + if ( + store === `collection` || + seed.key !== key || + seed.lost?.[store] === true || + this.sourceWroteBefore(store, key, entries, index) + ) + continue + return store === `row` + ? { present: true, value: seed.row } + : seed.metadata + } + const event = entry.event + if (event.truncate) { + if (store !== `collection` && truncatesRow(event, key)) return absent + continue + } + if (store === `rowMetadata` && this.clearsMetadata(entries, index, key)) + return absent + if (event.store !== store || event.key !== key) continue + if ( + !this.fullRows && + event.rowWrite === `update` && + event.value.present + ) { + const before = this.fold(`row`, key, entries, index) + if (before.present) + return { + present: true, + value: { + ...(before.value as object), + ...(event.value.value as object), + }, + } + } + return event.value + } + return absent + } + + /** Whether a source write to the object comes before `end` in the order. */ + private sourceWroteBefore( + store: ObjectStore, + key: string | number, + entries: ReadonlyArray, + end: number, + ): boolean { + return entries + .slice(0, end) + .some( + (entry, index) => + `event` in entry && + (entry.event.truncate + ? truncatesRow(entry.event, key) + : (entry.event.store === store && entry.event.key === key) || + (store === `rowMetadata` && + this.clearsMetadata(entries, index, key))), + ) + } + + /** Whether the entry at `index` is an insert that clears `key`'s metadata. */ + private clearsMetadata( + entries: ReadonlyArray, + index: number, + key: string | number, + ): boolean { + const entry = entries[index]! + return ( + `event` in entry && + isInsert(entry.event) && + entry.event.key === key && + entry.event.clears === true && + this.insertKind(entries, index) !== `reinsert` + ) + } + + /** + * An insert's kind against the row before it in the order. A seed + * accepted after the insert was written counts only beneath a source write + * to the row that a transaction committed after it, as that write's merge + * base. An open transaction's own writes over the seed do not count. + */ + private insertKind( + entries: ReadonlyArray, + index: number, + ): InsertKind { + const event = (entries[index] as { event: ModelWrite }).event + if (event.kind) return event.kind + const prefix = entries.slice(0, index) + const before = prefix.filter( + (entry, position) => + !(`seed` in entry) || + entry.seed.seq < event.seq || + prefix + .slice(position + 1) + .some( + (later) => + `event` in later && + !later.event.truncate && + later.event.store === `row` && + later.event.key === event.key && + this.frames[later.event.frame]!.status !== `open`, + ), + ) + const row = this.fold(`row`, event.key, before) + if ( + !row.present || + !this.sourceWroteBefore(`row`, event.key, before, before.length) + ) + return `insert` + return canonical(row.value) === + canonical((event.value as { value: unknown }).value) + ? `reinsert` + : `duplicate` + } +} + +function isInsert(event: ModelEvent): event is ModelWrite { + return !event.truncate && event.rowWrite === `insert` +} + +function encode(value: ModelValue): unknown { + return value.present ? value.value : absentMarker +} + +// --------------------------------------------------------------------------- +// Production driver contract +// --------------------------------------------------------------------------- + +/** + * A driver exercises one production path. It performs each action through + * the public sync API and records the observation at each checkpoint. A + * thrown error or an unexpected receipt rejection is an observation too, so + * the comparison can report it as a law violation rather than a crash. + */ +export type SessionDriver = { + begin: () => void + commit: () => void + cancel: () => void + rowSet: (key: SessionRowKey, value: unknown) => void + rowDelete: (key: SessionRowKey) => void + insert: (key: SessionRowKey, value: unknown, metadata: unknown) => void + update: (key: SessionRowKey, value: unknown, metadata: unknown) => void + delete: (key: SessionRowKey) => void + truncate: () => void + collectionSet: (key: SessionCollectionKey, value: unknown) => void + collectionDelete: (key: SessionCollectionKey) => void + /** + * `store` holds a wrapping sync's next durable step instead of core; + * `read` keeps a wrapping sync's storage read in flight. + */ + hold: (store: boolean, read?: boolean) => void + release: () => Promise + seed?: (key: SessionRowKey, row: unknown, metadata: unknown) => void + /** + * Another tab's source writes `row` and `metadata` (or deletes the row) + * in storage and sends its notice; resolves once this tab has handled it. + */ + peer?: ( + key: SessionRowKey, + notice: PeerNotice, + row: unknown, + metadata: unknown, + rewrites: boolean, + ) => Promise + /** Whether a thrown error is core's duplicate-key rejection. */ + isDuplicateKeyError?: (error: unknown) => boolean + /** + * Aborts the signal of the newest outermost transaction committed during + * the read hold. Lanes with read holds use it. + */ + abandon?: () => void + observe: () => SessionObservation + /** + * Each row as storage holds it. Where a lane models storage, the settled + * rows must match the model's: a restart reads them, and a full reload + * reads them back into this tab. + */ + storedRows?: () => Record + /** Lets receipt continuations run before an observation. */ + flush: () => Promise + /** Ends any hold and resolves every receipt; returns unexpected rejections. */ + settle: () => Promise> + /** + * When consumers and receipts are compared. `each-step` (the default) + * requires them at every checkpoint. `final` compares them only after + * `settle`, for a path that applies asynchronously after its own durable + * step; source reads are still compared at every checkpoint. + */ + consumerCheckpoint?: `each-step` | `final` + cleanup: () => Promise +} + +export type SessionMismatch = { + step: number + action: SessionAction | `settle` + law: string + expected: unknown + actual: unknown +} + +// --------------------------------------------------------------------------- +// Refinement check +// --------------------------------------------------------------------------- + +/** + * The runner applies each legal action to the model and the driver, then + * compares the full observation at the checkpoint after the action returns. + * Illegal actions for the current model state are skipped, so every + * generated history is legal: `commit`, `cancel`, and writes need an open + * transaction; `begin` respects the lane's depth; `hold` and `release` + * alternate; `remove` and `reinsert` need the row. A seed is legal at any + * point, including over a row the source has inserted. + * + * An inner transaction may write a row that a transaction it began inside + * has written, and may truncate. Committing first puts the inner write ahead + * of the outer one, so a source update can land on a row that is absent in + * commit order; it then writes its own fields. The outer insert's kind can + * change, or its transaction become invalid (see the model). + * + * Implicit row metadata follows the 2026-10-05 rules. An insert names the + * row's whole metadata, so one without `metadata` clears it; an update + * without `metadata` keeps it; a delete clears it. An `upsert` writes a + * fresh value, so only `reinsert` is an equal re-insert. A source insert + * replaces a seeded row, and `reinsert` over a row that only a seed supplied + * is an insert. + */ +export async function runSessionHistory( + lane: SessionLane, + history: ReadonlyArray, + createDriver: () => Promise, +): Promise<{ mismatch: SessionMismatch | undefined; steps: number }> { + const model = createSessionModel(lane) + const driver = await createDriver() + let mismatch: SessionMismatch | undefined + let steps = 0 + let primary: unknown + try { + for (const [step, action] of history.entries()) { + if (!isLegal(lane, model, action)) continue + steps++ + const value = { s: step } + let thrown: string | undefined + let lawError: string | undefined + try { + await planAction(model, action, value)(driver) + } catch (error) { + thrown = describeError(error) + if (error instanceof SessionLawError) lawError = error.law + } + if (thrown !== undefined) { + mismatch = { + step, + action, + law: lawError ?? `legal action threw`, + expected: lawError ? `a duplicate-key error` : `no error`, + actual: lawError ? `no error` : thrown, + } + break + } + await driver.flush() + mismatch = compare( + step, + action, + model.observe(), + driver.observe(), + driver.consumerCheckpoint === `final`, + ) + if (mismatch) break + } + if (!mismatch) { + const rejections = await driver.settle() + if (model.isHolding) model.release() + while (model.depth > 0) model.cancel() + if (rejections.length > 0) + mismatch = { + step: history.length, + action: `settle`, + law: `accepted transaction applies`, + expected: [], + actual: rejections, + } + else + mismatch = compare( + history.length, + `settle`, + model.observe(), + driver.observe(), + ) + if (!mismatch && lane.peers === true && driver.storedRows) { + const expected = model.storedRows() + const actual = driver.storedRows() + if (canonical(expected) !== canonical(actual)) + mismatch = { + step: history.length, + action: `settle`, + law: `storage holds the rows this tab shows once they are durable`, + expected, + actual, + } + } + } + } catch (error) { + primary = error + } + let cleanupFailure: unknown + try { + await driver.cleanup() + } catch (error) { + cleanupFailure = error ?? new Error(`cleanup failed`) + } + if (cleanupFailure !== undefined) { + // A cleanup failure must not hide the oracle result it follows. + const first = + primary ?? + (mismatch ? new Error(describeMismatch(history, mismatch)) : undefined) + if (first === undefined) throw cleanupFailure + throw new AggregateError( + [first, cleanupFailure], + `oracle run and cleanup failed`, + { + cause: first, + }, + ) + } + if (primary !== undefined) throw primary + return { mismatch, steps } +} + +/** + * A second formulation of hold invisibility, independent of the model. Two + * production runs execute the same history, one with every `hold` and + * `release` removed. The source's reads must match at each checkpoint of an + * action both runs perform: a hold may change when consumers see writes, + * never what the source reads. The model decides only legality, and removing + * holds does not change which other actions are legal. + */ +export async function runHoldInvariance( + lane: SessionLane, + history: ReadonlyArray, + createDriver: () => Promise, +): Promise { + const held = await createDriver() + const unheld = await createDriver() + const model = createSessionModel(lane) + let mismatch: SessionMismatch | undefined + try { + for (const [step, action] of history.entries()) { + if (!isLegal(lane, model, action)) continue + const value = { s: step } + let thrown: string | undefined + try { + const call = planAction(model, action, value) + await call(held) + if (action.type !== `hold` && action.type !== `release`) + await call(unheld) + } catch (error) { + thrown = describeError(error) + } + if (thrown !== undefined) { + mismatch = { + step, + action, + law: `legal action threw`, + expected: `no error`, + actual: thrown, + } + break + } + if (action.type === `hold` || action.type === `release`) continue + await held.flush() + await unheld.flush() + mismatch = compareSource(step, action, unheld.observe(), held.observe()) + if (mismatch) { + mismatch.law = `a hold does not change source reads (${mismatch.law})` + break + } + } + } finally { + await Promise.allSettled([held.settle(), unheld.settle()]) + await Promise.allSettled([held.cleanup(), unheld.cleanup()]) + } + return mismatch +} + +type PlannedCall = { step: number; action: SessionAction; call: DriverCall } + +/** The legal actions of a history and their driver calls, planned once. */ +function planHistory( + lane: SessionLane, + history: ReadonlyArray, +): Array { + const model = createSessionModel(lane) + const planned: Array = [] + for (const [step, action] of history.entries()) { + if (!isLegal(lane, model, action)) continue + planned.push({ step, action, call: planAction(model, action, { s: step }) }) + } + return planned +} + +async function performCall( + driver: SessionDriver, + { step, action, call }: PlannedCall, +): Promise { + try { + await call(driver) + } catch (error) { + return { + step, + action, + law: `legal action threw`, + expected: `no error`, + actual: describeError(error), + } + } + await driver.flush() + return undefined +} + +const otherRowKey: SessionRowKey = 2 + +/** Whether an action names row 2's data or metadata. */ +function namesOtherRow(action: SessionAction): boolean { + return `key` in action && action.key === otherRowKey +} + +/** Row 1, the collection metadata, and receipts: what row 2 must not change. */ +function withoutOtherRow(observation: SessionObservation): SessionObservation { + return { + ...observation, + rowMetadata: { 1: observation.rowMetadata[1] }, + base: { 1: observation.base[1] }, + } +} + +/** + * A second formulation of per-object independence (read your writes holds + * per object). Two production runs execute the same legal history, one with + * every action that names row 2 removed. Row 1's reads, the collection + * metadata, consumers' view of row 1, and receipts must match at each + * checkpoint of an action both runs perform. Row 2's actions never change + * which other actions are legal; `truncate` names every row and stays. + * Invalidation is per transaction: a row 2 insert that lands on a different + * row makes its whole transaction invalid, row 1's writes included. Histories + * where the model invalidates a transaction are therefore not compared. + */ +export async function runKeyIndependence( + lane: SessionLane, + history: ReadonlyArray, + createDriver: () => Promise, +): Promise { + if (historyRegimes(lane, history).has(`transaction invalidated`)) + return undefined + const both = await createDriver() + const rowOneOnly = await createDriver() + let mismatch: SessionMismatch | undefined + try { + for (const planned of planHistory(lane, history)) { + mismatch = await performCall(both, planned) + if (mismatch) break + if (namesOtherRow(planned.action)) continue + mismatch = await performCall(rowOneOnly, planned) + if (mismatch) break + mismatch = compare( + planned.step, + planned.action, + withoutOtherRow(rowOneOnly.observe()), + withoutOtherRow(both.observe()), + ) + if (mismatch) { + mismatch.law = `row 2's writes do not change row 1 (${mismatch.law})` + break + } + } + } finally { + await Promise.allSettled([both.settle(), rowOneOnly.settle()]) + await Promise.allSettled([both.cleanup(), rowOneOnly.cleanup()]) + } + return mismatch +} + +/** + * A second formulation of decisions D4 and D5: a seed is older than every + * source write, so its arrival time cannot change the outcome. Two + * production runs perform the same driver calls; the second performs every + * seed first, in their original order. The source's calls stay what they + * were, so a source that inserted a row before its seed arrived still calls + * `insert`. After both runs settle, every observation must match. The model + * plans the calls with seeds beneath every source write, and a source insert + * replaces a seeded row, so an equal re-insert planned in arrival order + * stays equal when the seed moves first. Under partial row updates a seed + * that arrives after a source update is ignored, while a seed moved first is + * the update's merge base, so consumers' rows are compared only under full + * row updates; metadata is compared in every lane. For the same reason a + * re-insert planned from the arrival-order row need not equal the row when + * the seed moves first (decision "b"), so partial lanes drop `reinsert` from + * both runs. + */ +export async function runSeedOrderInvariance( + lane: SessionLane, + history: ReadonlyArray, + createDriver: () => Promise, +): Promise { + const asArrived = await createDriver() + const seedsFirst = await createDriver() + const planned = planHistory( + lane, + lane.fullRows + ? history + : history.filter((action) => action.type !== `reinsert`), + ) + const isSeed = ({ action }: PlannedCall) => action.type === `seed` + let mismatch: SessionMismatch | undefined + try { + for (const next of planned) { + mismatch = await performCall(asArrived, next) + if (mismatch) break + } + if (!mismatch) + for (const next of [ + ...planned.filter(isSeed), + ...planned.filter((other) => !isSeed(other)), + ]) { + mismatch = await performCall(seedsFirst, next) + if (mismatch) break + } + if (!mismatch) { + await Promise.all([asArrived.settle(), seedsFirst.settle()]) + const comparable = (observation: SessionObservation) => + lane.fullRows ? observation : { ...observation, base: {} } + mismatch = compare( + history.length, + `settle`, + comparable(seedsFirst.observe()), + comparable(asArrived.observe()), + ) + if (mismatch) + mismatch.law = `a seed's arrival time does not change the outcome (${mismatch.law})` + } + } finally { + await Promise.allSettled([asArrived.settle(), seedsFirst.settle()]) + await Promise.allSettled([asArrived.cleanup(), seedsFirst.cleanup()]) + } + return mismatch +} + +const modelOnly: SessionDriver = { + begin: () => {}, + commit: () => {}, + abandon: () => {}, + cancel: () => {}, + rowSet: () => {}, + rowDelete: () => {}, + insert: () => {}, + update: () => {}, + delete: () => {}, + truncate: () => {}, + collectionSet: () => {}, + collectionDelete: () => {}, + hold: () => {}, + release: () => Promise.resolve(), + seed: () => {}, + peer: () => Promise.resolve(), + observe: () => new SessionModel().observe(), + flush: () => Promise.resolve(), + settle: () => Promise.resolve([]), + cleanup: () => Promise.resolve(), +} + +/** + * The model's observation at each checkpoint of a history, keyed by the step + * index of each legal action. Witness tests use it to check the model against + * expectations derived by hand from the laws. + */ +export async function modelCheckpoints( + lane: SessionLane, + history: ReadonlyArray, +): Promise> { + const model = createSessionModel(lane) + const checkpoints = new Map() + for (const [step, action] of history.entries()) { + if (!isLegal(lane, model, action)) continue + await planAction(model, action, { s: step })(modelOnly) + checkpoints.set(step, model.observe()) + } + return checkpoints +} + +function isLegal( + lane: SessionLane, + model: SessionModel, + action: SessionAction, +): boolean { + // An abandoned commit's read ends next. + if (model.awaitsAbandonment && action.type !== `release`) return false + switch (action.type) { + case `begin`: + return model.depth < lane.maxDepth + case `hold`: + return !model.isHolding + case `release`: + return model.isHolding + case `abandon`: + return lane.readHold === true && model.canAbandon + case `seed`: + return lane.seeds + case `peer`: + return ( + lane.peers === true && + !model.blocksPeerNotices && + (action.notice === `row` || + action.notice === `reload` || + action.notice === `reset` || + model.storedRow(action.key).present) + ) + case `insertOverSeed`: + return ( + (lane.seeds || lane.cached === true) && + model.depth > 0 && + model.sourceRow(action.key).present && + !model.sourceWroteRow(action.key) + ) + case `conflictingInsert`: + return ( + model.depth > 0 && + !model.innermostInvalid && + model.sourceWroteRow(action.key) + ) + case `remove`: + case `reinsert`: + return model.depth > 0 && model.sourceRow(action.key).present + default: + return model.depth > 0 + } +} + +type DriverCall = (driver: SessionDriver) => void | Promise + +/** + * Advances the model by one legal action and returns the matching driver + * call. The model decides once; the call can be replayed on several drivers. + */ +function planAction( + model: SessionModel, + action: SessionAction, + value: unknown, +): DriverCall { + switch (action.type) { + case `begin`: + model.begin() + return (driver) => driver.begin() + case `commit`: + model.commit() + return (driver) => driver.commit() + case `cancel`: + model.cancel() + return (driver) => driver.cancel() + case `abandon`: + model.abandon() + return (driver) => driver.abandon!() + case `rowSet`: + model.write(`rowMetadata`, action.key, { present: true, value }) + return (driver) => driver.rowSet(action.key, value) + case `rowDelete`: + model.write(`rowMetadata`, action.key, absent) + return (driver) => driver.rowDelete(action.key) + case `upsert`: { + // A row written with metadata also carries field `m`, so rows differ in + // shape and a full update is distinguishable from a merge. + const row = action.metadata + ? { id: action.key, v: value, m: value } + : { id: action.key, v: value } + const metadata = action.metadata ? value : undefined + const present = model.sourceRow(action.key).present + model.writeRow( + action.key, + row, + present ? `update` : `insert`, + metadata === undefined, + ) + if (metadata !== undefined) + model.write(`rowMetadata`, action.key, { + present: true, + value: metadata, + }) + return (driver) => + present + ? driver.update(action.key, row, metadata) + : driver.insert(action.key, row, metadata) + } + case `reinsert`: { + // An insert equal to a row the source itself wrote is an equal + // re-insert: without metadata it keeps the current value, like an + // update. Over a row that only a seed supplied it is an insert. + const current = model.sourceRow(action.key) + if (!current.present) throw new Error(`reinsert needs a present row`) + const row = structuredClone(current.value) + const metadata = action.metadata ? value : undefined + model.writeRow(action.key, row, `insert`, metadata === undefined) + if (metadata !== undefined) + model.write(`rowMetadata`, action.key, { + present: true, + value: metadata, + }) + return (driver) => driver.insert(action.key, row, metadata) + } + case `remove`: + model.write(`row`, action.key, absent) + model.write(`rowMetadata`, action.key, absent) + return (driver) => driver.delete(action.key) + case `truncate`: + model.truncate() + return (driver) => driver.truncate() + case `collectionSet`: { + const stored = action.undefinedValue === true ? undefined : value + model.write(`collection`, action.key, { present: true, value: stored }) + return (driver) => driver.collectionSet(action.key, stored) + } + case `collectionDelete`: + model.write(`collection`, action.key, absent) + return (driver) => driver.collectionDelete(action.key) + case `hold`: + model.hold( + action.store === true || action.read === true, + action.read === true, + ) + return (driver) => + driver.hold(action.store === true, action.read === true) + case `release`: + model.release() + return (driver) => driver.release() + case `seed`: { + // A seed usually carries a field the source never writes, so a source + // update that merges over it keeps it. A narrow seed omits the field, + // so a later seed of the same key shows whether it replaces the row. + // A bare seed carries no metadata, so a later seed of the same key + // shows whether the latest seed wins metadata too. + const row = + action.narrow === true + ? { id: action.key, v: value } + : { id: action.key, v: value, seeded: value } + const metadata = action.bare === true ? undefined : value + model.seed( + action.key, + row, + metadata === undefined ? absent : { present: true, value: metadata }, + ) + return (driver) => driver.seed!(action.key, row, metadata) + } + case `conflictingInsert`: { + // A fresh insert over a row the source wrote. Core rejects it with a + // duplicate-key error and records nothing; where inserts replace, it + // replaces the row and, without metadata, clears its metadata. + const row = action.metadata + ? { id: action.key, v: value, m: value } + : { id: action.key, v: value } + const metadata = action.metadata ? value : undefined + if (!model.insertsReplaceRows) + return async (driver) => { + // The model alone records nothing for a rejected insert. + if (driver === modelOnly) return + try { + await driver.insert(action.key, row, metadata) + } catch (error) { + if (driver.isDuplicateKeyError?.(error) === true) return + throw error + } + throw new SessionLawError( + `a source insert over a different source row is rejected`, + ) + } + model.writeRow(action.key, row, `insert`, metadata === undefined) + if (metadata !== undefined) + model.write(`rowMetadata`, action.key, { + present: true, + value: metadata, + }) + return (driver) => driver.insert(action.key, row, metadata) + } + case `peer`: { + const kept = + action.keepsValue === true ? model.acceptedValue(action.key) : undefined + const rowValue = kept ? kept.value : value + model.peer( + action.key, + action.notice, + value, + rowValue, + action.rewrites === true, + ) + return (driver) => + driver.peer!( + action.key, + action.notice, + { id: action.key, v: rowValue }, + { peer: value }, + action.rewrites === true, + ) + } + case `insertOverSeed`: { + // A fresh source insert over a row only a seed supplied replaces it. + const row = action.metadata + ? { id: action.key, v: value, m: value } + : { id: action.key, v: value } + const metadata = action.metadata ? value : undefined + model.writeRow(action.key, row, `insert`, metadata === undefined) + if (metadata !== undefined) + model.write(`rowMetadata`, action.key, { + present: true, + value: metadata, + }) + return (driver) => driver.insert(action.key, row, metadata) + } + } +} + +/** + * Each observation field is one public promise. The comparison names the + * law that the first differing field protects. + */ +function compare( + step: number, + action: SessionAction | `settle`, + expected: SessionObservation, + actual: SessionObservation, + sourceOnly = false, +): SessionMismatch | undefined { + const fields: Array<[keyof SessionObservation, string]> = [ + [`rowMetadata`, `source reads its row metadata (metadata.row.get)`], + [ + `collection`, + `source reads its collection metadata (metadata.collection.get)`, + ], + [`list`, `source lists its collection metadata (metadata.collection.list)`], + [`prefixList`, `source lists a prefix (metadata.collection.list(prefix))`], + [`base`, `consumers see whole applied transactions only (collection.base)`], + [`receipts`, `a receipt settles when its writes become visible`], + ] + for (const [field, law] of fields) { + if (sourceOnly && (field === `base` || field === `receipts`)) continue + if (canonical(expected[field]) !== canonical(actual[field])) + return { + step, + action, + law, + expected: expected[field], + actual: actual[field], + } + } + return undefined +} + +/** JSON with sorted object keys, so field order is not an observation. */ +function canonical(value: unknown): string { + return JSON.stringify(value, (_key, item: unknown) => + item !== null && typeof item === `object` && !Array.isArray(item) + ? Object.fromEntries( + Object.entries(item as Record).sort(([a], [b]) => + a.localeCompare(b), + ), + ) + : item, + ) +} + +function compareSource( + step: number, + action: SessionAction, + expected: SessionObservation, + actual: SessionObservation, +): SessionMismatch | undefined { + return compare(step, action, expected, actual, true) +} + +/** A driver call that observed a production result the law forbids. */ +class SessionLawError extends Error { + constructor(readonly law: string) { + super(law) + } +} + +function describeError(error: unknown): string { + return error instanceof Error + ? `${error.name}: ${error.message}` + : String(error) +} + +export function describeMismatch( + history: ReadonlyArray, + mismatch: SessionMismatch, +): string { + const trace = history + .slice( + 0, + typeof mismatch.action === `string` ? history.length : mismatch.step + 1, + ) + .map((action) => + action.type === `seed` + ? `seed(${[action.key, action.narrow === true ? `narrow` : ``, action.bare === true ? `bare` : ``].filter(Boolean).join(`,`)})` + : action.type === `peer` + ? `peer(${action.key},${action.notice}${action.keepsValue === true ? `,keeps` : ``})` + : `key` in action + ? `${action.type}(${action.key})` + : action.type === `hold` && action.store === true + ? `hold(store)` + : action.type, + ) + .join(` `) + return [ + `law: ${mismatch.law}`, + `at step ${mismatch.step} (${typeof mismatch.action === `string` ? mismatch.action : mismatch.action.type})`, + `history: ${trace}`, + `expected: ${JSON.stringify(mismatch.expected)}`, + `actual: ${JSON.stringify(mismatch.actual)}`, + ].join(`\n`) +} + +// The check runs a history through a driver and compares each observation +// with the model; the model enacts the session laws; the grammar generates +// legal histories and names the regimes they reach. +declareOracle({ + check: [ + runSessionHistory, + runHoldInvariance, + runKeyIndependence, + runSeedOrderInvariance, + ], + model: [SessionModel, createSessionModel, modelCheckpoints], + grammar: [sessionActionArbitrary, sessionHistoryArbitrary, historyRegimes], +}) diff --git a/packages/db/tests/sync-metadata-session.test.ts b/packages/db/tests/sync-metadata-session.test.ts new file mode 100644 index 0000000000..d8bf30bc81 --- /dev/null +++ b/packages/db/tests/sync-metadata-session.test.ts @@ -0,0 +1,112 @@ +import { describe, expect, it } from 'vitest' +import { DbClient, collectionOptions } from '../src/client.js' +import { createCollection } from '../src/collection/index.js' +import type { SyncConfig } from '../src/types.js' + +// Fixed boundaries of the sync metadata session. The session law itself is +// owned by sync-metadata-session-oracle.ts and its generated histories. + +type Row = { id: number; v: unknown } +type SyncActions = Parameters[`sync`]>[0] + +/** + * Boundary witnesses for P11. A sync run that cleanup has ended reads nothing + * and ignores writes; a metadata write with no open transaction throws. + */ +describe(`sync metadata session boundaries`, () => { + it(`a stale sync run reads nothing and ignores writes`, async () => { + let sync!: SyncActions + const collection = createCollection({ + id: `sync-metadata-session-stale`, + getKey: (row) => row.id, + startSync: true, + sync: { + sync: (actions) => { + sync = actions + actions.begin() + actions.metadata!.row.set(1, { m: 1 }) + actions.metadata!.collection.set(`cursor`, 1) + actions.commit() + actions.markReady() + }, + }, + }) + await collection.preload() + expect(sync.metadata!.collection.get(`cursor`)).toBe(1) + const stale = sync + await collection.cleanup() + expect(stale.metadata!.row.get(1)).toBeUndefined() + expect(stale.metadata!.collection.get(`cursor`)).toBeUndefined() + expect(stale.metadata!.collection.list()).toEqual([]) + expect(() => stale.metadata!.collection.set(`cursor`, 2)).not.toThrow() + }) + + // Cleanup abandons a held seed; its receipt is internal, so the abandonment + // must not surface as an unhandled rejection. + it(`cleanup with a held seed leaves no unhandled rejection`, async () => { + const unhandled: Array = [] + const listener = (reason: unknown) => unhandled.push(reason) + process.on(`unhandledRejection`, listener) + let release!: () => void + const client = new DbClient() + const collection = client.collection( + collectionOptions({ + id: `sync-metadata-session-held-seed-cleanup`, + getKey: (row) => row.id, + sync: { + sync: (actions) => { + actions.begin() + actions.commit() + actions.markReady() + }, + }, + onInsert: () => + new Promise((resolve) => { + release = resolve + }), + }), + ) + try { + await collection.preload() + collection.insert({ id: 1000, v: 0 }) + client.hydrate({ + collections: [ + { + collectionId: `sync-metadata-session-held-seed-cleanup`, + rows: [{ key: 1, value: { id: 1, v: 1 }, metadata: { m: 1 } }], + }, + ], + }) + await collection.cleanup() + release() + await new Promise((resolve) => setTimeout(resolve, 10)) + } finally { + process.off(`unhandledRejection`, listener) + } + expect(unhandled).toEqual([]) + }) + + it(`a metadata write with no open transaction throws`, async () => { + let sync!: SyncActions + const collection = createCollection({ + id: `sync-metadata-session-no-transaction`, + getKey: (row) => row.id, + startSync: true, + sync: { + sync: (actions) => { + sync = actions + actions.begin() + actions.commit() + actions.markReady() + }, + }, + }) + try { + await collection.preload() + expect(() => sync.metadata!.row.set(1, { m: 1 })).toThrow() + expect(() => sync.metadata!.collection.set(`cursor`, 1)).toThrow() + } finally { + await collection.cleanup() + } + }) +}) diff --git a/packages/db/tests/sync-write-row-replacement.test.ts b/packages/db/tests/sync-write-row-replacement.test.ts new file mode 100644 index 0000000000..c5e36b014a --- /dev/null +++ b/packages/db/tests/sync-write-row-replacement.test.ts @@ -0,0 +1,35 @@ +import { expect, it } from 'vitest' +import { createCollection } from '../src/collection/index.js' +import type { SyncConfig } from '../src/types.js' + +type Row = { id: number; a?: number; b?: number } + +// A partial update merges over the row before it. Only an internal marker, +// which the persisted wrapper sets on an insert it judged itself, makes a +// sync write name the whole row; a public message field cannot. +it(`merges a public partial update whatever fields the message carries`, async () => { + let sync!: Parameters[`sync`]>[0] + const collection = createCollection({ + id: `public-update-merges`, + getKey: (row) => row.id, + startSync: true, + sync: { + sync: (actions) => { + sync = actions + actions.begin() + actions.write({ type: `insert`, value: { id: 1, a: 1, b: 1 } }) + actions.commit() + actions.markReady() + }, + }, + }) + await collection.preload() + sync.begin() + sync.write({ + type: `update`, + value: { id: 1, a: 2 }, + originalSyncType: `insert`, + } as Parameters[0]) + sync.commit() + expect(collection.get(1)).toMatchObject({ id: 1, a: 2, b: 1 }) +}) diff --git a/packages/electric-db-collection/tests/electric-adapter.test.ts b/packages/electric-db-collection/tests/electric-adapter.test.ts index 47aabad576..717eeef741 100644 --- a/packages/electric-db-collection/tests/electric-adapter.test.ts +++ b/packages/electric-db-collection/tests/electric-adapter.test.ts @@ -352,9 +352,11 @@ describe(`Electric adapter laws`, () => { }).toEqual({ publicRows: [[1, `updated across callbacks`, `stable-1`]], durableRows: [], + // The source's insert is persisted as an insert, which replaces any + // cached row (decision C1 of 2026-10-23); its update merges. persistedMutations: [ { - type: `update`, + type: `insert`, key: 1, value: { id: 1, diff --git a/packages/electron-db-sqlite-persistence/tests/electron-ipc.test.ts b/packages/electron-db-sqlite-persistence/tests/electron-ipc.test.ts index e5b263f45c..38bc6924cc 100644 --- a/packages/electron-db-sqlite-persistence/tests/electron-ipc.test.ts +++ b/packages/electron-db-sqlite-persistence/tests/electron-ipc.test.ts @@ -1235,7 +1235,9 @@ describe(`electron sqlite persistence bridge`, () => { truncate: true, mutations: [ { - type: `update`, + // A source insert is persisted as an insert, which replaces any cached + // row (decision C1 of 2026-10-23). + type: `insert`, key: `fresh`, value: { id: `fresh`, title: `Durable through owner` }, }, diff --git a/packages/query-db-collection/src/query.ts b/packages/query-db-collection/src/query.ts index c570037cf0..19f4aab9c6 100644 --- a/packages/query-db-collection/src/query.ts +++ b/packages/query-db-collection/src/query.ts @@ -1522,11 +1522,10 @@ export function queryCollectionOptions( return true } - const getRowMetadata = (rowKey: string | number) => { - return (metadata?.row.get(rowKey) ?? - collection._state.syncedMetadata.get(rowKey)) as - Record | undefined - } + // The sync session's read already falls back to applied metadata, and a + // staged clear reads as undefined, so it must not fall back again. + const getRowMetadata = (rowKey: string | number) => + metadata?.row.get(rowKey) as Record | undefined const getOwnersFromMetadata = ( rowMetadata: unknown, @@ -1544,13 +1543,13 @@ export function queryCollectionOptions( const result = new Set(Object.keys(owners as Record)) if (rowKey === undefined) return result - // Resolved Queries supersede stale persisted edges; accepted in-memory - // edges supplement owners whose metadata has not published yet. + // Resolved Queries supersede stale persisted edges. An owner set that a + // held sync transaction staged is readable here, because metadata reads + // include every accepted sync transaction in commit order. result.forEach((owner) => { const rows = queryToRows.get(owner) if (rows && !rows.has(rowKey)) result.delete(owner) }) - rowToQueries.get(rowKey)?.forEach((owner) => result.add(owner)) return result } diff --git a/packages/query-db-collection/tests/ownership-lifecycle-support.ts b/packages/query-db-collection/tests/ownership-lifecycle-support.ts index b38654e946..7e3816db59 100644 --- a/packages/query-db-collection/tests/ownership-lifecycle-support.ts +++ b/packages/query-db-collection/tests/ownership-lifecycle-support.ts @@ -5,7 +5,7 @@ */ import { QueryClient, hashKey } from '@tanstack/query-core' -import { IR, createCollection } from '@tanstack/db' +import { DbClient, IR, collectionOptions, createCollection } from '@tanstack/db' import { expect, vi } from 'vitest' import { createDeferred } from '../../db/src/deferred.js' @@ -152,6 +152,99 @@ export function recordMetadata( } } +/** + * Diagnosis for a race seen only on CI: the held-cleanup retention witnesses + * sometimes see no expiry transaction staged. A fixture with a retention + * time-to-live records each timer that Query DB sets during the test, where + * it was set, whether it fired, and what cleared it, and each Query fetch. A + * failed retention wait reports that trace. + */ +type RetentionTrace = { + started: number + timers: Array<{ + delay: number + set: string + fired?: number + cleared?: string + }> + fetches: Array +} +let retentionTrace: RetentionTrace | undefined + +const queryDbCaller = () => + (new Error().stack ?? ``) + .split(`\n`) + .filter((line) => line.includes(`query-db-collection/src/`)) + .slice(0, 3) + .map((line) => line.trim().replace(/^at /, ``)) + .join(` < `) + +function traceRetentionTimers(): RetentionTrace { + const trace: RetentionTrace = { started: Date.now(), timers: [], fetches: [] } + const elapsed = () => `+${Date.now() - trace.started}ms` + const realSetTimeout = globalThis.setTimeout + const realClearTimeout = globalThis.clearTimeout + const traced = new Map() + const setSpy = vi.spyOn(globalThis, `setTimeout`).mockImplementation((( + callback: (...args: Array) => void, + delay?: number, + ...args: Array + ) => { + const caller = queryDbCaller() + if (!caller) return realSetTimeout(callback, delay, ...args) + const timer: RetentionTrace[`timers`][number] = { + delay: delay ?? 0, + set: `${elapsed()} ${caller}`, + } + trace.timers.push(timer) + const handle = realSetTimeout( + (...callbackArgs: Array) => { + timer.fired = Date.now() - trace.started + callback(...callbackArgs) + }, + delay, + ...args, + ) + traced.set(handle, timer) + return handle + }) as typeof setTimeout) + const clearSpy = vi.spyOn(globalThis, `clearTimeout`).mockImplementation((( + handle?: Parameters[0], + ) => { + const timer = traced.get(handle) + if (timer) timer.cleared = `${elapsed()} ${queryDbCaller()}` + realClearTimeout(handle) + }) as typeof clearTimeout) + retentionTrace = trace + cleanups.push(() => { + setSpy.mockRestore() + clearSpy.mockRestore() + if (retentionTrace === trace) retentionTrace = undefined + return Promise.resolve() + }) + return trace +} + +export async function waitForRetention( + check: () => void, + options: { timeout: number }, +) { + try { + await vi.waitFor(check, options) + } catch (error) { + const trace = retentionTrace + const report = trace + ? [ + `Query DB timers:`, + ...trace.timers.map((timer) => ` ${JSON.stringify(timer)}`), + `Query fetches: ${JSON.stringify(trace.fetches)}`, + ].join(`\n`) + : `no retention trace` + if (error instanceof Error) error.message = `${error.message}\n${report}` + throw error + } +} + export function createOwnershipFixture({ id, results, @@ -165,11 +258,19 @@ export function createOwnershipFixture({ persistedGcTime, }: OwnershipFixtureOptions): OwnershipFixture { const queryClient = createQueryClient(customHash, staleTime) + const trace = + persistedGcTime === undefined ? undefined : traceRetentionTimers() const queryFn = vi.fn<() => Promise>>() - results.forEach((result) => - queryFn.mockImplementationOnce(() => Promise.resolve(result)), + results.forEach((result, index) => + queryFn.mockImplementationOnce(() => { + trace?.fetches.push(`+${Date.now() - trace.started}ms result ${index}`) + return Promise.resolve(result) + }), ) - queryFn.mockRejectedValue(new Error(`Unexpected ownership refetch`)) + queryFn.mockImplementation(() => { + trace?.fetches.push(`+${Date.now() - trace.started}ms unexpected`) + return Promise.reject(new Error(`Unexpected ownership refetch`)) + }) const baseOptions = queryCollectionOptions({ id, queryClient, @@ -285,13 +386,52 @@ export function persistedOwners( return owners && typeof owners === `object` ? Object.keys(owners).sort() : [] } +// Reads a row's published metadata through the public dehydrate boundary of a +// DbClient. Owner keys are opaque, so owners are counted, not named. The seed's +// non-owner field `user` is read by name, and absence is `undefined`. +export function publishedRowMetadata( + client: DbClient, + rowId: string, +): { user: unknown; owners: number } { + const metadata = client + .dehydrate({ shouldDehydrateCollection: () => true }) + .collections.flatMap((chunk) => chunk.rows) + .find((row) => row.key === rowId)?.metadata + return { + user: (metadata as Record | undefined)?.user, + owners: persistedOwners(new Map([[rowId, metadata]]), rowId).length, + } +} + +// A DbClient over one on-demand Query collection whose every fetch returns +// `shared`. A hydration check builds a server and a browser client from it, so +// a browser release reads only the owners that the dehydrated state recorded. +export function createSharedRowClient(id: string, persistedGcTime?: number) { + const queryClient = createQueryClient() + const descriptor = collectionOptions( + queryCollectionOptions({ + id, + queryClient, + queryKey: [id], + queryFn: () => Promise.resolve([shared]), + getKey: (item) => item.id, + syncMode: `on-demand`, + persistedGcTime, + startSync: true, + }), + ) + return { client: new DbClient({ queryClient }), queryClient, descriptor } +} + export type StoredOwnership = { rows: Map rowMetadata: Map collectionMetadata: Map } -export function categorySubset(category: `detail` | `list`): LoadSubsetOptions { +export function categorySubset( + category: `detail` | `list` | `peer`, +): LoadSubsetOptions { return { where: new IR.Func(`in`, [ new IR.PropRef([`category`]), diff --git a/packages/query-db-collection/tests/ownership-lifecycle.test.ts b/packages/query-db-collection/tests/ownership-lifecycle.test.ts index 22c82d269b..24345cdcf1 100644 --- a/packages/query-db-collection/tests/ownership-lifecycle.test.ts +++ b/packages/query-db-collection/tests/ownership-lifecycle.test.ts @@ -39,6 +39,7 @@ import { createOwnershipStorage, createPersistedOwnershipFixture, createQueryClient, + createSharedRowClient, detailOnly, expectColdOwnerRevalidation, expectMutationLifecycleSnapshot, @@ -49,6 +50,7 @@ import { listOnly, observeColdOwnerRevalidation, persistedOwners, + publishedRowMetadata, readClaimedCacheRows, recordMetadata, recordRunCacheClaim, @@ -57,6 +59,7 @@ import { runCleanups, shared, storedItems, + waitForRetention, } from './ownership-lifecycle-support.js' import type { Item, @@ -1428,6 +1431,1196 @@ describe(`query collection ownership lifecycle`, () => { expect(queryFn).toHaveBeenCalledOnce() }) + // Retention cleanup runs on a real 10 ms GC timer followed by a storage + // scan and a sync transaction. A loaded CI runner can take longer than the + // default 1 s wait, so these witnesses wait up to 5 s for it. + const retentionWait = { timeout: 5_000 } + // Longer than a wait, so a stuck wait reports its own observation. + const retentionTestTimeout = 20_000 + + /** + * ## A claim made during a held retention cleanup does not publish the cleaned owner + * + * Law: each publication records only the observer leases that still own a + * row. A persisted retention entry keeps its Query's owner edges until its + * time-to-live expires. The expiry then removes those edges in a sync + * transaction. That transaction is accepted but held behind an optimistic + * blocker, so the edges stay applied while a claim of the same row runs. The + * claim must not copy the removed owner back into its own publication. Its + * owners come from the pending cleanup and the claimant's own edge, never + * from an applied edge whose owner has already been retired. + * + * Authority: the published-owner law above, under which a released lease is + * not an owner even while its release waits behind held publication, and + * the retention rule that expiry releases the Query's owner edges. + * + * Reference: one row shared by a retained `detail` demand and a `list` demand + * that claims it during the cleanup hold. After publication the row has one + * owner, `list`. + * + * History: load `detail`, hold publication behind an optimistic blocker, + * unload `detail` so that its retention is recorded, wait for the expiry to + * start its cleanup transaction, load `list`, then publish. The checkpoint + * reads the applied owners. + * + * Why it fails: the expiry's cleanup deletes the row, because `detail` was + * its only owner, and retires the released Query's owner tracking when the + * cleanup commits, before the cleanup applies. The claim is an insert of the + * deleted row. Its owner read runs before its own insert, and the newest + * transaction has no write for the row there. So the read takes the applied + * edge for `detail`, and the released-owner filter, with no tracking entry, + * keeps it. The published owners then include `detail`. The claim's insert + * clear and the `??` fallback are not the trigger. Dropping the fallback alone + * leaves this history failing, so it does not discriminate that repair. The + * peer history below isolates the update path. + * + * Observation: this fixture has no DbClient, so the applied metadata is read + * from `collection._state.syncedMetadata`. + * + * Limits: one row, one retained demand, one claimant, a 10 ms time-to-live + * on real timers, and an in-memory baseline (the persisted scan returns no + * rows). The history checks the applied owners after publication. It does + * not model the hydrated client or a row that a claimant removes while the + * cleanup is held. + */ + it( + `does not publish a retention-cleaned owner through a claim made during the cleanup hold`, + async () => { + const id = `retention-cleanup-hold-claim` + const detail = categorySubset(`detail`) + const list = categorySubset(`list`) + const blocker = { id: `blocker`, category: `local`, name: `Blocker` } + const persistence = createDeferred() + const { collection } = createOwnershipFixture({ + id, + results: [[shared], [shared]], + persistedGcTime: 10, + scanPersistedRows: () => Promise.resolve([]), + }) + const mutation = createTransaction({ + mutationFn: () => persistence.promise, + }) + cleanups.push(async () => { + persistence.resolve() + await mutation.isPersisted.promise.catch(() => undefined) + }) + const settleTurn = async () => { + for (let turn = 0; turn < 3; turn++) + await new Promise((resolve) => setTimeout(resolve, 0)) + } + + await collection._sync.loadSubset(detail) + // Earlier result transactions apply first, so the count below grows only + // with the expiry's held transaction. + await waitForRetention( + () => + expect(collection._state.pendingSyncedTransactions).toHaveLength(0), + retentionWait, + ) + mutation.mutate(() => collection.insert(blocker)) + collection._sync.unloadSubset(detail) + const pendingBeforeExpiry = + collection._state.pendingSyncedTransactions.length + // The expiry's owner release is accepted but held behind the blocker. + await waitForRetention( + () => + expect( + collection._state.pendingSyncedTransactions.length, + ).toBeGreaterThan(pendingBeforeExpiry), + retentionWait, + ) + const claimed = Promise.resolve(collection._sync.loadSubset(list)) + void claimed.catch(() => undefined) + await settleTurn() + persistence.resolve() + await mutation.isPersisted.promise + await settleTurn() + await claimed + + expect({ + rows: rows(collection), + owners: persistedOwners(collection._state.syncedMetadata, shared.id) + .length, + }).toEqual({ + rows: [shared.id], + owners: 1, + }) + }, + retentionTestTimeout, + ) + + /** + * ## A retention-cleaned owner does not keep its row after the claimant releases + * + * Law: a row stays public while a committed owner needs it. After the + * retention cleanup, `list` is the row's only owner, and `detail` has been + * released. Releasing `list` must therefore let the row go once its own + * retention expires. A released demand's edge must not keep the row alive. + * + * Authority: the published-owner law above, under which a released lease is + * not an owner, and the retention rule that expiry releases the Query's + * owner edges. This is the identity observation for the count in the + * claim-during-cleanup-hold history above. The count cannot say which owner + * it is, so this sibling releases the claimant and watches the row. + * + * Reference: the same history as above, with the claim published. The + * expected public rows after `list` releases and its retention expires are + * none. + * + * History: the claim-during-cleanup-hold history, then unload `list` with no + * further hold. The checkpoint waits for the row to leave the collection. + * + * Why it fails: the stale `detail` edge that the cleanup should have retired + * stays in the applied owners, so the row survives the claimant's release. + * + * Limits: the same as the claim-during-cleanup-hold history. The row is + * observed through the public collection rows, and the owner set is not + * read. + */ + it( + `removes a retention-cleaned row once its claimant releases it`, + async () => { + const id = `retention-cleanup-hold-release` + const detail = categorySubset(`detail`) + const list = categorySubset(`list`) + const blocker = { id: `blocker`, category: `local`, name: `Blocker` } + const persistence = createDeferred() + const { collection } = createOwnershipFixture({ + id, + results: [[shared], [shared]], + persistedGcTime: 10, + scanPersistedRows: () => Promise.resolve([]), + }) + const mutation = createTransaction({ + mutationFn: () => persistence.promise, + }) + cleanups.push(async () => { + persistence.resolve() + await mutation.isPersisted.promise.catch(() => undefined) + }) + const settleTurn = async () => { + for (let turn = 0; turn < 3; turn++) + await new Promise((resolve) => setTimeout(resolve, 0)) + } + + await collection._sync.loadSubset(detail) + // Earlier result transactions apply first, so the count below grows only + // with the expiry's held transaction. + await waitForRetention( + () => + expect(collection._state.pendingSyncedTransactions).toHaveLength(0), + retentionWait, + ) + mutation.mutate(() => collection.insert(blocker)) + collection._sync.unloadSubset(detail) + const pendingBeforeExpiry = + collection._state.pendingSyncedTransactions.length + await waitForRetention( + () => + expect( + collection._state.pendingSyncedTransactions.length, + ).toBeGreaterThan(pendingBeforeExpiry), + retentionWait, + ) + const claimed = Promise.resolve(collection._sync.loadSubset(list)) + void claimed.catch(() => undefined) + await settleTurn() + persistence.resolve() + await mutation.isPersisted.promise + await settleTurn() + await claimed + + collection._sync.unloadSubset(list) + await waitForRetention( + () => expect(rows(collection)).toEqual([]), + retentionWait, + ) + }, + retentionTestTimeout, + ) + + /** + * ## A claimed row with no retention history is collected on release + * + * Law: a row whose only owner releases it leaves the collection once that + * owner's retention expires. This control runs the release step of the + * sibling above without a prior demand, so a failure there can be attributed + * to the stale owner edge rather than to the release or expiry. + * + * Authority: the retention rule that expiry releases the Query's owner + * edges, and the published-owner law (a released lease is not an owner). + * + * Reference: one row `shared` claimed by `list` alone. After `list` releases + * and its retention expires, no rows remain. + * + * History: load `list`, unload it, and wait for the row to leave. + * + * Limits: one row, one demand, a 10 ms time-to-live on real timers, and an + * in-memory baseline. + */ + it( + `collects a claimed row with no retention history once its claimant releases it`, + async () => { + const id = `retention-release-control` + const list = categorySubset(`list`) + const { collection } = createOwnershipFixture({ + id, + results: [[shared]], + persistedGcTime: 10, + scanPersistedRows: () => Promise.resolve([]), + }) + + await collection._sync.loadSubset(list) + expect(rows(collection)).toEqual([shared.id]) + + collection._sync.unloadSubset(list) + await waitForRetention( + () => expect(rows(collection)).toEqual([]), + retentionWait, + ) + }, + retentionTestTimeout, + ) + + /** + * ## A held cleanup does not publish a released owner beside a surviving peer + * + * Law: each publication records only the observer leases that still own a + * row. When a retention expiry releases one owner of a row that another lease + * still owns, the expiry removes only the released owner's edge, and the row + * stays public. A claim made while that release is held reads the pending + * release, so it publishes the peer and itself, and not the released owner. + * + * Authority: the published-owner law above, under which a released lease is + * not an owner even while its release waits behind held publication, and the + * retention rule that expiry releases the Query's owner edges. + * + * Reference: one row `shared`, owned by a retained `detail` demand and a + * surviving `peer` demand. A `list` claim during the cleanup hold publishes + * exactly two owners, `peer` and `list`, and not `detail`. + * + * History: load `detail`, load `peer`, hold publication behind an optimistic + * blocker, unload `detail` so that its retention is recorded, wait for the + * expiry to start its cleanup transaction, load `list`, then publish. + * + * Why it fails: the expiry's cleanup writes the row's owners as `{ peer }`, + * but that write is in an older pending transaction. The claim's read sees + * only its own newest transaction. The row is present, so the claim is an + * update and the read has no write to find. The read takes the applied + * `{ detail, peer }`. The cleanup has already retired `detail`'s tracking, so + * the released-owner filter keeps `detail`. The claim is an update, so the + * insert clear and the `??` fallback are not involved. This history therefore + * fails with the fallback kept and with it removed, and passes only when the + * read sees the pending owner set or the released tombstone outlives the + * cleanup's commit. + * + * Observation: this fixture has no DbClient, so the applied metadata is read + * from `collection._state.syncedMetadata`. + * + * Limits: one row, three demands, a 10 ms time-to-live on real timers, and an + * in-memory baseline. It does not model a row that a claimant removes while + * the cleanup is held. + */ + it( + `does not publish a released owner beside a surviving peer through a held cleanup`, + async () => { + const id = `retention-cleanup-peer-claim` + const detail = categorySubset(`detail`) + const peer = categorySubset(`peer`) + const list = categorySubset(`list`) + const blocker = { id: `blocker`, category: `local`, name: `Blocker` } + const persistence = createDeferred() + const { collection } = createOwnershipFixture({ + id, + results: [[shared], [shared], [shared]], + persistedGcTime: 10, + scanPersistedRows: () => Promise.resolve([]), + }) + const mutation = createTransaction({ + mutationFn: () => persistence.promise, + }) + cleanups.push(async () => { + persistence.resolve() + await mutation.isPersisted.promise.catch(() => undefined) + }) + const settleTurn = async () => { + for (let turn = 0; turn < 3; turn++) + await new Promise((resolve) => setTimeout(resolve, 0)) + } + + await collection._sync.loadSubset(detail) + await collection._sync.loadSubset(peer) + // Earlier result transactions apply first, so the count below grows only + // with the expiry's held transaction. + await waitForRetention( + () => + expect(collection._state.pendingSyncedTransactions).toHaveLength(0), + retentionWait, + ) + mutation.mutate(() => collection.insert(blocker)) + collection._sync.unloadSubset(detail) + const pendingBeforeExpiry = + collection._state.pendingSyncedTransactions.length + // The expiry's owner release is accepted but held behind the blocker. + await waitForRetention( + () => + expect( + collection._state.pendingSyncedTransactions.length, + ).toBeGreaterThan(pendingBeforeExpiry), + retentionWait, + ) + const claimed = Promise.resolve(collection._sync.loadSubset(list)) + void claimed.catch(() => undefined) + await settleTurn() + persistence.resolve() + await mutation.isPersisted.promise + await settleTurn() + await claimed + + expect({ + rows: rows(collection), + owners: persistedOwners(collection._state.syncedMetadata, shared.id) + .length, + }).toEqual({ + rows: [shared.id], + owners: 2, + }) + }, + retentionTestTimeout, + ) + + /** + * ## A hydrated release of a held-cleanup row needs a recorded owner to keep it + * + * Law: a row stays public while a committed owner needs it. In the surviving + * peer history above, the released `detail` is not an owner, so its edge + * must not keep the row alive after `peer` and `list` release. A live + * collection cannot show this, because its applied owners are already + * filtered. The stored owners do show it: a hydrated client trusts every + * recorded owner. + * + * Authority: the published-owner law above, under which a released lease is + * not an owner, and the hydration rule of the published-owner histories, a + * hydrated row stays while any recorded owner remains. The count in the peer + * history cannot name the owner, so this sibling reads the stored owners + * through a hydrated release. + * + * Reference: the surviving-peer history, dehydrated after publication. A + * browser client that loads and then releases `peer` and `list` has no + * recorded owner left, so the row leaves. + * + * History: the surviving-peer history, with its stored metadata for `shared` + * taken as the hydration input. A fresh browser DbClient hydrates it over the + * same collection id. The browser loads and releases `peer` and `list`, and + * the checkpoint reads its rows. The fixture is the server because it supplies + * the persisted scan that retention needs, and a DbClient collection here does + * not. The stored metadata is the owner set that `dehydrate` would publish. + * + * Why it fails: the stale `detail` edge stays in the stored owners, so the + * hydrated row survives after its last real owners release. + * + * Limits: one row, three demands, a 10 ms server time-to-live, and a browser + * with no retention. It does not model a browser that holds the release. + */ + it( + `hydrated clients drop a held-cleanup row once its peer and claimant release it`, + async () => { + const id = `retention-cleanup-peer-hydrated` + const detail = categorySubset(`detail`) + const peer = categorySubset(`peer`) + const list = categorySubset(`list`) + const blocker = { id: `blocker`, category: `local`, name: `Blocker` } + const persistence = createDeferred() + // The fixture supplies the persisted scan that makes retention expire. + const { collection } = createOwnershipFixture({ + id, + results: [[shared], [shared], [shared]], + persistedGcTime: 10, + scanPersistedRows: () => Promise.resolve([]), + }) + const mutation = createTransaction({ + mutationFn: () => persistence.promise, + }) + cleanups.push(async () => { + persistence.resolve() + await mutation.isPersisted.promise.catch(() => undefined) + }) + const settleTurn = async () => { + for (let turn = 0; turn < 3; turn++) + await new Promise((resolve) => setTimeout(resolve, 0)) + } + + await collection._sync.loadSubset(detail) + await collection._sync.loadSubset(peer) + // Earlier result transactions apply first, so the count below grows only + // with the expiry's held transaction. + await waitForRetention( + () => + expect(collection._state.pendingSyncedTransactions).toHaveLength(0), + retentionWait, + ) + mutation.mutate(() => collection.insert(blocker)) + collection._sync.unloadSubset(detail) + const pendingBeforeExpiry = + collection._state.pendingSyncedTransactions.length + await waitForRetention( + () => + expect( + collection._state.pendingSyncedTransactions.length, + ).toBeGreaterThan(pendingBeforeExpiry), + retentionWait, + ) + const claimed = Promise.resolve(collection._sync.loadSubset(list)) + void claimed.catch(() => undefined) + await settleTurn() + persistence.resolve() + await mutation.isPersisted.promise + await settleTurn() + await claimed + + // The stored metadata of the row is the state that a dehydrate publishes. + const state = { + collections: [ + { + collectionId: id, + rows: [ + { + key: shared.id, + value: shared, + metadata: collection._state.syncedMetadata.get(shared.id), + }, + ], + }, + ], + } + const browser = createSharedRowClient(id) + cleanups.push(async () => { + await browser.client.cleanup() + browser.queryClient.clear() + }) + browser.client.hydrate(state) + const hydrated = browser.client.collection(browser.descriptor) + await hydrated._sync.loadSubset(peer) + await hydrated._sync.loadSubset(list) + hydrated._sync.unloadSubset(peer) + hydrated._sync.unloadSubset(list) + await waitForRetention( + () => expect(rows(hydrated)).toEqual([]), + retentionWait, + ) + }, + retentionTestTimeout, + ) + + /** + * ## A held owner stays published beside a later claim of its row + * + * Law: each publication records every observer lease that still owns a row. + * A lease whose result is held behind publication already owns its rows in + * memory. A later lease that claims the same row while that result is held + * must publish both leases as owners. It must not overwrite the held owner + * with its own edge alone, because the held owner's edge is not yet visible + * in the applied metadata. + * + * Authority: the published-owner law above (a row remains public while any + * committed owner needs it, and publication records the owner set), and the + * accepted-edge rule in the ownership read: accepted in-memory edges + * supplement owners whose metadata has not published yet. + * + * Reference: one row `shared`. `detail` and `list` both need it. Publication + * is held, so neither edge has published. After release, the published owners + * are exactly `detail` and `list`. + * + * History: hold publication behind an optimistic blocker, load `detail` and + * leave its result held, load `list` for the same row, release the blocker, + * then publish. The checkpoint reads the applied owners. + * + * Why it fails when the in-memory supplement is ignored: the claim reads only + * the applied metadata, which has no owner for the row. It publishes `{list}` + * and drops the held `detail` edge. + * + * Observation: this fixture has no DbClient, so the applied metadata is read + * from `collection._state.syncedMetadata`. + * + * Limits: one row, two demands, one blocker, in-memory on-demand loads, and no + * persistence. + */ + it(`publishes a held owner beside a later claim of its row`, async () => { + const id = `held-owner-beside-claim` + const detail = categorySubset(`detail`) + const list = categorySubset(`list`) + const blocker = { id: `blocker`, category: `local`, name: `Blocker` } + const persistence = createDeferred() + const { collection } = createOwnershipFixture({ + id, + results: [[shared], [shared]], + }) + const mutation = createTransaction({ + mutationFn: () => persistence.promise, + }) + cleanups.push(async () => { + persistence.resolve() + await mutation.isPersisted.promise.catch(() => undefined) + }) + const settleTurn = async () => { + for (let turn = 0; turn < 3; turn++) + await new Promise((resolve) => setTimeout(resolve, 0)) + } + + mutation.mutate(() => collection.insert(blocker)) + const held = Promise.resolve(collection._sync.loadSubset(detail)) + void held.catch(() => undefined) + await settleTurn() + const claimed = Promise.resolve(collection._sync.loadSubset(list)) + void claimed.catch(() => undefined) + await settleTurn() + persistence.resolve() + await mutation.isPersisted.promise + await settleTurn() + await Promise.all([held, claimed]) + + expect({ + rows: rows(collection), + owners: persistedOwners(collection._state.syncedMetadata, shared.id) + .length, + }).toEqual({ + rows: [shared.id], + owners: 2, + }) + }) + + /** + * ## A hydrated release keeps a held owner's row while the held owner is recorded + * + * Law: a row stays public while a committed owner needs it. In the held-owner + * history above, `detail` and `list` both own the row, and publication + * records both. A browser that hydrates the published state and releases + * `list` must keep the row through the recorded `detail` owner. + * + * Authority: the published-owner law above (publication records the owner + * set, and a row remains public while any committed owner needs it), and the + * hydration rule that a hydrated row stays while any recorded owner remains. + * The count in that history cannot name the owner. A live collection cannot + * show the drop either, because its in-memory `detail` lease still holds the + * row, so this sibling reads the stored owners through a hydrated release. + * + * Reference: the held-owner history, dehydrated after publication. A browser + * that loads and releases `list` keeps `shared`, because `detail` is recorded. + * + * History: the held-owner history on a server DbClient, dehydrate it, and + * hydrate a fresh browser DbClient over the same collection. The browser loads + * and then releases `list`. The checkpoint reads its rows after the release + * settles. + * + * Why it fails: publication dropped the held `detail` edge, so the stored + * owners hold only `list`. Releasing `list` in the browser removes the row. + * + * Limits: one row, two demands, one blocker, and a browser with no retention. + * The survival checkpoint waits three turns, so a release that is slower than + * that would not be caught by this test. + */ + it(`hydrated clients keep a held owner's row after its later claimant releases`, async () => { + const id = `held-owner-hydrated-release` + const detail = categorySubset(`detail`) + const list = categorySubset(`list`) + const blocker = { id: `blocker`, category: `local`, name: `Blocker` } + const persistence = createDeferred() + const server = createSharedRowClient(id) + const collection = server.client.collection(server.descriptor) + const mutation = createTransaction({ + mutationFn: () => persistence.promise, + }) + cleanups.push(async () => { + persistence.resolve() + await mutation.isPersisted.promise.catch(() => undefined) + await server.client.cleanup() + server.queryClient.clear() + }) + const settleTurn = async () => { + for (let turn = 0; turn < 3; turn++) + await new Promise((resolve) => setTimeout(resolve, 0)) + } + + mutation.mutate(() => collection.insert(blocker)) + const held = Promise.resolve(collection._sync.loadSubset(detail)) + void held.catch(() => undefined) + await settleTurn() + const claimed = Promise.resolve(collection._sync.loadSubset(list)) + void claimed.catch(() => undefined) + await settleTurn() + persistence.resolve() + await mutation.isPersisted.promise + await settleTurn() + await Promise.all([held, claimed]) + + const state = server.client.dehydrate({ + shouldDehydrateCollection: () => true, + }) + const browser = createSharedRowClient(id) + cleanups.push(async () => { + await browser.client.cleanup() + browser.queryClient.clear() + }) + browser.client.hydrate(state) + const hydrated = browser.client.collection(browser.descriptor) + await hydrated._sync.loadSubset(list) + hydrated._sync.unloadSubset(list) + await settleTurn() + expect(rows(hydrated)).toEqual([shared.id]) + }) + + /** + * ## A claim that inserts a row replaces its applied metadata + * + * Law: an insert names the whole metadata of its row. An insert that names + * none clears the row's previous metadata, and a later write in the same + * session reads its own earlier writes. Query's claim of an absent row is a + * source insert with no metadata. It stages its owners only after that + * clear, so the published metadata contains the owner edge and nothing from + * the applied metadata that the clear removed. Metadata written outside a + * Query, here a plain metadata write with no row data, is not a hydration + * seed. The source sees the row as absent, so the insert is not an update. + * + * Authority: the sync metadata session law (an insert without metadata + * clears it, and a read includes the session's own earlier write), and the + * insert comment in the result-application path ("An insert clears stale + * metadata for its key"). + * + * Reference: one absent row `shared` whose applied metadata carries a + * non-owner field `user`. A claim of the row by `detail` publishes the owner + * edge for `detail`, with no `user` field. + * + * History: seed the metadata through `setupMetadata` without any row data, + * then load `detail`. The result inserts the row and stages the owner edge + * in the same transaction. The checkpoint reads the published metadata. + * + * Why it fails: the owner write reads the row's metadata after the insert + * has staged the clear. `metadata.row.get` returns the clear as `undefined`, + * and `getRowMetadata` treats `undefined` as "no session value", falling + * back with `??` to the applied `{ user }`. The owner write spreads that + * fallback, so `user` survives the insert. + * + * Limits: one row, one claimant, no hold, and an in-memory baseline. The + * history checks the published row metadata. It does not model persistence + * or a row that already has data. + */ + it(`does not keep applied row metadata when a claim inserts its row`, async () => { + const id = `claim-insert-replaces-metadata` + const detail = categorySubset(`detail`) + const { collection } = createOwnershipFixture({ + id, + results: [[shared]], + setupMetadata: (metadata) => { + metadata.row.set(shared.id, { user: `stale` }) + }, + }) + + await collection._sync.loadSubset(detail) + + expect({ + rows: rows(collection), + user: ( + collection._state.syncedMetadata.get(shared.id) as + Record | undefined + )?.user, + owners: persistedOwners(collection._state.syncedMetadata, shared.id) + .length, + }).toEqual({ + rows: [shared.id], + user: undefined, + owners: 1, + }) + }) + + /** + * ## A persisted claim that inserts its row does not keep applied row metadata + * + * Law: the claim-insert law above also holds behind the persistence wrapper. + * The insert of a row without metadata clears the row's metadata for the + * session. The claim's owner write then reads its own session's metadata, + * and that read sees the clear. It must not restore the applied `user` field. + * + * Authority: the same law as the in-memory claim-insert witness, applied to + * the persisted sync API that a Query Collection receives when it is wrapped + * by `persistedCollectionOptions`. The wrapper's staged metadata read is the + * session's view, and core supplies only the applied metadata beneath it. + * + * Reference: applied row metadata `{ user: 'stale' }` with no row data. A + * claim by `detail` inserts `shared`. The published metadata keeps only the + * `queryCollection` key, and the owner set holds `detail`. + * + * History: a persisted Query Collection with an empty storage. Before the + * first Query result, one metadata-only transaction stages the applied + * metadata. Then a `detail` claim loads `shared`. The checkpoint reads the + * applied metadata of the row after the claim settles. + * + * Why it fails: the insert stages a clear in the wrapper's open transaction. + * The owner write then reads the row through the wrapper. Its + * `readStagedMetadata` returns the staged clear as `undefined`. `getRowMetadata` + * falls back with `??` to core's applied `{ user }`, and the owner write + * spreads it. The wrapper's own fallback to applied metadata answers only the + * earlier read, which runs before the insert stages its clear. + * + * Limits: one row, one claimant, no hold, and a controlled in-memory storage + * adapter behind the wrapper. + */ + it(`does not keep applied row metadata when a persisted claim inserts its row`, async () => { + const id = `persisted-claim-insert-replaces-metadata` + const detail = categorySubset(`detail`) + const storage = createOwnershipStorage() + const queryClient = createQueryClient() + const queryFn = vi.fn(() => Promise.resolve([shared])) + const baseOptions = queryCollectionOptions({ + id, + queryClient, + queryKey: [id], + queryFn, + getKey: (item) => item.id, + syncMode: `on-demand`, + persistedGcTime: Number.POSITIVE_INFINITY, + startSync: true, + }) + const originalSync = baseOptions.sync + let pendingSetup = true + const collection = createCollection( + persistedCollectionOptions< + Item, + string | number, + never, + QueryCollectionUtils + >({ + ...baseOptions, + sync: { + sync: (params) => { + if (!params.metadata) { + throw new Error(`Sync metadata API is unavailable`) + } + if (pendingSetup) { + pendingSetup = false + params.begin() + params.metadata.row.set(shared.id, { user: `stale` }) + params.commit() + } + return originalSync.sync(params) + }, + }, + persistence: { adapter: storage.adapter }, + }), + ) + cleanups.push(async () => { + storage.release() + try { + await collection.cleanup() + } finally { + queryClient.clear() + } + }) + + await collection._sync.loadSubset(detail) + + expect({ + rows: rows(collection), + user: ( + collection._state.syncedMetadata.get(shared.id) as + Record | undefined + )?.user, + owners: persistedOwners(collection._state.syncedMetadata, shared.id) + .length, + }).toEqual({ + rows: [shared.id], + user: undefined, + owners: 1, + }) + }) + + /** + * ## A persisted claim after a held removal does not restore the removed row's metadata + * + * Law: the held-removal law above, behind the persistence wrapper. A source + * delete clears the row's metadata for the session. A source insert that + * names no metadata clears it again. A claim reads its session's own earlier + * writes, so the removal's delete stays in force while it is held. + * + * Authority: the same sync metadata session law as the in-memory held-removal + * witness, and the published-owner law: a released lease is not an owner. + * + * Reference: a persisted row `shared` with applied metadata `{ user: 'seeded' }` + * that `detail` owns. A held removal of `detail` deletes the row. A `list` + * claim during the hold re-inserts it. The published metadata holds the `list` + * owner edge and no `user` field. + * + * History: a persisted Query Collection over storage that holds `shared` with + * metadata `{ user: 'seeded' }`, with `persistedGcTime: 0` so that a release + * removes the row. That storage is a restored state. Restart and resume reads + * belong to other owners, so this test uses the restored state only as a + * legal starting point. Load `detail`. Hold publication behind an optimistic + * blocker, unload `detail`, and wait for the removal to be accepted. Load + * `list` during the hold, then publish. The checkpoint reads the applied + * metadata of the row. + * + * Why it fails: the claim's insert stages a clear, and the owner write reads + * it as `undefined` through the wrapper. `getRowMetadata` falls back to core's + * applied `{ user: 'seeded' }`. The removal has not published, so core still + * holds that value. + * + * Limits: one row, one removed demand, one claimant, a controlled in-memory + * storage adapter behind the wrapper. + */ + it(`does not restore a removed row's applied metadata when a persisted claim follows a held removal`, async () => { + const id = `persisted-held-removal-claim-metadata` + const detail = categorySubset(`detail`) + const list = categorySubset(`list`) + const blocker = { id: `blocker`, category: `local`, name: `Blocker` } + const persistence = createDeferred() + const storage = createOwnershipStorage({ + rows: new Map([[shared.id, shared]]), + rowMetadata: new Map([[shared.id, { user: `seeded` }]]), + collectionMetadata: new Map(), + }) + const queryClient = createQueryClient() + const queryFn = vi.fn(() => Promise.resolve([shared])) + const collection = createCollection( + persistedCollectionOptions< + Item, + string | number, + never, + QueryCollectionUtils + >({ + ...queryCollectionOptions({ + id, + queryClient, + queryKey: [id], + queryFn, + getKey: (item) => item.id, + syncMode: `on-demand`, + persistedGcTime: 0, + startSync: true, + }), + persistence: { adapter: storage.adapter }, + }), + ) + const mutation = createTransaction({ + mutationFn: () => persistence.promise, + }) + cleanups.push(async () => { + persistence.resolve() + await mutation.isPersisted.promise.catch(() => undefined) + storage.release() + try { + await collection.cleanup() + } finally { + queryClient.clear() + } + }) + const settleTurn = async () => { + for (let turn = 0; turn < 3; turn++) + await new Promise((resolve) => setTimeout(resolve, 0)) + } + + await collection._sync.loadSubset(detail) + mutation.mutate(() => collection.insert(blocker)) + const pendingBeforeRemoval = + collection._state.pendingSyncedTransactions.length + collection._sync.unloadSubset(detail) + // The removal's delete is accepted but held behind the blocker. + await vi.waitFor(() => + expect( + collection._state.pendingSyncedTransactions.length, + ).toBeGreaterThan(pendingBeforeRemoval), + ) + const claimed = Promise.resolve(collection._sync.loadSubset(list)) + void claimed.catch(() => undefined) + await settleTurn() + persistence.resolve() + await mutation.isPersisted.promise + await settleTurn() + await claimed + + expect({ + user: ( + collection._state.syncedMetadata.get(shared.id) as + Record | undefined + )?.user, + owners: persistedOwners(collection._state.syncedMetadata, shared.id) + .length, + }).toEqual({ + user: undefined, + owners: 1, + }) + }) + + /** + * ## A claim of a hydrated row keeps the seed's metadata + * + * Law: a hydration seed sits beneath every source write and supplies only + * the objects that no source write has written. Query sees a hydrated row as + * present, so a claim with an equal result writes no row; it writes only the + * row's metadata, after reading the current metadata. At claim time no source + * write has replaced the seed's metadata, so that read returns it, and the + * claim keeps the seed's non-owner fields beside the claimant's owner edge. + * The owner write is itself a source write to that metadata, so the law + * afterwards is different: a later seed for the row must not replace it. The + * late-seed witness below checks that. This control is the boundary of the + * claim-insert history above, where the claim inserts an absent row and the + * insert clears the metadata. + * + * Authority: decisions D4 and D5 recorded in the sync metadata session + * oracle (a seed sits beneath source writes; a source insert replaces a + * seeded row), and the read-your-writes law that the owner write's read + * returns the current metadata. + * + * Reference: one row `shared` hydrated with metadata `{ user: 'seeded' }`. + * A claim by `detail` keeps `user` and publishes one `detail` owner edge. + * + * History: create a DbClient, hydrate the row with its metadata, create the + * Query Collection through that client, then load `detail`. The result + * carries the same row. The checkpoint reads the published metadata. + * + * Why this is a control: it runs the same claim as the claim-insert history, + * over a present hydrated row rather than an absent row. A repair that clears + * metadata on every claim would pass the claim-insert history and fail this + * one. + * + * Limits: one row, one claimant, no hold, and an in-memory client without + * persistence. + */ + it(`keeps a hydrated row's metadata when a claim reconciles the seeded row`, async () => { + const id = `claim-hydrated-keeps-metadata` + const detail = categorySubset(`detail`) + const queryClient = createQueryClient() + const descriptor = collectionOptions( + queryCollectionOptions({ + id, + queryClient, + queryKey: [id], + queryFn: () => Promise.resolve([shared]), + getKey: (item) => item.id, + syncMode: `on-demand`, + startSync: true, + }), + ) + const client = new DbClient({ queryClient }) + cleanups.push(async () => { + await client.cleanup() + queryClient.clear() + }) + client.hydrate({ + collections: [ + { + collectionId: id, + rows: [ + { + key: shared.id, + value: shared, + metadata: { user: `seeded` }, + }, + ], + }, + ], + }) + const collection = client.collection(descriptor) + + await collection._sync.loadSubset(detail) + + expect({ + rows: rows(collection), + published: publishedRowMetadata(client, shared.id), + }).toEqual({ + rows: [shared.id], + published: { user: `seeded`, owners: 1 }, + }) + }) + + /** + * ## A late hydration seed does not erase a claim's owner edge + * + * Law: a hydration seed supplies only the objects that no source write has + * written. Query's owner write is a source write to the row's metadata, so + * once a claim has published that metadata, a later seed for the row must + * not replace it. The claim's owner edge and the seed's `user` field both + * stay published. + * + * Authority: decision D5 recorded in the sync metadata session oracle (a seed + * sits beneath source writes, and row data and row metadata are separate + * objects). The claim's own owner write is the source write in this history. + * + * Reference: one row `shared` hydrated with `{ user: 'seeded' }`. A `detail` + * claim publishes `user: 'seeded'` and one owner. A late chunk for the same + * row carries `{ user: 'late' }`. Its metadata is ignored, so the published + * metadata is unchanged. + * + * History: the hydrated control's setup, then load `detail`, check the claim + * checkpoint, and apply a chunk that seeds `shared` with `{ user: 'late' }` + * through `DbClient.applyCollectionChunk`. The checkpoint after the chunk + * reads the published metadata through the client. + * + * Why it fails: the late seed replaces the published metadata with + * `{ user: 'late' }`. The `detail` owner edge is gone, and the seed's field + * is the one that survives. + * + * Limits: one row, one claimant, no hold, and an in-memory client without + * persistence. The chunk is applied while the claim is published and not + * held. Held or persisted seeds are not covered here. + */ + it(`ignores a late hydration seed for a row whose metadata a claim has published`, async () => { + const id = `late-seed-after-claim` + const detail = categorySubset(`detail`) + const queryClient = createQueryClient() + const descriptor = collectionOptions( + queryCollectionOptions({ + id, + queryClient, + queryKey: [id], + queryFn: () => Promise.resolve([shared]), + getKey: (item) => item.id, + syncMode: `on-demand`, + startSync: true, + }), + ) + const client = new DbClient({ queryClient }) + cleanups.push(async () => { + await client.cleanup() + queryClient.clear() + }) + client.hydrate({ + collections: [ + { + collectionId: id, + rows: [ + { key: shared.id, value: shared, metadata: { user: `seeded` } }, + ], + }, + ], + }) + const collection = client.collection(descriptor) + + await collection._sync.loadSubset(detail) + expect(publishedRowMetadata(client, shared.id)).toEqual({ + user: `seeded`, + owners: 1, + }) + + client.applyCollectionChunk({ + collectionId: id, + rows: [{ key: shared.id, value: shared, metadata: { user: `late` } }], + }) + + expect({ + rows: rows(collection), + published: publishedRowMetadata(client, shared.id), + }).toEqual({ + rows: [shared.id], + published: { user: `seeded`, owners: 1 }, + }) + }) + + /** + * ## A held removal's cleared metadata is not restored by a later claim + * + * Law: a source delete clears the row's metadata for the session. A source + * insert that names no metadata then clears it again. A claim reads its + * session's own earlier writes, including a held removal that deleted the + * row. Under the seed-ordering decision D5, a hydration seed supplies only + * objects that no surviving source write has written. The held removal's + * delete is such a write, so the seed's `user` field stays gone while the + * removal is in the session. + * + * Authority: the sync metadata session law (a delete clears, an insert names + * the whole metadata, a read includes the session's own earlier write), the + * seed-ordering decision D5, and the published-owner law above, under which a + * released lease is not an owner. + * + * Reference: a hydrated row `shared` with metadata `{ user: 'seeded' }` that + * `detail` owns. A held removal of `detail` deletes the row. A `list` claim + * during the hold re-inserts it. The published metadata holds the `list` + * owner edge and no `user` field. + * + * History: create a DbClient and hydrate the row with its metadata. Load + * `detail`. Hold publication behind an optimistic blocker, unload `detail`, + * and wait for the removal to commit. Load `list` during the hold, then + * publish. The checkpoint reads the published metadata through the client. + * + * Why it fails: the claim's owner write reads the row's metadata after its + * own insert's clear. The newest transaction holds a delete for the row, so + * the read returns `undefined` and falls back to the applied metadata. The + * removal has not applied, so the applied metadata still has `user`, and the + * owner write spreads it back. + * + * Reachability: the removal is a public unload and the claim is a public + * load. The only hold is an optimistic blocker. + * + * Limits: one row, one removed demand, one claimant, an in-memory client, and + * no persistence. + */ + it(`does not restore a removed row's applied metadata when a claim follows a held removal`, async () => { + const id = `held-removal-claim-metadata` + const detail = categorySubset(`detail`) + const list = categorySubset(`list`) + const blocker = { id: `blocker`, category: `local`, name: `Blocker` } + const persistence = createDeferred() + const queryClient = createQueryClient() + const descriptor = collectionOptions( + queryCollectionOptions({ + id, + queryClient, + queryKey: [id], + queryFn: () => Promise.resolve([shared]), + getKey: (item) => item.id, + syncMode: `on-demand`, + startSync: true, + }), + ) + const client = new DbClient({ queryClient }) + const mutation = createTransaction({ + mutationFn: () => persistence.promise, + }) + cleanups.push(async () => { + persistence.resolve() + await mutation.isPersisted.promise.catch(() => undefined) + await client.cleanup() + queryClient.clear() + }) + client.hydrate({ + collections: [ + { + collectionId: id, + rows: [ + { key: shared.id, value: shared, metadata: { user: `seeded` } }, + ], + }, + ], + }) + const collection = client.collection(descriptor) + const settleTurn = async () => { + for (let turn = 0; turn < 3; turn++) + await new Promise((resolve) => setTimeout(resolve, 0)) + } + + await collection._sync.loadSubset(detail) + mutation.mutate(() => collection.insert(blocker)) + const pendingBeforeRemoval = + collection._state.pendingSyncedTransactions.length + collection._sync.unloadSubset(detail) + // The removal's delete is accepted but held behind the blocker. + await vi.waitFor(() => + expect( + collection._state.pendingSyncedTransactions.length, + ).toBeGreaterThan(pendingBeforeRemoval), + ) + const claimed = Promise.resolve(collection._sync.loadSubset(list)) + void claimed.catch(() => undefined) + await settleTurn() + persistence.resolve() + await mutation.isPersisted.promise + await settleTurn() + await claimed + + expect(publishedRowMetadata(client, shared.id)).toEqual({ + user: undefined, + owners: 1, + }) + }) + /** * ## A reacquired lease's cached rows receive accepted direct writes * @@ -1455,61 +2648,72 @@ describe(`query collection ownership lifecycle`, () => { * distinguish the skip: the upsert keeps the row either way. The Query * cache, read through `QueryClient.getQueriesData`, does. */ - it(`patches a reacquired lease's cached rows while its result waits`, async () => { - const id = `reacquired-lease-direct-patch` - const detail = categorySubset(`detail`) - const blocker = { id: `blocker`, category: `local`, name: `Blocker` } - const persistence = createDeferred() - const barrier = createDeferred() - const { collection, queryClient, queryFn } = createOwnershipFixture({ - id, - results: [[shared]], - }) - const mutation = createTransaction({ - mutationFn: () => persistence.promise, - }) - const settleTurn = async () => { - for (let turn = 0; turn < 3; turn++) - await new Promise((resolve) => setTimeout(resolve, 0)) - } - cleanups.push(async () => { - collection.deferDataRefresh = null - barrier.resolve() - persistence.resolve() - await mutation.isPersisted.promise.catch(() => undefined) - }) + it( + `patches a reacquired lease's cached rows while its result waits`, + async () => { + const id = `reacquired-lease-direct-patch` + const detail = categorySubset(`detail`) + const blocker = { id: `blocker`, category: `local`, name: `Blocker` } + const persistence = createDeferred() + const barrier = createDeferred() + const { collection, queryClient, queryFn } = createOwnershipFixture({ + id, + results: [[shared]], + }) + const mutation = createTransaction({ + mutationFn: () => persistence.promise, + }) + const settleTurn = async () => { + for (let turn = 0; turn < 3; turn++) + await new Promise((resolve) => setTimeout(resolve, 0)) + } + cleanups.push(async () => { + collection.deferDataRefresh = null + barrier.resolve() + persistence.resolve() + await mutation.isPersisted.promise.catch(() => undefined) + }) - await collection._sync.loadSubset(detail) - mutation.mutate(() => collection.insert(blocker)) - collection._sync.unloadSubset(detail) - await settleTurn() - collection.deferDataRefresh = barrier.promise - const reacquired = Promise.resolve(collection._sync.loadSubset(detail)) - void reacquired.catch(() => undefined) - await settleTurn() - const write = Promise.resolve( - collection.utils.writeUpsert({ ...shared, name: `Manual` }), - ) - void write.catch(() => undefined) - await settleTurn() + await collection._sync.loadSubset(detail) + // Earlier result transactions apply first, so the count below grows only + // with the expiry's held transaction. + await waitForRetention( + () => + expect(collection._state.pendingSyncedTransactions).toHaveLength(0), + retentionWait, + ) + mutation.mutate(() => collection.insert(blocker)) + collection._sync.unloadSubset(detail) + await settleTurn() + collection.deferDataRefresh = barrier.promise + const reacquired = Promise.resolve(collection._sync.loadSubset(detail)) + void reacquired.catch(() => undefined) + await settleTurn() + const write = Promise.resolve( + collection.utils.writeUpsert({ ...shared, name: `Manual` }), + ) + void write.catch(() => undefined) + await settleTurn() - persistence.resolve() - await mutation.isPersisted.promise - collection.deferDataRefresh = null - barrier.resolve() - await reacquired.catch(() => undefined) - await write - await settleTurn() + persistence.resolve() + await mutation.isPersisted.promise + collection.deferDataRefresh = null + barrier.resolve() + await reacquired.catch(() => undefined) + await write + await settleTurn() - expect({ - row: collection.get(shared.id)?.name, - cache: queryClient - .getQueriesData>({ queryKey: [id] }) - .flatMap(([, data]) => data ?? []) - .map((item) => item.name), - queryCalls: queryFn.mock.calls.length, - }).toEqual({ row: `Manual`, cache: [`Manual`], queryCalls: 2 }) - }) + expect({ + row: collection.get(shared.id)?.name, + cache: queryClient + .getQueriesData>({ queryKey: [id] }) + .flatMap(([, data]) => data ?? []) + .map((item) => item.name), + queryCalls: queryFn.mock.calls.length, + }).toEqual({ row: `Manual`, cache: [`Manual`], queryCalls: 2 }) + }, + retentionTestTimeout, + ) /** * ## A release whose durable commit fails is terminal for its sync run