diff --git a/.changeset/fix-effect-left-join-startup.md b/.changeset/fix-effect-left-join-startup.md new file mode 100644 index 0000000000..934954eeac --- /dev/null +++ b/.changeset/fix-effect-left-join-startup.md @@ -0,0 +1,5 @@ +--- +'@tanstack/db': patch +--- + +Classify Effect deltas from preceding result membership so LEFT-join startup and coalesced source writes emit the correct enter, update, or exit callback, including functional projections with callable or opaque values. Unordered Effects track only result keys and preserve their existing update notifications without comparing or retaining projected values. diff --git a/docs/contributing/oracle-coverage.md b/docs/contributing/oracle-coverage.md index a6616dd4df..32a0bf9d44 100644 --- a/docs/contributing/oracle-coverage.md +++ b/docs/contributing/oracle-coverage.md @@ -1889,6 +1889,19 @@ The [round 7 ledger](oracle-reviews/utils-unit-retirement-round-7.md) retains the retired examples. Caller-modified built-ins and Collection publication are outside this direct-return owner. +Set performance work evaluated in PR #2301 is excluded by the maintainer's +2026-10-30 decision: "we've never had this perf problem ... let's remove it." +The production matcher and the existing `utils.1`–`utils.40` campaigns and +budgets remain unchanged. The synthetic duplicate-valued object-Set family +exposes factorial assignment work, and an 8,192-member flat assignment chain +exposes recursive stack depth. No application incident was established. +The [versioned replay inputs](oracle-reviews/effect-startup-delta.md#deferred-set-replay-inputs) +retain the descriptors, observations and limits. The primary owner remains +`packages/db/tests/utils-oracle.property.test.ts`; resume optimization when the +maintainer requests it or an application workload requires it. The new +`utils.41`/`utils.42` work guards and dedicated receivers leave with that work. +This records a scope decision, not closure of those performance limitations. + The [Duration query equality oracle](https://github.com/TanStack/db/blob/main/packages/db/tests/duration-query-equality-oracle.test.ts) uses ten public fields to predict direct BasicIndex/BTreeIndex exact lookups and public `eq` snapshots before and after indexing. The `in` evaluator owner also @@ -5759,9 +5772,10 @@ Held mutations, independent confirmations/outcomes, both settlement orders, remote updates, peer exit/reentry and next-use reach 5,512 lane/channel/cut cells exactly once. No implicit Effect-root metadata or output Collection ID is claimed. -Open P09: all 64 LEFT-join histories emit a startup update for a never-entered -unmatched occurrence after its load finished. Those initial comparisons remain -RED on unchanged `858b174a6`; histories continue and all later comparisons pass. +Original P09 evidence: all 64 LEFT-join histories emitted a startup update for +a never-entered unmatched occurrence after its load finished. Those initial +comparisons were RED on unchanged `858b174a6`; histories continued and all later +comparisons passed. The other 136 native histories and 16 controls pass. Independent hostile overlays reject dispatch, previous-value, duplication, colliding-key and extra undefined defects; lawful key renaming, fresh reentry and empty partitions retain @@ -5773,8 +5787,62 @@ in the RC chunk record for exact fingerprints, evidence and review. Limits: eager controlled core sources, scalar rows, one mutation per source, and these named paths. No real provider/framework, arbitrary scheduling, ordered repair, same-source overlap, callback-input sanitization or exact -internal graph-run callback-count guarantee follows. P09 requires production -repair after the full oracle campaign; this entry makes no closure claim. +internal graph-run callback-count guarantee follows. + +Historical initial P09 repair evidence for PR #2301: +The exact owner and `test:oracles` entry were restored before production edits. +On baseline `9b03e639f`, the fresh RED reproduced 64 startup failures with every +reach cell at count 1. Repaired production passes all 216 owner tests and all +2,756 checkpoint comparisons. Restoring the original classifier reproduces the +same 64 assertion failures; suppressing all unordered updates fails all 200 +native histories. Each of the 5,512 reach cells remains at count 1 in both +mutants and GREEN. All 1,690 neighboring tests pass before and after the edit. +See [the Effect startup follow-up](oracle-reviews/effect-startup-delta.md) for +commands, fingerprints and limits. Arbitrary scheduling, join multiplicity, +ordered repair, same-source overlap and callback-input sanitization remain +outside this bounded closure. Other deferred findings remain open. + +Subsequent checker calibration keeps both callback channels coherent when +injecting a bad update key. The `key-collision` and `key-change` controls now +reject deletion of the interval-continuity judgment itself: the full checker +mutant passes 214 controls/histories and fails those two controls. Repaired +production passed all 216 owner tests and the combined 619 targeted tests at +that historical subject, including subsequently removed Set work. +Native histories and all 5,512 reach cells are unchanged. See the follow-up's +[calibration evidence](oracle-reviews/effect-startup-delta.md#key-interval-checker-calibration) +for the new owner fingerprint and the limits of the continuity claim. + +The subsequent high-review repair extends the same owner with 24 finite +histories crossing scalar/callable/private-field payloads, one source/LEFT join, +initially absent/present membership, and separate/coalesced source transactions. +The independent source snapshots still determine the full delta. Both callback +channels preserve event multiplicity, previous readable payloads, and active key +intervals across initial, sequence, absent, reentry and next-use cuts. All 120 +new reach cells have count 1. On unchanged `d97c1aaf4`, 14 histories fail; +retaining preceding result membership repairs them without relying on payload +equality. The original classifier and a net-count-only classifier fail these +comparisons. This finite extension does not claim exact rich-payload instance +identity, arbitrary nondeterministic projections, changing join keys, provider +schedules or framework behavior. The earlier scalar and ordered-repair limits +remain. See the [Effect membership evidence](oracle-reviews/effect-startup-delta.md#membership-and-coalesced-source-writes). + +Current key-only repair: the metadata owner has **300 tests**, including 60 +callback-profile histories from `query/effect-callback-oracle.ts`. The actual +React driver `packages/react-db/tests/effect-callback-oracle.test.tsx` runs the +same 60 histories, model and comparison. Six payload representations cross +five callback profiles and `skipInitial`. Source-version updates, handler +replacement, exit, reentry, next use and disposal reach 480 cells on each path, +each at count 1. The original 5,512 cells and 120 membership cells remain. +On unchanged `2b3dd8c2d`, 56 core and 40 React histories fail. The repair passes +all **708 targeted tests**. The stale-value mutant survives the original 314 +tests and fails the extended histories at exit; gross-count membership fails +79 comparisons. Unordered Effects now retain keys only and never compare +projected values. Ordered repairs retain their preceding coherent values. +The maintainer explicitly chose preservation of unordered update notifications, +including equal projections, on 2026-10-30. The new histories cover eager core +sources and actual React hook commits; other providers/frameworks, arbitrary +schedules and ordered repair semantics retain their existing limits. See the +[key-only review evidence](oracle-reviews/effect-startup-delta.md#review-of-2b3dd8c2d-key-only-unordered-results). ### RC bound schema mutation receivers (2026-10-28) diff --git a/docs/contributing/oracle-reviews/2026-10-29-pr-2171-deferred-oracles.md b/docs/contributing/oracle-reviews/2026-10-29-pr-2171-deferred-oracles.md index fa6afe5e13..c92603d698 100644 --- a/docs/contributing/oracle-reviews/2026-10-29-pr-2171-deferred-oracles.md +++ b/docs/contributing/oracle-reviews/2026-10-29-pr-2171-deferred-oracles.md @@ -249,3 +249,14 @@ settlement and after it. The original oracle remains active and unchanged; there is no skip, narrowed comparison, retry-away, GREEN, or bug-class closure claim. Its CI RED remains an open baseline defect under this explicit release scope deferral. + +## P09 follow-up (2026-10-29) + +[PR #2301](https://github.com/TanStack/db/pull/2301) restores the exact accepted +Effect owner and repairs its LEFT-join startup delta. Fresh unchanged-production +RED precedes the production edit. The owner passes all 216 tests after repair, +with every one of its 5,512 reach cells at count 1. Original and overcorrection +mutants fail the shared comparisons. The +[versioned follow-up](effect-startup-delta.md) records commands, fingerprints, +neighboring guarantees and the bounded closure. This updates P09 only. All +other deferrals and limits in this release record remain open. diff --git a/docs/contributing/oracle-reviews/effect-startup-delta.md b/docs/contributing/oracle-reviews/effect-startup-delta.md new file mode 100644 index 0000000000..a5c4b38245 --- /dev/null +++ b/docs/contributing/oracle-reviews/effect-startup-delta.md @@ -0,0 +1,569 @@ +# Effect startup delta follow-up + +Current implementation: unordered Effects track keys only and preserve graph +update notifications. The [key-only review repair](#review-of-2b3dd8c2d-key-only-unordered-results) +supersedes the earlier retained-value/comparison designs below. Earlier counts, +fingerprints, commands and design claims apply only to their named subjects. + +## Baseline restoration + +This follow-up restores the accepted Effect metadata owner from +`db38b4e009146af53e46aeeef3206269f5968af8`, without editing its model, grammar, +driver, recorder, comparisons, or reach. Its SHA256 is +`4b738962e30f2b0f55b1894bab1f76334da8d66e6001773bbe24599408bb6130`. +The shared source-metadata model and cleanup support already exist on main. +The owner also rejoins the package's explicit `test:oracles` command. + +The baseline is main `bea84cd4554b7bd49df50cb0c569a34d8b2cd915`. Production was +unchanged in baseline restoration commit `9b03e639f769502dccccff7656ba85ab4ca124e8`. +That commit establishes a separately reviewable instrument before the repair. +No execution result was claimed at restoration. The maintainer subsequently +authorized targeted oracle tests, while retaining the prohibition on full builds. +The dated evidence below supersedes the pending execution status. + +## Law and preserved evidence + +Effect callbacks deliver the query's full before/after delta. An `update` +requires an occurrence that previously entered; a startup-only intermediate +change cannot invent such an occurrence. Explicitly selected source metadata, +previous values, event multiplicity, and active event-key intervals are part of +the comparison. Authority: public `EffectConfig` and `DeltaEvent`, the Effect +guide, `VirtualRowProps`, and the source settlement law. Batch partition is +unconstrained; the check allows empty batches and opaque event keys. + +The reviewed pre-release owner has 200 finite histories across structured and +functional projections, single sources, LEFT joins, and both populated-union +orders. It varies independent optimistic outcomes, confirmations, settlement +order, remote followups, peer exit/reentry, and surviving-row next use. It checks +both batch and named callbacks at every declared action cut, continuing after +a mismatch so startup failure cannot erase later evidence. + +The preserved review records 152 passes and 64 LEFT-join startup REDs, all +5,512 declared lane/channel/cut cells reached, and 2,692 of 2,756 checkpoint +comparisons passing. The failing checkpoint emits an update for a never-entered +unmatched occurrence. Five hostile callback overlays add intended comparison +failures; legal key renaming, fresh reentry keys and empty partitions retain the +baseline. These are historical results, not fresh results on this branch. + +The complete applicable ORC-001 through ORC-016 evaluation, fingerprints, +calibrations, omission attacks and execution commands remain in +[chunk 16 of the accepted campaign record](2026-10-28-pr-2171-oracle-chunks.md#chunk-16-explicit-source-metadata-reaches-effect-deltas-reviewed). +The [release deferral](2026-10-29-pr-2171-deferred-oracles.md) retains the broader +P09–P12 and type-receiver scope. This restoration changes none of those verdicts. + +## Baseline repair plan (completed below) + +Before production changes, run the restored owner on unchanged main and retain +the actual callback mismatch and all per-cell reach. Then run the same owner +after the repair and with the original defect restored as a hostile mutant. +Preserve neighboring single-source/union behavior, both callback channels, +previous values, empty partitions, and later optimistic settlement/next use. + +```sh +TANSTACK_DB_ORACLE_STATISTICS=1 pnpm --filter @tanstack/db exec vitest run \ + tests/query/virtual-row-fields-effect-oracle.test.ts --coverage.enabled=false +``` + +Limits remain eager controlled core sources, scalar projections, one mutation +per source, and the named paths. This supplies no real-provider/framework, +arbitrary scheduling, ordered repair, same-source overlap, or callback-input +sanitization evidence. No GREEN, production repair, or bug-class closure is +claimed by this baseline restoration. + +## Executed repair evidence (2026-10-29) + +Review identity: baseline restoration `9b03e639f769502dccccff7656ba85ab4ca124e8`, +with production still identical to main +`bea84cd4554b7bd49df50cb0c569a34d8b2cd915`. The repaired +`packages/db/src/query/effect.ts` has SHA256 +`b9ea784afe07e156507079224885c73bd21e8633116c3344af809c09971b0879`. +The owner retains the restoration fingerprint above. No model, history, +recorder, comparison, reach requirement, or run budget changed after RED. +This record accompanies the production repair in PR #2301. + +### Law, diagnosis, and repair + +An Effect update describes a changed row that stays in the result. For a +preloaded LEFT join with a matched row and an unmatched anchor, startup owes +only the two entering occurrences. It owes no update for an intermediate +unmatched occurrence that never entered. The public Effect guide's delta +sections and `DeltaEvent.previousValue` establish this judgment independently +of the graph output. + +The original classifier treated any insert plus delete as an unordered update. +Startup emits and retracts an intermediate unmatched row within one graph run. +Its inserted and deleted scalar projections are equal. The callback nevertheless +received that row as an update, followed by the two legitimate enters. +The update's previous and current values were identical, with all right fields +undefined. The independent before/after model rejected the extra event and the +absence of its prior enter in both callback channels. + +The initial repair applied the existing `deepEquals` update condition to +unordered Effects. That design used no retained unordered result rows and +checked only scalar projections. The later membership repair retained values, +and the final key-only repair removes both unordered value retention and +comparison. These historical designs are not current implementation claims. + +The earlier Effect examples lacked this exact startup delta comparison. The +restored owner checks the complete emitted delta and active key intervals, +rather than inferring correctness from final rows or selected event counts. +No missing-input hypothesis remains for this witness: both source Collections +finish preload before Effect creation, and the correct matched and anchor +enters arrive in the same startup ledger as the extra update. + +### Results + +The pre-edit inventory comprises structured and functional projections through +actual core `createEffect`, with onBatch and named handlers. Single sources, +LEFT joins and both populated-union orders receive the same source model. +There is no real-provider or framework claim requiring another receiving host. +The unchanged-production neighboring run preceded the production edit. + +| Run | Passed / failed tests | Declared reach | Passing checkpoint comparisons | +| --- | --- | --- | --- | +| Unchanged production RED | 152 / 64 | All 5,512 cells: count 1 each | 2,692 / 2,756 | +| Repaired owner GREEN | 216 / 0 | All 5,512 cells: count 1 each | 2,756 / 2,756 | +| Original production restored as mutant | 152 / 64 | All 5,512 cells: count 1 each | 2,692 / 2,756 | +| Suppress all unordered updates mutant | 16 / 200 | All 5,512 cells: count 1 each | Later changed-value comparisons fail | +| Final restored repair plus neighboring suites | 1,906 / 0 | Effect cells unchanged: count 1 each | 2,756 / 2,756 Effect comparisons | + +The 200 native histories comprise 8 single-source, 64 LEFT-join and 128 union +histories. The other 16 tests calibrate the shared judgment. Both structured and +functional LEFT-join startup cuts fail in RED. Every later cut continues and +passes, so startup failure does not hide settlement or next-use evidence. +The original mutant restores the entire production file byte-for-byte to the +baseline and reproduces the same startup assertion failures. The second mutant +requires `publishedRows` for updates, wrongly dropping all unordered updates. +It fails the intended changed-value comparisons in all 200 native histories. +Both outcomes are assertion failures at the shared comparison, not setup errors +or timeouts. Neither mutant remains in the source. + +Neighboring suites pass 1,690 tests both before and after the production edit: +`effect.test.ts` (74), `effect-disposal-oracle.test.ts` (12), and +`query/virtual-row-fields-oracle.test.ts` (1,604). These retain skipInitial, +previous values, object-valued Set updates, source failure, ordered Effects, +disposal and root metadata observations within their existing bounds. +The baseline remote DB-1 job also fails the same 64 startup cases: +[run 40930211146](https://github.com/TanStack/db/actions/runs/40930211146/job/123646325441). + +All local tests use Node 24.19.0 and installed Vitest 3.2.4 on macOS arm64. +The manifest requests Vitest 4.1.12. No dependencies or lockfiles changed. +Full builds remain unrun by maintainer instruction. + +Commands run from `packages/db` using the worktree's installed binary: + +```sh +TANSTACK_DB_ORACLE_STATISTICS=1 ../../node_modules/.bin/vitest run \ + --coverage.enabled=false tests/query/virtual-row-fields-effect-oracle.test.ts + +../../node_modules/.bin/vitest run --coverage.enabled=false \ + tests/effect.test.ts tests/effect-disposal-oracle.test.ts \ + tests/query/virtual-row-fields-oracle.test.ts + +TANSTACK_DB_ORACLE_STATISTICS=1 ../../node_modules/.bin/vitest run \ + --coverage.enabled=false tests/query/virtual-row-fields-effect-oracle.test.ts \ + tests/effect.test.ts tests/effect-disposal-oracle.test.ts \ + tests/query/virtual-row-fields-oracle.test.ts +``` + +Each executed command also uses default and JSON reporters. Full logs and JSON +reports remain in `/private/tmp/effect-startup-delta/` under `baseline-red`, +`baseline-neighbors`, `repaired-green`, `repaired-neighbors`, `original-mutant`, +`overcorrection-mutant`, and `final-green`. The finite owner reports every named +reach cell, so the table's count 1 applies to each lane/channel/cut separately. +It does not aggregate away a missing path. No random seed applies to this +exhaustive finite grammar. Running the whole owner reconstructs every witness. + +### Self-review against the oracle rubric + +The accepted chunk-16 audit remains applicable to the unchanged owner. This +follow-up adds executed evidence on the restored baseline and repaired source. +The review here is a self-review, not a fresh independent review. + +| Requirement | Follow-up outcome | +| --- | --- | +| ORC-001 | Public delta law and bounded closure are stated above. | +| ORC-002 | Independent source projection and array differencing remain unchanged. No production classifier computes expected deltas. | +| ORC-003 | Restored literate owner keeps contract, model, grammar, driver and comparison together. | +| ORC-004 | All 200 finite histories run. Accepted reconstruction, ablation, range and exclusion controls remain unchanged. | +| ORC-005 | Actual core Effect callbacks supply both ledgers at every action cut. Preloaded inputs and complete startup output resolve the concrete input-sufficiency question. | +| ORC-006 | Original and overcorrection mutants fail the intended comparisons. All 16 checker controls pass on repaired production. | +| ORC-007 | Not triggered by a random property. This grammar exhausts its finite domain and has direct reconstruction. | +| ORC-008 | No reference-model state changes. Existing source visibility and active key intervals retain their accepted distinguishing actions. | +| ORC-009 | Existing source rows, optimistic state, source origins and active event-key intervals retain their documented mapping. | +| ORC-010 | Primary and cleanup errors remain distinguishable through the shared cleanup wrapper. The owner continues after mismatches and drains after disposal. | +| ORC-011 | No new concrete shared semantic fault requires a second formulation. The matched/unmatched startup result follows directly from the public delta contract. | +| ORC-012 | This versioned record identifies baseline and repaired source fingerprints, results, limits and every requirement. | +| ORC-013 | Equal intermediate startup values distinguish suppression. Later changed values, both settlements and next use reject suppress-all-updates. | +| ORC-014 | No real-provider or framework handoff claim. Sources are controlled core Collections. | +| ORC-015 | Every one of the 5,512 declared cells reaches count 1 in RED, GREEN and both mutant runs. | +| ORC-016 | The restored owner contains shared model/driver comparisons and checker calibrations. No independent expected-answer regression was added. | + +### Closure and remaining limits + +P09 is repaired within this contract × history × production-path × observation +boundary: exact scalar deltas for eager preloaded core Effects, the 200 stated +histories, both projection forms and both callback channels, at every declared +action cut. No known counterexample remains inside those exercised cells. +This does not establish arbitrary join multiplicity or scheduling, changing join +keys, same-source overlap, ordered repair, real providers/frameworks, or +callback-input sanitization. P10, P11/P12 and the other deferred runtime, +retention and public-type findings remain open under their existing owners. +No broader metadata or general delta-classifier closure follows. + +## Subsequent prep-pr review + +Review of `a7ea76cddf7ab2fe03f49d84c6d596dd598fedba` found a newly exposed +factorial object-Set comparison path. The [historical Set-work repair](https://github.com/TanStack/db/blob/0b47ee825aaa0c36b199a3c42652cab8d9672ec9/docs/contributing/oracle-reviews/set-equality-work.md) +records its independent model, pre-edit RED, receiving paths, GREEN and mutants. +The scalar startup owner's native model and histories still pass all 216 cases. +Its separate opaque-key calibration repair is recorded below; startup and work +evidence alone would not close that test-integrity gap. + +### Key-interval checker calibration + +CodeRabbit review `5703351893` on `a7ea76cddf7ab2fe03f49d84c6d596dd598fedba` +identified a compound-fault calibration. The `key-collision` and `key-change` +controls changed only batch keys. Named callbacks retained their original keys, +so cross-channel disagreement rejected the ledger even without the interval +judgment. The checker still enforced continuity; these controls did not isolate +it. ORC-006 requires sensitivity at the intended judgment. + +The maintainer accepted this repair. Both controls now put the same bad key in +the matching named entry. `events()` constructs named entries directly from the +batch, preserving their order. No production behavior, independent model, +native history, assertion, exclusion, run budget or reach requirement changed. + +Before repair, deleting the interval judgment leaves both controls passing: +the checker mutant survives. After coherent mutation, that deletion causes +exactly two expected-to-throw assertion failures. Removing only the key-stability +assertion from the full repaired owner gives **214 passes and those same two +failures**, with zero skipped tests. Cross-channel agreement therefore cannot +substitute for continuity enforcement. + +At that earlier head, the repaired owner passed all **216 tests**. Combined +equality, Collection/Effect receiving, proxy-revert and startup checks passed +**619/619**, including the later-removed Set work. All 5,512 startup +reach cells still count 1; Set-work reach is unchanged. Targeted TypeScript, +ESLint, oracle lint and formatting pass. Tests use at most two workers; no full +build ran. Other applicable rubric outcomes above remain unchanged. + +This is a test-integrity repair with a checker mutant, not a new production RED. +These update faults calibrate continuity for a live occurrence. They do not +independently mutate the distinct-key condition on a fresh enter, or establish +arbitrary graph, provider, framework or rich named-callback coverage. + +The restoration and execution fingerprints above remain historical evidence. +The current owner intentionally changes only key-fault setup and its explanation. +Its SHA256 is `56de9e93892c6a4772f71846e25fc9e3700359b2cadbe3d6d9b359d7b9bcd46f`. +Review identity is parent `a7ea76cddf7ab2fe03f49d84c6d596dd598fedba` plus that +fingerprint and the Set-work fingerprints. The combined command is recorded in +the Set-work review. Raw checker variants, reports and logs remain under +`/private/tmp/effect-startup-delta/prep-pr/key-calibration/accepted-cr1/`. + +## Membership and coalesced source writes + +The high review of `d97c1aaf412661e8d5d3e0d528fe5c4667f69d65` exposed a +remaining membership distinction. An absent key can receive insert/delete/insert +in one flush and must enter. A present key receiving delete/insert/delete must +exit. Fresh callable and private-field projections also exposed a startup update +for an unmatched occurrence that never entered. Presence cannot be inferred from +payload equality or gross insert/delete counts. + +The existing owner adds 24 finite histories: scalar/callable/private payload, +one source/LEFT join, initially absent/present and separate/coalesced writes. +Independent source snapshots determine the whole expected delta at initial, +sequence, absent, reentry and next-use cuts. Both callback channels compare event +multiplicity, previous readable values and active opaque-key intervals. All 120 +new cells reach once. The original 200 histories and 5,512 reach cells stay. +The rich recorder preserves payload kind and readable result, without a claim +about exact re-evaluated instance identity. + +Before production edits, the expanded owner on unchanged `d97c1aaf4` failed +14 of these histories at delta/interval assertions. The complete recorded run +passed 631 tests and failed 16, including two separate Set stack probes. The +membership repair passes all 240 owner tests. Restoring the original classifier +fails 14 histories; inferring prior presence from net count alone fails all 24. +Public sync writes inside a real optimistic transaction's scheduler scope reach +the reported three-write histories. No production information is missing at +these observation cuts. + +Every Effect now retains its preceding coherent result, including a skipped +initial baseline. Prior membership plus net multiplicity decides membership +at the next cut. Only a continued row compares values to decide an update. +An Effect without update/batch handlers refreshes retained values without that +comparison or an update dispatch, preserving its later exit's value. Public +`EffectConfig`, `DeltaEvent` and the Effect guide supply the authority. + +The Effect production and owner files are unchanged from the verified subject +`375e9b26ca93b19cc266dd1ccdfcd10e97e820fc`. Their SHA256 values are: + +```text +6835de1fc044efec14f80ce7aa546451c9c1ce6134a1c5198779ee5ebd45c8b9 packages/db/src/query/effect.ts +469415454de90a75db84caaa5b6893ae8fb54a1f85536872070f9a3c54ded1a6 packages/db/tests/query/virtual-row-fields-effect-oracle.test.ts +``` + +The [exact-head high-review record](https://github.com/TanStack/db/blob/0b47ee825aaa0c36b199a3c42652cab8d9672ec9/docs/contributing/oracle-reviews/effect-startup-high-review.md) +preserves all eight original findings, experiments, hostile variants, grammar +controls, rubric outcomes and reviewer assessment. That historical record's +647-test result includes the now-removed Set work. It is not the current count. +Current claims remain bounded to the declared eager controlled core histories. +Arbitrary provider/framework schedules, changing join keys/multiplicity and +exact rich-payload instance identity remain outside this instrument. + +## Maintainer scope revision, 2026-10-30 + +The maintainer explicitly removed Set performance work from this PR because no +application performance problem had been reported. The old Set matcher returns +byte-for-byte, together with the existing `utils.1`–`utils.40` owner and registry. +New Set models, receivers, properties 41/42, scale/work guards and the dedicated +Set report are removed. Existing laws, comparisons, reach and budgets remain. +The Set work's own new guards retire only with this explicit scope decision. +Their known synthetic limitations remain recorded under the primary deep-equality +owner in the coverage map; resumption requires a maintainer request or a real +workload. Historical generators and full traces remain in commit `0b47ee825` and +`/private/tmp/effect-startup-delta/high-review/`. + +The high-review ledger now has three Effect repairs (1, 2, 4), one accepted-design +semantic finding with a release-note correction (3), three explicitly deferred +Set findings (5–7), and one refuted coverage allegation (8). The original +factorial Set work is deferred with those findings. None is relabeled refuted or +closed by removing its test. The Effect membership repair, checker calibration, +callback comparison bypass and equal-projection suppression remain intact. + +### Validation after Set removal + +The six retained suites pass **581/581**, with zero failures or skipped tests. +They cover the existing change-event equality owner, ordinary utils tests, +proxy revert, ordinary Effects, Effect disposal and the Effect metadata owner. +All 240 metadata-owner tests pass. The original 5,512 reach cells and 120 added +membership cells each reach count 1. Targeted TypeScript, ESLint, oracle lint, +formatting and whitespace checks pass. + +The executed local configuration uses the installed dependencies, jsdom, +`tests/test-setup.ts`, a source alias for `@tanstack/db-ivm`, no coverage, and +at most two workers: + +```sh +node node_modules/vitest/vitest.mjs run \ + --config /private/tmp/effect-startup-delta/remove-set/focused.config.mjs \ + --reporter=default --reporter=json \ + --outputFile=/private/tmp/effect-startup-delta/remove-set/focused.json +``` + +The config, log and JSON report remain under that directory. The source alias +changes dependency resolution only. It does not replace the production driver +or independent comparison. The normal repository selectors name the same six +test files. Every `test:oracles` path exists after the removed receiver leaves +the script. + +The maintainer separately authorized `pnpm size:compare`, including its normal +package builds. Against merge base `bea84cd4554b7bd49df50cb0c569a34d8b2cd915`, +the typical app remains **96,732 bytes gzip** (0-byte change). The Effects app +changes from **99,303 to 99,340 bytes gzip** (+37 bytes). All other app scenarios +remain unchanged. The DB package adds 20 bytes gzip. The log is +`/private/tmp/effect-startup-delta/remove-set/size-compare.log`. + +These results supersede the earlier combined 619/647 counts for the current +scope. Those counts remain historical evidence for their stated subjects. +The Effect source and owner fingerprints above still identify the verified +implementation. Set production, its former `utils.1`–`utils.40` owner and its +registry are byte-identical to the merge base. No full repository test or build +ran beyond the separately authorized size comparison's normal package builds. + +## Review of 2b3dd8c2d: key-only unordered results + +The ten-item review targets `2b3dd8c2d0a0f98a858dc1d08182f6673c9a7836`. +It supplied no executed tests or builds. This evaluation is a self-review. +The maintainer explicitly chose: "Preserve existing notifications; use keys." +That decision supersedes the earlier general equal-projection suppression +policy recorded above. No oracle expectation changed to match production. +The new model ran RED before production changed. + +### Law, experiment and inference + +`EffectConfig`, `DeltaEvent` and the live-query guide promise complete delta +values at each callback cut. Handler subsets do not change an exit's current +value or an update's pre-batch value. React promises current callbacks without +remounting. The maintainer's decision preserves unordered graph notifications, +including equal projections, without adding value comparison or retention. + +The primary owner adds a shared companion, `effect-callback-oracle.ts`, and an +actual React receiving driver. Sixty finite histories on each path cross: + +- Scalar, keyless opaque class, keyed private-field class, prototype method over + non-enumerable state, object Set and equal scalar projection. +- All callbacks, enter only, exit only, late update and late batch callbacks. +- Initial delivery and `skipInitial`. + +Authored scalar source versions determine expected public readable payloads. +Classes and Sets are constructed only in the actual functional projection, +so source-Collection comparison cannot mask or create the reported output cost. +The model never imports production equality or its delta classifier. Both paths +compare initial delivery, an unobserved update, callback replacement, another +update, exit, reentry, next use and disposal. Source updates while absent are +excluded. Removing any profile loses its callback premise; removing any payload +loses its specific comparison boundary. All combinations run, and every one of +480 declared cells on each receiving path reaches once. Original 200/24 +histories and their 5,512/120 cells remain unchanged. + +The recorder reads actual public payloads, including private-state accessors, +and retains event multiplicity and previous values. Synthesized Set getters +count reads outside the recorder. The restored unordered path traverses no +payload leaves, for any callback profile. This bounds added traversal in these +histories, not Set runtime or arbitrary graphs. The original metadata histories +still check opaque-key intervals and both channels independently. + +| Experiment | Result | +| --- | --- | +| Expanded tests on unchanged reviewed production | 264 pass, 96 intended failures: 56 core and 40 React. Original 240 tests pass. | +| Repaired core owner and React receiver | 360 pass, zero failures/skips. | +| Final eight targeted suites | 708 pass, zero failures/skips. | +| Reviewed classifier with refresh skipped for unobserved updates, older tests only | 314 pass; the 120 new histories are deliberately unselected for this diagnostic. The mutant survives. | +| Same stale-value mutant, expanded tests | 338 pass, 96 failures. An exit reports version 0 instead of current version 2. | +| Gross-count prior-membership mutant (`wasPresent = deletes > 0`) | 355 pass, 79 intended callback/membership failures. | + +All ordinary runs and full mutant runs preserve per-path reach. The diagnostic +old-test selection is not a normal GREEN claim. Initial React module resolution +failed before collection; explicit aliases to the already-installed React 19.2.4 +fixed that setup before the valid pre-production RED. No setup failure counts +as a product or mutant result. + +The final repair retains only keys for unordered Effects. The first deletion +in a batch supplies its previous/exiting value. Ordered Effects still retain +coherent values across held graph runs and use their existing comparison. +`observesUpdates` and its creation-time snapshot are removed. Dynamic callback +lookup remains live. The redundant value fallbacks are removed; `deleteValue` +is now necessary for unordered delivery rather than dead state. + +A deterministic cardinality probe at 8 and 64 rows confirms that reviewed +unordered Effects added one Map entry per result, holding the same row objects +as callbacks. It did not clone the rows. Disposal cleared that Map. The repaired +path retains no output values in a Map. No total-memory multiplier was measured. + +### Lossless disposition + +| ID | Technical verdict | PR action and durable value | +| --- | --- | --- | +| R2-01 | Extra per-row Map retention confirmed. "Full copy" and roughly double total memory are unsupported: rows are shared references. | Fixed now with key-only unordered tracking for every handler profile, rather than conditional retained values. | +| R2-02 | Confirmed new comparison path. Scalar sources that synthesize object Sets isolate it from source comparison. | Fixed now: unordered Effects never call the comparator. The removed Set rewrite stays removed. | +| R2-03 | Confirmed: React always supplies wrappers. The claimed unordered comparison bypass did not apply there. | Fixed now through the core repair and real React receiving witness. Existing ordered comparison is unchanged. | +| R2-04 | Confirmed test gap and mutant survival. | Fixed now with callback profiles, replacement, current exit values and a demonstrated stale-value mutant rejection. The obsolete optimization branch is removed. | +| R2-05 | Partially confirmed. Keyless private instances already compare by identity and notify. Keyed instances with constant enumerable fields can hide changed private/non-enumerable state and retain stale values. | Fixed now on the introduced unordered path. The grammar keeps both failing and non-failing representations. No new general class-equality contract is imposed on other callers. | +| R2-06 | Confirmed: the initial observer snapshot prevents later core update/batch installation. | Fixed now by removing the snapshot; actual core config mutation and React replacement both run through the shared comparison. | +| R2-07 | Dead fallbacks are confirmed at the reviewed Map-only subject. Deleting the accumulator field after the key-only repair would instead lose previous/exiting values. | Fixed now by removing redundant fallbacks and restoring the field's required role. No field removal is claimed. | +| R2-08 | Duplicate retention remedy of R2-01. "Wrong layer" is a design opinion, not a demonstrated contract violation. Prior membership is needed beyond a raw gross-count classification. | Duplicate of R2-01; keep this separate ID and its key-only suggestion. The maintainer selected the narrower design in the Effect boundary. | +| R2-09 | Confirmed ambiguous/stale evidence. | Fixed now: historical subjects are explicit, and current owner/receiver counts and implementation are separate. | +| R2-10 | Local-only traces are a durability limit. Squash does not automatically remove GitHub commit blobs: both this PR's earlier blob and merged #2299's pre-squash blob remain accessible. Future retention is not guaranteed by that observation. | Fixed now by retaining essential replay inputs below and finite histories in current versioned sources. Historical blob links are supplementary, and local logs are convenience artifacts. | + +Accounting: **10 = 9 fixed-now + 1 duplicate**. Each technical qualification +above survives its action verdict. No new item is deferred. Previously removed +Set engineering work retains the maintainer's explicit scope decision and its +primary equality owner; resumption requires a maintainer request or an actual +workload. Removing the new unordered comparator caller resolves the new +interaction without reopening that optimization project. + +Reviewer assessment: **lean hire for static defect discovery, with verification +required before approval**. Callback profiles, live replacement and React +wrappers expose important missing cases. The memory multiplier, blanket private +class claim, unconditional link-loss claim and field-deletion suggestion needed +qualification. The proposed membership-only direction was useful. Severity and +closure require executed evidence. + +### Oracle review and remaining limits + +This is bounded enforcement of unordered membership, notification compatibility +and callback-visible values for the declared eager source histories and actual +React commits. No known violation remains within these cells. It does not prove +arbitrary scheduling, provider delivery, changing join multiplicity, other +frameworks, ordered opaque-value semantics or exact rich-instance identity. +Other comparator callers retain their previously documented domains and limits. + +| Requirement | Current evidence | +| --- | --- | +| ORC-001 | Public delta contracts and explicit maintainer notification decision above. | +| ORC-002 | Authored source-version model; no production comparator/classifier import. | +| ORC-003 | Companion explains contract, model, grammar, drivers and comparisons beside code. | +| ORC-004 | Full finite crossing reconstructs the reports, states axis ablations and bounds, and excludes updates of absent sources. | +| ORC-005 | Core callbacks and actual React hook; current readable values checked after each host/act cut. Scalar source facts arrive before projection. | +| ORC-006 | Stale-value and gross-count mutants fail intended exit/value/membership comparisons. The former survives the old tests. | +| ORC-007 | Not triggered: bounded finite histories, no random property or new run budget. | +| ORC-008 | No stateful reference-state expansion; one authored prior source version suffices for public delta values. | +| ORC-009 | Source version abstracts readable selected payload, with the mapping stated beside model and projection. | +| ORC-010 | Per-cut mismatches accumulate; shared cleanup preserves primary failure and separate cleanup diagnostics. Disposal has its own cut. | +| ORC-011 | No unresolved shared semantic fault is identified for this version-to-readable-payload relation. Other formulations remain outside the bounded claim. | +| ORC-012 | Reviewed head, exact fingerprints, experiments, all IDs and every rubric requirement are versioned here. | +| ORC-013 | Equal versus changed output, keyless versus keyed opaque state, callback presence/replacement and exit after unobserved updates distinguish neighboring wrong designs. | +| ORC-014 | Actual React renderHook/act supplies the callback-replacement premise; each path has separate reach. No other framework/provider claim. | +| ORC-015 | 480 cells per new path, each 1; original 5,512 and added membership 120 remain each 1. | +| ORC-016 | Finite histories use shared model/driver/comparison without per-case expected answers. Scratch retention probes are separate diagnostics. | + +The final run selects utils oracle, utils, proxy revert, Effect, Effect disposal, +Effect metadata, React Effect callbacks and ordinary React Effect tests. It uses +jsdom, existing test setup, source aliases for DB/IVM, installed React 19.2.4, +Vitest 3.2.4, no coverage, and at most two workers. Standard selectors are: + +```sh +pnpm --filter @tanstack/db exec vitest run --coverage.enabled=false --maxWorkers=2 \ + tests/utils-oracle.property.test.ts tests/utils.test.ts \ + tests/proxy-revert-oracle.property.test.ts tests/effect.test.ts \ + tests/effect-disposal-oracle.test.ts tests/query/virtual-row-fields-effect-oracle.test.ts +pnpm --filter @tanstack/react-db exec vitest run --coverage.enabled=false --maxWorkers=2 \ + tests/effect-callback-oracle.test.tsx tests/useLiveQueryEffect.test.tsx +``` + +Local execution uses the same selectors in the source-alias config +`/private/tmp/effect-startup-delta/review-2b3dd8c/final.config.mjs`. Reports, +commands, original/mutant snapshots and the append-only raw review ledger remain +in that directory. These local files are supplementary to the versioned +histories, recipe and comparison. Targeted TypeScript and ESLint pass. +The authorized `pnpm size:compare` reports typical app 96,732→96,732 bytes gzip, +Effects app 99,303→99,309 (+6), and DB bundle 109,667→109,668 (+1), against merge +base `bea84cd45`. Other app scenarios are unchanged. No full test/build ran +beyond the size script's normal package builds. + +Exact subject: reviewed parent plus the following final source fingerprints. +Documentation-only changes do not alter these executable subjects. + +```text +354bbbcf9683c91047c61da1ea2323ed89623628eb6059dacfbb79bd56b85ce4 packages/db/src/query/effect.ts +47e862b092e643309bd804acab7c61c036ed7a345c873ca480f63b807aa4f24d packages/db/tests/query/virtual-row-fields-effect-oracle.test.ts +ca9e1336e7ceeb243a8e32a49ad2c3c881deaa38334540e85f848122cbaf2d38 packages/db/tests/query/effect-callback-oracle.ts +9ed573ae14bf9bd87058c9136ef1de57f33422da235f6253a9c97f90599cf98e packages/react-db/tests/effect-callback-oracle.test.tsx +``` + +### Deferred Set replay inputs + +These inputs preserve the earlier synthetic findings without reinstating the +Set rewrite, test campaigns or work budgets. No application incident or new +latency guarantee follows. The existing `utils-oracle.property.test.ts` owner +remains responsible for change-event equality; the maintainer controls whether +to resume the excluded engineering work. + +The factorial-work witness uses distinct object members. For size `n`, compare +`new Set(Array.from({length:n}, () => ({value:0})))` with a Set of `n-1` such +objects followed by `{value:1}`. The equality judgment is false. Instrument an +enumerable `value` getter to count reads. The earlier direct `n=4` experiment +observed 128 reads, against its proposed bound of 32. The eight-member Effect +receiver observed 219,200 reads against 1,296 while still returning the correct +equality result. Those engineering bounds are historical and retired. + +The stack witness uses `n=2**13` words of 13 coordinates. Let `g(j)=j^(j>>1)`. +For right word `j`, coordinate `k` is a distinct instance of class First or +Second according to bit `k` of `g(j)`, with one enumerable `value:0` field. +Two different classes differ; a plain `{value:0}` can match either class. +Left word `i (i+1)%n`, so equality is true. To create the false neighbor, change every +coordinate value in the last right word to 1, leaving an isolated unmatched +word. The earlier recursive matcher overflowed on both witnesses after setup. +This is a runtime failure, not an equality-assertion rejection. Smaller generated +versions used 2–6 bits and optional one-place right rotation. A recorded greedy +mutant replay used `utils.42`, seed `-75265719`, shrink path `1`; that retired +selector alone does not run on the current registry. The structural recipe +above supplies the inputs independently of the removed generator. + +Historical reports at `0b47ee825aaa0c36b199a3c42652cab8d9672ec9` retain broader +experiments. Their links are supplementary; the current source tree now holds +these essential distinguishing inputs, observations and limits. diff --git a/docs/guides/live-queries.md b/docs/guides/live-queries.md index ba264a1846..65b63485b9 100644 --- a/docs/guides/live-queries.md +++ b/docs/guides/live-queries.md @@ -2450,6 +2450,8 @@ interface DeltaEvent { | `exit` | Row left the query result | The exiting row | — | | `update` | Row changed but stayed in the result | The new row | The row before the change | +Unordered Effects classify graph deltas using result membership. They keep result keys without retaining projected rows or comparing projected values. A graph update can therefore notify handlers even when the projected fields compare equal. Ordered Effects retain the last coherent result for repair across graph runs and suppress equal-value updates. Callback presence does not change the delta values. + ### Named Callbacks Use the callback that matches the query-result transition you care about: diff --git a/packages/db/package.json b/packages/db/package.json index 05e430c107..1b8a299d76 100644 --- a/packages/db/package.json +++ b/packages/db/package.json @@ -23,7 +23,7 @@ "test": "vitest --run", "test:dist": "vitest run --config vitest.dist.config.ts", "test:facade-retention": "node --expose-gc --import tsx tests/facade-retention.probe.ts", - "test:oracles": "vitest --run --coverage.enabled=false tests/reverse-index-oracle.property.test.ts tests/query/compiler/in-evaluator-oracle.property.test.ts tests/cleanup-queue-oracle.property.test.ts tests/comparison-oracle.property.test.ts tests/duration-query-equality-oracle.test.ts tests/cursor-oracle.property.test.ts tests/index-update-oracle.property.test.ts tests/utils-oracle.property.test.ts tests/change-event-history-oracle.test.ts tests/index-suggestion-oracle.test.ts tests/btree-map-oracle.test.ts tests/paced-mutations-oracle.test.ts tests/db-client-hydration-authority-oracle.test.ts tests/local-storage-order-oracle.test.ts tests/local-storage-value-fidelity-oracle.property.test.ts tests/collection-cleanup-restart-oracle.test.ts tests/collection-gc-transaction-oracle.property.test.ts tests/collection-gc-retention-oracle.property.test.ts tests/collection-gc-mutation-oracle.property.test.ts tests/effect-disposal-oracle.test.ts tests/optimistic-transaction-oracle.property.test.ts tests/transaction-ownership-oracle.property.test.ts tests/optimistic-settlement-boundaries-oracle.test.ts tests/optimistic-history-publication-oracle.test.ts tests/optimistic-history-outcomes-oracle.test.ts tests/collection-metadata-publication-oracle.property.test.ts tests/collection-index-value-oracle.test.ts tests/collection-state-retention-oracle.property.test.ts tests/collection-subscription-lifecycle-history-oracle.property.test.ts tests/collection-subscription-lifecycle-oracle.test.ts tests/collection-subscription-reentrancy-oracle.test.ts tests/collection-subscription-lifecycle-publication-oracle.property.test.ts tests/collection-subscription-replay-oracle.property.test.ts tests/d2-source-reconciliation-oracle.property.test.ts tests/live-query-cache-options-oracle.test.ts tests/mutation-confirmation-oracle.test.ts tests/mutation-handler-types-oracle.test.ts tests/live-query-observer-history-oracle.property.test.ts tests/live-query-observer-hydration-oracle.property.test.ts tests/live-query-observer-snapshot-oracle.property.test.ts tests/query/cold-join-reconciliation-oracle.test.ts tests/query/identity-output-shape-oracle.test.ts tests/query/optimizer-semantics-oracle.test.ts tests/query/index-path-collision-oracle.test.ts tests/query/custom-index-order-selection-oracle.test.ts tests/query/mixed-index-order-oracle.test.ts tests/query/includes-collection-oracle.property.test.ts tests/query/includes-functional-projection-oracle.test.ts tests/query/includes-functional-input-boundary.test.ts tests/query/includes-context-transport-oracle.test.ts tests/query/includes-cross-formulation-oracle.property.test.ts tests/query/includes-optimistic-oracle.property.test.ts tests/query/includes-oracle.property.test.ts tests/query/includes-alias-shadowing-oracle.test.ts tests/query/includes-outer-join-route-work-oracle.test.ts tests/query/includes-publication-oracle.test.ts tests/query/includes-query-shape-oracle.test.ts tests/query/subquery-user-value-oracle.test.ts tests/query/includes-temporal-oracle.test.ts tests/query/includes-work-counter-oracle.test.ts tests/query/load-subset-oracle.property.test.ts tests/query/load-subset-replay-refinement-oracle.test.ts tests/query/load-subset-source-readiness-refinement-oracle.test.ts tests/query/load-subset-transaction-refinement-oracle.test.ts tests/query/ordered-source-loader-state-oracle.test.ts tests/query/ordered-demand-retirement.test.ts tests/query/ordered-default-work.test.ts tests/query/ordered-lifecycle-oracle.property.test.ts tests/query/ordered-work-oracle.property.test.ts tests/query/pagination-oracle.property.test.ts tests/query/includes-space-oracle.test.ts tests/query/virtual-row-fields-oracle.test.ts tests/query/virtual-row-fields-types-oracle.test.ts tests/query/where-predicate-publication-oracle.property.test.ts tests/query/join-result-key-oracle.property.test.ts tests/query/pooled-live-query-oracle.property.test.ts tests/live-query-deferred-acquisition-oracle.test.ts tests/flat-change-tracking-oracle.property.test.ts tests/truncate-readiness-oracle.test.ts tests/proxy-detachment-contract-oracle.test.ts tests/production-error-messages-oracle.test.ts tests/query/bucket-facade-rollback-oracle.property.test.ts tests/query/aggregate-select-oracle.property.test.ts tests/query/load-subset-joint-replay-oracle.property.test.ts", + "test:oracles": "vitest --run --coverage.enabled=false tests/reverse-index-oracle.property.test.ts tests/query/compiler/in-evaluator-oracle.property.test.ts tests/cleanup-queue-oracle.property.test.ts tests/comparison-oracle.property.test.ts tests/duration-query-equality-oracle.test.ts tests/cursor-oracle.property.test.ts tests/index-update-oracle.property.test.ts tests/utils-oracle.property.test.ts tests/change-event-history-oracle.test.ts tests/index-suggestion-oracle.test.ts tests/btree-map-oracle.test.ts tests/paced-mutations-oracle.test.ts tests/db-client-hydration-authority-oracle.test.ts tests/local-storage-order-oracle.test.ts tests/local-storage-value-fidelity-oracle.property.test.ts tests/collection-cleanup-restart-oracle.test.ts tests/collection-gc-transaction-oracle.property.test.ts tests/collection-gc-retention-oracle.property.test.ts tests/collection-gc-mutation-oracle.property.test.ts tests/effect-disposal-oracle.test.ts tests/optimistic-transaction-oracle.property.test.ts tests/transaction-ownership-oracle.property.test.ts tests/optimistic-settlement-boundaries-oracle.test.ts tests/optimistic-history-publication-oracle.test.ts tests/optimistic-history-outcomes-oracle.test.ts tests/collection-metadata-publication-oracle.property.test.ts tests/collection-index-value-oracle.test.ts tests/collection-state-retention-oracle.property.test.ts tests/collection-subscription-lifecycle-history-oracle.property.test.ts tests/collection-subscription-lifecycle-oracle.test.ts tests/collection-subscription-reentrancy-oracle.test.ts tests/collection-subscription-lifecycle-publication-oracle.property.test.ts tests/collection-subscription-replay-oracle.property.test.ts tests/d2-source-reconciliation-oracle.property.test.ts tests/live-query-cache-options-oracle.test.ts tests/mutation-confirmation-oracle.test.ts tests/mutation-handler-types-oracle.test.ts tests/live-query-observer-history-oracle.property.test.ts tests/live-query-observer-hydration-oracle.property.test.ts tests/live-query-observer-snapshot-oracle.property.test.ts tests/query/cold-join-reconciliation-oracle.test.ts tests/query/identity-output-shape-oracle.test.ts tests/query/optimizer-semantics-oracle.test.ts tests/query/index-path-collision-oracle.test.ts tests/query/custom-index-order-selection-oracle.test.ts tests/query/mixed-index-order-oracle.test.ts tests/query/includes-collection-oracle.property.test.ts tests/query/includes-functional-projection-oracle.test.ts tests/query/includes-functional-input-boundary.test.ts tests/query/includes-context-transport-oracle.test.ts tests/query/includes-cross-formulation-oracle.property.test.ts tests/query/includes-optimistic-oracle.property.test.ts tests/query/includes-oracle.property.test.ts tests/query/includes-alias-shadowing-oracle.test.ts tests/query/includes-outer-join-route-work-oracle.test.ts tests/query/includes-publication-oracle.test.ts tests/query/includes-query-shape-oracle.test.ts tests/query/subquery-user-value-oracle.test.ts tests/query/includes-temporal-oracle.test.ts tests/query/includes-work-counter-oracle.test.ts tests/query/load-subset-oracle.property.test.ts tests/query/load-subset-replay-refinement-oracle.test.ts tests/query/load-subset-source-readiness-refinement-oracle.test.ts tests/query/load-subset-transaction-refinement-oracle.test.ts tests/query/ordered-source-loader-state-oracle.test.ts tests/query/ordered-demand-retirement.test.ts tests/query/ordered-default-work.test.ts tests/query/ordered-lifecycle-oracle.property.test.ts tests/query/ordered-work-oracle.property.test.ts tests/query/pagination-oracle.property.test.ts tests/query/includes-space-oracle.test.ts tests/query/virtual-row-fields-oracle.test.ts tests/query/virtual-row-fields-effect-oracle.test.ts tests/query/virtual-row-fields-types-oracle.test.ts tests/query/where-predicate-publication-oracle.property.test.ts tests/query/join-result-key-oracle.property.test.ts tests/query/pooled-live-query-oracle.property.test.ts tests/live-query-deferred-acquisition-oracle.test.ts tests/flat-change-tracking-oracle.property.test.ts tests/truncate-readiness-oracle.test.ts tests/proxy-detachment-contract-oracle.test.ts tests/production-error-messages-oracle.test.ts tests/query/bucket-facade-rollback-oracle.property.test.ts tests/query/aggregate-select-oracle.property.test.ts tests/query/load-subset-joint-replay-oracle.property.test.ts", "bench:nested-includes": "vitest bench tests/query/includes-performance.bench.ts --run", "test:indexed-db:package": "pnpm --filter @tanstack/db... build && vitest run --config vitest.indexed-db.package.config.ts", "test:collection-gc:browser": "playwright test --config playwright.collection-gc.config.ts", diff --git a/packages/db/src/query/effect.ts b/packages/db/src/query/effect.ts index 5629064a7f..4355c7a753 100644 --- a/packages/db/src/query/effect.ts +++ b/packages/db/src/query/effect.ts @@ -483,9 +483,9 @@ class EffectPipelineRunner { // Output accumulator private pendingChanges: Map> = new Map() - // Callback-visible result. Held repair deltas are classified against this - // last coherent membership rather than their net multiplicity alone. - private readonly publishedRows = new Map() + // Unordered deltas need prior membership, not retained values. Ordered + // repairs additionally need the preceding coherent values across graph runs. + private publishedRows: Set | Map = new Set() // skipInitial state private readonly skipInitial: boolean @@ -742,6 +742,7 @@ class EffectPipelineRunner { // For ordered aliases with an index, trigger the initial limited snapshot. // This loads only the top N rows rather than the entire collection. if (orderByInfo) { + this.publishedRows = new Map() const loader = new OrderedSourceLoader( orderByInfo, subscription, @@ -1059,7 +1060,6 @@ class EffectPipelineRunner { } const shouldPublish = !this.skipInitial || this.initialLoadComplete - const tracksPublishedRows = this.orderedLoaders.size > 0 const events: Array> = [] @@ -1067,13 +1067,13 @@ class EffectPipelineRunner { const event = classifyDelta( key as TKey, changes, - tracksPublishedRows ? this.publishedRows : undefined, + this.publishedRows, ) if (event) { - if (tracksPublishedRows) { - if (event.type === `exit`) this.publishedRows.delete(key) - else this.publishedRows.set(key, event.value) - } + if (event.type === `exit`) this.publishedRows.delete(key) + else if (this.publishedRows instanceof Map) + this.publishedRows.set(key, event.value) + else this.publishedRows.add(key) if (shouldPublish) events.push(event) } } @@ -1319,18 +1319,17 @@ function accumulateEffectChanges( return acc } -/** Interpret a D2 batch against callback-visible state when a window holds it. */ +/** Interpret a D2 batch against the preceding coherent result membership. */ function classifyDelta( key: TKey, changes: EffectChanges, - publishedRows?: ReadonlyMap, + publishedRows: ReadonlySet | ReadonlyMap, ): DeltaEvent | undefined { const { inserts, deletes, insertValue, deleteValue } = changes - const wasPresent = publishedRows ? publishedRows.has(key) : deletes > 0 - const previousValue = publishedRows?.get(key) ?? deleteValue - const isPresent = publishedRows - ? Number(wasPresent) + inserts - deletes > 0 - : inserts > 0 + const wasPresent = publishedRows.has(key) + const previousValue = + publishedRows instanceof Map ? publishedRows.get(key) : deleteValue + const isPresent = Number(wasPresent) + inserts - deletes > 0 if (!wasPresent && isPresent) { // Row entered the query result @@ -1340,16 +1339,17 @@ function classifyDelta( if (wasPresent && !isPresent) { // Row exited the query result — value is the exiting value, // previousValue is omitted (it would be identical to value) - return { type: `exit`, key, value: previousValue ?? deleteValue! } + return { type: `exit`, key, value: previousValue! } } if ( wasPresent && isPresent && insertValue !== undefined && - (!publishedRows || !deepEquals(previousValue, insertValue)) + (publishedRows instanceof Set || !deepEquals(previousValue, insertValue)) ) { - // Row updated within the query result + // Unordered Effects preserve all graph updates without comparing values. + // Ordered repairs compare with the preceding coherent publication. return { type: `update`, key, @@ -1358,8 +1358,7 @@ function classifyDelta( } } - // The row was absent both before and after the held delta, or its final - // value is equal to the last callback-visible value. + // The row was absent at both cuts, or an ordered repair kept an equal value. return undefined } diff --git a/packages/db/tests/query/effect-callback-oracle.ts b/packages/db/tests/query/effect-callback-oracle.ts new file mode 100644 index 0000000000..333a98959e --- /dev/null +++ b/packages/db/tests/query/effect-callback-oracle.ts @@ -0,0 +1,305 @@ +/** + * Callback profiles must not change the query's delta values. EffectConfig and + * the Effect guide promise the current exiting value and the pre-batch update + * value. React additionally promises the latest callbacks without remounting. + * This extends the metadata owner's observation to handler subsets and values + * whose readable state is absent from enumerable fields. + * + * An authored scalar source version independently predicts each readable + * payload. The real query projection creates classes/Sets. The driver invokes + * production. The finite grammar crosses payload representation, callback profile and initial + * suppression. Every history updates before and after callback replacement, + * then exits, reenters and updates again. An update of an absent source is not + * legal. These eager single-source histories do not cover ordered repairs or + * arbitrary provider schedules. Both receiving drivers use this same judgment. + * The maintainer's 2026-10-30 decision preserves unordered notifications even + * for equal projections and permits key-only membership tracking. A synthesized + * Set counts leaf reads outside observation capture: these unordered Effects + * must not add traversal of projected values. This is not a Set latency law. + */ +import { expect } from 'vitest' +import { declareOracle } from '../oracle-declaration.js' +import { createOracleReach } from '../oracle-reach.js' +import { checkWithCleanup } from './virtual-row-fields-support.js' +import type { DeltaEvent, EffectConfig } from '../../src/query/effect.js' + +export type CallbackRow = { id: number; version: number } +export type CallbackOutput = { id: number; payload: unknown } +type Representation = + `scalar` | `opaque` | `private` | `prototype` | `set` | `equal` +export const callbackHistories = ( + [`scalar`, `opaque`, `private`, `prototype`, `set`, `equal`] as const +).flatMap((representation) => + ([`all`, `enter`, `exit`, `late-update`, `late-batch`] as const).flatMap( + (profile) => + [false, true].map((skipInitial) => ({ + representation, + profile, + skipInitial, + })), + ), +) +export type CallbackHistory = (typeof callbackHistories)[number] +export type CallbackHandlers = Pick< + EffectConfig, + `onEnter` | `onUpdate` | `onExit` | `onBatch` +> +export interface CallbackDriver { + mount: (options: { + skipInitial: boolean + project: (row: CallbackRow) => CallbackOutput + handlers: CallbackHandlers + }) => Promise<{ + write: ( + type: `insert` | `update` | `delete`, + version: number, + ) => Promise + replaceHandlers: (handlers: CallbackHandlers) => Promise + dispose: () => Promise + cleanup: () => Promise + }> +} +type Observation = { + channel: string + type: string + value: string + previous?: string +} +const cuts = [ + `initial`, + `unobserved`, + `replacement`, + `update`, + `exit`, + `reentry`, + `next-use`, + `disposed`, +] +export function callbackHistoryId(h: CallbackHistory) { + return `${h.representation}/${h.profile}/${h.skipInitial}` +} +export function callbackReach(path: string) { + return createOracleReach( + `Effect callback delivery ${path}`, + callbackHistories.flatMap((h) => + cuts.map((cut) => `${callbackHistoryId(h)}/${cut}`), + ), + { replaying: false }, + ) +} + +class PrivatePayload { + #version: number + constructor(version: number) { + this.#version = version + } + read() { + return this.#version + } +} +class PrototypePayload { + currency = `USD` + constructor(version: number) { + Object.defineProperty(this, `version`, { value: version }) + } + declare version: number + read(): number { + return this.version + } +} +class KeyedPrivatePayload extends PrivatePayload { + currency = `USD` +} + +// Full source-version recomputation defines readable values, not deepEquals or +// the production membership classifier. Sets retain three authored members. +function expectedPayload( + representation: Representation, + version: number, +): string { + return representation === `set` + ? `0,0,${version}` + : representation === `equal` + ? `0` + : String(version) +} +function readPayload(payload: unknown): string { + if (payload instanceof PrivatePayload || payload instanceof PrototypePayload) + return String(payload.read()) + if (payload instanceof Set) + return [...payload] + .map((v: { value: number }) => v.value) + .sort() + .join(`,`) + return String(payload) +} +function expectedDelivery( + representation: Representation, + before: number | undefined, + after: number | undefined, + channels: ReadonlySet, + suppressed: boolean, +): Array { + if (suppressed || before === after) return [] + const type = + before === undefined ? `enter` : after === undefined ? `exit` : `update` + const event = { + type, + value: expectedPayload(representation, after ?? before!), + ...(type === `update` + ? { previous: expectedPayload(representation, before!) } + : {}), + } + return [...channels] + .filter((c) => c === `batch` || c === type) + .map((channel) => ({ channel, ...event })) +} +function checkDelivery( + actual: Array, + expected: Array, +) { + const order = (a: Observation, b: Observation) => + a.channel.localeCompare(b.channel) + expect([...actual].sort(order)).toStrictEqual([...expected].sort(order)) +} +declareOracle({ + model: [expectedPayload, expectedDelivery], + grammar: [callbackHistories], + check: [checkDelivery], +}) + +export async function runCallbackHistory( + history: CallbackHistory, + driver: CallbackDriver, + reach: ReturnType, +) { + reach.beginHistory() + const observed: Array = [] + let recording = false + let comparisonReads = 0 + let channels = new Set( + history.profile === `all` + ? [`enter`, `update`, `exit`, `batch`] + : history.profile.startsWith(`late`) + ? [`exit`] + : [history.profile], + ) + const record = (channel: string, event: DeltaEvent) => { + recording = true + try { + observed.push({ + channel, + type: event.type, + value: readPayload(event.value.payload), + ...(event.type === `update` + ? { previous: readPayload(event.previousValue.payload) } + : {}), + }) + } finally { + recording = false + } + } + const handlers = (): CallbackHandlers => ({ + onEnter: channels.has(`enter`) ? (e) => record(`enter`, e) : undefined, + onUpdate: channels.has(`update`) ? (e) => record(`update`, e) : undefined, + onExit: channels.has(`exit`) ? (e) => record(`exit`, e) : undefined, + onBatch: channels.has(`batch`) + ? (events) => { + for (const e of events) record(`batch`, e) + } + : undefined, + }) + const project = ({ id, version }: CallbackRow): CallbackOutput => { + let payload: unknown = version + if (history.representation === `opaque`) + payload = new PrivatePayload(version) + if (history.representation === `private`) + payload = new KeyedPrivatePayload(version) + if (history.representation === `equal`) payload = 0 + if (history.representation === `prototype`) { + payload = new PrototypePayload(version) + } + if (history.representation === `set`) + payload = new Set( + [0, 0, version].map((value) => ({ + get value() { + if (!recording) comparisonReads++ + return value + }, + })), + ) + return { id, payload } + } + const failures: Array = [] + let handle: Awaited> | undefined + let before: number | undefined + const cut = (name: string, after: number | undefined, suppressed = false) => { + try { + checkDelivery( + observed, + expectedDelivery( + history.representation, + before, + after, + channels, + suppressed, + ), + ) + } catch (error) { + failures.push(new Error(`callback cut ${name}`, { cause: error })) + } + observed.length = 0 + before = after + reach.reach(`${callbackHistoryId(history)}/${name}`) + } + await checkWithCleanup(async () => { + handle = await driver.mount({ + skipInitial: history.skipInitial, + project, + handlers: handlers(), + }) + cut(`initial`, 0, history.skipInitial) + await handle.write(`update`, 1) + cut(`unobserved`, 1) + if (history.profile === `late-update`) channels.add(`update`) + if (history.profile === `late-batch`) channels.add(`batch`) + await handle.replaceHandlers(handlers()) + cut(`replacement`, 1) + await handle.write(`update`, 2) + cut(`update`, 2) + // Installing exit after unobserved updates distinguishes current values + // from an initial value retained only when update handlers were present. + channels = new Set([`exit`]) + await handle.replaceHandlers(handlers()) + await handle.write(`delete`, 2) + cut(`exit`, undefined) + channels = new Set([`enter`, `update`, `exit`, `batch`]) + await handle.replaceHandlers(handlers()) + await handle.write(`insert`, 3) + cut(`reentry`, 3) + await handle.write(`update`, 4) + cut(`next-use`, 4) + await handle.dispose() + await handle.write(`update`, 5) + cut(`disposed`, 4) + if (process.env.TANSTACK_DB_ORACLE_STATISTICS === `1`) + console.info( + `Effect comparison reads`, + callbackHistoryId(history), + comparisonReads, + ) + try { + expect(comparisonReads, `unordered output leaf traversal`).toBe(0) + } catch (error) { + failures.push(error) + } + if (failures.length) + throw new AggregateError(failures, `Effect callback delivery failed`, { + cause: failures[0], + }) + }, [ + async () => { + await handle?.cleanup() + }, + ]) +} diff --git a/packages/db/tests/query/virtual-row-fields-effect-oracle.test.ts b/packages/db/tests/query/virtual-row-fields-effect-oracle.test.ts new file mode 100644 index 0000000000..6456720898 --- /dev/null +++ b/packages/db/tests/query/virtual-row-fields-effect-oracle.test.ts @@ -0,0 +1,1287 @@ +/** + * # Do Effects deliver the source metadata a query explicitly selects? + * + * VirtualRowProps permits selecting source metadata. EffectConfig promises the + * query's complete deltas through onBatch and once per row through the named + * handlers; DeltaEvent gives updates the value before the batch. Authority: + * virtual-props.ts, query/effect.ts public types, and live-queries.md's Effect + * query-feature and delta sections. Optimistic-history-oracle supplies the + * source visibility and VirtualOrigin settlement law. + * + * The model authors source rows, optimistic snapshots and applied origins. It + * recomputes ordinary projected fields, then differences two result arrays. + * The actual Effect callbacks supply two independent ledgers. Each ledger must + * contain exactly that delta, including previous values and multiplicity. + * Source metadata is copied into ordinary observed* fields: no output Collection + * exists, so this oracle assigns no implicit Effect-root metadata or ID. + * + * The finite 200 histories cross structured/functional projection, one source, + * a LEFT join with a matched row and unmatched anchor, and populated unions in + * both branch orders. Each source has at most one persisting update. Outcomes, + * queued confirmations and settlement orders vary independently. Later remote + * writes and a peer's exit/reentry check next use and surviving-row isolation. + * Removing a dimension loses that receiving path or settlement distinction. + * Reject-before-update and update-of-absent-key histories are excluded. + * + * Sources are eager and preloaded. At each action cut a host turn drains queued + * delivery, then the whole new prefix of each callback ledger is checked. The + * recorder copies flat scalar fields when invoked, before another action can + * replace them. Earlier ledgers are retained. Each cut collects its mismatches + * without changing the authored next state; the history reports all failures. + * Batch partition, including empty partitions, is unconstrained here; exact + * event multiplicity remains observable in both callback channels. + * A malformed startup delta therefore cannot prevent later comparisons. Reach + * counts exercised cuts; separate counters report which comparisons passed. + * A final drain after disposal checks for extra delta events. This bounded schedule + * does not establish arbitrary provider delays, framework hooks, ordered + * repairs, changing join keys, same-source overlap or subquery sanitization. + * + * Event keys are opaque. Both callback channels must agree; distinct live + * occurrences need distinct keys, and a live occurrence keeps its key through + * updates. An exit retires that correspondence. Reentry may use a fresh key. + * Exact typed source identity is checked through ordinary observedKey instead. + * The four unclaimed Effect-root virtual names are excluded from the ordinary + * field projection, not interpreted as an output-row attribution policy. + * + * The callback-profile companion below also checks equal-projection graph + * updates under the maintainer's preservation decision. The original metadata + * histories change selected source facts and retain their array-difference + * judgment; they do not specify every unordered graph notification. + */ +import { isDeepStrictEqual } from 'node:util' +import { afterAll, expect, test } from 'vitest' +import { createCollection } from '../../src/collection/index.js' +import { Query, coalesce, eq } from '../../src/query/index.js' +import { createEffect } from '../../src/query/effect.js' +import { createTransaction } from '../../src/transactions.js' +import { declareOracle } from '../oracle-declaration.js' +import { createOracleReach } from '../oracle-reach.js' +import { flushPromises, mockSyncCollectionOptions } from '../utils.js' +import { expectedSourceMetadata } from './virtual-row-fields-source-oracle.js' +import { + checkWithCleanup, + virtualFieldNames, +} from './virtual-row-fields-support.js' +import { + callbackHistories, + callbackReach, + runCallbackHistory, +} from './effect-callback-oracle.js' +import type { DeltaEvent, EffectConfig } from '../../src/query/effect.js' +import type { Transaction } from '../../src/transactions.js' +import type { CallbackOutput, CallbackRow } from './effect-callback-oracle.js' + +type Key = string | number +type Side = `left` | `right` +type Origin = `local` | `remote` +type SourceRow = { + id: Key + parentId: string + side: Side + label: string +} +type SourceState = { + id: string + base: Array + origins: Map + overlay?: SourceRow + confirmation?: SourceRow +} +type Projection = `structured` | `functional` +type Kind = `single` | `join` | `union` +type History = { + kind: Kind + projection: Projection + reverse: boolean + confirm: ReadonlyArray + reject: ReadonlyArray + rightFirst: boolean +} +type ProjectedRow = Record & { id: Key; side: Side } +type ModelEvent = { + type: `enter` | `update` | `exit` + value: ProjectedRow + previousValue?: ProjectedRow +} +type RecordedEvent = { + type: string + key: Key + value: unknown + previousValue?: unknown +} +type NamedEvent = RecordedEvent & { route: ModelEvent[`type`] } +type Ledger = { + batches: Array> + named: Array +} +type KeyIntervals = Map + +function visibleSource(source: SourceState) { + return source.base.map((base) => { + const pending = source.overlay?.id === base.id + const row = pending ? source.overlay! : base + return { + ...row, + ...expectedSourceMetadata( + row.id, + source.id, + pending, + source.origins.get(row.id) ?? `remote`, + ), + } + }) +} +type SourceValue = ReturnType[number] + +// SELECT copies these source facts into ordinary fields. New output objects +// need no root metadata judgment to determine their explicitly selected values. +function projectedSource(row: SourceValue): ProjectedRow { + return { + id: row.id, + side: row.side, + label: row.label, + observedKey: row.$key, + observedCollectionId: row.$collectionId, + observedOrigin: row.$origin, + observedPending: row.$hasPendingWrites, + } +} +function expectedEffectRows( + kind: Kind, + left: SourceState, + right: SourceState, +): Array { + const a = visibleSource(left) + const b = visibleSource(right) + if (kind === `single`) return a.map(projectedSource) + if (kind === `union`) return [...a, ...b].map(projectedSource) + return a.map((row) => { + const joined = b.find((candidate) => candidate.parentId === row.id) + return { + ...projectedSource(row), + rightLabel: joined?.label, + rightKey: joined?.$key, + rightCollectionId: joined?.$collectionId, + rightOrigin: joined?.$origin, + rightPending: joined?.$hasPendingWrites, + } + }) +} +function occurrence(value: unknown): string { + if (!value || typeof value !== `object`) return `invalid:${String(value)}` + const row = value as Record + return `${String(row.side)}:${typeof row.id}:${String(row.id)}` +} +// Independent before/after differencing retains every occurrence. Equality +// distinguishes a missing own field from a field whose value is undefined. +function expectedEffectDelta( + before: ReadonlyArray, + after: ReadonlyArray, +): Array { + const delta: Array = [] + for (const old of before) { + if (!after.some((row) => occurrence(row) === occurrence(old))) + delta.push({ type: `exit`, value: old }) + } + for (const row of after) { + const old = before.find((prior) => occurrence(prior) === occurrence(row)) + if (!old) delta.push({ type: `enter`, value: row }) + else if (!isDeepStrictEqual(old, row)) + delta.push({ type: `update`, value: row, previousValue: old }) + } + return delta +} +function eventOrder(a: { value: unknown }, b: { value: unknown }): number { + return occurrence(a.value).localeCompare(occurrence(b.value)) +} +function eventValue(event: RecordedEvent) { + const { key: _key, ...selected } = event + return selected +} +// Named delivery and batch delivery each refine the independent full delta. +// Cross-channel agreement alone would allow both channels to share a defect. +function checkEffectDelta( + ledger: Ledger, + expected: ReadonlyArray, + active: KeyIntervals, +): void { + const batches = ledger.batches.flat() + const failures: Array = [] + const judge = (law: string, compare: () => void) => { + try { + compare() + } catch (error) { + failures.push(new Error(law, { cause: error })) + } + } + judge(`batch delta and previous values`, () => { + expect(batches.map(eventValue).sort(eventOrder)).toStrictEqual( + [...expected].sort(eventOrder), + ) + }) + judge(`named delta, route and previous values`, () => { + expect(ledger.named.map(eventValue).sort(eventOrder)).toStrictEqual( + expected + .map((event) => ({ ...event, route: event.type })) + .sort(eventOrder), + ) + }) + judge(`named and batch event keys agree`, () => { + expect( + ledger.named.map(({ route: _route, ...event }) => event).sort(eventOrder), + ).toStrictEqual([...batches].sort(eventOrder)) + }) + + // Report an invalid event without adopting it as a live occurrence. Later + // lawful enters in the same batch can still establish their own intervals. + for (const event of batches) { + const id = occurrence(event.value) + judge(`opaque key interval for ${event.type} ${id}`, () => { + expect([`string`, `number`]).toContain(typeof event.key) + if (event.type === `enter`) { + expect(active.has(id), `enter starts an absent occurrence`).toBe(false) + expect( + [...active.values()], + `live occurrence keys are distinct`, + ).not.toContain(event.key) + active.set(id, event.key) + } else { + expect(active.has(id), `update/exit names a live occurrence`).toBe(true) + expect(event.key, `key remains stable within its active interval`).toBe( + active.get(id), + ) + if (event.type === `exit`) active.delete(id) + } + }) + } + if (failures.length) + throw new AggregateError(failures, `Effect delta law failed`, { + cause: failures[0], + }) +} + +// One source has two outcomes and two confirmation states. Two sources have +// four independent outcomes, four confirmation masks and two settlement orders. +// Only unions vary branch order: it has observable source-selection machinery. +const effectMetadataHistories: Array = ( + [`structured`, `functional`] as const +).flatMap((projection) => + ([`single`, `join`, `union`] as const).flatMap((kind) => + (kind === `union` ? [false, true] : [false]).flatMap((reverse) => + (kind === `single` + ? [[], [`left`]] + : [[], [`left`], [`right`], [`left`, `right`]] + ).flatMap((confirm) => + (kind === `single` + ? [[], [`left`]] + : [[], [`left`], [`right`], [`left`, `right`]] + ).flatMap((reject) => + (kind === `single` ? [false] : [false, true]).map((rightFirst) => ({ + kind, + projection, + reverse, + confirm: confirm as Array, + reject: reject as Array, + rightFirst, + })), + ), + ), + ), + ), +) + +// Handler profiles and readable opaque values share their model/comparison +// with the real React receiver. This driver supplies only the core boundary. +const deliveryReach = callbackReach(`core`) +let ranDelivery = false +afterAll(() => { + if (ranDelivery) deliveryReach.assertReached() + if (process.env.TANSTACK_DB_ORACLE_STATISTICS === `1`) + console.info( + `Effect core callback reach`, + Object.fromEntries(deliveryReach.counts()), + ) +}) +test.each(callbackHistories)( + `delivers current values across callback profiles: %j`, + async (history) => { + ranDelivery = true + await runCallbackHistory( + history, + { + mount: async ({ skipInitial, project, handlers }) => { + const source = createCollection( + mockSyncCollectionOptions({ + id: `callback-core`, + getKey: (row) => row.id, + initialData: [{ id: 1, version: 0 }], + }), + ) + await source.preload() + const config: EffectConfig = { + query: () => + new Query() + .from({ row: source }) + .fn.select(({ row }) => project(row)), + skipInitial, + ...handlers, + } + const effect = createEffect(config) + await flushPromises() + return { + write: async (type, version) => { + source.utils.begin() + source.utils.write({ type, value: { id: 1, version } }) + source.utils.commit() + await flushPromises() + }, + replaceHandlers: async (replacement) => { + Object.assign(config, replacement) + await flushPromises() + }, + dispose: () => effect.dispose(), + cleanup: () => + checkWithCleanup(async () => { + await effect.dispose() + }, [ + async () => { + await source.cleanup() + }, + ]), + } + }, + }, + deliveryReach, + ) + }, +) +function pathOf(history: Pick) { + return `${history.kind}${history.kind === `union` ? (history.reverse ? `-right-first` : `-left-first`) : ``}/${history.projection}` +} +function sidesOf(history: History): Array { + return history.kind === `single` ? [`left`] : [`left`, `right`] +} +function historyId(history: History): string { + return `${pathOf(history)}/confirm-${history.confirm.join(`+`) || `none`}/reject-${history.reject.join(`+`) || `none`}/settle-${history.rightFirst ? `right` : `left`}-first` +} +// Declare each finite lane's owed comparisons independently. Aggregate path +// totals would let one omitted right-source action borrow another lane's reach. +// Outcome and confirmation premises determine which cuts are legal in a lane. +function requiredEffectCuts(history: History): Array { + const sides = sidesOf(history) + return [ + `initial`, + ...sides.map((side) => `${side}-pending`), + ...(sides.length === 2 ? [`two-pending`] : []), + ...history.confirm.map((side) => `${side}-accepted-held`), + ...sides.map( + (side) => + `${side}-${history.reject.includes(side) ? `rejected` : `fulfilled`}`, + ), + ...(sides.length === 2 + ? [`one-pending-${history.rightFirst ? `left` : `right`}`] + : []), + ...sides.map((side) => `${side}-remote-followup`), + `peer-exit`, + `peer-reentry`, + `survivor-next-use`, + `final-drain`, + ] +} +const effectMetadataReach = createOracleReach( + `effect explicit source metadata`, + effectMetadataHistories.flatMap((history) => + [`batch`, `named`].flatMap((channel) => + requiredEffectCuts(history).map( + (cut) => `${historyId(history)}/${channel}/${cut}`, + ), + ), + ), + // This exhaustive grammar has no seed/path replay. An unrelated replay + // environment must not exempt its required finite lanes from reach. + { replaying: false }, +) +const effectPassedCuts = new Map() +let enteredEffectHistories = 0 +afterAll(() => { + // A checker-only selector enters no production history. Once any native + // history runs, every finite lane still owes its complete declared reach. + if (enteredEffectHistories > 0) effectMetadataReach.assertReached() + if (process.env.TANSTACK_DB_ORACLE_STATISTICS === `1`) + console.info( + `effect metadata reach`, + Object.fromEntries(effectMetadataReach.counts()), + `passed comparisons`, + Object.fromEntries(effectPassedCuts), + ) +}) +declareOracle({ + model: [visibleSource, expectedEffectRows, expectedEffectDelta], + check: [checkEffectDelta], + grammar: [effectMetadataHistories], +}) + +function gate() { + let resolve!: () => void + let reject!: (reason: Error) => void + const promise = new Promise((yes, no) => { + resolve = yes + reject = no + }) + void promise.catch(() => {}) + return { promise, resolve, reject, calls: 0 } +} +function createSource(state: SourceState, hold: ReturnType) { + return createCollection({ + ...mockSyncCollectionOptions({ + id: state.id, + getKey: (row) => row.id, + initialData: state.base.map((row) => ({ ...row })), + }), + onUpdate: async () => { + hold.calls++ + return hold.promise + }, + }) +} +type Source = ReturnType +function captureSelected(value: unknown): unknown { + if (!value || typeof value !== `object`) return value + return Object.fromEntries( + Object.entries(value).filter( + ([name]) => !virtualFieldNames.some((field) => field === name), + ), + ) +} +function captureEvent(event: DeltaEvent): RecordedEvent { + return { + type: event.type, + key: event.key, + value: captureSelected(event.value), + ...(`previousValue` in event + ? { previousValue: captureSelected(event.previousValue) } + : {}), + } +} +// Driver projection: runtime source objects, not model source snapshots. +function selectSource(row: SourceValue) { + return { + id: row.id, + side: row.side, + label: row.label, + observedKey: row.$key, + observedCollectionId: row.$collectionId, + observedOrigin: row.$origin, + observedPending: row.$hasPendingWrites, + } +} +function startEffect( + history: History, + left: Source, + right: Source, + ledger: Ledger, + errors: Array, +) { + return createEffect({ + query: () => { + if (history.kind === `union`) { + const query = new Query().unionAll( + history.reverse ? { b: right, a: left } : { a: left, b: right }, + ) + return history.projection === `functional` + ? query.fn.select(({ a, b }) => { + const row = a ?? b + if (!row) throw new Error(`union has no active source`) + return selectSource(row) + }) + : query.select(({ a, b }) => ({ + id: coalesce(a.id, b.id), + side: coalesce(a.side, b.side), + label: coalesce(a.label, b.label), + observedKey: coalesce(a.$key, b.$key), + observedCollectionId: coalesce(a.$collectionId, b.$collectionId), + observedOrigin: coalesce(a.$origin, b.$origin), + observedPending: coalesce( + a.$hasPendingWrites, + b.$hasPendingWrites, + ), + })) + } + const base = new Query().from({ a: left }) + if (history.kind === `join`) { + const query = base.leftJoin({ b: right }, ({ a, b }) => + eq(a.id, b.parentId), + ) + return history.projection === `functional` + ? query.fn.select(({ a, b }) => ({ + ...selectSource(a), + rightLabel: b?.label, + rightKey: b?.$key, + rightCollectionId: b?.$collectionId, + rightOrigin: b?.$origin, + rightPending: b?.$hasPendingWrites, + })) + : query.select(({ a, b }) => ({ + id: a.id, + side: a.side, + label: a.label, + observedKey: a.$key, + observedCollectionId: a.$collectionId, + observedOrigin: a.$origin, + observedPending: a.$hasPendingWrites, + rightLabel: b.label, + rightKey: b.$key, + rightCollectionId: b.$collectionId, + rightOrigin: b.$origin, + rightPending: b.$hasPendingWrites, + })) + } + return history.projection === `functional` + ? base.fn.select(({ a }) => selectSource(a)) + : base.select(({ a }) => ({ + id: a.id, + side: a.side, + label: a.label, + observedKey: a.$key, + observedCollectionId: a.$collectionId, + observedOrigin: a.$origin, + observedPending: a.$hasPendingWrites, + })) + }, + onBatch: (events) => { + ledger.batches.push(events.map(captureEvent)) + }, + onEnter: (event) => { + ledger.named.push({ route: `enter`, ...captureEvent(event) }) + }, + onUpdate: (event) => { + ledger.named.push({ route: `update`, ...captureEvent(event) }) + }, + onExit: (event) => { + ledger.named.push({ route: `exit`, ...captureEvent(event) }) + }, + onError: (error) => errors.push(error), + onSourceError: (error) => errors.push(error), + }) +} + +test.each(effectMetadataHistories)( + `delivers explicitly selected source metadata through Effects: %j`, + async (history) => { + enteredEffectHistories++ + effectMetadataReach.beginHistory() + const path = pathOf(history) + const id = historyId(history) + const reach = (cut: string) => { + effectMetadataReach.reach(`${id}/batch/${cut}`) + effectMetadataReach.reach(`${id}/named/${cut}`) + } + const leftMain: SourceRow = { + id: history.kind === `union` ? 1 : `a`, + parentId: `unused`, + side: `left`, + label: `left-base`, + } + const peer: SourceRow = { + id: `anchor`, + parentId: `unused`, + side: `left`, + label: `peer`, + } + const rightMain: SourceRow = { + id: history.kind === `union` ? `1` : `r`, + parentId: `a`, + side: `right`, + label: `right-base`, + } + const states: Record = { + left: { + id: `effect-left`, + base: [{ ...leftMain }, { ...peer }], + origins: new Map(), + }, + right: { + id: `effect-right`, + base: [{ ...rightMain }], + origins: new Map(), + }, + } + const holds = { left: gate(), right: gate() } + let left: Source | undefined + let right: Source | undefined + let effect: ReturnType | undefined + const settlements: Partial>> = {} + const transactions = new Map() + const failures = { + left: new Error(`left rejection`), + right: new Error(`right rejection`), + } + const ledger: Ledger = { batches: [], named: [] } + const errors: Array = [] + const mismatches: Array = [] + const active: KeyIntervals = new Map() + let checkedBatches = 0 + let checkedNamed = 0 + let previous: Array = [] + const modelRows = () => + expectedEffectRows(history.kind, states.left, states.right) + const cut = async (name: string, premise?: () => void) => { + await flushPromises() + const next = modelRows() + let premiseReached = true + try { + premise?.() + } catch (error) { + premiseReached = false + mismatches.push( + new Error(`Effect metadata premise ${name}`, { cause: error }), + ) + } + try { + checkEffectDelta( + { + batches: ledger.batches.slice(checkedBatches), + named: ledger.named.slice(checkedNamed), + }, + expectedEffectDelta(previous, next), + active, + ) + expect(errors).toStrictEqual([]) + const key = `${path}/${name}` + effectPassedCuts.set(key, (effectPassedCuts.get(key) ?? 0) + 1) + } catch (error) { + mismatches.push( + new Error(`Effect metadata cut ${name}`, { cause: error }), + ) + } + checkedBatches = ledger.batches.length + checkedNamed = ledger.named.length + previous = next + if (premiseReached) reach(name) + } + const write = ( + side: Side, + type: `insert` | `update` | `delete`, + row: SourceRow, + ) => { + const source = side === `left` ? left! : right! + source.utils.begin() + source.utils.write({ type, value: { ...row } }) + return source.utils.commit() + } + const requirePersisting = (side: Side) => { + expect(holds[side].calls, `${side} handler entered once`).toBe(1) + expect( + transactions.get(side)?.state, + `${side} transaction persists`, + ).toBe(`persisting`) + } + const drive = async () => { + left = createSource(states.left, holds.left) + right = createSource(states.right, holds.right) + await Promise.all([left.preload(), right.preload()]) + effect = startEffect(history, left, right, ledger, errors) + await cut(`initial`) + const sides = sidesOf(history) + for (const side of sides) { + const source = side === `left` ? left : right + const row = states[side].base[0]! + states[side].overlay = { ...row, label: `${side}-pending` } + const transaction = source.update(row.id, (draft) => { + draft.label = `${side}-pending` + }) + transactions.set(side, transaction) + settlements[side] = transaction.when(`settled`).then( + () => undefined, + (error: unknown) => error, + ) + await cut(`${side}-pending`, () => requirePersisting(side)) + } + if (sides.length === 2) + await cut(`two-pending`, () => { + requirePersisting(`left`) + requirePersisting(`right`) + }) + for (const side of history.confirm) { + const row = { ...states[side].base[0]!, label: `${side}-server` } + states[side].confirmation = row + // Commit accepts this row; the persisting overlay still owns visibility. + write(side, `update`, row) + await cut(`${side}-accepted-held`, () => requirePersisting(side)) + } + const order: Array = + history.kind === `single` + ? [`left`] + : history.rightFirst + ? [`right`, `left`] + : [`left`, `right`] + for (const [index, side] of order.entries()) { + const state = states[side] + const reject = history.reject.includes(side) + state.overlay = undefined + if (state.confirmation) { + const confirmation = state.confirmation + state.base = state.base.map((row) => + row.id === confirmation.id ? { ...confirmation } : row, + ) + state.origins.set(confirmation.id, reject ? `remote` : `local`) + state.confirmation = undefined + } + if (reject) holds[side].reject(failures[side]) + else holds[side].resolve() + expect(await settlements[side]).toBe( + reject ? failures[side] : undefined, + ) + await cut(`${side}-${reject ? `rejected` : `fulfilled`}`, () => { + expect(holds[side].calls, `${side} handler entered once`).toBe(1) + expect( + transactions.get(side)?.state, + `${side} transaction settled`, + ).toBe(reject ? `failed` : `completed`) + }) + if (sides.length === 2 && index === 0) { + const stillPending = order[1]! + await cut(`one-pending-${stillPending}`, () => { + requirePersisting(stillPending) + expect(transactions.get(side)?.state).toBe( + reject ? `failed` : `completed`, + ) + }) + } + } + for (const side of sides) { + const state = states[side] + const row = { ...state.base[0]!, label: `${side}-later` } + state.base[0] = row + state.origins.set(row.id, `remote`) + write(side, `update`, row) + await cut(`${side}-remote-followup`) + } + states.left.base = states.left.base.filter((row) => row.id !== peer.id) + write(`left`, `delete`, peer) + await cut(`peer-exit`) + states.left.base.push({ ...peer }) + write(`left`, `insert`, peer) + await cut(`peer-reentry`) + const survivor = { ...states.left.base[0]!, label: `survivor-next-use` } + states.left.base[0] = survivor + write(`left`, `update`, survivor) + await cut(`survivor-next-use`) + await effect.dispose() + await cut(`final-drain`) + } + await checkWithCleanup(async () => { + try { + await drive() + } catch (error) { + mismatches.push( + new Error(`Effect metadata driver ${id}`, { cause: error }), + ) + } + if (mismatches.length) + throw new AggregateError(mismatches, `Effect metadata history failed`, { + cause: mismatches[0], + }) + }, [ + async () => { + holds.left.resolve() + holds.right.resolve() + await Promise.all(Object.values(settlements)) + }, + async () => { + await effect?.dispose() + }, + async () => { + await right?.cleanup() + }, + async () => { + await left?.cleanup() + }, + ]) + }, +) + +// The same delta judgment must reject wrong callback histories even when their +// final row is right. Controls seed active key intervals through initial enter +// delivery, so failures reach the update comparison rather than setup. +const effectFaults = [ + `omitted-peer`, + `duplicate-message`, + `duplicate-batch`, + `stale-pending`, + `wrong-source`, + `new-previous-value`, + `extra-undefined`, + `wrong-route`, + `missing-named`, + `delayed-extra`, + `key-collision`, + `key-change`, +] as const +declareOracle({ grammar: [effectFaults] }) +test.each(effectFaults)( + `rejects a hostile Effect metadata ledger: %s`, + (fault) => { + const row = (id: Key, pending: boolean): ProjectedRow => ({ + id, + side: `left`, + label: pending ? `pending` : `base`, + observedKey: id, + observedCollectionId: `source`, + observedOrigin: pending ? `local` : `remote`, + observedPending: pending, + }) + const before = [row(1, false), row(`1`, false)] + const after = [row(1, true), row(`1`, true)] + const events = (expected: Array, prefix: string): Ledger => { + const batch = expected.map((event) => ({ + ...structuredClone(event), + key: `${prefix}/${occurrence(event.value)}`, + })) + return { + batches: [batch], + named: batch.map((event) => ({ + ...structuredClone(event), + route: event.type, + })), + } + } + const active: KeyIntervals = new Map() + checkEffectDelta( + events(expectedEffectDelta([], before), `opaque`), + expectedEffectDelta([], before), + active, + ) + const expected = expectedEffectDelta(before, after) + const allowed = events(expected, `opaque`) + checkEffectDelta(structuredClone(allowed), expected, new Map(active)) + const hostile = structuredClone(allowed) + const first = hostile.batches[0]![0]! + const second = hostile.batches[0]![1]! + if (fault === `omitted-peer`) hostile.batches[0]!.pop() + if (fault === `duplicate-message`) + hostile.batches[0]!.push(structuredClone(first)) + if (fault === `duplicate-batch`) + hostile.batches.push(structuredClone(hostile.batches[0]!)) + if (fault === `stale-pending`) + (first.value as Record).observedPending = false + if (fault === `wrong-source`) + (first.value as Record).observedCollectionId = + `other-source` + if (fault === `new-previous-value`) + first.previousValue = structuredClone(first.value) + if (fault === `extra-undefined`) + (first.value as Record).extra = undefined + if (fault === `wrong-route`) hostile.named[0]!.route = `enter` + if (fault === `missing-named`) hostile.named.pop() + if (fault === `delayed-extra`) + hostile.batches.push([structuredClone(first)]) + // Both channels agree on each bad key, so only interval continuity can + // reject these faults; cross-channel agreement cannot mask its removal. + if (fault === `key-collision`) { + second.key = first.key + hostile.named[1]!.key = second.key + } + if (fault === `key-change`) { + first.key = `different-opaque-key` + hostile.named[0]!.key = first.key + } + expect(() => checkEffectDelta(hostile, expected, new Map(active))).toThrow() + }, +) + +// Allowed controls preserve implementation freedom: row order and opaque key +// representation do not define metadata, and an exited interval owns no key. +function calibrationRows(pending: boolean): Array { + const base: Array = [ + { id: 1, side: `left`, parentId: `unused`, label: `base` }, + { id: `1`, side: `left`, parentId: `unused`, label: `peer` }, + ] + return expectedEffectRows( + `single`, + { + id: `source`, + base, + origins: new Map(), + ...(pending ? { overlay: { ...base[0]!, label: `pending` } } : {}), + }, + { id: `unused`, base: [], origins: new Map() }, + ) +} +function calibrationLedger( + delta: ReadonlyArray, + keyFor: (value: ProjectedRow) => Key, +): Ledger { + const batch = delta.map((event) => ({ + ...structuredClone(event), + key: keyFor(event.value), + })) + return { + batches: batch.length ? [batch.reverse()] : [], + named: batch.map((event) => ({ + ...structuredClone(event), + route: event.type, + })), + } +} +test(`permits reordered delivery and fresh opaque keys after an exit`, () => { + const before = calibrationRows(false) + const after = calibrationRows(true) + const active: KeyIntervals = new Map() + const oldKey = (row: ProjectedRow) => `arbitrary/${occurrence(row)}` + const initial = expectedEffectDelta([], before) + checkEffectDelta(calibrationLedger(initial, oldKey), initial, active) + const update = expectedEffectDelta(before, after) + checkEffectDelta(calibrationLedger(update, oldKey), update, active) + const remaining = [after[0]!] + const exit = expectedEffectDelta(after, remaining) + checkEffectDelta(calibrationLedger(exit, oldKey), exit, active) + const reentry = expectedEffectDelta(remaining, after) + checkEffectDelta( + calibrationLedger(reentry, (row) => `fresh/${occurrence(row)}`), + reentry, + active, + ) + const next = [before[0]!, after[1]!] + const survivor = expectedEffectDelta(after, next) + checkEffectDelta(calibrationLedger(survivor, oldKey), survivor, active) +}) +test(`permits empty callback partitions while preserving exact delta events`, () => { + const before = calibrationRows(false) + const after = calibrationRows(true) + const active: KeyIntervals = new Map() + const keyFor = (row: ProjectedRow) => occurrence(row) + for (const delta of [ + expectedEffectDelta([], before), + expectedEffectDelta(before, after), + expectedEffectDelta(after, after), + ]) { + const ledger = calibrationLedger(delta, keyFor) + ledger.batches.unshift([]) + ledger.batches.push([]) + checkEffectDelta(ledger, delta, active) + } +}) +test(`records scalar callback values before later object mutation`, () => { + const before = calibrationRows(false) + const after = calibrationRows(true) + const keyFor = (row: ProjectedRow) => occurrence(row) + const active: KeyIntervals = new Map() + const initial = expectedEffectDelta([], before) + checkEffectDelta(calibrationLedger(initial, keyFor), initial, active) + const expected = expectedEffectDelta(before, after) + const row = structuredClone(expected[0]!.value) + const previousValue = structuredClone(expected[0]!.previousValue!) + const event: DeltaEvent = { + type: `update`, + key: keyFor(row), + value: row, + previousValue, + } + const ledger: Ledger = { + batches: [[captureEvent(event)]], + named: [{ route: `update`, ...captureEvent(event) }], + } + row.observedPending = false + previousValue.label = `mutated after delivery` + checkEffectDelta(ledger, expected, active) +}) +test(`preserves later interval judgments after an invalid unknown update`, () => { + const before = calibrationRows(false) + const after = calibrationRows(true) + const keyFor = (row: ProjectedRow) => occurrence(row) + const active: KeyIntervals = new Map() + const initial = expectedEffectDelta([], before) + const ledger = calibrationLedger(initial, keyFor) + const invalid = { + type: `update`, + key: `never-entered`, + value: structuredClone(before[0]!), + previousValue: structuredClone(before[0]!), + } + ledger.batches[0]!.unshift(invalid) + ledger.named.unshift({ ...structuredClone(invalid), route: `update` }) + expect(() => checkEffectDelta(ledger, initial, active)).toThrow() + const update = expectedEffectDelta(before, after) + checkEffectDelta(calibrationLedger(update, keyFor), update, active) +}) + +/** + * Membership is independent of the selected value's comparison domain. A fresh + * callable or opaque instance may compare unequal when the graph retracts a + * transient join contribution, but an absent occurrence still cannot update. + * These histories also coalesce legal source transactions inside a real + * optimistic-transaction scheduler scope. Their before/after source snapshots + * determine membership independently of output multiplicities. + * + * The rich-value observation records its kind and callable/readable result. + * It does not claim exact instance identity for a re-evaluated projection. + * Nondeterministic projections are not needed to expose this membership law. + */ +type BoundaryRepresentation = `scalar` | `function` | `private` +class BoundaryValue { + #label: string + constructor(label: string) { + this.#label = label + } + read() { + return this.#label + } +} +const boundaryHistories = ([`scalar`, `function`, `private`] as const).flatMap( + (representation) => + ([`single`, `join`] as const).flatMap((kind) => + [false, true].flatMap((present) => + [false, true].map((grouped) => ({ + representation, + kind, + present, + grouped, + })), + ), + ), +) +function boundaryId(history: (typeof boundaryHistories)[number]) { + return `${history.representation}/${history.kind}/${history.present}/${history.grouped}` +} +const boundaryCuts = [`initial`, `sequence`, `absent`, `reentry`, `next-use`] +const boundaryReach = createOracleReach( + `Effect membership independent of projection equality`, + boundaryHistories.flatMap((history) => + boundaryCuts.map((cut) => `${boundaryId(history)}/${cut}`), + ), + { replaying: false }, +) +let enteredBoundary = false +afterAll(() => { + if (enteredBoundary) boundaryReach.assertReached() + if (process.env.TANSTACK_DB_ORACLE_STATISTICS === `1`) + console.info( + `Effect membership reach`, + Object.fromEntries(boundaryReach.counts()), + ) +}) +function boundaryModelRows( + kind: `single` | `join`, + representation: BoundaryRepresentation, + left: SourceState, + right: SourceState, +): Array { + return expectedEffectRows(kind, left, right).map((row) => ({ + ...row, + payloadKind: representation, + payload: row.label, + })) +} +function captureBoundaryEvent(event: DeltaEvent): RecordedEvent { + const capture = (value: unknown) => { + const row = captureSelected(value) as Record + const payload = row.payload + return { + ...row, + payloadKind: + typeof payload === `function` + ? `function` + : payload instanceof BoundaryValue + ? `private` + : `scalar`, + payload: + typeof payload === `function` + ? payload() + : payload instanceof BoundaryValue + ? payload.read() + : payload, + } + } + return { + type: event.type, + key: event.key, + value: capture(event.value), + ...(`previousValue` in event + ? { previousValue: capture(event.previousValue) } + : {}), + } +} +declareOracle({ model: [boundaryModelRows], grammar: [boundaryHistories] }) +test.each(boundaryHistories)( + `classifies callback-visible membership across coalesced source histories: %j`, + async (history) => { + enteredBoundary = true + boundaryReach.beginHistory() + const main: SourceRow = { + id: `a`, + parentId: `unused`, + side: `left`, + label: `v0`, + } + const anchor: SourceRow = { + id: `anchor`, + parentId: `unused`, + side: `left`, + label: `anchor`, + } + const joined: SourceRow = { + id: `r`, + parentId: `a`, + side: `right`, + label: `right`, + } + const state: SourceState = { + id: `boundary-left`, + base: history.present ? [main, anchor] : [anchor], + origins: new Map(), + } + const rightState: SourceState = { + id: `boundary-right`, + base: [joined], + origins: new Map(), + } + const left = createSource(state, gate()) + const right = createSource(rightState, gate()) + const ledger: Ledger = { batches: [], named: [] } + const active: KeyIntervals = new Map() + const failures: Array = [] + const callbackErrors: Array = [] + let effect: ReturnType | undefined + let before: Array = [] + let checkedBatch = 0 + let checkedNamed = 0 + const cut = async (name?: string) => { + await flushPromises() + const after = boundaryModelRows( + history.kind, + history.representation, + state, + rightState, + ) + try { + checkEffectDelta( + { + batches: ledger.batches.slice(checkedBatch), + named: ledger.named.slice(checkedNamed), + }, + expectedEffectDelta(before, after), + active, + ) + expect(callbackErrors).toStrictEqual([]) + } catch (error) { + failures.push( + new Error(`membership cut ${name ?? `individual action`}`, { + cause: error, + }), + ) + } + before = after + checkedBatch = ledger.batches.length + checkedNamed = ledger.named.length + if (name) boundaryReach.reach(`${boundaryId(history)}/${name}`) + } + // The model applies the authored source operation directly. Production + // receives the same operation through public sync writes and scheduling. + const apply = (type: `insert` | `delete` | `update`, label: string) => { + const row = { ...main, label } + state.base = + type === `delete` + ? state.base.filter((value) => value.id !== main.id) + : type === `insert` + ? [...state.base, row] + : state.base.map((value) => (value.id === main.id ? row : value)) + left.utils.begin() + left.utils.write({ type, value: row }) + left.utils.commit() + } + await checkWithCleanup(async () => { + await Promise.all([left.preload(), right.preload()]) + effect = createEffect({ + query: () => { + const base = new Query().from({ a: left }) + const project = (a: SourceValue, b?: SourceValue) => { + const label = a.label + const payload = + history.representation === `function` + ? () => label + : history.representation === `private` + ? new BoundaryValue(label) + : label + return { + ...selectSource(a), + ...(history.kind === `join` + ? { + rightLabel: b?.label, + rightKey: b?.$key, + rightCollectionId: b?.$collectionId, + rightOrigin: b?.$origin, + rightPending: b?.$hasPendingWrites, + } + : {}), + payload, + } + } + return history.kind === `join` + ? base + .leftJoin({ b: right }, ({ a, b }) => eq(a.id, b.parentId)) + .fn.select(({ a, b }) => project(a, b)) + : base.fn.select(({ a }) => project(a)) + }, + onBatch: (events) => { + ledger.batches.push(events.map(captureBoundaryEvent)) + }, + onEnter: (event) => { + ledger.named.push({ route: `enter`, ...captureBoundaryEvent(event) }) + }, + onUpdate: (event) => { + ledger.named.push({ route: `update`, ...captureBoundaryEvent(event) }) + }, + onExit: (event) => { + ledger.named.push({ route: `exit`, ...captureBoundaryEvent(event) }) + }, + onError: (error) => callbackErrors.push(error), + onSourceError: (error) => callbackErrors.push(error), + }) + await cut(`initial`) + const commands: Array = + history.present + ? [ + [`delete`, `v0`], + [`insert`, `v1`], + [`delete`, `v1`], + ] + : [ + [`insert`, `v0`], + [`delete`, `v0`], + [`insert`, `v1`], + ] + if (history.grouped) { + createTransaction({ mutationFn: async () => {} }).mutate(() => { + for (const [type, label] of commands) apply(type, label) + }) + } else { + for (const [type, label] of commands) { + apply(type, label) + await cut() + } + } + await cut(`sequence`) + if (!history.present) apply(`delete`, `v1`) + await cut(`absent`) + apply(`insert`, `v2`) + await cut(`reentry`) + apply(`update`, `v3`) + await cut(`next-use`) + if (failures.length) + throw new AggregateError(failures, `Effect membership history failed`, { + cause: failures[0], + }) + }, [ + async () => { + await effect?.dispose() + }, + async () => { + await right.cleanup() + }, + async () => { + await left.cleanup() + }, + ]) + }, +) diff --git a/packages/react-db/tests/effect-callback-oracle.test.tsx b/packages/react-db/tests/effect-callback-oracle.test.tsx new file mode 100644 index 0000000000..8608fb34be --- /dev/null +++ b/packages/react-db/tests/effect-callback-oracle.test.tsx @@ -0,0 +1,102 @@ +/** + * React receiving driver for the core Effect metadata owner's callback law. + * The shared oracle authors source versions, callback changes and comparisons. + * Real renderHook/act commits supply the hook's callback-replacement premise. + * Observation cuts follow act and one host turn. This covers eager sources, + * not arbitrary host scheduling or other framework bindings. + */ +import { afterAll, test } from 'vitest' +import { act, renderHook } from '@testing-library/react' +import { Query, createCollection } from '@tanstack/db' +import { useLiveQueryEffect } from '../src/useLiveQueryEffect' +import { + callbackHistories, + callbackReach, + runCallbackHistory, +} from '../../db/tests/query/effect-callback-oracle' +import { checkWithCleanup } from '../../db/tests/query/virtual-row-fields-support' +import { flushPromises, mockSyncCollectionOptions } from '../../db/tests/utils' +import type { + CallbackOutput, + CallbackRow, +} from '../../db/tests/query/effect-callback-oracle' +import type { EffectConfig } from '@tanstack/db' + +const reach = callbackReach(`React`) +let ran = false +afterAll(() => { + if (ran) reach.assertReached() + if (process.env.TANSTACK_DB_ORACLE_STATISTICS === `1`) + console.info( + `Effect React callback reach`, + Object.fromEntries(reach.counts()), + ) +}) +test.each(callbackHistories)( + `receives current Effect values across callback profiles: %j`, + async (history) => { + ran = true + await runCallbackHistory( + history, + { + mount: async ({ skipInitial, project, handlers }) => { + const source = createCollection( + mockSyncCollectionOptions({ + id: `callback-react`, + getKey: (row) => row.id, + initialData: [{ id: 1, version: 0 }], + }), + ) + await source.preload() + const query = new Query() + .from({ row: source }) + .fn.select(({ row }) => project(row)) + const mounted = renderHook( + ({ config }: { config: EffectConfig }) => + useLiveQueryEffect(config), + { initialProps: { config: { query, skipInitial, ...handlers } } }, + ) + await act(async () => { + await flushPromises() + }) + return { + write: async (type, version) => { + await act(async () => { + source.utils.begin() + source.utils.write({ type, value: { id: 1, version } }) + source.utils.commit() + await flushPromises() + }) + }, + replaceHandlers: async (replacement) => { + mounted.rerender({ + config: { query, skipInitial, ...replacement }, + }) + await act(async () => { + await flushPromises() + }) + }, + dispose: async () => { + mounted.unmount() + await act(async () => { + await flushPromises() + }) + }, + cleanup: () => + checkWithCleanup(async () => { + mounted.unmount() + await act(async () => { + await flushPromises() + }) + }, [ + async () => { + await source.cleanup() + }, + ]), + } + }, + }, + reach, + ) + }, +)