D-MBX-A6-P3d: durable-witness reshape (crash-durable paired move) + temporal.rs layer-1 causal deinterlacing

.claude/board/EPIPHANIES.md

@@ -1,3 +1,41 @@
+## 2026-08-02 — E-THE-DURABLE-UNIT-IS-THE-CYCLE-NOT-THE-CAST-ONE-WAL-WRITE-PER-SWEEP-1 — the persistence seam is reshaped to the WAL-amortized generation; per-cast durability retired
+
+**Status:** FINDING (operator-ruled 2026-08-02 — the decision on the fork Correction 2(c) of `E-THE-PAIRED-MOVE-MUST-BE-DURABLE-CO-LOCATED-NOT-IN-MEMORY-ONLY-1` surfaced). **Confidence:** High for the **storage/race CONTRACT** (`lance-graph-planner` 344 lib tests, 13 in `persist_sink`, clippy+fmt clean); crash-durability remains contract-probed over an in-process fake, NOT storage-proven (`compile+test green ≠ storage proven`, the Ladybug lesson). Builds NO concrete sink.
+
+**The ruling.** The durable unit is the **cycle/sweep** — one globally-aligned 64k-row frame — NOT the per-cast `DurableWitness`. `persist_cast`-per-thought would pay WAL header + durability-wait + version-metadata 64k times; the reshape **amortizes the WAL**: 64k concurrent thoughts stage into an owned cast vector, one `persist_cycle` folds+freezes them, and `WalSink::commit_cycle` does exactly **one atomic append → one `DatasetVersion`**. cast ⊂ chunk ⊂ cycle.
+
+**The layer's ONLY concerns are storage + that 64k concurrent thoughts hit no physical or epistemic race.** It models no semantics: it mints NO witness / ancestry / awareness / branch / rung / temporal-policy type. `CycleFrame` is storage-only `{cycle, base_version}`; `SweepSlot` is a boring landing (`where`, never `why` — no `basis`).
+
+**Physical race → WRITE-SIDE order, never read-time repair.** Thoughts finish in arbitrary CPU order; `DetachedCycleBatch::freeze` stable-orders casts by their EXISTING `stream_position` (`order_cycle_stably`, generic over the key) BEFORE the single append, so the durable image is already canonical. `scan_sealed` returns stored order and **never sorts** (a read-time sort would lose the whole amortization). This write-side ordering lives in `persist_sink.rs`, **NOT `temporal.rs`** — that module owns query-time reader-horizon/version reading; the write-side deinterlace additions made there this session were reverted.
+
+**Epistemic race → the sealed read horizon.** Every thought reads only the sealed predecessor `Vn` (`frame.base_version`); the commit publishes `Vn+1` once, all-or-nothing. `scan_sealed` excludes any uncommitted cycle, so no thought reads a concurrent sibling's in-flight output as though it were sealed history (`read Vn / write Vn+1`).
+
+**Coalescing is a real per-row fold, not last-chunk-wins.** The final image is `row -> last payload in stream order`; chunks are disjoint/mergeable slices of that image.
+
+**Vocabulary reused, not re-minted.** `DatasetVersion` (scheduler), `KanbanMove`/`KanbanColumn`/`RubiconTransitionError` (kanban), `MailboxId`/`MailboxSoaOwner`, `stream_position` (the existing canonical key). `recover_and_apply` + the durable **watermark** idempotence (not phase equality — the lifecycle is cyclic) + the `StalePhase`/`OwnerMismatch` corruption guards survive from the prior model.
+
+**Retired (this reshape).** `DurableWitness`, `DurableReceipt`, `DurableCoordinate`, `DurableWrite`, `persist_cast`, `apply_durable_step`, `scan_witnesses`, `LandedWitness` — the per-cast durability surface. Their intent is preserved in the cycle machinery (`CycleFrame`, `SweepSlot`, `LandedSlot`, `DetachedCycleBatch`, `WalSink::commit_cycle`, `persist_cycle`, `order_cycle_stably`). Supersedes the seam-shape half of `E-THE-PAIRED-MOVE-MUST-BE-DURABLE-CO-LOCATED-NOT-IN-MEMORY-ONLY-1` (its ordering/recovery/watermark CONTRACT stands). Still deferred: the concrete Lance sink, gated on crash falsifiers. PR #878 (reshaped in place).
+
+## 2026-08-01 — E-THE-PAIRED-MOVE-MUST-BE-DURABLE-CO-LOCATED-NOT-IN-MEMORY-ONLY-1 — the persistence sink's crash gap, and temporal.rs's missing layer-1
+
+**Status:** FINDING (operator-ruled 2026-08-01; review-hardened twice — see the two Corrections below). **Confidence:** High for the **ordering/recovery CONTRACT** (349 planner lib tests, clippy+fmt clean); the crash-durability is **contract-probed over an in-process fake sink, NOT storage-proven** — real MemWAL/restart/atomic co-location durability is demonstrated only by the deferred concrete Lance sink (`compile+test green ≠ storage proven`, the Ladybug lesson). Builds NO concrete sink.
+
+**The gap the split-phase left open.** At the prior two-clock-domain shape, `persist_cast` appended only the PAYLOAD; the paired `KanbanMove` lived solely in the in-memory `DurableReceipt`. Sequence: WAL append lands (durable) → process dies before `apply_durable_step` → the receipt evaporates → the paired move is lost → the KanbanStep never fires — **even though the durable SoA state moved**. Silent: storage advanced, lifecycle did not. An in-memory receipt is not a durable record; a step that lives only in it is a step that a crash can delete.
+
+**The rule (operator).** The paired transition witness must be **durably co-located with the SoA state in the same persistence generation**. The in-memory receipt may REFERENCE that durable material (via its `DurableCoordinate`) but must never be its only copy. NOT via a separate ack/confirmation ledger (`E-ACK-ELIMINATED-1` stands) — the witness rides IN the generation and is read back with the state. So `DurableWrite::append(&DurableWitness, &payload)` lands both atomically; `scan_witnesses` reads them back; `recover_and_apply` replays the pending tail.
+
+**temporal.rs is TWO layers, and layer-1 was missing.** Layer-2 (epistemic projection — `classify`/`deinterlace`: contemporary/anachronistic/spoiler/unknowable) was well-developed. Layer-1 (**causal** deinterlacing) did not exist: the global durable log interleaves every owner's writes (`A@s0, C@s0, B@s0, A@s1`), and reconstructing owner A's OWN local chain `[A@s0, A@s1]` — with the interleaved owners *removed* from A's timeline — is what makes crash-replay in cast order possible. New `LocalCausalRow` + `local_trajectories`/`local_trajectory_of`. The two layers compose: layer-1 rebuilds the chain, layer-2 filters it by the reader's horizon. `DurableWitness` implements `LocalCausalRow`.
+
+**Durability proof is the coordinate, not a `LanceVersion`.** A MemWAL write is queryable/durable BEFORE any base manifest version attaches, so recovery identifies the latest LOCAL state from the co-located witness's cast order — not from a dataset version that does not yet exist (falsifier 5). This is why `DurableCoordinate` (shard/epoch/wal-position), never `DatasetVersion`, is what `append` returns.
+
+**Five integration falsifiers (all green).** (1) layer-1 deinterlaces the interleaved global log into an owner-local chain, with a vacuity guard that the other owners' rows are *dropped* not reordered; (2) one owner's batch replays in cast order over a deliberately seq-descending log; (3) WAL succeeds + crash (receipt dropped) → recovery reconstructs+applies the move from the durable witness, idempotent on re-run; (4) WAL fails → no witness lands → no move, no step; (5) WAL-visible-before-manifest → recovery uses cast order, no dataset version exists. Plus a synchronous-path `StalePhase` guard (a stale/out-of-order apply is surfaced loudly on `apply_durable_step`, while recovery silently skips a stale move as already-reflected).
+
+**Still deferred (operator ruling — do NOT build yet).** The concrete `LanceShardSink` (`ShardWriter::put`, `enable_memtable + durable_write`) comes AFTER, gated on these falsifiers; the durable local causal chain lands first. No confirmation ledger, replay queue, per-thought ack, custom WAL, or ractor hot-path callback. Extends `E-KANBANMOVE-IS-THE-PARCEL-ADDRESS-STEP-IS-THE-DELIVERY-SCAN-1`.
+
+**Correction (PR #878 review, same PR — Bugbot High + Codex/CodeRabbit Critical).** The first cut ordered replay by `cast_id` and inferred "already applied" from phase equality. Both are unsound and were fixed within the PR: (a) **`CastId` resets to 0 on every writer restart** (`BatchWriter::next_id`), so cross-lifetime witnesses collide — replay now orders by the durable **`DurableCoordinate::log_order`** (WAL entry position, monotonic across restarts, never reset); `cast_id` survives only as provenance. `scan_witnesses` returns `LandedWitness{coordinate, witness}` and is the `LocalCausalRow` implementor. (b) **Phase equality is not a sound idempotence key on a CYCLIC lifecycle** (`Planning→CognitiveWork→Evaluation→Plan→Planning`): after a lap the owner is back at `Planning`, so a phase-only check replays the whole lap. `recover_and_apply` now takes a durable **watermark** (`applied_through`) — skip ≤ watermark, and above it a non-matching `from` is genuine corruption (`StalePhase`), not a benign skip; it returns the new watermark to persist WITH the SoA phase. Also: `paired_move.mailbox == owner` is validated at persist time (no cross-owner move becomes durable); `scan_witnesses(from)` is bounded; `WriteFailed`/`PersistError` impl `Display`+`Error`. New falsifiers: cyclic-idempotence with a negative control proving the watermark is load-bearing, and cast_id-collision ordering-by-durable-position. The layer-1 `cast_seq` doc now states the monotonic-across-restarts precondition.
+
+**Correction 2 (PR #878 review — ChatGPT, taken with a grain of salt).** Three further points held up and were fixed: (a) **reordered receipts must stay retryable, NOT dropped** — a stale receipt dropped on the *happy path* would only be re-applied on a crash+restart (recovery is not the happy-path backstop for concurrent-completion reordering), so `apply_durable_step` now **borrows** `&DurableReceipt`; on `StalePhase` the caller keeps it and retries once the prefix lands (falsifier `a_reordered_receipt_is_retryable_not_lost`, no crash needed). (b) **API-honesty on the durable coordinate** — `ShardWriter::put` returns a *batch position*, not `WalAppendResult::entry_position`, so the field `wal_entry_position` was renamed to an opaque, backend-defined `seq` (the sink maps whichever its API yields); no field claims a WAL offset the chosen API does not return. (c) **honest test/claims framing** — the `FakeSink` tests are labelled **ordering/recovery CONTRACT probes**, not crash/WAL integration tests (no real WAL, restart, atomic RecordBatch co-location, or fencing); the fake now STORES the payload (was ignored) and asserts it landed alongside the witness, while the module doc states plainly that atomic co-location is unproven until the concrete sink. Points taken-with-salt and NOT actioned as blocking: the per-cast-vs-generation persistence *seam shape* (finding 5) is a real architectural fork but not a correctness bug (the operator's own `bb5fcdd` used per-cast) — surfaced to the operator for decision rather than unilaterally reshaped; a `DurableWitness::try_from_cast` constructor was deemed redundant given the persist-time guard.
+
 ## 2026-08-01 — E-KANBANMOVE-IS-THE-PARCEL-ADDRESS-STEP-IS-THE-DELIVERY-SCAN-1 — the D-MBX-A6 move is pre-write intent; the KanbanStep is post-write; the persistence sink wires Lance 7's existing MemWAL and invents nothing
 
 **Status:** FINDING (operator-ruled 2026-08-01). **Confidence:** High — the pre-write half is shipped with 5 falsifiable probes (`lance_graph_planner::owner_adapter`); the persistence half's API is verified against real `lance-7.0.0` source.

.claude/board/LATEST_STATE.md

@@ -1,3 +1,17 @@
+## 2026-08-02 — branch `claude/medcare-rs-continue-ufsazd` — D-MBX-A6-P3e persistence-sink reshaped to WAL-amortized cycle (one write per sweep)
+
+- `lance_graph_planner::persist_sink` — **reshaped in place** (PR #878) from the per-cast durable-witness model to the **cycle/WAL** model (operator ruling on the seam-shape fork surfaced 2026-08-01). The durable unit is the **cycle/sweep**, NOT the cast: 64k concurrent thoughts stage into an owned `Vec<SweepSlot>`, `persist_cycle(sink, frame, casts)` folds+freezes them, and `WalSink::commit_cycle(base, DetachedCycleBatch)` does **exactly one atomic append → one `DatasetVersion`** (WAL amortization — 64k thoughts, one write). Types: `CycleId`, `CycleFrame{cycle, base_version}` (storage-only — NO rung/branch/semantic tags), `SweepSlot{cycle, stream_position, owner, row, paired_move, payload}` (boring landing, no `basis`), `LandedSlot{version, slot}`, `DetachedCycleBatch{frame, landings, image}` (frozen, deinterlaced, coalesced). **Write-side ordering** = `order_cycle_stably<T, K: Ord>(rows, key)` (generic over the caller's canonical key; stable-orders casts by the EXISTING `stream_position` in `freeze` BEFORE the append) — completion order never becomes storage order (physical race); `scan_sealed` returns stored order and NEVER sorts. **Epistemic horizon** = sealed read (`read Vn / write Vn+1`); an uncommitted cycle is invisible to `scan_sealed`. **Coalescing** = real per-row fold (`row -> last payload in stream order`), not last-chunk-wins. `recover_and_apply(owner, sealed, applied_through)` + the durable **watermark** idempotence + `StalePhase`/`OwnerMismatch` guards survive unchanged. `versions()` = the cheap coarse timeline a downstream time-series consumer (stockfish-rs, another session) looks up, no landing replay.
+- **Write-side ordering lives HERE, not in `temporal.rs`** — that module owns query-time reader-horizon/version reading; only the write-side `deinterlace_write` additions made there earlier this session were **reverted**. `temporal.rs`'s existing query-time causal helpers (`LocalCausalRow`, `local_trajectories`/`local_trajectory_of`, layer-1 causal deinterlace) landed in the PR's earlier commits and **remain** — the reshape commit itself leaves `temporal.rs` untouched (this commit's `git diff` on it is empty), NOT the whole PR. The rung a cycle aligns at is decided by whoever schedules it, never modelled in `CycleFrame`.
+- **Retired:** `DurableWitness`, `DurableReceipt`, `DurableCoordinate`, `DurableWrite`, `persist_cast`, `apply_durable_step`, `scan_witnesses`, `LandedWitness` (the per-cast surface). Vocabulary reused, not re-minted: `DatasetVersion`, `KanbanMove`/`KanbanColumn`, `MailboxId`/`MailboxSoaOwner`, `stream_position`. **No concrete Lance sink built** (deferred per operator, gated on crash falsifiers). Tests are storage/race CONTRACT probes over an in-process `FakeWalSink`, honestly labelled — real MemWAL/restart/atomic-append durability UNPROVEN. See `EPIPHANIES.md` E-THE-DURABLE-UNIT-IS-THE-CYCLE-NOT-THE-CAST-ONE-WAL-WRITE-PER-SWEEP-1.
+
+## 2026-08-01 — branch `claude/medcare-rs-continue-ufsazd` — D-MBX-A6-P3d persistence-sink durable-witness reshape + temporal layer-1
+
+### Current Contract Inventory — new/changed modules (lance-graph-planner)
+- `lance_graph_planner::persist_sink` — the POST-write half, two clock domains, **crash-durable**. `DurableWitness{owner, cast_id, cycle, paired_move}` is CO-LOCATED with the SoA payload in one persistence generation via `DurableWrite::append(&witness, &payload)`; the in-memory `DurableReceipt` merely REFERENCES it (via `DurableCoordinate`), never the only copy of the move. **Replay order = the durable `DurableCoordinate::log_order` (WAL position, monotonic across restarts), NOT `cast_id`** (`BatchWriter`'s counter resets on restart → cross-lifetime collisions; `cast_id` is provenance only). `DurableWrite::scan_witnesses(from)` returns `LandedWitness{coordinate, witness}` (bounded tail read; `LandedWitness` is the `LocalCausalRow` implementor). `recover_and_apply(owner, landed, applied_through)` = crash recovery: temporal layer-1 → per-owner tail in durable order; **idempotence is a durable WATERMARK** (`applied_through`), NOT phase equality (unsound on the cyclic lifecycle — a completed lap returns to `Planning`); above the watermark a non-matching `from` is corruption (`StalePhase`); returns `Recovered{applied, watermark}` to persist with the SoA phase. `persist_cast` validates `paired_move.mailbox == owner` (no cross-owner move becomes durable); `apply_durable_step` keeps the sync `from==phase` guard (a stale drop is safe — the durable witness replays). `WriteFailed`/`PersistError` impl `Display`+`Error`. Async `persist_cast` (no owner borrow) / sync `apply_durable_step` (no await) split preserved. Durability proof = `DurableCoordinate`, NEVER `LanceVersion`. **No concrete `LanceShardSink` built** (deferred per operator, gated on the crash falsifiers).
+- `lance_graph_planner::temporal` — **layer-1 CAUSAL deinterlacing added** (the missing half): `LocalCausalRow{owner, cast_seq}` + `local_trajectories`/`local_trajectory_of` split a globally-interleaved durable log into per-owner LOCAL chains ordered by `cast_seq` (`A@s0,C@s0,B@s0,A@s1` → A's `[A@s0,A@s1]`; interleaved owners removed). `cast_seq` doc states the monotonic-across-restarts precondition (a resettable counter is NOT valid; use a durable-log position). Composes with the pre-existing layer-2 epistemic projection (`classify`/`deinterlace`). `LandedWitness` implements `LocalCausalRow`.
+- Round-2 review (ChatGPT, grain of salt): `apply_durable_step` now **borrows** `&DurableReceipt` so a reordered/stale receipt stays RETRYABLE on the happy path (not dropped-until-crash); `DurableCoordinate.wal_entry_position` renamed to opaque `seq` (API-honest — `ShardWriter::put` returns a batch position, not a WAL offset); the `FakeSink` tests are labelled **CONTRACT probes** (the fake now stores + asserts the payload) — real MemWAL/restart/atomic-co-location durability is UNPROVEN until the concrete sink (`compile+test green ≠ storage proven`).
+- **Honest status:** the ordering/recovery CONTRACT is green (349 planner lib tests, clippy + fmt clean); crash-durability is contract-probed over an in-process fake, NOT storage-proven. Deferred per operator: the concrete `LanceShardSink`; and the per-cast-vs-**generation/batch** persistence seam shape (finding 5 — surfaced for operator decision, not a correctness bug). `E-THE-PAIRED-MOVE-MUST-BE-DURABLE-CO-LOCATED-NOT-IN-MEMORY-ONLY-1`.
+
 ## 2026-08-01 — branch `claude/medcare-rs-continue-ufsazd` — D-MBX-A6-P3c owner-consume adapter (rebased onto main dcd9cc9)
 
 ### Current Contract Inventory — new module (lance-graph-planner)