diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index e6dafbcc2..430d5a0a5 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -13,7 +13,7 @@ flowchart LR C --> F[Typed document-event-entity graph] E --> G[Posterior topic coordinates] F --> G - G --> H[Longitudinal ESEM and DSEM] + G --> H[Longitudinal Modeling] G --> I[Topic and event networks] H --> J[Evidence-grounded interpretation] I --> J @@ -28,31 +28,37 @@ flowchart LR | `temporal_core` | instants, intervals, uncertain dates, partial orders, bitemporal availability and leakage gates | | `event_ontology` | event mentions, event instances, roles, subevents, products, factors, places, and evidence links | | `relation_graph` | typed document, segment, event, entity, revision, translation, evidence, and transition edges | -| `membership_model` | time-varying cross-classified and multiple-membership assignments | +| `membership_model` | time-varying multilevel, cross-classified, and multiple-membership assignments with auditable event-time validity | | `semantic_preprocessor` | Unicode, segmentation, morphology, dependency phrases, LLM span contracts, validation | | `concept_dictionary` | versioned multilingual concept alignment and unknown-concept review | | `topic_measurement` | shared-latent temporal/relational topic estimation and uncertainty | -| `compute_backend` | CPU `f64`, fixed-pool multithreading, CUDA/WGPU, sparse streaming, VRAM budgeting | +| `compute_backend` | CPU `f64`, fixed-pool multithreading, accelerator parity, sparse streaming, VRAM budgeting | | `model_selection` | fitted candidate-K scoring from the CPU reference, predictive fit, coherence, exclusivity, stability, alignment, fairness, blinded LLM review | -| `psychometric_core` | posterior-plausible-value ESEM, longitudinal invariance, DSEM, continuous-time paths | +| `psychometric_core` | posterior-aware non-temporal measurement inputs and structural measurement fitting; reusable generalized/static psychometric arithmetic is consumed from released fast-mlsirm contracts rather than re-owned here | +| `longitudinal_modeling` | event-time longitudinal state/trajectory composition, irregular-gap transitions, longitudinal invariance and DSEM/continuous-time mappings, time-varying membership composition, alignment, and recovery evidence | | `event_intelligence` | TDT segmentation/link/detection/first-story/tracking and CHRONOS schema reasoning | | `network_analysis` | log-ratio topic correlation, conditional networks, uncertainty, Leiden consensus clusters | -| `interpretation_gateway` | evidence-bounded LLM interpretation, independent verification, routing and ablations | +| `interpretation_gateway` | evidence-bounded LLM interpretation and independent verification; never numerical/scientific authority | | `artifact_service` | model registry, manifests, JSON-LD, GraphML, Arrow/Parquet, tables, SVG/PDF exports | -| `visual_analytics` | bitemporal lens, event graph, topic river, drift, ESEM/DSEM builder, invariance and leakage audit | +| `visual_analytics` | bitemporal lens, event graph, topic river, drift, longitudinal builder, invariance and leakage audit | -Every boundary must be independently usable and expose versioned contracts for integration with organization repositories, `naruon`, and `contextual-orchestrator`. +Every boundary must be independently usable and expose versioned contracts for integration with organization repositories. Cross-boundary integration is through released/versioned contracts and explicit ACLs; direct cross-service application-table access and mutable sibling-head dependencies are prohibited. -The `analysis_engine` vertical slice is intentionally separate from `tepp_api`: -the API owns wire contracts while the engine owns deterministic execution. It -does not replace the future topic or psychometric estimators and does not read -another service's application tables. +The `analysis_engine` vertical slice is intentionally separate from `tepp_api`: the API owns wire contracts while the engine owns deterministic execution. It does not replace topic, measurement, or Longitudinal Modeling estimators and does not read another service's application tables. + +### Longitudinal Modeling ownership + +Temporal composition is a TEPP domain responsibility. `longitudinal_modeling` owns the meaning and composition of event-time state/trajectory changes, irregular gaps, time-varying hierarchy/cross-classification/multiple membership, temporal alignment, rolling-origin leakage control, and longitudinal recovery evidence. Its Rust implementation path is `longitudinal_core`. + +Reusable static/generalized-mixed/dependence psychometric arithmetic is not duplicated in TEPP. fast-mlsirm is the canonical owner for that arithmetic and TEPP consumes only immutable released/versioned Published Language through an ACL. A TEPP temporal adapter may combine those static primitives with typed event-time state, membership, and evidence semantics, but it cannot copy the upstream implementation or make an open upstream PR head authoritative. + +`psychometric_core` retains legacy compatibility surfaces for existing measurement code while they are migrated through explicit adapters. Those legacy APIs do not establish new ownership of temporal composition. New temporal/state-transition behavior belongs in `longitudinal_core`; new reusable static psychometric primitives belong in fast-mlsirm. + +Detailed equations, literature claims, recovery fixtures, and exact implementation/test evidence belong in TRACEABILITY, doctoring/research documents, ADRs, and source tests. The responsibility tables below intentionally identify ownership rather than duplicate an equation catalogue. ## Implemented foundation topology -Task 1 materializes the first storage-independent workspace boundaries. The -crate names are stable implementation identifiers, while the broader service -boundaries above remain the target modular MSA architecture. +The crate names are implementation identifiers. A crate can be an adapter or an incremental slice without becoming a new bounded context or architectural authority. | Rust crate | Initial responsibility | |---|---| @@ -60,9 +66,9 @@ boundaries above remain the target modular MSA architecture. | `semantic_core` | span-grounded semantic units; language is not identity | | `location_membership` | location is not entity identity and not a language channel | | `temporal_core` | typed clocks, intervals, and temporal reasoning | -| `event_core` | event instances, span-grounded `EventMention` (the only constructible mention type), roles, provenance, and CHRONOS occurrence-prediction calibration | +| `event_core` | event instances, span-grounded `EventMention`, roles, provenance, and CHRONOS occurrence-prediction calibration | | `relation_graph` | typed relations and forward-transition validation | -| `membership_core` | time-varying cross-classified multiple membership, Kish ESS, nested ICC with non-nested refusal | +| `membership_core` | time-varying cross-classified/multiple membership, Kish ESS, nested ICC with non-nested refusal | | `role_contradiction` | customer and competitor cannot occupy the same group | | `relation_absence` | unobserved relation pairs are not evidence of no relationship | | `persistence_postgres` | PostgreSQL repositories and migrations | @@ -91,18 +97,18 @@ boundaries above remain the target modular MSA architecture. | `assertion_clock` | assertion time cannot be replaced by event, system, document, or available time | | `cutoff_clock` | knowledge cutoff cannot be replaced by event, system, or availability time | | `available_clock` | availability time cannot be replaced by event or system time | -| `document_clocks` | document rows must carry assertion time and document time | +| `document_clocks` | document rows carry assertion time and document time | | `revision_order` | later document revisions must have later system time | -| `encrypted_mapping` | purpose-bound in-memory AES-256-GCM identity mappings; no plaintext persistence or KMS integration | +| `encrypted_mapping` | purpose-bound in-memory AES-256-GCM identity mappings; no plaintext persistence or KMS claim | | `citation_edge` | citation, revision, translation, and retrospective edges are not state transitions | -| `psychometric_fit` | CPU `f64` ESEM loading recovery and event-time DSEM lag gates | -| `subevent_containment` | subevent event-time intervals must stay inside the parent | -| `prediction_contradiction` | Allen promotion gate: `before`/`after` stay contradictory; `meets`/`met_by` stay unsupported; coverage is required before unmatched predicted mass may be authorized for promotion | +| `psychometric_fit` | CPU `f64` ESEM loading recovery and event-time admission at the measurement/Longitudinal ACL | +| `subevent_containment` | subevent event-time intervals stay inside the parent | +| `prediction_contradiction` | Allen promotion gate: contradictory and unsupported relations fail closed unless required evidence exists | | `provider_receipt` | provider-disclosure field-code receipts; source text and identity are not disclosable | -| `operational_log` | operational logs; `try_record` is the only recording API; source text and source identity are not loggable; `persistence_postgres` `audit_event` inserts call the same gate | -| `service_tls` | production TLS bind gates and rustls server config | +| `operational_log` | operational logs; `try_record` is the recording gate and source text/source identity are not loggable | +| `service_tls` | production TLS bind gates and rustls server configuration | | `derived_sensitivity` | derived topic/factor/relation outputs inherit source sensitivity | -| `longitudinal_core` | active-PR: within/between decomposition; refuse between-as-within; component RMSE | +| `longitudinal_core` | Longitudinal Modeling temporal composition and recovery: within/between decomposition, event-time lagged correlation using both marginals, scalar event-time mappings, irregular residual log-rate, and typed interval admission; reusable static psychometric arithmetic is consumed only through released fast-mlsirm ACLs | | `topic_lineage` | global topic identity across active/dormant/reactivated states | | `network_analysis` | compositional cluster-pair gates; raw simplex is not Euclidean | | `interpretation_gateway` | evidence-bounded LLM interpretations; not estimators or observed facts | @@ -110,80 +116,41 @@ boundaries above remain the target modular MSA architecture. | `model_selection` | fitted candidate-`K` scoring from the CPU `f64` reference plus statistical/Pareto gates; LLM votes are not numerical authority | | `checkpoint_authority` | a model checkpoint is not the CPU `f64` estimator | | `compute_backend` | VRAM-budgeted streamed planning, executable OOM retry plans, and a compensated CPU `f64` reference | -| `episode_membership` | episode membership cannot escape the episode event-time interval | | `membership_target` | language, episode, template, department, and opportunity-pool targets cannot collapse into entity or project | | `topic_measurement` | logistic-normal ALR/ILR coordinates and the CPU `f64` TRSL-TM reference estimator | -| `analysis_engine` | bounded cutoff-safe temporal evidence readiness execution and digest-bound terminal artifacts | -| `psychometric_core` | posterior-aware structural input gates, CWC within/between OLS plus the contextual effect, event-time log-rate, unequal-interval discrete-lag remapping, constant-predictor discrete effect, time-varying-predictor discrete effect (Eq. 14), exact scalar discrete process noise (Driver et al., 2017, Eq. 3), lagged latent covariance and unconditional latent variance (Driver et al., 2017, Eq. 3–4), stationary within-subject variance (Driver et al., 2017, Eq. 4 as `Δt → ∞`; `asymDIFFUSION`), trait-plus-state variance (Driver et al., 2017, §4.3 `TRAITVAR`; not process noise), observed-indicator variance and lagged observed covariance (Driver et al., 2017, Eq. 5; Table 2 `MANIFESTVAR` is `Θ`, not `Var(y)`; `MANIFESTTRAITVAR` is not `MANIFESTVAR`; `Θ` does not enter lagged observed covariance; observed-indicator mean is `τ + λ μ`; `MANIFESTMEANS` is not `E(y)`; `CINT` is not `MANIFESTMEANS`; discrete latent mean is `exp(a Δt) μ_0 + (exp(a Δt) − 1)/a κ`; `T0MEANS` is not `μ_t`; `CINT` is not the discrete increment; evolved observed mean is `τ + λ μ_t`; `τ + λ μ_0` is not `E(y_t)`; contemporaneous `TDPREDEFFECT` impulse is `m x`, not `CINT`, not `TIPREDEFFECT`, and not Voelkle Eq. 14; Eq. 5 of that contemporaneous impulse is `τ + λ(μ_t + m x)`, and `τ + λ μ_t` is not that observed mean; time-independent `TIPREDEFFECT` increment is `A^{-1}[e^{A Δt} − I] B z`, not `CINT`, not `M x`, not Voelkle Eq. 14, and not the coefficient `B`; Eq. 5 of that increment is `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + m x)` is not that observed mean; `τ + λ(μ_t + e^{a(t−u)} m x)` is not that observed mean when `u ≠ t`; within-interval `TDPREDEFFECT` carry is `e^{A(t−u)} M x` for `t0 < u < t`, not the contemporaneous Dirac, not `CINT`, not `TIPREDEFFECT`, and not Voelkle Eq. 14; Eq. 5 of that carry is `τ + λ(μ_t + e^{a(t−u)} m x)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + m x)` is not that carried observed mean when `u ≠ t`; first-occasion `T0TIPREDEFFECT` shift is `t0_b z` and Eq. 3 first-summand carry is `e^{A Δt} t0_b z` (`T0TIPREDEFFECT` is not `TIPREDEFFECT` `B`; `t0_b z` is not `A^{-1}[e^{A Δt} − I] B z`; `e^{A Δt} t0_b z` is not `t0_b z`; Eq. 5 of that carry is `τ + λ(μ_t + e^{a Δt} t0_b z)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` is not that observed mean), first-occasion `T0TDPREDEFFECT` shift is `t0_m x0` and Eq. 3 first-summand carry is `e^{A Δt} t0_m x0` (`T0TDPREDEFFECT` is not `TDPREDEFFECT` `M`; `t0_m x0` is not `M x`; `e^{A Δt} t0_m x0` is not `t0_m x0`; `e^{A Δt} t0_m x0` is not `e^{A(t−u)} M x` for `t0 < u < t`; `t0_m x0` is not `t0_b z`; an impulse at `u ≤ t0` that used `M` is already in `η(t0)` as `TDPREDEFFECT`, not as `T0TDPREDEFFECT`; Eq. 5 of that carry is `τ + λ(μ_t + e^{a Δt} t0_m x0)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` is not that observed mean; `τ + λ(μ_t + e^{a Δt} t0_b z)` is not that observed mean; §7.2 level-change `CINT` is `κ = −a m x` with `a < 0` so `−κ / a = m x` (`−a m x` is not the dissipating Dirac, not a free `CINT`, not `TIPREDEFFECT`, and not the extra near-zero-drift latent process also named in §7.2; Eq. 3 of that setting is `(1 − e^{a Δt}) m x`, which is not `m x`, not `κ`, and not `TIPREDEFFECT`; §7.2 extra-process contribution is `a_{ηξ} x (e^{ε Δt} − e^{a Δt}) / (ε − a)` (`ε = a` is `a_{ηξ} x Δt e^{a Δt}`; identification `TDPREDEFFECT` on the extra process is 1; printed extra `DRIFT` is `−0.000001`; not `κ = −a m x`, not `(1 − e^{a Δt}) m x`, and not the dissipating Dirac `m x`; `ε ≥ 0` fails closed; Eq. 5 of that contribution is `τ + λ(μ_t + a_{ηξ} x (e^{ε Δt} − e^{a Δt}) / (ε − a)`; the extra process has `LAMBDA` 0 and is not an observed indicator; `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + m x)` is not that observed mean; the contribution is not `E(y_t)`; the evolved-plus-contribution latent mean is not `E(y_t)`; after-t0 extra-process `TDPREDEFFECT` is `a_{ηξ} x (e^{ε(t−u)} − e^{a(t−u)}) / (ε − a)` for `t0 < u < t` while `μ_t` uses `Δt`; Eq. 5 of that after-t0 contribution is `τ + λ(μ_t + a_{ηξ} x (e^{ε(t−u)} − e^{a(t−u)}) / (ε − a)`; the first-occasion extra-process observed mean is not that observed mean when `u ≠ t0`; `e^{a(t−u)} m x` is a Dirac on the original process, not this `DRIFT` drive; §7.2 `asymTIPREDEFFECT` is `-B z / a` for `a < 0` (`-B z / a` is not the coefficient `B`, not `A^{-1}[e^{A Δt} − I] B z`, not `CINT`, and not `M x`; §7.2 `addedTIPREDVAR` is `(B / a)² v`, not `TRAITVAR`, not `asymDIFFUSION`, and not `-B z / a`; Table 2 `asymCINT` is `-κ / a` for `a < 0` and is not `κ`, not `A^{-1}[e^{A Δt} − I] κ`, not `T0MEANS`, and not `-B z / a`; p. 16 stationary `T0MEANS` is `-κ / a + −B z / a` and is not free `T0MEANS`, not `asymCINT` alone, not `asymTIPREDEFFECT` alone, and not the finite-interval discrete latent mean; Eq. 5 of that constrained mean is `τ + λ(−κ / a + −B z / a)`; `τ + λ μ_0` is not that observed mean; `τ + λ(−κ / a)` is not that observed mean when `B z ≠ 0`; `τ + λ μ_t` is not that observed mean; `MANIFESTMEANS` is not `E(y_0)`; the constrained latent mean is not `E(y_0)`; stationary `T0VAR` is `trait + −q / (2 a) + (B / a)² v` (not free `T0VAR`, not `asymDIFFUSION` alone, not `TRAITVAR` alone, not `addedTIPREDVAR` alone, and not the finite-interval discrete latent variance. Eq. 5 of that constrained variance is `λ²(trait + −q / (2 a) + (B / a)² v) + θ + ψ` (JSS PDF re-opened 2026-08-22T03:20Z; form the stationary latent variance first, then `λ² p + θ + ψ`; `λ² p_0` is not that observed variance; `λ²(−q / (2 a)) + θ` is not that observed variance when `TRAITVAR` or `addedTIPREDVAR` is nonzero; `MANIFESTVAR` is not `Var(y_0)`; the constrained latent variance is not `Var(y_0)`); lagged stationary `T0VAR` is `trait + e^{a Δt}(−q / (2 a)) + (B / a)² v` (trait and `addedTIPREDVAR` do not decay; contemporaneous `T0VAR` is not that lagged map; decaying the constrained total as if it were all state is not that lagged map; Eq. 5 of that lagged covariance is `λ²(trait + e^{a Δt}(−q / (2 a)) + (B / a)² v) + ψ`; `Θ` does not enter; contemporaneous `Var(y_0)` is not that lagged observed covariance; the lagged latent covariance is not that observed covariance); later-occasion stationary `T0VAR` is `trait + e^{2 a Δt}(−q / (2 a)) + Q_Δt + (B / a)² v` (trait and `addedTIPREDVAR` do not enter `Q_Δt`; under stationarity that composition equals contemporaneous `T0VAR`; evolving the constrained total as if it were all state is not that later map; the lagged covariance omits `Q_Δt`; `Q_Δt` is not that later map; Eq. 5 of that later-occasion variance is `λ²(trait + e^{2 a Δt}(−q / (2 a)) + Q_Δt + (B / a)² v) + θ + ψ`; lagged observed covariance omits `Q_Δt` and `θ`; `MANIFESTVAR` is not `Var(y_t)`; the later-occasion latent variance is not `Var(y_t)`))), irregular already-centered residual lag, Rubin `T` on OLS loadings, and strong-gated latent means (two-observation residual variance is identically `0` and caps at strong/scalar; Putnick & Bornstein, 2016) | -| `validation_core` | RMSE, bias, coverage, graph, and Monte Carlo metrics | -| `tepp_api` | versioned DTO, schema, and export contracts | -| `psychometric_core` | posterior-aware structural input gates, CWC within/between OLS plus the contextual effect, event-time log-rate, unequal-interval discrete-lag remapping, constant-predictor discrete effect, time-varying-predictor discrete effect (Eq. 14), exact scalar discrete process noise (Driver et al., 2017, Eq. 3), lagged latent covariance and unconditional latent variance (Driver et al., 2017, Eq. 3–4), stationary within-subject variance (Driver et al., 2017, Eq. 4 as `Δt → ∞`; `asymDIFFUSION`), trait-plus-state variance (Driver et al., 2017, §4.3 `TRAITVAR`; not process noise), observed-indicator variance and lagged observed covariance (Driver et al., 2017, Eq. 5; Table 2 `MANIFESTVAR` is `Θ`, not `Var(y)`; `MANIFESTTRAITVAR` is not `MANIFESTVAR`; `Θ` does not enter lagged observed covariance; observed-indicator mean is `τ + λ μ`; `MANIFESTMEANS` is not `E(y)`; `CINT` is not `MANIFESTMEANS`; discrete latent mean is `exp(a Δt) μ_0 + (exp(a Δt) − 1)/a κ`; `T0MEANS` is not `μ_t`; `CINT` is not the discrete increment; evolved observed mean is `τ + λ μ_t`; `τ + λ μ_0` is not `E(y_t)`; contemporaneous `TDPREDEFFECT` impulse is `m x`, not `CINT`, not `TIPREDEFFECT`, and not Voelkle Eq. 14; Eq. 5 of that contemporaneous impulse is `τ + λ(μ_t + m x)`, and `τ + λ μ_t` is not that observed mean; time-independent `TIPREDEFFECT` increment is `A^{-1}[e^{A Δt} − I] B z`, not `CINT`, not `M x`, not Voelkle Eq. 14, and not the coefficient `B`; Eq. 5 of that increment is `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + m x)` is not that observed mean; `τ + λ(μ_t + e^{a(t−u)} m x)` is not that observed mean when `u ≠ t`; within-interval `TDPREDEFFECT` carry is `e^{A(t−u)} M x` for `t0 < u < t`, not the contemporaneous Dirac, not `CINT`, not `TIPREDEFFECT`, and not Voelkle Eq. 14; Eq. 5 of that carry is `τ + λ(μ_t + e^{a(t−u)} m x)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + m x)` is not that carried observed mean when `u ≠ t`; first-occasion `T0TIPREDEFFECT` shift is `t0_b z` and Eq. 3 first-summand carry is `e^{A Δt} t0_b z` (`T0TIPREDEFFECT` is not `TIPREDEFFECT` `B`; `t0_b z` is not `A^{-1}[e^{A Δt} − I] B z`; `e^{A Δt} t0_b z` is not `t0_b z`; Eq. 5 of that carry is `τ + λ(μ_t + e^{a Δt} t0_b z)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` is not that observed mean), first-occasion `T0TDPREDEFFECT` shift is `t0_m x0` and Eq. 3 first-summand carry is `e^{A Δt} t0_m x0` (`T0TDPREDEFFECT` is not `TDPREDEFFECT` `M`; `t0_m x0` is not `M x`; `e^{A Δt} t0_m x0` is not `t0_m x0`; `e^{A Δt} t0_m x0` is not `e^{A(t−u)} M x` for `t0 < u < t`; `t0_m x0` is not `t0_b z`; an impulse at `u ≤ t0` that used `M` is already in `η(t0)` as `TDPREDEFFECT`, not as `T0TDPREDEFFECT`; Eq. 5 of that carry is `τ + λ(μ_t + e^{a Δt} t0_m x0)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` is not that observed mean; `τ + λ(μ_t + e^{a Δt} t0_b z)` is not that observed mean; §7.2 level-change `CINT` is `κ = −a m x` with `a < 0` so `−κ / a = m x` (`−a m x` is not the dissipating Dirac, not a free `CINT`, not `TIPREDEFFECT`, and not the extra near-zero-drift latent process also named in §7.2; Eq. 3 of that setting is `(1 − e^{a Δt}) m x`, which is not `m x`, not `κ`, and not `TIPREDEFFECT`; §7.2 extra-process contribution is `a_{ηξ} x (e^{ε Δt} − e^{a Δt}) / (ε − a)` (`ε = a` is `a_{ηξ} x Δt e^{a Δt}`; identification `TDPREDEFFECT` on the extra process is 1; printed extra `DRIFT` is `−0.000001`; not `κ = −a m x`, not `(1 − e^{a Δt}) m x`, and not the dissipating Dirac `m x`; `ε ≥ 0` fails closed; Eq. 5 of that contribution is `τ + λ(μ_t + a_{ηξ} x (e^{ε Δt} − e^{a Δt}) / (ε − a)`; the extra process has `LAMBDA` 0 and is not an observed indicator; `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + m x)` is not that observed mean; the contribution is not `E(y_t)`; the evolved-plus-contribution latent mean is not `E(y_t)`; after-t0 extra-process `TDPREDEFFECT` is `a_{ηξ} x (e^{ε(t−u)} − e^{a(t−u)}) / (ε − a)` for `t0 < u < t` while `μ_t` uses `Δt`; Eq. 5 of that after-t0 contribution is `τ + λ(μ_t + a_{ηξ} x (e^{ε(t−u)} − e^{a(t−u)}) / (ε − a)`; the first-occasion extra-process observed mean is not that observed mean when `u ≠ t0`; `e^{a(t−u)} m x` is a Dirac on the original process, not this `DRIFT` drive; §7.2 `asymTIPREDEFFECT` is `-B z / a` for `a < 0` (`-B z / a` is not the coefficient `B`, not `A^{-1}[e^{A Δt} − I] B z`, not `CINT`, and not `M x`; §7.2 `addedTIPREDVAR` is `(B / a)² v`, not `TRAITVAR`, not `asymDIFFUSION`, and not `-B z / a`; Table 2 `asymCINT` is `-κ / a` for `a < 0` and is not `κ`, not `A^{-1}[e^{A Δt} − I] κ`, not `T0MEANS`, and not `-B z / a`; p. 16 stationary `T0MEANS` is `-κ / a + −B z / a` and is not free `T0MEANS`, not `asymCINT` alone, not `asymTIPREDEFFECT` alone, and not the finite-interval discrete latent mean; Eq. 5 of that constrained mean is `τ + λ(−κ / a + −B z / a)`; `τ + λ μ_0` is not that observed mean; `τ + λ(−κ / a)` is not that observed mean when `B z ≠ 0`; `τ + λ μ_t` is not that observed mean; `MANIFESTMEANS` is not `E(y_0)`; the constrained latent mean is not `E(y_0)`; stationary `T0VAR` is `trait + −q / (2 a) + (B / a)² v` (not free `T0VAR`, not `asymDIFFUSION` alone, not `TRAITVAR` alone, not `addedTIPREDVAR` alone, and not the finite-interval discrete latent variance. Eq. 5 of that constrained variance is `λ²(trait + −q / (2 a) + (B / a)² v) + θ + ψ` (JSS PDF re-opened 2026-08-22T03:20Z; form the stationary latent variance first, then `λ² p + θ + ψ`; `λ² p_0` is not that observed variance; `λ²(−q / (2 a)) + θ` is not that observed variance when `TRAITVAR` or `addedTIPREDVAR` is nonzero; `MANIFESTVAR` is not `Var(y_0)`; the constrained latent variance is not `Var(y_0)`); lagged stationary `T0VAR` is `trait + e^{a Δt}(−q / (2 a)) + (B / a)² v` (trait and `addedTIPREDVAR` do not decay; contemporaneous `T0VAR` is not that lagged map; decaying the constrained total as if it were all state is not that lagged map; Eq. 5 of that lagged covariance is `λ²(trait + e^{a Δt}(−q / (2 a)) + (B / a)² v) + ψ`; `Θ` does not enter; contemporaneous `Var(y_0)` is not that lagged observed covariance; the lagged latent covariance is not that observed covariance); later-occasion stationary `T0VAR` is `trait + e^{2 a Δt}(−q / (2 a)) + Q_Δt + (B / a)² v` (trait and `addedTIPREDVAR` do not enter `Q_Δt`; under stationarity that composition equals contemporaneous `T0VAR`; evolving the constrained total as if it were all state is not that later map; the lagged covariance omits `Q_Δt`; `Q_Δt` is not that later map; Eq. 5 of that later-occasion variance is `λ²(trait + e^{2 a Δt}(−q / (2 a)) + Q_Δt + (B / a)² v) + θ + ψ`; lagged observed covariance omits `Q_Δt` and `θ`; `MANIFESTVAR` is not `Var(y_t)`; the later-occasion latent variance is not `Var(y_t)`); predetermined later-occasion `T0VAR` is `trait + e^{2 a Δt} p_0 + Q_Δt + (B / a)² v` (free `T0VAR` `p_0` is not that later map; setting `p_0 = −q / (2 a)` recovers the stationary later-occasion map; stationary later variance uses `−q / (2 a)` in place of `p_0` and is not that later map when `p_0` is free; evolving `trait + p_0 + (B / a)² v` as if it were all state is not that later map; Eq. 5 of that predetermined later-occasion variance is `λ²(trait + e^{2 a Δt} p_0 + Q_Δt + (B / a)² v) + θ + ψ`; `MANIFESTVAR` is not `Var(y_t)`; the predetermined later-occasion latent variance is not `Var(y_t)`; stationary later observed variance is not that observed variance when `p_0` is free); predetermined lagged `T0VAR` is `trait + e^{a Δt} p_0 + (B / a)² v` (free `T0VAR` `p_0` is not that lagged map; setting `p_0 = −q / (2 a)` recovers the stationary lagged map; stationary lagged covariance uses `−q / (2 a)` in place of `p_0` and is not that lagged map when `p_0` is free; evolving `trait + p_0 + (B / a)² v` as if it were all state is not that lagged map; later-occasion variance includes `Q_Δt` and is not that lagged map; Eq. 5 of that predetermined lagged covariance is `λ²(trait + e^{a Δt} p_0 + (B / a)² v) + ψ`; `MANIFESTVAR` does not enter; the predetermined lagged latent covariance is not that observed covariance; predetermined later observed variance includes `Q_Δt` and `θ` and is not that lagged observed covariance; stationary lagged observed covariance is not that observed covariance when `p_0` is free; the predetermined first-occasion variance of §4.3 predetermined `T0VAR` is `trait + p_0 + (B / a)² v`; free `p_0` is not that map; stationary first-occasion variance uses `−q / (2 a)` in place of `p_0` and is not that map when `p_0` is free; lagged covariance decays the state and is not that map; later-occasion variance includes `Q_Δt` and is not that map; Eq. 5 of that predetermined first-occasion variance is `λ²(trait + p_0 + (B / a)² v) + θ + ψ`; `MANIFESTVAR` is not that first-occasion observed variance; the predetermined first-occasion latent variance is not that observed variance; stationary first-occasion observed variance is not that observed variance when `p_0` is free; predetermined later observed variance includes `Q_Δt` and is not that first-occasion observed variance; later-start lagged covariance of predetermined `T0VAR` is `trait + e^{a s}(e^{2 a u} p_0 + Q_u) + (B / a)² v` (Driver et al., 2017, §4.3 `startoffset`; Eq. 4; JSS PDF re-opened 2026-08-23T10:27Z; first-occasion lagged omits `e^{a s} Q_u`; later-occasion variance does not lag; stationary lagged uses `−q / (2 a)`; decaying the later total is not that map; Eq. 5 of that later-start lagged covariance is `λ²` of it plus `ψ`; `Θ` does not enter; first-occasion lagged observed omits `e^{a s} Q_u`; later observed variance includes `Q_u` and `θ`; later-start later-occasion variance of predetermined `T0VAR` is `trait + e^{2 a s}(e^{2 a u} p_0 + Q_u) + Q_s + (B / a)² v` (Driver et al., 2017, §4.3 `startoffset`; Eq. 3–4 Chapman–Kolmogorov `Q_{u+s} = e^{2 a s} Q_u + Q_s`; JSS PDF re-opened 2026-08-23T11:05Z; later-occasion variance at `u` omits `Q_s`; later-start lagged covariance omits `Q_s`; stationary later uses `−q / (2 a)`; evolving the later total as if it were all state is not that map; ignoring `startoffset` omits `e^{2 a s} Q_u`; Eq. 5 of that later-start later-occasion variance is `λ²` of it plus `θ + ψ`; `MANIFESTVAR` is not that observed variance; p. 16 `discreteDRIFTstd` is `e^{a Δt}` after strictly positive `asymDIFFUSION` `-q / (2 a)` (footnote 4; unstandardised `e^{a Δt}` is defined for growing `a ≥ 0` and for zero diffusion and is not `discreteDRIFTstd`; the §7.1 trait-plus-state autocorrelation uses `TRAITVAR` and is not `discreteDRIFTstd`; `TRAITVAR` is not the standardisation variance; p. 16 `discreteDIFFUSIONstd` is `Q_Δt / (−q / (2 a))` after strictly positive `asymDIFFUSION` `-q / (2 a)` (footnote 4; unstandardised `Q_Δt` is defined for growing `a ≥ 0` and for zero diffusion and is not `discreteDIFFUSIONstd`; the continuous standardisation `−2 a` is not `discreteDIFFUSIONstd`; `Q_Δt / (trait + p + added)` uses `TRAITVAR` and is not `discreteDIFFUSIONstd`; `TRAITVAR` is not the standardisation variance; p. 16 `DIFFUSIONstd` is `q / (−q / (2 a)) = −2 a` after strictly positive `asymDIFFUSION` `-q / (2 a)` (Driver et al., 2017, p. 16; Eq. 4; footnote 4; JSS PDF re-opened 2026-08-23T13:20Z; unstandardised `q` is defined for growing `a ≥ 0` and for zero diffusion and is not `DIFFUSIONstd`; the discrete standardisation `Q_Δt / (−q / (2 a))` depends on `Δt` and is not `DIFFUSIONstd`; `q / (trait + p + added)` uses `TRAITVAR` and is not `DIFFUSIONstd`; `TRAITVAR` is not the standardisation variance; p. 16 `DRIFTstd` is the continuous auto-effect after strictly positive `asymDIFFUSION` `-q / (2 a)` (Driver et al., 2017, p. 16; Eq. 1; footnote 4; JSS PDF re-opened 2026-08-23T13:28Z); unstandardised `a` is defined for growing `a ≥ 0` and for zero diffusion and is not `DRIFTstd`; the discrete standardisation `e^{a Δt}` depends on the event interval and is not `DRIFTstd`; `a p / (trait + p + added)` uses `TRAITVAR` and is not `DRIFTstd`; `TRAITVAR` is not the standardisation variance); p. 16 `asymTIPREDEFFECTstd` is `(-B / a) · √v / √(-q / (2 a))` after strictly positive `asymDIFFUSION` `-q / (2 a)` and strictly positive predictor variance `v` (Driver et al., 2017, p. 16; §7.2; footnote 4; JSS PDF re-opened 2026-08-23T14:25Z; unstandardised `-B / a` is defined for a zero coefficient and for zero predictor variance and is not `asymTIPREDEFFECTstd`; the finite-interval standardisation `A^{-1}[e^{A Δt} − I] B · √v / √p` depends on the event interval and is not `asymTIPREDEFFECTstd`; `(-B / a) · √v / √(trait + p + added)` uses `TRAITVAR` and is not `asymTIPREDEFFECTstd`; `TRAITVAR` is not the standardisation variance); p. 16 `TIPREDEFFECTstd` is `B · √v / √(-q / (2 a))` after strictly positive `asymDIFFUSION` `-q / (2 a)` and strictly positive predictor variance `v` (Driver et al., 2017, p. 16; §7.2; footnote 4; JSS PDF re-opened 2026-08-23T16:21Z; unstandardised `B` is defined for a zero coefficient and for zero predictor variance and is not `TIPREDEFFECTstd`; the asymptotic standardisation `(-B / a) · √v / √p` is the total change and is not `TIPREDEFFECTstd`; the finite-interval standardisation `A^{-1}[e^{A Δt} − I] B · √v / √p` depends on the event interval and is not `TIPREDEFFECTstd`; `B · √v / √(trait + p + added)` uses `TRAITVAR` and is not `TIPREDEFFECTstd`; `TRAITVAR` is not the standardisation variance); Table 3 `T0TIPREDEFFECTstd` is `t0_b · √v / √p_0` after strictly positive free `T0VAR` `p_0` and strictly positive predictor variance `v` (Driver et al., 2017, Table 3, p. 13; p. 16; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-23T17:20Z; the affected variance is free `T0VAR`, not `asymDIFFUSION`; unstandardised `t0_b` is defined for a zero coefficient and for zero predictor variance and is not `T0TIPREDEFFECTstd`; `TIPREDEFFECTstd` `B · √v / √(-q / (2 a))` is the continuous coefficient and is not `T0TIPREDEFFECTstd`; `asymTIPREDEFFECTstd` `(-B / a) · √v / √p` is the total change and is not `T0TIPREDEFFECTstd`; `t0_b · √v / √(trait + p_0 + added)` uses `TRAITVAR` and is not `T0TIPREDEFFECTstd`; `TRAITVAR` is not the standardisation variance); 2017-era `addedT0TIPREDVAR` is `t0_b² v` (Driver et al., 2017, Table 3, p. 13; p. 16; §7.2; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-23T18:20Z; `T0TIPREDEFFECT %*% TIPREDVAR %*% t(T0TIPREDEFFECT)` immediately after `T0TIPREDEFFECTstd`; form `t0_b` first, then square, then multiply by `v`; a zero coefficient or zero predictor variance is exactly zero; free `T0TIPREDEFFECT` does not require `a < 0`; `(B / a)² v` is `addedTIPREDVAR` and is not this first-occasion map; `t0_b · √v / √p_0` is `T0TIPREDEFFECTstd` and is not this variance; free `T0VAR` is not this extra TI variance; `TRAITVAR` is not this extra TI variance; Equation 5 of 2017-era `addedT0TIPREDVAR` is `λ² t0_b² v` (Driver et al., 2017, Eq. 5, p. 5; Table 3, p. 13; Table 2, p. 12; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-23T19:10Z; form `t0_b² v` first, then `(λ extra) λ` with `θ = 0`; a zero loading or zero extra is exactly zero; `t0_b² v` is the latent extra, not the observed extra; `λ² p_0 + θ` is first-occasion observed variance, not this extra; `λ² (B / a)² v` is Eq. 5 of `addedTIPREDVAR`, not this first-occasion observed extra; `MANIFESTVAR` `θ` is not this extra; Equation 5 of §7.2 `addedTIPREDVAR` is `λ² (B / a)² v`; form `(B / a)² v` first, then `(λ extra) λ` with `θ = 0`; a zero loading or zero extra is exactly zero; lasting asymptotic extra requires `a < 0`; `(B / a)² v` is the latent extra, not the observed extra; `λ² t0_b² v` is first-occasion extra observed TI variance, not this extra; `λ² p + θ` is stationary observed variance, not this extra; `MANIFESTVAR` `θ` is not this extra; p. 16 `TDPREDEFFECTstd` is `m · √v / √(-q / (2 a))` after strictly positive `asymDIFFUSION` and strictly positive time-dependent predictor variance; unstandardised `M` is not `TDPREDEFFECTstd`; `TIPREDEFFECTstd` is not `TDPREDEFFECTstd` even when `M = B`; intercept-style `A^{-1}[e^{A Δt} − I] M · √v / √p` is not `TDPREDEFFECTstd`; `m · √v / √(trait + p + added)` uses `TRAITVAR` and is not `TDPREDEFFECTstd`; Table 3 / p. 16 `T0TDPREDEFFECTstd` is `t0_m · √v / √p_0` after strictly positive free `T0VAR` and strictly positive TD predictor variance; unstandardised `t0_m` is not `T0TDPREDEFFECTstd`; `TDPREDEFFECTstd` uses `asymDIFFUSION` and is not `T0TDPREDEFFECTstd`; `T0TIPREDEFFECTstd` is not `T0TDPREDEFFECTstd` even when `t0_m = t0_b`; `t0_m · √v / √(trait + p_0 + added)` uses `TRAITVAR` and is not `T0TDPREDEFFECTstd`; free `T0VAR` does not require `a < 0`; p. 16 `T0VARstd` is `p_0 / p_0 = 1` after strictly positive free `T0VAR` (`solve(sqrt(diag(T0VAR))) %&% T0VAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; default ridge is 0); unstandardised `T0VAR` is not `T0VARstd`; `T0TDPREDEFFECTstd` is not `T0VARstd`; `addedT0TIPREDVAR` is not `T0VARstd`; p. 16 `TRAITVARstd` is `trait / trait = 1` after strictly positive `TRAITVAR` (`solve(sqrt(diag(TRAITVAR))) %&% TRAITVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; no ridge addend); unstandardised `TRAITVAR` is not `TRAITVARstd`; `T0VARstd` is not `TRAITVARstd` even when both equal 1; `addedT0TIPREDVAR` is not `TRAITVARstd`; p. 16 `MANIFESTTRAITVARstd` is `ψ / ψ = 1` after strictly positive `MANIFESTTRAITVAR` (`solve(sqrt(diag(MANIFESTTRAITVAR))) %&% MANIFESTTRAITVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0); unstandardised `MANIFESTTRAITVAR` is not `MANIFESTTRAITVARstd`; `TRAITVARstd` is not `MANIFESTTRAITVARstd` even when both equal 1; `MANIFESTVAR` is not `MANIFESTTRAITVARstd`; p. 16 `MANIFESTVARstd` is `θ / θ = 1` after strictly positive `MANIFESTVAR` (`solve(sqrt(diag(MANIFESTVAR))) %&% MANIFESTVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0; 2017-era `dimnames` assignment to `latentNames` is a source bug); unstandardised `MANIFESTVAR` is not `MANIFESTVARstd`; `MANIFESTTRAITVARstd` is not `MANIFESTVARstd` even when both equal 1; Equation 5 `Var(y)` is not `MANIFESTVARstd`; p. 16 `TIPREDVARstd` is `v / v = 1` after strictly positive `TIPREDVAR` (`solve(sqrt(diag(TIPREDVAR))) %&% TIPREDVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0; `dimnames` are `TIpredNames`); unstandardised `TIPREDVAR` is not `TIPREDVARstd`; `MANIFESTVARstd` is not `TIPREDVARstd` even when both equal 1; §7.2 `addedTIPREDVAR` is not `TIPREDVARstd`; p. 16 `asymDIFFUSIONstd` is `p / p = 1` after strictly positive `asymDIFFUSION` (`solve(sqrt(diag(asymDIFFUSION))) %&% asymDIFFUSION`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0; `dimnames` are `latentNames`); unstandardised `asymDIFFUSION` is not `asymDIFFUSIONstd`; `TIPREDVARstd` is not `asymDIFFUSIONstd` even when both equal 1; `DIFFUSIONstd` `−2 a` is not `asymDIFFUSIONstd`; p. 16 `discreteCINTstd` is `A^{-1}[e^{A Δt} − I] κ / √p` after strictly positive `asymDIFFUSION`; unstandardised `discreteCINT` is not `discreteCINTstd`; `κ / √p` is not `discreteCINTstd`; `(-κ / a) / √p` is not `discreteCINTstd`; `asymCINTstd` is `(-κ / a) / √p` after strictly positive `asymDIFFUSION`; unstandardised `asymCINT` is not `asymCINTstd`; `κ / √p` is not `asymCINTstd`; `discreteCINTstd` is not `asymCINTstd`; `T0MEANSstd` is `μ_0 / √p_0` after strictly positive free `T0VAR`; unstandardised `T0MEANS` is not `T0MEANSstd`; `T0VARstd` is not `T0MEANSstd`; `μ_0 / √asymDIFFUSION` is not `T0MEANSstd`; `MANIFESTMEANSstd` is `τ / √θ` after strictly positive `MANIFESTVAR`; unstandardised `MANIFESTMEANS` is not `MANIFESTMEANSstd`; `MANIFESTVARstd` is not `MANIFESTMEANSstd`; `τ / √(λ² Var(η) + θ)` is not `MANIFESTMEANSstd`; p. 16 `CINTstd` is `κ / √p` after strictly positive `asymDIFFUSION`; unstandardised `CINT` is not `CINTstd`; `asymCINTstd` is not `CINTstd`; `discreteCINTstd` is not `CINTstd`; `κ / √(trait + p + added)` is not `CINTstd`;))))), irregular already-centered residual lag, Rubin `T` on OLS loadings, and strong-gated latent means (two-observation residual variance is identically `0` and caps at strong/scalar; Putnick & Bornstein, 2016) | -| `psychometric_core` | posterior-aware structural input gates, CWC within/between OLS plus the contextual effect, event-time log-rate, unequal-interval discrete-lag remapping, constant-predictor discrete effect, time-varying-predictor discrete effect (Eq. 14), exact scalar discrete process noise (Driver et al., 2017, Eq. 3), lagged latent covariance and unconditional latent variance (Driver et al., 2017, Eq. 3–4), stationary within-subject variance (Driver et al., 2017, Eq. 4 as `Δt → ∞`; `asymDIFFUSION`), trait-plus-state variance (Driver et al., 2017, §4.3 `TRAITVAR`; not process noise), observed-indicator variance and lagged observed covariance (Driver et al., 2017, Eq. 5; Table 2 `MANIFESTVAR` is `Θ`, not `Var(y)`; `MANIFESTTRAITVAR` is not `MANIFESTVAR`; `Θ` does not enter lagged observed covariance; observed-indicator mean is `τ + λ μ`; `MANIFESTMEANS` is not `E(y)`; `CINT` is not `MANIFESTMEANS`; discrete latent mean is `exp(a Δt) μ_0 + (exp(a Δt) − 1)/a κ`; `T0MEANS` is not `μ_t`; `CINT` is not the discrete increment; evolved observed mean is `τ + λ μ_t`; `τ + λ μ_0` is not `E(y_t)`; contemporaneous `TDPREDEFFECT` impulse is `m x`, not `CINT`, not `TIPREDEFFECT`, and not Voelkle Eq. 14; Eq. 5 of that contemporaneous impulse is `τ + λ(μ_t + m x)`, and `τ + λ μ_t` is not that observed mean; time-independent `TIPREDEFFECT` increment is `A^{-1}[e^{A Δt} − I] B z`, not `CINT`, not `M x`, not Voelkle Eq. 14, and not the coefficient `B`; Eq. 5 of that increment is `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + m x)` is not that observed mean; `τ + λ(μ_t + e^{a(t−u)} m x)` is not that observed mean when `u ≠ t`; within-interval `TDPREDEFFECT` carry is `e^{A(t−u)} M x` for `t0 < u < t`, not the contemporaneous Dirac, not `CINT`, not `TIPREDEFFECT`, and not Voelkle Eq. 14; Eq. 5 of that carry is `τ + λ(μ_t + e^{a(t−u)} m x)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + m x)` is not that carried observed mean when `u ≠ t`; first-occasion `T0TIPREDEFFECT` shift is `t0_b z` and Eq. 3 first-summand carry is `e^{A Δt} t0_b z` (`T0TIPREDEFFECT` is not `TIPREDEFFECT` `B`; `t0_b z` is not `A^{-1}[e^{A Δt} − I] B z`; `e^{A Δt} t0_b z` is not `t0_b z`; Eq. 5 of that carry is `τ + λ(μ_t + e^{a Δt} t0_b z)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` is not that observed mean), first-occasion `T0TDPREDEFFECT` shift is `t0_m x0` and Eq. 3 first-summand carry is `e^{A Δt} t0_m x0` (`T0TDPREDEFFECT` is not `TDPREDEFFECT` `M`; `t0_m x0` is not `M x`; `e^{A Δt} t0_m x0` is not `t0_m x0`; `e^{A Δt} t0_m x0` is not `e^{A(t−u)} M x` for `t0 < u < t`; `t0_m x0` is not `t0_b z`; an impulse at `u ≤ t0` that used `M` is already in `η(t0)` as `TDPREDEFFECT`, not as `T0TDPREDEFFECT`; Eq. 5 of that carry is `τ + λ(μ_t + e^{a Δt} t0_m x0)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` is not that observed mean; `τ + λ(μ_t + e^{a Δt} t0_b z)` is not that observed mean; §7.2 level-change `CINT` is `κ = −a m x` with `a < 0` so `−κ / a = m x` (`−a m x` is not the dissipating Dirac, not a free `CINT`, not `TIPREDEFFECT`, and not the extra near-zero-drift latent process also named in §7.2; Eq. 3 of that setting is `(1 − e^{a Δt}) m x`, which is not `m x`, not `κ`, and not `TIPREDEFFECT`; §7.2 extra-process contribution is `a_{ηξ} x (e^{ε Δt} − e^{a Δt}) / (ε − a)` (`ε = a` is `a_{ηξ} x Δt e^{a Δt}`; identification `TDPREDEFFECT` on the extra process is 1; printed extra `DRIFT` is `−0.000001`; not `κ = −a m x`, not `(1 − e^{a Δt}) m x`, and not the dissipating Dirac `m x`; `ε ≥ 0` fails closed; Eq. 5 of that contribution is `τ + λ(μ_t + a_{ηξ} x (e^{ε Δt} − e^{a Δt}) / (ε − a)`; the extra process has `LAMBDA` 0 and is not an observed indicator; `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + m x)` is not that observed mean; the contribution is not `E(y_t)`; the evolved-plus-contribution latent mean is not `E(y_t)`; after-t0 extra-process `TDPREDEFFECT` is `a_{ηξ} x (e^{ε(t−u)} − e^{a(t−u)}) / (ε − a)` for `t0 < u < t` while `μ_t` uses `Δt`; Eq. 5 of that after-t0 contribution is `τ + λ(μ_t + a_{ηξ} x (e^{ε(t−u)} − e^{a(t−u)}) / (ε − a)`; the first-occasion extra-process observed mean is not that observed mean when `u ≠ t0`; `e^{a(t−u)} m x` is a Dirac on the original process, not this `DRIFT` drive; §7.2 `asymTIPREDEFFECT` is `-B z / a` for `a < 0` (`-B z / a` is not the coefficient `B`, not `A^{-1}[e^{A Δt} − I] B z`, not `CINT`, and not `M x`; §7.2 `addedTIPREDVAR` is `(B / a)² v`, not `TRAITVAR`, not `asymDIFFUSION`, and not `-B z / a`; Table 2 `asymCINT` is `-κ / a` for `a < 0` and is not `κ`, not `A^{-1}[e^{A Δt} − I] κ`, not `T0MEANS`, and not `-B z / a`; p. 16 stationary `T0MEANS` is `-κ / a + −B z / a` and is not free `T0MEANS`, not `asymCINT` alone, not `asymTIPREDEFFECT` alone, and not the finite-interval discrete latent mean; Eq. 5 of that constrained mean is `τ + λ(−κ / a + −B z / a)`; `τ + λ μ_0` is not that observed mean; `τ + λ(−κ / a)` is not that observed mean when `B z ≠ 0`; `τ + λ μ_t` is not that observed mean; `MANIFESTMEANS` is not `E(y_0)`; the constrained latent mean is not `E(y_0)`; stationary `T0VAR` is `trait + −q / (2 a) + (B / a)² v` (not free `T0VAR`, not `asymDIFFUSION` alone, not `TRAITVAR` alone, not `addedTIPREDVAR` alone, and not the finite-interval discrete latent variance. Eq. 5 of that constrained variance is `λ²(trait + −q / (2 a) + (B / a)² v) + θ + ψ` (JSS PDF re-opened 2026-08-22T03:20Z; form the stationary latent variance first, then `λ² p + θ + ψ`; `λ² p_0` is not that observed variance; `λ²(−q / (2 a)) + θ` is not that observed variance when `TRAITVAR` or `addedTIPREDVAR` is nonzero; `MANIFESTVAR` is not `Var(y_0)`; the constrained latent variance is not `Var(y_0)`); lagged stationary `T0VAR` is `trait + e^{a Δt}(−q / (2 a)) + (B / a)² v` (trait and `addedTIPREDVAR` do not decay; contemporaneous `T0VAR` is not that lagged map; decaying the constrained total as if it were all state is not that lagged map; Eq. 5 of that lagged covariance is `λ²(trait + e^{a Δt}(−q / (2 a)) + (B / a)² v) + ψ`; `Θ` does not enter; contemporaneous `Var(y_0)` is not that lagged observed covariance; the lagged latent covariance is not that observed covariance); later-occasion stationary `T0VAR` is `trait + e^{2 a Δt}(−q / (2 a)) + Q_Δt + (B / a)² v` (trait and `addedTIPREDVAR` do not enter `Q_Δt`; under stationarity that composition equals contemporaneous `T0VAR`; evolving the constrained total as if it were all state is not that later map; the lagged covariance omits `Q_Δt`; `Q_Δt` is not that later map; Eq. 5 of that later-occasion variance is `λ²(trait + e^{2 a Δt}(−q / (2 a)) + Q_Δt + (B / a)² v) + θ + ψ`; lagged observed covariance omits `Q_Δt` and `θ`; `MANIFESTVAR` is not `Var(y_t)`; the later-occasion latent variance is not `Var(y_t)`); predetermined later-occasion `T0VAR` is `trait + e^{2 a Δt} p_0 + Q_Δt + (B / a)² v` (free `T0VAR` `p_0` is not that later map; setting `p_0 = −q / (2 a)` recovers the stationary later-occasion map; stationary later variance uses `−q / (2 a)` in place of `p_0` and is not that later map when `p_0` is free; evolving `trait + p_0 + (B / a)² v` as if it were all state is not that later map; Eq. 5 of that predetermined later-occasion variance is `λ²(trait + e^{2 a Δt} p_0 + Q_Δt + (B / a)² v) + θ + ψ`; `MANIFESTVAR` is not `Var(y_t)`; the predetermined later-occasion latent variance is not `Var(y_t)`; stationary later observed variance is not that observed variance when `p_0` is free); predetermined lagged `T0VAR` is `trait + e^{a Δt} p_0 + (B / a)² v` (free `T0VAR` `p_0` is not that lagged map; setting `p_0 = −q / (2 a)` recovers the stationary lagged map; stationary lagged covariance uses `−q / (2 a)` in place of `p_0` and is not that lagged map when `p_0` is free; evolving `trait + p_0 + (B / a)² v` as if it were all state is not that lagged map; later-occasion variance includes `Q_Δt` and is not that lagged map; Eq. 5 of that predetermined lagged covariance is `λ²(trait + e^{a Δt} p_0 + (B / a)² v) + ψ`; `MANIFESTVAR` does not enter; the predetermined lagged latent covariance is not that observed covariance; predetermined later observed variance includes `Q_Δt` and `θ` and is not that lagged observed covariance; stationary lagged observed covariance is not that observed covariance when `p_0` is free; the predetermined first-occasion variance of §4.3 predetermined `T0VAR` is `trait + p_0 + (B / a)² v`; free `p_0` is not that map; stationary first-occasion variance uses `−q / (2 a)` in place of `p_0` and is not that map when `p_0` is free; lagged covariance decays the state and is not that map; later-occasion variance includes `Q_Δt` and is not that map; Eq. 5 of that predetermined first-occasion variance is `λ²(trait + p_0 + (B / a)² v) + θ + ψ`; `MANIFESTVAR` is not that first-occasion observed variance; the predetermined first-occasion latent variance is not that observed variance; stationary first-occasion observed variance is not that observed variance when `p_0` is free; predetermined later observed variance includes `Q_Δt` and is not that first-occasion observed variance; later-start lagged covariance of predetermined `T0VAR` is `trait + e^{a s}(e^{2 a u} p_0 + Q_u) + (B / a)² v` (Driver et al., 2017, §4.3 `startoffset`; Eq. 4; JSS PDF re-opened 2026-08-23T10:27Z; first-occasion lagged omits `e^{a s} Q_u`; later-occasion variance does not lag; stationary lagged uses `−q / (2 a)`; decaying the later total is not that map; Eq. 5 of that later-start lagged covariance is `λ²` of it plus `ψ`; `Θ` does not enter; first-occasion lagged observed omits `e^{a s} Q_u`; later observed variance includes `Q_u` and `θ`; later-start later-occasion variance of predetermined `T0VAR` is `trait + e^{2 a s}(e^{2 a u} p_0 + Q_u) + Q_s + (B / a)² v` (Driver et al., 2017, §4.3 `startoffset`; Eq. 3–4 Chapman–Kolmogorov `Q_{u+s} = e^{2 a s} Q_u + Q_s`; JSS PDF re-opened 2026-08-23T11:05Z; later-occasion variance at `u` omits `Q_s`; later-start lagged covariance omits `Q_s`; stationary later uses `−q / (2 a)`; evolving the later total as if it were all state is not that map; ignoring `startoffset` omits `e^{2 a s} Q_u`; Eq. 5 of that later-start later-occasion variance is `λ²` of it plus `θ + ψ`; `MANIFESTVAR` is not that observed variance; p. 16 `discreteDRIFTstd` is `e^{a Δt}` after strictly positive `asymDIFFUSION` `-q / (2 a)` (footnote 4; unstandardised `e^{a Δt}` is defined for growing `a ≥ 0` and for zero diffusion and is not `discreteDRIFTstd`; the §7.1 trait-plus-state autocorrelation uses `TRAITVAR` and is not `discreteDRIFTstd`; `TRAITVAR` is not the standardisation variance; p. 16 `discreteDIFFUSIONstd` is `Q_Δt / (−q / (2 a))` after strictly positive `asymDIFFUSION` `-q / (2 a)` (footnote 4; unstandardised `Q_Δt` is defined for growing `a ≥ 0` and for zero diffusion and is not `discreteDIFFUSIONstd`; the continuous standardisation `−2 a` is not `discreteDIFFUSIONstd`; `Q_Δt / (trait + p + added)` uses `TRAITVAR` and is not `discreteDIFFUSIONstd`; `TRAITVAR` is not the standardisation variance; p. 16 `DIFFUSIONstd` is `q / (−q / (2 a)) = −2 a` after strictly positive `asymDIFFUSION` `-q / (2 a)` (Driver et al., 2017, p. 16; Eq. 4; footnote 4; JSS PDF re-opened 2026-08-23T13:20Z; unstandardised `q` is defined for growing `a ≥ 0` and for zero diffusion and is not `DIFFUSIONstd`; the discrete standardisation `Q_Δt / (−q / (2 a))` depends on `Δt` and is not `DIFFUSIONstd`; `q / (trait + p + added)` uses `TRAITVAR` and is not `DIFFUSIONstd`; `TRAITVAR` is not the standardisation variance; p. 16 `DRIFTstd` is the continuous auto-effect after strictly positive `asymDIFFUSION` `-q / (2 a)` (Driver et al., 2017, p. 16; Eq. 1; footnote 4; JSS PDF re-opened 2026-08-23T13:28Z); unstandardised `a` is defined for growing `a ≥ 0` and for zero diffusion and is not `DRIFTstd`; the discrete standardisation `e^{a Δt}` depends on the event interval and is not `DRIFTstd`; `a p / (trait + p + added)` uses `TRAITVAR` and is not `DRIFTstd`; `TRAITVAR` is not the standardisation variance); p. 16 `asymTIPREDEFFECTstd` is `(-B / a) · √v / √(-q / (2 a))` after strictly positive `asymDIFFUSION` `-q / (2 a)` and strictly positive predictor variance `v` (Driver et al., 2017, p. 16; §7.2; footnote 4; JSS PDF re-opened 2026-08-23T14:25Z; unstandardised `-B / a` is defined for a zero coefficient and for zero predictor variance and is not `asymTIPREDEFFECTstd`; the finite-interval standardisation `A^{-1}[e^{A Δt} − I] B · √v / √p` depends on the event interval and is not `asymTIPREDEFFECTstd`; `(-B / a) · √v / √(trait + p + added)` uses `TRAITVAR` and is not `asymTIPREDEFFECTstd`; `TRAITVAR` is not the standardisation variance); p. 16 `TIPREDEFFECTstd` is `B · √v / √(-q / (2 a))` after strictly positive `asymDIFFUSION` `-q / (2 a)` and strictly positive predictor variance `v` (Driver et al., 2017, p. 16; §7.2; footnote 4; JSS PDF re-opened 2026-08-23T16:21Z; unstandardised `B` is defined for a zero coefficient and for zero predictor variance and is not `TIPREDEFFECTstd`; the asymptotic standardisation `(-B / a) · √v / √p` is the total change and is not `TIPREDEFFECTstd`; the finite-interval standardisation `A^{-1}[e^{A Δt} − I] B · √v / √p` depends on the event interval and is not `TIPREDEFFECTstd`; `B · √v / √(trait + p + added)` uses `TRAITVAR` and is not `TIPREDEFFECTstd`; `TRAITVAR` is not the standardisation variance); Table 3 `T0TIPREDEFFECTstd` is `t0_b · √v / √p_0` after strictly positive free `T0VAR` `p_0` and strictly positive predictor variance `v` (Driver et al., 2017, Table 3, p. 13; p. 16; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-23T17:20Z; the affected variance is free `T0VAR`, not `asymDIFFUSION`; unstandardised `t0_b` is defined for a zero coefficient and for zero predictor variance and is not `T0TIPREDEFFECTstd`; `TIPREDEFFECTstd` `B · √v / √(-q / (2 a))` is the continuous coefficient and is not `T0TIPREDEFFECTstd`; `asymTIPREDEFFECTstd` `(-B / a) · √v / √p` is the total change and is not `T0TIPREDEFFECTstd`; `t0_b · √v / √(trait + p_0 + added)` uses `TRAITVAR` and is not `T0TIPREDEFFECTstd`; `TRAITVAR` is not the standardisation variance); 2017-era `addedT0TIPREDVAR` is `t0_b² v` (Driver et al., 2017, Table 3, p. 13; p. 16; §7.2; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-23T18:20Z; `T0TIPREDEFFECT %*% TIPREDVAR %*% t(T0TIPREDEFFECT)` immediately after `T0TIPREDEFFECTstd`; form `t0_b` first, then square, then multiply by `v`; a zero coefficient or zero predictor variance is exactly zero; free `T0TIPREDEFFECT` does not require `a < 0`; `(B / a)² v` is `addedTIPREDVAR` and is not this first-occasion map; `t0_b · √v / √p_0` is `T0TIPREDEFFECTstd` and is not this variance; free `T0VAR` is not this extra TI variance; `TRAITVAR` is not this extra TI variance; Equation 5 of 2017-era `addedT0TIPREDVAR` is `λ² t0_b² v` (Driver et al., 2017, Eq. 5, p. 5; Table 3, p. 13; Table 2, p. 12; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-23T19:10Z; form `t0_b² v` first, then `(λ extra) λ` with `θ = 0`; a zero loading or zero extra is exactly zero; `t0_b² v` is the latent extra, not the observed extra; `λ² p_0 + θ` is first-occasion observed variance, not this extra; `λ² (B / a)² v` is Eq. 5 of `addedTIPREDVAR`, not this first-occasion observed extra; `MANIFESTVAR` `θ` is not this extra; Equation 5 of §7.2 `addedTIPREDVAR` is `λ² (B / a)² v`; form `(B / a)² v` first, then `(λ extra) λ` with `θ = 0`; a zero loading or zero extra is exactly zero; lasting asymptotic extra requires `a < 0`; `(B / a)² v` is the latent extra, not the observed extra; `λ² t0_b² v` is first-occasion extra observed TI variance, not this extra; `λ² p + θ` is stationary observed variance, not this extra; `MANIFESTVAR` `θ` is not this extra; p. 16 `TDPREDEFFECTstd` is `m · √v / √(-q / (2 a))` after strictly positive `asymDIFFUSION` and strictly positive time-dependent predictor variance; unstandardised `M` is not `TDPREDEFFECTstd`; `TIPREDEFFECTstd` is not `TDPREDEFFECTstd` even when `M = B`; intercept-style `A^{-1}[e^{A Δt} − I] M · √v / √p` is not `TDPREDEFFECTstd`; `m · √v / √(trait + p + added)` uses `TRAITVAR` and is not `TDPREDEFFECTstd`; Table 3 / p. 16 `T0TDPREDEFFECTstd` is `t0_m · √v / √p_0` after strictly positive free `T0VAR` and strictly positive TD predictor variance; unstandardised `t0_m` is not `T0TDPREDEFFECTstd`; `TDPREDEFFECTstd` uses `asymDIFFUSION` and is not `T0TDPREDEFFECTstd`; `T0TIPREDEFFECTstd` is not `T0TDPREDEFFECTstd` even when `t0_m = t0_b`; `t0_m · √v / √(trait + p_0 + added)` uses `TRAITVAR` and is not `T0TDPREDEFFECTstd`; free `T0VAR` does not require `a < 0`; p. 16 `T0VARstd` is `p_0 / p_0 = 1` after strictly positive free `T0VAR` (`solve(sqrt(diag(T0VAR))) %&% T0VAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; default ridge is 0); unstandardised `T0VAR` is not `T0VARstd`; `T0TDPREDEFFECTstd` is not `T0VARstd`; `addedT0TIPREDVAR` is not `T0VARstd`; p. 16 `TRAITVARstd` is `trait / trait = 1` after strictly positive `TRAITVAR` (`solve(sqrt(diag(TRAITVAR))) %&% TRAITVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; no ridge addend); unstandardised `TRAITVAR` is not `TRAITVARstd`; `T0VARstd` is not `TRAITVARstd` even when both equal 1; `addedT0TIPREDVAR` is not `TRAITVARstd`; p. 16 `MANIFESTTRAITVARstd` is `ψ / ψ = 1` after strictly positive `MANIFESTTRAITVAR` (`solve(sqrt(diag(MANIFESTTRAITVAR))) %&% MANIFESTTRAITVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0); unstandardised `MANIFESTTRAITVAR` is not `MANIFESTTRAITVARstd`; `TRAITVARstd` is not `MANIFESTTRAITVARstd` even when both equal 1; `MANIFESTVAR` is not `MANIFESTTRAITVARstd`; p. 16 `MANIFESTVARstd` is `θ / θ = 1` after strictly positive `MANIFESTVAR` (`solve(sqrt(diag(MANIFESTVAR))) %&% MANIFESTVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0; 2017-era `dimnames` assignment to `latentNames` is a source bug); unstandardised `MANIFESTVAR` is not `MANIFESTVARstd`; `MANIFESTTRAITVARstd` is not `MANIFESTVARstd` even when both equal 1; Equation 5 `Var(y)` is not `MANIFESTVARstd`; p. 16 `TIPREDVARstd` is `v / v = 1` after strictly positive `TIPREDVAR` (`solve(sqrt(diag(TIPREDVAR))) %&% TIPREDVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0; `dimnames` are `TIpredNames`); unstandardised `TIPREDVAR` is not `TIPREDVARstd`; `MANIFESTVARstd` is not `TIPREDVARstd` even when both equal 1; §7.2 `addedTIPREDVAR` is not `TIPREDVARstd`; p. 16 `asymDIFFUSIONstd` is `p / p = 1` after strictly positive `asymDIFFUSION` (`solve(sqrt(diag(asymDIFFUSION))) %&% asymDIFFUSION`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0; `dimnames` are `latentNames`); unstandardised `asymDIFFUSION` is not `asymDIFFUSIONstd`; `TIPREDVARstd` is not `asymDIFFUSIONstd` even when both equal 1; `DIFFUSIONstd` `−2 a` is not `asymDIFFUSIONstd`; p. 16 `discreteCINTstd` is `A^{-1}[e^{A Δt} − I] κ / √p` after strictly positive `asymDIFFUSION`; unstandardised `discreteCINT` is not `discreteCINTstd`; `κ / √p` is not `discreteCINTstd`; `(-κ / a) / √p` is not `discreteCINTstd`; `asymCINTstd` is `(-κ / a) / √p` after strictly positive `asymDIFFUSION`; unstandardised `asymCINT` is not `asymCINTstd`; `κ / √p` is not `asymCINTstd`; `discreteCINTstd` is not `asymCINTstd`; `T0MEANSstd` is `μ_0 / √p_0` after strictly positive free `T0VAR`; unstandardised `T0MEANS` is not `T0MEANSstd`; `T0VARstd` is not `T0MEANSstd`; `μ_0 / √asymDIFFUSION` is not `T0MEANSstd`;))))), irregular already-centered residual lag, Rubin `T` on OLS loadings, and strong-gated latent means (two-observation residual variance is identically `0` and caps at strong/scalar; Putnick & Bornstein, 2016) | +| `psychometric_core` | posterior-aware structural measurement gates and existing non-temporal fitting/compatibility surfaces; it is not the owner for new event-time temporal composition or reusable static generalized psychometric arithmetic | +Foundation crates expose only tested contracts. Empty façades are not public APIs. No crate exposes placeholder production behavior merely to reserve an API. -Foundation crates expose only tested contracts. Empty façades are not public -APIs. +## Immutable evidence boundary -No crate exposes placeholder production behavior in Task 1. This prevents an -empty façade from becoming a de facto public API before its invariants and tests -exist. +Stable RFC 9562 `UUIDv7` identities are independent from canonical `SHA-256` content digests. Source bytes and UTF-8 document text are copied into immutable owned storage, bounded before allocation, and verified without exposing mutable fields. -## Immutable evidence boundary +A source span records an owning document, a half-open UTF-8 byte range, the matching half-open Unicode-scalar range, and optional page/layout geometry. It fails closed for empty or reversed ranges, byte or scalar overflow, mid-code-point boundaries, coordinate disagreement, cross-document use, nonfinite geometry, nonpositive dimensions, and rectangles outside the page. Scalar coordinates are evidence locations rather than grapheme, word, or sentence boundaries; language-tailored segmentation remains a separate boundary. + +The evidence boundary exposes a strict versioned JSON wire contract without exposing private Rust fields. Reconstruction revalidates identifiers, canonical digests, content limits, exact text coordinates, ownership, and page geometry. Malformed/unsupported input and unknown nested fields fail closed with content-redacting errors. -Task 2 begins the executable `evidence_core` boundary. Stable RFC 9562 `UUIDv7` -identities are independent from canonical `SHA-256` content digests. Source -bytes and UTF-8 document text are copied into immutable owned storage, bounded -before allocation, and verified without exposing mutable fields. - -A source span records an owning document, a half-open UTF-8 byte range, the -matching half-open Unicode-scalar range, and optional page/layout geometry. It -fails closed for empty or reversed ranges, byte or scalar overflow, -mid-code-point boundaries, coordinate disagreement, cross-document use, -nonfinite geometry, nonpositive dimensions, and rectangles outside the page. -Scalar coordinates are evidence locations rather than grapheme, word, or -sentence boundaries; language-tailored segmentation remains a later module. - -The boundary now exposes a strict JSON wire version `1` without exposing private -Rust fields. Artifacts, documents, spans, and nested page locations are serialized -through explicit DTOs with unknown-field rejection. Reconstruction parses and -revalidates RFC 9562 identifiers, canonical digests, content limits, exact text -coordinates, document ownership, and page geometry. Artifact bytes and document -text are rehashed during reconstruction, and digest substitution fails closed. -Malformed JSON, unsupported versions, invalid byte values, and unknown nested -fields produce stable content-redacting errors. - -Persistence, JSON Schema publication, JSON-LD, GraphML, source acquisition -metadata, signatures, and W3C PROV remain outward adapters or later contracts. -They must depend inward on these validated domain values rather than defining -them. +Persistence, JSON Schema publication, JSON-LD, GraphML, source acquisition metadata, signatures, and W3C PROV remain outward adapters or later contracts. They depend inward on validated domain values rather than defining them. ## Quality architecture -The workspace centralizes package metadata and Rust/Clippy lints. Every member -inherits `unsafe_code = "forbid"`, `missing_docs = "deny"`, and warning denial. -Repository contract scripts independently verify the approved crate set, -workspace inheritance, action SHA pinning, absence of LLM credentials from -ordinary CI, and complete Rust documentation. +The workspace centralizes package metadata and Rust/Clippy lints. Every member inherits `unsafe_code = "forbid"`, `missing_docs = "deny"`, and warning denial. Repository contract scripts independently verify the approved crate set, workspace inheritance, action SHA pinning, absence of execution credentials from ordinary CI, and complete Rust documentation. -Stable Rust 1.98.0 is the compile, lint, test, and line-coverage reference. -Branch coverage runs in a pinned nightly lane because LLVM branch coverage -remains unstable in Rust. `cargo-nextest` runs tests without retries, while -doctests remain a separate `cargo test --doc` gate. `cargo-deny` enforces -advisory, license, ban, and source policy. Failed Rust coverage gates print the -exact missing source locations from the same instrumented run without weakening -the 100% contract. +Stable Rust is the compile/lint/test reference selected by repository toolchain policy. CPU `f64` is the scientific numerical reference. Branch/statement coverage, doctests, dependency policy, property/fuzz/security/concurrency tests, and exact missing-source evidence are enforced without retries or denominator manipulation. Accelerator claims require actual hardware execution and parity evidence rather than skipped tests. ## Temporal invariants -TEPP stores event/valid time, assertion time, document time, system time, available time, and knowledge cutoff independently. A historical analysis may include a document only when: +TEPP stores event/valid time, assertion time, document time, system time, available time, and knowledge cutoff independently. A historical analysis may include evidence only when: \[ \operatorname{available\_time}(d) \leq \operatorname{knowledge\_cutoff}. \] -When availability is an interval, every possible instant in that interval must satisfy the inequality. Unknown or open-ended availability that can extend past the cutoff fails closed; event time and document time cannot substitute for availability. +When availability is an interval, every admissible instant must satisfy the cutoff. Unknown or open-ended availability that can extend past the cutoff fails closed; event time and document time cannot substitute for availability. -Forward transition edges require a temporally valid partial order. Retrospective, revision, translation, citation, support, and contradiction relations retain their direction and provenance but do not create reverse state transitions. +Forward transition/state edges require a temporally valid partial order. Retrospective, revision, translation, citation, support, contradiction, and provenance relations retain their direction and evidence but do not become reverse state transitions. + +Longitudinal recovery preserves event-time spacing, available-time admission, irregular gaps, delayed/retrospective reports, missing occasions, changing memberships, language/source drift, and required alignment across translations/rotations/reflections or cluster labels. Scientific acceptance uses true-parameter state/trajectory recovery, RMSE/bias/coverage/convergence and Monte Carlo uncertainty; LLM output cannot activate a scientific candidate. ## Measurement invariants @@ -191,23 +158,25 @@ All languages share global topic identities and latent document coordinates. Lan Repeated report vocabulary is modeled through corpus-background, template, section, style, copied-text, prompt, modality, and substantive-topic sources. It is not silently removed by stopword lists, TF-IDF, or BM25. -Topic proportions are compositional (Aitchison, 1982). ESEM and network analysis consume logistic-normal latent coordinates or orthonormal log-ratio coordinates, with posterior uncertainty propagated through plausible values or a joint model (Asparouhov & Muthén, 2009; Asparouhov et al., 2018; Marsh et al., 2014). The product topic-estimator contract is TRSL-TM (ADR 0012); an STM-style logistic-normal family is the reference, not a shipped-backend claim (Blei & Lafferty, 2006; Roberts et al., 2014, 2019). TDT/CHRONOS event intelligence remains an accepted-target boundary (Allan, 2002; Anagnostopoulos et al., 2013). +Topic proportions are compositional (Aitchison, 1982). ESEM and network analysis consume logistic-normal latent coordinates or orthonormal log-ratio coordinates, with posterior uncertainty propagated through plausible values or a joint model (Asparouhov & Muthén, 2009; Asparouhov et al., 2018; Marsh et al., 2014). The product topic-estimator contract is TRSL-TM (ADR 0012); an STM-style logistic-normal family is a reference formulation, not a shipped-backend claim (Blei & Lafferty, 2006; Roberts et al., 2014, 2019). TDT/CHRONOS event intelligence remains evidence-gated (Allan, 2002; Anagnostopoulos et al., 2013). + +Rasch identity is distinct from generic 1PL. Cross-classification and multiple membership are distinct. Known hierarchy/testlet/rater/method/item-family effects precede residual latent-space dependence. Temporal candidate activation requires exact formulation identity, identification/alignment, Rust estimator, primary citations, required data support, and passing recovery; automatic enumeration is never automatic activation. ## Compute architecture -The CPU `f64` implementation is the numerical reference. Rayon-style fixed worker pools and thread-local sufficient statistics minimize context switching and oversubscription. GPU work is streamed; temporary responsibilities are never retained for the full corpus. The VRAM controller estimates peak allocation, reserves a safety margin, autotunes micro-batches, records telemetry, reduces batches after OOM, and falls back to CPU safely. +The CPU `f64` implementation is the numerical reference. Fixed worker pools and thread-local sufficient statistics minimize context switching and oversubscription. Accelerator work is streamed and parity-verified; temporary responsibilities are not retained for the full corpus. OOM is an expected state with bounded retry/fallback behavior, not a hidden sample-size reduction. ## Persistence -PostgreSQL is the reference relational store. Database objects use two-or-more-word `snake_case` names, including `document_record`, `temporal_interval`, `event_instance`, `event_mention`, `document_relation`, `segment_relation`, `entity_role_assignment`, `model_run`, `topic_definition`, `topic_correlation`, `topic_cluster`, `factor_solution`, `validation_metric`, and `audit_event`. `audit_event` inserts call `operational_log::try_record` before SQL is rendered so source text and source identity cannot enter the row. +PostgreSQL is the reference relational store. Database objects use two-or-more-word `snake_case` names. Temporal and membership persistence is normalized, interval/bitemporal constrained, idempotent where commands can be replayed, and immutable for evidence/provenance records. `audit_event` inserts call the operational logging gate before SQL is rendered so source text and source identity cannot enter the row. Cross-service SQL is prohibited. ## Security and trust boundaries -Documents and LLM outputs are untrusted. Exact spans, JSON Schema, size/depth limits, Unicode validity, prompt-injection isolation, provider allowlists, no-tool execution, tenant isolation, immutable audit events, dependency pinning, SBOM, provenance, and reproducible releases are mandatory. LLM live tests use `NVIDIA_NIM_API_KEY`; `COPILOT_GITHUB_TOKEN` is forbidden. +Documents and LLM outputs are untrusted. Exact spans, schema validation, size/depth limits, Unicode validity, prompt-injection isolation, tenant/purpose authorization, immutable audit evidence, dependency pinning, SBOM, provenance, and reproducible releases are mandatory. LLM live tests use `NVIDIA_NIM_API_KEY`; `COPILOT_GITHUB_TOKEN` is forbidden. The contextual-orchestrator migration is owned by its separate consumer-integration vehicle and is not silently folded into this Longitudinal Modeling branch. ## References -The full APA 7th register is [`docs/research/standards-and-literature.md`](docs/research/standards-and-literature.md). Method claims on this page use: +The full APA 7th register is [`docs/research/standards-and-literature.md`](docs/research/standards-and-literature.md). Detailed Longitudinal Modeling equations and exact implementation/recovery evidence are traced in TRACEABILITY/doctoring and source tests rather than duplicated in responsibility rows. Aitchison, J. (1982). The statistical analysis of compositional data. *Journal of the Royal Statistical Society: Series B, 44*(2), 139–177. https://doi.org/10.1111/j.2517-6161.1982.tb01195.x diff --git a/CHANGELOG.d/longitudinal-component-rmse-identity.md b/CHANGELOG.d/longitudinal-component-rmse-identity.md new file mode 100644 index 000000000..5ad924ceb --- /dev/null +++ b/CHANGELOG.d/longitudinal-component-rmse-identity.md @@ -0,0 +1,5 @@ +### Longitudinal Modeling + +- Known-truth component RMSE rejects duplicate admitted identities instead of letting repeated rows silently reweight the scientific recovery denominator. +- Stable `Between` recovery is unit-level and uses the decomposition contract's canonical `occasion_index = 0`; a nonzero occasion alias is rejected rather than counted as another target or implicit weight. `Within` recovery remains unit-and-occasion specific. +- Truth and recovered component rows are matched by admitted scientific identity and accumulated in canonical identity order, so serialization-order permutations preserve the same deterministic CPU `f64` recovery result instead of changing admission or the last-bit RMSE rounding path. diff --git a/CHANGELOG.d/longitudinal-occasion-mean-event-time.md b/CHANGELOG.d/longitudinal-occasion-mean-event-time.md new file mode 100644 index 000000000..3a1227fbb --- /dev/null +++ b/CHANGELOG.d/longitudinal-occasion-mean-event-time.md @@ -0,0 +1,7 @@ +# Longitudinal occasion-mean event-time composition + +- `longitudinal_core` adds Hamaker, Kuiper, and Grasman (2015, Eq. 1a) occasion-mean event-time composition. For each admitted occasion it forms `p_it = x_it - mu_t` from the occasion-specific group mean, not a person mean or sample-wide grand mean; those deviations retain between-person differences and therefore are not evidence of a within-person or RI-CLPM lag. +- Occasion identity is numeric, so `-0.0` and `+0.0` are one occasion. Each admitted occasion must contain at least two distinct units, duplicate unit-occasion membership fails closed, and at least two units must contribute a consecutive lag. +- Exact zero Hamaker Eq. 1a residuals use canonical public `+0.0`. IEEE-754 signed zero after subtracting a canonical occasion mean is not a directional deviation; exposing both encodings would give one scientific estimand two public identities. Already-centered caller-constructed pairs stay as given. These residuals still retain between-person differences and are not within-person or RI-CLPM lags. +- Occasion means reuse the Longitudinal-local scaled compensated mean used by CWC/decomposition; event intervals use the typed finite-positive `EventTimeInterval`, and log-rate recovery reuses the existing exact scalar irregular-residual boundary. Non-finite, sparse, duplicate, nonpositive-interval, or nonrepresentable inputs fail closed rather than being reinterpreted as within-person change. +- This fragment preserves the valid operational release note from PR #486 while its wrong-owner `psychometric_core` implementation, raw signed-zero bit identity, naive `sum/count` mean, and mixed-timezone provenance wording remain intentionally unadopted. Canonical root `CHANGELOG.md` assembly remains a landing prerequisite rather than being inferred from this fragment. diff --git a/CHANGELOG.md b/CHANGELOG.md index 8b495291a..34cbc4c66 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -40,6 +40,14 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang - Removed the repository-local hourly PR-maintenance caller now covered by the central required scheduler, retired stale workflow registrations, narrowed documentation triggers, keyed PR concurrency by fixed workflow name, repository, and pull-request number without cancelling non-PR runs, and combined line/branch coverage on one sequential runner while preserving both 100% gates and diagnostics. +- `longitudinal_core` canonicalizes exact-zero Hamaker Eq. 1a occasion-mean residuals to public `+0.0`. IEEE-754 signed zero after subtracting a canonical `+0.0` occasion mean is numerically equal to `+0.0` but bit-distinct; a zero occasion-mean deviation has no directional meaning, so exposing both encodings gave one scientific estimand two public identities. `center_occasion_mean_event_lags` now matches the CWC and `decompose_within_between` public-zero contracts. Already-centered caller-constructed pairs are unchanged. These residuals still retain between-person differences and are not within-person, RI-CLPM, or DSEM lags (Hamaker, Kuiper, & Grasman, 2015, Eq. 1a). Meredith (1993) remains unread. Mislevy (1991) remains unread (ERIC ED268155 is the 1985/86 report, not the 1991 *Psychometrika* article). + +- `longitudinal_core` canonicalizes exact-zero CWC lag residuals to public `+0.0`. IEEE-754 signed zero after person-mean subtraction is numerically equal to `+0.0` but bit-distinct; a zero within-person deviation has no directional meaning, so exposing both encodings gave one scientific estimand two public identities. `center_within_unit_event_lags` now matches the `decompose_within_between` public-zero contract. Already-centered caller-constructed pairs are unchanged. The crate also exposes the already-implemented unstandardised stationary within-person variance `p = -q / (2 a)` (Driver et al., 2017, Eq. 4 as `Δt → ∞`; p. 16 `asymDIFFUSION`) so standardised maps can keep cancelling `p` without leaving a dead production path. This is not DSEM, not raw-process drift, and not `asymDIFFUSIONstd`. Meredith (1993) remains unread. Mislevy (1991) remains unread (ERIC ED268155 is the 1985/86 report, not the 1991 *Psychometrika* article). + +- `longitudinal_core` folds Draft #327 unique CWC-then-irregular residual log-rate onto the Longitudinal Modeling landing vehicle. Consecutive unit-mean-centered pairs keep typed `EventTimeInterval`. The pairwise mean of the Driver, Oud, and Voelkle (2017, Eq. 3; JSS PDF opened 2026-09-02) scalar inverse `a = ln(|later| / |earlier|) / Δt` is taken on nonzero same-sign residuals, with a stable logarithm when that ratio overflows or underflows, and an incremental mean so two finite rates whose raw sum overflows stay representable. Already-centered pairs recover `ln(0.5)`; CWC of a raw AR path does not. Curran and Bauer (2011, pp. 583–619; PMC3059070 XML opened 2026-09-02; Eq. 36) show that person-mean centering of a time-related TVC is biased for the within-person effect; `refuse_cwc_residual_log_rate_as_raw_process_drift` always fails closed. T=2 CWC is `r, −r` (empty admissible). This is not Newton LS, not DSEM, and not a second invariance crate. Voelkle et al. (2012, Eq. 7) remains previously opened lineage; the ZORA PDF was not re-opened this cycle. Meredith (1993) remains unread. Mislevy (1991) remains unread. + +- `longitudinal_core` adds event-time lagged-correlation standardization that requires lagged covariance plus both occasion-specific marginal variances, enforces the exact binary64 covariance bound, and uses bounded division order to avoid representable overflow/underflow. The invalid one-sided covariance/earlier-variance `expected_autocorrelation` API is removed; this arithmetic boundary does not claim a DSEM/ctsem/state-transition estimator. + - `event_core` adds bounded Allen interval-consistency classification, atomic path-consistency closure, contradiction/resource refusals, and an explicit dependency-error fallback without claiming unrestricted global satisfiability. - `psychometric_core` recovers the Driver, Oud, and Voelkle (2017, Table 2, p. 12 `MANIFESTTRAITVAR`; §7.1, p. 19; p. 16 `MANIFESTTRAITVARstd`; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-27T14:20Z from https://www.jstatsoft.org/index.php/jss/article/download/v077i05/1104) scalar standardised manifest-trait variance on current main after `0ce16e8` dropped the pre-consolidation code while research notes already named the map (register items 83–84). Table 2 names `MANIFESTTRAITVAR` `Ψ_τ` the additional time-invariant variance-covariance on the measurement level and sets it `NULL` when there is no manifest trait. Equation 5 writes `Γ ~ N(τ, Ψ)` and names that covariance the manifest traits. Section 7.1 names manifest traits stable individual differences in indicator levels, distinct from process-level `TRAITVAR` `φ_ξ`. Page 16 prints standardised matrices with the suffix `std` when appropriate. The printed example on p. 16 is `discreteDRIFTstd`, not `MANIFESTTRAITVARstd`. Footnote 4 standardises using only the relevant variance, not the total. The relevant variance for that named indicator-level correlation is `MANIFESTTRAITVAR`, not process-level `TRAITVAR` and not residual `MANIFESTVAR` `θ`. The 2017-era source forms `MANIFESTTRAITVARstd` only when `MANIFESTTRAITVAR != 0`, as `solve(sqrt(diag(MANIFESTTRAITVAR) + ridging)) %&% MANIFESTTRAITVAR` when `verbose = TRUE`. OpenMx `%&%` is `t(A) %*% B %*% A`. Unlike `TRAITVARstd`, that formation adds `diag(c(ridging), n.manifest)`. The default `ridging = FALSE` adds 0, not `0.0001`; that ridge is a numerical hack and is not this exact map. The scalar correlation is `ψ / ψ = 1` after strictly positive `MANIFESTTRAITVAR`. Form strictly positive `ψ` first, then `1 / √ψ`, then `(1 / √ψ) ψ (1 / √ψ)`. Unstandardised `MANIFESTTRAITVAR` is defined for a zero trait; standardised `MANIFESTTRAITVAR` is not. Zero `MANIFESTTRAITVAR` skips forming `MANIFESTTRAITVARstd` in the 2017-era source and fails closed here. Indicator-level trait variance is an event-time structural quantity, so a non-event clock fails closed. `MANIFESTTRAITVAR` does not require stable `a < 0`. Distinct positive `ψ` recover the same 1. `trait / trait = 1` is `TRAITVARstd` and recovers the same number and remains a distinct named quantity. `θ` is `MANIFESTVAR` and is measurement error, not this correlation. Meredith (1993) remains unread (web search 2026-08-27T14:20Z: Springer/Cambridge Core paywalled; Unpaywall historically `is_oa: false`; Springer `content/pdf` is an HTML stub). Mislevy (1991, *Psychometrika, 56*, 177–196) remains unread on the same terms (DOI `10.1007/bf02294457`). Still not a Kalman filter, not a matrix `expm`, not ESEM estimation, not DSEM, and not ctsem estimation. @@ -448,4 +456,4 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang - Added known digest vectors, mutation detection, hostile multibyte Unicode, exact-coordinate, page-boundary, stable-error, and invalid-input regression tests for the first evidence slice. - Added strict wire round trips, unknown-field and version rejection, digest reconstruction, configured-limit, hostile JSON, and generated multilingual span tests. -The repository has not yet cut a stable implementation release, so no compare reference is published for `[Unreleased]` yet. +The repository has not yet cut a stable implementation release, so no compare reference is published for `[Unreleased]` yet. \ No newline at end of file diff --git a/CLAUDE.md b/CLAUDE.md index 339692cf0..a35b5aeef 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -15,8 +15,9 @@ Read and follow `AGENTS.md` before changing this repository. The repository-wide - Do not remove repeated report language with global stopword lists or use TF-IDF/BM25 as inferential weights. Model template, section, copied-text, style, modality, and corpus-background sources explicitly. - Do not treat raw topic proportions as ordinary Euclidean indicators. Use logistic-normal coordinates or valid log-ratio coordinates and propagate posterior uncertainty into ESEM/DSEM. - Do not treat metric/weak invariance as a latent-mean license. Strong (equal loading and intercept) or strict is required; `#84` `metric` licenses shared metric meaning only. Putnick and Bornstein (2016, PMC5145197 opened 2026-08-19T22:15Z) require scalar invariance before latent-mean comparison; residual invariance is not a prerequisite. Two-observation series have no residual degrees of freedom (`ordinary_least_squares_fit` returns residual variance `0`) and cap at strong/scalar; they still license means. This is two-group OLS, not MGCFA. Meredith (1993) names remain unread labels (Unpaywall/OpenAlex 2026-08-25T11:32Z: closed). -- Do not use the difference quotient as a continuous-time rate. The scalar map is `a = ln(φ) / Δt` on event time. Discrete lags from unequal event intervals are not one coefficient; remap them through that log-rate. Binary64 `exp(a Δt) = 0` is not a discrete lag. A constant predictor's discrete effect is Voelkle et al. (2012, Eq. 12), evaluated as `a_yx (expm1(z) / a_xx)` with `z = a_xx Δt` so a finite result is not lost when `z` overflows to `-∞` or when `a_yx Δt` overflows. When `expm1(z)` overflows at a finite `z`, rewrite in log space; a zero continuous effect is exactly zero; an overflowing `a_yx/a_xx` rewrite term fails closed. The first-order product is the underflow limit of that equation, not the general constant-predictor discrete effect. A time-varying predictor whose sampling interval equals its constancy interval uses Voelkle et al. (2012, Eq. 14): `b* = a_yx Δt`. Unmatched intervals fail closed (Oud & Jansen, 2000, unread). Discrete process noise is Driver et al. (2017, Eq. 3): `Q_Δt = 0.5 q (expm1(z) / a)` with `z = 2 (a Δt)` and `q = G G⊤ ≥ 0`; do not form `2 a` first; `a = 0` and `z → 0` recover `q Δt`; a zero diffusion is exactly zero; an overflowing rewrite scale `0.5 q / a` fails closed; this is not a Kalman filter. `Q_Δt` is `cov(η_t | η_{t-1})`, not `Var(η_t)`. The lagged covariance is `exp(a Δt) p` and the unconditional variance is `exp(2 a Δt) p + Q_Δt` (Driver et al., 2017, Eq. 3–4, pp. 4–5; JSS has no numbered §2.2). A zero diffusion whose `2 (a Δt)` overflows to `+∞` is not a finite `Var(η_t)`. The stationary within-subject variance is the `Δt → ∞` limit of Eq. 4: `-q / (2 a)` for stable `a < 0` (JSS p. 16 `asymDIFFUSION`; §4.3). When `2 a` is finite, form `q / -(2 a)` so `q / a` overflow does not lose a finite result (`q = MAX`, `a = -0.75` → `MAX / 1.5`). When `2 a` overflows, form `(q / a) * -0.5`. Do not form `0.5 q` first (`q = from_bits(1)` underflows). `a ≥ 0` has no finite stationary variance. Finite-interval `Q_Δt` is not that limit. Trait-plus-state variance is `trait + state` and lagged covariance is `trait + exp(a Δt) p` (Driver et al., 2017, §4.3, p. 9). Trait variance is not process noise and not `asymDIFFUSION`. Evolving the summed variance as if it were all state is not that map. This is not RI-CLPM. Observed-indicator variance is `λ² Var(η) + θ` when `MANIFESTTRAITVAR` is zero and `λ² Var(η) + θ + ψ` otherwise (Driver et al., 2017, Eq. 5, p. 5; Table 2, p. 12). Lagged observed covariance is `λ² cov(η_t, η_{t-1}) + ψ`; `MANIFESTVAR` does not enter. Observed-indicator mean is `τ + λ μ` (Driver et al., 2017, Eq. 5; Table 2, p. 12). `MANIFESTMEANS` is `τ`, not `E(y)`. `E(η)` is not `E(y)`. `CINT` is not `MANIFESTMEANS`. `T0MEANS` is not `E(y)`. The discrete latent mean is `μ_t = exp(a Δt) μ_0 + (exp(a Δt) − 1)/a κ` (Driver et al., 2017, Eq. 3, p. 4; Table 2, p. 12). `T0MEANS` is not `μ_t`. `CINT` is not that discrete increment. A zero drift is `κ Δt`. Underflow of `exp(a Δt)` to `+0` drops the carried `T0MEANS` and keeps `−κ / a`. The evolved observed mean is `τ + λ μ_t` (Driver et al., 2017, Eq. 5 of that Eq. 3 map). The first-occasion map `τ + λ μ_0` is not `E(y_t)`. `μ_t` is not `E(y_t)`. The contemporaneous time-dependent predictor impulse is `m x` (Driver et al., 2017, Eq. 3 fourth summand; Table 2 `TDPREDEFFECT` is `M`). Form `μ_t` first, then add `m x`. `TDPREDEFFECT` is not `CINT`. `M x` is not `A^{-1}[e^{A Δt} − I] B z` and is not Voelkle et al. (2012, Eq. 14). The §7.2 level-change form is not that impulse. The observed mean of that contemporaneous impulse is `τ + λ(μ_t + m x)` (Driver et al., 2017, Eq. 5 of the Eq. 3 fourth-summand composition). The evolved map `τ + λ μ_t` is not that observed mean. The carry map `τ + λ(μ_t + e^{a(t−u)} m x)` is not that observed mean when `u ≠ t`. The evolved-plus-impulse latent mean is not `E(y_t)`. The time-independent predictor increment is `A^{-1}[e^{A Δt} − I] B z` (Driver et al., 2017, Eq. 3 second summand; Table 2 `TIPREDEFFECT` is `B`). Form `B z` first, then the discrete intercept map. A zero drift is `B z Δt`. `TIPREDEFFECT` is `B`, not that discrete increment. `A^{-1}[e^{A Δt} − I] B z` is not `CINT`, not `M x`, and not Voelkle et al. (2012, Eq. 14). The observed mean of that increment is `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` (Driver et al., 2017, Eq. 5 of the Eq. 3 printed addend after the `T0MEANS` carry and the `CINT` increment). The evolved map `τ + λ μ_t` is not that observed mean. The contemporaneous map `τ + λ(μ_t + m x)` is not that observed mean. The carry map `τ + λ(μ_t + e^{a(t−u)} m x)` is not that observed mean when `u ≠ t`. The evolved-plus-increment latent mean is not `E(y_t)`. The within-interval time-dependent impulse carry is `e^{A(t−u)} M x` for `t0 < u < t` (Driver et al., 2017, Eq. 1–2 Green-function integral of Eq. 2; §7.2 dissipation). Form `m x` first, then `e^{a(t−u)} m x`. A zero drift is `m x` with no dissipation. Underflow of `e^{a(t−u)}` to `+0` is vanishing dissipation and is kept. `e^{A(t−u)} M x` is not the contemporaneous Dirac, not `CINT`, not `TIPREDEFFECT`, and not Voelkle et al. (2012, Eq. 14). An impulse at `u = t` is the contemporaneous map. An impulse at `u ≤ t0` is already in `η(t0)`. The observed mean of that carry is `τ + λ(μ_t + e^{a(t−u)} m x)` (Driver et al., 2017, Eq. 5 of the Eq. 1–2 carried latent mean). The evolved map `τ + λ μ_t` is not that observed mean. The contemporaneous map `τ + λ(μ_t + m x)` is not that observed mean when `u ≠ t`. `MANIFESTMEANS` is not `E(y_t)`. The carried latent mean is not `E(y_t)`. The first-occasion time-independent predictor shift is `t0_b z` (Driver et al., 2017, Table 3 `T0TIPREDEFFECT`; Eq. 3 first summand). Form `t0_b z` first, then `e^{a Δt} t0_b z`. Form `μ_t` first, then add that carry. A zero drift is `t0_b z`. Underflow of `e^{a Δt}` to `+0` is a vanishing carry of the first-occasion shift and is kept. `t0_b z` is not `A^{-1}[e^{A Δt} − I] B z`, not `CINT`, and not `M x`. `e^{A Δt} t0_b z` is not `t0_b z`. `T0TIPREDEFFECT` is the coefficient, not the shift. The observed mean of that first-occasion carry is `τ + λ(μ_t + e^{a Δt} t0_b z)` (Driver et al., 2017, Eq. 5 of the Table 3 / Eq. 3 first-summand composition). The evolved map `τ + λ μ_t` is not that observed mean. The process-increment map `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` is not that observed mean. The contemporaneous map `τ + λ(μ_t + m x)` is not that observed mean. The impulse-carry map `τ + λ(μ_t + e^{a(t−u)} m x)` is not that observed mean when `u ≠ t0`. The evolved-plus-carry latent mean is not `E(y_t)`. The first-occasion time-dependent predictor shift is `t0_m x0` (Driver et al., 2017, Table 3 `T0TDPREDEFFECT`; Eq. 3 first summand; JSS PDF re-opened 2026-08-20T19:10Z). Form `t0_m x0` first, then `e^{a Δt} t0_m x0`. Form `μ_t` first, then add that carry. A zero drift is `t0_m x0`. Underflow of `e^{a Δt}` to `+0` is a vanishing carry of the first-occasion shift and is kept. `t0_m x0` is not `M x`, not `e^{A(t−u)} M x` for `t0 < u < t`, not `t0_b z`, not `A^{-1}[e^{A Δt} − I] B z`, and not `CINT`. `e^{A Δt} t0_m x0` is not `t0_m x0`. `T0TDPREDEFFECT` is the coefficient, not the shift. An impulse at `u ≤ t0` that used `M` is already in `η(t0)` as `TDPREDEFFECT`, not as `T0TDPREDEFFECT`. The observed mean of that first-occasion TD carry is `τ + λ(μ_t + e^{a Δt} t0_m x0)` (Driver et al., 2017, Eq. 5 of the Table 3 / Eq. 3 first-summand TD composition; JSS PDF re-opened 2026-08-20T19:07Z). The evolved map `τ + λ μ_t` is not that observed mean. The process-increment map `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` is not that observed mean. The contemporaneous map `τ + λ(μ_t + m x)` is not that observed mean. The impulse-carry map `τ + λ(μ_t + e^{a(t−u)} m x)` is not that observed mean when `u ≠ t0`. The first-occasion TI map `τ + λ(μ_t + e^{a Δt} t0_b z)` is not that observed mean. The evolved-plus-carry latent mean is not `E(y_t)`. The lasting level-change `CINT` is `κ = −a m x` (Driver et al., 2017, §7.2, pp. 20–21; JSS PDF re-opened 2026-08-20T19:45Z). Form `m x` first, then multiply by `−a`. Stable `a < 0` is required so `−κ / a = m x` is an equilibrium offset. `a ≥ 0` cannot hold a new process mean. `−a m x` is not the dissipating Dirac `m x`, not a free `CINT`, and not `A^{-1}[e^{A Δt} − I] B z`. The extra near-zero-drift latent process also named in §7.2 is a different specification and is not this `CINT` setting. Equation 3 maps that intercept as `(1 − e^{a Δt}) m x` (JSS PDF re-opened 2026-08-20T19:50Z). Form the level-change `CINT` first, then the discrete intercept map. Underflow of `e^{a Δt}` to `+0` keeps `m x`. `(1 − e^{a Δt}) m x` is not `m x`, not `κ`, and not `A^{-1}[e^{A Δt} − I] B z`. The printed §7.2 lasting level change is an extra near-zero-drift latent process (Driver et al., 2017, §7.2, pp. 22–23; JSS PDF re-opened 2026-08-20T23:10Z). `T0MEANS`, `CINT`, `T0VAR`, `DIFFUSION`, and `TRAITVAR` of that process are fixed to 0; `TDPREDEFFECT` on it is fixed to 1; its `DRIFT` diagonal is very close to 0 (printed example `−0.000001`; precisely 0 causes computational problems); the original process is driven by the `DRIFT` coupling `a_{ηξ}`. After a unit identification impulse the scalar contribution is `a_{ηξ} x (e^{ε Δt} − e^{a Δt}) / (ε − a)` (`ε = a` is `a_{ηξ} x Δt e^{a Δt}`). Form `a_{ηξ} x` first. A zero coupling or zero predictor is exactly zero. `ε ≥ 0` fails closed. That contribution is not `κ = −a m x`, not `(1 − e^{a Δt}) m x`, and not the dissipating Dirac `m x`. The observed mean of that extra-process contribution is `τ + λ(μ_t + a_{ηξ} x (e^{ε Δt} − e^{a Δt}) / (ε − a))` (Driver et al., 2017, Eq. 5 of that §7.2 contribution; JSS PDF re-opened 2026-08-21T06:12Z). The extra process has `LAMBDA` 0 and is not an observed indicator. Original indicators load on the original process after the `DRIFT` coupling. The evolved map `τ + λ μ_t` is not that observed mean. The contemporaneous map `τ + λ(μ_t + m x)` is not that observed mean. The contribution is not `E(y_t)`. The evolved-plus-contribution latent mean is not `E(y_t)`. `T0TDPREDEFFECT` on the extra process begins at `t = 0` and uses `Δt = t − t0` for both the original-process evolution and the extra drive. `TDPREDEFFECT` after `t0` uses `t − u` with `t0 < u < t` for the extra drive while `μ_t` still uses `Δt`. The observed mean of that after-t0 extra-process contribution is `τ + λ(μ_t + a_{ηξ} x (e^{ε(t−u)} − e^{a(t−u)}) / (ε − a))` (Driver et al., 2017, Eq. 5 of that §7.2 after-t0 contribution; JSS PDF re-opened 2026-08-21T06:32Z). The first-occasion extra-process observed mean is not that observed mean when `u ≠ t0`. The impulse-carry map `τ + λ(μ_t + e^{a(t−u)} m x)` is a Dirac on the original process and is not that `DRIFT` drive. An impulse at `u = t0` or `u = t` is not interior. The asymptotic time-independent predictor effect is `-B z / a` (Driver et al., 2017, §7.2, pp. 20–21; JSS PDF opened 2026-08-21T13:08Z). Form `B z` first, then divide by `-a`. Stable `a < 0` is required. `a ≥ 0` cannot hold a finite process-mean change. `-B z / a` is not the coefficient `B`, not `A^{-1}[e^{A Δt} − I] B z`, not `CINT`, and not `M x`. The asymptotic time-independent predictor variance is `(B / a)² v` (Driver et al., 2017, §7.2, pp. 20–21 `addedTIPREDVAR`). Form the unit asymptotic effect first, then square, then multiply by `v`. `(B / a)² v` is not `TRAITVAR`, not `asymDIFFUSION`, and not `-B z / a`. The asymptotic continuous intercept is `-κ / a` (Driver et al., 2017, Table 2, p. 12 `asymCINT`; Eq. 3 as `Δt → ∞`; JSS PDF opened 2026-08-21T16:13Z). Form `κ` first, then divide by `-a`. Stable `a < 0` is required. `-κ / a` is not `κ`, not `A^{-1}[e^{A Δt} − I] κ`, not `T0MEANS`, and not `-B z / a`. The p. 16 stationary `T0MEANS` constraint is `-κ / a + −B z / a`. Form the intercept contribution first, then include the TI extra effect, then add. That constrained first-occasion mean is not free `T0MEANS`, not `asymCINT` alone, not `asymTIPREDEFFECT` alone, and not the finite-interval discrete latent mean. Equation 5 of that constrained mean is `τ + λ(−κ / a + −B z / a)` (Driver et al., 2017, §4.3, pp. 9–10; Eq. 5, p. 5; JSS PDF re-opened 2026-08-21T20:07Z). Form the stationary latent mean first, then `τ + λ` of that mean. `τ + λ μ_0` for free `T0MEANS` is not that composition. `τ + λ(−κ / a)` is not that composition when `B z ≠ 0`. `τ + λ μ_t` is not that composition. `MANIFESTMEANS` is not `E(y_0)`. The constrained latent mean is not `E(y_0)`. The p. 16 constrained first-occasion variance `trait + −q / (2 a) + (B / a)² v` is not free `T0VAR`, not `asymDIFFUSION` alone, not `TRAITVAR` alone, not `addedTIPREDVAR` alone, and not the finite-interval discrete latent variance. Eq. 5 of that constrained variance is `λ²(trait + −q / (2 a) + (B / a)² v) + θ + ψ` (JSS PDF re-opened 2026-08-22T03:20Z; form the stationary latent variance first, then `λ² p + θ + ψ`; `λ² p_0` is not that observed variance; `λ²(−q / (2 a)) + θ` is not that observed variance when `TRAITVAR` or `addedTIPREDVAR` is nonzero; `MANIFESTVAR` is not `Var(y_0)`; the constrained latent variance is not `Var(y_0)`). The lagged covariance of that constrained process is `trait + e^{a Δt}(−q / (2 a)) + (B / a)² v` (Driver et al., 2017, Eq. 3–4 of §4.3 / p. 16 `T0VAR`; JSS PDF re-opened 2026-08-22T19:13Z). Trait and `addedTIPREDVAR` do not decay with `e^{a Δt}`. Contemporaneous `T0VAR` is not that lagged map. Decaying the constrained total as if it were all state is not that lagged map. Equation 5 of that lagged covariance is `λ²(trait + e^{a Δt}(−q / (2 a)) + (B / a)² v) + ψ`. `Θ` does not enter. Contemporaneous `Var(y_0)` is not that lagged observed covariance. The lagged latent covariance is not that observed covariance. The later-occasion variance of that constrained process is `trait + e^{2 a Δt}(−q / (2 a)) + Q_Δt + (B / a)² v` (Driver et al., 2017, Eq. 3–4 of §4.3 / p. 16 `T0VAR`; JSS PDF re-opened 2026-08-22T23:12Z). Trait and `addedTIPREDVAR` do not enter `Q_Δt`. Under stationarity that composition equals contemporaneous `T0VAR`. Evolving the constrained total as if it were all state is not that later map. The lagged covariance omits `Q_Δt` and is not that later map. `Q_Δt` is not that later map. Equation 5 of that later-occasion variance is `λ²(trait + e^{2 a Δt}(−q / (2 a)) + Q_Δt + (B / a)² v) + θ + ψ`. The lagged observed covariance omits `Q_Δt` and `θ`. `MANIFESTVAR` is not `Var(y_t)`. The later-occasion latent variance is not `Var(y_t)`. The later-occasion variance of §4.3 predetermined `T0VAR` is `trait + e^{2 a Δt} p_0 + Q_Δt + (B / a)² v` (Driver et al., 2017, Eq. 3–4 of §4.3 predetermined first occasion; JSS PDF re-opened 2026-08-23T05:12Z). Trait and `addedTIPREDVAR` do not enter `Q_Δt`. Free `T0VAR` `p_0` is not that later map. Setting `p_0 = −q / (2 a)` recovers the stationary later-occasion map. Stationary later variance uses `−q / (2 a)` in place of `p_0` and is not that later map when `p_0` is free. Evolving `trait + p_0 + (B / a)² v` as if it were all state is not that later map. As `Δt → ∞` with stable `a < 0` the composition approaches contemporaneous stationary `T0VAR`. As `Δt → 0+` the composition approaches `trait + p_0 + (B / a)² v`. Nonzero diffusion with `a ≥ 0` is a growing process and is kept. Equation 5 of that predetermined later-occasion variance is `λ²(trait + e^{2 a Δt} p_0 + Q_Δt + (B / a)² v) + θ + ψ`. `MANIFESTVAR` is not `Var(y_t)`. The predetermined later-occasion latent variance is not `Var(y_t)`. Stationary later observed variance is not that observed variance when `p_0` is free. The lagged covariance of §4.3 predetermined `T0VAR` is `trait + e^{a Δt} p_0 + (B / a)² v` (Driver et al., 2017, Eq. 3–4 of §4.3 predetermined first occasion; JSS PDF re-opened 2026-08-23T09:04Z). Trait and `addedTIPREDVAR` do not decay with `e^{a Δt}`. Free `T0VAR` `p_0` is not that lagged map. Setting `p_0 = −q / (2 a)` recovers the stationary lagged map. Stationary lagged covariance uses `−q / (2 a)` in place of `p_0` and is not that lagged map when `p_0` is free. Evolving `trait + p_0 + (B / a)² v` as if it were all state is not that lagged map. Later-occasion variance includes `Q_Δt` and is not that lagged map. As `Δt → ∞` with stable `a < 0` the state term vanishes. As `Δt → 0+` the composition approaches `trait + p_0 + (B / a)² v`. Equation 5 of that predetermined lagged covariance is `λ²(trait + e^{a Δt} p_0 + (B / a)² v) + ψ`. `MANIFESTVAR` does not enter. The predetermined lagged latent covariance is not that observed covariance. Predetermined later observed variance includes `Q_Δt` and `θ` and is not that lagged observed covariance. Stationary lagged observed covariance is not that observed covariance when `p_0` is free. The predetermined first-occasion variance of §4.3 predetermined `T0VAR` is `trait + p_0 + (B / a)² v`. Free `p_0` is not that map. Stationary first-occasion variance uses `−q / (2 a)` in place of `p_0` and is not that map when `p_0` is free. Lagged covariance decays the state and is not that map. Later-occasion variance includes `Q_Δt` and is not that map. Equation 5 of that predetermined first-occasion variance is `λ²(trait + p_0 + (B / a)² v) + θ + ψ`. `MANIFESTVAR` is not that first-occasion observed variance. The predetermined first-occasion latent variance is not that observed variance. Stationary first-occasion observed variance is not that observed variance when `p_0` is free. Predetermined later observed variance includes `Q_Δt` and is not that first-occasion observed variance. Later-start lagged covariance of predetermined `T0VAR` is `trait + e^{a s}(e^{2 a u} p_0 + Q_u) + (B / a)² v` (Driver et al., 2017, §4.3 `startoffset`; Eq. 4; JSS PDF re-opened 2026-08-23T10:27Z). First-occasion lagged omits `e^{a s} Q_u`. Later-occasion variance does not lag. Stationary lagged uses `−q / (2 a)`. Decaying the later total is not that map. Equation 5 of that later-start lagged covariance is `λ²` of it plus `ψ`. Independent `ε_t` does not enter. First-occasion lagged observed omits `e^{a s} Q_u`. Predetermined later observed variance includes `Q_u` and `θ` and is not that later-start lagged observed covariance. Later-start later-occasion variance of predetermined `T0VAR` is `trait + e^{2 a s}(e^{2 a u} p_0 + Q_u) + Q_s + (B / a)² v` (Driver et al., 2017, §4.3 `startoffset`; Eq. 3–4 Chapman–Kolmogorov `Q_{u+s} = e^{2 a s} Q_u + Q_s`; JSS PDF re-opened 2026-08-23T11:05Z). Later-occasion variance at `u` omits `Q_s`. Later-start lagged covariance omits `Q_s`. Stationary later uses `−q / (2 a)`. Evolving the later total as if it were all state is not that map. Ignoring `startoffset` omits `e^{2 a s} Q_u`. Equation 5 of that later-start later-occasion variance is `λ²` of it plus `θ + ψ`. `MANIFESTVAR` is not that observed variance. Page 16 `discreteDRIFTstd` is `e^{a Δt}` after strictly positive `asymDIFFUSION` `-q / (2 a)` (Driver et al., 2017, p. 16; footnote 4; §7.1; JSS PDF re-opened 2026-08-23T11:40Z). Unstandardised `e^{a Δt}` is defined for growing `a ≥ 0` and for zero diffusion and is not `discreteDRIFTstd`. The §7.1 trait-plus-state autocorrelation `(trait + e^{a Δt} p + added) / (trait + p + added)` uses `TRAITVAR` and is not `discreteDRIFTstd`. `TRAITVAR` is not the standardisation variance. Page 16 `discreteDIFFUSIONstd` is `Q_Δt / (−q / (2 a))` after strictly positive `asymDIFFUSION` `-q / (2 a)` (Driver et al., 2017, p. 16; Eq. 4; footnote 4; JSS PDF re-opened 2026-08-23T13:06Z). Unstandardised `Q_Δt` is defined for growing `a ≥ 0` and for zero diffusion and is not `discreteDIFFUSIONstd`. The continuous standardisation `−2 a` is not `discreteDIFFUSIONstd`. `Q_Δt / (trait + p + added)` uses `TRAITVAR` and is not `discreteDIFFUSIONstd`. `TRAITVAR` is not the standardisation variance. Page 16 `DIFFUSIONstd` is `q / (−q / (2 a)) = −2 a` after strictly positive `asymDIFFUSION` `-q / (2 a)` (Driver et al., 2017, p. 16; Eq. 4; footnote 4; JSS PDF re-opened 2026-08-23T13:20Z). Unstandardised `q` is defined for growing `a ≥ 0` and for zero diffusion and is not `DIFFUSIONstd`. The discrete standardisation `Q_Δt / (−q / (2 a))` depends on `Δt` and is not `DIFFUSIONstd`. `q / (trait + p + added)` uses `TRAITVAR` and is not `DIFFUSIONstd`. `TRAITVAR` is not the standardisation variance. Page 16 `DRIFTstd` is the continuous auto-effect after strictly positive `asymDIFFUSION` `-q / (2 a)` (Driver et al., 2017, p. 16; Eq. 1; footnote 4; JSS PDF re-opened 2026-08-23T13:28Z). Unstandardised `a` is defined for growing `a ≥ 0` and for zero diffusion and is not `DRIFTstd`. The discrete standardisation `e^{a Δt}` depends on the event interval and is not `DRIFTstd`. `a p / (trait + p + added)` uses `TRAITVAR` and is not `DRIFTstd`. `TRAITVAR` is not the standardisation variance. Page 16 `asymTIPREDEFFECTstd` is `(-B / a) · √v / √(-q / (2 a))` after strictly positive `asymDIFFUSION` `-q / (2 a)` and strictly positive predictor variance `v` (Driver et al., 2017, p. 16; §7.2; footnote 4; JSS PDF re-opened 2026-08-23T14:25Z). Unstandardised `-B / a` is defined for a zero coefficient and for zero predictor variance and is not `asymTIPREDEFFECTstd`. The finite-interval standardisation `A^{-1}[e^{A Δt} − I] B · √v / √p` depends on the event interval and is not `asymTIPREDEFFECTstd`. `(-B / a) · √v / √(trait + p + added)` uses `TRAITVAR` and is not `asymTIPREDEFFECTstd`. `TRAITVAR` is not the standardisation variance. Page 16 `TIPREDEFFECTstd` is `B · √v / √(-q / (2 a))` after strictly positive `asymDIFFUSION` `-q / (2 a)` and strictly positive predictor variance `v` (Driver et al., 2017, p. 16; §7.2; footnote 4; JSS PDF re-opened 2026-08-23T16:21Z). Unstandardised `B` is defined for a zero coefficient and for zero predictor variance and is not `TIPREDEFFECTstd`. The asymptotic standardisation `(-B / a) · √v / √p` is the total change and is not `TIPREDEFFECTstd`. The finite-interval standardisation `A^{-1}[e^{A Δt} − I] B · √v / √p` depends on the event interval and is not `TIPREDEFFECTstd`. `B · √v / √(trait + p + added)` uses `TRAITVAR` and is not `TIPREDEFFECTstd`. `TRAITVAR` is not the standardisation variance. Page 16 / Table 3 `T0TIPREDEFFECTstd` is `t0_b · √v / √p_0` after strictly positive free `T0VAR` `p_0` and strictly positive predictor variance `v` (Driver et al., 2017, Table 3, p. 13; p. 16; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-23T17:20Z). The affected variance is free first-occasion `T0VAR`, not `asymDIFFUSION`. Unstandardised `t0_b` is defined for a zero coefficient and for zero predictor variance and is not `T0TIPREDEFFECTstd`. `TIPREDEFFECTstd` `B · √v / √(-q / (2 a))` is the continuous coefficient and is not `T0TIPREDEFFECTstd`. `asymTIPREDEFFECTstd` `(-B / a) · √v / √p` is the total change and is not `T0TIPREDEFFECTstd`. `t0_b · √v / √(trait + p_0 + added)` uses `TRAITVAR` and is not `T0TIPREDEFFECTstd`. `TRAITVAR` is not the standardisation variance. 2017-era `addedT0TIPREDVAR` is `t0_b² v` after a first-occasion time-independent predictor (Driver et al., 2017, Table 3, p. 13; p. 16; §7.2; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-23T18:20Z). Form `t0_b` first, then square, then multiply by `v`. A zero coefficient or zero predictor variance is exactly zero. Free `T0TIPREDEFFECT` does not require `a < 0`. `(B / a)² v` is `addedTIPREDVAR` and is not this first-occasion map. `t0_b · √v / √p_0` is `T0TIPREDEFFECTstd` and is not this variance. Free `T0VAR` is not this extra TI variance. `TRAITVAR` is not this extra TI variance. Equation 5 of 2017-era `addedT0TIPREDVAR` is `λ² t0_b² v` (Driver et al., 2017, Eq. 5, p. 5; Table 3, p. 13; Table 2, p. 12; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-23T19:10Z). Form `t0_b² v` first, then `(λ extra) λ` with `θ = 0`. A zero loading or zero extra is exactly zero. `t0_b² v` is the latent extra, not the observed extra. `λ² p_0 + θ` is first-occasion observed variance, not this extra. `λ² (B / a)² v` is Eq. 5 of `addedTIPREDVAR`, not this first-occasion observed extra. `MANIFESTVAR` `θ` is not this extra. Equation 5 of §7.2 `addedTIPREDVAR` is `λ² (B / a)² v` (Driver et al., 2017, Eq. 5, p. 5; Table 2, p. 12; §7.2, pp. 20–21; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-23T19:23Z). Form `(B / a)² v` first, then `(λ extra) λ` with `θ = 0`. A zero loading or zero extra is exactly zero. Lasting asymptotic extra requires `a < 0`. `(B / a)² v` is the latent extra, not the observed extra. `λ² t0_b² v` is first-occasion extra observed TI variance, not this extra. `λ² p + θ` is stationary observed variance, not this extra. `MANIFESTVAR` `θ` is not this extra. Page 16 `TDPREDEFFECTstd` is `m · √v / √(-q / (2 a))` after strictly positive `asymDIFFUSION` and strictly positive time-dependent predictor variance. Unstandardised `M` is not `TDPREDEFFECTstd`. `TIPREDEFFECTstd` is not `TDPREDEFFECTstd` even when `M = B`. intercept-style `A^{-1}[e^{A Δt} − I] M · √v / √p` is not `TDPREDEFFECTstd`. `m · √v / √(trait + p + added)` uses `TRAITVAR` and is not `TDPREDEFFECTstd`. Table 3 / p. 16 `T0TDPREDEFFECTstd` is `t0_m · √v / √p_0` after strictly positive free `T0VAR` and strictly positive TD predictor variance. Unstandardised `t0_m` is not `T0TDPREDEFFECTstd`. `TDPREDEFFECTstd` uses `asymDIFFUSION` and is not `T0TDPREDEFFECTstd`. `T0TIPREDEFFECTstd` is not `T0TDPREDEFFECTstd` even when `t0_m = t0_b`. `t0_m · √v / √(trait + p_0 + added)` uses `TRAITVAR` and is not `T0TDPREDEFFECTstd`. Free `T0VAR` does not require `a < 0`. Page 16 `T0VARstd` is `p_0 / p_0 = 1` after strictly positive free `T0VAR` (`solve(sqrt(diag(T0VAR))) %&% T0VAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; the default ridge is 0). Unstandardised `T0VAR` is not `T0VARstd`. `T0TDPREDEFFECTstd` is not `T0VARstd`. `addedT0TIPREDVAR` is not `T0VARstd`. Page 16 `TRAITVARstd` is `trait / trait = 1` after strictly positive `TRAITVAR` (`solve(sqrt(diag(TRAITVAR))) %&% TRAITVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; no ridge addend). Unstandardised `TRAITVAR` is not `TRAITVARstd`. `T0VARstd` is not `TRAITVARstd` even when both equal 1. `addedT0TIPREDVAR` is not `TRAITVARstd`. Page 16 `MANIFESTTRAITVARstd` is `ψ / ψ = 1` after strictly positive `MANIFESTTRAITVAR` (`solve(sqrt(diag(MANIFESTTRAITVAR))) %&% MANIFESTTRAITVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0). Unstandardised `MANIFESTTRAITVAR` is not `MANIFESTTRAITVARstd`. `TRAITVARstd` is not `MANIFESTTRAITVARstd` even when both equal 1. `MANIFESTVAR` is not `MANIFESTTRAITVARstd`. Page 16 `MANIFESTVARstd` is `θ / θ = 1` after strictly positive `MANIFESTVAR` (`solve(sqrt(diag(MANIFESTVAR))) %&% MANIFESTVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0; 2017-era `dimnames` assignment to `latentNames` is a source bug). Unstandardised `MANIFESTVAR` is not `MANIFESTVARstd`. `MANIFESTTRAITVARstd` is not `MANIFESTVARstd` even when both equal 1. Equation 5 `Var(y)` is not `MANIFESTVARstd`. Page 16 `TIPREDVARstd` is `v / v = 1` after strictly positive `TIPREDVAR` (`solve(sqrt(diag(TIPREDVAR))) %&% TIPREDVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0; `dimnames` are `TIpredNames`). Unstandardised `TIPREDVAR` is not `TIPREDVARstd`. `MANIFESTVARstd` is not `TIPREDVARstd` even when both equal 1. Section 7.2 `addedTIPREDVAR` is not `TIPREDVARstd`. Page 16 `asymDIFFUSIONstd` is `p / p = 1` after strictly positive `asymDIFFUSION` (`solve(sqrt(diag(asymDIFFUSION))) %&% asymDIFFUSION`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0; `dimnames` are `latentNames`). Unstandardised `asymDIFFUSION` is not `asymDIFFUSIONstd`. `TIPREDVARstd` is not `asymDIFFUSIONstd` even when both equal 1. `DIFFUSIONstd` `−2 a` is not `asymDIFFUSIONstd`. Page 16 `discreteCINTstd` is `A^{-1}[e^{A Δt} − I] κ / √p` after strictly positive `asymDIFFUSION`. Unstandardised `discreteCINT` is not `discreteCINTstd`. `κ / √p` is not `discreteCINTstd`. `(-κ / a) / √p` is not `discreteCINTstd`. `asymCINTstd` is `(-κ / a) / √p` after strictly positive `asymDIFFUSION`. Unstandardised `asymCINT` is not `asymCINTstd`. `κ / √p` is not `asymCINTstd`. `discreteCINTstd` is not `asymCINTstd`. `T0MEANSstd` is `μ_0 / √p_0` after strictly positive free `T0VAR`. Unstandardised `T0MEANS` is not `T0MEANSstd`. `T0VARstd` is not `T0MEANSstd`. `μ_0 / √asymDIFFUSION` is not `T0MEANSstd`. Page 16 `MANIFESTMEANSstd` is `τ / √θ` after strictly positive `MANIFESTVAR`. Unstandardised `MANIFESTMEANS` is not `MANIFESTMEANSstd`. `MANIFESTVARstd` is not `MANIFESTMEANSstd`. `τ / √(λ² Var(η) + θ)` is not `MANIFESTMEANSstd`. Page 16 `CINTstd` is `κ / √p` after strictly positive `asymDIFFUSION`. Unstandardised `CINT` is not `CINTstd`. `asymCINTstd` is not `CINTstd`. `discreteCINTstd` is not `CINTstd`. `κ / √(trait + p + added)` is not `CINTstd`. Evolving from that stationary start with `CINT` and `TIPREDEFFECT` stays at the stationary mean. Equation 1 is the latent SDE, not the measurement model. Form `(λ p) λ` then add `θ`, then add `ψ`. `MANIFESTVAR` is `Θ`, not `Var(y)`. `MANIFESTTRAITVAR` is `Ψ_τ`, not `Θ`. `TRAITVAR` is latent and scaled by `λ²`. `Var(η)` is not `Var(y)`. -- Separate cluster means before within-unit lag. CWC plus an event-time lag is not DSEM. Subtracting the person-specific mean from a raw autoregressive series does not isolate the lagged within-person effect (Curran & Bauer, 2011, pp. 607–608); already-centered residuals with irregular event intervals use the exact scalar map. +- Do not use the difference quotient as a continuous-time rate. The scalar map is `a = ln(φ) / Δt` on event time. Discrete lags from unequal event intervals are not one coefficient; remap them through that log-rate. Binary64 `exp(a Δt) = 0` is not a discrete lag. A constant predictor's discrete effect is Voelkle et al. (2012, Eq. 12), evaluated as `a_yx (expm1(z) / a_xx)` with `z = a_xx Δt` so a finite result is not lost when `z` overflows to `-∞` or when `a_yx Δt` overflows. When `expm1(z)` overflows at a finite `z`, rewrite in log space; a zero continuous effect is exactly zero; an overflowing `a_yx/a_xx` rewrite term fails closed. The first-order product is the underflow limit of that equation, not the general constant-predictor discrete effect. A time-varying predictor whose sampling interval equals its constancy interval uses Voelkle et al. (2012, Eq. 14): `b* = a_yx Δt`. Unmatched intervals fail closed (Oud & Jansen, 2000, unread). Discrete process noise is Driver et al. (2017, Eq. 3): `Q_Δt = 0.5 q (expm1(z) / a)` with `z = 2 (a Δt)` and `q = G G⊤ ≥ 0`; do not form `2 a` first; `a = 0` and `z → 0` recover `q Δt`; a zero diffusion is exactly zero; an overflowing rewrite scale `0.5 q / a` fails closed; this is not a Kalman filter. `Q_Δt` is `cov(η_t | η_{t-1})`, not `Var(η_t)`. The lagged covariance is `exp(a Δt) p` and the unconditional variance is `exp(2 a Δt) p + Q_Δt` (Driver et al., 2017, Eq. 3–4, pp. 4–5; JSS has no numbered §2.2). A zero diffusion whose `2 (a Δt)` overflows to `+∞` is not a finite `Var(η_t)`. The stationary within-subject variance is the `Δt → ∞` limit of Eq. 4: `-q / (2 a)` for stable `a < 0` (JSS p. 16 `asymDIFFUSION`; §4.3). When `2 a` is finite, form `q / -(2 a)` so `q / a` overflow does not lose a finite result (`q = MAX`, `a = -0.75` → `MAX / 1.5`). When `2 a` overflows, form `(q * 0.5) / |a|`. In that branch `|a|` is already so large that halving a minimum-subnormal `q` cannot discard a representable positive final stationary variance; the ratio-first rewrite can instead overflow before the final factor restores a representable result. `a ≥ 0` has no finite stationary variance. Finite-interval `Q_Δt` is not that limit. Trait-plus-state variance is `trait + state` and lagged covariance is `trait + exp(a Δt) p` (Driver et al., 2017, §4.3, p. 9). Trait variance is not process noise and not `asymDIFFUSION`. Evolving the summed variance as if it were all state is not that map. This is not RI-CLPM. Observed-indicator variance is `λ² Var(η) + θ` when `MANIFESTTRAITVAR` is zero and `λ² Var(η) + θ + ψ` otherwise (Driver et al., 2017, Eq. 5, p. 5; Table 2, p. 12). Lagged observed covariance is `λ² cov(η_t, η_{t-1}) + ψ`; `MANIFESTVAR` does not enter. Observed-indicator mean is `τ + λ μ` (Driver et al., 2017, Eq. 5; Table 2, p. 12). `MANIFESTMEANS` is `τ`, not `E(y)`. `E(η)` is not `E(y)`. `CINT` is not `MANIFESTMEANS`. `T0MEANS` is not `E(y)`. The discrete latent mean is `μ_t = exp(a Δt) μ_0 + (exp(a Δt) − 1)/a κ` (Driver et al., 2017, Eq. 3, p. 4; Table 2, p. 12). `T0MEANS` is not `μ_t`. `CINT` is not that discrete increment. A zero drift is `κ Δt`. Underflow of `exp(a Δt)` to `+0` drops the carried `T0MEANS` and keeps `−κ / a`. The evolved observed mean is `τ + λ μ_t` (Driver et al., 2017, Eq. 5 of that Eq. 3 map). The first-occasion map `τ + λ μ_0` is not `E(y_t)`. `μ_t` is not `E(y_t)`. The contemporaneous time-dependent predictor impulse is `m x` (Driver et al., 2017, Eq. 3 fourth summand; Table 2 `TDPREDEFFECT` is `M`). Form `μ_t` first, then add `m x`. `TDPREDEFFECT` is not `CINT`. `M x` is not `A^{-1}[e^{A Δt} − I] B z` and is not Voelkle et al. (2012, Eq. 14). The §7.2 level-change form is not that impulse. The observed mean of that contemporaneous impulse is `τ + λ(μ_t + m x)` (Driver et al., 2017, Eq. 5 of the Eq. 3 fourth-summand composition). The evolved map `τ + λ μ_t` is not that observed mean. The carry map `τ + λ(μ_t + e^{a(t−u)} m x)` is not that observed mean when `u ≠ t`. The evolved-plus-impulse latent mean is not `E(y_t)`. The time-independent predictor increment is `A^{-1}[e^{A Δt} − I] B z` (Driver et al., 2017, Eq. 3 second summand; Table 2 `TIPREDEFFECT` is `B`). Form `B z` first, then the discrete intercept map. A zero drift is `B z Δt`. `TIPREDEFFECT` is `B`, not that discrete increment. `A^{-1}[e^{A Δt} − I] B z` is not `CINT`, not `M x`, and not Voelkle et al. (2012, Eq. 14). The observed mean of that increment is `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` (Driver et al., 2017, Eq. 5 of the Eq. 3 printed addend after the `T0MEANS` carry and the `CINT` increment). The evolved map `τ + λ μ_t` is not that observed mean. The contemporaneous map `τ + λ(μ_t + m x)` is not that observed mean. The carry map `τ + λ(μ_t + e^{a(t−u)} m x)` is not that observed mean when `u ≠ t`. The evolved-plus-increment latent mean is not `E(y_t)`. The within-interval time-dependent impulse carry is `e^{A(t−u)} M x` for `t0 < u < t` (Driver et al., 2017, Eq. 1–2 Green-function integral of Eq. 2; §7.2 dissipation). Form `m x` first, then `e^{a(t−u)} m x`. A zero drift is `m x` with no dissipation. Underflow of `e^{a(t−u)}` to `+0` is vanishing dissipation and is kept. `e^{A(t−u)} M x` is not the contemporaneous Dirac, not `CINT`, not `TIPREDEFFECT`, and not Voelkle et al. (2012, Eq. 14). An impulse at `u = t` is the contemporaneous map. An impulse at `u ≤ t0` is already in `η(t0)`. The observed mean of that carry is `τ + λ(μ_t + e^{a(t−u)} m x)` (Driver et al., 2017, Eq. 5 of the Eq. 1–2 carried latent mean). The evolved map `τ + λ μ_t` is not that observed mean. The contemporaneous map `τ + λ(μ_t + m x)` is not that observed mean when `u ≠ t`. `MANIFESTMEANS` is not `E(y_t)`. The carried latent mean is not `E(y_t)`. The first-occasion time-independent predictor shift is `t0_b z` (Driver et al., 2017, Table 3 `T0TIPREDEFFECT`; Eq. 3 first summand). Form `t0_b z` first, then `e^{a Δt} t0_b z`. Form `μ_t` first, then add that carry. A zero drift is `t0_b z`. Underflow of `e^{a Δt}` to `+0` is a vanishing carry of the first-occasion shift and is kept. `t0_b z` is not `A^{-1}[e^{A Δt} − I] B z`, not `CINT`, and not `M x`. `e^{A Δt} t0_b z` is not `t0_b z`. `T0TIPREDEFFECT` is the coefficient, not the shift. The observed mean of that first-occasion carry is `τ + λ(μ_t + e^{a Δt} t0_b z)` (Driver et al., 2017, Eq. 5 of the Table 3 / Eq. 3 first-summand composition). The evolved map `τ + λ μ_t` is not that observed mean. The process-increment map `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` is not that observed mean. The contemporaneous map `τ + λ(μ_t + m x)` is not that observed mean. The impulse-carry map `τ + λ(μ_t + e^{a(t−u)} m x)` is not that observed mean when `u ≠ t0`. The evolved-plus-carry latent mean is not `E(y_t)`. The first-occasion time-dependent predictor shift is `t0_m x0` (Driver et al., 2017, Table 3 `T0TDPREDEFFECT`; Eq. 3 first summand; JSS PDF re-opened 2026-08-20T19:10Z). Form `t0_m x0` first, then `e^{a Δt} t0_m x0`. Form `μ_t` first, then add that carry. A zero drift is `t0_m x0`. Underflow of `e^{a Δt}` to `+0` is a vanishing carry of the first-occasion shift and is kept. `t0_m x0` is not `M x`, not `e^{A(t−u)} M x` for `t0 < u < t`, not `t0_b z`, not `A^{-1}[e^{A Δt} − I] B z`, and not `CINT`. `e^{A Δt} t0_m x0` is not `t0_m x0`. `T0TDPREDEFFECT` is the coefficient, not the shift. An impulse at `u ≤ t0` that used `M` is already in `η(t0)` as `TDPREDEFFECT`, not as `T0TDPREDEFFECT`. The observed mean of that first-occasion TD carry is `τ + λ(μ_t + e^{a Δt} t0_m x0)` (Driver et al., 2017, Eq. 5 of the Table 3 / Eq. 3 first-summand TD composition; JSS PDF re-opened 2026-08-20T19:07Z). The evolved map `τ + λ μ_t` is not that observed mean. The process-increment map `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` is not that observed mean. The contemporaneous map `τ + λ(μ_t + m x)` is not that observed mean. The impulse-carry map `τ + λ(μ_t + e^{a(t−u)} m x)` is not that observed mean when `u ≠ t0`. The first-occasion TI map `τ + λ(μ_t + e^{a Δt} t0_b z)` is not that observed mean. The evolved-plus-carry latent mean is not `E(y_t)`. The lasting level-change `CINT` is `κ = −a m x` (Driver et al., 2017, §7.2, pp. 20–21; JSS PDF re-opened 2026-08-20T19:45Z). Form `m x` first, then multiply by `−a`. Stable `a < 0` is required so `−κ / a = m x` is an equilibrium offset. `a ≥ 0` cannot hold a new process mean. `−a m x` is not the dissipating Dirac `m x`, not a free `CINT`, and not `A^{-1}[e^{A Δt} − I] B z`. The extra near-zero-drift latent process also named in §7.2 is a different specification and is not this `CINT` setting. Equation 3 maps that intercept as `(1 − e^{a Δt}) m x` (JSS PDF re-opened 2026-08-20T19:50Z). Form the level-change `CINT` first, then the discrete intercept map. Underflow of `e^{a Δt}` to `+0` keeps `m x`. `(1 − e^{a Δt}) m x` is not `m x`, not `κ`, and not `A^{-1}[e^{A Δt} − I] B z`. The printed §7.2 lasting level change is an extra near-zero-drift latent process (Driver et al., 2017, §7.2, pp. 22–23; JSS PDF re-opened 2026-08-20T23:10Z). `T0MEANS`, `CINT`, `T0VAR`, `DIFFUSION`, and `TRAITVAR` of that process are fixed to 0; `TDPREDEFFECT` on it is fixed to 1; its `DRIFT` diagonal is very close to 0 (printed example `−0.000001`; precisely 0 causes computational problems); the original process is driven by the `DRIFT` coupling `a_{ηξ}`. After a unit identification impulse the scalar contribution is `a_{ηξ} x (e^{ε Δt} − e^{a Δt}) / (ε − a)` (`ε = a` is `a_{ηξ} x Δt e^{a Δt}`). Form `a_{ηξ} x` first. A zero coupling or zero predictor is exactly zero. `ε ≥ 0` fails closed. That contribution is not `κ = −a m x`, not `(1 − e^{a Δt}) m x`, and not the dissipating Dirac `m x`. The observed mean of that extra-process contribution is `τ + λ(μ_t + a_{ηξ} x (e^{ε Δt} − e^{a Δt}) / (ε − a))` (Driver et al., 2017, Eq. 5 of that §7.2 contribution; JSS PDF re-opened 2026-08-21T06:12Z). The extra process has `LAMBDA` 0 and is not an observed indicator. Original indicators load on the original process after the `DRIFT` coupling. The evolved map `τ + λ μ_t` is not that observed mean. The contemporaneous map `τ + λ(μ_t + m x)` is not that observed mean. The contribution is not `E(y_t)`. The evolved-plus-contribution latent mean is not `E(y_t)`. `T0TDPREDEFFECT` on the extra process begins at `t = 0` and uses `Δt = t − t0` for both the original-process evolution and the extra drive. `TDPREDEFFECT` after `t0` uses `t − u` with `t0 < u < t` for the extra drive while `μ_t` still uses `Δt`. The observed mean of that after-t0 extra-process contribution is `τ + λ(μ_t + a_{ηξ} x (e^{ε(t−u)} − e^{a(t−u)}) / (ε − a))` (Driver et al., 2017, Eq. 5 of that §7.2 after-t0 contribution; JSS PDF re-opened 2026-08-21T06:32Z). The first-occasion extra-process observed mean is not that observed mean when `u ≠ t0`. The impulse-carry map `τ + λ(μ_t + e^{a(t−u)} m x)` is a Dirac on the original process and is not that `DRIFT` drive. An impulse at `u = t0` or `u = t` is not interior. The asymptotic time-independent predictor effect is `-B z / a` (Driver et al., 2017, §7.2, pp. 20–21; JSS PDF opened 2026-08-21T13:08Z). Form `B z` first, then divide by `-a`. Stable `a < 0` is required. `a ≥ 0` cannot hold a finite process-mean change. `-B z / a` is not the coefficient `B`, not `A^{-1}[e^{A Δt} − I] B z`, not `CINT`, and not `M x`. The asymptotic time-independent predictor variance is `(B / a)² v` (Driver et al., 2017, §7.2, pp. 20–21 `addedTIPREDVAR`). Form the unit asymptotic effect first, then square, then multiply by `v`. `(B / a)² v` is not `TRAITVAR`, not `asymDIFFUSION`, and not `-B z / a`. The asymptotic continuous intercept is `-κ / a` (Driver et al., 2017, Table 2, p. 12 `asymCINT`; Eq. 3 as `Δt → ∞`; JSS PDF opened 2026-08-21T16:13Z). Form `κ` first, then divide by `-a`. Stable `a < 0` is required. `-κ / a` is not `κ`, not `A^{-1}[e^{A Δt} − I] κ`, not `T0MEANS`, and not `-B z / a`. The p. 16 stationary `T0MEANS` constraint is `-κ / a + −B z / a`. Form the intercept contribution first, then include the TI extra effect, then add. That constrained first-occasion mean is not free `T0MEANS`, not `asymCINT` alone, not `asymTIPREDEFFECT` alone, and not the finite-interval discrete latent mean. Equation 5 of that constrained mean is `τ + λ(−κ / a + −B z / a)` (Driver et al., 2017, §4.3, pp. 9–10; Eq. 5, p. 5; JSS PDF re-opened 2026-08-21T20:07Z). Form the stationary latent mean first, then `τ + λ` of that mean. `τ + λ μ_0` for free `T0MEANS` is not that composition. `τ + λ(−κ / a)` is not that composition when `B z ≠ 0`. `τ + λ μ_t` is not that composition. `MANIFESTMEANS` is not `E(y_0)`. The constrained latent mean is not `E(y_0)`. The p. 16 constrained first-occasion variance `trait + −q / (2 a) + (B / a)² v` is not free `T0VAR`, not `asymDIFFUSION` alone, not `TRAITVAR` alone, not `addedTIPREDVAR` alone, and not the finite-interval discrete latent variance. Eq. 5 of that constrained variance is `λ²(trait + −q / (2 a) + (B / a)² v) + θ + ψ` (JSS PDF re-opened 2026-08-22T03:20Z; form the stationary latent variance first, then `λ² p + θ + ψ`; `λ² p_0` is not that observed variance; `λ²(−q / (2 a)) + θ` is not that observed variance when `TRAITVAR` or `addedTIPREDVAR` is nonzero; `MANIFESTVAR` is not `Var(y_0)`; the constrained latent variance is not `Var(y_0)`). The lagged covariance of that constrained process is `trait + e^{a Δt}(−q / (2 a)) + (B / a)² v` (Driver et al., 2017, Eq. 3–4 of §4.3 / p. 16 `T0VAR`; JSS PDF re-opened 2026-08-22T19:13Z). Trait and `addedTIPREDVAR` do not decay with `e^{a Δt}`. Contemporaneous `T0VAR` is not that lagged map. Decaying the constrained total as if it were all state is not that lagged map. Equation 5 of that lagged covariance is `λ²(trait + e^{a Δt}(−q / (2 a)) + (B / a)² v) + ψ`. `Θ` does not enter. Contemporaneous `Var(y_0)` is not that lagged observed covariance. The lagged latent covariance is not that observed covariance. The later-occasion variance of that constrained process is `trait + e^{2 a Δt}(−q / (2 a)) + Q_Δt + (B / a)² v` (Driver et al., 2017, Eq. 3–4 of §4.3 / p. 16 `T0VAR`; JSS PDF re-opened 2026-08-22T23:12Z). Trait and `addedTIPREDVAR` do not enter `Q_Δt`. Under stationarity that composition equals contemporaneous `T0VAR`. Evolving the constrained total as if it were all state is not that later map. The lagged covariance omits `Q_Δt` and is not that later map. `Q_Δt` is not that later map. Equation 5 of that later-occasion variance is `λ²(trait + e^{2 a Δt}(−q / (2 a)) + Q_Δt + (B / a)² v) + θ + ψ`. The lagged observed covariance omits `Q_Δt` and `θ`. `MANIFESTVAR` is not `Var(y_t)`. The later-occasion latent variance is not `Var(y_t)`. The later-occasion variance of §4.3 predetermined `T0VAR` is `trait + e^{2 a Δt} p_0 + Q_Δt + (B / a)² v` (Driver et al., 2017, Eq. 3–4 of §4.3 predetermined first occasion; JSS PDF re-opened 2026-08-23T05:12Z). Trait and `addedTIPREDVAR` do not enter `Q_Δt`. Free `T0VAR` `p_0` is not that later map. Setting `p_0 = −q / (2 a)` recovers the stationary later-occasion map. Stationary later variance uses `−q / (2 a)` in place of `p_0` and is not that later map when `p_0` is free. Evolving `trait + p_0 + (B / a)² v` as if it were all state is not that later map. As `Δt → ∞` with stable `a < 0` the composition approaches contemporaneous stationary `T0VAR`. As `Δt → 0+` the composition approaches `trait + p_0 + (B / a)² v`. Nonzero diffusion with `a ≥ 0` is a growing process and is kept. Equation 5 of that predetermined later-occasion variance is `λ²(trait + e^{2 a Δt} p_0 + Q_Δt + (B / a)² v) + θ + ψ`. `MANIFESTVAR` is not `Var(y_t)`. The predetermined later-occasion latent variance is not `Var(y_t)`. Stationary later observed variance is not that observed variance when `p_0` is free. The lagged covariance of §4.3 predetermined `T0VAR` is `trait + e^{a Δt} p_0 + (B / a)² v` (Driver et al., 2017, Eq. 3–4 of §4.3 predetermined first occasion; JSS PDF re-opened 2026-08-23T09:04Z). Trait and `addedTIPREDVAR` do not decay with `e^{a Δt}`. Free `T0VAR` `p_0` is not that lagged map. Setting `p_0 = −q / (2 a)` recovers the stationary lagged map. Stationary lagged covariance uses `−q / (2 a)` in place of `p_0` and is not that lagged map when `p_0` is free. Evolving `trait + p_0 + (B / a)² v` as if it were all state is not that lagged map. Later-occasion variance includes `Q_Δt` and is not that lagged map. As `Δt → ∞` with stable `a < 0` the state term vanishes. As `Δt → 0+` the composition approaches `trait + p_0 + (B / a)² v`. Equation 5 of that predetermined lagged covariance is `λ²(trait + e^{a Δt} p_0 + (B / a)² v) + ψ`. `MANIFESTVAR` does not enter. The predetermined lagged latent covariance is not that observed covariance. Predetermined later observed variance includes `Q_Δt` and `θ` and is not that lagged observed covariance. Stationary lagged observed covariance is not that observed covariance when `p_0` is free. The predetermined first-occasion variance of §4.3 predetermined `T0VAR` is `trait + p_0 + (B / a)² v`. Free `p_0` is not that map. Stationary first-occasion variance uses `−q / (2 a)` in place of `p_0` and is not that map when `p_0` is free. Lagged covariance decays the state and is not that map. Later-occasion variance includes `Q_Δt` and is not that map. Equation 5 of that predetermined first-occasion variance is `λ²(trait + p_0 + (B / a)² v) + θ + ψ`. `MANIFESTVAR` is not that first-occasion observed variance. The predetermined first-occasion latent variance is not that observed variance. Stationary first-occasion observed variance is not that observed variance when `p_0` is free. Predetermined later observed variance includes `Q_Δt` and is not that first-occasion observed variance. Later-start lagged covariance of predetermined `T0VAR` is `trait + e^{a s}(e^{2 a u} p_0 + Q_u) + (B / a)² v` (Driver et al., 2017, §4.3 `startoffset`; Eq. 4; JSS PDF re-opened 2026-08-23T10:27Z). First-occasion lagged omits `e^{a s} Q_u`. Later-occasion variance does not lag. Stationary lagged uses `−q / (2 a)`. Decaying the later total is not that map. Equation 5 of that later-start lagged covariance is `λ²` of it plus `ψ`. Independent `ε_t` does not enter. First-occasion lagged observed omits `e^{a s} Q_u`. Predetermined later observed variance includes `Q_u` and `θ` and is not that later-start lagged observed covariance. Later-start later-occasion variance of predetermined `T0VAR` is `trait + e^{2 a s}(e^{2 a u} p_0 + Q_u) + Q_s + (B / a)² v` (Driver et al., 2017, §4.3 `startoffset`; Eq. 3–4 Chapman–Kolmogorov `Q_{u+s} = e^{2 a s} Q_u + Q_s`; JSS PDF re-opened 2026-08-23T11:05Z). Later-occasion variance at `u` omits `Q_s`. Later-start lagged covariance omits `Q_s`. Stationary later uses `−q / (2 a)`. Evolving the later total as if it were all state is not that map. Ignoring `startoffset` omits `e^{2 a s} Q_u`. Equation 5 of that later-start later-occasion variance is `λ²` of it plus `θ + ψ`. `MANIFESTVAR` is not that observed variance. Page 16 `discreteDRIFTstd` is `e^{a Δt}` after strictly positive `asymDIFFUSION` `-q / (2 a)` (Driver et al., 2017, p. 16; footnote 4; §7.1; JSS PDF re-opened 2026-08-23T11:40Z). Unstandardised `e^{a Δt}` is defined for growing `a ≥ 0` and for zero diffusion and is not `discreteDRIFTstd`. The §7.1 trait-plus-state autocorrelation `(trait + e^{a Δt} p + added) / (trait + p + added)` uses `TRAITVAR` and is not `discreteDRIFTstd`. `TRAITVAR` is not the standardisation variance. Page 16 `discreteDIFFUSIONstd` is `Q_Δt / (−q / (2 a))` after strictly positive `asymDIFFUSION` `-q / (2 a)` (Driver et al., 2017, p. 16; Eq. 4; footnote 4; JSS PDF re-opened 2026-08-23T13:06Z). Unstandardised `Q_Δt` is defined for growing `a ≥ 0` and for zero diffusion and is not `discreteDIFFUSIONstd`. The continuous standardisation `−2 a` is not `discreteDIFFUSIONstd`. `Q_Δt / (trait + p + added)` uses `TRAITVAR` and is not `discreteDIFFUSIONstd`. `TRAITVAR` is not the standardisation variance. Page 16 `DIFFUSIONstd` is `q / (−q / (2 a)) = −2 a` after strictly positive `asymDIFFUSION` `-q / (2 a)` (Driver et al., 2017, p. 16; Eq. 4; footnote 4; JSS PDF re-opened 2026-08-23T13:20Z). Unstandardised `q` is defined for growing `a ≥ 0` and for zero diffusion and is not `DIFFUSIONstd`. The discrete standardisation `Q_Δt / (−q / (2 a))` depends on `Δt` and is not `DIFFUSIONstd`. `q / (trait + p + added)` uses `TRAITVAR` and is not `DIFFUSIONstd`. `TRAITVAR` is not the standardisation variance. Page 16 `DRIFTstd` is the continuous auto-effect after strictly positive `asymDIFFUSION` `-q / (2 a)` (Driver et al., 2017, p. 16; Eq. 1; footnote 4; JSS PDF re-opened 2026-08-23T13:28Z). Unstandardised `a` is defined for growing `a ≥ 0` and for zero diffusion and is not `DRIFTstd`. The discrete standardisation `e^{a Δt}` depends on the event interval and is not `DRIFTstd`. `a p / (trait + p + added)` uses `TRAITVAR` and is not `DRIFTstd`. `TRAITVAR` is not the standardisation variance. Page 16 `asymTIPREDEFFECTstd` is `(-B / a) · √v / √(-q / (2 a))` after strictly positive `asymDIFFUSION` `-q / (2 a)` and strictly positive predictor variance `v` (Driver et al., 2017, p. 16; §7.2; footnote 4; JSS PDF re-opened 2026-08-23T14:25Z). Unstandardised `-B / a` is defined for a zero coefficient and for zero predictor variance and is not `asymTIPREDEFFECTstd`. The finite-interval standardisation `A^{-1}[e^{A Δt} − I] B · √v / √p` depends on the event interval and is not `asymTIPREDEFFECTstd`. `(-B / a) · √v / √(trait + p + added)` uses `TRAITVAR` and is not `asymTIPREDEFFECTstd`. `TRAITVAR` is not the standardisation variance. Page 16 `TIPREDEFFECTstd` is `B · √v / √(-q / (2 a))` after strictly positive `asymDIFFUSION` `-q / (2 a)` and strictly positive predictor variance `v` (Driver et al., 2017, p. 16; §7.2; footnote 4; JSS PDF re-opened 2026-08-23T16:21Z). Unstandardised `B` is defined for a zero coefficient and for zero predictor variance and is not `TIPREDEFFECTstd`. The asymptotic standardisation `(-B / a) · √v / √p` is the total change and is not `TIPREDEFFECTstd`. The finite-interval standardisation `A^{-1}[e^{A Δt} − I] B · √v / √p` depends on the event interval and is not `TIPREDEFFECTstd`. `B · √v / √(trait + p + added)` uses `TRAITVAR` and is not `TIPREDEFFECTstd`. `TRAITVAR` is not the standardisation variance. Page 16 / Table 3 `T0TIPREDEFFECTstd` is `t0_b · √v / √p_0` after strictly positive free `T0VAR` `p_0` and strictly positive predictor variance `v` (Driver et al., 2017, Table 3, p. 13; p. 16; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-23T17:20Z). The affected variance is free first-occasion `T0VAR`, not `asymDIFFUSION`. Unstandardised `t0_b` is defined for a zero coefficient and for zero predictor variance and is not `T0TIPREDEFFECTstd`. `TIPREDEFFECTstd` `B · √v / √(-q / (2 a))` is the continuous coefficient and is not `T0TIPREDEFFECTstd`. `asymTIPREDEFFECTstd` `(-B / a) · √v / √p` is the total change and is not `T0TIPREDEFFECTstd`. `t0_b · √v / √(trait + p_0 + added)` uses `TRAITVAR` and is not `T0TIPREDEFFECTstd`. `TRAITVAR` is not the standardisation variance. 2017-era `addedT0TIPREDVAR` is `t0_b² v` after a first-occasion time-independent predictor (Driver et al., 2017, Table 3, p. 13; p. 16; §7.2; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-23T18:20Z). Form `t0_b` first, then square, then multiply by `v`. A zero coefficient or zero predictor variance is exactly zero. Free `T0TIPREDEFFECT` does not require `a < 0`. `(B / a)² v` is `addedTIPREDVAR` and is not this first-occasion map. `t0_b · √v / √p_0` is `T0TIPREDEFFECTstd` and is not this variance. Free `T0VAR` is not this extra TI variance. `TRAITVAR` is not this extra TI variance. Equation 5 of 2017-era `addedT0TIPREDVAR` is `λ² t0_b² v` (Driver et al., 2017, Eq. 5, p. 5; Table 3, p. 13; Table 2, p. 12; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-23T19:10Z). Form `t0_b² v` first, then `(λ extra) λ` with `θ = 0`. A zero loading or zero extra is exactly zero. `t0_b² v` is the latent extra, not the observed extra. `λ² p_0 + θ` is first-occasion observed variance, not this extra. `λ² (B / a)² v` is Eq. 5 of `addedTIPREDVAR`, not this first-occasion observed extra. `MANIFESTVAR` `θ` is not this extra. Equation 5 of §7.2 `addedTIPREDVAR` is `λ² (B / a)² v` (Driver et al., 2017, Eq. 5, p. 5; Table 2, p. 12; §7.2, pp. 20–21; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-23T19:23Z). Form `(B / a)² v` first, then `(λ extra) λ` with `θ = 0`. A zero loading or zero extra is exactly zero. Lasting asymptotic extra requires `a < 0`. `(B / a)² v` is the latent extra, not the observed extra. `λ² t0_b² v` is first-occasion extra observed TI variance, not this extra. `λ² p + θ` is stationary observed variance, not this extra. `MANIFESTVAR` `θ` is not this extra. Page 16 `TDPREDEFFECTstd` is `m · √v / √(-q / (2 a))` after strictly positive `asymDIFFUSION` and strictly positive time-dependent predictor variance. Unstandardised `M` is not `TDPREDEFFECTstd`. `TIPREDEFFECTstd` is not `TDPREDEFFECTstd` even when `M = B`. intercept-style `A^{-1}[e^{A Δt} − I] M · √v / √p` is not `TDPREDEFFECTstd`. `m · √v / √(trait + p + added)` uses `TRAITVAR` and is not `TDPREDEFFECTstd`. Table 3 / p. 16 `T0TDPREDEFFECTstd` is `t0_m · √v / √p_0` after strictly positive free `T0VAR` and strictly positive TD predictor variance. Unstandardised `t0_m` is not `T0TDPREDEFFECTstd`. `TDPREDEFFECTstd` uses `asymDIFFUSION` and is not `T0TDPREDEFFECTstd`. `T0TIPREDEFFECTstd` is not `T0TDPREDEFFECTstd` even when `t0_m = t0_b`. `t0_m · √v / √(trait + p_0 + added)` uses `TRAITVAR` and is not `T0TDPREDEFFECTstd`. Free `T0VAR` does not require `a < 0`. Page 16 `T0VARstd` is `p_0 / p_0 = 1` after strictly positive free `T0VAR` (`solve(sqrt(diag(T0VAR))) %&% T0VAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; the default ridge is 0). Unstandardised `T0VAR` is not `T0VARstd`. `T0TDPREDEFFECTstd` is not `T0VARstd`. `addedT0TIPREDVAR` is not `T0VARstd`. Page 16 `TRAITVARstd` is `trait / trait = 1` after strictly positive `TRAITVAR` (`solve(sqrt(diag(TRAITVAR))) %&% TRAITVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; no ridge addend). Unstandardised `TRAITVAR` is not `TRAITVARstd`. `T0VARstd` is not `TRAITVARstd` even when both equal 1. `addedT0TIPREDVAR` is not `TRAITVARstd`. Page 16 `MANIFESTTRAITVARstd` is `ψ / ψ = 1` after strictly positive `MANIFESTTRAITVAR` (`solve(sqrt(diag(MANIFESTTRAITVAR))) %&% MANIFESTTRAITVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0). Unstandardised `MANIFESTTRAITVAR` is not `MANIFESTTRAITVARstd`. `TRAITVARstd` is not `MANIFESTTRAITVARstd` even when both equal 1. `MANIFESTVAR` is not `MANIFESTTRAITVARstd`. Page 16 `MANIFESTVARstd` is `θ / θ = 1` after strictly positive `MANIFESTVAR` (`solve(sqrt(diag(MANIFESTVAR))) %&% MANIFESTVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0; 2017-era `dimnames` assignment to `latentNames` is a source bug). Unstandardised `MANIFESTVAR` is not `MANIFESTVARstd`. `MANIFESTTRAITVARstd` is not `MANIFESTVARstd` even when both equal 1. Equation 5 `Var(y)` is not `MANIFESTVARstd`. Page 16 `TIPREDVARstd` is `v / v = 1` after strictly positive `TIPREDVAR` (`solve(sqrt(diag(TIPREDVAR))) %&% TIPREDVAR`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0; `dimnames` are `TIpredNames`). Unstandardised `TIPREDVAR` is not `TIPREDVARstd`. `MANIFESTVARstd` is not `TIPREDVARstd` even when both equal 1. Section 7.2 `addedTIPREDVAR` is not `TIPREDVARstd`. Page 16 `asymDIFFUSIONstd` is `p / p = 1` after strictly positive `asymDIFFUSION` (`solve(sqrt(diag(asymDIFFUSION))) %&% asymDIFFUSION`; OpenMx `%&%` is `t(A) %*% B %*% A`; 2017-era source adds ridging; default ridge is 0; `dimnames` are `latentNames`). Unstandardised `asymDIFFUSION` is not `asymDIFFUSIONstd`. `TIPREDVARstd` is not `asymDIFFUSIONstd` even when both equal 1. `DIFFUSIONstd` `−2 a` is not `asymDIFFUSIONstd`. Page 16 `discreteCINTstd` is `A^{-1}[e^{A Δt} − I] κ / √p` after strictly positive `asymDIFFUSION`. Unstandardised `discreteCINT` is not `discreteCINTstd`. `κ / √p` is not `discreteCINTstd`. `(-κ / a) / √p` is not `discreteCINTstd`. `asymCINTstd` is `(-κ / a) / √p` after strictly positive `asymDIFFUSION`. Unstandardised `asymCINT` is not `asymCINTstd`. `κ / √p` is not `asymCINTstd`. `discreteCINTstd` is not `asymCINTstd`. `T0MEANSstd` is `μ_0 / √p_0` after strictly positive free `T0VAR`. Unstandardised `T0MEANS` is not `T0MEANSstd`. `T0VARstd` is not `T0MEANSstd`. `μ_0 / √asymDIFFUSION` is not `T0MEANSstd`. Page 16 `MANIFESTMEANSstd` is `τ / √θ` after strictly positive `MANIFESTVAR`. Unstandardised `MANIFESTMEANS` is not `MANIFESTMEANSstd`. `MANIFESTVARstd` is not `MANIFESTMEANSstd`. `τ / √(λ² Var(η) + θ)` is not `MANIFESTMEANSstd`. Page 16 `CINTstd` is `κ / √p` after strictly positive `asymDIFFUSION`. Unstandardised `CINT` is not `CINTstd`. `asymCINTstd` is not `CINTstd`. `discreteCINTstd` is not `CINTstd`. `κ / √(trait + p + added)` is not `CINTstd`. Evolving from that stationary start with `CINT` and `TIPREDEFFECT` stays at the stationary mean. Equation 1 is the latent SDE, not the measurement model. Form `(λ p) λ` then add `θ`, then add `ψ`. `MANIFESTVAR` is `Θ`, not `Var(y)`. `MANIFESTTRAITVAR` is `Ψ_τ`, not `Θ`. `TRAITVAR` is latent and scaled by `λ²`. `Var(η)` is not `Var(y)`. +- Separate cluster means before within-unit lag. CWC plus an event-time lag is not DSEM. Subtracting the person-specific mean from a raw autoregressive series does not isolate the lagged within-person effect (Curran & Bauer, 2011, pp. 583–619; PMC3059070 XML opened 2026-09-02; Eq. 36). `longitudinal_core` owns CWC-then-pairwise-mean residual log-rate on typed `EventTimeInterval`; for an already-centered `(1, 0.5)` pair it recovers `ln(0.5) / Δt`, equal to `ln(0.5)` only when `Δt = 1`. Exact zero CWC residuals use canonical public `+0.0`. CWC log-rate is not raw-process drift. +- Occasion-mean residuals are Hamaker, Kuiper, and Grasman (2015, Eq. 1a) `x_it = μ_t + p_it` deviations, not CWC and not RI-CLPM. Exact zero occasion-mean residuals use canonical public `+0.0`. They still retain between-person differences and are not within-person change. - Do not treat the CWC cluster-mean coefficient as the between-cluster effect. It is the contextual effect `between − within` (Enders & Tofighi, 2007, Table 2, pp. 124–127). - Never use future-available evidence in historical model fits. - Do not blanket-mask PII when identity/role/linkage is scientifically required. Follow the purpose-bound separation, opaque-ID, encryption, retention, and audit contract in `docs/PRIVACY_DATA_GOVERNANCE.md`. diff --git a/crates/longitudinal_core/Cargo.toml b/crates/longitudinal_core/Cargo.toml index 72828a1af..b50d6dfe0 100644 --- a/crates/longitudinal_core/Cargo.toml +++ b/crates/longitudinal_core/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "longitudinal_core" -description = "Within/between decomposition gates and component RMSE." +description = "Longitudinal modeling gates, within/between decomposition, CWC-then-irregular residual log-rate, temporal association standardization, and recovery metrics." version.workspace = true edition.workspace = true rust-version.workspace = true diff --git a/crates/longitudinal_core/src/association.rs b/crates/longitudinal_core/src/association.rs new file mode 100644 index 000000000..679db33f7 --- /dev/null +++ b/crates/longitudinal_core/src/association.rs @@ -0,0 +1,385 @@ +//! Event-time lagged association standardization. +//! +//! This module belongs to the Longitudinal Modeling bounded context. It +//! standardizes a lagged covariance only when both marginal variances are +//! available. A one-sided covariance/initial-variance ratio is deliberately +//! not exposed as an autocorrelation. + +use crate::{EventTimeInterval, LongitudinalError}; + +/// Decompose a positive finite binary64 value into an exact integer +/// significand and a power-of-two exponent. +fn positive_binary_components(value: f64) -> (u64, i32) { + let bits = value.to_bits(); + let exponent_bits = ((bits >> 52) & 0x7ff) as i32; + let fraction = bits & ((1_u64 << 52) - 1); + if exponent_bits == 0 { + (fraction, -1074) + } else { + ((1_u64 << 52) | fraction, exponent_bits - 1023 - 52) + } +} + +/// Compare two positive `u128 * 2^exponent` values without overflowing the +/// integer significands. +fn scaled_integer_leq( + left_significand: u128, + left_exponent: i32, + right_significand: u128, + right_exponent: i32, +) -> bool { + let left_bits = (u128::BITS - left_significand.leading_zeros()).cast_signed(); + let right_bits = (u128::BITS - right_significand.leading_zeros()).cast_signed(); + let left_order = left_exponent + left_bits; + let right_order = right_exponent + right_bits; + if left_order != right_order { + return left_order < right_order; + } + if left_exponent == right_exponent { + return left_significand <= right_significand; + } + if left_exponent > right_exponent { + let shift = (left_exponent - right_exponent).cast_unsigned(); + return (left_significand << shift) <= right_significand; + } + let shift = (right_exponent - left_exponent).cast_unsigned(); + left_significand <= (right_significand << shift) +} + +/// Return whether the supplied covariance is within and exactly on the +/// Cauchy–Schwarz boundary for the supplied binary64 marginals. +/// +/// Both answers are derived from exact integer significands and powers of two; +/// rounded square roots are not authoritative for scientific endpoint claims. +fn covariance_binary_bound_relation( + lagged_covariance: f64, + earlier_total_variance: f64, + later_total_variance: f64, +) -> (bool, bool) { + let covariance_magnitude = lagged_covariance.abs(); + if covariance_magnitude == 0.0 { + return (true, false); + } + let (covariance_significand, covariance_exponent) = + positive_binary_components(covariance_magnitude); + let (earlier_significand, earlier_exponent) = + positive_binary_components(earlier_total_variance); + let (later_significand, later_exponent) = positive_binary_components(later_total_variance); + + let covariance_square = u128::from(covariance_significand) * u128::from(covariance_significand); + let variance_product = u128::from(earlier_significand) * u128::from(later_significand); + let covariance_square_exponent = covariance_exponent * 2; + let variance_product_exponent = earlier_exponent + later_exponent; + let within = scaled_integer_leq( + covariance_square, + covariance_square_exponent, + variance_product, + variance_product_exponent, + ); + let boundary = within + && scaled_integer_leq( + variance_product, + variance_product_exponent, + covariance_square, + covariance_square_exponent, + ); + (within, boundary) +} + +/// Recover a Pearson correlation for an event-time lag from its covariance and +/// both marginal variances. +/// +/// For observations at event times `t` and `t + Δ`, the correlation is +/// +/// `Cov(Y_t, Y_{t+Δ}) / sqrt(Var(Y_t) * Var(Y_{t+Δ}))`. +/// +/// Requiring both marginals is essential for nonstationary processes. Driver, +/// Oud, and Voelkle (2017) provide the continuous-time state-transition and +/// covariance components from which occasion-specific marginals can be built; +/// they do not justify replacing the second marginal variance with the first +/// when the process is nonstationary. +/// +/// [`EventTimeInterval`] makes substantive event-time ownership explicit. This +/// function does not infer either marginal variance and does not estimate a +/// state process. Exact binary64 Cauchy–Schwarz equality is authoritative for +/// a perfect-correlation endpoint; rounded square roots and divisions may +/// neither invent nor weaken that endpoint. +/// +/// # Errors +/// +/// Returns [`LongitudinalError::InvalidTemporalAssociationInput`] for +/// non-finite covariance or marginal inputs, when a nonzero exact correlation +/// is too small to be represented as binary64, or when a strict interior +/// covariance is rounded to a false exact ±1 correlation, +/// [`LongitudinalError::NonPositiveMarginalVariance`] when either marginal +/// variance is not strictly positive, and +/// [`LongitudinalError::CovarianceBoundViolation`] when the supplied covariance +/// is incompatible with the two marginal variances. +pub(crate) fn recover_event_time_lagged_correlation( + lagged_covariance: f64, + earlier_total_variance: f64, + later_total_variance: f64, + _event_interval: EventTimeInterval, +) -> Result { + if !lagged_covariance.is_finite() + || !earlier_total_variance.is_finite() + || !later_total_variance.is_finite() + { + return Err(LongitudinalError::InvalidTemporalAssociationInput); + } + if earlier_total_variance <= 0.0 || later_total_variance <= 0.0 { + return Err(LongitudinalError::NonPositiveMarginalVariance); + } + let (within_bound, on_exact_bound) = covariance_binary_bound_relation( + lagged_covariance, + earlier_total_variance, + later_total_variance, + ); + if !within_bound { + return Err(LongitudinalError::CovarianceBoundViolation); + } + + // The exact represented relation is authoritative at the scientific + // endpoint. For example, covariance=4 with marginals 2 and 8 lies exactly + // on the boundary, while rounded sqrt/division arithmetic evaluates to the + // predecessor of 1.0. Returning that rounded interior value would weaken a + // perfect-correlation claim that is exact in the supplied binary64 data. + if on_exact_bound { + return Ok(if lagged_covariance.is_sign_negative() { + -1.0 + } else { + 1.0 + }); + } + + let earlier_scale = earlier_total_variance.sqrt(); + let later_scale = later_total_variance.sqrt(); + // Divide by the smaller scale first. The exact covariance-bound gate above + // guarantees the intermediate magnitude cannot exceed the remaining + // marginal scale, while this order avoids underflow when the marginals are + // separated by hundreds of binary exponents. + let (first_scale, second_scale) = if earlier_scale <= later_scale { + (earlier_scale, later_scale) + } else { + (later_scale, earlier_scale) + }; + let correlation = (lagged_covariance / first_scale) / second_scale; + + // A nonzero covariance cannot scientifically become an exact zero + // correlation merely because binary64 cannot represent the standardized + // magnitude. Fail closed rather than report a false no-association result. + if lagged_covariance != 0.0 && correlation == 0.0 { + return Err(LongitudinalError::InvalidTemporalAssociationInput); + } + + // Exact boundary cases returned above. Any remaining rounded ±1 endpoint + // necessarily strengthens a strict-interior covariance and must fail + // closed rather than be clamped into a perfect-correlation claim. + if correlation.abs() >= 1.0 { + return Err(LongitudinalError::InvalidTemporalAssociationInput); + } + + Ok(correlation) +} + +#[cfg(test)] +mod tests { + use super::{recover_event_time_lagged_correlation, scaled_integer_leq}; + use crate::{EventTimeInterval, LongitudinalError}; + + fn event_time(value: f64) -> EventTimeInterval { + EventTimeInterval::new(value).expect("test interval must be valid event time") + } + + #[test] + fn nonstationary_marginals_do_not_use_the_earlier_variance_twice() { + let recovered = recover_event_time_lagged_correlation(1.5, 1.0, 4.0, event_time(1.0)) + .expect("valid nonstationary covariance should standardize"); + assert!((recovered - 0.75).abs() < f64::EPSILON * 4.0); + } + + #[test] + fn covariance_bound_is_fail_closed() { + assert_eq!( + recover_event_time_lagged_correlation(2.000_000_000_1, 1.0, 4.0, event_time(1.0)), + Err(LongitudinalError::CovarianceBoundViolation) + ); + assert_eq!( + recover_event_time_lagged_correlation(-2.000_000_000_1, 1.0, 4.0, event_time(1.0)), + Err(LongitudinalError::CovarianceBoundViolation) + ); + } + + #[test] + fn exact_binary_bound_rejects_one_ulp_excess_for_both_signs() { + let variance = 2.0_f64; + let one_ulp_above = f64::from_bits(variance.to_bits() + 1); + assert_eq!( + recover_event_time_lagged_correlation( + one_ulp_above, + variance, + variance, + event_time(1.0), + ), + Err(LongitudinalError::CovarianceBoundViolation) + ); + assert_eq!( + recover_event_time_lagged_correlation( + -one_ulp_above, + variance, + variance, + event_time(1.0), + ), + Err(LongitudinalError::CovarianceBoundViolation) + ); + } + + #[test] + fn exact_binary_bound_accepts_extreme_and_subnormal_boundaries() { + assert_eq!( + recover_event_time_lagged_correlation(f64::MAX, f64::MAX, f64::MAX, event_time(1.0),), + Ok(1.0) + ); + assert_eq!( + recover_event_time_lagged_correlation(-f64::MAX, f64::MAX, f64::MAX, event_time(1.0),), + Ok(-1.0) + ); + let minimum_subnormal = f64::from_bits(1); + assert_eq!( + recover_event_time_lagged_correlation( + minimum_subnormal, + minimum_subnormal, + minimum_subnormal, + event_time(1.0), + ), + Ok(1.0) + ); + assert_eq!( + recover_event_time_lagged_correlation( + -minimum_subnormal, + minimum_subnormal, + minimum_subnormal, + event_time(1.0), + ), + Ok(-1.0) + ); + } + + #[test] + fn nonzero_covariance_cannot_collapse_to_exact_zero_correlation() { + let minimum_subnormal = f64::from_bits(1); + assert_eq!( + recover_event_time_lagged_correlation( + minimum_subnormal, + f64::MAX, + f64::MAX, + event_time(1.0), + ), + Err(LongitudinalError::InvalidTemporalAssociationInput) + ); + } + + #[test] + fn gross_subnormal_bound_violation_is_classified_before_division() { + let minimum_subnormal = f64::from_bits(1); + assert_eq!( + recover_event_time_lagged_correlation( + 1.0, + minimum_subnormal, + minimum_subnormal, + event_time(1.0), + ), + Err(LongitudinalError::CovarianceBoundViolation) + ); + } + + #[test] + fn unequal_scales_do_not_underflow_a_representable_correlation() { + let recovered = recover_event_time_lagged_correlation( + f64::MIN_POSITIVE, + f64::MIN_POSITIVE, + f64::MAX, + event_time(1.0), + ) + .expect("valid unequal-scale covariance should remain representable"); + assert!(recovered > 0.0); + assert!(recovered.is_finite()); + } + + #[test] + fn scale_order_is_symmetric_when_the_earlier_marginal_is_larger() { + let recovered = recover_event_time_lagged_correlation(1.5, 4.0, 1.0, event_time(1.0)) + .expect("reversing marginal scale order should still standardize"); + assert!((recovered - 0.75).abs() < f64::EPSILON * 4.0); + } + + #[test] + fn zero_covariance_is_valid() { + assert_eq!( + recover_event_time_lagged_correlation(0.0, 1.0, 4.0, event_time(1.0)), + Ok(0.0) + ); + } + + #[test] + fn scaled_integer_comparison_covers_alignment_directions() { + assert!(scaled_integer_leq(1, 2, 4, 0)); + assert!(!scaled_integer_leq(5, 0, 1, 2)); + assert!(scaled_integer_leq(3, 0, 6, -1)); + assert!(scaled_integer_leq(6, -1, 3, 0)); + assert!(!scaled_integer_leq(7, -1, 3, 0)); + } + + #[test] + fn exact_boundary_correlations_are_allowed() { + assert_eq!( + recover_event_time_lagged_correlation(2.0, 1.0, 4.0, event_time(1.0)), + Ok(1.0) + ); + assert_eq!( + recover_event_time_lagged_correlation(-2.0, 1.0, 4.0, event_time(1.0)), + Ok(-1.0) + ); + } + + #[test] + fn square_root_scaling_avoids_avoidable_variance_product_overflow() { + let variance = f64::MAX / 4.0; + let recovered = recover_event_time_lagged_correlation( + variance / 2.0, + variance, + variance, + event_time(1.0), + ) + .expect("representable standardized covariance should remain representable"); + assert!((recovered - 0.5).abs() < 1.0e-15); + } + + #[test] + fn every_non_finite_input_position_fails_closed() { + assert_eq!( + recover_event_time_lagged_correlation(f64::NAN, 1.0, 1.0, event_time(1.0)), + Err(LongitudinalError::InvalidTemporalAssociationInput) + ); + assert_eq!( + recover_event_time_lagged_correlation(0.0, f64::INFINITY, 1.0, event_time(1.0)), + Err(LongitudinalError::InvalidTemporalAssociationInput) + ); + assert_eq!( + recover_event_time_lagged_correlation(0.0, 1.0, f64::INFINITY, event_time(1.0)), + Err(LongitudinalError::InvalidTemporalAssociationInput) + ); + } + + #[test] + fn either_non_positive_marginal_fails_closed() { + assert_eq!( + recover_event_time_lagged_correlation(0.0, 0.0, 1.0, event_time(1.0)), + Err(LongitudinalError::NonPositiveMarginalVariance) + ); + assert_eq!( + recover_event_time_lagged_correlation(0.0, 1.0, 0.0, event_time(1.0)), + Err(LongitudinalError::NonPositiveMarginalVariance) + ); + } +} diff --git a/crates/longitudinal_core/src/component.rs b/crates/longitudinal_core/src/component.rs index 846487af6..193689174 100644 --- a/crates/longitudinal_core/src/component.rs +++ b/crates/longitudinal_core/src/component.rs @@ -1,5 +1,7 @@ //! Known-truth RMSE for within/between components. +use std::collections::HashMap; + use crate::{ComponentLevel, LongitudinalError}; /// One unit-specific within or between component. @@ -14,8 +16,9 @@ pub struct ComponentValue { impl ComponentValue { /// Construct a component record from its identity fields and raw value. /// - /// The value is stored exactly as given, including non-finite values; - /// this constructor performs no validation. + /// The value and identity fields are stored exactly as given; this + /// constructor performs no validation. Recovery admission enforces that a + /// stable between-unit component uses canonical occasion index `0`. #[must_use] pub const fn new( unit_index: u32, @@ -56,16 +59,65 @@ impl ComponentValue { } } +fn validated_component_identity( + row: &ComponentValue, +) -> Result<(u32, u32, &'static str), LongitudinalError> { + if row.level() == ComponentLevel::Between && row.occasion_index() != 0 { + return Err(LongitudinalError::InvalidComponentPayload); + } + Ok(( + row.unit_index(), + row.occasion_index(), + row.level().wire_name(), + )) +} + +fn add_scaled_square( + scale: &mut f64, + scaled_sum_squares: &mut f64, + residual_scale: f64, + residual_ratio: f64, +) { + if residual_scale == 0.0 || residual_ratio == 0.0 { + return; + } + if *scale == 0.0 { + *scale = residual_scale; + *scaled_sum_squares = residual_ratio * residual_ratio; + } else if residual_scale > *scale { + let ratio = *scale / residual_scale; + *scaled_sum_squares = *scaled_sum_squares * ratio * ratio + residual_ratio * residual_ratio; + *scale = residual_scale; + } else { + let ratio = residual_scale / *scale; + let normalized = ratio * residual_ratio; + *scaled_sum_squares += normalized * normalized; + } +} + /// RMSE of recovered components against known-truth components. /// -/// The sum of squared residuals is accumulated with max-magnitude scaling so -/// large finite residuals cannot overflow to infinity. +/// Direct finite residuals keep their own magnitude so small recovery errors +/// are not erased merely because an unrelated matched component is extreme. +/// If one finite endpoint subtraction overflows, that residual alone is +/// represented as `endpoint_scale × normalized_difference`, where the latter +/// is bounded by two. The root-mean-square accumulator then rescales those +/// representations without ever materializing a non-representable residual. +/// Stable between-unit components have one unit-level scientific identity and +/// therefore use canonical occasion index `0`; within-unit components retain +/// their actual `(unit, occasion)` identity. Every admitted identity may +/// contribute exactly once. Duplicate or aliased identities would silently +/// change the recovery denominator and therefore fail closed. Slice order is +/// not scientific identity: truth and recovered rows are aligned by the +/// admitted identity tuple and accumulated in canonical tuple order so row +/// permutations cannot change the deterministic `f64` reference. /// /// # Errors /// /// Returns [`LongitudinalError::InvalidComponentPayload`] when either slice is -/// empty, the lengths differ, a unit/occasion/level identity mismatches, a -/// value or a computed residual is non-finite. +/// empty, the lengths differ, a stable between-unit component uses a nonzero +/// occasion index, an identity is missing or duplicated, an input value is +/// non-finite, or the final RMSE is not representable. pub fn component_root_mean_square_error( truth: &[ComponentValue], decided: &[ComponentValue], @@ -73,37 +125,70 @@ pub fn component_root_mean_square_error( if truth.is_empty() || truth.len() != decided.len() { return Err(LongitudinalError::InvalidComponentPayload); } - let mut scale = 0.0_f64; - let mut scaled_sum_squares = 0.0_f64; - for (truth_row, decided_row) in truth.iter().zip(decided) { - if truth_row.unit_index() != decided_row.unit_index() - || truth_row.occasion_index() != decided_row.occasion_index() - || truth_row.level() != decided_row.level() - || !truth_row.value().is_finite() - || !decided_row.value().is_finite() - { + + let mut truth_by_identity = HashMap::with_capacity(truth.len()); + for truth_row in truth { + if !truth_row.value().is_finite() { return Err(LongitudinalError::InvalidComponentPayload); } - let residual = decided_row.value() - truth_row.value(); - if !residual.is_finite() { + let identity = validated_component_identity(truth_row)?; + if truth_by_identity.insert(identity, truth_row).is_some() { return Err(LongitudinalError::InvalidComponentPayload); } - let magnitude = residual.abs(); - if magnitude > scale { - let ratio = scale / magnitude; - scaled_sum_squares = 1.0 + scaled_sum_squares * ratio * ratio; - scale = magnitude; - } else if scale > 0.0 { - let ratio = magnitude / scale; - scaled_sum_squares += ratio * ratio; + } + + let mut decided_by_identity = HashMap::with_capacity(decided.len()); + for decided_row in decided { + if !decided_row.value().is_finite() { + return Err(LongitudinalError::InvalidComponentPayload); } + let identity = validated_component_identity(decided_row)?; + if decided_by_identity.insert(identity, decided_row).is_some() { + return Err(LongitudinalError::InvalidComponentPayload); + } + } + + let mut identities: Vec<_> = truth_by_identity.keys().copied().collect(); + identities.sort_unstable(); + + let mut scale = 0.0_f64; + let mut scaled_sum_squares = 0.0_f64; + for identity in identities { + let truth_row = truth_by_identity[&identity]; + let Some(decided_row) = decided_by_identity.get(&identity).copied() else { + return Err(LongitudinalError::InvalidComponentPayload); + }; + + let residual = decided_row.value() - truth_row.value(); + if residual.is_finite() { + add_scaled_square(&mut scale, &mut scaled_sum_squares, residual.abs(), 1.0); + } else { + let endpoint_scale = truth_row.value().abs().max(decided_row.value().abs()); + let normalized_residual = + decided_row.value() / endpoint_scale - truth_row.value() / endpoint_scale; + add_scaled_square( + &mut scale, + &mut scaled_sum_squares, + endpoint_scale, + normalized_residual.abs(), + ); + } + } + + if scale == 0.0 { + return Ok(0.0); + } + let rmse = scale * (scaled_sum_squares / truth.len() as f64).sqrt(); + if rmse.is_finite() && rmse != 0.0 { + Ok(rmse) + } else { + Err(LongitudinalError::InvalidComponentPayload) } - Ok(scale * (scaled_sum_squares / truth.len() as f64).sqrt()) } #[cfg(test)] mod tests { - use super::{ComponentValue, component_root_mean_square_error}; + use super::{ComponentValue, add_scaled_square, component_root_mean_square_error}; use crate::{ComponentLevel, LongitudinalError}; #[test] @@ -129,6 +214,41 @@ mod tests { assert!((got - expected).abs() <= expected * 4.0 * f64::EPSILON); } + #[test] + fn representable_rmse_survives_an_overflowing_individual_residual() { + let truth = [ + ComponentValue::new(0, 0, ComponentLevel::Between, -f64::MAX), + ComponentValue::new(1, 0, ComponentLevel::Between, 0.0), + ComponentValue::new(2, 0, ComponentLevel::Between, 0.0), + ComponentValue::new(3, 0, ComponentLevel::Between, 0.0), + ]; + let decided = [ + ComponentValue::new(0, 0, ComponentLevel::Between, f64::MAX), + ComponentValue::new(1, 0, ComponentLevel::Between, 0.0), + ComponentValue::new(2, 0, ComponentLevel::Between, 0.0), + ComponentValue::new(3, 0, ComponentLevel::Between, 0.0), + ]; + assert_eq!( + component_root_mean_square_error(&truth, &decided), + Ok(f64::MAX) + ); + } + + #[test] + fn finite_residual_precision_is_not_lost_to_unrelated_endpoint_scale() { + let truth = [ + ComponentValue::new(0, 0, ComponentLevel::Between, f64::MAX), + ComponentValue::new(1, 0, ComponentLevel::Between, 0.0), + ]; + let decided = [ + ComponentValue::new(0, 0, ComponentLevel::Between, f64::MAX), + ComponentValue::new(1, 0, ComponentLevel::Between, 1.0), + ]; + let expected = 1.0 / f64::sqrt(2.0); + let got = component_root_mean_square_error(&truth, &decided).expect("finite residual"); + assert!((got - expected).abs() <= expected * 4.0 * f64::EPSILON); + } + #[test] fn overflowing_residual_fails_closed() { let truth = [ComponentValue::new(0, 0, ComponentLevel::Within, -f64::MAX)]; @@ -182,4 +302,29 @@ mod tests { 3 ); } + + #[test] + fn duplicate_decided_identity_fails_closed() { + let truth = [ + ComponentValue::new(0, 0, ComponentLevel::Between, 0.5), + ComponentValue::new(1, 0, ComponentLevel::Between, 0.25), + ]; + let decided = [ + ComponentValue::new(0, 0, ComponentLevel::Between, 0.5), + ComponentValue::new(0, 0, ComponentLevel::Between, 0.4), + ]; + assert_eq!( + component_root_mean_square_error(&truth, &decided), + Err(LongitudinalError::InvalidComponentPayload) + ); + } + + #[test] + fn zero_ratio_scaled_square_is_a_no_op() { + let mut scale = 2.0_f64; + let mut sum = 3.0_f64; + add_scaled_square(&mut scale, &mut sum, 4.0, 0.0); + assert_eq!(scale.to_bits(), 2.0_f64.to_bits()); + assert_eq!(sum.to_bits(), 3.0_f64.to_bits()); + } } diff --git a/crates/longitudinal_core/src/decompose.rs b/crates/longitudinal_core/src/decompose.rs index 0a24ca650..b3c020b8b 100644 --- a/crates/longitudinal_core/src/decompose.rs +++ b/crates/longitudinal_core/src/decompose.rs @@ -2,6 +2,7 @@ use std::collections::HashSet; +use crate::irregular_residual::scaled_compensated_mean; use crate::{ComponentLevel, ComponentValue, LongitudinalError}; /// One occasion score for one unit. @@ -42,17 +43,29 @@ impl OccasionObservation { } } +fn stable_unit_mean(rows: &[OccasionObservation]) -> Result { + let values: Vec = rows.iter().map(|row| row.score()).collect(); + scaled_compensated_mean(&values).map_err(|_| LongitudinalError::InvalidObservationPayload) +} + /// Decompose occasion scores into unit means and within residuals. /// /// Each unit contributes one between component at occasion `0` and one within /// residual per observed occasion. Units and occasions are emitted in sorted -/// order so recovery tests can pair known truth without extra matching. +/// order so recovery tests can pair known truth without extra matching. Unit +/// means use the same Longitudinal-local overflow-safe, cancellation-safe, +/// halfway-rounding numerical authority as CWC and occasion means, so +/// decomposition does not maintain a shadow averaging algorithm. Exact zero +/// within residuals use canonical public `+0.0` because a zero deviation has no +/// directional measurement meaning; signed zero remains available to private +/// numerical intermediates where it can carry diagnostic information. /// /// # Errors /// /// Returns [`LongitudinalError::InvalidObservationPayload`] when fewer than two /// units are present, any unit has fewer than two occasions, a `(unit, -/// occasion)` pair is duplicated, or a score is non-finite. +/// occasion)` pair is duplicated, a score is non-finite, or a resulting mean +/// or within residual is not representable. pub fn decompose_within_between( observations: &[OccasionObservation], ) -> Result, LongitudinalError> { @@ -93,21 +106,19 @@ pub fn decompose_within_between( if count < 2 { return Err(LongitudinalError::InvalidObservationPayload); } - let mut total = 0.0_f64; - for row in &rows[start..end] { - total += row.score(); - } - let mean = total / count as f64; - if !mean.is_finite() { - return Err(LongitudinalError::InvalidObservationPayload); - } + let mean = stable_unit_mean(&rows[start..end])?; components.push(ComponentValue::new(unit, 0, ComponentLevel::Between, mean)); for row in &rows[start..end] { + let residual = row.score() - mean; + if !residual.is_finite() { + return Err(LongitudinalError::InvalidObservationPayload); + } + let public_residual = if residual == 0.0 { 0.0 } else { residual }; components.push(ComponentValue::new( unit, row.occasion_index(), ComponentLevel::Within, - row.score() - mean, + public_residual, )); } } @@ -170,16 +181,6 @@ mod tests { decompose_within_between(&nan), Err(LongitudinalError::InvalidObservationPayload) ); - let overflow = [ - OccasionObservation::new(0, 0, f64::MAX), - OccasionObservation::new(0, 1, f64::MAX), - OccasionObservation::new(1, 0, 0.0), - OccasionObservation::new(1, 1, 0.0), - ]; - assert_eq!( - decompose_within_between(&overflow), - Err(LongitudinalError::InvalidObservationPayload) - ); let recovered = decompose_within_between(&[ OccasionObservation::new(1, 1, 4.0), OccasionObservation::new(0, 1, 2.0), @@ -192,4 +193,52 @@ mod tests { assert!((recovered[0].value() - 1.0).abs() < f64::EPSILON); assert_eq!(OccasionObservation::new(9, 8, 0.0).unit_index(), 9); } + + #[test] + fn representable_unit_mean_survives_raw_sum_overflow() { + let recovered = decompose_within_between(&[ + OccasionObservation::new(0, 0, f64::MAX), + OccasionObservation::new(0, 1, f64::MAX), + OccasionObservation::new(1, 0, 0.0), + OccasionObservation::new(1, 1, 0.0), + ]) + .expect("representable unit mean must not fail on an overflowing partial sum"); + + assert_eq!(recovered[0].level(), ComponentLevel::Between); + assert_eq!(recovered[0].value().to_bits(), f64::MAX.to_bits()); + assert_eq!(recovered[1].value().to_bits(), 0.0_f64.to_bits()); + assert_eq!(recovered[2].value().to_bits(), 0.0_f64.to_bits()); + } + + #[test] + fn representable_subnormal_unit_mean_survives_extreme_cancellation() { + let minimum_subnormal = f64::from_bits(1); + let recovered = decompose_within_between(&[ + OccasionObservation::new(0, 0, f64::MAX), + OccasionObservation::new(0, 1, -f64::MAX), + OccasionObservation::new(0, 2, f64::from_bits(4)), + OccasionObservation::new(1, 0, 0.0), + OccasionObservation::new(1, 1, 0.0), + ]) + .expect("a representable subnormal unit mean must survive extreme cancellation"); + + assert_eq!(recovered[0].level(), ComponentLevel::Between); + assert_eq!(recovered[0].value().to_bits(), minimum_subnormal.to_bits()); + } + + #[test] + fn overflowing_within_residual_after_finite_mean_fails_closed() { + // Need >=2 units for admission. Unit 0 mean is MAX/3; -MAX - MAX/3 overflows. + let rows = [ + OccasionObservation::new(0, 0, f64::MAX), + OccasionObservation::new(0, 1, f64::MAX), + OccasionObservation::new(0, 2, -f64::MAX), + OccasionObservation::new(1, 0, 1.0), + OccasionObservation::new(1, 1, 0.5), + ]; + assert_eq!( + decompose_within_between(&rows), + Err(LongitudinalError::InvalidObservationPayload) + ); + } } diff --git a/crates/longitudinal_core/src/diffusion.rs b/crates/longitudinal_core/src/diffusion.rs new file mode 100644 index 000000000..59de0c07d --- /dev/null +++ b/crates/longitudinal_core/src/diffusion.rs @@ -0,0 +1,254 @@ +//! Scalar diffusion standardisation on substantive event time. +//! +//! These functions preserve the scientific evidence from TEPP PRs #476 and +//! #477 while moving temporal/state composition out of `psychometric_core`. +//! Driver, Oud, and Voelkle (2017) print the underlying continuous/discrete +//! diffusion transformations and describe relevant-variance standardisation, +//! but the 2017 ctsem summary source does not emit named `DIFFUSIONstd` or +//! `discreteDIFFUSIONstd` matrices. The scalar maps below are therefore +//! research-candidate extensions, not canonical ctsem output and not a DSEM or +//! ctsem estimator. + +use crate::{EventTimeInterval, LongitudinalError, stationary::validate_stationary_process_inputs}; + +/// Recover the scalar research-candidate `DIFFUSIONstd = q / p` map. +/// +/// `q` is the continuous diffusion variance-rate input and `p` is the strictly +/// positive stationary within-person variance `-q/(2a)`. For every finite +/// positive `q` and stable finite `a`, that real-valued variance exists and the +/// scalar standardisation cancels it exactly to `-2a`. The implementation +/// therefore validates the stationary process without materializing `p` as +/// binary64; an intermediate `p` below or above the `f64` range must not erase +/// a representable final standardized ratio. Equal numeric values still do not +/// collapse this estimand into `asymDIFFUSIONstd` or another variance +/// standardisation. +/// +/// # Errors +/// +/// Returns [`LongitudinalError::InvalidTemporalTransformInput`] for non-finite +/// inputs, negative diffusion, or a non-representable final `-2a` ratio. Returns +/// [`LongitudinalError::StationaryVarianceRequiresStableDrift`] unless `a < 0`. +/// Returns [`LongitudinalError::StandardisedDiffusionRequiresPositiveWithinVariance`] +/// when continuous diffusion is exactly zero, because the stationary variance +/// is then zero rather than merely outside the binary64 range. +pub fn recover_event_time_standardised_continuous_diffusion( + continuous_diffusion: f64, + log_rate: f64, +) -> Result { + validate_stationary_process_inputs(continuous_diffusion, log_rate)?; + if continuous_diffusion == 0.0 { + return Err(LongitudinalError::StandardisedDiffusionRequiresPositiveWithinVariance); + } + + // Algebraically q / (-q/(2a)) == -2a for every positive q. The stationary + // process admission above establishes that cancellation in the real-valued + // model without requiring the cancelled p itself to fit in binary64. + let ratio = -2.0 * log_rate; + if !ratio.is_finite() { + return Err(LongitudinalError::InvalidTemporalTransformInput); + } + Ok(ratio) +} + +/// Recover the scalar research-candidate `discreteDIFFUSIONstd = Q_delta / p` map. +/// +/// For a stable scalar continuous-time process, dividing discrete process noise +/// over an event interval by stationary within-person variance yields +/// `1 - exp(2 a delta)`. The implementation evaluates this ratio directly with +/// `exp_m1` after proving positive stationary variance algebraically from finite +/// positive diffusion and stable drift. It does not materialize the cancelled +/// stationary variance, so a below-range or above-range `p` cannot reject a +/// representable final ratio. The [`EventTimeInterval`] value object prevents +/// measurement occasion, document, assertion, system, or availability durations +/// from being passed as event time accidentally. +/// +/// # Errors +/// +/// Returns [`LongitudinalError::InvalidTemporalTransformInput`] for invalid +/// diffusion/drift, a doubled event-time exponent that underflows to signed +/// zero or overflows, or a finite-interval final ratio that collapses to the +/// false binary64 endpoints zero or one. Returns +/// [`LongitudinalError::StationaryVarianceRequiresStableDrift`] unless `a < 0`. +/// Returns [`LongitudinalError::StandardisedDiffusionRequiresPositiveWithinVariance`] +/// when continuous diffusion is exactly zero. +pub fn recover_event_time_standardised_discrete_diffusion( + continuous_diffusion: f64, + log_rate: f64, + event_interval: EventTimeInterval, +) -> Result { + validate_stationary_process_inputs(continuous_diffusion, log_rate)?; + if continuous_diffusion == 0.0 { + return Err(LongitudinalError::StandardisedDiffusionRequiresPositiveWithinVariance); + } + + let interval = event_interval.as_f64(); + let doubled_interval = interval * 2.0; + let exponent = if doubled_interval.is_finite() { + // Multiplication by two is exact while finite. Scaling the interval + // before the only rounded product preserves a representable `2aΔ` + // when `aΔ` alone would round to signed zero, and it avoids forming + // `2a`, whose intermediate can overflow for an extreme stable drift. + log_rate * doubled_interval + } else { + // If 2Δ overflows, Δ is already enormous. Form aΔ first and then apply + // the exact factor two; this branch cannot suffer the tiny-interval + // underflow that motivated the primary ordering above. A non-finite + // product cannot be reported as the exact finite-interval endpoint one. + let half_exponent = log_rate * interval; + if !half_exponent.is_finite() { + return Err(LongitudinalError::InvalidTemporalTransformInput); + } + half_exponent * 2.0 + }; + + // For finite stable a and finite positive Δ, the exact 2aΔ is finite and + // strictly negative. Signed zero or infinity therefore means the target + // exponent itself is not representable in binary64 and must fail closed. + if !exponent.is_finite() || exponent == 0.0 { + return Err(LongitudinalError::InvalidTemporalTransformInput); + } + + let ratio = -exponent.exp_m1(); + // Stable a < 0 with positive Δ yields a strictly negative exponent, so + // -exp_m1 is in (0, 1] when finite. The only false scientific boundary that + // remains is rounding up to the exact unit endpoint. + if ratio >= 1.0 { + return Err(LongitudinalError::InvalidTemporalTransformInput); + } + Ok(ratio) +} + +/// Refuse treating a continuous standardised diffusion ratio as a discrete one. +/// +/// `q/p` is interval-independent while `Q_delta/p` is an event-interval +/// quantity. Numerical equality at a particular parameter value does not make +/// the named estimands interchangeable. +/// +/// # Errors +/// +/// Always returns [`LongitudinalError::ContinuousDiffusionIsNotDiscreteDiffusion`]. +pub fn refuse_standardised_continuous_diffusion_as_standardised_discrete_diffusion( + continuous_standardised_diffusion: f64, + discrete_standardised_diffusion: f64, +) -> Result { + let _ = ( + continuous_standardised_diffusion, + discrete_standardised_diffusion, + ); + Err(LongitudinalError::ContinuousDiffusionIsNotDiscreteDiffusion) +} + +/// Refuse treating an unstandardised diffusion quantity as a standardised one. +/// +/// # Errors +/// +/// Always returns [`LongitudinalError::UnstandardisedDiffusionIsNotStandardisedDiffusion`]. +pub fn refuse_unstandardised_diffusion_as_standardised_diffusion( + unstandardised_diffusion: f64, + standardised_diffusion: f64, +) -> Result { + let _ = (unstandardised_diffusion, standardised_diffusion); + Err(LongitudinalError::UnstandardisedDiffusionIsNotStandardisedDiffusion) +} + +/// Refuse scaling diffusion by total trait-plus-state variance as standardisation. +/// +/// Driver et al.'s relevant-variance rule for these scalar research candidates +/// uses within-person stationary variance, not a total that also contains +/// between-unit trait or added time-independent-predictor variance. +/// +/// # Errors +/// +/// Always returns [`LongitudinalError::TotalVarianceScaledDiffusionIsNotStandardisedDiffusion`]. +pub fn refuse_total_variance_scaled_diffusion_as_standardised_diffusion( + total_variance_scaled_diffusion: f64, + standardised_diffusion: f64, +) -> Result { + let _ = (total_variance_scaled_diffusion, standardised_diffusion); + Err(LongitudinalError::TotalVarianceScaledDiffusionIsNotStandardisedDiffusion) +} + +#[cfg(test)] +mod tests { + use super::{ + recover_event_time_standardised_continuous_diffusion, + recover_event_time_standardised_discrete_diffusion, + }; + use crate::{EventTimeInterval, LongitudinalError}; + + #[test] + fn discrete_candidate_preserves_representable_minimum_subnormal_ratio() { + let tiny = EventTimeInterval::new(f64::from_bits(1)).expect("minimum subnormal interval"); + let recovered = recover_event_time_standardised_discrete_diffusion(1.0, -0.5, tiny) + .expect("the final standardized ratio is the minimum positive subnormal"); + assert_eq!(recovered.to_bits(), 1); + } + + #[test] + fn finite_event_interval_must_not_collapse_to_false_unit_diffusion() { + let huge = EventTimeInterval::new(f64::MAX).expect("finite positive interval"); + assert_eq!( + recover_event_time_standardised_discrete_diffusion(1.0, -1.0, huge), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + + let finite_saturating = EventTimeInterval::new(50.0).expect("finite event interval"); + assert_eq!( + recover_event_time_standardised_discrete_diffusion(1.0, -1.0, finite_saturating), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + } + + #[test] + fn standardised_diffusion_does_not_materialise_a_cancelled_stationary_variance() { + let minimum_subnormal = f64::from_bits(1); + let continuous_underflow = + recover_event_time_standardised_continuous_diffusion(minimum_subnormal, -1.0) + .expect("q/p cancels a positive real stationary variance below binary64 range"); + assert_eq!(continuous_underflow.to_bits(), 2.0_f64.to_bits()); + + let continuous_overflow = + recover_event_time_standardised_continuous_diffusion(f64::MAX, -0.25) + .expect("q/p cancels a positive real stationary variance above binary64 range"); + assert_eq!(continuous_overflow.to_bits(), 0.5_f64.to_bits()); + + let interval = EventTimeInterval::new(1.0).expect("unit event interval"); + let discrete_underflow = + recover_event_time_standardised_discrete_diffusion(minimum_subnormal, -1.0, interval) + .expect("Q_delta/p cancels the unrepresentable stationary intermediate"); + let underflow_truth = -(-2.0_f64).exp_m1(); + assert!((discrete_underflow - underflow_truth).abs() <= f64::EPSILON); + + let discrete_overflow = + recover_event_time_standardised_discrete_diffusion(f64::MAX, -0.25, interval) + .expect("finite standardized discrete diffusion must survive p overflow"); + let overflow_truth = -(-0.5_f64).exp_m1(); + assert!((discrete_overflow - overflow_truth).abs() <= f64::EPSILON); + } + + #[test] + fn continuous_candidate_rejects_nonrepresentable_ratio() { + assert_eq!( + recover_event_time_standardised_continuous_diffusion(f64::MAX, -f64::MAX), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + } + + #[test] + fn doubled_interval_overflow_rejects_nonfinite_half_exponent() { + let huge = EventTimeInterval::new(f64::MAX).expect("finite positive interval"); + assert_eq!( + recover_event_time_standardised_discrete_diffusion(1.0, -f64::MAX, huge), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + } + + #[test] + fn underflowed_exact_exponent_fails_closed() { + let tiny = EventTimeInterval::new(1e-300).expect("tiny interval"); + assert_eq!( + recover_event_time_standardised_discrete_diffusion(1.0, -1e-300, tiny), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + } +} diff --git a/crates/longitudinal_core/src/discrete_drift.rs b/crates/longitudinal_core/src/discrete_drift.rs new file mode 100644 index 000000000..4743d0161 --- /dev/null +++ b/crates/longitudinal_core/src/discrete_drift.rs @@ -0,0 +1,106 @@ +//! Event-time standardisation for scalar continuous-time drift. + +use crate::{EventTimeInterval, LongitudinalError, stationary::validate_stationary_process_inputs}; + +/// Recover the scalar p. 16 `discreteDRIFTstd` on event time. +/// +/// Driver, Oud, and Voelkle (2017) define the discrete-time transition over an +/// interval as `exp(A * delta_t)`. Their standardisation uses the relevant +/// within-person asymptotic variance. In the scalar stationary case the +/// affecting/affected standard-deviation ratio is one, so the standardised +/// auto-effect is numerically `exp(a * delta_t)` after stable negative drift +/// and positive continuous diffusion establish a positive real stationary +/// within-person variance. The cancelled stationary variance is not materialized +/// as binary64, because its representability does not constrain the final scalar +/// standardized map. Equal numerical values still do not make unstandardised +/// `discreteDRIFT` and `discreteDRIFTstd` the same estimand. +/// +/// This function is temporal composition, not a ctsem/DSEM estimator. It does +/// not estimate `a`, process noise, uncertainty, or a latent state. The caller +/// supplies the continuous diffusion intensity, stable scalar drift, and an +/// [`EventTimeInterval`] admitted by the Longitudinal Modeling bounded context. +/// +/// # Errors +/// +/// Returns [`LongitudinalError::InvalidTemporalTransformInput`] for non-finite +/// diffusion/drift inputs, negative diffusion, an `a * delta_t` product that +/// overflows or underflows to signed zero, or an exponential whose nonzero +/// change is not representable and therefore rounds to zero or one. Returns +/// [`LongitudinalError::StationaryVarianceRequiresStableDrift`] unless `a < 0`. +/// Returns [`LongitudinalError::StandardisedDriftRequiresPositiveWithinVariance`] +/// when continuous diffusion is exactly zero, because the stationary variance +/// is then zero rather than merely outside the binary64 range. +pub fn recover_event_time_standardised_discrete_drift( + continuous_diffusion: f64, + log_rate: f64, + event_interval: EventTimeInterval, +) -> Result { + validate_stationary_process_inputs(continuous_diffusion, log_rate)?; + if continuous_diffusion == 0.0 { + return Err(LongitudinalError::StandardisedDriftRequiresPositiveWithinVariance); + } + + let exponent = log_rate * event_interval.as_f64(); + // `log_rate` is strictly negative and EventTimeInterval is strictly + // positive, so an exact product cannot be zero. A signed zero therefore + // proves binary64 multiplication underflow and must fail closed instead of + // silently becoming exp(-0.0) == 1.0. + if !exponent.is_finite() || exponent == 0.0 { + return Err(LongitudinalError::InvalidTemporalTransformInput); + } + let discrete_drift = exponent.exp(); + // For every admitted finite interval and stable finite drift the exact + // transition lies strictly inside (0, 1). Returning either endpoint would + // erase a nonzero scientific effect solely because binary64 cannot express + // it, so both endpoint collapses fail closed. + if discrete_drift == 0.0 || discrete_drift.to_bits() == 1.0_f64.to_bits() { + return Err(LongitudinalError::InvalidTemporalTransformInput); + } + Ok(discrete_drift) +} + +/// Refuse treating unstandardised `discreteDRIFT` as `discreteDRIFTstd`. +/// +/// The scalar stationary values may coincide numerically while the named +/// quantities and their admissibility conditions remain distinct. +/// +/// # Errors +/// +/// Always returns [`LongitudinalError::UnstandardisedDriftIsNotStandardisedDrift`]. +pub fn refuse_unstandardised_discrete_drift_as_standardised_discrete_drift( + unstandardised_discrete_drift: f64, + standardised_discrete_drift: f64, +) -> Result { + let _ = (unstandardised_discrete_drift, standardised_discrete_drift); + Err(LongitudinalError::UnstandardisedDriftIsNotStandardisedDrift) +} + +/// Refuse treating a trait-plus-state lagged association as `discreteDRIFTstd`. +/// +/// A trait-plus-state association mixes stable between-unit variance with +/// within-person dynamics. Driver et al.'s drift standardisation uses the +/// relevant within-person variance instead. +/// +/// # Errors +/// +/// Always returns [`LongitudinalError::TraitStateAssociationIsNotStandardisedDrift`]. +pub fn refuse_trait_plus_state_association_as_standardised_discrete_drift( + trait_plus_state_association: f64, + standardised_discrete_drift: f64, +) -> Result { + let _ = (trait_plus_state_association, standardised_discrete_drift); + Err(LongitudinalError::TraitStateAssociationIsNotStandardisedDrift) +} + +/// Refuse using between-unit trait variance as the drift standardisation variance. +/// +/// # Errors +/// +/// Always returns [`LongitudinalError::TraitVarianceIsNotDriftStandardisationVariance`]. +pub fn refuse_trait_variance_as_standardisation_variance( + trait_variance: f64, + within_person_variance: f64, +) -> Result { + let _ = (trait_variance, within_person_variance); + Err(LongitudinalError::TraitVarianceIsNotDriftStandardisationVariance) +} diff --git a/crates/longitudinal_core/src/error.rs b/crates/longitudinal_core/src/error.rs index eb0dba3ab..3240db0f1 100644 --- a/crates/longitudinal_core/src/error.rs +++ b/crates/longitudinal_core/src/error.rs @@ -1,8 +1,8 @@ -//! Fail-closed longitudinal within/between errors. +//! Fail-closed longitudinal modeling errors. use std::fmt; -/// A fail-closed longitudinal-decomposition error. +/// A fail-closed longitudinal-modeling error. #[derive(Clone, Copy, Debug, Eq, PartialEq)] #[non_exhaustive] pub enum LongitudinalError { @@ -14,6 +14,38 @@ pub enum LongitudinalError { InvalidComponentPayload, /// Observations were empty, sparse, duplicated, or non-finite. InvalidObservationPayload, + /// A lagged covariance, marginal variance, or event interval was non-finite. + InvalidTemporalAssociationInput, + /// A temporal transform input or intermediate value was not representable. + InvalidTemporalTransformInput, + /// The requested irregular-rate estimand is typed but not yet activated. + IrregularRateEstimandUnavailable, + /// At least one marginal variance was not strictly positive. + NonPositiveMarginalVariance, + /// The event-time interval was not strictly positive. + NonPositiveEventInterval, + /// The covariance violates the Cauchy-Schwarz bound implied by the two marginals. + CovarianceBoundViolation, + /// A stationary within-person variance was requested without stable negative drift. + StationaryVarianceRequiresStableDrift, + /// A standardised diffusion candidate was requested without positive within variance. + StandardisedDiffusionRequiresPositiveWithinVariance, + /// `discreteDRIFTstd` was requested without positive stationary within-person variance. + StandardisedDriftRequiresPositiveWithinVariance, + /// Unstandardised `discreteDRIFT` was substituted for `discreteDRIFTstd`. + UnstandardisedDriftIsNotStandardisedDrift, + /// A trait-plus-state association was substituted for `discreteDRIFTstd`. + TraitStateAssociationIsNotStandardisedDrift, + /// Between-unit trait variance was used as the drift standardisation variance. + TraitVarianceIsNotDriftStandardisationVariance, + /// A continuous diffusion ratio was substituted for a discrete diffusion ratio. + ContinuousDiffusionIsNotDiscreteDiffusion, + /// An unstandardised diffusion quantity was substituted for a standardised one. + UnstandardisedDiffusionIsNotStandardisedDiffusion, + /// Diffusion was scaled by total variance instead of relevant within variance. + TotalVarianceScaledDiffusionIsNotStandardisedDiffusion, + /// Cluster-mean-centered residual log-rate was treated as raw-process AR drift. + CwcResidualLogRateIsNotRawProcessDrift, } impl fmt::Display for LongitudinalError { @@ -23,6 +55,50 @@ impl fmt::Display for LongitudinalError { Self::UnknownComponentLevel => "unknown component level", Self::InvalidComponentPayload => "invalid longitudinal component payload", Self::InvalidObservationPayload => "invalid longitudinal observation payload", + Self::InvalidTemporalAssociationInput => "invalid temporal association input", + Self::InvalidTemporalTransformInput => "invalid temporal transform input", + Self::IrregularRateEstimandUnavailable => { + "requested irregular-rate estimand is not yet available" + } + Self::NonPositiveMarginalVariance => { + "temporal correlation requires strictly positive marginal variances" + } + Self::NonPositiveEventInterval => { + "temporal composition requires a strictly positive event-time interval" + } + Self::CovarianceBoundViolation => { + "lagged covariance is incompatible with the supplied marginal variances" + } + Self::StationaryVarianceRequiresStableDrift => { + "stationary within-person variance requires strictly negative drift" + } + Self::StandardisedDiffusionRequiresPositiveWithinVariance => { + "standardised diffusion requires positive stationary within-person variance" + } + Self::StandardisedDriftRequiresPositiveWithinVariance => { + "standardised discrete drift requires positive stationary within-person variance" + } + Self::UnstandardisedDriftIsNotStandardisedDrift => { + "unstandardised discrete drift is not standardised discrete drift" + } + Self::TraitStateAssociationIsNotStandardisedDrift => { + "trait-plus-state association is not standardised discrete drift" + } + Self::TraitVarianceIsNotDriftStandardisationVariance => { + "trait variance is not the drift standardisation variance" + } + Self::ContinuousDiffusionIsNotDiscreteDiffusion => { + "continuous standardised diffusion is not discrete standardised diffusion" + } + Self::UnstandardisedDiffusionIsNotStandardisedDiffusion => { + "unstandardised diffusion is not standardised diffusion" + } + Self::TotalVarianceScaledDiffusionIsNotStandardisedDiffusion => { + "total-variance-scaled diffusion is not relevant-variance-standardised diffusion" + } + Self::CwcResidualLogRateIsNotRawProcessDrift => { + "cluster-mean-centered residual log-rate is not the raw-process autoregressive drift" + } }; formatter.write_str(message) } @@ -53,6 +129,70 @@ mod tests { LongitudinalError::InvalidObservationPayload, "invalid longitudinal observation payload", ), + ( + LongitudinalError::InvalidTemporalAssociationInput, + "invalid temporal association input", + ), + ( + LongitudinalError::InvalidTemporalTransformInput, + "invalid temporal transform input", + ), + ( + LongitudinalError::IrregularRateEstimandUnavailable, + "requested irregular-rate estimand is not yet available", + ), + ( + LongitudinalError::NonPositiveMarginalVariance, + "temporal correlation requires strictly positive marginal variances", + ), + ( + LongitudinalError::NonPositiveEventInterval, + "temporal composition requires a strictly positive event-time interval", + ), + ( + LongitudinalError::CovarianceBoundViolation, + "lagged covariance is incompatible with the supplied marginal variances", + ), + ( + LongitudinalError::StationaryVarianceRequiresStableDrift, + "stationary within-person variance requires strictly negative drift", + ), + ( + LongitudinalError::StandardisedDiffusionRequiresPositiveWithinVariance, + "standardised diffusion requires positive stationary within-person variance", + ), + ( + LongitudinalError::StandardisedDriftRequiresPositiveWithinVariance, + "standardised discrete drift requires positive stationary within-person variance", + ), + ( + LongitudinalError::UnstandardisedDriftIsNotStandardisedDrift, + "unstandardised discrete drift is not standardised discrete drift", + ), + ( + LongitudinalError::TraitStateAssociationIsNotStandardisedDrift, + "trait-plus-state association is not standardised discrete drift", + ), + ( + LongitudinalError::TraitVarianceIsNotDriftStandardisationVariance, + "trait variance is not the drift standardisation variance", + ), + ( + LongitudinalError::ContinuousDiffusionIsNotDiscreteDiffusion, + "continuous standardised diffusion is not discrete standardised diffusion", + ), + ( + LongitudinalError::UnstandardisedDiffusionIsNotStandardisedDiffusion, + "unstandardised diffusion is not standardised diffusion", + ), + ( + LongitudinalError::TotalVarianceScaledDiffusionIsNotStandardisedDiffusion, + "total-variance-scaled diffusion is not relevant-variance-standardised diffusion", + ), + ( + LongitudinalError::CwcResidualLogRateIsNotRawProcessDrift, + "cluster-mean-centered residual log-rate is not the raw-process autoregressive drift", + ), ] { assert_eq!(error.to_string(), message); } diff --git a/crates/longitudinal_core/src/event_time.rs b/crates/longitudinal_core/src/event_time.rs new file mode 100644 index 000000000..a4813fd4d --- /dev/null +++ b/crates/longitudinal_core/src/event_time.rs @@ -0,0 +1,64 @@ +//! Event-time value objects for longitudinal composition. + +use crate::LongitudinalError; + +/// A finite, strictly positive interval on TEPP's substantive event-time clock. +/// +/// Constructing this value is the admission boundary between generic numeric +/// durations and Longitudinal Modeling operations that are explicitly defined +/// on event time. Measurement-occasion, assertion, document, system, and +/// availability-clock durations must not be re-labelled as this type without +/// an owning-context conversion that proves the semantic mapping. +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct EventTimeInterval(f64); + +impl EventTimeInterval { + /// Admit a finite, strictly positive interval as substantive event time. + /// + /// # Errors + /// + /// Returns [`LongitudinalError::NonPositiveEventInterval`] when `value` is + /// non-finite, zero, or negative. + pub fn new(value: f64) -> Result { + if !value.is_finite() || value <= 0.0 { + return Err(LongitudinalError::NonPositiveEventInterval); + } + Ok(Self(value)) + } + + /// Return the admitted interval in the model's declared event-time unit. + #[must_use] + pub const fn as_f64(self) -> f64 { + self.0 + } +} + +#[cfg(test)] +mod tests { + use super::EventTimeInterval; + use crate::LongitudinalError; + + #[test] + fn admits_only_finite_positive_event_intervals() { + assert_eq!( + EventTimeInterval::new(0.0), + Err(LongitudinalError::NonPositiveEventInterval) + ); + assert_eq!( + EventTimeInterval::new(-1.0), + Err(LongitudinalError::NonPositiveEventInterval) + ); + assert_eq!( + EventTimeInterval::new(f64::NAN), + Err(LongitudinalError::NonPositiveEventInterval) + ); + assert_eq!( + EventTimeInterval::new(f64::INFINITY), + Err(LongitudinalError::NonPositiveEventInterval) + ); + assert_eq!( + EventTimeInterval::new(0.25).map(EventTimeInterval::as_f64), + Ok(0.25) + ); + } +} diff --git a/crates/longitudinal_core/src/irregular_rate_estimand.rs b/crates/longitudinal_core/src/irregular_rate_estimand.rs new file mode 100644 index 000000000..89d6a627f --- /dev/null +++ b/crates/longitudinal_core/src/irregular_rate_estimand.rs @@ -0,0 +1,213 @@ +//! Versioned weighting policy for CWC irregular-residual log-rate summaries. +//! +//! Longitudinal units can contribute different numbers of consecutive event-time +//! pairs. That multiplicity is part of the estimand, not a numerical detail. +//! This module makes the currently supported lag-pair-average target explicit +//! and reports its unit/pair denominators. Equal-unit aggregation stays typed +//! but fail-closed until the reusable correctly-rounded finite-mean contract is +//! available from its canonical numerical owner. + +use std::collections::BTreeMap; + +use crate::LongitudinalError; +use crate::irregular_residual::{ + EventTimedObservation, center_within_unit_event_lags, driver_same_sign_log_rate, + same_sign_nonzero, scaled_compensated_mean, +}; + +/// Versioned scientific weighting target for irregular residual log-rate summaries. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum IrregularRateEstimand { + /// Every admitted consecutive lag pair receives equal weight. + LagPairAverageV1, + /// Each unit first receives one within-unit summary and then equal unit weight. + /// + /// This target is reserved but not yet numerically activated. Requesting it + /// fails closed until TEPP can consume the released reusable finite-mean + /// contract instead of adding another local generic mean implementation. + UnitAverageV1, +} + +impl IrregularRateEstimand { + /// Stable external name of this estimand contract. + #[must_use] + pub const fn wire_name(self) -> &'static str { + match self { + Self::LagPairAverageV1 => "tepp.irregular_rate.lag_pair_average.v1", + Self::UnitAverageV1 => "tepp.irregular_rate.unit_average.v1", + } + } +} + +/// Denominator-bearing evidence for one irregular residual log-rate summary. +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct IrregularRateSummary { + estimand: IrregularRateEstimand, + estimate: Option, + candidate_units: usize, + contributing_units: usize, + candidate_pairs: usize, + admitted_pairs: usize, + sign_or_zero_refused_pairs: usize, + nonrepresentable_rate_refused_pairs: usize, +} + +impl IrregularRateSummary { + /// Return the versioned weighting target used by this summary. + #[must_use] + pub const fn estimand(self) -> IrregularRateEstimand { + self.estimand + } + + /// Return the represented estimate, or `None` when no candidate pair was admissible. + #[must_use] + pub const fn estimate(self) -> Option { + self.estimate + } + + /// Return the number of units with at least two admitted event-time occasions. + #[must_use] + pub const fn candidate_units(self) -> usize { + self.candidate_units + } + + /// Return the number of candidate units contributing at least one admitted rate. + #[must_use] + pub const fn contributing_units(self) -> usize { + self.contributing_units + } + + /// Return the number of consecutive event-time pairs considered. + #[must_use] + pub const fn candidate_pairs(self) -> usize { + self.candidate_pairs + } + + /// Return the number of pairs admitted to the reported numerical estimate. + #[must_use] + pub const fn admitted_pairs(self) -> usize { + self.admitted_pairs + } + + /// Return pairs refused because a residual was zero or the pair changed sign. + #[must_use] + pub const fn sign_or_zero_refused_pairs(self) -> usize { + self.sign_or_zero_refused_pairs + } + + /// Return same-sign nonzero pairs whose represented log-rate was not admissible. + #[must_use] + pub const fn nonrepresentable_rate_refused_pairs(self) -> usize { + self.nonrepresentable_rate_refused_pairs + } + + /// Return the total number of candidate pairs refused by log-rate admission. + #[must_use] + pub const fn refused_pairs(self) -> usize { + self.sign_or_zero_refused_pairs + self.nonrepresentable_rate_refused_pairs + } +} + +/// Recover an explicitly named irregular residual log-rate estimand with denominators. +/// +/// `LagPairAverageV1` preserves the existing TEPP behavior: every admissible +/// consecutive pair receives equal weight. The returned evidence makes the +/// weighting population observable by carrying candidate/contributing units and +/// candidate/admitted/refused pairs. Zero or opposite-sign pairs and represented +/// same-sign pairs whose log-rate cannot be admitted have separate refusal +/// denominators instead of silently changing the weighting population. +/// +/// `UnitAverageV1` is intentionally fail-closed. Equal-unit aggregation needs a +/// second finite-mean operation over within-unit summaries. Reusable finite mean +/// arithmetic is not owned by TEPP, so activation waits for an immutable released +/// owner contract and parity evidence. +/// +/// # Errors +/// +/// Propagates event-time/CWC admission failures. `UnitAverageV1` returns +/// [`LongitudinalError::IrregularRateEstimandUnavailable`] while the released +/// reusable mean contract is unavailable. The pair-average numerical mean can +/// return [`LongitudinalError::InvalidTemporalTransformInput`] if its currently +/// shared compatibility arithmetic cannot represent the final admitted-pair mean. +pub fn recover_within_unit_irregular_rate_summary( + rows: &[EventTimedObservation], + estimand: IrregularRateEstimand, +) -> Result { + if estimand == IrregularRateEstimand::UnitAverageV1 { + return Err(LongitudinalError::IrregularRateEstimandUnavailable); + } + + let lagged = center_within_unit_event_lags(rows)?; + let pair_counts = consecutive_pair_counts(rows); + let candidate_units = pair_counts.len(); + let candidate_pairs = lagged.len(); + let mut admitted_rates = Vec::with_capacity(candidate_pairs); + let mut admitted_pairs = 0_usize; + let mut sign_or_zero_refused_pairs = 0_usize; + let mut nonrepresentable_rate_refused_pairs = 0_usize; + let mut contributing_units = 0_usize; + let mut offset = 0_usize; + + for pair_count in pair_counts.values().copied() { + let end = offset + pair_count; + let unit_pairs = &lagged[offset..end]; + let mut unit_contributed = false; + for pair in unit_pairs { + if !same_sign_nonzero(pair.earlier_residual(), pair.later_residual()) { + sign_or_zero_refused_pairs += 1; + continue; + } + match driver_same_sign_log_rate( + pair.earlier_residual(), + pair.later_residual(), + pair.event_interval(), + ) { + Ok(rate) => { + admitted_rates.push(rate); + admitted_pairs += 1; + unit_contributed = true; + } + Err(LongitudinalError::InvalidTemporalTransformInput) => { + nonrepresentable_rate_refused_pairs += 1; + } + Err(error) => return Err(error), + } + } + if unit_contributed { + contributing_units += 1; + } + offset = end; + } + + let refused_pairs = sign_or_zero_refused_pairs + nonrepresentable_rate_refused_pairs; + debug_assert_eq!(offset, candidate_pairs); + debug_assert_eq!(admitted_pairs + refused_pairs, candidate_pairs); + + let estimate = if admitted_rates.is_empty() { + None + } else { + Some(scaled_compensated_mean(&admitted_rates)?) + }; + + Ok(IrregularRateSummary { + estimand, + estimate, + candidate_units, + contributing_units, + candidate_pairs, + admitted_pairs, + sign_or_zero_refused_pairs, + nonrepresentable_rate_refused_pairs, + }) +} + +fn consecutive_pair_counts(rows: &[EventTimedObservation]) -> BTreeMap { + let mut occasion_counts = BTreeMap::::new(); + for row in rows { + *occasion_counts.entry(row.unit_index()).or_default() += 1; + } + occasion_counts + .into_iter() + .filter_map(|(unit, count)| (count >= 2).then_some((unit, count - 1))) + .collect() +} diff --git a/crates/longitudinal_core/src/irregular_residual.rs b/crates/longitudinal_core/src/irregular_residual.rs new file mode 100644 index 000000000..8714ce1ee --- /dev/null +++ b/crates/longitudinal_core/src/irregular_residual.rs @@ -0,0 +1,1052 @@ +//! CWC-then-irregular residual log-rate on substantive event time. +//! +//! Unique evidence folded from Draft #327. Temporal composition belongs here, +//! not in a generic psychometric kernel. This is not DSEM, not Newton LS, and +//! not raw-process autoregressive drift. + +use std::collections::BTreeMap; + +use crate::{EventTimeInterval, LongitudinalError}; + +/// One unit's score at one event-time occasion. +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct EventTimedObservation { + unit_index: u32, + event_time: f64, + score: f64, +} + +impl EventTimedObservation { + /// Construct an event-timed observation. + /// + /// The constructor stores the fields as given. Admission of finite scores, + /// strictly positive consecutive intervals, and at least two lag-contributing + /// units happens in [`center_within_unit_event_lags`]. + #[must_use] + pub const fn new(unit_index: u32, event_time: f64, score: f64) -> Self { + Self { + unit_index, + event_time, + score, + } + } + + /// Return the unit index. + #[must_use] + pub const fn unit_index(self) -> u32 { + self.unit_index + } + + /// Return the event time. + #[must_use] + pub const fn event_time(self) -> f64 { + self.event_time + } + + /// Return the observed score. + #[must_use] + pub const fn score(self) -> f64 { + self.score + } +} + +/// One already-formed lagged within residual pair on event time. +/// +/// The interval is admitted event time. Residuals are stored as given; the +/// recover functions decide whether a pair is an admissible log-rate input. +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct LaggedWithinResidual { + earlier_residual: f64, + later_residual: f64, + event_interval: EventTimeInterval, +} + +impl LaggedWithinResidual { + /// Construct a lagged within-residual pair on an admitted event interval. + #[must_use] + pub const fn new( + earlier_residual: f64, + later_residual: f64, + event_interval: EventTimeInterval, + ) -> Self { + Self { + earlier_residual, + later_residual, + event_interval, + } + } + + /// Return the earlier within residual. + #[must_use] + pub const fn earlier_residual(self) -> f64 { + self.earlier_residual + } + + /// Return the later within residual. + #[must_use] + pub const fn later_residual(self) -> f64 { + self.later_residual + } + + /// Return the admitted event-time interval. + #[must_use] + pub const fn event_interval(self) -> EventTimeInterval { + self.event_interval + } +} + +/// Cluster-mean-center consecutive event-time lags inside each unit. +/// +/// Stable between-unit means are removed first (CWC). Consecutive within-unit +/// residuals then become [`LaggedWithinResidual`] pairs on possibly irregular +/// event intervals. Exact zero within residuals use canonical public `+0.0` +/// because a zero person-mean deviation has no directional measurement +/// meaning; signed zero remains available to private numerical intermediates +/// and to caller-constructed already-centered pairs. Singleton units do not +/// contribute lags and therefore do not count toward the two-unit longitudinal +/// evidence floor. Curran and Bauer +/// (2011, pp. 583–619; PMC3059070 XML opened 2026-09-02; Eq. 36) show that +/// person-mean subtraction of a time-varying covariate related to time is +/// biased for the within-person effect. The returned pairs are therefore not a +/// license to recover raw-process drift `a`. +/// +/// # Errors +/// +/// Returns [`LongitudinalError::InvalidObservationPayload`] for empty, +/// singleton-only, fewer-than-two lag-contributing units, or non-finite rows, +/// including non-representable stable unit means and overflowing CWC residuals +/// after a finite mean, and [`LongitudinalError::NonPositiveEventInterval`] +/// when any consecutive event interval is not strictly positive. +pub fn center_within_unit_event_lags( + rows: &[EventTimedObservation], +) -> Result, LongitudinalError> { + if rows.len() < 2 { + return Err(LongitudinalError::InvalidObservationPayload); + } + let mut groups: BTreeMap> = BTreeMap::new(); + for &row in rows { + if !row.event_time().is_finite() || !row.score().is_finite() { + return Err(LongitudinalError::InvalidObservationPayload); + } + groups.entry(row.unit_index()).or_default().push(row); + } + if groups.len() < 2 { + return Err(LongitudinalError::InvalidObservationPayload); + } + let lag_contributing_units = groups + .values() + .filter(|occasions| occasions.len() >= 2) + .count(); + if lag_contributing_units < 2 { + return Err(LongitudinalError::InvalidObservationPayload); + } + let mut pairs = Vec::new(); + for occasions in groups.values_mut() { + if occasions.len() < 2 { + continue; + } + occasions.sort_by(|left, right| left.event_time().total_cmp(&right.event_time())); + let scores: Vec = occasions.iter().map(|row| row.score()).collect(); + let mean = scaled_compensated_mean(&scores) + .map_err(|_| LongitudinalError::InvalidObservationPayload)?; + for window in occasions.windows(2) { + let earlier_residual = window[0].score() - mean; + let later_residual = window[1].score() - mean; + if !earlier_residual.is_finite() || !later_residual.is_finite() { + return Err(LongitudinalError::InvalidObservationPayload); + } + let event_delta = window[1].event_time() - window[0].event_time(); + let event_interval = EventTimeInterval::new(event_delta)?; + pairs.push(LaggedWithinResidual::new( + if earlier_residual == 0.0 { + 0.0 + } else { + earlier_residual + }, + if later_residual == 0.0 { + 0.0 + } else { + later_residual + }, + event_interval, + )); + } + } + // Two lag-contributing units each yield at least one validated window, so + // the pair list cannot be empty once the admission checks above succeed. + Ok(pairs) +} + +/// Pairwise-mean exact log-rate after CWC on irregular event intervals. +/// +/// This is [`center_within_unit_event_lags`] then the pairwise mean of the +/// Driver, Oud, and Voelkle (2017, Eq. 3) scalar inverse +/// `a = ln(|later| / |earlier|) / Δt` on nonzero same-sign residuals. For +/// magnitudes within a factor of two, the represented magnitude difference is +/// converted with `ln_1p` so an adjacent-float change is not rounded once by a +/// quotient and again by the logarithm. More extreme ratios use the direct +/// quotient when representable and otherwise fall back to +/// `ln|later| − ln|earlier|`. Opposite-sign and zero residuals have no real +/// logarithm and are skipped. The pairwise mean cancels opposite-signed finite +/// rates from largest magnitude downward before averaging the surviving +/// same-sign residuals. This avoids both overflowing a raw same-sign sum and +/// destroying representable subnormal terms by pre-scaling them. An empty +/// admissible set fails closed. This is not Newton LS and does not recover +/// raw-process drift from CWC of a raw AR path (Curran & Bauer, 2011, +/// pp. 583–619; Eq. 36). +/// +/// # Errors +/// +/// Propagates centering errors from [`center_within_unit_event_lags`]. A +/// non-finite log-rate or an empty admissible list is +/// [`LongitudinalError::InvalidTemporalTransformInput`]. +pub fn recover_within_unit_irregular_residual_log_rate( + rows: &[EventTimedObservation], +) -> Result { + let lagged = center_within_unit_event_lags(rows)?; + pairwise_same_sign_log_rate(&lagged) +} + +/// Mean exact scalar log-rate on already-centered residuals. +/// +/// Each pair is `a = ln(|later| / |earlier|) / Δt` (Driver et al., 2017, +/// Eq. 3 inverse). The function does **not** center again. Residuals must be +/// finite, nonzero, and of equal sign. Nearby represented magnitudes are +/// compared through their difference and `ln_1p`, avoiding a ratio-first +/// rounding step that can nearly double an adjacent-float change at a +/// power-of-two boundary. Extreme ratios retain direct-ratio and log-domain +/// fallbacks so a representable final log-rate is not rejected because of a +/// non-representable intermediate. This is the known-truth path that recovers +/// `ln(0.5)` from already-centered pairs `(1, 0.5)` over unit event time. +/// +/// # Errors +/// +/// Returns [`LongitudinalError::InvalidObservationPayload`] for an empty +/// series or non-finite residuals, and +/// [`LongitudinalError::InvalidTemporalTransformInput`] for zero or +/// opposite-sign residuals, a non-finite log-rate, or a non-representable +/// final mean. +pub fn recover_centered_irregular_residual_log_rate( + pairs: &[LaggedWithinResidual], +) -> Result { + if pairs.is_empty() { + return Err(LongitudinalError::InvalidObservationPayload); + } + let mut rates = Vec::with_capacity(pairs.len()); + for pair in pairs { + if !pair.earlier_residual().is_finite() || !pair.later_residual().is_finite() { + return Err(LongitudinalError::InvalidObservationPayload); + } + if !same_sign_nonzero(pair.earlier_residual(), pair.later_residual()) { + return Err(LongitudinalError::InvalidTemporalTransformInput); + } + rates.push(driver_same_sign_log_rate( + pair.earlier_residual(), + pair.later_residual(), + pair.event_interval(), + )?); + } + scaled_compensated_mean(&rates) +} + +/// Refuse treating a CWC residual log-rate as raw-process AR drift. +/// +/// Always fails closed. Curran and Bauer (2011, pp. 583–619; PMC3059070 XML +/// opened 2026-09-02) show that person-mean centering of a time-varying +/// covariate related to time is biased for the within-person effect. Licensed +/// detrend is the person-specific OLS residual of the covariate on time +/// (Eq. 36). Use [`recover_centered_irregular_residual_log_rate`] on +/// already-centered residuals for the raw-process estimand. +/// +/// # Errors +/// +/// Always returns [`LongitudinalError::CwcResidualLogRateIsNotRawProcessDrift`]. +pub fn refuse_cwc_residual_log_rate_as_raw_process_drift( + cwc_log_rate: f64, + raw_process_drift: f64, +) -> Result { + let _ = (cwc_log_rate, raw_process_drift); + Err(LongitudinalError::CwcResidualLogRateIsNotRawProcessDrift) +} + +fn pairwise_same_sign_log_rate(lagged: &[LaggedWithinResidual]) -> Result { + let mut rates = Vec::with_capacity(lagged.len()); + for pair in lagged { + if !same_sign_nonzero(pair.earlier_residual(), pair.later_residual()) { + continue; + } + rates.push(driver_same_sign_log_rate( + pair.earlier_residual(), + pair.later_residual(), + pair.event_interval(), + )?); + } + scaled_compensated_mean(&rates) +} + +/// Overflow-safe mean for finite values across binary64 scales. +/// +/// Same-sign inputs are normalized by an exact power-of-two scale derived from +/// their maximum magnitude before a deterministic compensated sum. Power-of-two +/// scaling avoids the extra division rounding that can turn an exact halfway +/// subnormal mean into the wrong even-neighbour result. Mixed-sign inputs are +/// partitioned by sign and sorted from largest magnitude downward. Opposite +/// signs are cancelled before any scale reduction, so a subnormal addend is +/// never divided into zero merely to protect an unrelated extreme term. Before +/// an opposite-side term can be rounded away unchanged, adjacent same-sign mass +/// on that side is coalesced and retried so collectively representable low-order +/// mass is not discarded one row at a time. Each cancellation is an +/// opposite-sign addition and therefore cannot overflow. The remaining terms +/// have one sign and are normalized and summed before the original sample-count +/// denominator is applied. This avoids rounding a retained-only mean and then +/// weighting that rounded intermediate, which can move a representable +/// mixed-sign subnormal result by one ULP. A nonzero same-sign residual mass +/// whose final real mean is below binary64's positive range fails closed instead +/// of being reported as exact zero. +pub(crate) fn scaled_compensated_mean(values: &[f64]) -> Result { + if values.is_empty() { + return Err(LongitudinalError::InvalidTemporalTransformInput); + } + + let mut positives = Vec::new(); + let mut negatives = Vec::new(); + for &value in values { + if !value.is_finite() { + return Err(LongitudinalError::InvalidTemporalTransformInput); + } + if value > 0.0 { + positives.push(value); + } else if value < 0.0 { + negatives.push(value); + } + } + + if positives.is_empty() && negatives.is_empty() { + return Ok(0.0); + } + + if positives.is_empty() || negatives.is_empty() { + return same_sign_mean_over_total(values, values.len()); + } + + positives.sort_by(|left, right| right.total_cmp(left)); + negatives.sort_by(f64::total_cmp); + + let mut positive_index = 0_usize; + let mut negative_index = 0_usize; + let mut positive = positives[0]; + let mut negative = negatives[0]; + let mut residuals = Vec::with_capacity(values.len()); + + loop { + let residual = positive + negative; + if residual.to_bits() == positive.to_bits() && negative_index + 1 < negatives.len() { + negative += negatives[negative_index + 1]; + negative_index += 1; + continue; + } + if residual.to_bits() == negative.to_bits() && positive_index + 1 < positives.len() { + positive += positives[positive_index + 1]; + positive_index += 1; + continue; + } + if residual > 0.0 { + positive = residual; + negative_index += 1; + if negative_index == negatives.len() { + residuals.push(positive); + residuals.extend_from_slice(&positives[positive_index + 1..]); + break; + } + negative = negatives[negative_index]; + } else if residual < 0.0 { + negative = residual; + positive_index += 1; + if positive_index == positives.len() { + residuals.push(negative); + residuals.extend_from_slice(&negatives[negative_index + 1..]); + break; + } + positive = positives[positive_index]; + } else { + positive_index += 1; + negative_index += 1; + if positive_index == positives.len() || negative_index == negatives.len() { + residuals.extend_from_slice(&positives[positive_index..]); + residuals.extend_from_slice(&negatives[negative_index..]); + break; + } + positive = positives[positive_index]; + negative = negatives[negative_index]; + } + } + + if residuals.is_empty() { + return Ok(0.0); + } + same_sign_mean_over_total(&residuals, values.len()) +} + +fn same_sign_mean_over_total(values: &[f64], total_count: usize) -> Result { + let max_magnitude = values + .iter() + .map(|value| value.abs()) + .max_by(f64::total_cmp) + .ok_or(LongitudinalError::InvalidTemporalTransformInput)?; + if max_magnitude == 0.0 { + return Ok(0.0); + } + + let scale = exact_power_of_two_scale(max_magnitude); + let mut normalized: Vec = values.iter().map(|value| *value / scale).collect(); + normalized.sort_by(f64::total_cmp); + let mut sum = 0.0_f64; + let mut compensation = 0.0_f64; + for value in normalized { + let adjusted = value - compensation; + let next = sum + adjusted; + compensation = (next - sum) - adjusted; + sum = next; + } + let mean = (sum / total_count as f64) * scale; + if mean == 0.0 { + return Err(LongitudinalError::InvalidTemporalTransformInput); + } + require_finite(mean) +} + +fn exact_power_of_two_scale(max_magnitude: f64) -> f64 { + let bits = max_magnitude.to_bits(); + let exponent = (bits >> 52) & 0x7ff; + if exponent == 0 { + let significand = bits & 0x000f_ffff_ffff_ffff; + let highest_bit = significand.ilog2(); + f64::from_bits(1_u64 << highest_bit) + } else { + f64::from_bits(exponent << 52) + } +} + +/// Nonzero residuals of equal sign admit a real Driver Eq. 3 logarithm. +pub(crate) fn same_sign_nonzero(earlier: f64, later: f64) -> bool { + earlier != 0.0 && later != 0.0 && earlier.is_sign_positive() == later.is_sign_positive() +} + +/// Driver et al. (2017, Eq. 3) inverse `a = ln(|later| / |earlier|) / Δt`. +/// +/// Caller already established same-sign nonzero residuals and an admitted +/// event interval. When the represented magnitudes differ by no more than a +/// factor of two, subtraction is exact in binary floating point and `ln_1p` +/// preserves the represented relative change without first rounding a quotient. +/// Growth uses `-ln1p(-(later-earlier)/later)` and decay uses +/// `ln1p((later-earlier)/earlier)`. More extreme scales retain the finite direct +/// ratio and log-domain fallbacks. A represented zero rate is accepted only +/// when the residual magnitudes are exactly equal; otherwise it is a +/// non-representable nonzero change and fails closed. +pub(crate) fn driver_same_sign_log_rate( + earlier: f64, + later: f64, + event_interval: EventTimeInterval, +) -> Result { + let earlier_magnitude = earlier.abs(); + let later_magnitude = later.abs(); + let log_ratio = if later_magnitude >= earlier_magnitude + && later_magnitude <= earlier_magnitude * 2.0 + { + let relative_loss_from_later = (later_magnitude - earlier_magnitude) / later_magnitude; + -(-relative_loss_from_later).ln_1p() + } else if earlier_magnitude > later_magnitude && earlier_magnitude <= later_magnitude * 2.0 { + let relative_change_from_earlier = + (later_magnitude - earlier_magnitude) / earlier_magnitude; + relative_change_from_earlier.ln_1p() + } else { + let ratio = later_magnitude / earlier_magnitude; + if ratio.is_finite() && ratio > 0.0 { + ratio.ln() + } else { + later_magnitude.ln() - earlier_magnitude.ln() + } + }; + let rate = log_ratio / event_interval.as_f64(); + if !rate.is_finite() + || (rate == 0.0 && later_magnitude.to_bits() != earlier_magnitude.to_bits()) + { + Err(LongitudinalError::InvalidTemporalTransformInput) + } else { + Ok(rate) + } +} + +fn require_finite(value: f64) -> Result { + if value.is_finite() { + Ok(value) + } else { + Err(LongitudinalError::InvalidTemporalTransformInput) + } +} + +#[cfg(test)] +mod tests { + use super::{ + EventTimedObservation, LaggedWithinResidual, center_within_unit_event_lags, + driver_same_sign_log_rate, recover_centered_irregular_residual_log_rate, + recover_within_unit_irregular_residual_log_rate, + refuse_cwc_residual_log_rate_as_raw_process_drift, require_finite, + same_sign_mean_over_total, same_sign_nonzero, scaled_compensated_mean, + }; + use crate::{EventTimeInterval, LongitudinalError}; + + fn timed(unit: u32, event_time: f64, score: f64) -> EventTimedObservation { + EventTimedObservation::new(unit, event_time, score) + } + + fn unit_interval() -> EventTimeInterval { + EventTimeInterval::new(1.0).expect("unit interval") + } + + fn lagged(earlier: f64, later: f64, delta: f64) -> LaggedWithinResidual { + LaggedWithinResidual::new( + earlier, + later, + EventTimeInterval::new(delta).expect("test interval"), + ) + } + + fn decaying_scores(drift: f64) -> [EventTimedObservation; 8] { + [ + timed(0, 0.0, 10.0 + 1.0), + timed(0, 1.0, 10.0 + drift.exp()), + timed(0, 2.0, 10.0 + (drift * 2.0).exp()), + timed(0, 3.0, 10.0 + (drift * 3.0).exp()), + timed(1, 0.0, 4.0 + 1.0), + timed(1, 1.0, 4.0 + drift.exp()), + timed(1, 2.0, 4.0 + (drift * 2.0).exp()), + timed(1, 3.0, 4.0 + (drift * 3.0).exp()), + ] + } + + #[test] + fn already_centered_irregular_pairs_recover_true_log_rate() { + let drift = -0.35_f64; + let pairs = [ + lagged(1.4, 1.4 * (drift * 0.4).exp(), 0.4), + lagged(0.9, 0.9 * (drift * 1.6).exp(), 1.6), + lagged(-0.7, -0.7 * (drift * 2.2).exp(), 2.2), + ]; + let recovered = recover_centered_irregular_residual_log_rate(&pairs).expect("centered"); + assert!((recovered - drift).abs() < 1e-12); + let half = recover_centered_irregular_residual_log_rate(&[lagged(1.0, 0.5, 1.0)]) + .expect("ln(0.5)"); + assert!((half - 0.5_f64.ln()).abs() < 1e-15); + } + + #[test] + fn cwc_of_raw_ar_does_not_recover_process_drift() { + let drift = -0.3_f64; + let rows = decaying_scores(drift); + let extracted = center_within_unit_event_lags(&rows).expect("cwc pairs"); + let composed = + recover_within_unit_irregular_residual_log_rate(&rows).expect("cwc pairwise"); + assert!((composed - drift).abs() > 1e-6); + assert_eq!( + refuse_cwc_residual_log_rate_as_raw_process_drift(composed, drift), + Err(LongitudinalError::CwcResidualLogRateIsNotRawProcessDrift) + ); + let admissible: Vec = extracted + .iter() + .copied() + .filter(|pair| same_sign_nonzero(pair.earlier_residual(), pair.later_residual())) + .collect(); + let from_pairs = + recover_centered_irregular_residual_log_rate(&admissible).expect("admissible"); + assert!((composed - from_pairs).abs() < 1e-15); + assert_eq!( + extracted[0].event_interval().as_f64().to_bits(), + 1.0_f64.to_bits() + ); + assert_eq!(timed(9, 8.0, 0.0).unit_index(), 9); + assert_eq!(timed(9, 8.0, 0.0).event_time().to_bits(), 8.0_f64.to_bits()); + assert_eq!(timed(9, 8.0, 0.0).score().to_bits(), 0.0_f64.to_bits()); + assert_eq!( + lagged(1.0, 0.5, 1.0).earlier_residual().to_bits(), + 1.0_f64.to_bits() + ); + assert_eq!( + lagged(1.0, 0.5, 1.0).later_residual().to_bits(), + 0.5_f64.to_bits() + ); + } + + #[test] + fn two_occasion_cwc_is_empty_admissible() { + assert_eq!( + recover_within_unit_irregular_residual_log_rate(&[ + timed(0, 0.0, 1.0), + timed(0, 1.0, 0.5), + timed(1, 0.0, 2.0), + timed(1, 1.0, 1.0), + ]), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + } + + #[test] + fn arithmetic_progression_has_zero_residual_and_fails_closed() { + assert_eq!( + recover_within_unit_irregular_residual_log_rate(&[ + timed(0, 0.0, 1.0), + timed(0, 1.0, 2.0), + timed(0, 2.0, 3.0), + timed(1, 0.0, 4.0), + timed(1, 1.0, 5.0), + timed(1, 2.0, 6.0), + ]), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + } + + #[test] + fn same_sign_nonzero_rejects_zero_and_opposite_signs() { + assert!(!same_sign_nonzero(0.0, 1.0)); + assert!(!same_sign_nonzero(1.0, 0.0)); + assert!(!same_sign_nonzero(-1.0, 2.0)); + assert!(same_sign_nonzero(1e-160, 1e160)); + assert!(same_sign_nonzero(-0.4, -1.2)); + } + + #[test] + fn driver_same_sign_uses_relative_change_for_nearby_residuals() { + let earlier = 1e20_f64; + let later = earlier * (-1e-12_f64).exp(); + let relative_change = (later.abs() - earlier.abs()) / earlier.abs(); + let from_relative_change = relative_change.ln_1p(); + let rate = driver_same_sign_log_rate(earlier, later, unit_interval()).expect("near-equal"); + assert_eq!(rate.to_bits(), from_relative_change.to_bits()); + + let adjacent_earlier = f64::from_bits(2.0_f64.to_bits() - 1); + let adjacent_growth = driver_same_sign_log_rate(adjacent_earlier, 2.0, unit_interval()) + .expect("adjacent power-of-two growth"); + let exact_adjacent_growth = -(-(f64::EPSILON / 2.0)).ln_1p(); + assert_eq!(adjacent_growth.to_bits(), exact_adjacent_growth.to_bits()); + + let overflow_rate = driver_same_sign_log_rate(f64::from_bits(1), f64::MAX, unit_interval()) + .expect("overflow arm"); + let overflow_logs = f64::MAX.ln() - f64::from_bits(1).ln(); + assert_eq!(overflow_rate.to_bits(), overflow_logs.to_bits()); + let underflow_rate = + driver_same_sign_log_rate(1e300_f64, 1e-300_f64, unit_interval()).expect("underflow"); + let underflow_logs = (1e-300_f64).ln() - (1e300_f64).ln(); + assert_eq!(underflow_rate.to_bits(), underflow_logs.to_bits()); + } + + #[test] + fn scaled_compensated_mean_keeps_extreme_cancellation_and_overflow_safe_mean() { + let large = 1.45e308_f64; + assert!(!(large + large).is_finite()); + let same_sign = scaled_compensated_mean(&[large, large]).expect("same-sign mean"); + assert!(same_sign.is_finite()); + assert!((same_sign - large).abs() < 1.0); + let negative = scaled_compensated_mean(&[-4.0, -2.0]).expect("negative same-sign mean"); + assert_eq!(negative.to_bits(), (-3.0_f64).to_bits()); + let mixed = scaled_compensated_mean(&[large, -large]).expect("mixed mean"); + assert!(mixed.abs() < 1.0); + + let recovered = scaled_compensated_mean(&[1.0e100, 1.0, -1.0e100]) + .expect("small signal survives cancellation"); + assert!((recovered - (1.0 / 3.0)).abs() <= 1.0e-12); + let full_range = scaled_compensated_mean(&[f64::MAX, 1.0e-16, -f64::MAX]) + .expect("full exponent range cancellation"); + assert_eq!(full_range.to_bits(), (1.0e-16_f64 / 3.0).to_bits()); + let minimum_subnormal = f64::from_bits(1); + let subnormal = + scaled_compensated_mean(&[f64::MAX, f64::from_bits(2), f64::from_bits(2), -f64::MAX]) + .expect("subnormal cancellation residue"); + assert_eq!(subnormal.to_bits(), minimum_subnormal.to_bits()); + let finite_after_mass_overflow = scaled_compensated_mean(&[large, large, -1.0, -1.0]) + .expect("representable final mean after retained-mass overflow"); + assert!(finite_after_mass_overflow.is_finite()); + assert!( + (finite_after_mass_overflow - large / 2.0).abs() <= (large / 2.0) * 4.0 * f64::EPSILON + ); + assert_eq!( + scaled_compensated_mean(&[]), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + assert_eq!(scaled_compensated_mean(&[0.0, 0.0]), Ok(0.0)); + assert_eq!( + scaled_compensated_mean(&[f64::INFINITY]), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + } + + fn overflowing_cwc_rate_rows(unit: u32, growing: bool) -> [EventTimedObservation; 3] { + // Representable leftover after MAX/-MAX cancellation (ae5081 false-zero + // mean fail-closed). 1e-100 keeps the unit mean nonzero while ln(MAX/r)/Δt + // with Δt=1e-305 still exceeds MAX/2 so two pairwise rates overflow when + // added, exercising compensated-mean recovery of the overflowed sum. + let delta = 1e-305_f64; + let leftover = 1e-100_f64; + let (first, second) = if growing { + (leftover, f64::MAX) + } else { + (f64::MAX, leftover) + }; + [ + timed(unit, 0.0, first), + timed(unit, delta, second), + timed(unit, 2.0 * delta, -f64::MAX), + ] + } + + #[test] + fn cwc_pairwise_keeps_overflowed_rate_sum_via_compensated_mean() { + let mut rows = overflowing_cwc_rate_rows(1, true).to_vec(); + rows.extend(overflowing_cwc_rate_rows(2, true)); + let recovered = + recover_within_unit_irregular_residual_log_rate(&rows).expect("compensated"); + assert!(recovered.is_finite()); + let extracted = center_within_unit_event_lags(&rows).expect("extract"); + let mut rates = Vec::new(); + for pair in extracted { + if !same_sign_nonzero(pair.earlier_residual(), pair.later_residual()) { + continue; + } + rates.push( + driver_same_sign_log_rate( + pair.earlier_residual(), + pair.later_residual(), + pair.event_interval(), + ) + .expect("pair rate"), + ); + } + assert_eq!(rates.len(), 2); + assert!(!(rates[0] + rates[1]).is_finite()); + let expected = scaled_compensated_mean(&rates).expect("reference compensated mean"); + assert!((recovered - expected).abs() <= expected.abs() * 1e-15); + let mut mixed = overflowing_cwc_rate_rows(1, true).to_vec(); + mixed.extend(overflowing_cwc_rate_rows(2, false)); + let mixed_mean = + recover_within_unit_irregular_residual_log_rate(&mixed).expect("mixed-sign"); + assert!(mixed_mean.abs() < recovered.abs() * 1e-12); + } + + #[test] + fn cwc_pairwise_tiny_interval_with_huge_log_ratio_fails_closed() { + assert_eq!( + recover_within_unit_irregular_residual_log_rate(&[ + timed(1, 0.0, 1e-160), + timed(1, f64::from_bits(1), 1e160), + timed(1, 1.0, -1e160), + timed(2, 0.0, 1.0), + timed(2, 1.0, 0.5), + ]), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + } + + #[test] + fn cwc_pairwise_skips_zero_residuals_and_keeps_same_sign_pairs() { + let recovered = recover_within_unit_irregular_residual_log_rate(&[ + timed(1, 0.0, 7.0), + timed(1, 1.0, 5.0), + timed(1, 2.0, 4.0), + timed(1, 3.0, 4.0), + timed(2, 0.0, -1.2), + timed(2, 1.0, -0.4), + timed(2, 2.0, -0.8), + ]) + .expect("skip zeros keep same-sign"); + assert!(recovered.is_finite()); + let extracted = center_within_unit_event_lags(&[ + timed(1, 0.0, 7.0), + timed(1, 1.0, 5.0), + timed(1, 2.0, 4.0), + timed(1, 3.0, 4.0), + timed(2, 0.0, -1.2), + timed(2, 1.0, -0.4), + timed(2, 2.0, -0.8), + ]) + .expect("extract"); + assert!( + extracted + .iter() + .any(|pair| pair.later_residual().to_bits() == 0.0_f64.to_bits()) + ); + assert!( + extracted + .iter() + .any(|pair| pair.earlier_residual().to_bits() == 0.0_f64.to_bits()) + ); + assert!(extracted.iter().any(|pair| same_sign_nonzero( + pair.earlier_residual(), + pair.later_residual() + ) && !pair.earlier_residual().is_sign_positive())); + } + + #[test] + fn cwc_observation_payload_paths_fail_closed() { + assert_eq!( + recover_within_unit_irregular_residual_log_rate(&[]), + Err(LongitudinalError::InvalidObservationPayload) + ); + assert_eq!( + recover_within_unit_irregular_residual_log_rate(&[timed(1, 0.0, 1.0)]), + Err(LongitudinalError::InvalidObservationPayload) + ); + assert_eq!( + recover_within_unit_irregular_residual_log_rate(&[ + timed(1, 0.0, 1.0), + timed(1, 1.0, 0.5), + ]), + Err(LongitudinalError::InvalidObservationPayload) + ); + assert_eq!( + recover_within_unit_irregular_residual_log_rate(&[ + timed(1, 0.0, 1.0), + timed(2, 1.0, 0.5), + ]), + Err(LongitudinalError::InvalidObservationPayload) + ); + assert_eq!( + center_within_unit_event_lags(&[ + timed(1, 0.0, 1.0), + timed(1, 0.0, 1.2), + timed(2, 0.0, 2.0), + timed(2, 1.0, 1.5), + ]), + Err(LongitudinalError::NonPositiveEventInterval) + ); + assert_eq!( + recover_within_unit_irregular_residual_log_rate(&[ + timed(1, 0.0, f64::NAN), + timed(1, 1.0, 1.0), + timed(2, 0.0, 1.0), + timed(2, 1.0, 0.5), + ]), + Err(LongitudinalError::InvalidObservationPayload) + ); + assert_eq!( + recover_within_unit_irregular_residual_log_rate(&[ + timed(1, 0.0, f64::INFINITY), + timed(1, 1.0, 1.0), + timed(2, 0.0, 1.0), + timed(2, 1.0, 0.5), + ]), + Err(LongitudinalError::InvalidObservationPayload) + ); + assert_eq!( + recover_within_unit_irregular_residual_log_rate(&[ + timed(1, 0.0, f64::MAX), + timed(1, 1.0, f64::MAX), + timed(2, 0.0, 1.0), + timed(2, 1.0, 0.5), + ]), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + assert_eq!( + center_within_unit_event_lags(&[ + timed(1, f64::MAX, 1.0), + timed(1, -f64::MAX, 0.5), + timed(2, 0.0, 1.0), + timed(2, 1.0, 0.5), + ]), + Err(LongitudinalError::NonPositiveEventInterval) + ); + } + + #[test] + fn already_centered_and_curran_refusal_paths_fail_closed() { + assert_eq!( + recover_centered_irregular_residual_log_rate(&[]), + Err(LongitudinalError::InvalidObservationPayload) + ); + assert_eq!( + recover_centered_irregular_residual_log_rate(&[lagged(f64::NAN, 0.5, 1.0)]), + Err(LongitudinalError::InvalidObservationPayload) + ); + assert_eq!( + recover_centered_irregular_residual_log_rate(&[lagged(1.0, f64::NAN, 1.0)]), + Err(LongitudinalError::InvalidObservationPayload) + ); + assert_eq!( + recover_centered_irregular_residual_log_rate(&[lagged(0.0, 0.5, 1.0)]), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + assert_eq!( + recover_centered_irregular_residual_log_rate(&[lagged(1.0, -0.5, 1.0)]), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + let overflow_ratio = recover_centered_irregular_residual_log_rate(&[lagged( + f64::from_bits(1), + f64::MAX, + 1.0, + )]) + .expect("finite log-domain overflow fallback"); + let expected_overflow = f64::MAX.ln() - f64::from_bits(1).ln(); + assert_eq!(overflow_ratio.to_bits(), expected_overflow.to_bits()); + let underflow_ratio = recover_centered_irregular_residual_log_rate(&[lagged( + f64::MAX, + f64::from_bits(1), + 1.0, + )]) + .expect("finite log-domain underflow fallback"); + let expected_underflow = f64::from_bits(1).ln() - f64::MAX.ln(); + assert_eq!(underflow_ratio.to_bits(), expected_underflow.to_bits()); + assert_eq!( + recover_centered_irregular_residual_log_rate(&[lagged( + 1e-160, + 1e160, + f64::from_bits(1) + )]), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + assert_eq!( + refuse_cwc_residual_log_rate_as_raw_process_drift(f64::NAN, f64::INFINITY), + Err(LongitudinalError::CwcResidualLogRateIsNotRawProcessDrift) + ); + } + + #[test] + fn cwc_orders_unsorted_event_times_before_lag_pairs() { + let pairs = center_within_unit_event_lags(&[ + timed(1, 2.0, 0.4), + timed(2, 3.0, -0.8), + timed(1, 0.5, 1.6), + timed(2, 1.0, 0.2), + ]) + .expect("unsorted"); + assert_eq!(pairs.len(), 2); + assert!((pairs[0].event_interval().as_f64() - 1.5).abs() < 1e-15); + assert!((pairs[1].event_interval().as_f64() - 2.0).abs() < 1e-15); + let cluster_one_mean = f64::midpoint(1.6, 0.4); + assert!((pairs[0].earlier_residual() - (1.6 - cluster_one_mean)).abs() < 1e-15); + assert!((pairs[0].later_residual() - (0.4 - cluster_one_mean)).abs() < 1e-15); + } + + #[test] + fn singleton_second_unit_does_not_satisfy_longitudinal_unit_floor() { + let drift = -0.2_f64; + let mixed = [ + timed(1, 0.0, 10.0 + 1.0), + timed(1, 1.0, 10.0 + drift.exp()), + timed(1, 2.0, 10.0 + (drift * 2.0).exp()), + timed(1, 3.0, 10.0 + (drift * 3.0).exp()), + timed(2, 0.0, 4.0), + ]; + assert_eq!( + recover_within_unit_irregular_residual_log_rate(&mixed), + Err(LongitudinalError::InvalidObservationPayload) + ); + assert_eq!( + recover_within_unit_irregular_residual_log_rate(&[ + timed(1, 0.0, 1.0), + timed(2, 1.0, 0.5) + ]), + Err(LongitudinalError::InvalidObservationPayload) + ); + } + + #[test] + fn overflowing_same_sign_cwc_pairs_keep_stable_log() { + // Representable unit mean: after MAX/-MAX cancel, leftover 1.0 yields mean + // 1/3 and same-sign CWC residuals (2/3, MAX). Subnormal leftovers underflow + // that mean to a false zero and fail closed before rate recovery. + let overflowed_both = recover_within_unit_irregular_residual_log_rate(&[ + timed(1, 0.0, 1.0), + timed(1, 1.0, f64::MAX), + timed(1, 2.0, -f64::MAX), + timed(2, 0.0, 1.0), + timed(2, 1.0, f64::MAX), + timed(2, 2.0, -f64::MAX), + ]) + .expect("stable log of overflowed same-sign CWC pairs"); + let overflow_rate = driver_same_sign_log_rate(2.0 / 3.0, f64::MAX, unit_interval()) + .expect("two-thirds/MAX after representable CWC mean"); + assert!((overflowed_both - overflow_rate).abs() < 1e-9); + } + + #[test] + fn one_sided_residual_overflow_fails_closed() { + assert_eq!( + recover_within_unit_irregular_residual_log_rate(&[ + timed(1, 0.0, -f64::MAX), + timed(1, 1.0, -f64::MAX), + timed(1, 2.0, -f64::MAX), + timed(1, 3.0, f64::MAX), + timed(2, 0.0, 1.0), + timed(2, 1.0, 0.8), + ]), + Err(LongitudinalError::InvalidObservationPayload) + ); + assert_eq!( + center_within_unit_event_lags(&[ + timed(1, 0.0, f64::MAX), + timed(1, 1.0, -f64::MAX), + timed(1, 2.0, -f64::MAX / 2.0), + timed(2, 0.0, 1.0), + timed(2, 1.0, 0.5), + ]), + Err(LongitudinalError::InvalidObservationPayload) + ); + } + + #[test] + fn singleton_unit_is_skipped_once_two_lag_units_admit() { + let rows = [ + timed(1, 0.0, 1.0), + timed(1, 1.0, 0.5), + timed(2, 0.0, 2.0), + timed(2, 1.0, 1.0), + timed(3, 0.0, 9.0), + ]; + let pairs = center_within_unit_event_lags(&rows).expect("singleton skipped"); + assert_eq!(pairs.len(), 2); + } + + #[test] + fn continued_exact_cancel_keeps_remaining_same_sign_terms() { + let mean = scaled_compensated_mean(&[3.0, 1.0, -3.0, -1.0]).expect("continued cancel"); + assert_eq!(mean.to_bits(), 0.0_f64.to_bits()); + } + + #[test] + fn same_sign_mean_over_total_accepts_all_zero_magnitudes() { + assert_eq!(same_sign_mean_over_total(&[0.0, 0.0], 2), Ok(0.0)); + } + + #[test] + fn require_finite_rejects_non_finite_values() { + assert_eq!( + require_finite(f64::INFINITY), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + assert_eq!( + require_finite(f64::NAN), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + assert_eq!(require_finite(1.5), Ok(1.5)); + } + + #[test] + fn non_finite_event_time_or_score_fails_closed() { + assert_eq!( + center_within_unit_event_lags(&[ + timed(1, f64::NAN, 1.0), + timed(1, 1.0, 0.5), + timed(2, 0.0, 2.0), + timed(2, 1.0, 1.0), + ]), + Err(LongitudinalError::InvalidObservationPayload) + ); + assert_eq!( + center_within_unit_event_lags(&[ + timed(1, 0.0, f64::INFINITY), + timed(1, 1.0, 0.5), + timed(2, 0.0, 2.0), + timed(2, 1.0, 1.0), + ]), + Err(LongitudinalError::InvalidObservationPayload) + ); + } +} diff --git a/crates/longitudinal_core/src/lib.rs b/crates/longitudinal_core/src/lib.rs index 31996b824..c909c770b 100644 --- a/crates/longitudinal_core/src/lib.rs +++ b/crates/longitudinal_core/src/lib.rs @@ -1,15 +1,29 @@ #![forbid(unsafe_code)] #![deny(missing_docs)] #![allow(clippy::cast_precision_loss)] -//! Within/between decomposition for longitudinal scores. +//! Longitudinal modeling primitives for temporal psychometrics. //! //! Stable between-unit components cannot be scored as within-unit change. -//! Recovery reports computed component RMSE against known truth (ADR 0005). +//! Event-time lagged associations require both marginal variances before a +//! covariance can be standardized as a correlation. Event-interval response +//! transforms and scalar diffusion standardisation candidates live here rather +//! than in a generic psychometric kernel. Person-mean centering of a time-related +//! series is not raw-process drift (Curran & Bauer, 2011). Recovery reports +//! computed component RMSE against known truth (ADR 0005). +mod association; mod component; mod decompose; +mod diffusion; +mod discrete_drift; mod error; +mod event_time; +mod irregular_rate_estimand; +mod irregular_residual; mod level; +mod occasion_mean; +mod stationary; +mod temporal_association; /// One unit-specific within or between component. pub use component::ComponentValue; @@ -19,9 +33,58 @@ pub use component::component_root_mean_square_error; pub use decompose::OccasionObservation; /// Decompose occasion scores into unit means and within residuals. pub use decompose::decompose_within_between; -/// Fail-closed longitudinal-decomposition errors. +/// Recover the scalar research-candidate continuous diffusion standardisation on event time. +pub use diffusion::recover_event_time_standardised_continuous_diffusion; +/// Recover the scalar research-candidate discrete diffusion standardisation on event time. +pub use diffusion::recover_event_time_standardised_discrete_diffusion; +/// Refuse continuous standardised diffusion as discrete standardised diffusion. +pub use diffusion::refuse_standardised_continuous_diffusion_as_standardised_discrete_diffusion; +/// Refuse total-variance scaling as relevant-variance diffusion standardisation. +pub use diffusion::refuse_total_variance_scaled_diffusion_as_standardised_diffusion; +/// Refuse unstandardised diffusion as standardised diffusion. +pub use diffusion::refuse_unstandardised_diffusion_as_standardised_diffusion; +/// Recover scalar Driver p.16 `discreteDRIFTstd` on substantive event time. +pub use discrete_drift::recover_event_time_standardised_discrete_drift; +/// Refuse trait-plus-state association as `discreteDRIFTstd`. +pub use discrete_drift::refuse_trait_plus_state_association_as_standardised_discrete_drift; +/// Refuse trait variance as the drift standardisation variance. +pub use discrete_drift::refuse_trait_variance_as_standardisation_variance; +/// Refuse unstandardised `discreteDRIFT` as `discreteDRIFTstd`. +pub use discrete_drift::refuse_unstandardised_discrete_drift_as_standardised_discrete_drift; +/// Fail-closed longitudinal-modeling errors. pub use error::LongitudinalError; +/// A finite, strictly positive interval admitted on substantive event time. +pub use event_time::EventTimeInterval; +/// Versioned weighting target for irregular residual log-rate summaries. +pub use irregular_rate_estimand::IrregularRateEstimand; +/// Denominator-bearing evidence for an irregular residual log-rate summary. +pub use irregular_rate_estimand::IrregularRateSummary; +/// Recover an explicitly named irregular residual log-rate estimand with denominators. +pub use irregular_rate_estimand::recover_within_unit_irregular_rate_summary; +/// One unit's score at one event-time occasion. +pub use irregular_residual::EventTimedObservation; +/// One already-formed lagged within residual pair on event time. +pub use irregular_residual::LaggedWithinResidual; +/// Cluster-mean-center consecutive event-time lags inside each unit. +pub use irregular_residual::center_within_unit_event_lags; +/// Mean exact scalar log-rate on already-centered residuals with stable count weighting. +pub use irregular_residual::recover_centered_irregular_residual_log_rate; +/// Pairwise-mean exact log-rate after CWC with stable count weighting. +pub use irregular_residual::recover_within_unit_irregular_residual_log_rate; +/// Refuse treating a CWC residual log-rate as raw-process AR drift. +pub use irregular_residual::refuse_cwc_residual_log_rate_as_raw_process_drift; /// Established longitudinal component level. pub use level::ComponentLevel; /// Refuse to treat a between-unit component as within-unit change. pub use level::refuse_between_as_within_change; +/// Form consecutive event-time lags after subtracting each occasion's group mean. +pub use occasion_mean::center_occasion_mean_event_lags; +/// Recover the exact scalar log-rate of occasion-mean residuals. +pub use occasion_mean::recover_occasion_mean_centered_irregular_residual_log_rate; +/// Refuse treating occasion-mean residual lag as within-person change. +pub use occasion_mean::refuse_occasion_mean_centered_log_rate_as_within_person_lag; +/// Recover the scalar Driver p.16 unstandardised stationary within-person variance. +pub use stationary::recover_stationary_within_variance; +/// Recover a valid event-time lagged correlation from covariance and both +/// marginal variances through the typed event-time boundary. +pub use temporal_association::recover_event_time_lagged_correlation; diff --git a/crates/longitudinal_core/src/occasion_mean.rs b/crates/longitudinal_core/src/occasion_mean.rs new file mode 100644 index 000000000..1e0f01a5d --- /dev/null +++ b/crates/longitudinal_core/src/occasion_mean.rs @@ -0,0 +1,266 @@ +//! Occasion-mean event-time composition for longitudinal modeling. +//! +//! Hamaker, Kuiper, and Grasman (2015, Eq. 1a) decompose an observed score as +//! `x_it = mu_t + p_it`, where `mu_t` is the occasion-specific group mean. +//! These deviations are distinct from person-mean CWC residuals and still +//! contain stable between-person differences, so they are not within-person +//! effects and are not RI-CLPM residuals. + +use std::collections::{BTreeMap, BTreeSet}; + +use crate::irregular_residual::{ + EventTimedObservation, LaggedWithinResidual, recover_centered_irregular_residual_log_rate, + scaled_compensated_mean, +}; +use crate::{EventTimeInterval, LongitudinalError}; + +/// Form consecutive event-time lags after subtracting each occasion's group mean. +/// +/// Numeric event-time identity is used for occasion membership. In particular, +/// `-0.0` and `+0.0` are one occasion rather than two binary encodings. Every +/// admitted occasion must contain at least two distinct units, and at least two +/// units must contribute a consecutive lag. Occasion means use the same +/// Longitudinal-local overflow-safe, cancellation-safe, halfway-rounding +/// numerical authority as CWC means and are bit-stable under row permutation. +/// Exact zero occasion-mean residuals use canonical public `+0.0` because a +/// zero Hamaker Eq. 1a deviation has no directional measurement meaning; +/// signed zero remains available to private numerical intermediates and to +/// caller-constructed already-centered pairs. The returned pairs still retain +/// between-person differences and are not within-person or RI-CLPM lags. +/// +/// # Errors +/// +/// Returns [`LongitudinalError::InvalidObservationPayload`] for empty, sparse, +/// duplicated unit-occasion, or non-finite observations, and +/// [`LongitudinalError::NonPositiveEventInterval`] when a unit's consecutive +/// event times do not form a finite strictly positive interval. +pub fn center_occasion_mean_event_lags( + rows: &[EventTimedObservation], +) -> Result, LongitudinalError> { + if rows.len() < 2 { + return Err(LongitudinalError::InvalidObservationPayload); + } + + let mut by_time: BTreeMap> = BTreeMap::new(); + let mut by_unit: BTreeMap> = BTreeMap::new(); + for &row in rows { + if !row.event_time().is_finite() || !row.score().is_finite() { + return Err(LongitudinalError::InvalidObservationPayload); + } + by_time + .entry(canonical_event_time_key(row.event_time())) + .or_default() + .push(row); + by_unit.entry(row.unit_index()).or_default().push(row); + } + + let lag_contributing_units = by_unit + .values() + .filter(|occasions| occasions.len() >= 2) + .count(); + if lag_contributing_units < 2 { + return Err(LongitudinalError::InvalidObservationPayload); + } + + let mut occasion_means = BTreeMap::new(); + for (&time_key, occasion_rows) in &by_time { + let mut seen_units = BTreeSet::new(); + let mut scores = Vec::with_capacity(occasion_rows.len()); + for row in occasion_rows { + if !seen_units.insert(row.unit_index()) { + return Err(LongitudinalError::InvalidObservationPayload); + } + scores.push(row.score()); + } + if seen_units.len() < 2 { + return Err(LongitudinalError::InvalidObservationPayload); + } + occasion_means.insert(time_key, occasion_mean(&scores)?); + } + + let mut pairs = Vec::new(); + for occasions in by_unit.values_mut() { + if occasions.len() < 2 { + continue; + } + occasions.sort_by(|left, right| left.event_time().total_cmp(&right.event_time())); + for window in occasions.windows(2) { + let earlier = window[0]; + let later = window[1]; + let earlier_mean = occasion_means[&canonical_event_time_key(earlier.event_time())]; + let later_mean = occasion_means[&canonical_event_time_key(later.event_time())]; + let earlier_residual = earlier.score() - earlier_mean; + let later_residual = later.score() - later_mean; + if !earlier_residual.is_finite() || !later_residual.is_finite() { + return Err(LongitudinalError::InvalidObservationPayload); + } + let event_interval = EventTimeInterval::new(later.event_time() - earlier.event_time())?; + pairs.push(LaggedWithinResidual::new( + if earlier_residual == 0.0 { + 0.0 + } else { + earlier_residual + }, + if later_residual == 0.0 { + 0.0 + } else { + later_residual + }, + event_interval, + )); + } + } + + // Two lag-contributing units each yield at least one validated window, so + // the pair list cannot be empty once occasion means and intervals validate. + Ok(pairs) +} + +/// Recover the mean exact scalar log-rate of occasion-mean residuals. +/// +/// This composes [`center_occasion_mean_event_lags`] with the existing +/// longitudinal exact-log-rate boundary. It is an event-time association of +/// Hamaker Eq. 1a deviations, not a within-person autoregressive effect. +/// +/// # Errors +/// +/// Propagates centering and exact-log-rate admission failures. +pub fn recover_occasion_mean_centered_irregular_residual_log_rate( + rows: &[EventTimedObservation], +) -> Result { + let pairs = center_occasion_mean_event_lags(rows)?; + recover_centered_irregular_residual_log_rate(&pairs) +} + +/// Refuse treating an occasion-mean residual log-rate as within-person change. +/// +/// Hamaker Eq. 1a deviations retain between-person differences unless a +/// person-specific stable component is removed by a model that identifies it. +/// +/// # Errors +/// +/// Always returns [`LongitudinalError::BetweenIsNotWithinChange`]. +pub fn refuse_occasion_mean_centered_log_rate_as_within_person_lag( + log_rate: f64, +) -> Result { + let _ = log_rate; + Err(LongitudinalError::BetweenIsNotWithinChange) +} + +fn canonical_event_time_key(event_time: f64) -> u64 { + if event_time == 0.0 { + 0.0_f64.to_bits() + } else { + event_time.to_bits() + } +} + +fn occasion_mean(values: &[f64]) -> Result { + scaled_compensated_mean(values).map_err(|_| LongitudinalError::InvalidObservationPayload) +} + +#[cfg(test)] +mod tests { + use super::{ + center_occasion_mean_event_lags, recover_occasion_mean_centered_irregular_residual_log_rate, + }; + use crate::LongitudinalError; + use crate::irregular_residual::EventTimedObservation; + + fn timed(unit: u32, event_time: f64, score: f64) -> EventTimedObservation { + EventTimedObservation::new(unit, event_time, score) + } + + #[test] + fn fewer_than_two_lag_units_fail_closed() { + assert_eq!( + center_occasion_mean_event_lags(&[ + timed(1, 0.0, 1.0), + timed(1, 1.0, 0.5), + timed(2, 0.0, 2.0), + ]), + Err(LongitudinalError::InvalidObservationPayload) + ); + } + + #[test] + fn singleton_unit_is_skipped_when_two_lag_units_admit() { + // Units 1 and 2 contribute lags; unit 3 appears only once at a shared time so + // occasion means still have >=2 units while the singleton is skipped in the + // per-unit lag loop. + let pairs = center_occasion_mean_event_lags(&[ + timed(1, 0.0, 1.0), + timed(1, 1.0, 0.5), + timed(2, 0.0, 2.0), + timed(2, 1.0, 1.0), + timed(3, 0.0, 9.0), + ]) + .expect("singleton unit skipped"); + assert_eq!(pairs.len(), 2); + } + + #[test] + fn overflowing_occasion_mean_residual_fails_closed() { + // Three units at t=1 yield occasion mean -MAX/3; MAX - (-MAX/3) overflows. + assert_eq!( + center_occasion_mean_event_lags(&[ + timed(1, 0.0, 0.0), + timed(1, 1.0, f64::MAX), + timed(2, 0.0, 0.0), + timed(2, 1.0, -f64::MAX), + timed(3, 0.0, 0.0), + timed(3, 1.0, -f64::MAX), + ]), + Err(LongitudinalError::InvalidObservationPayload) + ); + } + + #[test] + fn happy_path_recovers_finite_log_rate() { + let rate = recover_occasion_mean_centered_irregular_residual_log_rate(&[ + timed(1, 0.0, 1.0), + timed(1, 1.0, 0.5), + timed(2, 0.0, 2.0), + timed(2, 1.0, 1.0), + ]) + .expect("occasion-mean rate"); + assert!(rate.is_finite()); + } + + #[test] + fn non_finite_event_time_or_score_fails_closed() { + assert_eq!( + center_occasion_mean_event_lags(&[ + timed(1, f64::NAN, 1.0), + timed(1, 1.0, 0.5), + timed(2, 0.0, 2.0), + timed(2, 1.0, 1.0), + ]), + Err(LongitudinalError::InvalidObservationPayload) + ); + assert_eq!( + center_occasion_mean_event_lags(&[ + timed(1, 0.0, f64::INFINITY), + timed(1, 1.0, 0.5), + timed(2, 0.0, 2.0), + timed(2, 1.0, 1.0), + ]), + Err(LongitudinalError::InvalidObservationPayload) + ); + } + + #[test] + fn overflowing_earlier_occasion_mean_residual_fails_closed() { + assert_eq!( + center_occasion_mean_event_lags(&[ + timed(1, 0.0, f64::MAX), + timed(1, 1.0, 0.0), + timed(2, 0.0, -f64::MAX), + timed(2, 1.0, 0.0), + timed(3, 0.0, -f64::MAX), + timed(3, 1.0, 0.0), + ]), + Err(LongitudinalError::InvalidObservationPayload) + ); + } +} diff --git a/crates/longitudinal_core/src/stationary.rs b/crates/longitudinal_core/src/stationary.rs new file mode 100644 index 000000000..027cc2e78 --- /dev/null +++ b/crates/longitudinal_core/src/stationary.rs @@ -0,0 +1,155 @@ +//! Internal stationary within-person variance arithmetic. + +use crate::LongitudinalError; + +/// Validate inputs for a scalar stationary continuous-time process. +/// +/// Finite `q >= 0` and strictly negative finite drift establish a finite, +/// non-negative stationary variance in the mathematical real-valued model. +/// This admission deliberately does not materialize `q / (-2a)` as binary64: +/// standardized scalar maps may cancel that variance algebraically even when +/// the intermediate itself lies below or above the representable `f64` range. +pub(crate) fn validate_stationary_process_inputs( + continuous_diffusion: f64, + log_rate: f64, +) -> Result<(), LongitudinalError> { + if !continuous_diffusion.is_finite() || continuous_diffusion < 0.0 || !log_rate.is_finite() { + return Err(LongitudinalError::InvalidTemporalTransformInput); + } + if log_rate >= 0.0 { + return Err(LongitudinalError::StationaryVarianceRequiresStableDrift); + } + Ok(()) +} + +/// Recover scalar stationary within-person variance `p = -q / (2a)`. +/// +/// This primitive is intentionally private to Longitudinal Modeling. It admits +/// finite non-negative continuous diffusion and strictly negative drift while +/// avoiding an otherwise unnecessary overflow in the intermediate `2a`. +/// Unlike standardized maps in which `p` cancels algebraically, this function +/// actually returns `p`, so the stationary variance itself must be representable +/// as binary64. Callers decide whether zero stationary variance is admissible +/// for their named estimand. +/// +/// # Errors +/// +/// Returns [`LongitudinalError::InvalidTemporalTransformInput`] for non-finite +/// diffusion/drift inputs, negative diffusion, or when a strictly positive real +/// stationary variance is not representable as binary64 (including collapse to +/// exact zero). Returns +/// [`LongitudinalError::StationaryVarianceRequiresStableDrift`] unless `a < 0`. +pub fn recover_stationary_within_variance( + continuous_diffusion: f64, + log_rate: f64, +) -> Result { + validate_stationary_process_inputs(continuous_diffusion, log_rate)?; + if continuous_diffusion == 0.0 { + return Ok(0.0); + } + + let twice_rate = log_rate * 2.0; + let stationary = if twice_rate.is_finite() { + continuous_diffusion / -twice_rate + } else { + // In this branch |a| > MAX/2, so halving the finite numerator before + // division is the exact power-of-two rescaling q/(2|a|). It avoids the + // predecessor's division-then-quarter sequence, which could round twice + // at the subnormal boundary. If q/2 itself underflows here, q/(2|a|) + // is necessarily far below the minimum representable positive value. + (continuous_diffusion * 0.5) / -log_rate + }; + // Positive q with stable finite a implies strictly positive real p. If the + // binary64 evaluation collapses that requested estimand to zero, zero is a + // false scientific boundary rather than a representable stationary + // variance. Standardized callers that algebraically cancel p use the input + // validator above and therefore are not rejected by this materialization + // rule. + if !stationary.is_finite() || stationary == 0.0 { + return Err(LongitudinalError::InvalidTemporalTransformInput); + } + Ok(stationary) +} + +#[cfg(test)] +mod tests { + use super::{recover_stationary_within_variance, validate_stationary_process_inputs}; + use crate::LongitudinalError; + + #[test] + fn stationary_input_admission_does_not_materialise_the_variance() { + assert_eq!( + validate_stationary_process_inputs(f64::from_bits(1), -1.0), + Ok(()) + ); + assert_eq!(validate_stationary_process_inputs(f64::MAX, -0.25), Ok(())); + } + + #[test] + fn avoids_doubling_overflow_when_final_stationary_variance_is_representable() { + let stationary = recover_stationary_within_variance(f64::MAX, -f64::MAX) + .expect("q / (-2a) remains representable"); + assert_eq!(stationary.to_bits(), 0.5_f64.to_bits()); + } + + #[test] + fn preserves_minimum_subnormal_stationary_variance_after_drift_overflow() { + // Exact q / (-2a) lies above the half-ulp threshold for the minimum + // positive subnormal. Dividing by |a| first rounds to one subnormal; + // halving that rounded intermediate incorrectly erases the result. + let diffusion = 1.332_267_629_550_187_7e-15_f64; + let stationary = recover_stationary_within_variance(diffusion, -f64::MAX) + .expect("positive subnormal stationary variance is representable"); + assert_eq!(stationary.to_bits(), 1); + } + + #[test] + fn overflow_fallback_does_not_double_round_a_minimum_subnormal_result() { + // The exact q / (-2a) rounds to the minimum positive subnormal. The + // predecessor fallback rounded once during division and again during + // its final quarter-scale, returning two subnormal ulps instead. + let diffusion = f64::from_bits(0x3cda_d6b3_492a_639e); + let log_rate = -f64::from_bits(0x7fe2_342c_9564_2bec); + let stationary = recover_stationary_within_variance(diffusion, log_rate) + .expect("the final stationary variance is representable"); + assert_eq!(stationary.to_bits(), 1); + } + + #[test] + fn positive_stationary_variance_below_binary64_range_fails_closed() { + let minimum_subnormal = f64::from_bits(1); + assert_eq!( + recover_stationary_within_variance(minimum_subnormal, -1.0), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + } + + #[test] + fn stationary_variance_admission_is_fail_closed() { + assert_eq!(recover_stationary_within_variance(0.0, -0.5), Ok(0.0)); + assert_eq!( + recover_stationary_within_variance(1.0, 0.0), + Err(LongitudinalError::StationaryVarianceRequiresStableDrift) + ); + assert_eq!( + recover_stationary_within_variance(-1.0, -0.5), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + } + + #[test] + fn overflowed_twice_rate_rejects_underflowed_stationary_variance() { + assert_eq!( + recover_stationary_within_variance(f64::from_bits(1), -f64::MAX), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + } + + #[test] + fn overflowed_stationary_variance_fails_closed() { + assert_eq!( + recover_stationary_within_variance(f64::MAX, -f64::from_bits(1)), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + } +} diff --git a/crates/longitudinal_core/src/temporal_association.rs b/crates/longitudinal_core/src/temporal_association.rs new file mode 100644 index 000000000..b6aa1eae5 --- /dev/null +++ b/crates/longitudinal_core/src/temporal_association.rs @@ -0,0 +1,84 @@ +//! Typed public boundary for event-time lagged association. + +use crate::{EventTimeInterval, LongitudinalError, association}; + +/// Recover a Pearson correlation for a substantive event-time lag. +/// +/// The caller supplies the lagged covariance and both occasion-specific +/// marginal variances. [`EventTimeInterval`] makes the public clock ownership +/// explicit; assertion-, document-, system-, availability-, or method-occasion +/// intervals cannot enter this API as bare numeric durations. +/// +/// Exact binary64 Cauchy–Schwarz comparison authorizes both admissibility and +/// the scientific ±1 endpoint. Rounded square-root/division arithmetic may not +/// promote a strict interior covariance to a perfect-correlation claim. +/// +/// Exact zero covariance has one public scientific encoding: `+0.0`. IEEE-754 +/// signed zero is an arithmetic representation detail, not evidence of a +/// directional association, so the projection boundary canonicalizes either +/// zero sign after the numerical primitive has completed validation. +/// +/// # Errors +/// +/// Returns [`LongitudinalError::InvalidTemporalAssociationInput`] for invalid +/// covariance or marginal inputs, when a nonzero exact correlation is too +/// small to be represented as binary64, or when a strict interior covariance +/// would round to a false exact ±1 correlation, +/// [`LongitudinalError::NonPositiveMarginalVariance`] when either marginal +/// variance is non-positive, and [`LongitudinalError::CovarianceBoundViolation`] +/// when the covariance exceeds the exact binary64 Cauchy–Schwarz bound. +pub fn recover_event_time_lagged_correlation( + lagged_covariance: f64, + earlier_total_variance: f64, + later_total_variance: f64, + event_interval: EventTimeInterval, +) -> Result { + let correlation = association::recover_event_time_lagged_correlation( + lagged_covariance, + earlier_total_variance, + later_total_variance, + event_interval, + )?; + + if correlation == 0.0 { + return Ok(0.0); + } + + Ok(correlation) +} + +#[cfg(test)] +mod tests { + use super::recover_event_time_lagged_correlation; + use crate::{EventTimeInterval, LongitudinalError}; + + #[test] + fn public_boundary_requires_admitted_event_time() { + let interval = EventTimeInterval::new(1.0).expect("valid event time"); + assert_eq!( + recover_event_time_lagged_correlation(2.0, 1.0, 4.0, interval), + Ok(1.0) + ); + } + + #[test] + fn public_boundary_does_not_report_underflowed_nonzero_correlation_as_zero() { + let interval = EventTimeInterval::new(1.0).expect("valid event time"); + assert_eq!( + recover_event_time_lagged_correlation(f64::from_bits(1), f64::MAX, f64::MAX, interval,), + Err(LongitudinalError::InvalidTemporalAssociationInput) + ); + } + + #[test] + fn wrong_clock_shaped_numeric_values_fail_at_value_object_admission() { + assert_eq!( + EventTimeInterval::new(0.0), + Err(LongitudinalError::NonPositiveEventInterval) + ); + assert_eq!( + EventTimeInterval::new(f64::NAN), + Err(LongitudinalError::NonPositiveEventInterval) + ); + } +} diff --git a/crates/longitudinal_core/tests/component_rmse_duplicate_identity_contract.rs b/crates/longitudinal_core/tests/component_rmse_duplicate_identity_contract.rs new file mode 100644 index 000000000..9973047ec --- /dev/null +++ b/crates/longitudinal_core/tests/component_rmse_duplicate_identity_contract.rs @@ -0,0 +1,45 @@ +//! Known-truth recovery denominators require unique component identities. + +use longitudinal_core::{ + ComponentLevel, ComponentValue, LongitudinalError, component_root_mean_square_error, +}; + +#[test] +fn duplicate_component_identity_cannot_reweight_known_truth_rmse() { + let truth = [ + ComponentValue::new(0, 0, ComponentLevel::Between, 1.0), + ComponentValue::new(0, 0, ComponentLevel::Between, 1.0), + ComponentValue::new(1, 0, ComponentLevel::Between, 3.0), + ]; + let decided = [ + ComponentValue::new(0, 0, ComponentLevel::Between, 1.0), + ComponentValue::new(0, 0, ComponentLevel::Between, 1.0), + ComponentValue::new(1, 0, ComponentLevel::Between, 5.0), + ]; + + // The duplicate (unit, occasion, level) would count the zero-error unit + // twice and silently lower the RMSE denominator from the unique-component + // target. Recovery evidence must fail closed instead of changing weight by + // duplicate identity multiplicity. + assert_eq!( + component_root_mean_square_error(&truth, &decided), + Err(LongitudinalError::InvalidComponentPayload) + ); +} + +#[test] +fn duplicate_truth_identity_fails_even_when_recovered_identities_are_unique() { + let truth = [ + ComponentValue::new(0, 0, ComponentLevel::Between, 1.0), + ComponentValue::new(0, 0, ComponentLevel::Between, 1.0), + ]; + let decided = [ + ComponentValue::new(0, 0, ComponentLevel::Between, 1.0), + ComponentValue::new(1, 0, ComponentLevel::Between, 1.0), + ]; + + assert_eq!( + component_root_mean_square_error(&truth, &decided), + Err(LongitudinalError::InvalidComponentPayload) + ); +} diff --git a/crates/longitudinal_core/tests/component_rmse_permutation_identity_contract.rs b/crates/longitudinal_core/tests/component_rmse_permutation_identity_contract.rs new file mode 100644 index 000000000..c12dd64b6 --- /dev/null +++ b/crates/longitudinal_core/tests/component_rmse_permutation_identity_contract.rs @@ -0,0 +1,22 @@ +//! Known-truth component RMSE is invariant to recovered-row transport order. +//! +//! Recovered components are matched by scientific identity rather than slice +//! position, so a transport permutation cannot change admission or RMSE. + +use longitudinal_core::{ComponentLevel, ComponentValue, component_root_mean_square_error}; + +#[test] +fn component_rmse_aligns_by_scientific_identity_not_row_position() { + let truth = [ + ComponentValue::new(7, 0, ComponentLevel::Between, 1.25), + ComponentValue::new(7, 3, ComponentLevel::Within, -0.5), + ComponentValue::new(11, 0, ComponentLevel::Between, 4.0), + ]; + let decided = [ + ComponentValue::new(11, 0, ComponentLevel::Between, 4.0), + ComponentValue::new(7, 0, ComponentLevel::Between, 1.25), + ComponentValue::new(7, 3, ComponentLevel::Within, -0.5), + ]; + + assert_eq!(component_root_mean_square_error(&truth, &decided), Ok(0.0)); +} diff --git a/crates/longitudinal_core/tests/component_rmse_stable_between_identity_contract.rs b/crates/longitudinal_core/tests/component_rmse_stable_between_identity_contract.rs new file mode 100644 index 000000000..cdb44e7a2 --- /dev/null +++ b/crates/longitudinal_core/tests/component_rmse_stable_between_identity_contract.rs @@ -0,0 +1,27 @@ +//! Stable Between components keep a canonical occasion identity of zero. +//! +//! Longitudinal decomposition emits one Between component per unit. An occasion +//! alias must not become a second stable target or an implicit RMSE weight. + +use longitudinal_core::{ + ComponentLevel, ComponentValue, LongitudinalError, component_root_mean_square_error, +}; + +#[test] +fn stable_between_component_cannot_gain_weight_from_occasion_aliases() { + let truth = [ + ComponentValue::new(7, 0, ComponentLevel::Between, 1.25), + ComponentValue::new(7, 2, ComponentLevel::Between, 1.25), + ComponentValue::new(11, 1, ComponentLevel::Within, 0.0), + ]; + let decided = [ + ComponentValue::new(7, 0, ComponentLevel::Between, 1.25), + ComponentValue::new(7, 2, ComponentLevel::Between, 1.25), + ComponentValue::new(11, 1, ComponentLevel::Within, 3.0), + ]; + + assert_eq!( + component_root_mean_square_error(&truth, &decided), + Err(LongitudinalError::InvalidComponentPayload) + ); +} diff --git a/crates/longitudinal_core/tests/component_rmse_truth_permutation_contract.rs b/crates/longitudinal_core/tests/component_rmse_truth_permutation_contract.rs new file mode 100644 index 000000000..b9150da95 --- /dev/null +++ b/crates/longitudinal_core/tests/component_rmse_truth_permutation_contract.rs @@ -0,0 +1,34 @@ +//! Known-truth component RMSE is invariant to truth-row serialization order. +//! +//! Residuals accumulate in canonical scientific identity order, so permuting +//! the truth slice cannot change the deterministic binary64 recovery metric. + +use longitudinal_core::{ComponentLevel, ComponentValue, component_root_mean_square_error}; + +#[test] +fn component_rmse_is_bit_identical_under_truth_row_permutation() { + let just_below_one = f64::from_bits(1.0_f64.to_bits() - 1); + let truth_a = [ + ComponentValue::new(0, 0, ComponentLevel::Between, 0.0), + ComponentValue::new(1, 0, ComponentLevel::Between, 0.0), + ComponentValue::new(2, 0, ComponentLevel::Between, 0.0), + ComponentValue::new(3, 0, ComponentLevel::Between, 0.0), + ]; + let truth_b = [ + ComponentValue::new(0, 0, ComponentLevel::Between, 0.0), + ComponentValue::new(1, 0, ComponentLevel::Between, 0.0), + ComponentValue::new(3, 0, ComponentLevel::Between, 0.0), + ComponentValue::new(2, 0, ComponentLevel::Between, 0.0), + ]; + let decided = [ + ComponentValue::new(0, 0, ComponentLevel::Between, 1.0), + ComponentValue::new(1, 0, ComponentLevel::Between, 1e-100), + ComponentValue::new(2, 0, ComponentLevel::Between, 3.0), + ComponentValue::new(3, 0, ComponentLevel::Between, just_below_one), + ]; + + let first = component_root_mean_square_error(&truth_a, &decided).expect("first RMSE"); + let permuted = component_root_mean_square_error(&truth_b, &decided).expect("permuted RMSE"); + + assert_eq!(first.to_bits(), permuted.to_bits()); +} diff --git a/crates/longitudinal_core/tests/component_rmse_underflow_contract.rs b/crates/longitudinal_core/tests/component_rmse_underflow_contract.rs new file mode 100644 index 000000000..89ab9196c --- /dev/null +++ b/crates/longitudinal_core/tests/component_rmse_underflow_contract.rs @@ -0,0 +1,29 @@ +//! Nonzero known-truth RMSE must not underflow to false perfect recovery. + +use longitudinal_core::{ + ComponentLevel, ComponentValue, LongitudinalError, component_root_mean_square_error, +}; + +#[test] +fn nonzero_recovery_error_cannot_collapse_to_perfect_rmse() { + let truth = [ + ComponentValue::new(0, 0, ComponentLevel::Within, 0.0), + ComponentValue::new(1, 0, ComponentLevel::Within, 0.0), + ComponentValue::new(2, 0, ComponentLevel::Within, 0.0), + ComponentValue::new(3, 0, ComponentLevel::Within, 0.0), + ComponentValue::new(4, 0, ComponentLevel::Within, 0.0), + ]; + let recovered = [ + ComponentValue::new(0, 0, ComponentLevel::Within, f64::from_bits(1)), + ComponentValue::new(1, 0, ComponentLevel::Within, 0.0), + ComponentValue::new(2, 0, ComponentLevel::Within, 0.0), + ComponentValue::new(3, 0, ComponentLevel::Within, 0.0), + ComponentValue::new(4, 0, ComponentLevel::Within, 0.0), + ]; + + assert_eq!( + component_root_mean_square_error(&truth, &recovered), + Err(LongitudinalError::InvalidComponentPayload), + "a nonzero exact recovery error whose RMSE is below binary64 range must fail closed rather than report perfect recovery", + ); +} diff --git a/crates/longitudinal_core/tests/correlation_false_perfect_contract.rs b/crates/longitudinal_core/tests/correlation_false_perfect_contract.rs new file mode 100644 index 000000000..a43c49a79 --- /dev/null +++ b/crates/longitudinal_core/tests/correlation_false_perfect_contract.rs @@ -0,0 +1,61 @@ +//! Strict-interior covariance cannot become a false exact ±1 correlation. + +use longitudinal_core::{ + EventTimeInterval, LongitudinalError, recover_event_time_lagged_correlation, +}; + +#[test] +fn strict_binary_covariance_bound_cannot_round_to_perfect_correlation() { + let earlier_variance = f64::from_bits(4_607_182_418_800_016_408); + let later_variance = f64::from_bits(4_607_182_418_800_016_427); + let covariance = f64::from_bits(4_607_182_418_800_016_417); + let interval = EventTimeInterval::new(1.0).expect("unit event-time interval"); + + // For these exact binary64 inputs covariance² is strictly below + // earlier_variance * later_variance, but rounded sqrt/division arithmetic + // produces ±1.0. Reporting either endpoint as perfect association would + // strengthen the scientific claim beyond the supplied covariance evidence. + for signed_covariance in [covariance, -covariance] { + assert_eq!( + recover_event_time_lagged_correlation( + signed_covariance, + earlier_variance, + later_variance, + interval, + ), + Err(LongitudinalError::InvalidTemporalAssociationInput) + ); + } +} + +#[test] +fn exact_binary_covariance_boundary_cannot_round_below_perfect_correlation() { + let earlier_variance = 2.0_f64; + let later_variance = 8.0_f64; + let covariance = 4.0_f64; + let interval = EventTimeInterval::new(1.0).expect("unit event-time interval"); + + // These represented inputs satisfy covariance² == earlier_variance * + // later_variance exactly. Rounded square roots make the naive two-step + // standardization equal to 0x1.fffffffffffffp-1 instead of 1.0. The exact + // binary covariance relation therefore has to remain authoritative for + // the endpoint in both sign directions. + assert_eq!( + recover_event_time_lagged_correlation( + covariance, + earlier_variance, + later_variance, + interval, + ), + Ok(1.0) + ); + assert_eq!( + recover_event_time_lagged_correlation( + -covariance, + earlier_variance, + later_variance, + interval, + ), + Ok(-1.0) + ); +} diff --git a/crates/longitudinal_core/tests/correlation_signed_zero_contract.rs b/crates/longitudinal_core/tests/correlation_signed_zero_contract.rs new file mode 100644 index 000000000..8666d14ce --- /dev/null +++ b/crates/longitudinal_core/tests/correlation_signed_zero_contract.rs @@ -0,0 +1,16 @@ +//! Exact-zero lagged covariance has one public no-association encoding. + +use longitudinal_core::{EventTimeInterval, recover_event_time_lagged_correlation}; + +#[test] +fn exact_zero_covariance_has_one_canonical_no_association_encoding() { + let interval = EventTimeInterval::new(1.0).expect("unit event-time interval"); + + let positive_zero = recover_event_time_lagged_correlation(0.0, 1.0, 4.0, interval) + .expect("exact zero covariance is a valid no-association result"); + let negative_zero = recover_event_time_lagged_correlation(-0.0, 1.0, 4.0, interval) + .expect("signed-zero covariance is the same exact no-association result"); + + assert_eq!(positive_zero.to_bits(), 0.0_f64.to_bits()); + assert_eq!(negative_zero.to_bits(), 0.0_f64.to_bits()); +} diff --git a/crates/longitudinal_core/tests/cwc_irregular_residual_contract.rs b/crates/longitudinal_core/tests/cwc_irregular_residual_contract.rs new file mode 100644 index 000000000..4cf657cc8 --- /dev/null +++ b/crates/longitudinal_core/tests/cwc_irregular_residual_contract.rs @@ -0,0 +1,133 @@ +//! CWC-then-irregular residual log-rate is not raw-process drift. + +use longitudinal_core::{ + EventTimedObservation, LaggedWithinResidual, LongitudinalError, center_within_unit_event_lags, + recover_centered_irregular_residual_log_rate, recover_within_unit_irregular_residual_log_rate, + refuse_cwc_residual_log_rate_as_raw_process_drift, +}; + +fn timed(unit: u32, event_time: f64, score: f64) -> EventTimedObservation { + EventTimedObservation::new(unit, event_time, score) +} + +#[test] +fn already_centered_unit_interval_recovers_ln_half() { + let pair = LaggedWithinResidual::new( + 1.0, + 0.5, + longitudinal_core::EventTimeInterval::new(1.0).expect("event time"), + ); + let recovered = recover_centered_irregular_residual_log_rate(&[pair]).expect("ln(0.5)"); + assert!((recovered - 0.5_f64.ln()).abs() < f64::EPSILON); +} + +#[test] +fn centered_log_rate_survives_nonrepresentable_intermediate_ratios() { + let interval = longitudinal_core::EventTimeInterval::new(1.0).expect("event time"); + let overflow_pair = LaggedWithinResidual::new(f64::MIN_POSITIVE, f64::MAX, interval); + assert!(!(f64::MAX / f64::MIN_POSITIVE).is_finite()); + let overflow_expected = f64::MAX.ln() - f64::MIN_POSITIVE.ln(); + let overflow_recovered = recover_centered_irregular_residual_log_rate(&[overflow_pair]) + .expect("finite log-domain rate despite overflowing direct ratio"); + assert_eq!(overflow_recovered.to_bits(), overflow_expected.to_bits()); + + let underflow_pair = LaggedWithinResidual::new(f64::MAX, f64::MIN_POSITIVE, interval); + assert_eq!((f64::MIN_POSITIVE / f64::MAX).to_bits(), 0.0_f64.to_bits()); + let underflow_expected = f64::MIN_POSITIVE.ln() - f64::MAX.ln(); + let underflow_recovered = recover_centered_irregular_residual_log_rate(&[underflow_pair]) + .expect("finite log-domain rate despite underflowing direct ratio"); + assert_eq!(underflow_recovered.to_bits(), underflow_expected.to_bits()); +} + +#[test] +fn cwc_mean_is_deterministic_under_input_row_permutation() { + let canonical = [ + timed(1, 0.0, 1.0e16), + timed(1, 1.0, -1.0e16), + timed(1, 2.0, 1.0), + timed(2, 0.0, 2.0), + timed(2, 1.0, 1.0), + ]; + let shuffled = [ + timed(1, 0.0, 1.0e16), + timed(1, 2.0, 1.0), + timed(1, 1.0, -1.0e16), + timed(2, 1.0, 1.0), + timed(2, 0.0, 2.0), + ]; + + let expected = center_within_unit_event_lags(&canonical).expect("canonical ordering"); + let reordered = center_within_unit_event_lags(&shuffled).expect("shuffled ordering"); + assert_eq!( + reordered, expected, + "CWC output must not depend on input row order" + ); +} + +#[test] +fn cwc_requires_two_units_with_repeated_event_time_evidence() { + let rows = [ + timed(1, 0.0, 11.0), + timed(1, 1.0, 10.8), + timed(1, 2.0, 10.6), + timed(2, 0.0, 4.0), + ]; + assert_eq!( + center_within_unit_event_lags(&rows), + Err(LongitudinalError::InvalidObservationPayload), + "a singleton second unit must not make a one-unit longitudinal sample look multilevel" + ); + assert_eq!( + recover_within_unit_irregular_residual_log_rate(&rows), + Err(LongitudinalError::InvalidObservationPayload), + "CWC recovery requires at least two lag-contributing units" + ); +} + +#[test] +fn cwc_mean_survives_overflowing_raw_sum_when_centered_values_are_finite() { + let rows = [ + timed(1, 0.0, f64::MAX * 0.75), + timed(1, 1.0, f64::MAX * 0.75), + timed(1, 2.0, -f64::MAX * 0.5), + timed(2, 0.0, 2.0), + timed(2, 1.0, 1.0), + ]; + assert!( + (rows[0].score() + rows[1].score()).is_infinite(), + "fixture must overflow naive same-order summation" + ); + + let pairs = center_within_unit_event_lags(&rows) + .expect("finite CWC residuals must not be rejected because a raw sum overflows"); + assert_eq!(pairs.len(), 3); + assert!( + pairs.iter().all(|pair| { + pair.earlier_residual().is_finite() && pair.later_residual().is_finite() + }) + ); +} + +#[test] +fn cwc_of_raw_autoregressive_path_is_not_process_drift() { + let drift = (0.5_f64).ln(); + let rows = [ + timed(0, 0.0, 8.0 + 1.0), + timed(0, 1.0, 8.0 + drift.exp()), + timed(0, 2.5, 8.0 + (drift * 2.5).exp()), + timed(1, 0.0, 3.0 + 1.0), + timed(1, 0.8, 3.0 + (drift * 0.8).exp()), + timed(1, 2.0, 3.0 + (drift * 2.0).exp()), + ]; + let composed = recover_within_unit_irregular_residual_log_rate(&rows).expect("cwc pairwise"); + assert!( + (composed - drift).abs() > 1e-4, + "Curran & Bauer: CWC of a time-related AR path must not recover raw drift" + ); + assert_eq!( + refuse_cwc_residual_log_rate_as_raw_process_drift(composed, drift), + Err(LongitudinalError::CwcResidualLogRateIsNotRawProcessDrift) + ); + let extracted = center_within_unit_event_lags(&rows).expect("pairs"); + assert!(extracted.len() >= 2); +} diff --git a/crates/longitudinal_core/tests/cwc_signed_zero_contract.rs b/crates/longitudinal_core/tests/cwc_signed_zero_contract.rs new file mode 100644 index 000000000..8571b0a81 --- /dev/null +++ b/crates/longitudinal_core/tests/cwc_signed_zero_contract.rs @@ -0,0 +1,30 @@ +//! Exact-zero CWC residuals have one public encoding. + +use longitudinal_core::{EventTimedObservation, center_within_unit_event_lags}; + +fn timed(unit: u32, event_time: f64, score: f64) -> EventTimedObservation { + EventTimedObservation::new(unit, event_time, score) +} + +#[test] +fn exact_zero_cwc_residuals_use_one_canonical_public_encoding() { + let pairs = center_within_unit_event_lags(&[ + timed(0, 0.0, -1.0), + timed(0, 1.0, -0.0), + timed(0, 2.0, 1.0), + timed(1, 0.0, 4.0), + timed(1, 1.0, 4.0), + timed(1, 2.0, 4.0), + ]) + .expect("finite repeated observations admit CWC lags"); + + assert_eq!(pairs.len(), 4); + assert_eq!(pairs[0].earlier_residual().to_bits(), (-1.0_f64).to_bits()); + assert_eq!(pairs[0].later_residual().to_bits(), 0.0_f64.to_bits()); + assert_eq!(pairs[1].earlier_residual().to_bits(), 0.0_f64.to_bits()); + assert_eq!(pairs[1].later_residual().to_bits(), 1.0_f64.to_bits()); + for pair in &pairs[2..] { + assert_eq!(pair.earlier_residual().to_bits(), 0.0_f64.to_bits()); + assert_eq!(pair.later_residual().to_bits(), 0.0_f64.to_bits()); + } +} diff --git a/crates/longitudinal_core/tests/decomposition_mean_rounding_contract.rs b/crates/longitudinal_core/tests/decomposition_mean_rounding_contract.rs new file mode 100644 index 000000000..21fb16567 --- /dev/null +++ b/crates/longitudinal_core/tests/decomposition_mean_rounding_contract.rs @@ -0,0 +1,22 @@ +//! Unit-mean decomposition uses the shared compensated-mean authority. + +use longitudinal_core::{ComponentLevel, OccasionObservation, decompose_within_between}; + +#[test] +fn decomposition_unit_mean_preserves_subnormal_ties_to_even() { + let ulp = f64::from_bits(1); + let recovered = decompose_within_between(&[ + OccasionObservation::new(0, 0, ulp), + OccasionObservation::new(0, 1, f64::from_bits(2)), + OccasionObservation::new(1, 0, 0.0), + OccasionObservation::new(1, 1, 0.0), + ]) + .expect("the representable unit mean must survive binary64 halfway rounding"); + + assert_eq!(recovered[0].level(), ComponentLevel::Between); + assert_eq!(recovered[0].value().to_bits(), f64::from_bits(2).to_bits()); + assert_eq!(recovered[1].level(), ComponentLevel::Within); + assert_eq!(recovered[1].value().to_bits(), (-ulp).to_bits()); + assert_eq!(recovered[2].level(), ComponentLevel::Within); + assert_eq!(recovered[2].value().to_bits(), 0.0_f64.to_bits()); +} diff --git a/crates/longitudinal_core/tests/decomposition_signed_zero_contract.rs b/crates/longitudinal_core/tests/decomposition_signed_zero_contract.rs new file mode 100644 index 000000000..a14f6d522 --- /dev/null +++ b/crates/longitudinal_core/tests/decomposition_signed_zero_contract.rs @@ -0,0 +1,26 @@ +//! Exact-zero within residuals have one public encoding. + +use longitudinal_core::{ComponentLevel, OccasionObservation, decompose_within_between}; + +#[test] +fn exact_zero_within_residuals_use_one_canonical_public_encoding() { + let recovered = decompose_within_between(&[ + OccasionObservation::new(0, 0, -0.0), + OccasionObservation::new(0, 1, 0.0), + OccasionObservation::new(1, 0, 1.0), + OccasionObservation::new(1, 1, 1.0), + ]) + .expect("finite repeated observations admit within/between decomposition"); + + let unit_zero_within: Vec<_> = recovered + .iter() + .filter(|component| { + component.unit_index() == 0 && component.level() == ComponentLevel::Within + }) + .collect(); + + assert_eq!(unit_zero_within.len(), 2); + for component in unit_zero_within { + assert_eq!(component.value().to_bits(), 0.0_f64.to_bits()); + } +} diff --git a/crates/longitudinal_core/tests/intermediate_overflow_regression.rs b/crates/longitudinal_core/tests/intermediate_overflow_regression.rs new file mode 100644 index 000000000..fa66edf51 --- /dev/null +++ b/crates/longitudinal_core/tests/intermediate_overflow_regression.rs @@ -0,0 +1,30 @@ +//! Finite log-rate pairs must survive overflowing raw ratio intermediates. + +use longitudinal_core::{ + EventTimeInterval, LaggedWithinResidual, recover_centered_irregular_residual_log_rate, +}; + +#[test] +fn representable_mixed_sign_mean_survives_retained_mass_intermediate_overflow() { + let target_positive_rate = 1.45e308_f64; + let extreme_interval = + EventTimeInterval::new(1.0 / target_positive_rate).expect("positive extreme interval"); + let unit_interval = EventTimeInterval::new(1.0).expect("unit interval"); + + let pairs = [ + LaggedWithinResidual::new(1.0, std::f64::consts::E, extreme_interval), + LaggedWithinResidual::new(1.0, std::f64::consts::E, extreme_interval), + LaggedWithinResidual::new(std::f64::consts::E, 1.0, unit_interval), + LaggedWithinResidual::new(std::f64::consts::E, 1.0, unit_interval), + ]; + + let positive_rate = std::f64::consts::E.ln() / extreme_interval.as_f64(); + assert!(positive_rate.is_finite()); + assert!(!(positive_rate * 2.0).is_finite()); + let expected = positive_rate / 2.0; + + let recovered = recover_centered_irregular_residual_log_rate(&pairs) + .expect("finite mathematical mean must not fail on intermediate overflow"); + assert!(recovered.is_finite()); + assert!((recovered - expected).abs() <= expected.abs() * 4.0 * f64::EPSILON); +} diff --git a/crates/longitudinal_core/tests/irregular_rate_adjacent_growth_contract.rs b/crates/longitudinal_core/tests/irregular_rate_adjacent_growth_contract.rs new file mode 100644 index 000000000..6e37928ea --- /dev/null +++ b/crates/longitudinal_core/tests/irregular_rate_adjacent_growth_contract.rs @@ -0,0 +1,33 @@ +#![forbid(unsafe_code)] + +//! Regression contract for adjacent-float irregular residual growth. +//! +//! Driver et al. (2017, Eq. 3) requires the logarithm of the exact represented +//! magnitude ratio. Forming that ratio first can round an adjacent-float growth +//! step at a power-of-two boundary from roughly half an epsilon to one epsilon, +//! doubling the recovered log-rate before the logarithm is evaluated. + +use longitudinal_core::{ + EventTimeInterval, LaggedWithinResidual, recover_centered_irregular_residual_log_rate, +}; + +#[test] +fn adjacent_power_of_two_growth_keeps_the_represented_relative_change() { + let earlier = f64::from_bits(2.0_f64.to_bits() - 1); + let later = 2.0_f64; + let interval = EventTimeInterval::new(1.0).expect("unit event-time interval"); + let pair = LaggedWithinResidual::new(earlier, later, interval); + + let exact_relative_change = f64::EPSILON / 2.0; + let expected = -(-exact_relative_change).ln_1p(); + let rounded_ratio_log = (later / earlier).ln(); + assert!( + rounded_ratio_log > expected * 1.9, + "the fixture must reproduce ratio-first double rounding" + ); + + let recovered = recover_centered_irregular_residual_log_rate(&[pair]) + .expect("adjacent represented growth must remain recoverable"); + + assert_eq!(recovered.to_bits(), expected.to_bits()); +} diff --git a/crates/longitudinal_core/tests/irregular_rate_estimand_contract.rs b/crates/longitudinal_core/tests/irregular_rate_estimand_contract.rs new file mode 100644 index 000000000..307886aee --- /dev/null +++ b/crates/longitudinal_core/tests/irregular_rate_estimand_contract.rs @@ -0,0 +1,276 @@ +//! Deterministic public contract for pair-weighted irregular-rate estimand identity. +//! +//! Realistic rate-associated missingness and Monte Carlo uncertainty are exercised +//! separately in `irregular_rate_monte_carlo_contract.rs`. This file keeps only +//! deterministic estimand, weighting, refusal, and fail-closed fixtures. + +use longitudinal_core::{ + EventTimedObservation, IrregularRateEstimand, LongitudinalError, + recover_within_unit_irregular_rate_summary, recover_within_unit_irregular_residual_log_rate, +}; + +fn timed(unit: u32, event_time: f64, score: f64) -> EventTimedObservation { + EventTimedObservation::new(unit, event_time, score) +} + +fn unequal_pair_count_rows() -> [EventTimedObservation; 7] { + [ + timed(1, 0.0, -3.0), + timed(1, 1.0, -2.0), + timed(1, 2.0, -1.0), + timed(1, 3.0, 6.0), + timed(2, 0.0, 3.0), + timed(2, 1.0, 1.0), + timed(2, 2.0, -4.0), + ] +} + +#[test] +fn lag_pair_average_reports_estimand_and_failure_denominators() { + let rows = unequal_pair_count_rows(); + let summary = + recover_within_unit_irregular_rate_summary(&rows, IrregularRateEstimand::LagPairAverageV1) + .expect("pair-average evidence"); + + assert_eq!( + summary.estimand().wire_name(), + "tepp.irregular_rate.lag_pair_average.v1" + ); + assert_eq!(summary.candidate_units(), 2); + assert_eq!(summary.contributing_units(), 2); + assert_eq!(summary.candidate_pairs(), 5); + assert_eq!(summary.admitted_pairs(), 3); + assert_eq!(summary.sign_or_zero_refused_pairs(), 2); + assert_eq!(summary.nonrepresentable_rate_refused_pairs(), 0); + assert_eq!(summary.refused_pairs(), 2); + assert_eq!( + summary.admitted_pairs() + summary.refused_pairs(), + summary.candidate_pairs() + ); + + let pair_average = summary.estimate().expect("three admitted rates"); + let legacy = recover_within_unit_irregular_residual_log_rate(&rows).expect("legacy pair mean"); + assert_eq!(pair_average.to_bits(), legacy.to_bits()); + + let unit_one = f64::midpoint((2.0_f64 / 3.0).ln(), (1.0_f64 / 2.0).ln()); + let unit_two = (1.0_f64 / 3.0).ln(); + let equal_unit_comparison = f64::midpoint(unit_one, unit_two); + assert!( + (pair_average - equal_unit_comparison).abs() > 1.0e-3, + "unequal admitted-pair counts must make pair and equal-unit targets distinguishable" + ); +} + +#[test] +fn extra_admitted_pair_changes_pair_weight_without_changing_unit_target() { + let base = unequal_pair_count_rows(); + let extended = [ + base[0], + base[1], + base[2], + base[3], + base[4], + base[5], + base[6], + timed(2, 3.0, -4.0 / 3.0), + timed(2, 4.0, 4.0 / 3.0), + ]; + + let base_summary = + recover_within_unit_irregular_rate_summary(&base, IrregularRateEstimand::LagPairAverageV1) + .expect("base pair-average evidence"); + let extended_summary = recover_within_unit_irregular_rate_summary( + &extended, + IrregularRateEstimand::LagPairAverageV1, + ) + .expect("extended pair-average evidence"); + + assert_eq!(base_summary.admitted_pairs(), 3); + assert_eq!(extended_summary.admitted_pairs(), 4); + assert_eq!(base_summary.candidate_pairs(), 5); + assert_eq!(extended_summary.candidate_pairs(), 7); + assert_eq!(base_summary.refused_pairs(), 2); + assert_eq!(extended_summary.refused_pairs(), 3); + + let base_pair_average = base_summary.estimate().expect("base estimate"); + let extended_pair_average = extended_summary.estimate().expect("extended estimate"); + assert!( + extended_pair_average < base_pair_average, + "an extra admitted ln(1/3) rate must shift the pair-weighted target toward unit two" + ); + assert!( + (extended_pair_average - base_pair_average).abs() > 0.05, + "the multiplicity perturbation must remain scientifically visible" + ); + + let unit_one = f64::midpoint((2.0_f64 / 3.0).ln(), (1.0_f64 / 2.0).ln()); + let unit_two_base = (1.0_f64 / 3.0).ln(); + let unit_two_extended = f64::midpoint(unit_two_base, unit_two_base); + assert_eq!(unit_two_extended.to_bits(), unit_two_base.to_bits()); + + let equal_unit_base = f64::midpoint(unit_one, unit_two_base); + let equal_unit_extended = f64::midpoint(unit_one, unit_two_extended); + assert_eq!(equal_unit_extended.to_bits(), equal_unit_base.to_bits()); +} + +#[test] +fn balanced_follow_up_recovers_declared_pair_weighted_truth() { + let rows = [ + timed(1, 0.0, -8.0), + timed(1, 1.0, -4.0), + timed(1, 2.0, -2.0), + timed(1, 3.0, 14.0), + timed(2, 0.0, -27.0), + timed(2, 1.0, -9.0), + timed(2, 2.0, -3.0), + timed(2, 3.0, 39.0), + ]; + let summary = + recover_within_unit_irregular_rate_summary(&rows, IrregularRateEstimand::LagPairAverageV1) + .expect("balanced known-truth evidence"); + + assert_eq!(summary.candidate_units(), 2); + assert_eq!(summary.contributing_units(), 2); + assert_eq!(summary.candidate_pairs(), 6); + assert_eq!(summary.admitted_pairs(), 4); + assert_eq!(summary.refused_pairs(), 2); + + let half_rate = 0.5_f64.ln(); + let third_rate = (1.0_f64 / 3.0).ln(); + let expected = f64::midpoint(half_rate, third_rate); + let actual = summary.estimate().expect("balanced pair target"); + assert!( + (actual - expected).abs() <= 8.0 * f64::EPSILON, + "balanced follow-up must recover the equal pair-weighted known truth" + ); +} + +#[test] +fn rate_associated_highly_unbalanced_follow_up_recovers_pair_weighted_truth() { + let mut rows = Vec::with_capacity(21); + let mut event_time = 0.0; + for exponent in (1..=16).rev() { + rows.push(timed(1, event_time, -2.0_f64.powi(exponent))); + event_time += 1.0; + } + rows.push(timed(1, event_time, 131_070.0)); + rows.extend([ + timed(2, 0.0, -27.0), + timed(2, 1.0, -9.0), + timed(2, 2.0, -3.0), + timed(2, 3.0, 39.0), + ]); + + let summary = + recover_within_unit_irregular_rate_summary(&rows, IrregularRateEstimand::LagPairAverageV1) + .expect("rate-associated follow-up evidence"); + + assert_eq!(summary.candidate_units(), 2); + assert_eq!(summary.contributing_units(), 2); + assert_eq!(summary.candidate_pairs(), 19); + assert_eq!(summary.admitted_pairs(), 17); + assert_eq!(summary.refused_pairs(), 2); + + let half_rate = 0.5_f64.ln(); + let third_rate = (1.0_f64 / 3.0).ln(); + let expected_pair_target = (15.0 * half_rate + 2.0 * third_rate) / 17.0; + let equal_unit_target = f64::midpoint(half_rate, third_rate); + let actual = summary.estimate().expect("unbalanced pair target"); + + assert!( + (actual - expected_pair_target).abs() <= 16.0 * f64::EPSILON, + "declared pair weighting must recover the known 15:2 follow-up target" + ); + assert!( + (actual - equal_unit_target).abs() > 0.1, + "rate-associated follow-up must not silently masquerade as equal-unit weighting" + ); +} + +#[test] +fn lag_pair_average_is_invariant_to_input_row_permutation() { + let canonical = unequal_pair_count_rows(); + let shuffled = [ + canonical[5], + canonical[2], + canonical[6], + canonical[0], + canonical[3], + canonical[4], + canonical[1], + ]; + + let expected = recover_within_unit_irregular_rate_summary( + &canonical, + IrregularRateEstimand::LagPairAverageV1, + ) + .expect("canonical"); + let reordered = recover_within_unit_irregular_rate_summary( + &shuffled, + IrregularRateEstimand::LagPairAverageV1, + ) + .expect("shuffled"); + + assert_eq!(reordered, expected); +} + +#[test] +fn zero_admissible_rates_still_return_explicit_pair_denominator() { + let rows = [ + timed(1, 0.0, 1.0), + timed(1, 1.0, -1.0), + timed(2, 0.0, 2.0), + timed(2, 1.0, -2.0), + ]; + let summary = + recover_within_unit_irregular_rate_summary(&rows, IrregularRateEstimand::LagPairAverageV1) + .expect("denominator evidence survives zero admissible rates"); + + assert_eq!(summary.estimate(), None); + assert_eq!(summary.candidate_units(), 2); + assert_eq!(summary.contributing_units(), 0); + assert_eq!(summary.candidate_pairs(), 2); + assert_eq!(summary.admitted_pairs(), 0); + assert_eq!(summary.sign_or_zero_refused_pairs(), 2); + assert_eq!(summary.nonrepresentable_rate_refused_pairs(), 0); + assert_eq!(summary.refused_pairs(), 2); +} + +#[test] +fn nonrepresentable_same_sign_rate_has_its_own_failure_denominator() { + let adjacent_one = f64::from_bits(1.0_f64.to_bits() + 1); + let rows = [ + timed(1, -f64::MAX, 1.0), + timed(1, 0.0, adjacent_one), + timed(1, f64::MAX, -2.0), + timed(2, 0.0, 3.0), + timed(2, 1.0, 1.0), + timed(2, 2.0, -4.0), + ]; + + let summary = + recover_within_unit_irregular_rate_summary(&rows, IrregularRateEstimand::LagPairAverageV1) + .expect("pair-level transform refusal remains reportable evidence"); + + assert_eq!(summary.candidate_units(), 2); + assert_eq!(summary.candidate_pairs(), 4); + assert_eq!(summary.admitted_pairs(), 1); + assert_eq!(summary.sign_or_zero_refused_pairs(), 2); + assert_eq!(summary.nonrepresentable_rate_refused_pairs(), 1); + assert_eq!(summary.refused_pairs(), 3); + assert_eq!(summary.contributing_units(), 1); + assert!(summary.estimate().is_some()); +} + +#[test] +fn unit_average_is_versioned_but_fails_closed_until_owner_mean_release() { + let rows = unequal_pair_count_rows(); + assert_eq!( + IrregularRateEstimand::UnitAverageV1.wire_name(), + "tepp.irregular_rate.unit_average.v1" + ); + assert_eq!( + recover_within_unit_irregular_rate_summary(&rows, IrregularRateEstimand::UnitAverageV1), + Err(LongitudinalError::IrregularRateEstimandUnavailable) + ); +} diff --git a/crates/longitudinal_core/tests/irregular_rate_mean_underflow_contract.rs b/crates/longitudinal_core/tests/irregular_rate_mean_underflow_contract.rs new file mode 100644 index 000000000..63f340079 --- /dev/null +++ b/crates/longitudinal_core/tests/irregular_rate_mean_underflow_contract.rs @@ -0,0 +1,28 @@ +//! A nonzero mean irregular-residual log-rate must not collapse to exact zero. + +use longitudinal_core::{ + EventTimeInterval, LaggedWithinResidual, LongitudinalError, + recover_centered_irregular_residual_log_rate, +}; + +#[test] +fn nonzero_mean_log_rate_that_is_not_binary64_representable_fails_closed() { + let tiny_interval = EventTimeInterval::new(1.0e307).expect("finite positive event interval"); + let unit_interval = EventTimeInterval::new(1.0).expect("unit interval"); + let earlier = 1.0_f64; + let later = f64::from_bits(earlier.to_bits() + 1); + let nonzero_pair = LaggedWithinResidual::new(earlier, later, tiny_interval); + + let one_pair_rate = recover_centered_irregular_residual_log_rate(&[nonzero_pair]) + .expect("the individual positive log-rate is representable"); + assert!(one_pair_rate > 0.0); + + let mut pairs = vec![LaggedWithinResidual::new(1.0, 1.0, unit_interval); 15]; + pairs.push(nonzero_pair); + + assert_eq!( + recover_centered_irregular_residual_log_rate(&pairs), + Err(LongitudinalError::InvalidTemporalTransformInput), + "a mathematically positive mean rate must not be reported as exact no-change" + ); +} diff --git a/crates/longitudinal_core/tests/irregular_rate_monte_carlo_contract.rs b/crates/longitudinal_core/tests/irregular_rate_monte_carlo_contract.rs new file mode 100644 index 000000000..42d15e75e --- /dev/null +++ b/crates/longitudinal_core/tests/irregular_rate_monte_carlo_contract.rs @@ -0,0 +1,291 @@ +//! Scientific acceptance for the declared pair-weighted irregular-rate estimand. +//! +//! This test deliberately keeps Monte Carlo arithmetic in the test boundary. Production +//! finite-binary64 mean ownership remains outside TEPP. + +use longitudinal_core::{ + EventTimedObservation, IrregularRateEstimand, recover_within_unit_irregular_rate_summary, +}; + +const REPLICATES: usize = 4_096; +const FIRST_QUARTER_REPLICATES: usize = REPLICATES / 4; +const INNOVATION_HALF_WIDTH: f64 = 0.12; +const Z_95: f64 = 1.96; + +fn timed(unit: u32, event_time: f64, score: f64) -> EventTimedObservation { + EventTimedObservation::new(unit, event_time, score) +} + +fn exact_test_count(value: usize) -> f64 { + f64::from(u32::try_from(value).expect("test count must fit in u32")) +} + +#[derive(Clone, Copy)] +struct SplitMix64 { + state: u64, +} + +impl SplitMix64 { + fn new(seed: u64) -> Self { + Self { state: seed } + } + + fn next_u64(&mut self) -> u64 { + self.state = self.state.wrapping_add(0x9E37_79B9_7F4A_7C15); + let mut value = self.state; + value = (value ^ (value >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + value = (value ^ (value >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + value ^ (value >> 31) + } + + fn signed_unit(&mut self) -> f64 { + let mantissa = self.next_u64() >> 11; + let high = u32::try_from(mantissa >> 32).expect("53-bit mantissa high part must fit u32"); + let low = u32::try_from(mantissa & u64::from(u32::MAX)) + .expect("53-bit mantissa low part must fit u32"); + let unit = (f64::from(high) * 4_294_967_296.0 + f64::from(low)) / 9_007_199_254_740_992.0; + 2.0 * unit - 1.0 + } +} + +#[derive(Clone, Copy, Debug)] +struct GeneratedUnit { + admitted_pairs: usize, + pair_rate_variance_sum: f64, +} + +fn append_noisy_missingness_unit( + rows: &mut Vec, + rng: &mut SplitMix64, + unit: u32, + true_log_rate: f64, + retain_three_of_four: bool, +) -> GeneratedUnit { + const LAST_NEGATIVE_TIME: usize = 12; + let innovation_variance = INNOVATION_HALF_WIDTH * INNOVATION_HALF_WIDTH / 3.0; + + let mut log_magnitude = 0.0; + let first_score = -1.0; + let mut retained_times = Vec::with_capacity(LAST_NEGATIVE_TIME + 1); + let mut negative_sum = first_score; + retained_times.push(0_usize); + rows.push(timed(unit, 0.0, first_score)); + + for event_time in 1..=LAST_NEGATIVE_TIME { + let innovation = INNOVATION_HALF_WIDTH * rng.signed_unit(); + log_magnitude += true_log_rate + innovation; + + let retain = if event_time == LAST_NEGATIVE_TIME { + true + } else { + let draw = rng.next_u64(); + if retain_three_of_four { + draw.trailing_zeros() < 2 + } else { + draw.trailing_zeros() >= 2 + } + }; + + if retain { + let score = -log_magnitude.exp(); + rows.push(timed(unit, exact_test_count(event_time), score)); + negative_sum += score; + retained_times.push(event_time); + } + } + + rows.push(timed( + unit, + exact_test_count(LAST_NEGATIVE_TIME + 1), + -negative_sum, + )); + + let admitted_pairs = retained_times.len() - 1; + let pair_rate_variance_sum = retained_times + .windows(2) + .map(|times| { + let gap = exact_test_count(times[1] - times[0]); + innovation_variance / gap + }) + .sum(); + + GeneratedUnit { + admitted_pairs, + pair_rate_variance_sum, + } +} + +#[derive(Clone, Copy, Debug)] +struct MonteCarloEvidence { + attempted_replicates: usize, + recovered_replicates: usize, + failed_replicates: usize, + half_rate_admitted_pairs: usize, + third_rate_admitted_pairs: usize, + bias: f64, + bias_monte_carlo_se: f64, + rmse: f64, + first_quarter_rmse: f64, + coverage_95: f64, + coverage_wilson_lower_95: f64, + coverage_wilson_upper_95: f64, +} + +fn run_noisy_rate_associated_missingness(seed: u64) -> MonteCarloEvidence { + let half_rate = 0.5_f64.ln(); + let third_rate = (1.0_f64 / 3.0).ln(); + let mut rng = SplitMix64::new(seed); + + let mut recovered_replicates = 0_usize; + let mut failed_replicates = 0_usize; + let mut first_quarter_recovered = 0_usize; + let mut half_rate_admitted_pairs = 0_usize; + let mut third_rate_admitted_pairs = 0_usize; + let mut error_sum = 0.0; + let mut squared_error_sum = 0.0; + let mut first_quarter_squared_error_sum = 0.0; + let mut covered_replicates = 0_usize; + + for replicate_index in 0..REPLICATES { + let mut rows = Vec::with_capacity(32); + let half = append_noisy_missingness_unit(&mut rows, &mut rng, 1, half_rate, true); + let third = append_noisy_missingness_unit(&mut rows, &mut rng, 2, third_rate, false); + half_rate_admitted_pairs += half.admitted_pairs; + third_rate_admitted_pairs += third.admitted_pairs; + + let Ok(summary) = recover_within_unit_irregular_rate_summary( + &rows, + IrregularRateEstimand::LagPairAverageV1, + ) else { + failed_replicates += 1; + continue; + }; + let Some(estimate) = summary.estimate() else { + failed_replicates += 1; + continue; + }; + + let admitted_pairs = half.admitted_pairs + third.admitted_pairs; + assert_eq!(summary.candidate_units(), 2); + assert_eq!(summary.contributing_units(), 2); + assert_eq!(summary.admitted_pairs(), admitted_pairs); + assert_eq!(summary.refused_pairs(), 2); + assert_eq!( + summary.candidate_pairs(), + summary.admitted_pairs() + summary.refused_pairs() + ); + + let half_pairs = exact_test_count(half.admitted_pairs); + let third_pairs = exact_test_count(third.admitted_pairs); + let admitted_pair_count = exact_test_count(admitted_pairs); + let true_pair_target = + (half_pairs * half_rate + third_pairs * third_rate) / admitted_pair_count; + let error = estimate - true_pair_target; + let conditional_standard_error = + (half.pair_rate_variance_sum + third.pair_rate_variance_sum).sqrt() + / admitted_pair_count; + + recovered_replicates += 1; + error_sum += error; + squared_error_sum += error * error; + if replicate_index < FIRST_QUARTER_REPLICATES { + first_quarter_recovered += 1; + first_quarter_squared_error_sum += error * error; + } + if error.abs() <= Z_95 * conditional_standard_error { + covered_replicates += 1; + } + } + + assert!(recovered_replicates > 1); + assert!(first_quarter_recovered > 0); + + let recovered = exact_test_count(recovered_replicates); + let bias = error_sum / recovered; + let rmse = (squared_error_sum / recovered).sqrt(); + let first_quarter_rmse = + (first_quarter_squared_error_sum / exact_test_count(first_quarter_recovered)).sqrt(); + let sample_error_variance = + (squared_error_sum - recovered * bias * bias).max(0.0) / (recovered - 1.0); + let bias_monte_carlo_se = (sample_error_variance / recovered).sqrt(); + let coverage_95 = exact_test_count(covered_replicates) / recovered; + + let z_squared = Z_95 * Z_95; + let wilson_denominator = 1.0 + z_squared / recovered; + let wilson_center = (coverage_95 + z_squared / (2.0 * recovered)) / wilson_denominator; + let wilson_margin = Z_95 + * ((coverage_95 * (1.0 - coverage_95) / recovered) + + z_squared / (4.0 * recovered * recovered)) + .sqrt() + / wilson_denominator; + + MonteCarloEvidence { + attempted_replicates: REPLICATES, + recovered_replicates, + failed_replicates, + half_rate_admitted_pairs, + third_rate_admitted_pairs, + bias, + bias_monte_carlo_se, + rmse, + first_quarter_rmse, + coverage_95, + coverage_wilson_lower_95: wilson_center - wilson_margin, + coverage_wilson_upper_95: wilson_center + wilson_margin, + } +} + +#[test] +fn noisy_rate_associated_missingness_recovers_declared_pair_estimand() { + let first = run_noisy_rate_associated_missingness(0x4950_2026_0912); + let replay = run_noisy_rate_associated_missingness(0x4950_2026_0912); + + assert_eq!(first.attempted_replicates, REPLICATES); + assert_eq!(first.recovered_replicates, REPLICATES); + assert_eq!(first.failed_replicates, 0); + assert!( + first.half_rate_admitted_pairs > 2 * first.third_rate_admitted_pairs, + "rate-associated retention must create informative pair multiplicity" + ); + + assert!(first.bias.abs() <= 3.0 * first.bias_monte_carlo_se); + assert!(first.rmse > 0.01 && first.rmse < 0.03); + assert!( + (first.rmse - first.first_quarter_rmse).abs() <= 0.25 * first.rmse, + "RMSE should stabilize rather than depend on the first quarter of replicates" + ); + assert!(first.coverage_95 > 0.90 && first.coverage_95 < 0.99); + assert!(first.coverage_wilson_lower_95 <= 0.95); + assert!(first.coverage_wilson_upper_95 >= 0.95); + + assert_eq!(first.attempted_replicates, replay.attempted_replicates); + assert_eq!(first.recovered_replicates, replay.recovered_replicates); + assert_eq!(first.failed_replicates, replay.failed_replicates); + assert_eq!( + first.half_rate_admitted_pairs, + replay.half_rate_admitted_pairs + ); + assert_eq!( + first.third_rate_admitted_pairs, + replay.third_rate_admitted_pairs + ); + assert_eq!(first.bias.to_bits(), replay.bias.to_bits()); + assert_eq!( + first.bias_monte_carlo_se.to_bits(), + replay.bias_monte_carlo_se.to_bits() + ); + assert_eq!(first.rmse.to_bits(), replay.rmse.to_bits()); + assert_eq!( + first.first_quarter_rmse.to_bits(), + replay.first_quarter_rmse.to_bits() + ); + assert_eq!(first.coverage_95.to_bits(), replay.coverage_95.to_bits()); + assert_eq!( + first.coverage_wilson_lower_95.to_bits(), + replay.coverage_wilson_lower_95.to_bits() + ); + assert_eq!( + first.coverage_wilson_upper_95.to_bits(), + replay.coverage_wilson_upper_95.to_bits() + ); +} diff --git a/crates/longitudinal_core/tests/irregular_rate_underflow_contract.rs b/crates/longitudinal_core/tests/irregular_rate_underflow_contract.rs new file mode 100644 index 000000000..afe5d0f5c --- /dev/null +++ b/crates/longitudinal_core/tests/irregular_rate_underflow_contract.rs @@ -0,0 +1,30 @@ +//! Near-equal large residuals keep a finite log-rate instead of underflowing. + +use longitudinal_core::{ + EventTimeInterval, LaggedWithinResidual, LongitudinalError, + recover_centered_irregular_residual_log_rate, +}; + +#[test] +fn distinct_residuals_with_nonrepresentable_tiny_rate_fail_closed() { + let event_interval = EventTimeInterval::new(f64::MAX).expect("finite positive event interval"); + let earlier = 1.0_f64; + let later = f64::from_bits(earlier.to_bits() + 1); + assert_ne!(earlier.to_bits(), later.to_bits()); + + let pair = LaggedWithinResidual::new(earlier, later, event_interval); + assert_eq!( + recover_centered_irregular_residual_log_rate(&[pair]), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); +} + +#[test] +fn equal_residuals_preserve_exact_zero_rate() { + let event_interval = EventTimeInterval::new(f64::MAX).expect("finite positive event interval"); + let pair = LaggedWithinResidual::new(1.0, 1.0, event_interval); + + let recovered = recover_centered_irregular_residual_log_rate(&[pair]) + .expect("equal residual magnitudes are exact no-change"); + assert_eq!(recovered.to_bits(), 0.0_f64.to_bits()); +} diff --git a/crates/longitudinal_core/tests/irregular_residual_mean_accuracy_contract.rs b/crates/longitudinal_core/tests/irregular_residual_mean_accuracy_contract.rs new file mode 100644 index 000000000..88825aae5 --- /dev/null +++ b/crates/longitudinal_core/tests/irregular_residual_mean_accuracy_contract.rs @@ -0,0 +1,98 @@ +//! Accuracy contract for means of extreme but finite event-time log rates. + +use longitudinal_core::{ + EventTimeInterval, LaggedWithinResidual, recover_centered_irregular_residual_log_rate, +}; + +fn pair(rate: f64, interval: f64) -> LaggedWithinResidual { + let earlier = 1.0_f64; + let later = (rate * interval).exp(); + LaggedWithinResidual::new( + earlier, + later, + EventTimeInterval::new(interval).expect("positive event interval"), + ) +} + +fn ratio_pair(later: f64, interval: f64) -> LaggedWithinResidual { + LaggedWithinResidual::new( + 1.0, + later, + EventTimeInterval::new(interval).expect("positive event interval"), + ) +} + +#[test] +fn compensated_mean_preserves_small_signal_between_extreme_rates() { + let pairs = [ + pair(1.0e100, 1.0e-100), + pair(1.0, 1.0), + pair(-1.0e100, 1.0e-100), + ]; + let recovered = recover_centered_irregular_residual_log_rate(&pairs) + .expect("finite extreme-rate mean remains identifiable"); + let expected = 1.0 / 3.0; + assert!((recovered - expected).abs() <= 1.0e-12); +} + +#[test] +fn full_exponent_range_cancellation_preserves_representable_tiny_mean() { + let tiny_interval = 4.0e-309_f64; + let next_after_one = f64::from_bits(1.0_f64.to_bits() + 1); + let tiny_rate = next_after_one.ln(); + let pairs = [ + ratio_pair(2.0, tiny_interval), + ratio_pair(next_after_one, 1.0), + ratio_pair(0.5, tiny_interval), + ]; + + let recovered = recover_centered_irregular_residual_log_rate(&pairs) + .expect("opposing extreme rates retain the finite low-order mean"); + let expected = tiny_rate / 3.0; + assert!(expected > 0.0 && expected.is_finite()); + assert_eq!(recovered.to_bits(), expected.to_bits()); +} + +#[test] +fn cancellation_does_not_underflow_subnormal_rates_before_the_mean() { + let minimum_subnormal = f64::from_bits(1); + let target_small_rate = f64::from_bits(2); + let next_after_one = f64::from_bits(1.0_f64.to_bits() + 1); + let small_interval = next_after_one.ln() / target_small_rate; + let extreme_interval = 4.0e-309_f64; + let pairs = [ + ratio_pair(2.0, extreme_interval), + ratio_pair(next_after_one, small_interval), + ratio_pair(next_after_one, small_interval), + ratio_pair(0.5, extreme_interval), + ]; + + let recovered = recover_centered_irregular_residual_log_rate(&pairs) + .expect("large cancellation must retain a representable subnormal mean"); + assert_eq!(recovered.to_bits(), minimum_subnormal.to_bits()); +} + +#[test] +fn same_sign_rate_mean_is_bit_stable_under_pair_permutation() { + let pairs_a = [ + ratio_pair(2.0, 1.0), + ratio_pair(4.0, 1.0), + ratio_pair(8.0, 1.0), + ]; + let pairs_b = [ + ratio_pair(2.0, 1.0), + ratio_pair(8.0, 1.0), + ratio_pair(4.0, 1.0), + ]; + + let recovered_a = recover_centered_irregular_residual_log_rate(&pairs_a) + .expect("first permutation remains identifiable"); + let recovered_b = recover_centered_irregular_residual_log_rate(&pairs_b) + .expect("second permutation remains identifiable"); + + assert_eq!( + recovered_a.to_bits(), + recovered_b.to_bits(), + "scientific evidence order must not change the binary64 reference mean" + ); +} diff --git a/crates/longitudinal_core/tests/irregular_residual_subnormal_mean_contract.rs b/crates/longitudinal_core/tests/irregular_residual_subnormal_mean_contract.rs new file mode 100644 index 000000000..6c883f7ca --- /dev/null +++ b/crates/longitudinal_core/tests/irregular_residual_subnormal_mean_contract.rs @@ -0,0 +1,28 @@ +//! Binary64 contract for CWC means at the minimum-subnormal boundary. + +use longitudinal_core::{EventTimedObservation, center_within_unit_event_lags}; + +fn timed(unit: u32, event_time: f64, score: f64) -> EventTimedObservation { + EventTimedObservation::new(unit, event_time, score) +} + +#[test] +fn representable_subnormal_cwc_mean_preserves_round_to_even() { + let minimum_subnormal = f64::from_bits(1); + let two_ulps = f64::from_bits(2); + let pairs = center_within_unit_event_lags(&[ + timed(1, 0.0, minimum_subnormal), + timed(1, 1.0, two_ulps), + timed(2, 0.0, 2.0), + timed(2, 1.0, 4.0), + ]) + .expect("both units contribute an admitted event-time lag"); + + assert_eq!(pairs.len(), 2); + assert_eq!( + pairs[0].earlier_residual().to_bits(), + (-minimum_subnormal).to_bits(), + "the exact 1.5-ULP mean must round to the even 2-ULP value" + ); + assert_eq!(pairs[0].later_residual().to_bits(), 0.0_f64.to_bits()); +} diff --git a/crates/longitudinal_core/tests/mean_rounding_contract.rs b/crates/longitudinal_core/tests/mean_rounding_contract.rs new file mode 100644 index 000000000..1aa92518d --- /dev/null +++ b/crates/longitudinal_core/tests/mean_rounding_contract.rs @@ -0,0 +1,53 @@ +//! Binary64 rounding contracts shared by longitudinal centering paths. + +use longitudinal_core::{ + EventTimedObservation, center_occasion_mean_event_lags, center_within_unit_event_lags, +}; + +fn timed(unit: u32, event_time: f64, score_bits: u64) -> EventTimedObservation { + EventTimedObservation::new(unit, event_time, f64::from_bits(score_bits)) +} + +#[test] +fn cwc_mean_rounds_halfway_subnormal_to_even_after_normalization() { + let pairs = center_within_unit_event_lags(&[ + timed(1, 0.0, 1), + timed(1, 1.0, 14), + timed(2, 0.0, 4), + timed(2, 1.0, 12), + ]) + .expect("both units contribute finite event-time lags"); + + assert_eq!(pairs.len(), 2); + assert_eq!( + pairs[0].earlier_residual().to_bits(), + (-f64::from_bits(7)).to_bits(), + "the exact 7.5-ULP unit mean must round to the even 8-ULP value" + ); + assert_eq!( + pairs[0].later_residual().to_bits(), + f64::from_bits(6).to_bits() + ); +} + +#[test] +fn occasion_mean_rounds_halfway_subnormal_to_even_after_normalization() { + let pairs = center_occasion_mean_event_lags(&[ + timed(1, 0.0, 1), + timed(2, 0.0, 14), + timed(1, 1.0, 4), + timed(2, 1.0, 12), + ]) + .expect("both occasions and units satisfy the longitudinal evidence floor"); + + assert_eq!(pairs.len(), 2); + assert_eq!( + pairs[0].earlier_residual().to_bits(), + (-f64::from_bits(7)).to_bits(), + "the exact 7.5-ULP occasion mean must round to the even 8-ULP value" + ); + assert_eq!( + pairs[0].later_residual().to_bits(), + (-f64::from_bits(4)).to_bits() + ); +} diff --git a/crates/longitudinal_core/tests/mixed_sign_mean_rounding_contract.rs b/crates/longitudinal_core/tests/mixed_sign_mean_rounding_contract.rs new file mode 100644 index 000000000..512955d4c --- /dev/null +++ b/crates/longitudinal_core/tests/mixed_sign_mean_rounding_contract.rs @@ -0,0 +1,138 @@ +//! Mixed-sign CWC and occasion means divide by the original sample count. + +use longitudinal_core::{ + EventTimeInterval, EventTimedObservation, LaggedWithinResidual, + center_occasion_mean_event_lags, center_within_unit_event_lags, + recover_centered_irregular_residual_log_rate, +}; + +fn timed(unit: u32, event_time: f64, score: f64) -> EventTimedObservation { + EventTimedObservation::new(unit, event_time, score) +} + +fn lagged(earlier: f64, later: f64, interval: f64) -> LaggedWithinResidual { + LaggedWithinResidual::new( + earlier, + later, + EventTimeInterval::new(interval).expect("positive event interval"), + ) +} + +#[test] +fn mixed_sign_subnormal_cwc_mean_rounds_once_at_the_final_denominator() { + let ulp = f64::from_bits(1); + let rows = [ + timed(1, 0.0, -20.0 * ulp), + timed(1, 1.0, -20.0 * ulp), + timed(1, 2.0, 9.0 * ulp), + timed(2, 0.0, 1.0), + timed(2, 1.0, 2.0), + ]; + + let pairs = center_within_unit_event_lags(&rows).expect("admissible CWC rows"); + + // The exact unit-1 mean is -31/3 minimum-subnormal ULPs, which rounds once + // to -10 ULPs. The predecessor rounded the retained mixed-sign residual + // mean first and then weighted it, producing -11 ULPs instead. + assert_eq!( + pairs[0].earlier_residual().to_bits(), + (-f64::from_bits(10)).to_bits() + ); + assert_eq!( + pairs[0].later_residual().to_bits(), + (-f64::from_bits(10)).to_bits() + ); + assert_eq!( + pairs[1].earlier_residual().to_bits(), + (-f64::from_bits(10)).to_bits() + ); + assert_eq!( + pairs[1].later_residual().to_bits(), + f64::from_bits(19).to_bits() + ); +} + +#[test] +fn mixed_sign_subnormal_occasion_mean_uses_the_same_single_rounding_authority() { + let ulp = f64::from_bits(1); + let rows = [ + timed(1, 0.0, -20.0 * ulp), + timed(2, 0.0, -20.0 * ulp), + timed(3, 0.0, 9.0 * ulp), + timed(1, 1.0, 0.0), + timed(2, 1.0, 0.0), + timed(3, 1.0, 0.0), + ]; + + let pairs = center_occasion_mean_event_lags(&rows).expect("admissible occasion rows"); + + // Occasion t=0 has the same exact mean -31/3 ULPs. The public occasion + // path must share the CWC numerical authority rather than reintroducing a + // retained-only mean followed by a second weighting round. + assert_eq!( + pairs[0].earlier_residual().to_bits(), + (-f64::from_bits(10)).to_bits() + ); + assert_eq!( + pairs[1].earlier_residual().to_bits(), + (-f64::from_bits(10)).to_bits() + ); + assert_eq!( + pairs[2].earlier_residual().to_bits(), + f64::from_bits(19).to_bits() + ); +} + +#[test] +fn repeated_small_opposite_rates_change_the_correctly_rounded_mean() { + let log_two = -(-0.5_f64).ln_1p(); + let expected = 3_333_333_333_333_332.5_f64; + + let positive_dominant = [ + lagged(1.0, 2.0, log_two / 1.0e16_f64), + lagged(1.0, 0.5, log_two), + lagged(1.0, 0.5, log_two), + ]; + let recovered = recover_centered_irregular_residual_log_rate(&positive_dominant) + .expect("finite positive-dominant mixed-sign mean remains identifiable"); + assert_eq!( + recovered.to_bits(), + expected.to_bits(), + "two -1 rates are jointly significant and must not be rounded away one at a time against 1e16", + ); + + let mirrored_pairs = [ + lagged(1.0, 0.5, log_two / 1.0e16_f64), + lagged(1.0, 2.0, log_two), + lagged(1.0, 2.0, log_two), + ]; + let mirrored = recover_centered_irregular_residual_log_rate(&mirrored_pairs) + .expect("finite mirrored mixed-sign mean remains identifiable"); + assert_eq!( + mirrored.to_bits(), + (-expected).to_bits(), + "two +1 rates are jointly significant and must not be rounded away one at a time against the opposite extreme", + ); +} + +#[test] +fn half_ulp_tail_changes_the_final_mixed_sign_rounding() { + let log_two = -(-0.5_f64).ln_1p(); + let dominant_rate = f64::from_bits(0x4698_0000_0000_0000); + let half_ulp_rate = f64::from_bits(0x4340_0000_0000_0000); + let expected = f64::from_bits(0x467f_ffff_ffff_ffff); + + let pairs = [ + lagged(1.0, 2.0, log_two / dominant_rate), + lagged(1.0, 0.5, log_two / half_ulp_rate), + lagged(1.0, 0.5, log_two), + ]; + let recovered = recover_centered_irregular_residual_log_rate(&pairs) + .expect("finite mixed-sign mean remains identifiable"); + + assert_eq!( + recovered.to_bits(), + expected.to_bits(), + "the -1 tail sits below the -2^53 accumulator ULP but moves the final cancellation below the dominant half-ULP tie", + ); +} diff --git a/crates/longitudinal_core/tests/occasion_mean_event_time_contract.rs b/crates/longitudinal_core/tests/occasion_mean_event_time_contract.rs new file mode 100644 index 000000000..80c155071 --- /dev/null +++ b/crates/longitudinal_core/tests/occasion_mean_event_time_contract.rs @@ -0,0 +1,197 @@ +//! Occasion-mean event-time lags are Hamaker Eq. 1a deviations, not CWC. + +use longitudinal_core::{ + EventTimedObservation, LongitudinalError, center_occasion_mean_event_lags, + recover_occasion_mean_centered_irregular_residual_log_rate, + recover_within_unit_irregular_residual_log_rate, + refuse_occasion_mean_centered_log_rate_as_within_person_lag, +}; + +fn observation(unit: u32, event_time: f64, score: f64) -> EventTimedObservation { + EventTimedObservation::new(unit, event_time, score) +} + +#[test] +fn signed_zero_is_one_numeric_occasion() { + let drift = -0.5_f64; + let phi = drift.exp(); + let rows = [ + observation(1, -0.0, 1.0), + observation(1, 1.0, phi), + observation(2, 0.0, -1.0), + observation(2, 1.0, -phi), + ]; + + let pairs = center_occasion_mean_event_lags(&rows).expect("numeric zero is one occasion"); + assert_eq!(pairs.len(), 2); + + let recovered = recover_occasion_mean_centered_irregular_residual_log_rate(&rows) + .expect("recover occasion-mean residual rate"); + assert!((recovered - drift).abs() < 1.0e-12); + assert_eq!( + refuse_occasion_mean_centered_log_rate_as_within_person_lag(recovered), + Err(LongitudinalError::BetweenIsNotWithinChange) + ); +} + +#[test] +fn occasion_mean_residual_rate_is_not_cwc_rate_on_the_same_panel() { + let drift = -0.4_f64; + let phi = drift.exp(); + let rows = [ + observation(1, 0.0, 1.2), + observation(1, 1.0, 5.0 + 1.2 * phi), + observation(1, 2.0, 11.0 + 1.2 * (drift * 2.0).exp()), + observation(2, 0.0, -0.8), + observation(2, 1.0, 5.0 - 0.8 * phi), + observation(2, 2.0, 11.0 - 0.8 * (drift * 2.0).exp()), + ]; + + let occasion = recover_occasion_mean_centered_irregular_residual_log_rate(&rows) + .expect("occasion-mean residual rate"); + assert!((occasion - drift).abs() < 1.0e-12); + + let cwc = recover_within_unit_irregular_residual_log_rate(&rows).expect("CWC residual rate"); + assert!( + (cwc - drift).abs() > 1.0e-6, + "Hamaker Eq. 1a occasion deviations and person-mean CWC residuals are different estimands: occasion={occasion}, CWC={cwc}" + ); +} + +#[test] +fn signed_zero_duplicate_unit_is_rejected_as_one_occasion() { + let rows = [ + observation(1, -0.0, 1.0), + observation(1, 0.0, 1.5), + observation(2, 0.0, -1.0), + observation(2, 1.0, -0.5), + ]; + + assert_eq!( + center_occasion_mean_event_lags(&rows), + Err(LongitudinalError::InvalidObservationPayload) + ); +} + +#[test] +fn representable_occasion_mean_is_not_rejected_for_intermediate_sum_overflow() { + let max = f64::MAX; + let rows = [ + observation(1, 0.0, 0.75 * max), + observation(1, 1.0, 1.0), + observation(2, 0.0, 0.75 * max), + observation(2, 1.0, 1.0), + observation(3, 0.0, -0.5 * max), + observation(3, 1.0, -2.0), + ]; + + let pairs = center_occasion_mean_event_lags(&rows) + .expect("finite occasion mean must survive same-sign intermediate overflow"); + assert_eq!(pairs.len(), 3); + assert!(pairs.iter().all(|pair| { + pair.earlier_residual().is_finite() + && pair.later_residual().is_finite() + && pair.event_interval().as_f64().is_finite() + })); +} + +#[test] +fn representable_subnormal_occasion_mean_preserves_round_to_even() { + let minimum_subnormal = f64::from_bits(1); + let two_subnormals = f64::from_bits(2); + let rows = [ + observation(1, 0.0, minimum_subnormal), + observation(1, 1.0, 0.0), + observation(2, 0.0, two_subnormals), + observation(2, 1.0, 0.0), + ]; + + let pairs = center_occasion_mean_event_lags(&rows) + .expect("subnormal occasion mean must remain representable"); + assert_eq!(pairs.len(), 2); + assert_eq!( + pairs[0].earlier_residual().to_bits(), + (-minimum_subnormal).to_bits(), + "mean([1 ulp, 2 ulp]) is 1.5 ulp and rounds ties-to-even to 2 ulp" + ); + assert_eq!(pairs[1].earlier_residual().to_bits(), 0.0_f64.to_bits()); +} + +#[test] +fn occasion_mean_is_bit_stable_under_row_permutation() { + let large = f64::MAX * 0.5; + let next_one = f64::from_bits(1.0_f64.to_bits() + 1); + let rows_a = [ + observation(1, 0.0, 1.0), + observation(1, 1.0, 1.0), + observation(2, 0.0, next_one), + observation(2, 1.0, 1.0), + observation(3, 0.0, large), + observation(3, 1.0, 1.0), + ]; + let rows_b = [ + observation(1, 0.0, 1.0), + observation(1, 1.0, 1.0), + observation(3, 0.0, large), + observation(3, 1.0, 1.0), + observation(2, 0.0, next_one), + observation(2, 1.0, 1.0), + ]; + + let pairs_a = center_occasion_mean_event_lags(&rows_a).expect("first permutation"); + let pairs_b = center_occasion_mean_event_lags(&rows_b).expect("second permutation"); + assert_eq!(pairs_a, pairs_b); +} + +#[test] +fn sparse_unaligned_and_nonfinite_occasion_inputs_fail_closed() { + assert_eq!( + center_occasion_mean_event_lags(&[]), + Err(LongitudinalError::InvalidObservationPayload) + ); + assert_eq!( + center_occasion_mean_event_lags(&[observation(1, 0.0, 1.0)]), + Err(LongitudinalError::InvalidObservationPayload) + ); + + let unaligned = [ + observation(1, 0.0, 1.0), + observation(1, 1.0, 0.5), + observation(2, 0.1, -1.0), + observation(2, 1.1, -0.5), + ]; + assert_eq!( + center_occasion_mean_event_lags(&unaligned), + Err(LongitudinalError::InvalidObservationPayload) + ); + + let nonfinite = [ + observation(1, f64::NAN, 1.0), + observation(1, 1.0, 0.5), + observation(2, 0.0, -1.0), + observation(2, 1.0, -0.5), + ]; + assert_eq!( + center_occasion_mean_event_lags(&nonfinite), + Err(LongitudinalError::InvalidObservationPayload) + ); +} + +#[test] +fn singleton_wave_unit_does_not_manufacture_or_block_lag_evidence() { + let drift = -0.5_f64; + let phi = drift.exp(); + let rows = [ + observation(1, 0.0, 1.0), + observation(1, 1.0, 4.0 + phi), + observation(2, 0.0, -1.0), + observation(2, 1.0, 4.0 - phi), + observation(3, 0.0, 0.0), + ]; + + let pairs = center_occasion_mean_event_lags(&rows).expect("two lag-contributing units remain"); + assert_eq!(pairs.len(), 2); + let recovered = recover_occasion_mean_centered_irregular_residual_log_rate(&rows) + .expect("singleton-wave unit is not a lag contributor"); + assert!((recovered - drift).abs() < 1.0e-12); +} diff --git a/crates/longitudinal_core/tests/occasion_mean_signed_zero_contract.rs b/crates/longitudinal_core/tests/occasion_mean_signed_zero_contract.rs new file mode 100644 index 000000000..f0835aa21 --- /dev/null +++ b/crates/longitudinal_core/tests/occasion_mean_signed_zero_contract.rs @@ -0,0 +1,30 @@ +//! Exact-zero occasion-mean residuals have one public encoding. + +use longitudinal_core::{EventTimedObservation, center_occasion_mean_event_lags}; + +fn timed(unit: u32, event_time: f64, score: f64) -> EventTimedObservation { + EventTimedObservation::new(unit, event_time, score) +} + +#[test] +fn exact_zero_occasion_mean_residuals_use_one_canonical_public_encoding() { + let pairs = center_occasion_mean_event_lags(&[ + timed(0, 0.0, -1.0), + timed(0, 1.0, -0.0), + timed(0, 2.0, 1.0), + timed(1, 0.0, 1.0), + timed(1, 1.0, 0.0), + timed(1, 2.0, -1.0), + ]) + .expect("finite aligned occasions admit Hamaker Eq. 1a lags"); + + assert_eq!(pairs.len(), 4); + assert_eq!(pairs[0].earlier_residual().to_bits(), (-1.0_f64).to_bits()); + assert_eq!(pairs[0].later_residual().to_bits(), 0.0_f64.to_bits()); + assert_eq!(pairs[1].earlier_residual().to_bits(), 0.0_f64.to_bits()); + assert_eq!(pairs[1].later_residual().to_bits(), 1.0_f64.to_bits()); + assert_eq!(pairs[2].earlier_residual().to_bits(), 1.0_f64.to_bits()); + assert_eq!(pairs[2].later_residual().to_bits(), 0.0_f64.to_bits()); + assert_eq!(pairs[3].earlier_residual().to_bits(), 0.0_f64.to_bits()); + assert_eq!(pairs[3].later_residual().to_bits(), (-1.0_f64).to_bits()); +} diff --git a/crates/longitudinal_core/tests/standardised_diffusion_contract.rs b/crates/longitudinal_core/tests/standardised_diffusion_contract.rs new file mode 100644 index 000000000..578134ad6 --- /dev/null +++ b/crates/longitudinal_core/tests/standardised_diffusion_contract.rs @@ -0,0 +1,117 @@ +//! RED/GREEN contract for scalar standardised diffusion maps in Longitudinal Modeling. +//! +//! Driver, Oud, and Voelkle (2017) print the underlying continuous/discrete +//! diffusion transformations and describe relevant-variance standardisation, +//! but the 2017 ctsem summary source does not emit named `DIFFUSIONstd` or +//! `discreteDIFFUSIONstd` matrices. These scalar maps therefore remain explicit +//! research-candidate extensions rather than canonical ctsem output. + +use longitudinal_core::{ + EventTimeInterval, LongitudinalError, recover_event_time_standardised_continuous_diffusion, + recover_event_time_standardised_discrete_diffusion, + refuse_standardised_continuous_diffusion_as_standardised_discrete_diffusion, + refuse_total_variance_scaled_diffusion_as_standardised_diffusion, + refuse_unstandardised_diffusion_as_standardised_diffusion, +}; + +#[test] +fn continuous_diffusion_candidate_recovers_relevant_variance_ratio() { + let recovered = recover_event_time_standardised_continuous_diffusion(0.4, -0.25) + .expect("positive stationary within variance"); + assert!((recovered - 0.5).abs() < 1e-15); + + let scale_invariant = recover_event_time_standardised_continuous_diffusion(1.6, -0.25) + .expect("same scalar standardisation at a different q scale"); + assert!((scale_invariant - recovered).abs() < 1e-15); + + let max_path = recover_event_time_standardised_continuous_diffusion(f64::MAX, -0.75) + .expect("representable q/p must not fail on an avoidable intermediate overflow"); + assert!((max_path - 1.5).abs() < 1e-15); +} + +#[test] +fn continuous_diffusion_candidate_does_not_lose_cancellation_to_subnormal_rounding() { + let minimum_subnormal = f64::from_bits(1); + let recovered = recover_event_time_standardised_continuous_diffusion(minimum_subnormal, -0.75) + .expect("positive rounded stationary variance remains admissible"); + assert_eq!(recovered.to_bits(), 1.5_f64.to_bits()); + + let slightly_larger_subnormal = f64::from_bits(3); + let recovered_larger = + recover_event_time_standardised_continuous_diffusion(slightly_larger_subnormal, -0.75) + .expect("scale must not alter the standardized scalar identity"); + assert_eq!(recovered_larger.to_bits(), 1.5_f64.to_bits()); +} + +#[test] +fn discrete_diffusion_candidate_preserves_event_interval_semantics() { + let one = EventTimeInterval::new(1.0).expect("positive event interval"); + let two = EventTimeInterval::new(2.0).expect("positive event interval"); + + let recovered = recover_event_time_standardised_discrete_diffusion(0.4, -0.25, one) + .expect("positive stationary within variance"); + let expected = 1.0 - (-0.5_f64).exp(); + assert!((recovered - expected).abs() < 1e-15); + + let later = recover_event_time_standardised_discrete_diffusion(0.4, -0.25, two) + .expect("later event interval"); + assert!( + later > recovered, + "stable-process noise fraction must accumulate with event time" + ); + assert!(later < 1.0); + + let rescaled = recover_event_time_standardised_discrete_diffusion(1.6, -0.25, one) + .expect("same scalar standardisation at a different q scale"); + assert!((rescaled - recovered).abs() < 1e-15); +} + +#[test] +fn standardised_diffusion_candidates_fail_closed_without_positive_stationarity() { + let one = EventTimeInterval::new(1.0).expect("positive event interval"); + + assert_eq!( + recover_event_time_standardised_continuous_diffusion(0.0, -0.25), + Err(LongitudinalError::StandardisedDiffusionRequiresPositiveWithinVariance) + ); + assert_eq!( + recover_event_time_standardised_discrete_diffusion(0.0, -0.25, one), + Err(LongitudinalError::StandardisedDiffusionRequiresPositiveWithinVariance) + ); + assert_eq!( + recover_event_time_standardised_continuous_diffusion(0.4, 0.0), + Err(LongitudinalError::StationaryVarianceRequiresStableDrift) + ); + assert_eq!( + recover_event_time_standardised_discrete_diffusion(0.4, 0.25, one), + Err(LongitudinalError::StationaryVarianceRequiresStableDrift) + ); + assert_eq!( + recover_event_time_standardised_continuous_diffusion(f64::NAN, -0.25), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); +} + +#[test] +fn equal_numbers_do_not_collapse_named_diffusion_estimands() { + let one = EventTimeInterval::new(1.0).expect("positive event interval"); + let continuous = recover_event_time_standardised_continuous_diffusion(0.4, -0.5) + .expect("continuous candidate"); + let discrete = recover_event_time_standardised_discrete_diffusion(0.4, -0.5, one) + .expect("discrete candidate"); + + assert_eq!( + refuse_standardised_continuous_diffusion_as_standardised_discrete_diffusion( + continuous, discrete + ), + Err(LongitudinalError::ContinuousDiffusionIsNotDiscreteDiffusion) + ); + assert_eq!( + refuse_unstandardised_diffusion_as_standardised_diffusion(continuous, continuous), + Err(LongitudinalError::UnstandardisedDiffusionIsNotStandardisedDiffusion) + ); + assert_eq!( + refuse_total_variance_scaled_diffusion_as_standardised_diffusion(discrete, discrete), + Err(LongitudinalError::TotalVarianceScaledDiffusionIsNotStandardisedDiffusion) + ); +} diff --git a/crates/longitudinal_core/tests/standardised_discrete_drift_contract.rs b/crates/longitudinal_core/tests/standardised_discrete_drift_contract.rs new file mode 100644 index 000000000..23df97a39 --- /dev/null +++ b/crates/longitudinal_core/tests/standardised_discrete_drift_contract.rs @@ -0,0 +1,172 @@ +//! Scalar Driver p.16 `discreteDRIFTstd` contract on event time. + +use longitudinal_core::{ + EventTimeInterval, LongitudinalError, recover_event_time_standardised_discrete_drift, + refuse_trait_plus_state_association_as_standardised_discrete_drift, + refuse_trait_variance_as_standardisation_variance, + refuse_unstandardised_discrete_drift_as_standardised_discrete_drift, +}; + +fn event_time(value: f64) -> EventTimeInterval { + EventTimeInterval::new(value).expect("test interval must be valid event time") +} + +#[test] +fn driver_page_sixteen_scalar_standardised_drift_recovers_on_event_time() { + let recovered = recover_event_time_standardised_discrete_drift(0.4, -0.5, event_time(1.0)) + .expect("positive stationary within-person variance"); + assert!((recovered - (-0.5_f64).exp()).abs() < 1e-15); + + let longer = recover_event_time_standardised_discrete_drift(0.4, -0.5, event_time(2.5)) + .expect("irregular positive event interval"); + assert!((longer - (-1.25_f64).exp()).abs() < 1e-15); + assert!(longer < recovered); +} + +#[test] +fn known_truth_grid_has_machine_precision_rmse() { + let cases = [ + (0.1_f64, -0.15_f64, 0.25_f64), + (0.4, -0.5, 1.0), + (1.2, -1.25, 0.8), + (3.0, -0.05, 7.0), + ]; + let mut squared_error_sum = 0.0; + for (diffusion, rate, delta) in cases { + let recovered = + recover_event_time_standardised_discrete_drift(diffusion, rate, event_time(delta)) + .expect("known-truth case"); + let truth = (rate * delta).exp(); + squared_error_sum += (recovered - truth).powi(2); + } + let case_count = u32::try_from(cases.len()).expect("known-truth grid fits in u32"); + let rmse = (squared_error_sum / f64::from(case_count)).sqrt(); + assert!(rmse <= f64::EPSILON); +} + +#[test] +fn extreme_stable_rate_preserves_representable_stationary_result() { + let delta = 1.0 / f64::MAX; + let recovered = + recover_event_time_standardised_discrete_drift(f64::MAX, -f64::MAX, event_time(delta)) + .expect("doubling the drift must not create an avoidable overflow"); + let truth = (-f64::MAX * delta).exp(); + assert!((recovered - truth).abs() <= f64::EPSILON); +} + +#[test] +fn standardised_drift_does_not_materialise_a_cancelled_stationary_variance() { + let underflowed_stationary = + recover_event_time_standardised_discrete_drift(f64::from_bits(1), -1.0, event_time(1.0)) + .expect("positive real stationary variance cancels from the scalar standardisation"); + assert!((underflowed_stationary - (-1.0_f64).exp()).abs() <= f64::EPSILON); + + let overflowed_stationary = + recover_event_time_standardised_discrete_drift(f64::MAX, -0.25, event_time(1.0)) + .expect("an out-of-range stationary intermediate must not erase a finite final map"); + assert!((overflowed_stationary - (-0.25_f64).exp()).abs() <= f64::EPSILON); + + let extreme_scale = recover_event_time_standardised_discrete_drift( + f64::from_bits(1), + -1.0e307, + event_time(1.0e-307), + ) + .expect("the exact scalar standardisation is independent of diffusion scale"); + assert!((extreme_scale - (-1.0_f64).exp()).abs() <= f64::EPSILON); +} + +#[test] +fn event_time_exponent_underflow_fails_closed_instead_of_becoming_one() { + assert_eq!( + recover_event_time_standardised_discrete_drift( + f64::MIN_POSITIVE, + -f64::MIN_POSITIVE, + event_time(f64::MIN_POSITIVE), + ), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); +} + +#[test] +fn standardised_drift_fails_closed_without_positive_stationary_within_variance() { + assert_eq!( + recover_event_time_standardised_discrete_drift(0.0, -0.5, event_time(1.0)), + Err(LongitudinalError::StandardisedDriftRequiresPositiveWithinVariance) + ); + assert_eq!( + recover_event_time_standardised_discrete_drift(0.4, 0.0, event_time(1.0)), + Err(LongitudinalError::StationaryVarianceRequiresStableDrift) + ); + assert_eq!( + recover_event_time_standardised_discrete_drift(0.4, 0.5, event_time(1.0)), + Err(LongitudinalError::StationaryVarianceRequiresStableDrift) + ); +} + +#[test] +fn event_time_value_object_rejects_wrong_clock_payload_shape() { + assert_eq!( + EventTimeInterval::new(0.0), + Err(LongitudinalError::NonPositiveEventInterval) + ); + assert_eq!( + EventTimeInterval::new(-1.0), + Err(LongitudinalError::NonPositiveEventInterval) + ); + assert_eq!( + EventTimeInterval::new(f64::NAN), + Err(LongitudinalError::NonPositiveEventInterval) + ); + assert_eq!( + EventTimeInterval::new(f64::INFINITY), + Err(LongitudinalError::NonPositiveEventInterval) + ); +} + +#[test] +fn standardised_drift_rejects_nonfinite_negative_or_unrepresentable_inputs() { + for diffusion in [f64::NAN, f64::INFINITY, -0.1] { + assert_eq!( + recover_event_time_standardised_discrete_drift(diffusion, -0.5, event_time(1.0)), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + } + for rate in [f64::NAN, f64::NEG_INFINITY] { + assert_eq!( + recover_event_time_standardised_discrete_drift(0.4, rate, event_time(1.0)), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + } + assert_eq!( + recover_event_time_standardised_discrete_drift( + f64::MAX, + -f64::MIN_POSITIVE, + event_time(1.0), + ), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + assert_eq!( + recover_event_time_standardised_discrete_drift(0.4, -2.0, event_time(f64::MAX)), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); + assert_eq!( + recover_event_time_standardised_discrete_drift(0.4, -800.0, event_time(1.0)), + Err(LongitudinalError::InvalidTemporalTransformInput) + ); +} + +#[test] +fn equal_scalar_value_does_not_conflate_named_estimands() { + assert_eq!( + refuse_unstandardised_discrete_drift_as_standardised_discrete_drift(0.5, 0.5), + Err(LongitudinalError::UnstandardisedDriftIsNotStandardisedDrift) + ); + assert_eq!( + refuse_trait_plus_state_association_as_standardised_discrete_drift(0.5, 0.5), + Err(LongitudinalError::TraitStateAssociationIsNotStandardisedDrift) + ); + assert_eq!( + refuse_trait_variance_as_standardisation_variance(1.0, 0.4), + Err(LongitudinalError::TraitVarianceIsNotDriftStandardisationVariance) + ); +} diff --git a/docs/TRD.md b/docs/TRD.md index 031c2113a..38c436e34 100644 --- a/docs/TRD.md +++ b/docs/TRD.md @@ -1,7 +1,7 @@ # TEPP Technical Requirements Document **Status:** Accepted technical baseline aligned to approved PRD v0.4 -**Last reviewed:** 2026-08-16 +**Last reviewed:** 2026-09-12 ## 1. Technical objective @@ -47,6 +47,10 @@ Shared-latent temporal/relational topic estimation provides posterior uncertaint Longitudinal ESEM/DSEM (Asparouhov & Muthén, 2009; Asparouhov et al., 2018; Marsh et al., 2014) must distinguish stable between-unit differences from within-unit temporal change, test measurement invariance where comparisons require it (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014), account for irregular intervals when necessary, and propagate topic-posterior uncertainty through plausible values or joint estimation. These psychometric targets remain accepted-target. +Irregular-rate summaries must expose their weighting target as a versioned scientific contract rather than leaving contribution multiplicity implicit. `tepp.irregular_rate.lag_pair_average.v1` assigns equal weight to every admitted consecutive event-time pair and therefore permits a unit's total contribution to vary with admitted pair count. Its result evidence must carry candidate/contributing units and candidate/admitted/refused pairs, including refusal categories that alter the admitted population. `tepp.irregular_rate.unit_average.v1` is a separate equal-unit target that preserves unit identity through the within-unit summary before combining units. It remains fail closed until TEPP can consume an immutable released reusable finite-mean contract from fast-mlsirm; TEPP must not implement a second generic mean kernel to activate it. Occasion count, follow-up duration, missingness, or pair multiplicity must not be inferred as design or cross-classified/multiple-membership weights. Any future design-weighted target requires its own versioned contract. + +Scientific acceptance for irregular-rate weighting requires deterministic unequal-follow-up fixtures, permutation invariance, explicit refusal denominators, and leakage-safe known-truth simulation. Informative-missingness Monte Carlo evidence must report attempted/recovered/failed replicates, the truth corresponding to the actually admitted weighting population, bias, RMSE, interval coverage, convergence/replay evidence where applicable, and Monte Carlo uncertainty. The alternative weighting policy should be retained as a comparison or negative estimator when scientifically useful; numerical equality in one fixture does not collapse the estimand identities. + ## 8. Compute requirements Production mathematical/psychometric arithmetic is Rust. CPU `f64` is the numerical reference. CPU parallelism uses bounded fixed worker pools/thread-local sufficient statistics to reduce context switching and oversubscription. GPU execution is introduced only when computationally material, streamed under a VRAM budget, and parity-tested against the CPU reference. OOM triggers bounded batch reduction and safe CPU fallback rather than uncontrolled failure. diff --git a/docs/adr/longitudinal-modeling-ownership-addendum.md b/docs/adr/longitudinal-modeling-ownership-addendum.md new file mode 100644 index 000000000..b081e44ef --- /dev/null +++ b/docs/adr/longitudinal-modeling-ownership-addendum.md @@ -0,0 +1,47 @@ +# ADR 0005 ownership addendum — Longitudinal Modeling + +**Parent decision:** ADR 0005 — Posterior-aware ESEM/DSEM and structural interpretation + +**Decision identity:** ADR 0005; this addendum does not mint a new ADR number + +**Status:** Proposed clarification + +**Recorded:** 2026-09-01 + +## Decision clarification + +Temporal/event composition of longitudinal psychometric quantities belongs to the TEPP Longitudinal Modeling bounded context. Public operations whose meaning depends on substantive event time must therefore expose an event-time domain type at the context boundary rather than accepting an unqualified numeric duration. + +`longitudinal_core` owns this TEPP temporal composition. `fast-mlsirm` remains the canonical owner of reusable static/generalized-mixed/dependence-aware psychometric kernels. A numerically reusable primitive must migrate through the fast-mlsirm Published Language/ACL boundary rather than turning `longitudinal_core` into a second static psychometric kernel. + +For the scalar Driver, Oud, and Voelkle (2017) p. 16 `discreteDRIFTstd` special case, TEPP owns only the event-time composition and admissibility policy. The function requires stable negative drift, positive stationary within-person variance, and an admitted `EventTimeInterval`. In this scalar standardisation the stationary variance cancels algebraically: finite positive diffusion plus stable finite drift establish a positive real-valued stationary variance even when its materialized `f64` value would underflow or overflow. The cancelled intermediate therefore must not reject a representable final standardized map. An exact nonzero transition that itself collapses to a false binary64 endpoint remains fail-closed. + +The same numerical rule applies to TEPP's research-candidate scalar diffusion-standardisation compositions: a cancelled stationary variance is not a second admission gate, while the final standardized quantity must remain representable. This does not promote those named diffusion extensions to canonical ctsem output and does not move reusable static arithmetic ownership out of fast-mlsirm. + +The lagged-correlation boundary similarly requires both occasion-specific marginal variances and an `EventTimeInterval`; a covariance divided only by the earlier variance is not exposed as an autocorrelation. + +CWC-then-irregular residual log-rate is also Longitudinal Modeling composition. Person-mean centering of a time-related series is not raw-process drift (Curran & Bauer, 2011, pp. 583–619; PMC3059070 XML opened 2026-09-02; Eq. 36). The unique pairwise-mean-after-CWC evidence from Draft #327 is folded here with typed `EventTimeInterval` rather than grown on `psychometric_core`. Already-centered irregular pairs `(1, 0.5)` recover `ln(0.5)` only for `Δt = 1`; for a general admitted interval the exact map is `ln(0.5) / Δt`. CWC of a raw AR path does not recover raw-process drift. + +Repeated-pair multiplicity is part of the estimand rather than a numerical implementation detail. `tepp.irregular_rate.lag_pair_average.v1` weights every admitted consecutive pair equally and therefore lets a unit's total contribution vary with its admitted pair count. Its evidence records candidate/contributing units and candidate/admitted/refused pairs so missingness, follow-up duration, and refusal policy cannot silently disappear from the weighting population. `tepp.irregular_rate.unit_average.v1` is a different target: unit identity must survive until within-unit rates have first been summarized and the resulting unit summaries are combined with equal unit weight. It remains fail closed while the required reusable finite-mean operation has no immutable released fast-mlsirm contract. Occasion count is not a membership weight, including under time-varying multilevel, cross-classified, or multiple-membership composition. An explicit design-weighted target, if later required, needs a separate versioned contract instead of overloading either existing estimand. + +Occasion-mean event-time composition is likewise Longitudinal Modeling authority. Hamaker, Kuiper, and Grasman (2015, Eq. 1a) write `x_it = μ_t + p_it`, where `μ_t` is the occasion-specific group mean. TEPP forms those deviations only across numeric event-time occasions, treats IEEE-754 `-0.0` and `+0.0` as one numeric occasion, admits at most one observation per unit and occasion, and preserves each consecutive unit-specific interval as an `EventTimeInterval`. The resulting `p_it` still contains stable between-person differences and therefore does not become a within-person lag merely because it is lagged. It is distinct from person-mean CWC, a sample-wide grand mean, RI-CLPM, and DSEM. The occasion-specific mean must remain deterministic under row permutation and must not reject a representable final mean merely because a naive same-sign intermediate sum would overflow. + +## DDD consequences + +- `psychometric_core` is not the authority for temporal transforms merely because an earlier branch placed them there. +- `EventTimeInterval` is a value object of Longitudinal Modeling. Assertion-, document-, system-, availability-, and method-occasion intervals require explicit owning-context conversion before they can be admitted as substantive event time. +- Occasion identity is numeric event time, not the raw binary encoding of an otherwise equal numeric zero. +- Irregular-rate weighting policy is a versioned scientific contract. Pair-weighted, equal-unit, and any future design-weighted target remain distinct even when a particular fixture produces the same number. +- Cross-classified and multiple-membership weights come from their owning composition contract; occasion or pair count must never be inferred as membership weight. +- One transform, route, clock, refusal, or weighting option does not create a bounded context or a new ADR identity. +- Compatibility adapters may preserve public callers during a landed migration, but domain ownership and dependency direction must remain explicit. + +## Verification + +PR #310 is the current landing vehicle for this clarification. Its RED lineage includes an extreme stable-drift case that failed because `-2a` overflowed despite a representable stationary variance, a cancelled-stationary case in which the final scalar standardisation is representable even though `q / (-2a)` itself lies outside binary64 range, and a typed event-time contract that could not compile before the value object existed. The repaired source keeps actual stationary-variance recovery fail-closed when `p` itself is requested, but standardized scalar maps validate stationarity algebraically and avoid materializing an intermediate that cancels. The #327 fold adds CWC-then-pairwise-mean residual log-rate with Curran refusal and known-truth already-centered recovery of `ln(0.5)` at `Δt = 1`; arbitrary admitted intervals recover the exact `ln(0.5) / Δt` map. + +Issue #495 and PR #310 make the irregular-rate weighting identity explicit. `LagPairAverageV1` preserves the existing pair-weighted scalar while exposing unit/pair denominators and refusal counts. Unequal-follow-up fixtures distinguish that target from an independently computed equal-unit comparator, including a highly unbalanced rate-associated follow-up case. A deterministic informative-missingness Monte Carlo contract evaluates the declared pair target against truth conditioned on actually admitted lag-pair counts and reports attempted/recovered/failed replicates, bias, RMSE, Monte Carlo uncertainty, interval coverage, and replay reproducibility. `UnitAverageV1` remains typed but unavailable rather than being approximated with another TEPP-local generic mean. + +The #486 fold is verified against the same landing vehicle rather than retained as a second `psychometric_core` authority. Its Hamaker claim boundary is implemented by `longitudinal_core::center_occasion_mean_event_lags` and `recover_occasion_mean_centered_irregular_residual_log_rate`. Regressions cover numeric signed-zero identity, duplicate-unit admission, intermediate-sum overflow with a representable occasion mean, permutation stability, known drift recovery, and the same-panel distinction between occasion-mean residuals and person-mean CWC residuals. The dedicated research note records the primary-source trace and explicitly refuses promotion to a within-person effect. + +This addendum remains Proposed while its landing vehicle is Draft. Accepted/protected-main maturity is not claimed until the exact landing head passes the live ruleset, the required immutable numerical dependency is released and consumed, and the change is merged normally. diff --git a/docs/product/prd-v0.4-amendment-longitudinal-time-ownership.md b/docs/product/prd-v0.4-amendment-longitudinal-time-ownership.md new file mode 100644 index 000000000..21a6bc178 --- /dev/null +++ b/docs/product/prd-v0.4-amendment-longitudinal-time-ownership.md @@ -0,0 +1,24 @@ +# TEPP PRD v0.4 amendment — Longitudinal event-time ownership + +**Parent product baseline:** `docs/product/prd-v0.4-approved.md` +**Amendment:** v0.4-LONGITUDINAL-TIME-1 +**Status:** Proposed clarification; protected-main implementation and acceptance remain pending +**Recorded:** 2026-09-01 + +This amendment narrows implementation authority without changing TEPP's approved product thesis or scientific claims. + +TEPP Longitudinal Modeling owns temporal/event composition for longitudinal psychometrics. Product APIs that interpret a duration as substantive event time must accept an event-time domain value rather than a bare numeric duration. This preserves the PRD's distinction among event/valid, assertion, document, system, availability time, and knowledge cutoff and prevents measurement occasion or another clock from being silently substituted for substantive event time. + +For event-time lagged association, the product contract requires lagged covariance, both occasion-specific marginal variances, and an admitted positive finite event-time interval. Nonstationary correlation must not replace the later marginal variance with the earlier one. + +CWC-then-irregular residual log-rate is Longitudinal Modeling composition, not a generic psychometric kernel. Consecutive unit-mean-centered pairs keep typed event-time intervals. The pairwise mean of the Driver, Oud, and Voelkle (2017, Eq. 3) scalar inverse on nonzero same-sign residuals is not raw-process drift (Curran & Bauer, 2011, Eq. 36). Already-centered irregular pairs may recover that inverse, including `ln(0.5)`. This is not DSEM and not Newton least-squares. + +The weighting target is part of that scientific contract. `tepp.irregular_rate.lag_pair_average.v1` gives every admitted consecutive lag pair equal weight, so units with more admitted pairs contribute more total weight. Its evidence must report candidate/contributing units and candidate/admitted/refused pairs, including refusal categories that change the admitted-pair population. `tepp.irregular_rate.unit_average.v1` is a distinct equal-unit target: each unit must first retain identity through its within-unit summary before units are combined with equal weight. The latter remains fail closed until TEPP can consume the immutable released reusable finite-mean contract from fast-mlsirm. Occasion count, follow-up length, missingness, or cross-classified/multiple-membership composition must not be silently reinterpreted as a design or membership weight. A future explicitly design-weighted target requires its own versioned contract and denominator policy. + +Occasion-mean event-time composition is also Longitudinal Modeling composition. For Hamaker, Kuiper, and Grasman (2015, Eq. 1a), the product may form `p_it = x_it - μ_t` only from an occasion-specific group mean on numeric event time. Numeric `-0.0` and `+0.0` are one occasion, a unit may contribute at most one observation to an occasion, each admitted occasion must retain multilevel evidence, and consecutive unit-specific pairs keep their own positive finite event-time interval. Occasion means must be deterministic under row permutation and must remain recoverable when the final mean is representable even if a naive intermediate same-sign sum would overflow. These deviations are not person-mean CWC residuals and are not within-person effects by themselves; stable between-person differences may remain. TEPP therefore refuses promoting the resulting occasion-residual log-rate to RI-CLPM, DSEM, causal, or within-person authority. + +For the scalar stationary Driver, Oud, and Voelkle (2017) p. 16 `discreteDRIFTstd` mapping, TEPP may expose a deterministic event-time transform only when stable drift and positive stationary within-person variance are established. Numerical equality with `exp(a Δt)` in that scalar stationary special case does not make unstandardised `discreteDRIFT` and `discreteDRIFTstd` the same estimand. When the stationary variance cancels algebraically from a standardized scalar map, positive real-valued stationarity is established from finite positive diffusion and stable finite drift; the cancelled variance is not required to fit in binary64 if the final standardized result is representable. Conversely, a nonzero exact transition that rounds to a false endpoint such as `1.0` or `0.0` remains fail-closed rather than being reported as no change or complete dissipation. + +Reusable static/generalized-mixed/dependence-aware psychometric arithmetic remains owned by `ContextualWisdomLab/fast-mlsirm`; TEPP consumes versioned contracts through an anti-corruption layer and does not retain duplicate production kernels. LLM execution remains `contextual-orchestrator`-owned and cannot substitute for numerical estimation or scientific acceptance. + +Acceptance for this amendment requires typed event-time admission in the public longitudinal boundary, regression coverage for wrong-clock-shaped numeric input and extreme finite binary64 cases, explicit occasion-vs-CWC claim-boundary recovery on the same longitudinal panel, deterministic unequal-follow-up fixtures that distinguish pair-weighted from equal-unit targets, realistic leakage-safe Monte Carlo recovery for the declared estimand with attempted/recovered/failed denominators and Monte Carlo uncertainty, current-head Rust/documentation/security evidence, and normal protected-main integration. Branch-local implementation does not constitute released product capability. diff --git a/docs/research/component-recovery-denominator-identity.md b/docs/research/component-recovery-denominator-identity.md new file mode 100644 index 000000000..cd45b89f8 --- /dev/null +++ b/docs/research/component-recovery-denominator-identity.md @@ -0,0 +1,44 @@ +# Component recovery denominator identity + +## Decision + +Known-truth component RMSE in the Longitudinal Modeling bounded context uses one contribution per admitted scientific component identity. A stable `Between` component is unit-level, not occasion-level: TEPP's decomposition emits exactly one such component per unit and encodes its non-temporal carrier with canonical `occasion_index = 0`. A `Within` component remains occasion-specific and is identified by `(unit_index, occasion_index, component_level)`. A `Between` row with a nonzero occasion index is therefore an invalid alias rather than a second target or an implicit weight. + +Duplicate identity multiplicity is invalid evidence and fails closed with `LongitudinalError::InvalidComponentPayload`; it is not an implicit observation weight. The order in which truth and recovered rows are serialized is also not part of scientific identity. Both slices are admitted as identity maps and residuals are accumulated in canonical identity order. + +This boundary matters in three distinct ways. Repeating an already-matched component changes the RMSE denominator and can improve or worsen the reported recovery metric without adding a new known-truth target. Giving a stable between-unit component different occasion numbers creates the same denominator distortion while disguising the duplicate as two tuple identities. Requiring pairwise row order can instead reject two scientifically identical recovery sets merely because an upstream repository, query, or transport emitted the same unique component identities in a different order. Even after identity lookup removes pairwise order coupling, iterating residuals in caller-provided truth order can still change the last bit of the scaled binary64 sum of squares. TEPP therefore treats bounded-context component identity as authoritative for admission, alignment, denominator membership, and deterministic accumulation. + +Curran and Bauer (2011) distinguish stable between-person information from occasion-specific within-person deviations in longitudinal decomposition, while Hamaker, Kuiper, and Grasman (2015) explicitly separate stable between-person differences from the within-person process. Those sources support the scientific level distinction; `occasion_index = 0` is TEPP's canonical storage/projection encoding for its one stable between component per unit, not a claim that the stable component occurs at substantive occasion zero. Morris, White, and Crowther (2019) frame simulation evaluation around explicitly defined estimands and performance measures. For TEPP's known-truth recovery contract, denominator membership and truth-to-recovery alignment therefore follow the identified longitudinal component rather than accidental row multiplicity, serialization order, or a synthetic occasion alias. The current published *Standards for Educational and Psychological Testing* remains the 2014 AERA/APA/NCME edition while a revision is in progress; TEPP treats reproducible evidence and clearly specified score/evidence interpretation as governing constraints rather than inferring validity from transport shape. + +## RED → repair trace + +- Duplicate-identity RED `698f12f5b2f7a3c194e9d1d3f00c5aeaf10591f8`: `crates/longitudinal_core/tests/component_rmse_duplicate_identity_contract.rs` supplies the same `(unit, occasion, level)` twice in both truth and decided series. The predecessor implementation accepted it and therefore allowed duplicate multiplicity to change the recovery denominator. +- Duplicate-identity repair `2fae4cb2e7df2845270bd27192000ca370fb05ad`: `crates/longitudinal_core/src/component.rs` records admitted component identities before residual accumulation and rejects a duplicate. +- Cross-slice row-order RED `8ad72ac91cbddd5ce6432fc70630bad6ce7072ce`: `crates/longitudinal_core/tests/component_rmse_permutation_identity_contract.rs` supplies the same three unique known-truth components and recovered values in a different recovered-row order. The predecessor pairwise `zip` path rejects the scientifically identical perfect recovery solely because serialization order differs. +- Initial identity-alignment repair `2dd9537e04dd2048559ba707fecd2404db5a2a31`: recovered rows are indexed by `(unit, occasion, level)` so decided-row permutation no longer changes admission. Review of that repair found that residual accumulation still followed caller-provided truth order, leaving deterministic binary64 output order-sensitive. +- Truth-order rounding RED `5fb93c40eddbd9e7920196ef09594457b8ac72d3`: `crates/longitudinal_core/tests/component_rmse_truth_permutation_contract.rs` uses residual magnitudes `1`, `1e-100`, `3`, and the representable value immediately below `1`. Two truth-row permutations contain exactly the same identity-value pairs but drive the scaled sum-of-squares through different rounding paths and produce different RMSE bit patterns on the predecessor identity-aligned implementation. +- Causal deterministic repair `025dce7fd98cfb4f94ea790cacd555b744095377`: both truth and recovered slices are admitted into unique identity maps, truth identities are sorted by `(unit, occasion, level wire name)`, and residual accumulation follows that canonical order. Missing identities still fail closed. Existing overflow-safe residual scaling, nonzero-underflow refusal, and exact perfect-recovery zero remain unchanged. +- Edge-coverage reinforcement `976ce7d710125717ff2f8daeb943d54278c4acde`: a truth-side duplicate with unique recovered identities exercises the independent truth uniqueness gate rather than relying only on a payload where both slices contain duplicates. +- Stable-between alias RED `0a03041c5c1c2152fe90f86204edf2c957cebbfe`: `crates/longitudinal_core/tests/component_rmse_stable_between_identity_contract.rs` supplies two stable `Between` rows for one unit under different occasion indices. The predecessor identity tuple accepts both as separate targets and lets the alias change the recovery denominator. +- Fixture correction `53563e975dd10e4f18218880cca2bb4cd79d0e73`: the earlier cross-slice permutation contract now keeps all stable `Between` fixtures at canonical occasion index `0`; transport-order coverage remains intact without encoding a scientifically invalid between-at-occasion fixture. +- Stable-between identity repair `ec2c1219127e834d89051415f1de8e1cb48507aa`: `component_root_mean_square_error` validates the bounded-context identity before either map insertion. `Between` requires canonical occasion index `0`; `Within` retains its actual occasion. The repair changes admission only and leaves residual arithmetic, canonical accumulation order, nonzero-underflow refusal, and exact perfect-recovery zero unchanged. +- Public API: `component_root_mean_square_error`. +- Domain owner: `crates/longitudinal_core`; this is longitudinal Validation Evidence identity/admission policy, not reusable static psychometric arithmetic for `fast-mlsirm`. + +## Invariant + +For an admitted recovery vector of length `n`, there are exactly `n` unique scientific component identities in each slice and the two identity sets are equal. Each unit can contribute at most one stable `Between` target, encoded with `occasion_index = 0`; each `Within` target is identified by its actual unit and occasion. Any permutation of either slice that preserves those admitted identity-value pairs yields bit-identical deterministic CPU `f64` RMSE. A caller that needs weighted recovery must use a separately named, explicitly weighted contract with its own denominator and validation evidence; duplicate rows and between-component occasion aliases are not weights. + +## References + +American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). *Standards for educational and psychological testing*. American Educational Research Association. https://www.testingstandards.net/open-access-files.html + +Curran, P. J., & Bauer, D. J. (2011). The disaggregation of within-person and between-person effects in longitudinal models of change. *Annual Review of Psychology, 62*, 583–619. https://doi.org/10.1146/annurev.psych.093008.100356 + +Hamaker, E. L., Kuiper, R. M., & Grasman, R. P. P. P. (2015). A critique of the cross-lagged panel model. *Psychological Methods, 20*(1), 102–116. https://doi.org/10.1037/a0038889 + +Morris, T. P., White, I. R., & Crowther, M. J. (2019). Using simulation studies to evaluate statistical methods. *Statistics in Medicine, 38*(11), 2074–2102. https://doi.org/10.1002/sim.8086 + +### Standards currency note + +The joint AERA/APA/NCME site still distributes the 2014 edition as the current published edition. A joint revision process is active; this note should be revisited when the successor edition is formally published rather than treating revision activity as a released standard. diff --git a/docs/research/cwc-irregular-residual-log-rate.md b/docs/research/cwc-irregular-residual-log-rate.md new file mode 100644 index 000000000..d126a29ec --- /dev/null +++ b/docs/research/cwc-irregular-residual-log-rate.md @@ -0,0 +1,189 @@ +# CWC-then-irregular residual log-rate + +## Domain owner + +This contract belongs to TEPP's **Longitudinal Modeling** bounded context. +Public operations whose meaning depends on substantive event time expose +`EventTimeInterval` rather than a generic clock label. Unique evidence is +folded from Draft #327 into `longitudinal_core` on landing vehicle #310. +Reusable static/generalized-mixed kernels remain fast-mlsirm-owned. + +This is not DSEM, not Newton least-squares, not ctsem estimation, and not +raw-process autoregressive drift. + +## Estimands + +1. `center_within_unit_event_lags` subtracts the unit mean (CWC) and emits + consecutive [`LaggedWithinResidual`] pairs on admitted event intervals. + Singleton units are skipped and at least two units must contribute lags. +2. `recover_within_unit_irregular_residual_log_rate` is the mean of the + Driver, Oud, and Voelkle (2017, Eq. 3) scalar inverse + `a = ln(|later| / |earlier|) / Δt` on nonzero same-sign residuals. Nearby + represented magnitudes are evaluated from their exact binary difference + with `ln_1p`; more extreme ratios use a direct finite quotient when possible + and otherwise fall back to `ln|later| − ln|earlier|`. +3. `recover_centered_irregular_residual_log_rate` is the already-centered + path. It does **not** re-center. Residuals must be nonzero and have equal + sign. Known-truth pairs `(1, 0.5)` with `Δt = 1` recover `ln(0.5)`; for a + general admitted interval the scalar result is `ln(0.5) / Δt`. +4. `refuse_cwc_residual_log_rate_as_raw_process_drift` always fails closed. + +## Log-ratio numerical contract + +The log-rate must represent the ratio of the two **represented residual +magnitudes**, not the ratio after an avoidable intermediate quotient rounding. +That distinction is material near a power-of-two boundary. If `earlier` is the +binary64 value immediately below `2.0` and `later = 2.0`, the represented +magnitude change is one ULP below the boundary. Forming `later / earlier` +first rounds that ratio to `1 + 2^-52`; applying `ln` then reports a change +close to `2^-52`. The correctly rounded logarithm of the represented endpoint +ratio is instead close to `2^-53`, so the quotient-first path nearly doubles +the temporal rate. + +RED `766ddc7a3d95102450d663f06d2c577306964d79` adds +`irregular_rate_adjacent_growth_contract.rs` through the public already-centered +recovery API. Causal repair `16f21d9a223fb34d52f24d148c7cad9b385e1d91` +uses the exact represented magnitude difference for values within a factor of +two. Growth evaluates `-ln1p(-(later-earlier)/later)` and decay evaluates +`ln1p((later-earlier)/earlier)`. This keeps the difference subtraction in the +Sterbenz exact-subtraction region and avoids the quotient-first rounding step. +Direct-ratio and log-domain fallbacks remain for extreme scale differences, so +previous overflow/underflow recovery behavior is retained rather than traded +for local near-equality accuracy. + +This is a numerical implementation repair of the Driver et al. (2017, Eq. 3) +Longitudinal estimand. It does not create a second arithmetic owner, change the +scientific target, or authorize CWC residual rates as raw-process drift. + +## Numerical mean contract + +CWC means, occasion means, and admitted irregular-rate means share +`irregular_residual::scaled_compensated_mean`. Same-sign finite terms use an +exact power-of-two scale, deterministic ordering, and compensated summation so +an avoidable raw partial-sum overflow does not reject a representable mean and +non-power-of-two normalization does not add an earlier rounding step. + +Mixed-sign inputs cancel opposite signs from the largest magnitudes first. +The surviving one-sign residuals are then normalized and compensated, but the +division uses the **original sample count directly** before scaling back. The +implementation must not first round a mean over only the surviving residuals +and then weight that rounded intermediate. For the minimum-subnormal ULP `u`, +`[-20u, -20u, 9u]` has exact mean `-31u/3`, which rounds once to `-10u` in +binary64; the retired retained-mean-then-weight path produced `-11u`. + +RED `ae5e61f9a829adbfed2ea13c5705d4b85d80b0d6` pins this through the public +CWC API. Causal repair `39469067aca2fa93e2fa4c914848f7cec8031811` +applies the original denominator inside the shared normalized sum. Contract +`mixed_sign_mean_rounding_contract.rs` also exercises the public occasion-mean +path so the shared numerical authority cannot silently split again. + +A second boundary applies after the normalized sum. If the retained residual +mass has one sign and is nonzero, its real mean is also nonzero. Binary64 may +still round that final mean to exact zero when the magnitude lies below half +the minimum subnormal. Reporting that zero would change a nonzero temporal +rate into exact no-change. `same_sign_mean_over_total` therefore accepts zero +only when its admitted mass was itself exact zero; otherwise final zero is +`InvalidTemporalTransformInput`. + +Public RED `96f1c3342071173ba870e6ef5e11b826391c7621` adds +`irregular_rate_mean_underflow_contract.rs`. One already-centered pair has an +exact zero rate and one has a representable positive subnormal rate; averaging +that nonzero rate with fifteen exact-zero rates makes the real mean positive +but too small for binary64. Causal repair +`ae5081d8ae580c19e73aff7f03711e50c3c631dd` fails closed at the shared mean +boundary rather than returning exact zero. Exact cancellation of mixed-sign +rates still returns zero before the one-sign helper, and an all-zero input +still returns canonical `+0.0`. + +This arithmetic remains a Longitudinal composition primitive in this stack. +It is not a new reusable static psychometric kernel. A fast-mlsirm handoff +requires semantic-equivalence evidence and an immutable released owner +contract rather than source copying. + +## Identification and admissibility + +Curran and Bauer (2011, pp. 583–619; PMC3059070 XML opened 2026-09-02) +show that person-mean centering of a time-varying covariate related to time +is biased for the within-person effect. The licensed alternative is the +person-specific OLS residual of the covariate on time (their Eq. 36). An +autoregressive series is related to time, so CWC of a raw AR path does not +recover process drift `a`. Traditional person-mean centering is the +horizontal line in their Figure 9; detrend uses the individual regression +line. + +T=2 CWC is always `r, −r` (empty admissible). T=3 arithmetic progression +has a zero residual (empty admissible). Opposite-sign and zero residuals +are skipped. An empty admissible set fails closed. + +Driver et al. (2017, Eq. 3, p. 5; JSS PDF opened 2026-09-02) write the +discrete solution `η(t) = e^{A(t−t0)} η(t0) + ⋯`. The noiseless scalar +inverse is `a = ln(later / earlier) / Δt`. Voelkle, Oud, Davidov, and +Schmidt (2012, Eq. 7) print the same exponential map; the ZORA PDF was +not re-opened this cycle (invalid cross-reference table) and is cited +only as previously opened lineage. + +No claim is made for matrix `expm`, Kalman filtering, ESEM, DSEM, or +ctsem estimation. + +## Recovery evidence + +Already-centered irregular pairs recover known `a` at machine precision. +CWC of a raw AR path with a stable between-unit offset does **not** recover +that `a`. Fail-closed cases cover empty and singleton-only rows, fewer than +two lag-contributing units, non-positive intervals, non-finite scores, +non-representable means, overflowing CWC residuals after a finite mean, +tiny intervals with huge log-ratios, underflowed nonzero individual rates, +underflowed nonzero final mean rates, and the Curran refusal. Adjacent-float +power-of-two growth additionally verifies that a quotient-rounding artifact +cannot nearly double a representable temporal change. + +Exact zero CWC residuals also have one public identity. IEEE 754 binary64 +has distinct `+0.0` and `-0.0` encodings, and subtraction from a canonical +`+0.0` unit mean can therefore leave `-0.0` when an observed score is +signed zero or otherwise equals that mean. That sign bit does not represent +positive versus negative within-person change: the person-mean deviation is +exactly zero. The public CWC lag boundary therefore canonicalizes only +validated exact-zero residuals to `+0.0`; private numerical intermediates +and caller-constructed already-centered pairs remain free to retain signed +zero. IEEE Std 754-2019 remains the active published floating-point +standard while IEEE P754 is the active revision project as of 2026-09-03. +This is the same public-identity contract as +`decompose_within_between`; it is not a license to treat CWC residuals as +raw-process drift (Curran & Bauer, 2011, Eq. 36). + +Signed-zero traceability: + +- RED `crates/longitudinal_core/tests/cwc_signed_zero_contract.rs` drives + `center_within_unit_event_lags` with a `-0.0` score equal to the unit + mean so the public later/earlier residual pair cannot keep `-0.0`. +- Causal repair `f98ee093f6f7fd318ad6623ab44313385195f956` canonicalizes an exact-zero CWC residual only + after finite-result validation. +- Standard authority — IEEE. (2019). *IEEE standard for floating-point + arithmetic* (IEEE Std 754-2019). IEEE. The canonical repository register + is `docs/research/standards-and-literature.md`. + +The current public numerical regressions include same-sign raw-sum overflow, +full-exponent mixed-sign cancellation, minimum-subnormal cancellation, +halfway ties-to-even for same-sign means, the mixed-sign `-31u/3` case, +nonzero irregular-rate means that are not representable in binary64, and +adjacent-float log-rate changes across a power-of-two boundary. Hosted +exact-head CI and independent review remain delivery gates; these source +contracts do not by themselves establish release readiness. + +## Traceability + +Curran, P. J., & Bauer, D. J. (2011). The disaggregation of within-person +and between-person effects in longitudinal models of change. *Annual Review +of Psychology, 62*, 583–619. https://doi.org/10.1146/annurev.psych.093008.100356 + +Driver, C. C., Oud, J. H. L., & Voelkle, M. C. (2017). Continuous time +structural equation modeling with R package ctsem. *Journal of Statistical +Software, 77*(5), 1–35. https://doi.org/10.18637/jss.v077.i05 + +Voelkle, M. C., Oud, J. H. L., Davidov, E., & Schmidt, P. (2012). An SEM +approach to continuous time modeling of panel data: Relating +authoritarianism and anomia. *Psychological Methods, 17*(2), 176–192. +https://doi.org/10.1037/a0027543 + +IEEE. (2019). *IEEE standard for floating-point arithmetic* (IEEE Std +754-2019). IEEE. diff --git a/docs/research/irregular-rate-estimand-weighting.md b/docs/research/irregular-rate-estimand-weighting.md new file mode 100644 index 000000000..5b859b48c --- /dev/null +++ b/docs/research/irregular-rate-estimand-weighting.md @@ -0,0 +1,96 @@ +# Irregular residual log-rate weighting and estimand identity + +## Status and scope + +This note records a Longitudinal Modeling estimand boundary exposed by TEPP PR #310 and issue #495. The branch now contains a typed first-release candidate contract, but it remains Draft and is not protected-main or release authority. + +Scientific finding head: `df0b4d6d3e0622de3c988b840114fbdb41e5d1b0`. Typed-contract implementation lineage starts at `c7f55acb347ceac38675d9566eff767372739dba`; public contract tests start at `f45e83ba5223ad9d89022482e759ad9e705b229c`; refusal-class denominators are completed by `a3065f6d7589d5a310e54f85d6447b27414955ba`; the explicit unavailable-estimand error boundary lands through `432da36b44af921557f6cb92f4a71b0c8a56105d` and `8e579512c80e9544d594261186f279d9d896c5e2`; a represented same-sign rate-refusal denominator is exercised by the corrected extreme-interval fixture at `f2d31ac6120de7a2be235df2f5809556060d0985`. + +`center_within_unit_event_lags` forms one consecutive event-time lag pair for each admitted adjacent occasion inside a unit. `LaggedWithinResidual` carries the earlier residual, later residual, and typed event interval but not the originating unit identity. The existing scalar recovery therefore computes a lag-pair-average estimand: every admissible pair enters one common rate vector before averaging. + +The Draft candidate now names that target explicitly as `tepp.irregular_rate.lag_pair_average.v1`. `tepp.irregular_rate.unit_average.v1` is also a typed name, but it fails closed with `LongitudinalError::IrregularRateEstimandUnavailable` until an immutable released reusable finite-mean contract can support the second aggregation step without adding another TEPP-local generic summation kernel. A scientifically valid but not-yet-activated estimand request is therefore not misclassified as malformed temporal input. + +## Two distinct estimands + +Let unit `i` contribute `k_i >= 1` admitted consecutive lag pairs after all temporal and residual admission rules. Let `r_ij` be the scalar event-time log-rate for admitted pair `j` of unit `i`. + +The current pair-average target is + +\[ +\theta_{pair} += \frac{\sum_i \sum_{j=1}^{k_i} r_{ij}} + {\sum_i k_i}. +\] + +Every admitted lag pair receives equal weight. A unit with more admitted occasions can therefore receive more total weight through more consecutive pairs. + +A different, equal-unit target is + +\[ +\theta_{unit} += \frac{1}{I}\sum_{i=1}^{I} + \left(\frac{1}{k_i}\sum_{j=1}^{k_i} r_{ij}\right). +\] + +Every contributing unit receives equal final weight after its admitted pair rates are summarized within unit. The targets are not algebraically interchangeable when `k_i` varies. They may also target different populations when follow-up duration, observation count, missingness, or pair admissibility is associated with the longitudinal process. + +The current pair-average summary attributes candidate/contributing-unit denominators using the same deterministic `BTreeMap` unit order and consecutive-pair counts used by `center_within_unit_event_lags`. This is sufficient for denominator evidence for the existing pair-weighted target. It is deliberately not treated as the future equal-unit computation boundary: activating `unit_average.v1` must preserve unit identity through within-unit numerical aggregation instead of depending on flattened-pair reconstruction. + +## Failure denominators are part of the estimand + +The Draft `IrregularRateSummary` reports: + +- candidate units with at least two admitted event-time occasions; +- units contributing at least one admitted scalar rate; +- candidate consecutive pairs; +- admitted pairs; +- zero/opposite-sign refusals; +- same-sign pairs whose represented scalar log-rate is not admissible. + +`refused_pairs()` is the sum of the two pair-level refusal classes. Non-finite input rows remain a payload-level admission failure before a scientific summary is constructed; they are not silently converted into missing pair observations. + +The second refusal class matters separately from sign/zero exclusion. A pair can contain two finite, nonzero, same-sign centered residuals and still fail represented-rate admission: for example, adjacent residual magnitudes over an extreme but finite event-time interval can imply a nonzero real log-rate whose binary64 quotient collapses to zero. That pair is counted in `nonrepresentable_rate_refused_pairs()`, not silently erased or conflated with an opposite-sign transition. + +If no pair is numerically admissible after otherwise valid CWC/event-time admission, the summary returns `estimate = None` while retaining the complete unit/pair denominators. The legacy scalar recovery remains fail-closed for that case. This separation lets evidence reporting preserve its failure population without changing legacy scalar semantics. + +Dropping a refused pair changes `k_i` and can change both the pair-average weight and whether a unit contributes to a unit-average target. A unit with no admitted pairs must therefore not disappear silently from a denominator whose interpretation says otherwise. + +## Public contract evidence + +`crates/longitudinal_core/tests/irregular_rate_estimand_contract.rs` fixes deterministic fixtures for unequal pair counts and pair-level refusal classes. The contract establishes that: + +- `LagPairAverageV1` bit-matches the legacy pair-average scalar recovery; +- the pair-average differs from the independently computed equal-unit comparison when admitted pair counts differ; +- row permutation leaves the typed summary unchanged; +- a two-unit fixture with no admissible rates retains `candidate_units = 2`, `candidate_pairs = 2`, `admitted_pairs = 0`, `refused_pairs = 2`, and `estimate = None`; +- a finite same-sign adjacent-magnitude pair over an event interval of `f64::MAX` increments `nonrepresentable_rate_refused_pairs()` while a second unit supplies an admitted rate, proving that this failure denominator is observable without turning the whole evidence summary into an error; +- `UnitAverageV1` has a stable external name and fails closed with `IrregularRateEstimandUnavailable` while the owner mean release is unavailable. + +This evidence resolves the naming/denominator ambiguity for the pair-weighted first-release candidate. It does not claim equal-unit scientific acceptance and does not repair #310's independent mixed-sign binary64 mean RED. + +## Relation to unequal and informative cluster size + +This is not a cluster-randomized treatment-effect model. However, the statistical identification issue is analogous: unequal numbers of observations inside a higher-level unit can induce different weighting targets, and informative cluster or subcluster size can make those targets materially different. The estimator has to match the declared estimand rather than inheriting weights accidentally from record multiplicity. + +Wang, Kong, and Datta (2011) study clustered longitudinal data and show that informative cluster size can invalidate ordinary marginal inference when cluster size is related to the outcome distribution. Huang and Leroux (2011) further show that the appropriate weights depend on the population of interest and on within-cluster covariate structure. Kahan et al. (2023) give a clear modern estimand distinction between equal participant weighting and equal cluster weighting. TEPP does not import their treatment-effect estimands; it imports the narrower methodological requirement that aggregation weights are part of estimand identity. + +## Decision and documentation boundary + +Issue #495 remains open because a branch-local typed API is not the complete scientific acceptance package. Before buyer-facing promotion: + +1. The first released target must be named consistently in PRD/TRD/TRACEABILITY and a superseding scientific-estimand ADR. The current ADR directory already contains historical number collisions, so this Draft does not mint another potentially colliding ADR identifier; canonical documentation ownership must allocate and repair that identity before acceptance. +2. A unit-average path must preserve unit identity until within-unit rates are summarized; occasion count cannot stand in for an externally defined design or membership weight. +3. Known-truth acceptance must include balanced and highly unbalanced occasion counts, informative missing/follow-up patterns, irregular intervals, row permutation, worker-count determinism, and explicit failure denominators. +4. Cross-classified and multiple-membership extensions must retain their declared membership structure and must not collapse to a primary group merely to obtain one scalar weight. +5. Neither target is automatically a raw-process autoregressive/DSEM effect. Existing refusal boundaries around CWC and occasion-mean residuals remain in force. +6. Reusable finite binary64 sum/mean arithmetic remains owned by `ContextualWisdomLab/fast-mlsirm`. TEPP may consume it only from an immutable released contract; this note does not authorize another generic summation implementation in `longitudinal_core`. + +The current mixed-sign binary64 mean RED in #310 remains independent. Clarifying the estimand does not make that numerical RED pass and does not authorize a merge. + +## References + +Huang, Y., & Leroux, B. (2011). Informative cluster sizes for subcluster-level covariates and weighted generalized estimating equations. *Biometrics, 67*(3), 843–851. https://doi.org/10.1111/j.1541-0420.2010.01542.x + +Kahan, B. C., Li, F., Blette, B., Jairath, V., Copas, A., & Harhay, M. O. (2023). Informative cluster size in cluster-randomised trials: A case study from the TRIGGER trial. *Clinical Trials, 20*(6), 661–669. https://doi.org/10.1177/17407745231186094 + +Wang, M., Kong, M., & Datta, S. (2011). Inference for marginal linear models for clustered longitudinal data with potentially informative cluster sizes. *Statistical Methods in Medical Research, 20*(4), 347–367. https://doi.org/10.1177/0962280209347043 diff --git a/docs/research/longitudinal-binary64-mean-owner-handoff.md b/docs/research/longitudinal-binary64-mean-owner-handoff.md new file mode 100644 index 000000000..6361fd6a2 --- /dev/null +++ b/docs/research/longitudinal-binary64-mean-owner-handoff.md @@ -0,0 +1,104 @@ +# Longitudinal binary64 mean owner handoff + +## Decision status + +**Proposed consumer boundary.** This record does not activate a new numerical algorithm. TEPP keeps the current Longitudinal Modeling RED until the reusable numerical owner has protected-integrated and immutably released a contract that satisfies the evidence below. + +## Consumer finding + +TEPP PR #310 exposes a compact public RED through `crates/longitudinal_core/tests/mixed_sign_mean_rounding_contract.rs`. + +The represented rate values are `P = 0x1.8p+106` (`0x4698000000000000`), `-2^53`, and `-1`. The exact real sum is `P - 2^53 - 1`. The correctly rounded mean is `0x1.fffffffffffffp+104` (`0x467fffffffffffff`), while TEPP's current `scaled_compensated_mean` path returns `0x4680000000000000`, one ULP high because the same-side coalescing step can lose the final `-1` before the original-count division. + +This is an ordinary finite three-value input through the public already-centered residual path. It is not a resource-extreme synthetic witness. The latest fully terminal predecessor evidence before this documentation repair is TEPP PR #310 head `8eb95d14f43ac40b06d0d039ef4d94ad1a053924`: Rust Foundation run `34669350804` passes exact-head checkout, pinned toolchain/tool verification, formatting and Clippy, then reports 1,577 PASS / 1 FAIL / 0 skipped across 1,578 tests, with the single failure being the public mixed-sign half-ULP rounding contract. Production line and branch coverage generation on the same run also fails before authored-denominator enforcement because that product RED stops the instrumented suite. This paragraph is predecessor evidence, not a self-referential claim about the live PR head; current acceptance authority is always the live PR head plus its own terminal checks. No coverage denominator relaxation or alternate local mean is authorized. + +## Bounded-context ownership + +TEPP owns the temporal estimand: event-time admission, Driver-style log-rate construction, CWC semantics, longitudinal evidence composition, and the decision to publish or refuse a result. + +Reusable finite binary64 sum/mean arithmetic is domain-neutral numerical infrastructure and belongs to `ContextualWisdomLab/fast-mlsirm`. Canonical owner issue `fast-mlsirm#1814` and implementation PR `fast-mlsirm#1816` own this primitive. TEPP must not copy the owner source, pin a mutable PR head, introduce a second generic summation algorithm, or reinterpret an unreleased owner branch as dependency authority. + +Repository-owned GPU/CI acquisition and parity evidence is separately owned by `fast-mlsirm#1717`. TEPP and the numerical PR must not duplicate that CI repair, lower its GPU capability invariant, substitute CPU fallback as GPU evidence, or treat a skipped parity test as acceptance. + +The immutable fast-mlsirm release remains `v0.9.1`; it predates this contract. + +## Fresh owner evidence + +The live `fast-mlsirm#1816` head is `432765ccf633c9802e0f796ceeb4d6d572059acf` on protected `main@493326f2de49ea1704da0ded19868ed05d2fe00f`. The PR is open, **Draft**, mergeable, and unmerged. Its owner lane has currentized the body to the same head and Draft-containment state; no predecessor review or hosted success is promoted across the head change. + +The numerical implementation entered at `b5fec1d33c02e8ea969405e59747f0505e690d20`. Subsequent ordinary-forward commits added the independent exact oracle and directed cancellation/non-finite fixtures (`087df1d...`), Proposed ADR-0029 (`9a8614f...`), doctoring/changelog currentization, touched-function rustdoc coverage (`f00c31c...`), and finally official ISO/IEEE record links (`432765c...`). The `f00c31c... -> 432765c...` delta changes only `docs/doctoring/correctly-rounded-binary64-mean.md`; the production arithmetic is unchanged. + +Current exact-head hosted evidence on `432765c...` is mixed and therefore non-mergeable: + +- native CodeQL `34606295285`, Security Scan `34606295363`, SAST Semgrep `34606295323`, and ClusterFuzzLite `34606295428` are GREEN; +- repository CI `34606295259` is RED because `gpu-smoke` job `103285576158` failed while installing the software Vulkan adapter, before Vulkan availability or GPU parity executed. Package and fuzz jobs are GREEN. That run is historical evidence of a failed acquisition path, not the current root-cause authority for GPU readiness; +- required delegated CodeQL PR `34606295286` is RED. Python job `103285476466` and Actions job `103285476530` both successfully read the current-head dispatch verdict and then failed at `Release runner or enforce current-head CodeQL verdict`; only afterward did dispatch job `103286411396` succeed. The live canonical repair vehicle is `.github#2051@558693e0333e48012beea142f739bc634b0674a7`, which now records a versioned rollout/bootstrap prerequisite because the protected handler and PR-head client currently use incompatible exact run-name identities. TEPP and fast-mlsirm must not copy that control-plane repair, synthesize receipts, or loop no-op reruns; +- current formal reviews are COMMENTED only. The CodeRabbit oracle/standards-link findings are resolved, but there is no qualifying submitted current-head `APPROVED` review; +- current Noema evidence admitted `orchestrator/free` but the verdict request ended in HTTP 502; the exact consumer evidence belongs to the contextual-orchestrator owner path rather than a leaf provider/model retry; +- current central coverage contexts still do not provide authoritative exact-head Rust owned line+branch evidence for this material Rust implementation; no denominator trick or Python line-touch substitute is acceptable; +- current Strix evidence failed before an authoritative scan during sandbox/bootstrap control, so absence of a scan is not a clean security result. + +## Canonical GPU/CI owner evidence + +The stronger current GPU diagnosis is `fast-mlsirm#1717`, not another patch inside #1816. Its live exact head is `0b31640928e07f4362ce27dad3d310e630ab1b5d`, open/Draft/mergeable on the same protected fast-mlsirm base. + +#1717's controlled predecessor admission successfully acquired and initialized image-local SwiftShader, then measured the actual adapter contract. `SwiftShader Device (Subzero)` exposes `max_storage_buffers_per_shader_stage = 10`, while the current marginal GPU layout requires at least 18 storage buffers per shader stage. The adapter therefore cannot execute the governed kernel topology. Lowering the 18-buffer requirement without redesign evidence, accepting CPU fallback, or relabeling skipped GPU parity as success would weaken the product contract. + +Exact source inspection narrows the mismatch to storage topology rather than the statistical equations. The E-step shader binds one uniform plus 17 storage buffers. The score shader binds one uniform plus 18 storage buffers: 14 read-only logical arrays and four read-write outputs (`theta_eap`, `theta_sd`, `xi_eap`, `loglik`). Because the limit is per shader stage, distributing the same storage bindings over additional bind groups does not reduce the requirement. + +A non-weakening owner-side candidate is therefore **arena packing**, not a lower capability assertion. Pack read-only `f32` tables into one storage arena, pack `u32`/CSR indices into a second arena, and pack output/state into a third read-write arena, with fixed `u32` offsets carried in the uniform block. The score path can then target roughly three storage bindings plus one uniform; the E-step can use the same resource shape. This proposal changes storage addressing, not the WGSL equations or the intended `f32` GPU arithmetic. It is a Proposed repair candidate for #1717 or a verified successor, not an accepted implementation and not TEPP source authority. + +The arena candidate is acceptable only if the owner proves, on one exact head: generated score/E-step layouts use no more than ten storage buffers; packing offsets and arena round trips are deterministic including empty/padded segments; CPU-`f64` parity thresholds on realistic marginal/EAP fixtures remain unchanged; the existing SwiftShader capacity probe is followed by the real explicit GPU parity path rather than skip/fallback; realistic per-iteration or p95 evidence shows the packing does not erase the GPU benefit; and recovery/error behavior remains intact. If the owner rejects arena packing, the resource-topology reason belongs in its ADR/evidence rather than restoring an arbitrary 18-buffer admission floor. + +The #1717 forward repair also moved the environment-specific capacity probe out of ordinary workspace tests into `crates/mlsirm-core/examples/gpu_adapter_capacity.rs`, so ordinary `cargo test --workspace` no longer performs hardware acquisition outside the dedicated GPU lane. Exact-head CI `34607995391` is terminal: `gpu-smoke` job `103291084354` configures image-local SwiftShader, proves the Vulkan loader, runs the isolated capacity probe, reproduces the 10-versus-18 buffer mismatch, and fails closed at that invariant; package job `103291084379` is GREEN. Rust job `103291084335`, Python 3.12 `103291084281`, Python 3.14 `103291084331`, and fuzz `103291084064` are CANCELLED, while aggregate `python` job `103292403176` is RED because its required matrix/GPU evidence is incomplete. Those cancellation conclusions are terminal facts only; this record does not infer their cause or promote them to GREEN. + +The next causal GPU repair therefore belongs to #1717 or its verified successor. #1816 must reacquire its own exact-head repository evidence only after that canonical CI path is protected-integrated; TEPP must wait for the resulting immutable numerical release. + +## Proposed numerical contract + +The owner candidate publishes the proposed identity `fast_mlsirm.binary64_mean@1.0.0`. Every finite binary64 value is represented exactly as an integer multiple of `q = 2^-1074`. Positive and negative totals are accumulated separately in fixed 34×`u64` magnitudes, the exact signed magnitude is divided by the original slice cardinality, and only the final rational result is projected to binary64 with round-to-nearest, ties-to-even. `exact_zero` distinguishes exact represented cancellation from a nonzero mean that rounds to signed zero. + +For the largest finite binary64 value, the coefficient in `q` units is `(2^53 - 1) * 2^2045`, requiring 2,098 magnitude bits. On supported `usize::BITS <= 64` targets, any materializable same-sign slice total is `< 2^2162`; 34 `u64` limbs provide 2,176 magnitude bits. The fixed width is therefore a representation bound, not a psychometric sample ceiling. + +The final mean must be rounded from the exact rational `(S / n) * 2^-1074`, not from a binary64-rounded sum. This also permits same-sign cases whose exact intermediate sum exceeds binary64 while their mean remains representable. + +## Owner acceptance already encoded in tests + +The current owner test surface includes the TEPP half-ULP counterexample and mirrored sign, `[1e16, -1, -1]`, `[f64::MAX, 1e-16, -f64::MAX]`, exact cancellation versus nonzero underflow, minimum-subnormal residue after MAX cancellation, subnormal/normal boundaries, normal and subnormal ties-to-even, binade carry, same-sign `f64::MAX`, permutation invariance, and empty/NaN/±infinity refusal. It also carries a deterministic 10,000-case subnormal-domain oracle using independent test-only exact integer/rational arithmetic rather than the production 34-limb accumulator. + +Those tests are necessary but not sufficient. One unchanged owner head still needs repository Rust/rustdoc/Clippy, actual owned statement and branch/edge coverage, package/fuzz/security/supply-chain evidence, zero valid unresolved findings, qualifying independent review, and terminal central controls. The canonical GPU capacity RED, delegated CodeQL failure, missing authoritative Rust coverage evidence, Noema owner failure, and pre-scan Strix failure keep that bar open. + +## Rejected TEPP-local repairs + +The following are not causal acceptance: + +- adding another swallowed-term or pair-order special case to `scaled_compensated_mean`; +- replacing it with plain Kahan, Neumaier, sorting/coalescing, or pre-scaling without a final-rounding proof; +- computing a rounded floating sum first and dividing by the sample count afterward; +- adding a product-specific sample ceiling to simplify the numerical proof; +- copying fast-mlsirm #1536 private partials or #1816 source into TEPP; +- pinning a mutable owner PR/head; +- copying or modifying #1717's GPU/CI acquisition logic in TEPP or #1816; +- weakening the 18-buffer GPU capability contract, accepting CPU fallback as GPU parity, or accepting a skipped GPU test; +- describing faithful or approximate behavior as correctly rounded; +- treating native CodeQL/security success as a substitute for failed repository/GPU evidence, delegated CodeQL receipt, authoritative Rust coverage, independent review, semantic-review evidence, or immutable release. + +## Release and consumer acceptance + +Before TEPP changes production arithmetic, the fast-mlsirm owner chain must close in order: #1717 or a verified successor must establish normal GPU/CI acceptance; `fast-mlsirm#1816` must then reacquire its exact-current repository evidence, land through the normal protected path, and publish the numerical contract in a new immutable versioned release with version/tag/package, SBOM/provenance, reproducibility and rollback evidence. ADR-0029 remains Proposed until that acceptance is complete. + +After release, TEPP must pin the released contract through the approved dependency/ACL boundary, remove the local generic mean heuristic rather than retaining two numerical authorities, rerun the public longitudinal RED to GREEN, and reacquire exact-head formatting, Clippy, no-retry tests, rustdoc, 100% owned line/branch/edge coverage, dependency/security policy, SBOM/provenance, live PostgreSQL, OpenCode/Noema, independent review, protected merge, and TEPP release evidence. LLM review cannot substitute for numerical or scientific acceptance. + +## Research and standards trace + +International Organization for Standardization. (2020). *Information technology—Microprocessor systems—Floating-point arithmetic* (ISO/IEC 60559:2020). https://www.iso.org/standard/80985.html + +Institute of Electrical and Electronics Engineers. (2019). *IEEE standard for floating-point arithmetic* (IEEE Std 754-2019). https://standards.ieee.org/ieee/315/6210/ + +Ogita, T., Rump, S. M., & Oishi, S. (2005). Accurate sum and dot product. *SIAM Journal on Scientific Computing, 26*(6), 1955–1988. https://doi.org/10.1137/030601818 + +Rump, S. M., Ogita, T., & Oishi, S. (2008a). Accurate floating-point summation part I: Faithful rounding. *SIAM Journal on Scientific Computing, 31*(1), 189–224. https://doi.org/10.1137/050645671 + +Rump, S. M., Ogita, T., & Oishi, S. (2008b). Accurate floating-point summation part II: Sign, K-fold faithful and rounding to nearest. *SIAM Journal on Scientific Computing, 31*(2), 1269–1302. https://doi.org/10.1137/07068816X + +These sources motivate the numerical design and final-rounding requirements. They do not prove the concrete fast-mlsirm implementation or authorize TEPP production activation by themselves. diff --git a/docs/research/longitudinal-within-between.md b/docs/research/longitudinal-within-between.md index 48c565f91..583c02ca3 100644 --- a/docs/research/longitudinal-within-between.md +++ b/docs/research/longitudinal-within-between.md @@ -8,7 +8,84 @@ as within-unit change. Recovered components are scored with computed RMSE against known truth. This slice does not fit DSEM, claim lagged or causal paths, or treat -irregular intervals as equally spaced. +irregular intervals as equally spaced. CWC-then-irregular residual +log-rate lives in the same crate; see +`docs/research/cwc-irregular-residual-log-rate.md`. + +## Numerical authority + +Unit means in `decompose_within_between` use the same Longitudinal-local +`scaled_compensated_mean` primitive as CWC and occasion-mean composition. +The decomposition path must not maintain a shadow running-mean algorithm. + +This matters at binary64 boundaries. With minimum positive subnormal ULP +`u`, the unit scores `[u, 2u]` have exact mean `1.5u`, which IEEE 754 +round-to-nearest, ties-to-even represents as `2u`. The predecessor running +update `mean += (value - mean) / count` rounded the half-ULP update to zero +and returned `u`; the derived within residuals were therefore shifted from +`[-u, 0]` to `[0, u]`. The public decomposition contract now pins the +single-rounding result. + +Traceability: + +- RED `7dc87aa8ad4de4a73a502646c5667d01656e9dbd` — + `crates/longitudinal_core/tests/decomposition_mean_rounding_contract.rs`. +- Causal repair `97c8ad35d1a1a483d8feb2d617e5fbc035c5ead9` — + `crates/longitudinal_core/src/decompose.rs` delegates unit means to the + existing Longitudinal-local compensated mean authority. +- Public API under test — `decompose_within_between` with + `OccasionObservation`; between/within component identity remains unchanged. + +Exact zero within residuals also have one public identity. IEEE 754 binary64 +has distinct `+0.0` and `-0.0` encodings, and subtraction can therefore leave +`-0.0` when a negative-zero observed score equals a canonical zero unit mean. +That sign bit does not represent positive versus negative within-person change: +the deviation is exactly zero. The public decomposition boundary therefore +canonicalizes only validated exact-zero residuals to `+0.0`; private numerical +intermediates remain free to retain signed zero where it is diagnostically +meaningful. IEEE Std 754-2019 is the cited floating-point standard for this +signed-zero contract; no time-dependent IEEE P754 project-status claim is used +as scientific evidence. + +Signed-zero traceability: + +- RED `aeb008a38bc333ba0f1bec0651739e361426e66d` — + `crates/longitudinal_core/tests/decomposition_signed_zero_contract.rs` + exercises the public decomposition API with `-0.0` and `+0.0` observations. +- Causal repair `a9a70baa5c2a40ec8bf3fc77748bc3a5eaf92cd8` — + `crates/longitudinal_core/src/decompose.rs` canonicalizes an exact-zero + within residual only after finite-result validation. +- Standard authority — IEEE. (2019). *IEEE standard for floating-point + arithmetic* (IEEE Std 754-2019). IEEE. The canonical repository register is + `docs/research/standards-and-literature.md`. + +Known-truth recovery metrics have the same fail-closed representability +boundary. `component_root_mean_square_error` may return exact zero only when +all admitted matched component residuals are exactly zero. If at least one +residual is nonzero but the positive real-valued RMSE falls below binary64 +range, returning `0.0` would convert a numerical limitation into false perfect +recovery. For five matched components with one minimum-subnormal residual `u` +and four exact-zero residuals, the mathematical RMSE is `u / sqrt(5) > 0` +but binary64 rounds that final product to zero. That case must therefore be +reported as `InvalidComponentPayload`, consistent with the public contract +that a non-representable final RMSE fails closed. + +Recovery-metric traceability: + +- RED `496583c6b62cbe0ad1be0e65b51f01d7f72acd5a` — + `crates/longitudinal_core/tests/component_rmse_underflow_contract.rs` drives + the public `component_root_mean_square_error` API with identity-matched + within components and one minimum-subnormal nonzero recovery error. +- Causal repair `a82b383b5940126a0139180d66729d2e6aa4baf7` — + `crates/longitudinal_core/src/component.rs` keeps exact-zero recovery on the + existing `scale == 0` path and rejects a later rounded `rmse == 0` once a + nonzero residual scale has already been established. +- Acceptance boundary — an unrepresentable nonzero error is not a recovered + parameter and cannot count toward perfect RMSE or scientific claim promotion. + +This consolidation is local to Longitudinal Modeling. It does not create a +second reusable psychometric arithmetic owner and does not move static +psychometric truth out of fast-mlsirm. ## Authority @@ -30,4 +107,4 @@ https://doi.org/10.1037/a0038889 Asparouhov, T., Hamaker, E. L., & Muthén, B. (2018). Dynamic structural equation models. *Structural Equation Modeling: A Multidisciplinary -Journal, 25*(3), 359–388. https://doi.org/10.1080/10705511.2017.1406803 +Journal, 25*(3), 359–388. https://doi.org/10.1080/10705511.2017.1406803 \ No newline at end of file diff --git a/docs/research/multilevel-event-time-recovery.md b/docs/research/multilevel-event-time-recovery.md index 3701dcb4b..e0dc7f9f9 100644 --- a/docs/research/multilevel-event-time-recovery.md +++ b/docs/research/multilevel-event-time-recovery.md @@ -180,6 +180,7 @@ The Voelkle et al. (2012) ZORA accepted manuscript was re-opened 2026-08-18T21:0 - **Lagged predetermined observed covariance.** Driver et al. (2017, Eq. 5, p. 5; Eq. 3–4, pp. 4–5; Table 2, p. 12; JSS PDF re-opened 2026-08-23T09:04Z): independent `ε_t` does not enter `cov(y_t,y_{t-1})`. The scalar composition is `λ²(trait + e^{aΔt} p_0 + (B/a)²v) + ψ`. Form the predetermined lagged latent covariance first, then `λ²c+ψ`. A zero loading is exactly `ψ`. A zero trait, a zero initial variance, and a zero TI contribution is exactly `ψ`. Setting `p_0=−q/(2a)` recovers the stationary lagged observed covariance. Stationary lagged observed covariance is not this composition when `p_0` is free. `MANIFESTVAR` is not this composition. The predetermined lagged latent covariance is not this observed covariance. Predetermined later observed variance includes `Q_Δt` and `θ` and is not this composition. An overflowing product or sum fails closed. This is not a Kalman filter and not ctsem estimation. - **Level-change discrete increment.** Driver et al. (2017, §7.2, pp. 20–21; Eq. 3, pp. 4–5; Table 2, p. 12; JSS PDF re-opened 2026-08-20T19:50Z): Equation 3 maps `CINT` through `A^{-1}[e^{AΔt}−I]κ`. With `κ=−a m x` the scalar increment is `(e^{aΔt}−1)/a·(−a m x)=(1−e^{aΔt})m x`. Form the level-change `CINT` first, then the discrete intercept map. Underflow of `e^{aΔt}` to `+0` keeps `m x`. A zero effect or zero predictor is exactly zero. `(1−e^{aΔt})m x` is not `m x`, not `κ`, and not `A^{-1}[e^{AΔt}−I]Bz`. An overflowing product or increment fails closed. This is not a Kalman filter and not ctsem estimation. - **CWC-then-lag.** Sample cluster means are removed first. Consecutive within residuals are then fitted by least squares to \(r_{t+\Delta t}\approx\exp(a\Delta t)\,r_{t}\) on event time. Same-sign pair-wise logs initialize the scalar Newton step. Sign-flipping \(T=2\) CWC pairs have no real logarithm and fail closed. Curran and Bauer (2011, pp. 607–608) show that this person-mean subtraction on a raw autoregressive series does **not** isolate the lagged within-person effect; the helper therefore does not claim to recover the raw-process drift. +- **CWC-then-irregular pairwise mean.** Unique Draft #327 evidence folded into `longitudinal_core` on landing vehicle #310. Consecutive unit-mean-centered pairs keep typed `EventTimeInterval`. The pairwise mean of the Driver et al. (2017, Eq. 3; JSS PDF opened 2026-09-02, p. 4) scalar inverse \(a=\ln(|r_{t+\Delta t}|/|r_t|)/\Delta t\) is taken on nonzero same-sign residuals. This is not the Newton least-squares fit. For an already-centered pair `(1, 0.5)`, the recovered rate is \(\ln(0.5)/\Delta t\), which equals \(\ln(0.5)\) only when \(\Delta t=1\). See `docs/research/cwc-irregular-residual-log-rate.md`. - **Already-centered irregular residual.** The caller supplies lagged within residuals. The mean of \(a=\ln(r_{t+\Delta t}/r_t)/\Delta t\) is the exact scalar map. Intervals may be irregular. The helper does not center again. This is not DSEM. - **Standardised initial latent mean.** Driver et al. (2017, Table 2, p. 12; p. 16; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-26T04:09Z): Table 2 names `T0MEANS` the latent process means at the first time point `T0`. Footnote 4 standardises using only the relevant variance, not the total. The first-occasion relevant variance is free `T0VAR` `p_0`, not `asymDIFFUSION`. The 2017-era source forms unstandardised `T0MEANS` and does not form `T0MEANSstd`. The scalar map is `μ_0/√p_0`. Form strictly positive `p_0` first, then divide. A zero mean is exactly zero. Zero `p_0` has no positive SD and fails closed. `T0` is an event-time occasion. Free `T0MEANS` does not require `a<0`. `T0VARstd` is not this map even when both equal 1. `μ_0/√asymDIFFUSION` is not this map. An overflowing quotient fails closed. This is not a Kalman filter and not ctsem estimation. - **Standardised initial latent variance.** Driver et al. (2017, Table 2, p. 12; p. 16; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-26T07:17Z): Table 2 names `T0VAR` the latent process initial variance/covariance. Footnote 4 standardises using only the relevant variance, not the total. The first-occasion relevant variance is free `T0VAR` `p_0`, not `asymDIFFUSION`. The 2017-era source forms `T0VARstd` as `solve(sqrt(diag(T0VAR))) %&% T0VAR`. OpenMx `%&%` is `t(A) %*% B %*% A`. The default ridge is 0. The scalar correlation is `p_0/p_0=1`. Form strictly positive `p_0` first, then `1/√p_0`, then `(1/√p_0) p_0 (1/√p_0)`. Zero `p_0` has no positive SD and fails closed. `T0` is an event-time occasion. Free `T0VAR` does not require `a<0`. Distinct positive `p_0` recover the same 1. `T0MEANSstd` is not this map even when both equal 1. `asymDIFFUSIONstd` is not this map even when both equal 1. An overflowing quadratic form fails closed. This is not a Kalman filter and not ctsem estimation. diff --git a/docs/research/occasion-mean-event-time-composition.md b/docs/research/occasion-mean-event-time-composition.md new file mode 100644 index 000000000..7872c6344 --- /dev/null +++ b/docs/research/occasion-mean-event-time-composition.md @@ -0,0 +1,53 @@ +# Occasion-mean event-time composition + +Status: active PR evidence on #310; not protected-main or release evidence. + +## Scientific claim + +Hamaker, Kuiper, and Grasman (2015, Eq. 1a) write an observed score as an occasion-specific mean plus a deviation, `x_it = μ_t + p_it`. In TEPP, `μ_t` is the group mean for one numeric event-time occasion and `p_it = x_it - μ_t`. This is distinct from person/unit-mean centering (CWC), a sample-wide grand mean, and person-specific detrending. + +The resulting `p_it` deviations are not, by themselves, within-person effects. Stable between-person differences can remain in them; Hamaker et al.'s random-intercept extension removes a person-specific stable component before interpreting within-person dynamics. TEPP therefore refuses promoting the occasion-mean residual log-rate to a within-person lag estimand. + +Aligned occasions may be irregularly spaced. Each consecutive unit-specific pair retains its own positive finite `EventTimeInterval`; unequal event intervals are not pooled as one discrete coefficient. + +## Admission and numerical invariants + +`longitudinal_core::center_occasion_mean_event_lags` owns the temporal composition boundary. + +- Occasion identity is numeric event time. IEEE-754 `-0.0` and `+0.0` are one occasion, not two bit-pattern identities. +- Exact zero Hamaker Eq. 1a residuals use one public encoding: canonical `+0.0`. A `-0.0` observed score equal to a canonical `+0.0` occasion mean must not leak a signed-zero public residual. IEEE-754 signed zeros are numerically equal but bit-distinct; exact zero occasion-mean deviation has no positive/negative measurement direction. Private numerical intermediates and caller-constructed already-centered pairs may still retain signed zero. This is the same public-identity contract as CWC and `decompose_within_between`; it is not a license to treat occasion-mean residuals as within-person or RI-CLPM lags. +- A unit may contribute at most one observation to one numeric occasion. +- Each admitted occasion contains at least two distinct units, and at least two units contribute consecutive lags. This prevents an atomistic one-unit series from manufacturing multilevel evidence. +- Scores and event times must be finite. Consecutive event intervals must be finite and strictly positive. +- A representable occasion mean is not rejected merely because a same-sign intermediate partial sum would overflow binary64. Mixed signs cancel before bounded same-sign averaging; same-sign values are normalized by a finite maximum magnitude, accumulated in deterministic total order with compensation, divided by count, and then rescaled. +- A representable subnormal occasion mean must retain IEEE-754 round-to-nearest, ties-to-even behavior. In particular, the mean of one and two minimum-subnormal ULPs is 1.5 ULPs and rounds to the even two-ULP representation rather than losing the half-ULP update in an incremental recurrence. +- Occasion means are bit-stable under permutation of the same admitted rows. Input arrival order is not scientific evidence. +- `recover_occasion_mean_centered_irregular_residual_log_rate` composes the centered pairs with the existing Longitudinal Modeling exact-log-rate boundary. It does not create a second static psychometric arithmetic authority. + +## Test trace + +Current #310 branch tests: + +- `crates/longitudinal_core/tests/occasion_mean_event_time_contract.rs::signed_zero_is_one_numeric_occasion` — `-0.0`/`+0.0` share one occasion and the known scalar event-time log-rate is recovered. +- `crates/longitudinal_core/tests/occasion_mean_signed_zero_contract.rs` — a `-0.0` observed score equal to a canonical `+0.0` occasion mean cannot keep a signed-zero public residual; nonzero mixed-sign residuals stay bit-identical. +- `occasion_mean_residual_rate_is_not_cwc_rate_on_the_same_panel` — the same time-varying-group-mean panel recovers the known Hamaker Eq. 1a occasion-residual rate while person-mean CWC does not recover that rate. This preserves the valid scientific distinction from Draft #486 without retaining its wrong `psychometric_core` ownership. +- `signed_zero_duplicate_unit_is_rejected_as_one_occasion` — the same unit cannot use signed zero to bypass duplicate occasion admission. +- `representable_occasion_mean_is_not_rejected_for_intermediate_sum_overflow` — `[0.75·MAX, 0.75·MAX, -0.5·MAX]` at one occasion retains a finite representable mean and finite centered residuals rather than failing on the naive partial sum. +- `representable_subnormal_occasion_mean_preserves_round_to_even` — `[1 ULP, 2 ULP]` at one occasion requires the 1.5-ULP mathematical mean to round ties-to-even to 2 ULP; the first centered residual is therefore exactly `-1 ULP` rather than zero. +- `occasion_mean_is_bit_stable_under_row_permutation` — the same three-unit occasion with `{1, nextafter(1,+∞), MAX/2}` yields bit-identical centered pairs when row arrival order changes. +- `sparse_unaligned_and_nonfinite_occasion_inputs_fail_closed` — empty/singleton payloads, unaligned one-unit occasions, and non-finite event-time input fail closed at the Longitudinal boundary. +- `singleton_wave_unit_does_not_manufacture_or_block_lag_evidence` — a one-wave unit may contribute to an admitted occasion mean but cannot count toward the two-unit lag-evidence floor; two genuine lag-contributing units still recover the known rate. + +The initial occasion-composition RED is `75b0184d2f6341ef23cf14fc84398c68d8d95d22`. Deterministic-order RED `8a59019ed3112a3e27dd0dcd1b6b86d8d45e5435` exposes order-dependent same-sign averaging; causal repair `465d139dce6101c4958c8b0827b6ef5d674b54c2` orders same-sign values before averaging. Successor-evidence commit `b9e952bb8a893f62aaead59cdf825b5e3c6251c6` ports #486's valid same-panel occasion-vs-CWC scientific claim boundary into the owner-correct Longitudinal test surface. Admission-evidence commit `aad56b502bbdfab08ba896b7d3560884c87fc589` preserves #486's sparse/unaligned/non-finite and singleton-wave cases without retaining its wrong crate ownership. Subnormal-rounding RED `9aff817f9e0f82b9cdb2077f3f62bb3e6a987103` exposes the incremental same-sign mean's loss of a half-ULP update at the binary64 floor; causal repair `40e057b83980a0cc501ad936c02e2d59f90a6fe9` replaces that recurrence with deterministic normalized compensated averaging. Signed-zero residual RED `crates/longitudinal_core/tests/occasion_mean_signed_zero_contract.rs` drives `center_occasion_mean_event_lags` with a T=3 mixed-sign panel whose middle score is `-0.0`; causal repair `238c4def7a5a5f19963d2e5cc936dd750d55c5a1` canonicalizes only a validated exact-zero occasion-mean residual to public `+0.0`. IEEE Std 754-2019 is the cited floating-point standard used for the signed-zero and ties-to-even contract; no time-dependent IEEE P754 project-status claim is used as scientific evidence. The owner-correct implementation remains in `crates/longitudinal_core/src/occasion_mean.rs`. Only the current exact branch head may be used for merge evidence. + +## Claim boundary + +This slice is an occasion-mean residual event-time association. It is not RI-CLPM, DSEM, a causal effect, or proof of within-person dynamics. LLM output does not estimate or activate this quantity. Reusable static psychometric arithmetic remains fast-mlsirm-owned and may be consumed only through an immutable released contract. + +## Primary source + +Hamaker, E. L., Kuiper, R. M., & Grasman, R. P. P. P. (2015). A critique of the cross-lagged panel model. *Psychological Methods, 20*(1), 102–116. https://doi.org/10.1037/a0038889 + +IEEE. (2019). *IEEE standard for floating-point arithmetic* (IEEE Std 754-2019). IEEE. + +Repository evidence was cross-checked on 2026-09-03 against the Utrecht University/UvA-DARE final-published-version record and PubMed record (PMID 25822208). The repository record identifies the DOI, journal, volume, issue, pages, and peer-reviewed publication. No timezone-suffixed access timestamp is used as scientific evidence. diff --git a/docs/research/standardised-discrete-drift.md b/docs/research/standardised-discrete-drift.md new file mode 100644 index 000000000..d4f7af00d --- /dev/null +++ b/docs/research/standardised-discrete-drift.md @@ -0,0 +1,37 @@ +# Scalar event-time `discreteDRIFTstd` recovery + +## Domain owner + +This contract belongs to TEPP's **Longitudinal Modeling** bounded context because it composes a continuous-time drift over a substantive event-time interval. It is not a reusable static psychometric response/dependence kernel and does not establish `psychometric_core` as an owner. Reusable generalized-mixed, LSIRM/MLSIRM/DLSJM numerical kernels remain fast-mlsirm-owned. + +## Estimand + +For scalar continuous-time drift `a` and positive event interval `Δt`, Driver, Oud, and Voelkle's discrete-time transition is + +`φ(Δt) = exp(a Δt)`. + +The 2017 ctsem standardisation uses the relevant within-person asymptotic variance. For a scalar stationary process with continuous diffusion intensity `q > 0` and stable `a < 0`, + +`p = -q / (2a) > 0`. + +The affecting/affected standard-deviation ratio is therefore `sqrt(p) / sqrt(p) = 1`, so the scalar p. 16 `discreteDRIFTstd` is numerically `exp(a Δt)`. Numerical equality does not collapse the named estimands: unstandardised `discreteDRIFT`, `discreteDRIFTstd`, trait-plus-state lagged association, trait variance, process noise and a fitted ctsem/DSEM model remain different contracts. + +`recover_event_time_standardised_discrete_drift` accepts only an event interval; it does not accept a generic clock enum. This makes the event-time ownership structural at the API boundary rather than relying on a caller-supplied clock label. + +## Identification and admissibility + +This deterministic transform requires finite `q`, finite `a`, finite positive `Δt`, `q > 0`, stable `a < 0`, and a representable positive stationary within-person variance. If `exp(a Δt)` underflows to zero, the binary64 result is refused because zero cannot be inverted through the corresponding real log-rate map. The function does not estimate `a`, `q`, latent states, measurement parameters, or uncertainty. + +No claim is made for matrix standardisation, time-varying drift, nonstationary state variance, ESEM, DSEM, or ctsem estimation. Those require their own equations, identification, estimators and recovery evidence. + +## Recovery evidence + +The integration contract uses deterministic known truth over irregular positive intervals and multiple stable drift/diffusion settings. It computes `truth = exp(a Δt)` independently and requires machine-precision RMSE. Additional cases cover zero/underflowed within-person variance, unstable drift, non-positive/non-finite intervals, negative/non-finite diffusion, non-finite/overflowed intermediates, exponential underflow, and named-estimand refusal. + +This is exact arithmetic recovery rather than a fitted stochastic estimator, so Monte Carlo interval coverage is not manufactured for this function. Monte Carlo RMSE/bias/coverage remains mandatory for estimators that infer drift, diffusion, latent states or uncertainty from sampled data. + +## Traceability + +Driver, C. C., Oud, J. H. L., & Voelkle, M. C. (2017). Continuous time structural equation modeling with R package ctsem. *Journal of Statistical Software, 77*(5), 1–35. https://doi.org/10.18637/jss.v077.i05 + +Relevant evidence: Eq. 3 discrete transition; Table 2 continuous/discrete parameter names; p. 16 standardised output; footnote 4 standardisation rule; §7.1 separation of stable trait and within-person dynamics. The historical #310 branch also records inspection of the 2017-era `summary.ctsemFit.R`; that implementation lineage is preserved in Git history rather than keeping the temporal transform in the wrong bounded context. diff --git a/docs/research/standards-and-literature.md b/docs/research/standards-and-literature.md index 8ce0ecc77..e399eb866 100644 --- a/docs/research/standards-and-literature.md +++ b/docs/research/standards-and-literature.md @@ -47,6 +47,14 @@ Browne, W. J., Goldstein, H., & Rasbash, J. (2001). Multiple membership multiple Jones, K. (1991). Specifying and estimating multi-level models for geographical research. *Transactions of the Institute of British Geographers, 16*(2), 148–160. https://doi.org/10.2307/622612 +Huang, Y., & Leroux, B. (2011). Informative cluster sizes for subcluster-level covariates and weighted generalized estimating equations. *Biometrics, 67*(3), 843–851. https://doi.org/10.1111/j.1541-0420.2010.01542.x + +Kahan, B. C., Li, F., Blette, B., Jairath, V., Copas, A., & Harhay, M. O. (2023). Informative cluster size in cluster-randomised trials: A case study from the TRIGGER trial. *Clinical Trials, 20*(6), 661–669. https://doi.org/10.1177/17407745231186094 + +Wang, M., Kong, M., & Datta, S. (2011). Inference for marginal linear models for clustered longitudinal data with potentially informative cluster sizes. *Statistical Methods in Medical Research, 20*(4), 347–367. https://doi.org/10.1177/0962280209347043 + +For TEPP's irregular-rate estimand contract, these informative-cluster-size sources are methodological analogies rather than treatment-effect authority. They support the narrower requirement that record multiplicity and follow-up intensity can change the weighting population, so pair-weighted, equal-unit, and any future design-weighted target must remain explicitly distinct. See `docs/research/irregular-rate-estimand-weighting.md`; the repository does not import the papers' treatment estimands into TEPP. + TEPP applies these sources to construct definition, score interpretation, reliability, validity evidence, uncertainty, consequences, longitudinal invariance, ESEM cross-loadings, and DSEM. Topic outputs are treated as fallible indicators or components only after their construct role is evaluated. Reflective, formative, and network classes remain distinct (Bollen & Lennox, 1991). Complete-data OLS loadings across posterior indicator draws are combined with Rubin (1996) \(T_m\); the arithmetic-mean helper remains a point estimate. Mislevy (1991, *Psychometrika, 56*, 177–196, DOI 10.1007/bf02294457) remains unread (Unpaywall/OpenAlex/Semantic Scholar 2026-08-18T03:07Z: closed). The 1988 ETS RR-88-45 / DTIC ADA200179 technical report of the same title was opened 2026-08-17T12:04Z from archive.org; it is not the 1991 journal article and is not used as Mislevy plausible-value authority. Temporal precedence is not causal identification (Holland, 1986). Within/between OLS follows Enders and Tofighi (2007), Curran and Bauer (2011), and Hamaker et al. (2015). Enders and Tofighi (2007, Table 2, pp. 124–127; PDF opened 2026-08-17) show that the CWC cluster-mean coefficient is the contextual effect (`between − within`), not the between-cluster effect. Curran and Bauer (2011, pp. 607–608) reject person-mean subtraction on a raw autoregressive series as the lagged within-person residual; already-centered irregular residuals use the Voelkle et al. (2012, Eq. 7) / Driver et al. (2017, Eq. 3) scalar map. Discrete lags from unequal event intervals are remapped through that log-rate (Voelkle et al., 2012, ZORA accepted manuscript re-opened 2026-08-17T13:13Z) and are not pooled. Driver, Oud, and Voelkle (2017, Eq. 3 and p. 4) write \(A_{\Delta t}=\operatorname{expm}(A\Delta t)\) and restate the discrete intercept as a function of \(A\) and \(\Delta t\). A binary64 underflow of \(\exp(a\Delta t)\) to `+0` is refused because discrete auto-effects are strictly positive. The discrete effect of a constant predictor is Voelkle et al. (2012, Eq. 12; ZORA accepted manuscript re-opened 2026-08-17T14:20Z, Introducing Intercepts, manuscript p. 20), evaluated as \(a_{yx}(\operatorname{expm1}(z)/a_{xx})\) with \(z=a_{xx}\Delta t\) so a finite result is not lost when \(z\) overflows to \(-\infty\) or when \(a_{yx}\Delta t\) overflows, and in log space when `expm1(z)` overflows at a finite \(z\); a zero continuous effect is exactly zero; an overflowing \(a_{yx}/a_{xx}\) rewrite term fails closed; the first-order product is the underflow limit of that equation, not the general constant-predictor discrete effect. The discrete effect of a time-varying predictor with matched sampling and constancy intervals is Voelkle et al. (2012, Eq. 14; manuscript p. 21): \(b^{*}_{y.x}(\Delta t)=a_{yx}\Delta t\). That product is not Eq. 12. Unmatched intervals fail closed (Oud & Jansen, 2000, unread). The exact scalar discrete process noise is Driver, Oud, and Voelkle (2017, Eq. 3; JSS PDF re-opened 2026-08-18T14:04Z, p. 4): \(Q_{\Delta t}=0.5 q(\operatorname{expm1}(z)/a)\) with \(z=2(a\Delta t)\) for \(a\neq 0\) and \(q=GG^{\top}\ge 0\); do not form \(2a\) first; \(a=0\) recovers \(q\Delta t\); an overflowing rewrite scale \(0.5 q/a\) fails closed. This is not a Kalman filter. Driver, Oud, and Voelkle (2017, Eq. 3; JSS PDF re-opened 2026-08-18T14:04Z) write the same discrete intercept as \(A^{-1}[e^{A\Delta t}-I]\xi\). The lagged covariance is \(\mathrm{e}^{a\Delta t}p\) and the unconditional variance is \(\mathrm{e}^{2a\Delta t}p+Q_{\Delta t}\) (Driver et al., 2017, Eq. 3–4, pp. 4–5); a zero diffusion whose \(2(a\Delta t)\) overflows to \(+\infty\) fails closed. The stationary within-subject variance is the \(\Delta t\to\infty\) limit of Eq. 4: \(-q/(2a)\) for stable \(a<0\) (JSS p. 16 `asymDIFFUSION`; §4.3; PDF re-opened 2026-08-18T18:03Z). Finite-interval \(Q_{\Delta t}\) is not that limit. Trait-plus-state variance is \(\mathrm{trait}+\mathrm{state}\) and lagged covariance is \(\mathrm{trait}+\mathrm{e}^{a\Delta t}p\) (Driver et al., 2017, §4.3, p. 9; JSS PDF re-opened 2026-08-18T21:07Z). Trait variance is not process noise and not `asymDIFFUSION`. The first-occasion map `τ + λ μ_0` is not `E(y_t)`. The contemporaneous `TDPREDEFFECT` impulse is `m x` (Driver et al., 2017, Eq. 3 fourth summand; Table 2; §7.2; JSS PDF re-opened 2026-08-20T07:10Z). `TDPREDEFFECT` is not `CINT`. `M x` is not `A^{-1}[e^{A Δt} − I] B z` and is not Voelkle et al. (2012, Eq. 14). Metric/weak invariance does not license latent-mean comparison. Putnick and Bornstein (2016, PMC author manuscript PMC5145197 opened 2026-08-19T22:15Z) require scalar invariance before latent-mean comparison and state that residual invariance is not a prerequisite. Two-observation OLS residual variance is identically `0` and is not strict. Meredith (1993) remains unread (Unpaywall/OpenAlex 2026-08-25T11:32Z: `is_oa: false`; Springer `content/pdf` is a 3038-byte HTML stub; Cambridge Core DOI 10.1007/BF02294825 remains a closed product page). Vandenberg and Lance (2000) remains unread. Mislevy (1991, *Psychometrika, 56*, 177–196, DOI 10.1007/bf02294457) remains unread (Unpaywall/OpenAlex 2026-08-25T11:32Z: `is_oa: false`; Springer `content/pdf` is a 3038-byte HTML stub). ERIC ED334221 is Singer and Willett (1991), not the 1991 journal article. ERIC ED333032 is Mislevy, Sheehan, and Wingersky (1990), ETS RR-90-17-ONR, not the 1991 journal article. Oud and Jansen (2000) remains unread (Unpaywall/OpenAlex 2026-08-18T21:07Z: closed). For Meredith (1993), the Cambridge Core original-paper page and abstract were opened on 2026-08-21; Unpaywall, Springer `content/pdf`, OpenAlex, Semantic Scholar, and Cambridge Core PDF lookup were re-tried 2026-08-25T11:32Z and remain closed. The earlier `remains unread` note means that the full text was not available, not that the authoritative record was unverified. @@ -447,3 +455,11 @@ Nielsen, S., Cetin, E., Schwendeman, P., Sun, Q., Xu, J., & Tang, Y. (2026). Lea Tang, Y., Cetin, E., Xu, J., Sun, Q., Nielsen, S., Richard, V., Goda, H., Tymchenko, I., Nguyen, N., Lee, H., Ashiga, M., Kotyan, S., Kuroki, S., & Clanuwat, T. (2026). *Sakana Fugu technical report* [Preprint]. arXiv. https://arxiv.org/abs/2606.21228 TRINITY motivates lightweight learned model/role delegation over multiple turns; Conductor motivates query-adaptive natural-language workflow/topology/instruction generation and recursive test-time scaling; Fugu demonstrates a production-oriented family of query-adaptive agentic scaffolds building on these research lines (Xu et al., 2026; Nielsen et al., 2026; Tang et al., 2026). TEPP therefore treats direct routing, verification, fixed multi-agent workflows, adaptive orchestration, stage count, decomposition, recursion, access lists, role-specific reasoning effort, and total test-time budget as explicit experimental variables. `tepp_api::route_orchestration` is the deterministic selector for those variables, while `orchestrator_live` records the selected mode on `POST /v1/interpretation-runs` and refuses scientific-authority promotion. Deeper/more-agent orchestration is never assumed better by default. Comparable-budget ablation, evidence support, calibration, disagreement, safety, cost, and failure behavior are required before a production claim. See `docs/research/adaptive-orchestration-router.md` and `docs/research/orchestrator-live-http.md`. + +## Correlation and covariance-bound foundations + +Bouniakowsky, V. (1859). Sur quelques inégalités concernant les intégrales ordinaires et les intégrales aux différences finies. *Mémoires de l’Académie Impériale des Sciences de Saint-Pétersbourg, VIIe Série, 1*(9), 1–18. + +Pearson, K. (1895). Notes on regression and inheritance in the case of two parents. *Proceedings of the Royal Society of London, 58*, 240–242. + +Rodgers, J. L., & Nicewander, W. A. (1988). Thirteen ways to look at the correlation coefficient. *The American Statistician, 42*(1), 59–66. https://doi.org/10.1080/00031305.1988.10475524 diff --git a/docs/research/trait-state-lagged-association.md b/docs/research/trait-state-lagged-association.md new file mode 100644 index 000000000..d9d8650ca --- /dev/null +++ b/docs/research/trait-state-lagged-association.md @@ -0,0 +1,73 @@ +# Trait-plus-state lagged association boundary + +## Status + +Research boundary note for the Longitudinal Modeling bounded context. This note does not activate a production scoring or estimator target. + +## Scientific correction + +A Pearson correlation standardizes covariance by both marginal standard deviations; this normalization is the defining feature of the coefficient and is not replaceable by a one-sided variance ratio (Pearson, 1895; Rodgers & Nicewander, 1988). For event times `t` and `t + Δ`: + +\[ +\rho_{t,t+\Delta}= +\frac{\operatorname{Cov}(Y_t,Y_{t+\Delta})} +{\sqrt{\operatorname{Var}(Y_t)\operatorname{Var}(Y_{t+\Delta})}}. +\] + +The Cauchy–Bunyakovsky–Schwarz inequality gives the corresponding covariance bound. Bouniakowsky (1859, pp. 3–4) states the integral form that bounds a squared cross-product integral by the product of the two squared-function integrals; applying that inner-product inequality to centered random variables yields `Cov² <= Var_t Var_{t+Δ}`. TEPP therefore treats a supplied covariance outside the exact binary64 representation of that bound as incompatible input rather than rounding it into range. Rodgers and Nicewander (1988) remain supplementary statistical context, not the primary source for the inequality. + +Continuous-time state models need not have equal occasion-specific marginal variances when the initial distribution is not stationary or when time-varying inputs alter state uncertainty. Driver, Oud, and Voelkle (2017, §§4.3, 7.1–7.2) explicitly distinguish initial variance, asymptotic diffusion, trait variance, predictor-related variance, and time evolution. Consequently, a one-sided ratio such as + +\[ +\frac{\text{stable variance}+\text{lagged state covariance}} +{\text{stable variance}+\text{earlier state variance}} +\] + +can exceed one and must not be published as an autocorrelation. Driver et al. (2017) provide the continuous-time state-transition and covariance ingredients used by ctsem, but they do not print or validate that one-sided ratio as an autocorrelation. + +The unmerged implementation that introduced a public `expected_autocorrelation` API is therefore retained only as RED/scientific-failure lineage. Rejecting only positive drift would not repair the definition: a nonstationary initial state can still require a distinct later marginal variance even under negative drift (Driver et al., 2017, §§4.3, 7.1). + +## TEPP contract + +`longitudinal_core::recover_event_time_lagged_correlation` accepts a lagged covariance, the earlier marginal variance, the later marginal variance, and an admitted `EventTimeInterval`. The typed value object is carried through the public boundary into the internal association implementation rather than being erased back to a bare duration. The operation performs only temporal association standardization and does not infer state variance, process noise, or a psychometric response kernel. + +Admissibility and perfect-correlation endpoint claims both use the exact binary64 covariance relation. The implementation decomposes the covariance and marginal variances into integer significands and powers of two, compares `Cov²` with `Var_t Var_{t+Δ}` without a rounded square-root product, and records whether the relation is strict or exact. Rounded `sqrt`/division arithmetic is then used only to form an interior representable coefficient. Exact binary64 equality at the covariance boundary is itself authoritative for the `±1` endpoint, so floating-point square-root rounding may neither invent nor weaken perfect association. + +This distinction is necessary in both directions. For the exact binary64 inputs + +- earlier variance `f64::from_bits(4_607_182_418_800_016_408)`, +- later variance `f64::from_bits(4_607_182_418_800_016_427)`, and +- covariance magnitude `f64::from_bits(4_607_182_418_800_016_417)`, + +`Cov²` is strictly smaller than the exact product of the two supplied marginal variances, yet the rounded square roots followed by the two divisions produce `1.0`. Returning that endpoint would convert an interior association into a scientifically stronger perfect-correlation claim. RED `683b28eeeda3ad72ac11f5317c5aea54f34e0692` fixes both covariance signs through the public API in `crates/longitudinal_core/tests/correlation_false_perfect_contract.rs`; causal repair `9eeb373df2cd333fe7543df2197ea0cc0c492780` prevents a rounded strict-interior endpoint from being promoted. + +The inverse rounding failure also occurs on an exact boundary. With represented inputs `Var_t = 2`, `Var_{t+Δ} = 8`, and `|Cov| = 4`, exact binary arithmetic gives `Cov² = Var_t Var_{t+Δ}` and therefore `|ρ| = 1`. The predecessor square-root/division path evaluates the positive coefficient as `0x1.fffffffffffffp-1`, one ULP below one, which would weaken an exactly perfect represented association. RED `c25000901eb429a43817552f8b76cf4aae04e522` adds positive and negative public-API cases; causal repair `d06259ec1e036558d8d2f775c266b2b9db4e42c4` returns the exact sign endpoint directly when the exact integer covariance relation is on the boundary and reserves floating-point standardization for strict-interior coefficients. + +Exact zero covariance has no directional association, so its public scientific representation must not depend on the IEEE-754 sign bit. Before the projection repair, a supplied `-0.0` covariance passed numerical validation and emerged as `-0.0`, creating a bit-distinct public value even though `+0.0` and `-0.0` denote the same zero covariance estimand. RED `e15d053100c17ac46b48cc04ad61440d0185e729` fixes this through `crates/longitudinal_core/tests/correlation_signed_zero_contract.rs`; causal repair `fc61f7bdb2fd8c67719c41a9ce9ef6436beab38a` canonicalizes any validated zero result to `+0.0` at `crates/longitudinal_core/src/temporal_association.rs`. The private association primitive retains numerical validation and endpoint arithmetic; canonical scientific encoding belongs to the public Temporal Association projection boundary. + +This preserves the DDD ownership boundary: + +- reusable static/generalized-mixed/dependence psychometric kernels remain owned by `ContextualWisdomLab/fast-mlsirm`; +- TEPP owns event-time composition and longitudinal semantics; +- measurement occasion, rater, and method facets are not substitutes for substantive event time; +- callers must assemble occasion-specific marginals from an identified temporal model before asking for a correlation. + +The function fails closed when either marginal is non-positive, covariance or marginal inputs are non-finite, the supplied covariance violates the Cauchy–Bunyakovsky–Schwarz covariance bound, a nonzero exact coefficient would collapse to binary64 zero, or a strict interior covariance would round to exact `±1`. `EventTimeInterval` itself fails admission for non-finite or non-positive durations. Pearson standardization does not estimate or transform time; the event-time value object exists to preserve clock ownership through the API. + +## Regression evidence + +The regression suite includes a nonstationary case with earlier variance `1`, later variance `4`, and lagged covariance `1.5`. The retired one-sided ratio would be `1.5`; correct standardization yields `0.75`. It also verifies exact `±1` boundaries at ordinary, unequal-marginal (`2`, `8`, `4`), `f64::MAX`, and minimum-subnormal scales; rejects one-ULP over-bound covariance for both signs; rejects strict-interior covariances whose rounded standardization would otherwise become false exact `±1`; classifies gross subnormal violations before division; canonicalizes both IEEE zero covariance signs to public `+0.0`; rejects invalid event-time value construction; and avoids forming `Var_t * Var_{t+Δ}` directly. + +The scalar `discreteDRIFTstd` regressions separately require monotone temporal ordering for stable negative drift and fail closed when a finite negative drift multiplied by a positive admitted event interval underflows to signed zero. That case must not silently become `exp(-0.0) == 1.0`. + +These are arithmetic and contract regressions, not an estimator-recovery study. A future production autocorrelation derived from a specific DSEM or continuous-time state model must additionally identify and recover the state transition, process-noise contribution, both marginal variances, uncertainty, and leakage-safe rolling-origin performance. Production promotion therefore requires model-specific true-parameter RMSE, bias, interval coverage, irregular-gap behavior, delayed/retrospective-report handling, and temporal-ordering evidence rather than borrowing these standardization tests as scientific-estimator acceptance (Driver et al., 2017). + +## Research basis + +Bouniakowsky, V. (1859). Sur quelques inégalités concernant les intégrales ordinaires et les intégrales aux différences finies. *Mémoires de l’Académie Impériale des Sciences de Saint-Pétersbourg, VIIe Série, 1*(9), 1–18. + +Driver, C. C., Oud, J. H. L., & Voelkle, M. C. (2017). Continuous time structural equation modeling with R package ctsem. *Journal of Statistical Software, 77*(5), 1–35. https://doi.org/10.18637/jss.v077.i05 + +Pearson, K. (1895). Notes on regression and inheritance in the case of two parents. *Proceedings of the Royal Society of London, 58*, 240–242. + +Rodgers, J. L., & Nicewander, W. A. (1988). Thirteen ways to look at the correlation coefficient. *The American Statistician, 42*(1), 59–66. https://doi.org/10.1080/00031305.1988.10475524 diff --git a/scripts/actions_workflow_fleet.py b/scripts/actions_workflow_fleet.py index e5dfb8c97..62b7576d5 100644 --- a/scripts/actions_workflow_fleet.py +++ b/scripts/actions_workflow_fleet.py @@ -9,6 +9,7 @@ from __future__ import annotations import argparse +import contextlib import http.client import json import os @@ -575,7 +576,12 @@ def main( err = stderr if stderr is not None else sys.stderr env = dict(os.environ) if environ is None else environ try: - parsed = parser.parse_args(list(arguments if arguments is not None else sys.argv[1:])) + # Argparse writes usage/errors to sys.stderr; bind that to the caller-provided + # stream so unknown-command and other parse failures are observable in tests. + with contextlib.redirect_stderr(err): + parsed = parser.parse_args( + list(arguments if arguments is not None else sys.argv[1:]) + ) transport = build_transport(env) audit = audit_repository(transport, parsed.owner, parsed.repo) if parsed.command == "audit": diff --git a/tests/quality/test_actions_workflow_fleet.py b/tests/quality/test_actions_workflow_fleet.py index 14c055d46..acef72f77 100644 --- a/tests/quality/test_actions_workflow_fleet.py +++ b/tests/quality/test_actions_workflow_fleet.py @@ -548,8 +548,18 @@ def test_cli_audit_and_disable_and_unknown_command(self) -> None: ) self.assertTrue(applier.call_args.kwargs["apply_changes"]) - with self.assertRaises(SystemExit): - fleet.main(["wat"], environ={"GITHUB_TOKEN": "token"}, stdout=io.StringIO()) + unknown_stderr = io.StringIO() + with self.assertRaises(SystemExit) as unknown_command: + fleet.main( + ["not-a-fleet-command"], + environ={"GITHUB_TOKEN": "token"}, + stdout=io.StringIO(), + stderr=unknown_stderr, + ) + self.assertEqual(unknown_command.exception.code, 2) + self.assertIn("invalid choice", unknown_stderr.getvalue()) + self.assertIn("audit", unknown_stderr.getvalue()) + self.assertIn("disable-orphans", unknown_stderr.getvalue()) def test_cli_reads_process_environment_when_environ_omitted(self) -> None: """Invoking the CLI without an environ argument uses the real process environment.""" diff --git a/tests/quality/test_hourly_nim_product_development.py b/tests/quality/test_hourly_nim_product_development.py index e2a1a03a0..fce4c3f5c 100644 --- a/tests/quality/test_hourly_nim_product_development.py +++ b/tests/quality/test_hourly_nim_product_development.py @@ -37,12 +37,12 @@ class HourlyNimProductDevelopmentContractTests(unittest.TestCase): """Structural tests for the credential-separated product-development loop.""" def test_hourly_workflow_schedule_credentials_and_queue_gate(self) -> None: - """Run at minute 47 with provider discovery and fail closed around inventory.""" + """Keep central-admission entrypoint, provider discovery, and fail-closed inventory gates.""" text = _text(WORKFLOW) bootstrap = _text(BOOTSTRAP) for token in ( - 'cron: "47 * * * *"', + "# cwl-org-commercial-entrypoint: v1", "workflow_dispatch:", "dry_run:", "hourly-nim-product-development-${{ github.repository }}", diff --git a/tests/quality/test_longitudinal_architecture_ownership.py b/tests/quality/test_longitudinal_architecture_ownership.py new file mode 100644 index 000000000..940a5da39 --- /dev/null +++ b/tests/quality/test_longitudinal_architecture_ownership.py @@ -0,0 +1,62 @@ +"""Architecture fitness for Longitudinal Modeling ownership boundaries.""" + +from pathlib import Path + + +ARCHITECTURE = (Path(__file__).parents[2] / "ARCHITECTURE.md").read_text(encoding="utf-8") + + +def _section(start: str, end: str) -> str: + """Return one architecture section bounded by exact headings/markers.""" + return ARCHITECTURE.split(start, 1)[1].split(end, 1)[0] + + +def test_conceptual_boundaries_do_not_assign_temporal_composition_to_psychometric_core() -> None: + """Longitudinal/event-time composition belongs to its own bounded context.""" + bounded = _section("## Bounded services and Rust crates", "Every boundary must") + psychometric_row = next( + line for line in bounded.splitlines() if line.startswith("| `psychometric_core` |") + ) + + for leaked_authority in ("longitudinal invariance", "DSEM", "continuous-time paths"): + assert leaked_authority not in psychometric_row, ( + "psychometric_core still claims Longitudinal Modeling authority: " + f"{leaked_authority}" + ) + + assert "| `longitudinal_modeling` |" in bounded + longitudinal_row = next( + line for line in bounded.splitlines() if line.startswith("| `longitudinal_modeling` |") + ) + for owned_semantic in ("event-time", "longitudinal", "state"): + assert owned_semantic in longitudinal_row + + +def test_implementation_topology_has_one_psychometric_and_one_longitudinal_owner_row() -> None: + """Implementation topology cannot duplicate or conflate responsibility rows.""" + topology = _section("## Implemented foundation topology", "Foundation crates expose") + psychometric_rows = [ + line for line in topology.splitlines() if line.startswith("| `psychometric_core` |") + ] + longitudinal_rows = [ + line for line in topology.splitlines() if line.startswith("| `longitudinal_core` |") + ] + + assert len(psychometric_rows) == 1, "psychometric_core has duplicate responsibility rows" + assert len(longitudinal_rows) == 1, "longitudinal_core has duplicate responsibility rows" + + psychometric_row = psychometric_rows[0] + for temporal_implementation in ( + "event-time log-rate", + "discrete process noise", + "stationary within-subject variance", + "TDPREDEFFECT", + ): + assert temporal_implementation not in psychometric_row, ( + "temporal composition is still assigned to psychometric_core: " + f"{temporal_implementation}" + ) + + longitudinal_row = longitudinal_rows[0] + assert "temporal composition" in longitudinal_row + assert "fast-mlsirm" in longitudinal_row diff --git a/tests/quality/test_longitudinal_scientific_authority_sync.py b/tests/quality/test_longitudinal_scientific_authority_sync.py new file mode 100644 index 000000000..0750990c6 --- /dev/null +++ b/tests/quality/test_longitudinal_scientific_authority_sync.py @@ -0,0 +1,76 @@ +"""Keep Longitudinal Modeling numerical guidance synchronized with production contracts.""" + +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[2] +CLAUDE = ROOT / "CLAUDE.md" +PRD_AMENDMENT = ROOT / "docs/product/prd-v0.4-amendment-longitudinal-time-ownership.md" +TRD = ROOT / "docs/TRD.md" +OWNERSHIP_ADR = ROOT / "docs/adr/longitudinal-modeling-ownership-addendum.md" +LITERATURE_REGISTER = ROOT / "docs/research/standards-and-literature.md" + + +def test_stationary_variance_overflow_guidance_matches_source_contract() -> None: + """Reject the retired ratio-first overflow rewrite and require the current fallback.""" + guidance = CLAUDE.read_text(encoding="utf-8") + + assert "When `2 a` overflows, form `(q / a) * -0.5`." not in guidance + assert "Do not form `0.5 q` first (`q = from_bits(1)` underflows)." not in guidance + assert "When `2 a` overflows, form `(q * 0.5) / |a|`." in guidance + + +def test_prd_names_irregular_rate_estimands_and_weighting_population() -> None: + """Keep the product target explicit when follow-up multiplicity changes pair weights.""" + prd = PRD_AMENDMENT.read_text(encoding="utf-8") + + assert "`tepp.irregular_rate.lag_pair_average.v1`" in prd + assert "`tepp.irregular_rate.unit_average.v1`" in prd + assert "candidate/contributing units" in prd + assert "candidate/admitted/refused pairs" in prd + assert "fail closed" in prd + + +def test_trd_requires_versioned_irregular_rate_weighting_and_denominators() -> None: + """Keep the technical contract aligned with the public estimand API.""" + trd = TRD.read_text(encoding="utf-8") + + assert "`tepp.irregular_rate.lag_pair_average.v1`" in trd + assert "`tepp.irregular_rate.unit_average.v1`" in trd + assert "candidate/contributing units" in trd + assert "candidate/admitted/refused pairs" in trd + assert "attempted/recovered/failed" in trd + + +def test_ownership_adr_keeps_pair_and_unit_estimands_distinct() -> None: + """Prevent the implemented pair target from silently becoming an equal-unit target.""" + adr = OWNERSHIP_ADR.read_text(encoding="utf-8") + + assert "`tepp.irregular_rate.lag_pair_average.v1`" in adr + assert "`tepp.irregular_rate.unit_average.v1`" in adr + assert "occasion count" in adr + assert "membership weight" in adr + assert "fail closed" in adr + + +def test_irregular_rate_weighting_sources_are_doctored_in_canonical_register() -> None: + """Keep claim-specific informative-size sources complete enough for APA traceability.""" + literature = LITERATURE_REGISTER.read_text(encoding="utf-8") + + assert ( + "Huang, Y., & Leroux, B. (2011). Informative cluster sizes for subcluster-level " + "covariates and weighted generalized estimating equations. *Biometrics, 67*(3), " + "843\u2013851." in literature + ) + assert ( + "Kahan, B. C., Li, F., Blette, B., Jairath, V., Copas, A., & Harhay, M. O. " + "(2023). Informative cluster size in cluster-randomised trials: A case study from " + "the TRIGGER trial. *Clinical Trials, 20*(6), 661\u2013669." in literature + ) + assert ( + "Wang, M., Kong, M., & Datta, S. (2011). Inference for marginal linear models for " + "clustered longitudinal data with potentially informative cluster sizes. " + "*Statistical Methods in Medical Research, 20*(4), 347\u2013367." in literature + ) + assert "irregular-rate estimand" in literature + assert "record multiplicity" in literature