Repository navigation
Release 0.25 - #984
Merged
Merged
Release 0.25#984
Conversation
The classification table gains the parametric and semi-parametric regressions, trees and forests, ParametricCompetingRisks, the renewal models and Binomial. The multivariate section lists the copulas as they are (seven families with rotations, standard errors and bounds) and the shared-frailty regressions, which the atlas called future work; the Wiener and gamma degradation processes, also called future work, are described. A Gaps section lists each unbuilt cell with its issue (#804-#811), and says which empty cells are empty by choice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRhKL2fJBiAAfNo9rihwts
…uatqe Model atlas: today's models, and the gaps filed as issues
…ry core The release run took 81 minutes: the calibration studies ran inside the 3.12 parts, with coverage, and one part drew most of the long ones. A study is one long test, so the hash split cannot spread it. - The 3.12 parts run the suite only (with coverage), like 3.11 and 3.13. - A calibration job runs the studies and scenario cards on 3.12 in seven groups that run at once, chosen by the studies' times on the CI runners so that none takes much over a quarter of an hour. Together the groups hold every calibration and scenario test once (277, checked by collection against -m 'calibration or scenarios'). - #617's bootstrap coverage study (54 minutes on one core on a runner) draws its 200 data sets in order from its one stream, then runs the repetitions on every core: each bootstrap has its own seed, so the numbers are those of the loop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRhKL2fJBiAAfNo9rihwts
…uatqe CI: calibration in seven groups by measured time; #617's study on every core
include=[] used to try the default candidates, as if include had been left out. It is now refused, naming the fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRhKL2fJBiAAfNo9rihwts
…uatqe fit_best refuses an empty include (0.24 review)
…0.24 review) The printout of a fit with maximum 'unverified' or 'no finite maximum' read as an ordinary fit; only the warning raised by fit said otherwise, and it is easy to miss or filter. It now has a Maximum line saying which, and what it means. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRhKL2fJBiAAfNo9rihwts
…uatqe A parametric model's printout says when its maximum is not verified (0.24 review)
One refit has no spread, and its BCa bounds came back NaN without a warning. check_n_boot now asks for at least 2 and says 1000 or more is usual; the message keeps its 'must be a positive integer' start. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRhKL2fJBiAAfNo9rihwts
…uatqe Parametric and regression bootstraps refuse n_boot=1 (0.24 review)
- The MLE-failure warning: 'closer.A good way' run together, and 'it's inverse'. - A censoring flag other than -1, 0 or 1 is refused saying what each means and that interval censoring is c=2 with [left, right] pairs. - A non-numeric alpha_ci is refused saying a number is expected. - The degradation process fits document that the estimated y0 can be below zero, and y0=0.0 for paths known to start at zero. - The 0.24 changelog says the 200 K warning is for a temperature column entirely below 200 K, as it behaves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRhKL2fJBiAAfNo9rihwts
…uatqe Messages and docs from the 0.24 review
Past a group's largest observed time, with its survival estimate still above zero, the curve is not estimated; rmst_diff held it at its last value out to tau, so the RMSTs and the difference grew with tau (tau=1e6 gave a difference of 25,000 on data to 120), with no warning. It is now refused, naming the group, its last time and the common follow-up. A curve that has reached zero needs no extrapolation and is not refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRhKL2fJBiAAfNo9rihwts
…uatqe rmst_diff refuses a tau past a group's follow-up (0.24 review)
…0.24 review) AttributeError was the first thing an engineer looking for the Arrhenius model met: it is life_models.Exponential. Arrhenius is the same class, so a model fitted with either is the same model and saves as one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRhKL2fJBiAAfNo9rihwts
…uatqe life_models.Arrhenius, the Exponential life model by its usual name (0.24 review)
….24 review) how='mpp' worked but heuristic='blom' was refused; and Benard's approximation is often spelled 'Bernard'. canonical_heuristic gives the name as PLOTTING_METHODS spells it, at the three places a heuristic enters (a fit's input check, plotting_positions and the probability plot's adjust_heuristic), so every comparison after sees the one spelling. The refusal adds 'Case does not matter.'. The papercut test's example of an unknown heuristic was 'median', now Median's: it is 'medium'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRhKL2fJBiAAfNo9rihwts
…uatqe The plotting heuristic read whatever its case; 'Bernard' is Benard (0.24 review)
The recurrent fitters took tr and tl only as a scalar or one value per row, so CrowAMSAA.fit(x, i=i, tr=[300, 300, 400]) for three units was refused with "'tl' and 'tr' must be the same length as 'x'". handle_xicn, the handler every recurrent fit goes through (NHPP and HPP, the renewal models, proportional intensity, NonParametricCounting, the cause-specific MCF and NHPP, and fit_from_df via fit), now expands a tl or tr with one entry per unit to one per row before format_truncation reads it. With i given, the values are taken in the sorted order of the unit ids (np.unique(i)), the order of RecurrentEventData.items and of an array T in the trend tests, and each row gets its own unit's value wherever the unit's rows are in x. Everything downstream, the stored data and its serialisation included, sees the per-row arrays it sees today. A bound with one entry per row is per row even when that is also the number of units, as before. Any other length is refused naming both accepted lengths. The univariate handler, t, and gapped observation (windows, which refuses tl and tr) are unchanged. The fitters' tl/tr docstrings, handle_xicn's, and the recurrent user guide say so; regression tests in test_truncation.py check per-unit against per-row (string, unsorted and non-contiguous labels, interleaved rows) and the wrong-length message. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRhKL2fJBiAAfNo9rihwts
…uatqe Recurrent tr and tl may be given per unit (0.24 review)
… path (0.24 review)
MixtureModel(m=0) failed inside numpy with "number sections must be
larger than 0"; an m that is not a whole number of at least 1 is now
refused, saying so. from_json reads a string as a file path unless it
starts with '{', so a garbled JSON string reported only a missing file;
the message now says it was read as a path.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TRhKL2fJBiAAfNo9rihwts
…uatqe MixtureModel checks m; from_json says a non-JSON string was read as a path (0.24 review)
…d (0.24 review) success_run(2.5) returned a bound for half a trial; a number of successes that is not a whole number is now refused. cb, param_cb, standard_errors and the other bounds on a model built by weibayes said only "Only MLE has confidence bounds" or that the model carries no covariance; they now say the model is itself the lower bound at the alpha_ci it was built with, and to call weibayes again for another level. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRhKL2fJBiAAfNo9rihwts
…uatqe success_run takes whole counts; Weibayes models say they are the bound (0.24 review)
…nce (#821) qf solved ff(t) = u with brentq one quantile at a time, so above u = 1/2 it kept only the digits of 1 - 1e-16 (sf(qf(1 - s)) off by 1e-4 at s = 1e-12), and took about 0.6 ms a quantile. It now solves every quantile together by Newton steps with the density, kept in the bracket of the components' quantiles by bisection, against s - sf(t) above 1/2 and ff(t) - u below: about 1e-14 at both ends for Weibull, LogNormal, Gamma, Normal, Gumbel, Exponential and LogLogistic mixtures, and 20,000 quantiles in about 0.06 s. Discrete mixtures keep their bisection. Closes #821. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRhKL2fJBiAAfNo9rihwts
…uatqe MixtureModel.qf: full precision in the upper tail, all quantiles at once (#821)
mean(tau), rmst(tau) and mean_cb(tau) held the survival curve at its last value out to tau, so a tau past the data gave an RMST of whatever size tau made it. They now refuse it while the estimate at the largest observed time is above zero, as rmst_diff does; the default tau and a curve that reaches zero are unaffected. rmst_diff shares the check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRhKL2fJBiAAfNo9rihwts
…uatqe Single-group RMST refuses a tau past the data (0.24 review)
The model is a distribution only where the hazard h_0(x) + beta'Z and its integral H(x | Z) are both non-negative, and nothing kept the fit there: the likelihood rises without limit as H falls below 0 at an observation, so fits put sf above 1 at their own data -- on ordinary positive data with a protective covariate (WeibullAH, LogNormalAH and GammaAH on the regression test fixture), and, on a baseline over the whole real line, with sf up to 1e22 at the data and mean(Z) up to 1e39. - The fit keeps every observed point inside the support (H >= 0 at the times, interval bounds and finite truncation times; h >= 0 at the times and bounds), with the NaN barrier it already used for log h. On simulated real-line data with half the times negative it recovers the truth. - A fit that ends against that limit (H = 0 at an observation, where that unit's support starts) is a maximum on the boundary: it warns, its maximum is "unverified", and the search's checks no longer read it as having no finite maximum. A level with no events now has a finite coefficient, held at that limit. - Predictions outside the support (sf, ff, df, hf, Hf, qf, cb, random and the time-varying functions, and sf_tvc given an age outside it) are nan with one warning, where they returned sf above 1 or negative ff. - mean(Z) of a real-line row with beta'Z > 0 is the mean from where its support starts; nan, with a warning, where the model is not a distribution along the way. Bootstrap refits are 0 before their own support starts. - The conformance registry's AH cases fit their own fixture (ah_data: covariates that add hazard), on which each is an ordinary maximum; the boundary has tests of its own. Likelihood-ratio bounds whose region reaches the limit can stop short of its edge: a strict xfail, #837. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRhKL2fJBiAAfNo9rihwts
…uatqe Additive hazards models stay inside their support (#828)
The four test_541_* tests near rho = +-1 failed on arm64 macOS and passed on x86-64 Linux. Their reference densities and h-functions formed 1 - rho^2 and the quadratic forms in np.longdouble, which is 80-bit only on x86-64 Linux; on arm64 and Windows it is double, so there the references, not the copulas, were off by about 1e-9. The references now form those terms in exact rational arithmetic on the doubles (fractions.Fraction), and the copulas meet the same tolerances as before (1e-11 on the log density, rtol 1e-11 on the h-function) on this machine too. No tolerance, point or rho changed. CLAUDE.md no longer lists them as known failures, and says why a reference must not rely on long double. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Km2eWBiPJ5JNkme3vRg4B
Copula #541 tests: references exact on every platform
DynamicPrediction.to_dict(x=None, alpha_ci=0.05) gives one item's prediction as strict JSON for a report or a chart: its age, covariates by name, readings as [age, value] pairs, sf, ff and hf on a grid of ages (by default from the age now to about the 0.99 quantile) with the two-sided Wald bounds on sf, the remaining life's 0.1, 0.5 and 0.9 quantiles, and the marker's path with its interval from the first reading. Infinite and missing values are None with the package's "non_finite" record, as every to_dict writes them. DECISIONS.md: the maintainer kept hazard_covariates and path_covariates. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Km2eWBiPJ5JNkme3vRg4B
Joint models: separate hazard and path covariates, interval data, dynamic prediction, calibration
…plot # Conflicts: # docs/changelog.rst
surpyval.at_risk_table(models, times=None, rows=("At risk",),
counts="cumulative", *, labels=None) returns {"times", "rows", "counts":
{label: {row: array}}}: the counts plot_nonparametric draws under its
plot, from the same code, without drawing or importing a plotting
library. The counting (risk sets from the data, or the ladder of a model
without its data), the model and row checks move to
nonparametric/at_risk.py, which plot_estimates imports. The default times
are the start of the axis and every distinct time in the estimates.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Km2eWBiPJ5JNkme3vRg4B
DemonstrationPlan, LifeTestPlan, SimulatedPrecision, AcceleratedTestPlan, SequentialMTBFTest and SequentialDemonstrationTest gain to_dict(): native Python values (tuples and arrays as lists, numpy scalars as floats and ints), with an infinite or missing number written as None and recorded under "non_finite", through the package's encode_non_finite, so decode_non_finite restores it. The sequential tests also tabulate their boundaries: accept and reject times by failure count (to the truncation failure, or 10), and accepted and rejecting failures by number of units (to the truncation, or twice the units it takes to accept with no failures), each row count overridable. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Km2eWBiPJ5JNkme3vRg4B
The v0.24 deprecations are due in v0.25, and the removal test fails once the version reaches REMOVED_IN with a shim left. Each old form now raises the error Python would, naming its replacement and the release: - CompetingRisksProportionalHazards.beta (an AttributeError: use 'betas' or 'params'), and phi_e given a row index of betas (a ValueError naming the cause's label); - NonParametric.cb's old order cb(x, on, bound, ...) (a TypeError when a bound comes third); - MixtureModel.p, the EM responsibilities (internal to fit); - ProportionalIntensityNHPP.fit(dist=) (use 'baseline'). REMOVED_IN is now "0.26", for the names deprecated in v0.25 (RecurrentEventData; EventHistoryData.items and get_times_to_first_events), which keep their "until v0.26" messages, and REMOVED_IN_NEXT "0.27". Their tests and the changelog's Removed section follow. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Km2eWBiPJ5JNkme3vRg4B
Add plot_nonparametric: estimates with a number-at-risk table
Remove the names deprecated in v0.24; REMOVED_IN is 0.26
…to-dict # Conflicts: # docs/changelog.rst
Test planning results: to_dict(), strict JSON
pyproject.toml and surpyval.__version__ are 0.25. The changelog's Unreleased section becomes v0.25 (17 October 2026), with highlights (joint models with covariates and dynamic prediction, test planning, the intervention test, every bound, non-parametric plotting with an at-risk table) and what to change when upgrading from 0.24. It also restores the page title's underline, cut short by a merge. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Km2eWBiPJ5JNkme3vRg4B
The release PR's docs job builds with -W, and its lint job installs the latest isort; both would have failed on dev: - The test planning reference page documented every function and class twice (its automodule took autodoc's default members, then each autofunction again); it now names the module with py:module. - A section of Parametric SurPyval Modelling was underlined with the wrong character for its level (a CRITICAL error that stops the build). - The changelog's nested lists lacked the blank lines around them. - The deprecated notes of the data-format converters and check_ph's Returns had literal blocks that numpydoc-style parsing broke into definition lists; they are prose now. - Two test modules' imports were sorted differently by the pinned pre-commit isort (8.0.1) and the latest (9.0.2); both now agree. The hooks also dropped needless quotes from data_formats' annotations (pyupgrade). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Km2eWBiPJ5JNkme3vRg4B
Version 0.25, its changelog, and the release checks
The release PR's first full run failed on things no local run on arm64 macOS showed: - Regression qf: Newton's finish rejected every step when the bracketed answer sat at an end of its bracket and the cumulative hazard's sign there flipped between batches (numpy 2.5.4's vectorised exp/log on Linux round the same time differently in a batch of another size), so LogNormalPH's qf(0.9) kept the solver's 1.1e-12 error. The bracket is widened by 1e-10 of t; a step must still reduce |gap|. Reproduced and fixed in Linux containers (arm64 and amd64). - test_828 bounded the error of ff(qf(s)) near an additive model's support by 1e-8 of s, about two ulps of the terms H0 + x beta'Z that cancel there; x86-64 rounds them to 1.1e-8. The bound is now 16 ulps of |H0(q)|, which is the limit any qf or ff has there (about 5 at most on either platform). - NormalAH's numerical standard errors (the reference) are 1.05e-4 from the exact ones on x86-64, 6.2e-5 on arm64; it joins OLD_ROUNDING, whose check at ten times the step agrees to 6.7e-7. - Doctests run one after another in one process in CI; three that draw on plt.gca() drew on an earlier example's axes. Every doctest now starts with no figure open (root conftest). - The refit calibration registry neither planned nor excluded WeibullJointPH; it is excluded, its own study being calibration/test_coverage_joint.py. The hooks also moved _prediction's Callable to collections.abc. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Km2eWBiPJ5JNkme3vRg4B
Release 0.25 CI: platform-independent qf, doctests and checks
with_params (and from_params) divided the weights by their sum, so weights that summed to 1 within an ulp each moved by an ulp: on 3.11 x86-64 (numpy 2.4.6) the fitted weights did, and test_with_params_keeps_dist_and_m_and_reads_the_covariance_order failed on the release run. Weights within 8 m ulps of summing to 1 are kept as given; others (up to 1e-9 off, as before) are still renormalised. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Km2eWBiPJ5JNkme3vRg4B
MixtureModel.with_params keeps the weights bit for bit
The release run's calibration jobs ran for hours: - conformance-sweeps: the ExpoWeibull's mrl_cb likelihood-ratio sweep. Its mean residual life is an integral at every point the search tries, in a three-parameter valley; one two-sided bound at one time took 414 s on four cores, and the sweep asks for dozens. It is left out, as the offset, LFP and ZI variants' already were for the same reason; mrl_cb[lr] is still swept on 36 cases, the Weibull's taking the same path in closed form. The rest of the sweep: 242 tests, 10.4 minutes on four workers locally. - extra-parameters: its four studies took 52 minutes in one group (offset 1284 s, on the edge 903, lfp 651, zi 321). They run in three groups now, offset, on the edge, and lfp with zi, none of them cut. Every calibration group gets timeout-minutes: 45, so a hang fails in three quarters of an hour rather than GitHub's six. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Km2eWBiPJ5JNkme3vRg4B
Calibration CI: groups that fit, and a timeout for each
A hidden cell in Regression Modelling asserted that each partial-effects curve equals pump_model.sf(x, row) exactly. The plot evaluates every row in one batch and the check one row at a time, and numpy's vectorised exp/log on Linux can round the same value differently in another batch, so the docs build failed on one release run and passed on the one before. It compares to rtol 1e-12 now. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Km2eWBiPJ5JNkme3vRg4B
Docs: the partial-effects check compares to rounding, not bit for bit
Without the ExpoWeibull's mrl_cb sweep the conformance sweeps took 9.5 minutes on four workers locally, but on the release run they were 59% done when the 45-minute timeout stopped them. They run in three groups of about equal time by their local per-case totals (core-seconds): the offset LogNormal (703), the ExpoWeibull (691) and the rest (788); the three -k filters select 23, 23 and 196 of the 242 tests. The timeout is 60 minutes: the longest group on the day's runs, #617's bootstrap, took 24 to 36, and the runners' speed varied by about two (extra-parameters took 52 minutes on one run and 96 on another). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Km2eWBiPJ5JNkme3vRg4B
Calibration CI: the conformance sweeps in three groups
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 0.25:
dev→main. The full changelog is the v0.25 section ofdocs/changelog.rst.Highlights:
JointModel.predict→DynamicPrediction, withto_dict()), plus interval and left censored items, truncation, and a calibration study.surpyval.test_planning: demonstration, sequential, life-test precision, accelerated test plans, log-rank and rate-ratio power. Every result hasto_dict().surpyval.recurrent.intervention_test.plot_nonparametric, with a number-at-risk table, andat_risk_tablefor the same table as data.RecurrentEventData, nowEventHistoryData(until v0.26).After merge:
gh workflow run publish.yml --ref main -f version=0.25(dry run first with-f dry_run=true).🤖 Generated with Claude Code
https://claude.ai/code/session_019Km2eWBiPJ5JNkme3vRg4B