[fbonc]

Adds EnbPI from Xu and Xie, "Conformal prediction interval for dynamic time-series", ICML 2021, as the first time series conformal method in pyhealth.calib. It builds prediction intervals for time series regression and updates them online as labels are revealed, without assuming exchangeability. ACI and AgACI will follow on the same interface.

Changes

• pyhealth.calib.predictionset.EnbPI trains a bootstrap ensemble from a model_fn factory, fills a sliding window with out-of-bag absolute residuals, and returns the center plus or minus the conformal (1 - alpha) quantile of the window. reset() restores the calibration window, and state_dict() carries the full calibration state for checkpoints.
• IntervalPredictor in pyhealth/calib/base_classes.py declares calibrate, forward and update, like SetPredictor.
• pyhealth/metrics/prediction_interval.py adds miscoverage_pi, interval_width and rolling_miscoverage_pi@W, available through regression_metrics_fn(..., y_lower=..., y_upper=...).
• API docs, a README section, examples/conformal_timeseries/enbpi_synthetic.py and tests.

Key decisions

• The default center is phi over the out-of-bag predictions, as in the paper's experiments and MAPIE. center="quantile" reproduces Algorithm 1 line 12, whose (1 - alpha) quantile biases the point prediction upward.
• EnbPI trains its own ensemble from a model_fn factory, following MAPIE's clone-per-bootstrap convention, so the bootstrap bags always match the models. No calibrator in PyHealth trained models before, so there was no in-repo convention.
• IntervalPredictor fixes only the public interface. Each method implements its own online loop, because ACI and AgACI update in ways a shared loop would constrain.
• forward is stateless and works inside Trainer.inference, and update slides the window. predict_sequential is a convenience method for evaluation that runs the paper's loop over a labeled test stream.
• Every calibrate call trains a new ensemble on the data passed in, like other calibrators recompute from their data. reset() restores the calibration window without retraining, so evaluations can be rerun from the same start.
• state_dict() saves the whole predictor, including the bootstrap indices and the residual window, through PyTorch's extra state hooks, so load_state_dict and Trainer checkpoints restore calibrated inference and resume online updates. The hooks sit in a separate checkpointing section at the end of EnbPI, apart from the algorithm.
• calibrate raises when no training point has an out-of-bag model, since every interval would be NaN. The paper does not address this case, and with the recommended B = 20 to 30 it essentially never happens.
• Scope is regression mode with one stream in time order and absolute residuals, as in the ICML paper. Many series at once (timeseries_regression) and signed residuals are follow-ups.

[github-actions]

Test Coverage

Total: 74.27% (20594 / 27729 lines)

Modified file	Coverage	Lines
pyhealth/calib/base_classes.py	75.00%	18 / 24
pyhealth/calib/predictionset/__init__.py	100.00%	10 / 10
pyhealth/calib/predictionset/timeseries/__init__.py	100.00%	3 / 3
pyhealth/calib/predictionset/timeseries/enbpi.py	100.00%	188 / 188
pyhealth/calib/predictionset/timeseries/utils.py	100.00%	22 / 22
pyhealth/metrics/prediction_interval.py	100.00%	20 / 20
pyhealth/metrics/regression.py	90.20%	46 / 51

Workflow run · d68b819