Online Conformal Prediction via Online Optimization

We introduce a family of algorithms for online conformal prediction with coverage guarantees for both adversarial and stochastic data. In the adversarial setting, we establish the standard guarantee: over time, a pre-specified target fraction of confidence sets cover the ground truth. For stochastic data, we provide a guarantee at every time instead of just on average over time: the probability that a confidence set covers the ground truth-conditioned on past observations-converges to a pre-specified target when the conditional quantiles of the errors are a linear function of past data. Complementary to our theory, our experiments spanning over 15 datasets suggest that the performance improvement of our methods over baselines grows with the magnitude of the data's dependence, even when baselines are tuned on the test set. We put these findings to the test by pre-registering an experiment for electricity demand forecasting in Texas, where our algorithms achieve over a 10% reduction in confidence set sizes, a more than a 30% improvement in quantile and absolute losses with respect to the observed errors, and significant outcomes on all 78 out of 78 pre-registered hypotheses. We provide documentation for the pypi package implementing our algorithms here: https: //conformalopt.readthedocs.io/ .