Conformal Prediction via Regression-as-Classification

Conformal prediction (CP) for regression can be challenging, especially when the output distribution is heteroscedastic, multimodal, or skewed. Some of the issues can be addressed by estimating a distribution over the output, but in reality, such approaches can be sensitive to estimation error and yield unstable intervals. Here, we circumvent the challenges by converting regression to a classification problem and then use CP for classification to obtain CP sets for regression. To preserve the ordering of the continuous-output space, we design a new loss function and make necessary modifications to the CP classification techniques. Empirical results on many benchmarks shows that this simple approach gives surprisingly good results on many practical problems. INTRODUCTION Quantifying and estimating the uncertainty of machine-learning models is an important task for many problems, especially mission-critical applications where reliable predictions are required. Conformal Prediction (CP) (Vovk et al., 2005) has recently gained popularity and has been used successfully in applications such as breast cancer detection (Lambrou et al., 2009) , stroke risk prediction (Lambrou et al., 2010) , and drug discovery (Cortés-Ciriano & Bender, 2020) . Under mild conditions, CP techniques aim to construct a prediction set that, for given test inputs, is guaranteed to contain the true (unknown) output with high probability. The set is built using a conformity score, which, roughly speaking, indicates the similarity between a new test example and the training examples. The conformal set merely gathers examples that have large conformity scores. Despite its popularity, CP for regression can be challenging, especially when the output distribution is heteroscedastic, multimodal, or skewed (Lei & Wasserman, 2014) . The main challenge lies in the design of the conformity score. It is common to use a simple choice for score functions such as distance to mean regressor, but such choices may ignore the subtle features of the shape of the output distribution. For instance, this could lead to symmetric intervals or ignoring the heteroscedasticity. In theory, it is better to estimate the (conditional) distribution over the output, for example, by using kernel density estimation and directly using it to build a confidence interval. However, such estimation approaches are also challenging, and estimates can be sensitive to the choice of kernel and hyperparameters, which can yield unstable results. We circumvent the challenges by exploiting the existing CP techniques for classification. We proceed by first converting regression to a classification problem and then using CP techniques for classification to obtain a conformal set. Regression-as-classification approaches are popular for various applications in computer vision and have led to more accurate training than only-regression training (