The Entropic Overhead of Conformal Correction

Conformal prediction (CP) provides a comprehensive framework to produce statistically rigorous uncertainty sets for black-box machine learning models. To further improve the efficiency, conformal correction extends standard CP by wrapping the base model with an extra module. In this work, we empirically and theoretically identify a trade-off in conformal prediction: its efficiency is at odds with the entropy of model prediction. We further observe that existing conformal correction methods are essentially trading entropy for efficiency. Based on the above observations, we then propose an entropy-constrained conformal correction method, better exploring the Pareto frontier between efficiency and entropy. Extensive experimental results on both computer vision and graph datasets demonstrate the efficacy of the proposed method. For instance, it can significantly improve the efficiency of state-of-the-art CP methods by up to 34.4%, given an entropy threshold. An implementation of our conformal correction approach can be accessed at the following anonymous link: https://github.com/Xusr1123/Conformal-Correction.