Ensemble Conformal Predictor (EnCP): A New Conformal Predictor with Robustness Guarantees Against Data Poisoning Attacks

Conformal Prediction (CP) is a popular statistical framework for uncertainty quantification by producing prediction sets with valid coverage guarantees (e.g., ensuring the true label falls within the predicted set with a user-defined confidence level such as 95 %). It has recently gained popularity in both classical machine learning (ML) tasks (e.g., image classification) and large language model (LLM) applications (e.g., toxic content classification). However, recent works show CP is vulnerable to adversarial attacks in both the learning and inference phases, affecting its reliability in real-world applications. While several studies investigated defenses against inference-phase attacks, the threat of learning-phase (particularly data poisoning) attacks remains largely under-explored. We take the first step towards developing a provably robust CP framework (called EnCP) against data poisoning attacks, by addressing critical challenges including the sensitivity of the ML model and conformal predictor to poisoned data and the difficulty of maintaining both valid coverage and moderate prediction set size under the attack. Our EnCP is inspired by ensemble learning and can inherently bound the effect of poisoned samples on CP's coverage and prediction set, enabling us to derive the certified coverage and certified prediction set size. Our results demonstrate strong robustness of EnCP on both image classification benchmarks and LLM for toxicity text classification, showing that EnCP preserves both high coverage and compact prediction sets under data poisoning attacks. Source code is available at: https://github.com/Yuxin104/EnCP.