Unveiling AI's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
Shuangpeng Han, Mengmi Zhang
Abstract
AI models make mistakes when recognizing images-whether in-domain, out-of-domain, or adversarial. Predicting these errors is critical for improving system reliability, reducing costly mistakes, and enabling proactive corrections in real-world applications such as healthcare, finance, and autonomous systems. However, understanding what mistakes AI models make, why they occur, and how to predict them remains an open challenge. Here, we conduct comprehensive empirical evaluations using a "mentor" model-a deep neural network designed to predict another "mentee" model's errors. Our findings show that the mentor excels at learning from a mentee's mistakes on adversarial images with small perturbations and generalizes effectively to predict in-domain and out-of-domain errors of the mentee. Additionally, transformer-based mentor models excel at predicting errors across various mentee architectures. Subsequently, we draw insights from these observations and develop an "oracle" mentor model, dubbed SuperMentor, that can outperform baseline mentors in predicting errors across different error types from the ImageNet-1K dataset. Our framework paves the way for future research on anticipating and correcting AI model behaviors, ultimately increasing trust in AI systems. Our data and code are available at here.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Peering into the Unknown: Active View Selection with Neural Uncertainty Maps for 3D Reconstruction cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Out-of-Distribution Detection with Deep Nearest Neighbors cites Energy-based Out-of-distribution Detection
- Out-of-Distribution Detection with Deep Nearest Neighbors cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Adaptive Confidence Regularization for Multimodal Failure Detection cites Energy-based Out-of-distribution Detection
- Adaptive Confidence Regularization for Multimodal Failure Detection cites Out-of-Distribution Detection with Deep Nearest Neighbors
Cited by top-tier papers2
Builds on22
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
- Towards Evaluating the Robustness of Neural NetworksNicholas Carlini, David A. WagnerS&P 2017 · 9,786 citations
- The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution GeneralizationDan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath et al.ICCV 2021 · 2,294 citations
- Energy-based Out-of-distribution DetectionWeitang Liu, Xiaoyun Wang, John D. Owens, Yixuan LiNeurIPS 2020 · 2,213 citations
- Out-of-Distribution Detection with Deep Nearest NeighborsYiyou Sun, Yifei Ming, Xiaojin Zhu, Yixuan LiICML 2022 · 789 citations
Related papers
- ImageNet-X: Understanding Model Mistakes with Factor of Variation AnnotationsBadr Youbi Idrissi, Diane Bouchacourt, Randall Balestriero, Ivan Evtimov et al.ICLR 2023 · 11 citations
- Learning to Predict Trustworthiness with Steep Slope LossYan Luo, Yongkang Wong, Mohan S. Kankanhalli, Qi ZhaoNeurIPS 2021 · 17 citations
- Editable Neural NetworksAnton Sinitsin, Vsevolod Plokhotnyuk, Dmitry V. Pyrkin, Sergei Popov et al.ICLR 2020 · 210 citations
- Misbehaviour prediction for autonomous driving systemsAndrea Stocco, Michael Weiss, Marco Calzana, Paolo TonellaICSE 2020 · 138 citations
- Self-Checking Deep Neural Networks in DeploymentYan Xiao, Ivan Beschastnikh, David S. Rosenblum, Changsheng Sun et al.ICSE 2021 · 36 citations
