From Sampling to Cognition: Modeling Internal Cognitive Confidence in Language Models for Robust Uncertainty Calibration
Hao Li, Tao He, Jiafeng Liang, Zheng Chu, Ming Liu
Abstract
Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of tasks, yet they generally lack self-awareness, often displaying overconfidence when confronted with questions beyond their knowledge boundaries. This limitation severely hinders their trustworthiness in high-stakes scenarios. Existing calibration methods typically rely on sampling accuracy, derived from multiple outputs, as a proxy for model confidence. However, this coarse-grained metric fails to capture the model’s internal cognitive states, such as confusion, hallucination, or persistent belief in false knowledge. To address this, we propose CogConf (Cognitive Confidence), a cognitively grounded uncertainty signal that extends sampling accuracy by incorporating the semantic diversity of incorrect answers and the model’s abstention behaviors. By shifting the focus from sampling-based to cognition-oriented uncertainty modeling, CogConf offers a more faithful reflection of the model's internal beliefs. Building on this signal, we introduce CogAlign, a simple yet effective alignment framework that explicitly aligns the model’s verbalized confidence with CogConf, thereby producing uncertainty estimates that better reflect the model’s internal cognition. Experimental results on six knowledge-intensive in-domain and out-of-domain QA datasets demonstrate that CogConf robustly characterizes the model's internal uncertainty. Building on this foundation, CogAlign guides the model's expression to significantly enhance the trustworthiness and utility of its uncertainty calibration without compromising its underlying QA capabilities, while also demonstrating strong cross-task generalization and output stability. Offering a new pathway toward building more trustworthy LLMs.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on14
- Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question AnsweringPan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu et al.NeurIPS 2022 · 2,727 citations
- Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMsMiao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li et al.ICLR 2024 · 867 citations
- Scalable Best-of-N Selection for Large Language Models via Self-CertaintyZhewei Kang, Xuandong Zhao, Dawn SongNeurIPS 2025 · 211 citations
- Do Large Language Models Know What They Don't Know?Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu et al.ACL Findings 2023 · 75 citations
- Calibrating Large Language Models with Sample ConsistencyQing Lyu, Kumar Shridhar, Chaitanya Malaviya, Li Zhang et al.AAAI 2025 · 72 citations
Related papers
- Do LVLMs Know What They Know? A Systematic Study of Knowledge Boundary Perception in LVLMsZhikai Ding, Shiyu Ni, Keping BiEMNLP Findings 2025
- UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language ModelsBoyang Xue, Fei Mi, Qi Zhu, Hongru Wang et al.ACL 2025 · 10 citations
- ADVICE: Answer-Dependent Verbalized Confidence EstimationKi Jung Seo, Sehun Lim, Taeuk KimACL 2026 · 4 citations
- MetaFaith: Faithful Natural Language Uncertainty Expression in LLMsGabrielle Kaili-May Liu, Gal Yona, Avi Caciularu, Idan Szpektor et al.EMNLP 2025
- Decoupling Metacognition from Cognition: A Framework for Quantifying Metacognitive Ability in LLMsGuoqing Wang, Wen Wu, Guangze Ye, Zhenxiao Cheng et al.AAAI 2025 · 15 citations
