From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Models

While Large Language Models (LLMs) show remarkable capabilities, their unreliability remains a critical barrier to deployment in highstakes domains. This survey charts a functional evolution in addressing this challenge: the evolution of uncertainty from a passive diagnostic metric to an active control signal guiding real-time model behavior. We demonstrate how uncertainty is leveraged as an active control signal across three frontiers: in advanced reasoning to optimize computation and trigger self-correction; in autonomous agents to govern metacognitive decisions about tool use and information seeking; and in reinforcement learning to mitigate reward hacking and enable self-improvement via intrinsic rewards. By grounding these advancements in emerging theoretical frameworks like Bayesian methods and Conformal Prediction, we provide a unified perspective on this transformative trend. This survey provides a comprehensive overview, critical analysis, and practical design patterns, arguing that mastering the new trend of uncertainty is essential for building the next generation of scalable, reliable, and trustworthy AI. Method / Framework Key Advantage(s) Key Disadvantage(s) / Failure Mode(s) Cost Complexity Between Reasoning Paths CISC -More efficient than standard self-consistency. -A single bad step can sink a good path score. -Relies on well-calibrated confidence. High Low CER -Robust for long-chain reasoning. -Focuses on the most important steps. -Must correctly identify "critical" steps. -Can amplify errors from miscalibrated confidence. Very High Medium Inside a Reasoning Path UAG / SPOC -Enables real-time error correction. -No retraining required. -LLMs often fail at true self-correction. -Can get stuck in correction loops. Medium High Uncertainty-Aware FT -Fundamentally improves model calibration. -Benefits all downstream tasks. -Data-intensive training process. -Risk of harming in-distribution performance. Low High Optimizing Cognitive Effort UnCert-CoT -Excellent efficiency-performance balance. -Simple and intuitive concept. -Performance is highly sensitive to the threshold value. Low Low MUR -More stable control via momentum. -Finer-grained resource allocation. -More complex than simple triggers. -Adds more hyperparameters to tune. Low-Medium Medium Method / Framework Key Advantage(s) Key Disadvantage(s) / Failure Mode(s) Cost Complexity Function: Responding to Internal Uncertainty Abstention -Simple, robust safety mechanism. -Prevents generating harmful misinformation. -Can be overly conservative, reducing helpfulness. -Performance is highly sensitive to the threshold. Low Low Proactive Inquiry (UoT) -Actively reduces uncertainty, improving final quality. -Mimics intelligent, collaborative behavior. -Can increase user burden with too many questions. -Requires a complex (often RL-trained) policy. Medium-High High Function: Tool-Use Decision Boundary UALA -Greatly improves efficiency vs. always-use-tool. -Simple threshold-based logic. -Does not account for tool unreliability (blind trust). -Static threshold may not generalize well. Low Low SMARTAgent -Internalizes knowledge boundaries via training. -More robust than a simple static threshold. -Requires creating a specialized fine-tuning dataset. -Higher upfront training cost. Low (inference) High Function: Uncertainty Propagation SAUP -Pragmatic and intuitive approach. -Context-aware weighting is powerful. -Situational weights can be heuristic and hard to define formally across different tasks. Medium Medium UProp -Principled, information-theoretic foundation. -Clearly separates intrinsic vs. extrinsic uncertainty. -Computationally expensive to estimate mutual info. -Can be less practical for real-time applications. High High Table 5: Critical Analysis of Methods in Autonomous Agents. This table provides a comparative overview of key methodologies, focusing on their advantages, failure modes, computational costs, and implementation complexity. Method / Framework Key Advantage(s) Key Disadvantage(s) / Failure Mode(s) Cost Complexity Function: Robust Reward Models URM -Explicitly models data ambiguity (aleatoric uncertainty). -Simple architectural change. -May not capture model's own ignorance (epistemic). -Requires changing the training objective. Low (inference) Medium Bayesian RMs -Principled way to capture model uncertainty (epistemic). -Provides a theoretically-grounded penalty for RL. -Can be computationally expensive to train and run. -More complex to implement correctly. Medium-High High