ACL2026
NeuroSym-Cal: Bridging the Reasoning-Execution Gap in Code Generation via Hierarchical Calibration
Peiyang Liu, Yining Wang, Youru Li, Long Li, Zhi Cai, Wei Ye
Abstract
While Chain-of-Thought (CoT) reasoning enhances code generation in Large Language Models (LLMs), it introduces a critical challenge in uncertainty estimation: Confidence Saturation. Existing calibration methods, such as Self-Consistency, rely on the assumption that consensus implies correctness. This assumption fails under systematic errors, where models confidently repeat flawed logic, leading to miscalibrated high-confidence predictions. To address this, we introduce NEUROSYM-CAL, a hierarchical calibration framework. We posit that reliable confidence requires interrogating the model at two complementary levels: the extrinsic consensus of its symbolic outputs and the intrinsic self-assessment of its generated logic. Specifically, we propose Self-Verification Analysis, which prompts the model to holistically re-evaluate its completed candidate, exploiting the cognitive asymmetry between autoregressive generation and post-hoc reviewing. This provides a fine-grained continuous signal that persists even when output consensus saturates. These orthogonal features, augmented by code-level descriptors, are fused by a Contextual Calibration Network to predict correctness. Experiments across state-of-the-art reasoning models (e.g., DeepSeek-R1) demonstrate that NEUROSYM-CAL effectively desaturates overconfident errors, achieving stateof-the-art Expected Calibration Error (ECE) and superior selective generation performance on Out-Of-Domain (OOD) benchmarks.