Efficient Hallucination Detection in Automatic Code Generation

Large language models (LLMs) frequently produce source code that seems correct and wellformed, yet includes hallucinated elements that cause downstream test failures. In this study, we benchmark state-of-the-art uncertainty quantification methods and existing baselines for the task of hallucination detection in source code and introduce a diff-based pipeline to construct a code dataset annotated with line-level hallucinations. Building on this, we train a lightweight Transformer-based detector that uses LLM internal representations to identify hallucinations, substantially outperforming existing methods across several code generation domains. The detector also shows particular promise for enabling self-correction in LLM-based coding agents. We release the first publicly available dataset of line-level code hallucinations, along with the corresponding source code and trained hallucination detectors https://github.com/ datapaf/CodeHallucinationDetection Generated Code (Collu-Bench) Correct Code (Collu-Bench) Line Splitting Diff Utility Lines of Correct Code