Beyond Noise: Characterizing Creative Potential in Unverifiable LLM Hallucinations
Yu Yan, Chunhong Zhang, Haiyu Zhao, Ziyang Zeng, Zihao Liu, Yongkang Wu, Jianzhou Diao, Yijie Chen, Shujie Wang, Zheng Hu
Abstract
In knowledge-intensive creative tasks, Large Language Models (LLMs) often generate outputs that extend beyond established knowledge, making direct verification against current evidence impractical. Unlike factual hallucinations checked against ground truth, such outputs arise naturally in creative generation, where extending beyond current knowledge is often the goal. Yet prior work debates whether hallucination should be suppressed or embraced without empirically analyzing this unverifiable subclass. On the ideation evaluation side, existing work focuses on individual outputs without characterizing the unverifiable space as a whole. To address this gap, we propose a novelty-verifiability characterization that distinguishes Creative Synthesis (Region A) from Groundless Fabrication (Region B), and study it through a conceptual creation task where LLMs synthesize novel scientific concepts. Through 32,400 generations across three technical domains and 1,080 human judgments, we find that Region A is non-negligible (4.7%) and robust, persisting across generation strategies, models, domains, and embedding choices. A retrospective recovery experiment further shows that LLMs can approximate postcutoff scientific concepts in controlled combinatorial settings. Our findings suggest that the unverifiable space is not uniformly noise but exhibits empirically distinguishable internal structure, providing an empirical basis for more selective hallucination governance. 1
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 06f9376a-3498-4372-8add-93dfe38ba30fBuilds on7
- The Linear Representation Hypothesis and the Geometry of Large Language ModelsKiho Park, Yo Joong Choe, Victor VeitchICML 2024 · 461 citations
- SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language ModelsPotsawee Manakul, Adian Liusie, Mark J. F. GalesEMNLP 2023 · 331 citations
- HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language ModelsJunyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie et al.EMNLP 2023 · 224 citations
- Cache-to-Cache: Direct Semantic Communication Between Large Language ModelsTianyu Fu, Zihan Min, Hanling Zhang, Jichao Yan et al.ICLR 2026 · 59 citations
- Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP ResearchersChenglei Si, Diyi Yang, Tatsunori HashimotoICLR 2025
Related papers
- HALoGEN: Fantastic LLM Hallucinations and Where to Find ThemAbhilasha Ravichander, Shrusti Ghela, David Wadden, Yejin ChoiACL 2025 · 35 citations
- HalluLens: LLM Hallucination BenchmarkYejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn et al.ACL 2025
- Can Large Language Models Unlock Novel Scientific Research Ideas?Sandeep Kumar, Tirthankar Ghosal, Vinayak Goyal, Asif EkbalEMNLP 2025 · 3 citations
- Confabulation: The Surprising Value of Large Language Model HallucinationsPeiqi Sui, Eamon Duede, Sophie Wu, Richard Jean SoACL 2024
- CodeHalu: Investigating Code Hallucinations in LLMs via Execution-based VerificationYuchen Tian, Weixiang Yan, Qian Yang, Xuandong Zhao et al.AAAI 2025 · 41 citations
