Understanding Uncertainty In LLMs

It is important to understand the uncertainty in large language models (LLMs) explanations because they reveal more information about the reasoning process and thus provide insights into the reliability of LLMs' answers regarding a question. In this work, we propose a novel framework that quantifies uncertainty in LLM explanations through reasoning topologies. By designing a structural elicitation strategy, we guide the LLM to frame the explanations on how it derives the answers into graph topologies. This strategy first queries knowledge-related sub-question and sub-answer pairs, and then guides the LLM to connect the pairs through a topological reasoning process. Based on the reasoning topologies, we revisit the Graph Edit Distance and provide a variant that can better quantify the LLM uncertainty from both semantic and reasoning structure dimensions. The topology structure further brings convenience to assess redundancy by extracting and comparing the valid reasoning path to the raw explanation. Extensive experiments show the effectiveness of the proposed framework, and interesting findings on reasoning patterns and efficiency are discussed.