Can LLMs Reason Soundly in Law? Auditing Inference Patterns for Legal Judgment
Lu Chen, Yuxuan Huang, Yixing Li, Dongrui Liu, Qihan Ren, Shuai Zhao, Kun Kuang, Zilong Zheng, Quanshi Zhang
Abstract
This paper presents a method to analyze the inference patterns used by Large Language Models (LLMs) for judgment in a case study on legal LLMs, so as to identify potential incorrect representations of the LLM, according to human domain knowledge. Unlike traditional evaluations on language generation results, we propose to evaluate the correctness of the detailed inference patterns of an LLM behind its seemingly correct outputs. To this end, we quantify the interactions between input phrases used by the LLM as primitive inference patterns, because recent theoretical achievements (26; 42) have proven several mathematical guarantees of the faithfulness of the interaction-based explanation. We design a set of metrics to evaluate the detailed inference patterns of LLMs. Experiments show that even when the language generation results appear correct, a significant portion of the inference patterns used by the LLM for the legal judgment may represent misleading or irrelevant logic 1 . He is with the Department of Computer Science and Engineering, the John Hopcroft Center, at the Shanghai Jiao Tong University, China. 1 The names used in the legal cases follow an alphabetical convention, e.g., Andy, Bob, Charlie, etc., which do not represent any bias against actual individuals. On the (A) morning of December 22, 2013, the defendants Andy and Bob deceived Charlie and the three of them (B) had an argument. Andy (C) chased Charlie (D) with an axe and (E) bit Charlie, causing Charlie to be (F) slightly injured. Bob (G) hit Charlie (H) with a shovel, (I) injuring Charlie and causing Charlie's (J) death. Irrelevant phrases Relevant phrases Forbidden phrases Input legal case 𝒙 LLM's output 𝒗("𝐴𝑠𝑠𝑎𝑢𝑙𝑡"|𝒙) Input legal case 𝒙 Equivalently modeling Rel iab le inte rac tion effe cts Un re lia bl e in te ra ct io n ef fe ct s 𝑅 !
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- TruthfulQA: Measuring How Models Mimic Human Falsehoods cites Training language models to follow instructions with human feedback
- Training language models to follow instructions with human feedback cites TruthfulQA: Measuring How Models Mimic Human Falsehoods
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Training language models to follow instructions with human feedback
Builds on16
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 citations
- The Shapley Taylor Interaction IndexMukund Sundararajan, Kedar Dhamdhere, Ashish AgarwalICML 2020 · 199 citations
- How Do Transformers Learn Topic Structure: Towards a Mechanistic UnderstandingYuchen Li, Yuanzhi Li, Andrej RisteskiICML 2023 · 87 citations
Related papers
- KoBLEX: Open Legal Question Answering with Multi-hop ReasoningJihyung Lee, Daehui Kim, Seonjeong Hwang, Hyounghun Kim et al.EMNLP 2025
- Conditional and Modal Reasoning in Large Language ModelsWesley H. Holliday, Matthew Mandelkern, Cedegao ZhangEMNLP 2024 · 5 citations
- Evaluating Test-Time Scaling LLMs for Legal Reasoning: OpenAI o1, DeepSeek-R1, and BeyondYinghao Hu, Yaoyao Yu, Leilei Gan, Bin Wei et al.EMNLP Findings 2025 · 1 citation
- Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMsSiyuan Wang, Zhongyu Wei, Yejin Choi, Xiang RenACL 2024
- Real-World Summarization: When Evaluation Reaches Its LimitsPatrícia Schmidtová, Ondrej Dusek, Saad MahamoodEMNLP Findings 2025
