FLARE: Faithful Logic-Aided Reasoning and Exploration
Erik Arakelyan, Pasquale Minervini, Patrick S. H. Lewis, Pat Verga, Isabelle Augenstein
Abstract
Modern Question Answering (QA) and Reasoning approaches with Large Language Models (LLMs) commonly use Chain-of-Thought (CoT) prompting but struggle with generating outputs faithful to their intermediate reasoning chains. While neuro-symbolic methods like Faithful CoT (F-CoT) offer higher faithfulness through external solvers, they require codespecialized models and struggle with ambiguous tasks. We introduce Faithful Logic-Aided Reasoning and Exploration (FLARE), which uses LLMs to plan solutions, formalize queries into logic programs, and simulate code execution through multi-hop search without external solvers. Our method achieves SOTA results on 7 out of 9 diverse reasoning benchmarks and 3 out of 3 logic inference benchmarks while enabling measurement of reasoning faithfulness. We demonstrate that model faithfulness correlates with performance and that successful reasoning traces show an 18.1% increase in unique emergent facts, 8.6% higher overlap between code-defined and execution-trace relations, and 3.6% reduction in unused relations.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions cites Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
- Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions cites CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning
- Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions cites Large Language Models are Zero-Shot Reasoners
- Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting cites Large Language Models are Zero-Shot Reasoners
- Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting cites Solving Quantitative Reasoning Problems with Language Models
- CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning cites Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
- CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning cites Large Language Models are Zero-Shot Reasoners
- CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning cites Solving Quantitative Reasoning Problems with Language Models
- Large Language Models are Zero-Shot Reasoners cites Language Models are Few-Shot Learners
- Solving Quantitative Reasoning Problems with Language Models cites Language Models are Few-Shot Learners
- Graph of Thoughts: Solving Elaborate Problems with Large Language Models cites Language Models are Few-Shot Learners
Cited by top-tier papers2
Builds on20
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- Solving Quantitative Reasoning Problems with Language ModelsAitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer et al.NeurIPS 2022 · 2,039 citations
- Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought PromptingMiles Turpin, Julian Michael, Ethan Perez, Samuel R. BowmanNeurIPS 2023 · 1,792 citations
- Graph of Thoughts: Solving Elaborate Problems with Large Language ModelsMaciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger et al.AAAI 2024 · 1,292 citations
Related papers
- Matrix as Plan: Structured Logical Reasoning with Feedback-Driven ReplanningKe Chen, Jiandian Zeng, Zihao Peng, Guo Li et al.WWW 2026
- Faithful Logical Reasoning via Symbolic Chain-of-ThoughtJundong Xu, Hao Fei, Liangming Pan, Qian Liu et al.ACL 2024
- GCoT-Decoding: Unlocking Deep Reasoning Paths for Universal Question AnsweringGuanran Luo, Wentao Qiu, Zhongquan Jian, Meihong Wang et al.ACL Findings 2026
- Unveiling Factual Recall Behaviors of Large Language Models through Knowledge NeuronsYifei Wang, Yuheng Chen, Wanting Wen, Yu Sheng et al.EMNLP 2024 · 3 citations
- AnchorCoT: Anchors Pave the Way for Multi-hop ReasoningTianshi Ming, Xian Wu, Yingying Zhang, Zichuan Fu et al.ACL Findings 2025 · 2 citations
