Lune

ACL Findings2023

Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Text

Zhun Yang, Adam Ishay, Joohyung Lee

Abstract

While large language models (LLMs), such as GPT-3, appear to be robust and general, their reasoning ability is not at a level to compete with the best models trained for specific natural language reasoning problems.In this study, we observe that a large language model can serve as a highly effective few-shot semantic parser.It can convert natural language sentences into a logical form that serves as input for answer set programs, a logic-based declarative knowledge representation formalism.The combination results in a robust and general system that can handle multiple question-answering tasks without requiring retraining for each new task.It only needs a few examples to guide the LLM's adaptation to a specific task, along with reusable ASP knowledge modules that can be applied to multiple tasks.We demonstrate that this method achieves state-of-the-art performance on several NLP benchmarks, including bAbI, StepGame, CLUTRR, and gSCAN.Additionally, it successfully tackles robot planning tasks that an LLM alone fails to solve.C is to the top right of D. C is to the right and above D at an angle of about 45 degrees.C is at a 45 degree angle to D, in the upper righthand corner.C is directly north east of D. C is above D at 2 o'clock.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

14 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Advancing Spatial Reasoning in Large Language Models: An In-Depth Evaluation and Enhancement Using the StepGame Benchmark cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • TopViewRS: Vision-Language Models as Top-View Spatial Reasoners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction cites Language Models are Few-Shot Learners
  • Graph-enhanced Large Language Models in Asynchronous Plan Reasoning cites Language Models are Few-Shot Learners
  • Graph-enhanced Large Language Models in Asynchronous Plan Reasoning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • NeSyPr: Neurosymbolic Proceduralization For Efficient Embodied Reasoning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • (Comet-) Atomic 2020: On Symbolic and Neural Commonsense Knowledge Graphs cites Language Models are Few-Shot Learners
  • Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models cites Language Models are Few-Shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers14

  1. Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language ModelsWenshan Wu, Shaoguang Mao, Yadong Zhang, Yan Xia et al.NeurIPS 2024 · 100 citations
  2. Advancing Spatial Reasoning in Large Language Models: An In-Depth Evaluation and Enhancement Using the StepGame BenchmarkFangjun Li, David C. Hogg, Anthony G. CohnAAAI 2024 · 60 citations
  3. Graph-enhanced Large Language Models in Asynchronous Plan ReasoningFangru Lin, Emanuele La Malfa, Valentin Hofmann, Elle Michelle Yang et al.ICML 2024 · 33 citations
  4. TopViewRS: Vision-Language Models as Top-View Spatial ReasonersChengzu Li, Caiqi Zhang, Han Zhou, Nigel Collier et al.EMNLP 2024 · 5 citations
  5. NeSyPr: Neurosymbolic Proceduralization For Efficient Embodied ReasoningWonje Choi, Jooyoung Kim, Honguk WooNeurIPS 2025 · 4 citations

Builds on19

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  3. CLEVRER: Collision Events for Video Representation and ReasoningKexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli et al.ICLR 2020 · 584 citations
  4. KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation ExtractionXiang Chen, Ningyu Zhang, Xin Xie, Shumin Deng et al.WWW 2022 · 488 citations
  5. (Comet-) Atomic 2020: On Symbolic and Neural Commonsense Knowledge GraphsJena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da et al.AAAI 2021 · 458 citations

Related papers

  1. Selection-Inference: Exploiting Large Language Models for Interpretable Logical ReasoningAntonia Creswell, Murray Shanahan, Irina HigginsICLR 2023 · 110 citations
  2. Constrained Language Models Yield Few-Shot Semantic ParsersRichard Shin, Christopher H. Lin, Sam Thomson, Charles Chen et al.EMNLP 2021 · 131 citations
  3. DePlot: One-shot visual language reasoning by plot-to-table translationFangyu Liu, Julian Martin Eisenschlos, Francesco Piccinno, Syrine Krichene et al.ACL Findings 2023 · 72 citations
  4. Mixing Expert Knowledge: Bring Human Thoughts Back To the Game of GoYichuan Ma, Linyang Li, Yongkang Chen, Peiji Li et al.NeurIPS 2025 · 2 citations
  5. Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-ThoughtAbulhair Saparov, He HeICLR 2023 · 38 citations
Dusk over the sea between two cliffs drawn in fine vertical lines