Lune

EMNLP Findings2023

Reasoning about Ambiguous Definite Descriptions

Stefan F. Schouten, Peter Bloem, Ilia Markov, Piek Vossen

Abstract

Natural language reasoning plays an increasingly important role in improving language models' ability to solve complex language understanding tasks.An interesting use case for reasoning is the resolution of contextdependent ambiguity.But no resources exist to evaluate how well Large Language Models can use explicit reasoning to resolve ambiguity in language.We propose to use ambiguous definite descriptions for this purpose and create and publish the first benchmark dataset consisting of such phrases.Our method includes all information required to resolve the ambiguity in the prompt, which means a model does not require anything but reasoning to do well.We find this to be a challenging task for recent LLMs.Code and data available at: https://github.com/ sfschouten/exploiting-ambiguity

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Identifying & Interactively Refining Ambiguous User Goals for Data Visualization Code Generation cites AmbigQA: Answering Ambiguous Open-domain Questions
  • Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Reasoning with Language Model Prompting: A Survey cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Reasoning with Language Model Prompting: A Survey cites Towards Reasoning in Large Language Models: A Survey
  • Towards Reasoning in Large Language Models: A Survey cites Reasoning with Language Model Prompting: A Survey
  • Towards Reasoning in Large Language Models: A Survey cites Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
  • Towards Reasoning in Large Language Models: A Survey cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Identifying & Interactively Refining Ambiguous User Goals for Data Visualization Code GenerationMert Inan, Anthony Sicilia, Alex Xie, Saujas Vaduguru et al.EMNLP 2025

Builds on6

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Towards Reasoning in Large Language Models: A SurveyJie Huang, Kevin Chen-Chuan ChangACL Findings 2023 · 428 citations
  3. Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve ThemMirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann et al.ACL Findings 2023 · 253 citations
  4. AmbigQA: Answering Ambiguous Open-domain QuestionsSewon Min, Julian Michael, Hannaneh Hajishirzi, Luke ZettlemoyerEMNLP 2020 · 162 citations
  5. Reasoning with Language Model Prompting: A SurveyShuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen et al.ACL 2023 · 124 citations

Related papers

  1. AmbiK: Dataset of Ambiguous Tasks in Kitchen EnvironmentAnastasiia Ivanova, Eva Bakaeva, Zoya Volovikova, Alexey K. Kovalev et al.ACL 2025
  2. Task Ambiguity in Humans and Language ModelsAlex Tamkin, Kunal Handa, Avash Shrestha, Noah D. GoodmanICLR 2023 · 8 citations
  3. Large Language Models Can Be Easily Distracted by Irrelevant ContextFreda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales et al.ICML 2023 · 970 citations
  4. VAGUE: Visual Contexts Clarify Ambiguous ExpressionsHeejeong Nam, Jinwoo Ahn, Keummin Ka, Jiwan Chung et al.ICCV 2025 · 1 citation
  5. Meaningful Learning: Enhancing Abstract Reasoning in Large Language Models via Generic Fact GuidanceKai Xiong, Xiao Ding, Ting Liu, Bing Qin et al.NeurIPS 2024
Dusk over the sea between two cliffs drawn in fine vertical lines