Lune

EMNLP Findings2023

That was the last straw, we need more: Are Translation Systems Sensitive to Disambiguating Context?

Jaechan Lee, Alisa Liu, Orevaoghene Ahia, Hila Gonen, Noah A. Smith

Abstract

The translation of ambiguous text presents a challenge for translation systems, as it requires using the surrounding context to disambiguate the intended meaning as much as possible.

While prior work has studied ambiguities that result from different grammatical features of the source and target language, we study semantic ambiguities that exist in the source (English in this work) itself. In particular, we focus on idioms that are open to both literal and figurative interpretations (e.g., goose egg), and collect TIDE, 1 a dataset of 512 pairs of English sentences containing idioms with disambiguating context such that one is literal (it laid a goose egg) and another is figurative (they scored a goose egg, as in a score of zero). In experiments, we compare MT-specific models and language models for (i) their preference when given an ambiguous subsentence, (ii) their sensitivity to disambiguating context, and (iii) the performance disparity between figurative and literal source sentences. We find that current MT models consistently translate English idioms literally, even when the context suggests a figurative interpretation. On the other hand, LMs are far more context-aware, although there remain disparities across target languages. Our findings underline the potential of LMs as a strong backbone for context-aware translation.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • We're Afraid Language Models Aren't Modeling Ambiguity cites WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation
  • We're Afraid Language Models Aren't Modeling Ambiguity cites AmbigQA: Answering Ambiguous Open-domain Questions
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Voices Unheard: NLP Resources and Models for Yorùbá Regional DialectsOrevaoghene Ahia, Anuoluwapo Aremu, Diana Abagyan, Hila Gonen et al.EMNLP 2024 · 1 citation
  2. Semantic and Expressive Variations in Image Captions Across LanguagesAndre Ye, Sebastin Santy, Jena D. Hwang, Amy X. Zhang et al.CVPR 2025
  3. Memorization or Reasoning? Exploring the Idiom Understanding of LLMsJisu Kim, Youngwoo Shin, Uiji Hwang, Jihun Choi et al.EMNLP 2025

Builds on13

  1. AmbigQA: Answering Ambiguous Open-domain QuestionsSewon Min, Julian Michael, Hannaneh Hajishirzi, Luke ZettlemoyerEMNLP 2020 · 162 citations
  2. WANLI: Worker and AI Collaboration for Natural Language Inference Dataset CreationAlisa Liu, Swabha Swayamdipta, Noah A. Smith, Yejin ChoiEMNLP Findings 2022 · 122 citations
  3. BLEURT: Learning Robust Metrics for Text GenerationThibault Sellam, Dipanjan Das, Ankur P. ParikhACL 2020 · 40 citations
  4. We're Afraid Language Models Aren't Modeling AmbiguityAlisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr et al.EMNLP 2023 · 35 citations
  5. FLUTE: Figurative Language Understanding through Textual ExplanationsTuhin Chakrabarty, Arkadiy Saakyan, Debanjan Ghosh, Smaranda MuresanEMNLP 2022 · 35 citations

Related papers

  1. It's Not a Walk in the Park! Challenges of Idiom Translation in Speech-to-text SystemsIuliia Zaitova, Badr M. Abdullah, Wei Xue, Dietrich Klakow et al.ACL 2025 · 1 citation
  2. Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss WeightingEmmy Liu, Aditi Chaudhary, Graham NeubigEMNLP 2023 · 2 citations
  3. Rolling the DICE on Idiomaticity: How LLMs Fail to Grasp ContextMaggie Mi, Aline Villavicencio, Nafise Sadat MoosaviACL 2025 · 9 citations
  4. Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine TranslationVerna Dankers, Christopher G. Lucas, Ivan TitovACL 2022
  5. Multilingual Idioms in Sentences and Conversations Across High-, Medium-, and Low-Resource LanguagesSaeed Almheiri, Bilal Elbouardi, Salsabila Zahirah Pranida, Irina Nikishina et al.ACL 2026
Dusk over the sea between two cliffs drawn in fine vertical lines