G-IdiomAlign: A Gloss-Pivoted Benchmark for Cross-Lingual Idiom Alignment
Fengying Ye, Yanming Sun, Runzhe Zhan, Lidia S. Chao, Zheqi Zhang, Derek F. Wong
Abstract
Idioms are difficult to transfer across languages due to their non-compositionality and weak surface-form grounding, making literal mappings unreliable. We present G-IdiomAlign, a gloss-pivoted benchmark where each idiom is anchored by an English gloss from Wiktionary. We further construct a high-confidence reference alignment set for reproducible evaluation. G-IdiomAlign supports two protocols: (1) a controlled Multiple-Choice Idiom Equivalence with typed distractors for error attribution; and (2) a Gloss-Contrastive Generation contrasting No-gloss and With-gloss inputs to isolate the effect of an explicit semantic pivot. Across diverse LLMs, a bias to literal translation is a dominant failure mode, especially when the target is a low-resource language. Glosses consistently improve Gloss-Contrastive Generation under an embedding-based semantic proxy, but performance remains modest, indicating substantial headroom in the open output space. Subsequent analysis on Qwen3-8B further suggests that cross-condition differences are concentrated more in attention heads than in layers, while better With-gloss generations coincide with stronger gloss anchoring 1 .
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext b2d32650-4e7b-4ac5-8f1a-95a8c2b58a1cCited by top-tier papers1
Ask how each one uses itBuilds on9
- Translate Meanings, Not Just Words: IdiomKB's Role in Optimizing Idiomatic Translation with Language ModelsShuang Li, Jiangjie Chen, Siyu Yuan, Xinyi Wu et al.AAAI 2024 · 44 citations
- Rolling the DICE on Idiomaticity: How LLMs Fail to Grasp ContextMaggie Mi, Aline Villavicencio, Nafise Sadat MoosaviACL 2025 · 9 citations
- Probing Semantic Alignment, Lexical Invariance, and Syntactic Influence in LLM Metaphor ProcessingFengying Ye, Shanshan Wang, Lidia S. Chao, Derek F. WongACL 2026 · 7 citations
- Enhancing Bilingual Lexicon Induction via Bi-directional Translation Pair RetrievingQiuyu Ding, Hailong Cao, Tiejun ZhaoAAAI 2024 · 3 citations
- Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss WeightingEmmy Liu, Aditi Chaudhary, Graham NeubigEMNLP 2023 · 2 citations
Related papers
- DeReA: Improving Idiom Translation with Detect-Retrieve-Arbitrate ReasoningRongqing Jiang, Xuebo Liu, Shengxin Liu, Yutong Wang et al.ACL 2026
- More Than Meets the Eye: Measuring the Semiotic Gap in Vision-Language Models via Semantic AnchorageWei HeACL 2026
- CHENGYU-BENCH: Benchmarking Large Language Models for Chinese Idiom Understanding and UseYicheng Fu, Zhemin Huang, Liuxin Yang, Yumeng Lu et al.EMNLP 2025
- EIFFEL: a novel benchmark to measure bias of English heavy training on French idiomatic expressionsCharlotte Noel, Nicholas Asher, Olivier Gouvert, Farah Benamara et al.ACL 2026
- Multilingual Idioms in Sentences and Conversations Across High-, Medium-, and Low-Resource LanguagesSaeed Almheiri, Bilal Elbouardi, Salsabila Zahirah Pranida, Irina Nikishina et al.ACL 2026
