Lune

ACL Findings2024

Enhancing Idiomatic Representation in Multiple Languages via an Adaptive Contrastive Triplet Loss

Wei He, Marco Idiart, Carolina Scarton, Aline Villavicencio

Abstract

Accurately modeling idiomatic or noncompositional language has been a longstanding challenge in Natural Language Processing (NLP). This is partly because these expressions do not derive their meanings solely from their constituent words, but also due to the scarcity of relevant data resources, and their impact on the performance of downstream tasks such as machine translation and simplification. In this paper we propose an approach to model idiomaticity effectively using a triplet loss that incorporates the asymmetric contribution of components words to an idiomatic meaning for training language models by using adaptive contrastive learning and resampling miners to build an idiomatic-aware learning objective. Our proposed method is evaluated on a SemEval challenge and outperforms previous alternatives significantly in many metrics. Our code is available at our project 1 .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. From Input Perception to Predictive Insight: Modeling Model Blind Spots Before They Become ErrorsMaggie Mi, Aline Villavicencio, Nafise Sadat MoosaviEMNLP 2025

Builds on25

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Supervised Contrastive LearningPrannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna et al.NeurIPS 2020 · 7,049 citations
  3. SimCSE: Simple Contrastive Learning of Sentence EmbeddingsTianyu Gao, Xingcheng Yao, Danqi ChenEMNLP 2021 · 2,496 citations
  4. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text RetrievalLee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang et al.ICLR 2021 · 1,547 citations
  5. Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary et al.ACL 2020 · 539 citations

Related papers

  1. Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine TranslationVerna Dankers, Christopher G. Lucas, Ivan TitovACL 2022
  2. CLCL: Non-compositional Expression Detection with Contrastive Learning and Curriculum LearningJianing Zhou, Ziheng Zeng, Suma BhatACL 2023 · 2 citations
  3. Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss WeightingEmmy Liu, Aditi Chaudhary, Graham NeubigEMNLP 2023 · 2 citations
  4. Non-compositional Expression Generation and its Continual LearningJianing Zhou, Suma BhatACL Findings 2024
  5. Mitigating Idiom Inconsistency: A Multi-Semantic Contrastive Learning Method for Chinese Idiom Reading ComprehensionMingmin Wu, Yuxue Hu, Yongcheng Zhang, Zhi Zeng et al.AAAI 2024 · 10 citations
Dusk over the sea between two cliffs drawn in fine vertical lines