EMNLP Findings2023
Unsupervised Lexical Simplification with Context Augmentation
Takashi Wada, Timothy Baldwin, Jey Han Lau
Abstract
We propose a new unsupervised lexical simplification method that uses only monolingual data and pre-trained language models. Given a target word and its context, our method generates substitutes based on the target context and also additional contexts sampled from monolingual data. We conduct experiments in English, Portuguese, and Spanish on the TSAR-2022 shared task, and show that our model substantially outperforms other unsupervised systems across all languages. We also establish a new state-of-the-art by ensembling our model with GPT-3.5. Lastly, we evaluate our model on the SWORDS lexical substitution data set, achieving a state-of-the-art result. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing cites Language Models are Few-Shot Learners
- Training language models to follow instructions with human feedback cites Language Models are Few-Shot Learners
Builds on7
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding SharingPengcheng He, Jianfeng Gao, Weizhu ChenICLR 2023 · 394 citations
- Lexical Simplification with Pretrained EncodersJipeng Qiang, Yun Li, Yi Zhu, Yunhao Yuan et al.AAAI 2020 · 86 citations
- BLEURT: Learning Robust Metrics for Text GenerationThibault Sellam, Dipanjan Das, Ankur P. ParikhACL 2020 · 40 citations
Related papers
- Teaching the Pre-trained Model to Generate Simple Texts for Text SimplificationRenliang Sun, Wei Xu, Xiaojun WanACL Findings 2023 · 14 citations
- Unsupervised Paraphrasing of Multiword ExpressionsTakashi Wada, Yuji Matsumoto, Timothy Baldwin, Jey Han LauACL Findings 2023 · 1 citation
- GeneSis: A Generative Approach to Substitutes in ContextCaterina Lacerra, Rocco Tripodi, Roberto NavigliEMNLP 2021
- Enhancing Sentence Simplification in Portuguese: Leveraging Paraphrases, Context, and Linguistic FeaturesArthur Scalercio, Maria José Finatto, Aline PaesACL Findings 2024 · 3 citations
- BLESS: Benchmarking Large Language Models on Sentence SimplificationTannon Kew, Alison Chi, Laura Vásquez-Rodríguez, Sweta Agrawal et al.EMNLP 2023 · 15 citations
