Multilingual Relation Classification via Efficient and Effective Prompting
Yuxuan Chen, David Harbecke, Leonhard Hennig
Abstract
Prompting pre-trained language models has achieved impressive performance on various NLP tasks, especially in low data regimes. Despite the success of prompting in monolingual settings, applying prompt-based methods in multilingual scenarios has been limited to a narrow set of tasks, due to the high cost of handcrafting multilingual prompts. In this paper, we present the first work on prompt-based multilingual relation classification (RC), by introducing an efficient and effective method that constructs prompts from relation triples and involves only minimal translation for the class labels. We evaluate its performance in fully supervised, few-shot and zero-shot scenarios, and analyze its effectiveness across 14 languages, prompt variants, and English-task training in cross-lingual settings. We find that in both fully supervised and few-shot scenarios, our prompt method beats competitive baselines: fine-tuning XLM-R_EM and null prompts. It also outperforms the random baseline by a large margin in zero-shot experiments. Our method requires little in-language knowledge and can be used as a strong baseline for similar multilingual classification tasks.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext d9e14745-c117-4af9-b131-c38a27d14117Cited by top-tier papers2
- InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language UnderstandingJunda Wu, Tong Yu, Rui Wang, Zhao Song et al.NeurIPS 2023 · 48 citations
- REDFM: a Filtered and Multilingual Relation Extraction DatasetPere-Lluís Huguet Cabot, Simone Tedeschi, Axel-Cyrille Ngonga Ngomo, Roberto NavigliACL 2023 · 9 citations
Builds on16
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- Multitask Prompted Training Enables Zero-Shot Task GeneralizationVictor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach et al.ICLR 2022 · 1,976 citations
- Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order SensitivityYao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel et al.ACL 2022 · 1,494 citations
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
- Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary et al.ACL 2020 · 539 citations
Related papers
- Zero-shot Approach to Overcome Perturbation Sensitivity of PromptsMohna Chakraborty, Adithya Kulkarni, Qi LiACL 2023 · 6 citations
- TransPrompt: Towards an Automatic Transferable Prompting Framework for Few-shot Text ClassificationChengyu Wang, Jianing Wang, Minghui Qiu, Jun Huang et al.EMNLP 2021 · 39 citations
- Zero-shot Cross-lingual Transfer of Prompt-based Tuning with a Unified Multilingual PromptLianzhe Huang, Shuming Ma, Dongdong Zhang, Furu Wei et al.EMNLP 2022 · 27 citations
- Soft Language Clustering for Multilingual Model Pre-trainingJiali Zeng, Yufan Jiang, Yongjing Yin, Yi Jing et al.ACL 2023 · 1 citation
- Efficient Unseen Language Adaptation for Multilingual Pre-Trained Language ModelsPo-Heng Chen, Yun-Nung ChenEMNLP 2024
