Hybrid Knowledge Transfer for Improved Cross-Lingual Event Detection via Hierarchical Sample Selection
Luis Guzman-Nateras, Franck Dernoncourt, Thien Huu Nguyen
Abstract
In this paper, we address the Event Detection task under a zero-shot cross-lingual setting where a model is trained on a source language but evaluated on a distinct target language for which there is no labeled data available. Most recent efforts in this field follow a direct transfer approach in which the model is trained using language-invariant features and then directly applied to the target language. However, we argue that these methods fail to take advantage of the benefits of the data transfer approach where a cross-lingual model is trained on target-language data and is able to learn task-specific information from syntactical features or word-label relations in the target language. As such, we propose a hybrid knowledge-transfer approach that leverages a teacher-student framework where the teacher and student networks are trained following the direct and data transfer approaches, respectively. Our method is complemented by a hierarchical training-sample selection scheme designed to address the issue of noisy labels being generated by the teacher model. Our model achieves state-of-the-art results on 9 morphologically-diverse target languages across 3 distinct datasets, highlighting the importance of exploiting the benefits of hybrid transfer.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext ad8c8cc1-77df-439b-9275-8e87a5119281Cited by top-tier papers1
Ask how each one uses itBuilds on11
- Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary et al.ACL 2020 · 539 citations
- Acquiring Knowledge from Pre-Trained Model to Neural Machine TranslationRongxiang Weng, Heng Yu, Shujian Huang, Shanbo Cheng et al.AAAI 2020 · 71 citations
- Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target LanguageQianhui Wu, Zijia Lin, Börje Karlsson, Jianguang Lou et al.ACL 2020 · 59 citations
- Towards Oracle Knowledge Distillation with Neural Architecture SearchMinsoo Kang, Jonghwan Mun, Bohyung HanAAAI 2020 · 48 citations
- Language Model Priming for Cross-Lingual Event ExtractionSteven Fincke, Shantanu Agarwal, Scott Miller, Elizabeth BoscheeAAAI 2022 · 31 citations
Related papers
- Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument ExtractionKuan-Hao Huang, I-Hung Hsu, Prem Natarajan, Kai-Wei Chang et al.ACL 2022
- Wider & Closer: Mixture of Short-channel Distillers for Zero-shot Cross-lingual Named Entity RecognitionJun-Yu Ma, Beiduo Chen, Jia-Chen Gu, Zhenhua Ling et al.EMNLP 2022 · 3 citations
- Discrepancy and Uncertainty Aware Denoising Knowledge Distillation for Zero-Shot Cross-Lingual Named Entity RecognitionLing Ge, Chunming Hu, Guanghui Ma, Jihong Liu et al.AAAI 2024 · 9 citations
- Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information ExtractionMahsa Yarmohammadi, Shijie Wu, Marc Marone, Haoran Xu et al.EMNLP 2021
- On the Importance of Word Order Information in Cross-lingual Sequence LabelingZihan Liu, Genta Indra Winata, Samuel Cahyawijaya, Andrea Madotto et al.AAAI 2021 · 29 citations
