EMNLP Findings2021
Cross-lingual Transfer for Text Classification with Dictionary-based Heterogeneous Graph
Nuttapong Chairatanakul, Noppayut Sriwatanasakdi, Nontawat Charoenphakdee, Xin Liu, Tsuyoshi Murata
Abstract
In cross-lingual text classification, it is required that task-specific training data in high-resource source languages are available, where the task is identical to that of a low-resource target language. However, collecting such training data can be infeasible because of the labeling cost, task characteristics, and privacy concerns. This paper proposes an alternative solution that uses only task-independent word embeddings of high-resource languages and bilingual dictionaries. First, we construct a dictionary-based heterogeneous graph (DHG) from bilingual dictionaries. This opens the possibility to use graph neural networks for crosslingual transfer. The remaining challenge is the heterogeneity of DHG because multiple languages are considered. To address this challenge, we propose dictionary-based heterogeneous graph neural network (DHGNet) that effectively handles the heterogeneity of DHG by two-step aggregations, which are word-level and language-level aggregations. Experimental results demonstrate that our method outperforms pretrained models even though it does not access to large corpora. Furthermore, it can perform well even though dictionaries contain many incorrect translations. Its robustness allows the usage of a wider range of dictionaries such as an automatically constructed dictionary and crowdsourced dictionary, which are convenient for real-world applications.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers1
Builds on10
- Simple and Deep Graph Convolutional NetworksMing Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding et al.ICML 2020 · 1,910 citations
- Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain AdaptationJian Liang, Dapeng Hu, Jiashi FengICML 2020 · 1,624 citations
- Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary et al.ACL 2020 · 539 citations
- iNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian LanguagesDivyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, Gokul N. C. et al.EMNLP Findings 2020 · 465 citations
- A Mathematical Exploration of Why Language Models Help Solve Downstream TasksNikunj Saunshi, Sadhika Malladi, Sanjeev AroraICLR 2021 · 93 citations
Related papers
- Graph-Based Multilingual Label Propagation for Low-Resource Part-of-Speech TaggingAyyoob Imani, Silvia Severini, Masoud Jalili Sabet, François Yvon et al.EMNLP 2022 · 8 citations
- Harnessing Language Model for Cross-Heterogeneity Graph Knowledge TransferJinyu Yang, Ruijia Wang, Cheng Yang, Bo Yan et al.AAAI 2025 · 4 citations
- Beyond Offline Mapping: Learning Cross-lingual Word Embeddings through Context AnchoringAitor Ormazabal, Mikel Artetxe, Aitor Soroa, Gorka Labaka et al.ACL 2021
- A Graph-based Coarse-to-fine Method for Unsupervised Bilingual Lexicon InductionShuo Ren, Shujie Liu, Ming Zhou, Shuai MaACL 2020 · 13 citations
- Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine TranslationDi Wu, Christof MonzEMNLP 2023 · 5 citations
