Lune

EMNLP Findings2021

Cross-lingual Transfer for Text Classification with Dictionary-based Heterogeneous Graph

Nuttapong Chairatanakul, Noppayut Sriwatanasakdi, Nontawat Charoenphakdee, Xin Liu, Tsuyoshi Murata

Abstract

In cross-lingual text classification, it is required that task-specific training data in high-resource source languages are available, where the task is identical to that of a low-resource target language. However, collecting such training data can be infeasible because of the labeling cost, task characteristics, and privacy concerns. This paper proposes an alternative solution that uses only task-independent word embeddings of high-resource languages and bilingual dictionaries. First, we construct a dictionary-based heterogeneous graph (DHG) from bilingual dictionaries. This opens the possibility to use graph neural networks for crosslingual transfer. The remaining challenge is the heterogeneity of DHG because multiple languages are considered. To address this challenge, we propose dictionary-based heterogeneous graph neural network (DHGNet) that effectively handles the heterogeneity of DHG by two-step aggregations, which are word-level and language-level aggregations. Experimental results demonstrate that our method outperforms pretrained models even though it does not access to large corpora. Furthermore, it can perform well even though dictionaries contain many incorrect translations. Its robustness allows the usage of a wider range of dictionaries such as an automatically constructed dictionary and crowdsourced dictionary, which are convenient for real-world applications.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • iNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages cites Unsupervised Cross-lingual Representation Learning at Scale
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Latent Constraints on Unsupervised Text-Graph Alignment with Information AsymmetryJidong Tian, Wenqing Chen, Yitian Li, Caoyun Fan et al.AAAI 2023

Builds on10

  1. Simple and Deep Graph Convolutional NetworksMing Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding et al.ICML 2020 · 1,910 citations
  2. Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain AdaptationJian Liang, Dapeng Hu, Jiashi FengICML 2020 · 1,624 citations
  3. Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary et al.ACL 2020 · 539 citations
  4. iNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian LanguagesDivyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, Gokul N. C. et al.EMNLP Findings 2020 · 465 citations
  5. A Mathematical Exploration of Why Language Models Help Solve Downstream TasksNikunj Saunshi, Sadhika Malladi, Sanjeev AroraICLR 2021 · 93 citations

Related papers

  1. Graph-Based Multilingual Label Propagation for Low-Resource Part-of-Speech TaggingAyyoob Imani, Silvia Severini, Masoud Jalili Sabet, François Yvon et al.EMNLP 2022 · 8 citations
  2. Harnessing Language Model for Cross-Heterogeneity Graph Knowledge TransferJinyu Yang, Ruijia Wang, Cheng Yang, Bo Yan et al.AAAI 2025 · 4 citations
  3. Beyond Offline Mapping: Learning Cross-lingual Word Embeddings through Context AnchoringAitor Ormazabal, Mikel Artetxe, Aitor Soroa, Gorka Labaka et al.ACL 2021
  4. A Graph-based Coarse-to-fine Method for Unsupervised Bilingual Lexicon InductionShuo Ren, Shujie Liu, Ming Zhou, Shuai MaACL 2020 · 13 citations
  5. Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine TranslationDi Wu, Christof MonzEMNLP 2023 · 5 citations
Dusk over the sea between two cliffs drawn in fine vertical lines