Lune

EMNLP Findings2022

Adapters for Enhanced Modeling of Multilingual Knowledge and Text

Yifan Hou, Wenxiang Jiao, Meizhen Liu, Carl Allen, Zhaopeng Tu, Mrinmaya Sachan

Abstract

Large language models appear to learn facts from the large text corpora they are trained on. Such facts are encoded implicitly within their many parameters, making it difficult to verify or manipulate what knowledge has been learned. Language models have recently been extended to multilingual language models (MLLMs), enabling knowledge to be learned across hundreds of languages. Meanwhile, knowledge graphs contain facts in an explicit triple format, which require careful and costly curation and are only available in a few highresource languages, restricting their research and application. To address these issues, we propose to enhance MLLMs with knowledge from multilingual knowledge graphs (MLKGs) so as to tackle language and knowledge graph tasks across many languages, including lowresource ones. Specifically, we introduce a lightweight adapter set to enhance MLLMs with cross-lingual entity alignment and facts from MLKGs for many languages. Experiments on common benchmarks show that such enhancement benefits both MLLMs and MLKGs, achieving: (1) comparable or improved performance for knowledge graph completion and entity alignment relative to baselines, especially for low-resource languages (for which knowledge graphs are unavailable); and (2) improved MLLM performance on language understanding tasks that require multilingual factual knowledge; all while maintaining performance on other general language tasks. 1 1 Our code, models, and data (e.g., integration corpus and extended datasets) are available at https://github.com/yifan- h/Multilingual_Space.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • On Learning Universal Representations Across Languages cites Unsupervised Cross-lingual Representation Learning at Scale
  • Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models cites Language Models are Few-Shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Cross-Lingual Consistency of Factual Knowledge in Multilingual Language ModelsJirui Qi, Raquel Fernández, Arianna BisazzaEMNLP 2023 · 9 citations

Builds on10

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. K-BERT: Enabling Language Representation with Knowledge GraphWeijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang et al.AAAI 2020 · 898 citations
  3. Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary et al.ACL 2020 · 539 citations
  4. K-Adapter: Infusing Knowledge into Pre-Trained Models with AdaptersRuize Wang, Duyu Tang, Nan Duan, Zhongyu Wei et al.ACL Findings 2021 · 473 citations
  5. On Learning Universal Representations Across LanguagesXiangpeng Wei, Rongxiang Weng, Yue Hu, Luxi Xing et al.ICLR 2021 · 93 citations

Related papers

  1. Multi-Modal Fact Knowledge Generation for Imbalanced Cross-Source Entity AlignmentQian Li, Cheng Ji, Zhaoji Liang, Yuzheng Zhang et al.AAAI 2026
  2. Enhancing Multilingual Language Model with Massive Multilingual Knowledge TriplesLinlin Liu, Xin Li, Ruidan He, Lidong Bing et al.EMNLP 2022 · 15 citations
  3. Multilingual Knowledge Graph Completion via Efficient Multilingual Knowledge SharingCunli Mao, Xiaofei Gao, Ran Song, Shizhu He et al.EMNLP Findings 2025 · 1 citation
  4. Prix-LM: Pretraining for Multilingual Knowledge Base ConstructionWenxuan Zhou, Fangyu Liu, Ivan Vulic, Nigel Collier et al.ACL 2022 · 21 citations
  5. MultiHal: Multilingual Dataset for Knowledge-Graph Grounded Evaluation of LLM HallucinationsErnests Lavrinovics, Russa Biswas, Katja Hose, Johannes BjervaICML 2026 · 5 citations
Dusk over the sea between two cliffs drawn in fine vertical lines