Lune

ACL Findings2024

KG-Adapter: Enabling Knowledge Graph Integration in Large Language Models through Parameter-Efficient Fine-Tuning

Shiyu Tian, Yangyang Luo, Tianze Xu, Caixia Yuan, Huixing Jiang, Chen Wei, Xiaojie Wang

Abstract

Although large language models (LLMs) show remarkable capabilities and generalizability across various tasks, they are criticized for lack of expertise. One promising solution is to combine knowledge graphs (KGs) with LLMs, and recent studies focus on integrating KGs into LLMs through prompt-based methods. However, these approaches fail to use the structural information of the KGs, suffer from the problem of knowledge conflict, and over-reliance on super LLMs. To address these challenges, we propose KG-Adapter, a parameter-level KG integration method based on parameterefficient fine-tuning (PEFT). Specifically, we introduce a novel adapter structure designed for decoder-only LLMs, which can encode KGs from both node-centered and relation-centered perspectives, and then perform joint reasoning with LLMs to generate responses end-toend. Experiments with diverse models on four datasets for two different tasks all demonstrate significant improvements. With only 28M parameters trained, we make the 7B-parameter LLM outperform the previous full-parameter fine-tuned state-of-the-art method and comparable to the prompt-based ChatGPT methods.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • LoRA: Low-Rank Adaptation of Large Language Models cites Language Models are Few-Shot Learners
  • How Much Can Transfer? BRIDGE: Bounded Multi-Domain Graph Foundation Model with Generalization Guarantees cites Language Models are Few-Shot Learners
  • GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning cites How Much Can Transfer? BRIDGE: Bounded Multi-Domain Graph Foundation Model with Generalization Guarantees
  • GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning cites Language Models are Few-Shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuningHaonan Yuan, Qingyun Sun, Junhua Shi, Xingcheng Fu et al.NeurIPS 2025 · 17 citations
  2. Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and OpportunitiesChuangtao Ma, Yongrui Chen, Tianxing Wu, Arijit Khan et al.EMNLP 2025 · 6 citations
  3. MultiHal: Multilingual Dataset for Knowledge-Graph Grounded Evaluation of LLM HallucinationsErnests Lavrinovics, Russa Biswas, Katja Hose, Johannes BjervaICML 2026 · 5 citations
  4. How Much Can Transfer? BRIDGE: Bounded Multi-Domain Graph Foundation Model with Generalization GuaranteesHaonan Yuan, Qingyun Sun, Junhua Shi, Xingcheng Fu et al.ICML 2025

Builds on27

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  3. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  4. Toolformer: Language Models Can Teach Themselves to Use ToolsTimo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu et al.NeurIPS 2023 · 5,989 citations
  5. K-BERT: Enabling Language Representation with Knowledge GraphWeijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang et al.AAAI 2020 · 898 citations

Related papers

  1. LightPROF: A Lightweight Reasoning Framework for Large Language Model on Knowledge GraphTu Ao, Yanhua Yu, Yuling Wang, Yang Deng et al.AAAI 2025 · 28 citations
  2. Exploration-and-Thinking: Agentic Reasoning over Knowledge Graphs via an LLM-RL Synergized FrameworkYi Xia, Gang Zhou, Jing Chen, Xiaohui Chen et al.SIGIR 2026
  3. LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language ModelsZhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu et al.EMNLP 2023 · 200 citations
  4. Plan Then Retrieve: Reinforcement Learning-Guided Complex Reasoning over Knowledge GraphsYanlin Song, Ben Liu, Víctor Gutiérrez-Basulto, Zhiwei Hu et al.WWW 2026
  5. An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language ModelsXiongtao Zhou, Jie He, Yuhua Ke, Guangyao Zhu et al.ACL Findings 2024 · 23 citations
Dusk over the sea between two cliffs drawn in fine vertical lines