Capability Decomposition for Unified Information Extraction via Hierarchical Mixture-of-Experts
Jing Zhou, Peng Wang, Wenjun Ke, Jiajun Liu, Yao He
摘要
Unified Information Extraction (UIE) aims to handle heterogeneous IE tasks within a single framework, but existing methods often suffer from inconsistent schema representation, implicitly intermediate reasoning and fullparameter adaptation, which limit generalization, interpretability and parameter efficiency. To address these issues, we propose UC-UIE (Universal Capabilities-based Unified Information Extractor), a unified framework based on Large Language Model (LLM), which introduces a unified frame-and-slots schema for IE tasks and explicitly decomposes IE reasoning into three universal capabilities: judging, locating, and associating. Furthermore, UC-UIE adopts a Low-Rank Adaptation (LoRA) based hierarchical Mixture-of-Experts (MoE) adapter to fine-tune LLMs for IE tasks, which explicitly models these three capabilities in a task-driven way while ensuring parameter efficiency. With only 1.24% trainable parameters, UC-UIE outperforms full-parameter tuning methods, showing excellent parameter efficiency. Zero-shot evaluation reveals its strong generalization ability to unseen domains and schemas, benefiting from unified schema representation and explicit capability decomposition. Further experiments validate that the hierarchical MoE adapter learns capability specialization and composition, which enhances both UIE performance and interpretability.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper23
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu 等ICLR 2022 · 被引用 18,833 次
- QLoRA: Efficient Finetuning of Quantized LLMsTim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke ZettlemoyerNeurIPS 2023 · 被引用 5,863 次
- GShard: Scaling Giant Models with Conditional Computation and Automatic ShardingDmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen 等ICLR 2021 · 被引用 1,954 次
- CrossNER: Evaluating Cross-Domain Named Entity RecognitionZihan Liu, Yan Xu, Tiezheng Yu, Wenliang Dai 等AAAI 2021 · 被引用 201 次
- Mixture of LoRA ExpertsXun Wu, Shaohan Huang, Furu WeiICLR 2024 · 被引用 174 次
相关 Paper
- Unified Structure Generation for Universal Information ExtractionYaojie Lu, Qing Liu, Dai Dai, Xinyan Xiao 等ACL 2022
- MeteoRA: Multiple-tasks Embedded LoRA for Large Language ModelsJingwei Xu, Junyu Lai, Yunpeng HuangICLR 2025
- UniEX: An Effective and Efficient Framework for Unified Information Extraction via a Span-extractive PerspectiveYang Ping, Junyu Lu, Ruyi Gan, Junjie Wang 等ACL 2023 · 被引用 4 次
- Each Rank Could be an Expert: Single-Ranked Mixture of Experts LoRA for Multi-task LearningZiyu Zhao, Yixiao Zhou, Xin Yu, Zhi Zhang 等KDD 2026 · 被引用 13 次
- KnowCoder: Coding Structured Knowledge into LLMs for Universal Information ExtractionZixuan Li, Yutao Zeng, Yuxin Zuo, Weicheng Ren 等ACL 2024 · 被引用 19 次
