Lune

EMNLP Findings2024

Is There a One-Model-Fits-All Approach to Information Extraction? Revisiting Task Definition Biases

Wenhao Huang, Qianyu He, Zhixu Li, Jiaqing Liang, Yanghua Xiao

Abstract

Definition bias is a negative phenomenon that can mislead models. Definition bias in information extraction appears not only across datasets from different domains but also within datasets sharing the same domain. We identify two types of definition bias in IE: bias among information extraction datasets and bias between information extraction datasets and instruction tuning datasets. To systematically investigate definition bias, we conduct three probing experiments to quantitatively analyze it and discover the limitations of unified information extraction and large language models in solving definition bias. To mitigate definition bias in information extraction, we propose a multi-stage framework consisting of definition bias measurement, biasaware fine-tuning, and task-specific bias mitigation. Experimental results demonstrate the effectiveness of our framework in addressing definition bias. 1 .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction cites LoRA: Low-Rank Adaptation of Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on11

  1. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  2. Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
  3. MPNet: Masked and Permuted Pre-training for Language UnderstandingKaitao Song, Xu Tan, Tao Qin, Jianfeng Lu et al.NeurIPS 2020 · 1,957 citations
  4. OpenChat: Advancing Open-source Language Models with Mixed-Quality DataGuan Wang, Sijie Cheng, Xianyuan Zhan, Xiangang Li et al.ICLR 2024 · 328 citations
  5. GoLLIE: Annotation Guidelines improve Zero-Shot Information-ExtractionOscar Sainz, Iker García-Ferrero, Rodrigo Agerri, Oier Lopez de Lacalle et al.ICLR 2024 · 168 citations

Related papers

  1. A Joint Model for Definition Extraction with Syntactic Connection and Semantic ConsistencyAmir Pouran Ben Veyseh, Franck Dernoncourt, Dejing Dou, Thien Huu NguyenAAAI 2020 · 43 citations
  2. ADELIE: Aligning Large Language Models on Information ExtractionYunjia Qi, Hao Peng, Xiaozhi Wang, Bin Xu et al.EMNLP 2024 · 8 citations
  3. Towards Debiasing NLU Models from Unknown BiasesPrasetya Ajie Utama, Nafise Sadat Moosavi, Iryna GurevychEMNLP 2020 · 3 citations
  4. Universal Information Extraction with Meta-Pretrained Self-RetrievalXin Cong, Bowen Yu, Mengcheng Fang, Tingwen Liu et al.ACL Findings 2023 · 5 citations
  5. Can We Debias Multimodal Large Language Models via Model Editing?Zecheng Wang, Xinye Li, Zhanyue Qin, Chunshan Li et al.ACM MM 2024 · 2 citations
Dusk over the sea between two cliffs drawn in fine vertical lines