Lune

ACL Findings2023

GNN-SL: Sequence Labeling Based on Nearest Examples via GNN

Shuhe Wang, Yuxian Meng, Rongbin Ouyang, Jiwei Li, Tianwei Zhang, Lingjuan Lyu, Guoyin Wang

Abstract

To better handle long-tail cases in the sequence labeling (SL) task, in this work, we introduce graph neural networks sequence labeling (GNN-SL), which augments the vanilla SL model output with similar tagging examples retrieved from the whole training set. Since not all the retrieved tagging examples benefit the model prediction, we construct a heterogeneous graph, and leverage graph neural networks (GNNs) to transfer information between the retrieved tagging examples and the input word sequence. The augmented node which aggregates information from neighbors is used to do prediction. This strategy enables the model to directly acquire similar tagging examples and improves the general quality of predictions. We conduct a variety of experiments on three typical sequence labeling tasks: Named Entity Recognition (NER), Part of Speech Tagging (POS), and Chinese Word Segmentation (CWS) to show the significant performance of our GNN-SL. Notably, GNN-SL achieves SOTA results of 96.9 (+0.2) on PKU, 98.3 (+0.4) on CITYU, 98.5 (+0.2) on MSR, and 96.9 (+0.2) on AS for the CWS task, and results comparable to SOTA performances on NER datasets, and POS datasets. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval cites Generalization through Memorization: Nearest Neighbor Language Models
  • Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks cites Generalization through Memorization: Nearest Neighbor Language Models
  • Retrieval Augmented Language Model Pre-Training cites Generalization through Memorization: Nearest Neighbor Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on11

  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
  2. Retrieval Augmented Language Model Pre-TrainingKelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat et al.ICML 2020 · 2,937 citations
  3. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text RetrievalLee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang et al.ICLR 2021 · 1,547 citations
  4. Generalization through Memorization: Nearest Neighbor Language ModelsUrvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer et al.ICLR 2020 · 1,038 citations
  5. A Unified MRC Framework for Named Entity RecognitionXiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han et al.ACL 2020 · 617 citations

Related papers

  1. When Generative Adversarial Networks Meet Sequence Labeling ChallengesYu Tong, Ge Chen, Guokai Zheng, Rui Li et al.EMNLP 2024 · 2 citations
  2. GNN-LM: Language Modeling based on Global Contexts via GNNYuxian Meng, Shi Zong, Xiaoya Li, Xiaofei Sun et al.ICLR 2022 · 46 citations
  3. Label-Specific Dual Graph Neural Network for Multi-Label Text ClassificationQianwen Ma, Chunyuan Yuan, Wei Zhou, Songlin HuACL 2021
  4. Lexicon Enhanced Chinese Sequence Labeling Using BERT AdapterWei Liu, Xiyan Fu, Yue Zhang, Wenming XiaoACL 2021
  5. On Size-Oriented Long-Tailed Graph Classification of Graph Neural NetworksZemin Liu, Qiheng Mao, Chenghao Liu, Yuan Fang et al.WWW 2022 · 27 citations
Dusk over the sea between two cliffs drawn in fine vertical lines