ACL Findings2023
AttenWalker: Unsupervised Long-Document Question Answering via Attention-based Graph Walking
Yuxiang Nie, Heyan Huang, Wei Wei, Xian-Ling Mao
Abstract
Annotating long-document question answering (long-document QA) pairs is time-consuming and expensive. To alleviate the problem, it might be possible to generate long-document QA pairs via unsupervised question answering (UQA) methods. However, existing UQA tasks are based on short documents, and can hardly incorporate long-range information. To tackle the problem, we propose a new task, named unsupervised long-document question answering (ULQA), aiming to generate highquality long-document QA instances in an unsupervised manner. Besides, we propose Atten-Walker, a novel unsupervised method to aggregate and generate answers with long-range dependency so as to construct long-document QA pairs. Specifically, AttenWalker is composed of three modules, i.e., span collector, span linker and answer aggregator. Firstly, the span collector takes advantage of constituent parsing and reconstruction loss to select informative candidate spans for constructing answers. Secondly, by going through the attention graph of a pre-trained long-document model, potentially interrelated text spans (that might be far apart) could be linked together via an attentionwalking algorithm. Thirdly, in the answer aggregator, linked spans are aggregated into the final answer via the mask-filling ability of a pre-trained model. Extensive experiments show that AttenWalker outperforms previous methods on Qasper and NarrativeQA. In addition, AttenWalker also shows strong performance in the few-shot learning setting. 1 1 We have released our codes and data in https://github. com/JerrryNie/Unsupervised-Long-Document-QA.
2 Usually, the term 'short-document QA' is simplified as 'QA' in the literature, which refers to the QA task with a short Today's increasing flood of information on the web creates a need for ...
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on2
- Harvesting and Refining Question-Answer Pairs for Unsupervised QAZhongli Li, Wenhui Wang, Li Dong, Furu Wei et al.ACL 2020 · 29 citations
- Capturing Global Structural Information in Long Document Question Answering with Compressive Graph Selector NetworkYuxiang Nie, Heyan Huang, Wei Wei, Xian-Ling MaoEMNLP 2022 · 11 citations
Related papers
- Regularization of Distinct Strategies for Unsupervised Question GenerationJunmo Kang, Giwon Hong, Haritz Puerto San Roman, Sung-Hyon MyaengEMNLP Findings 2020 · 3 citations
- RikiNet: Reading Wikipedia Pages for Natural Question AnsweringDayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan et al.ACL 2020 · 55 citations
- Improving Retrieval Augmented Open-Domain Question-Answering with Vectorized ContextsZhuo Chen, Xinyu Wang, Yong Jiang, Pengjun Xie et al.ACL Findings 2024 · 8 citations
- UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering Over Knowledge GraphJinhao Jiang, Kun Zhou, Xin Zhao, Ji-Rong WenICLR 2023 · 28 citations
- WebWalker: Benchmarking LLMs in Web TraversalJialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang et al.ACL 2025
