ACL Findings2025
Optimizing Multi-Hop Document Retrieval Through Intermediate Representations
Linjiaen Linjiaen, Jingyu Liu, Yingbo Liu
Abstract
Retrieval-augmented generation (RAG) encounters challenges when addressing complex queries, particularly multi-hop questions. While several methods tackle multi-hop queries by iteratively generating internal queries and retrieving external documents, these approaches are computationally expensive. In this paper, we identify a three-stage information processing pattern in LLMs during layer-by-layer reasoning, consisting of extraction, processing, and subsequent extraction steps. This observation suggests that the representations in intermediate layers contain richer information compared to those in other layers. Building on this insight, we propose Layer-wise RAG (L-RAG). Unlike prior methods that focus on generating new internal queries, L-RAG leverages intermediate representations from the middle layers, which capture next-hop information, to retrieve external knowledge. L-RAG achieves performance comparable to multi-step approaches while maintaining inference overhead similar to that of standard RAG. Experimental results show that L-RAG outperforms existing RAG methods on open-domain multihop question-answering datasets, including MuSiQue, HotpotQA, and 2WikiMultiHopQA. The code is available in https://anonymous. 4open.science/r/L-RAG-ADD5/.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Measuring and Narrowing the Compositionality Gap in Language Models cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Do not Abstain! Identify and Solve the Uncertainty cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Active Retrieval Augmented Generation cites Measuring and Narrowing the Compositionality Gap in Language Models
- Active Retrieval Augmented Generation cites Precise Zero-Shot Dense Retrieval without Relevance Labels
- Active Retrieval Augmented Generation cites Measuring Massive Multitask Language Understanding
- Active Retrieval Augmented Generation cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Cited by top-tier papers1
Builds on13
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
- Active Retrieval Augmented GenerationZhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun et al.EMNLP 2023 · 315 citations
- Measuring and Narrowing the Compositionality Gap in Language ModelsOfir Press, Muru Zhang, Sewon Min, Ludwig Schmidt et al.EMNLP Findings 2023 · 296 citations
- Precise Zero-Shot Dense Retrieval without Relevance LabelsLuyu Gao, Xueguang Ma, Jimmy Lin, Jamie CallanACL 2023 · 211 citations
Related papers
- CompactRAG: Reducing LLM Calls and Token Overhead in Multi-Hop Question AnsweringHao Yang, Zhiyu Yang, Xupeng Zhang, Wei Wei et al.WWW 2026
- PropRAG: Guiding Retrieval with Beam Search over Proposition PathsJingjin Wang, Jiawei HanEMNLP 2025
- Cooperative Retrieval-Augmented Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting LayersYoumin Ko, Sungjong Seo, Hyunjoon KimNeurIPS 2025 · 2 citations
- LiR3AG: A Lightweight Rerank Reasoning Strategy Framework for Retrieval-Augmented GenerationGuo Chen, Junjie Huang, Huaijin Xie, Fei Sun et al.AAAI 2026
- Iterative Multi-Granular RAG with Contextual Hierarchical GraphYanli Hu, Teng Liu, Zhuangyi Zhou, Weixin Zeng et al.AAAI 2026
