Lune

EMNLP Findings2025

ImpRAG: Retrieval-Augmented Generation with Implicit Queries

Wenzheng Zhang, Xi Victoria Lin, Karl Stratos, Wen-tau Yih, Mingda Chen

Abstract

Retrieval-Augmented Generation (RAG) systems traditionally treat retrieval and generation as separate processes, requiring explicit textual queries to connect them. This separation can limit the ability of models to generalize across diverse tasks. In this work, we propose a queryfree RAG system, named ImpRAG, which integrates retrieval and generation into a unified model. ImpRAG allows models to implicitly express their information needs, eliminating the need for human-specified queries. By dividing pretrained decoder-only language models into specialized layer groups, ImpRAG optimizes retrieval and generation tasks simultaneously. Our approach employs a two-stage inference process, using the same model parameters and forward pass for both retrieval and generation, thereby minimizing the disparity between retrievers and language models. Experiments on 8 knowledge-intensive tasks demonstrate that ImpRAG significantly enhances both retrieval and generation performance, with exact match scores increasing by 3.6-11.5 points and retrieval recalls improving by 5.0-23.2 points for unseen tasks with diverse formats, highlighting its effectiveness in enabling models to articulate their own information needs and generalize across tasks. Our analysis underscores the importance of balancing retrieval and generation parameters and leveraging generation perplexities as retrieval training objectives for enhanced performance.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Measuring and Narrowing the Compositionality Gap in Language Models cites Improving Language Models by Retrieving from Trillions of Tokens
  • Measuring and Narrowing the Compositionality Gap in Language Models cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • RAGPerf: An End-to-End Benchmarking Framework for Retrieval-Augmented Generation Systems cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Improving Language Models by Retrieving from Trillions of Tokens cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Improving Language Models by Retrieving from Trillions of Tokens cites Retrieval Augmented Language Model Pre-Training
  • Active Retrieval Augmented Generation cites Measuring and Narrowing the Compositionality Gap in Language Models
  • Active Retrieval Augmented Generation cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. RAGPerf: An End-to-End Benchmarking Framework for Retrieval-Augmented Generation SystemsShaobo Li, Yirui Zhou, Yuan Xu, Kevin Chen et al.VLDB 2026 · 3 citations

Builds on21

  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
  2. Retrieval Augmented Language Model Pre-TrainingKelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat et al.ICML 2020 · 2,937 citations
  3. Improving Language Models by Retrieving from Trillions of TokensSebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai et al.ICML 2022 · 1,629 citations
  4. Active Retrieval Augmented GenerationZhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun et al.EMNLP 2023 · 315 citations
  5. Measuring and Narrowing the Compositionality Gap in Language ModelsOfir Press, Muru Zhang, Sewon Min, Ludwig Schmidt et al.EMNLP Findings 2023 · 296 citations

Related papers

  1. UniRAG: Unified Query Understanding Method for Retrieval Augmented GenerationRui Li, Liyang He, Qi Liu, Zheng Zhang et al.ACL 2025
  2. Decoupling Knowledge and Context: An Efficient and Effective Retrieval Augmented Generation Framework via Cross AttentionQian Dong, Qingyao Ai, Hongning Wang, Yiding Liu et al.WWW 2025 · 19 citations
  3. Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented GenerationShicheng Xu, Liang Pang, Mo Yu, Fandong Meng et al.ACL 2024
  4. From RAG to Riches: Retrieval Interlaced with Sequence GenerationPalak Jain, Livio Baldini Soares, Tom KwiatkowskiEMNLP 2024 · 2 citations
  5. Embedding-Free RAGJessica Maghakian, Raunak Sinha, Max Schettewi, Gunkirat KaurEMNLP Findings 2025 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines