Lune

EMNLP Findings2024

OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMs

Jintian Zhang, Cheng Peng, Mengshu Sun, Xiang Chen, Lei Liang, Zhiqiang Zhang, Jun Zhou, Huajun Chen, Ningyu Zhang

Abstract

Despite the recent advancements in Large Language Models (LLMs), which have significantly enhanced the generative capabilities for various NLP tasks, LLMs still face limitations in directly handling retrieval tasks. However, many practical applications demand the seamless integration of both retrieval and generation. This paper introduces a novel and efficient One-pass Generation and retrieval framework (OneGen), designed to improve LLMs' performance on tasks that require both generation and retrieval. The proposed framework bridges the traditionally separate training approaches for generation and retrieval by incorporating retrieval tokens generated autoregressively. This enables a single LLM to handle both tasks simultaneously in a unified forward pass. We conduct experiments on two distinct types of composite tasks, RAG and Entity Linking, to validate the pluggability, effectiveness, and efficiency of OneGen in training and inference. Furthermore, our results show that integrating generation and retrieval within the same context preserves the generative capabilities of LLMs while improving retrieval performance. To the best of our knowledge, OneGen is the first to enable LLMs to conduct vector retrieval during the generation.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

5 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • LoRA: Low-Rank Adaptation of Large Language Models cites Language Models are Few-Shot Learners
  • MAGNET: Augmenting Generative Decoders with Representation Learning and Infilling Capabilities cites LoRA: Low-Rank Adaptation of Large Language Models
  • Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answering cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Language Models are Few-Shot Learners cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations cites LoRA: Low-Rank Adaptation of Large Language Models
  • UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generation cites LoRA: Low-Rank Adaptation of Large Language Models
  • RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generation cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers5

  1. UniConv: Unifying Retrieval and Response Generation for Large Language Models in ConversationsFengran Mo, Yifan Gao, Chuan Meng, Xin Liu et al.ACL 2025 · 22 citations
  2. Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question AnsweringRongzhi Zhu, Xiangyu Liu, Zequn Sun, Yiwei Wang et al.ACL 2025 · 14 citations
  3. Learning Contextual Retrieval for Robust Conversational SearchSeunghan Yang, Juntae Lee, Jihwan Bang, Kyuhong Shim et al.EMNLP 2025 · 3 citations
  4. MAGNET: Augmenting Generative Decoders with Representation Learning and Infilling CapabilitiesSavya Khosla, Aditi Tiwari, Kushal Kafle, Simon Jenni et al.ACL 2025
  5. RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within GenerationXiaoxi Li, Jiajie Jin, Yujia Zhou, Yongkang Wu et al.ACL 2025

Builds on24

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  3. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
  4. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  5. Toolformer: Language Models Can Teach Themselves to Use ToolsTimo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu et al.NeurIPS 2023 · 5,989 citations

Related papers

  1. Bridging the Preference Gap between Retrievers and LLMsZixuan Ke, Weize Kong, Cheng Li, Mingyang Zhang et al.ACL 2024 · 8 citations
  2. ImpRAG: Retrieval-Augmented Generation with Implicit QueriesWenzheng Zhang, Xi Victoria Lin, Karl Stratos, Wen-tau Yih et al.EMNLP Findings 2025
  3. UniGen: A Unified Generative Framework for Retrieval and Question Answering with Large Language ModelsXiaoxi Li, Yujia Zhou, Zhicheng DouAAAI 2024 · 24 citations
  4. A + B: A General Generator-Reader Framework for Optimizing LLMs to Unleash Synergy PotentialWei Tang, Yixin Cao, Jiahao Ying, Bo Wang et al.ACL Findings 2024 · 3 citations
  5. One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language ModelsYutao Zhu, Zhaoheng Huang, Zhicheng Dou, Ji-Rong WenAAAI 2025 · 9 citations
Dusk over the sea between two cliffs drawn in fine vertical lines