Lune

ACL Findings2026

Agentic Conversational Search with Contextualized Reasoning via Reinforcement Learning

Fengran Mo, Yifan Gao, Sha Li, Hansi Zeng, Xin Liu, Zhaoxuan Tan, Xian Li, Jianshu Chen, Dakuo Wang, Meng Jiang

Abstract

Large Language Models (LLMs) have become a popular interface for human-AI interaction, supporting information seeking and task assistance through natural, multi-turn dialogue. To respond to users within multiturn dialogues, the context-dependent user intent evolves across interactions, requiring contextual interpretation, query reformulation, and dynamic coordination between retrieval and generation. Existing studies usually follow static "rewrite, retrieve, and generate" pipelines, which optimize different procedures separately and overlook the mixed-initiative action optimization simultaneously. Although the recent developments in deep search agents demonstrate the effectiveness in jointly optimizing retrieval and generation via reasoning, these approaches focus on single-turn scenarios, which might lack the ability to handle multi-turn interactions. We introduce a conversational agent that interleaves search and reasoning across turns, enabling exploratory and adaptive behaviors learned through reinforcement learning (RL) training with tailored rewards towards evolving user goals. The experimental results across four widely used conversational benchmarks demonstrate the effectiveness of our methods by surpassing several existing strong baselines.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • RATT: A Thought Structure for Coherent and Correct LLM Reasoning cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • ChatQA: Surpassing GPT-4 on Conversational QA and RAG cites Open-Retrieval Conversational Question Answering
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on25

  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
  2. ChatQA: Surpassing GPT-4 on Conversational QA and RAGZihan Liu, Wei Ping, Rajarshi Roy, Peng Xu et al.NeurIPS 2024 · 121 citations
  3. Open-Retrieval Conversational Question AnsweringChen Qu, Liu Yang, Cen Chen, Minghui Qiu et al.SIGIR 2020 · 84 citations
  4. Large Language Models Know Your Contextual Search Intent: A Prompting Framework for Conversational SearchKelong Mao, Zhicheng Dou, Fengran Mo, Jiewen Hou et al.EMNLP Findings 2023 · 57 citations
  5. RATT: A Thought Structure for Coherent and Correct LLM ReasoningJinghan Zhang, Xiting Wang, Weijieying Ren, Lu Jiang et al.AAAI 2025 · 51 citations

Related papers

  1. R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement LearningQingfei Zhao, Ruobing Wang, Dingling Xu, Daren Zha et al.ACL Findings 2026
  2. ChatR1: Reinforcement Learning for Conversational Reasoning and Retrieval Augmented Question AnsweringSimon Lupart, Mohammad Aliannejadi, Evangelos KanoulasACL 2026 · 5 citations
  3. ReSearch: Learning to Reason with Search for LLMs via Reinforcement LearningMingyang Chen, Linzhuang Sun, Tianpeng Li, Haoze Sun et al.NeurIPS 2025 · 125 citations
  4. Learning Contextual Retrieval for Robust Conversational SearchSeunghan Yang, Juntae Lee, Jihwan Bang, Kyuhong Shim et al.EMNLP 2025 · 3 citations
  5. A Survey of Large Language Model-Based Search AgentsYunjia Xi, Jianghao Lin, Yongzhao Xiao, Zheli Zhou et al.ACL 2026 · 1,216 citations
Dusk over the sea between two cliffs drawn in fine vertical lines