ACL Findings2026
Bridging Reasoning and Action: Hybrid LLM-RL Framework for Efficient Cross-Domain Task-Oriented Dialogue
Yangyang Zhao, Linfan Dai, Li Cai, Bowen Xing, Libo Qin
Abstract
Cross-domain task-oriented dialogue requires reasoning over implicit and explicit feasibility constraints while planning long-horizon, multi-turn actions. Large language models (LLMs) can infer such constraints but are unreliable over long horizons, while Reinforcement learning (RL) optimizes long-horizon behavior yet cannot recover constraints from raw dialogue. Naively coupling LLMs with RL is therefore brittle: unverified or unstructured LLM outputs can corrupt state representations and misguide policy learning. Motivated by this, we propose Verified LLM-Knowledge empowered RL (VLK-RL), a hybrid framework that makes LLM-derived constraint reasoning usable for RL. VLK-RL first elicits candidate constraints with an LLM and then verifies them via a dual-role crossexamination procedure to suppress hallucinations and cross-turn inconsistencies. The verified constraints are mapped into ontologyaligned slot-value representations, yielding a structured, constraint-aware state for RL policy optimization. Experiments across multiple benchmarks demonstrate that VLK-RL significantly improves generalization and robustness, outperforming strong single-model baselines on long-horizon tasks.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions cites GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-supervised Learning and Explicit Policy Injection
- Zero-shot Cross-domain Dialogue State Tracking via Context-aware Auto-prompting and Instruction-following Contrastive Decoding cites Towards LLM-driven Dialogue State Tracking
Builds on7
- GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-supervised Learning and Explicit Policy InjectionWanwei He, Yinpei Dai, Yinhe Zheng, Yuchuan Wu et al.AAAI 2022 · 181 citations
- LM vs LM: Detecting Factual Errors via Cross ExaminationRoi Cohen, May Hamri, Mor Geva, Amir GlobersonEMNLP 2023 · 41 citations
- Towards LLM-driven Dialogue State TrackingYujie Feng, Zexin Lu, Bo Liu, Liming Zhan et al.EMNLP 2023 · 25 citations
- End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future DirectionsLibo Qin, Wenbo Pan, Qiguang Chen, Lizi Liao et al.EMNLP 2023 · 12 citations
- Zero-shot Cross-domain Dialogue State Tracking via Context-aware Auto-prompting and Instruction-following Contrastive DecodingXiaoyu Dong, Yujie Feng, Zexin Lu, Guangyuan Shi et al.EMNLP 2024 · 3 citations
Related papers
- KnowRL: Exploring Knowledgeable Reinforcement Learning for FactualityBaochang Ren, Shuofei Qiao, Ningyu Zhang, Da Zheng et al.ACL 2026 · 12 citations
- OntoGuard: Enforcing Action Admissibility for LLM Agents in Complex Interactive EnvironmentsPengxiang Liu, Tao Ren, Wei Xiong, Tingrui Yang et al.ACL Findings 2026 · 1 citation
- CARL: Constraint-Aware Reinforcement Learning for Planning with LLMsQiuyi Qi, Jinjian Zhang, Mutian Bao, Tian Liang et al.ACL Findings 2026
- PlanLLM: Video Procedure Planning with Refinable Large Language ModelsDejie Yang, Zijing Zhao, Yang LiuAAAI 2025 · 8 citations
- Theory of Mind for Multi-Agent Collaboration via Large Language ModelsHuao Li, Yu Quan Chong, Simon Stepputtis, Joseph Campbell et al.EMNLP 2023 · 57 citations
