Lune

ACL Findings2026

Bridging Reasoning and Action: Hybrid LLM-RL Framework for Efficient Cross-Domain Task-Oriented Dialogue

Yangyang Zhao, Linfan Dai, Li Cai, Bowen Xing, Libo Qin

Abstract

Cross-domain task-oriented dialogue requires reasoning over implicit and explicit feasibility constraints while planning long-horizon, multi-turn actions. Large language models (LLMs) can infer such constraints but are unreliable over long horizons, while Reinforcement learning (RL) optimizes long-horizon behavior yet cannot recover constraints from raw dialogue. Naively coupling LLMs with RL is therefore brittle: unverified or unstructured LLM outputs can corrupt state representations and misguide policy learning. Motivated by this, we propose Verified LLM-Knowledge empowered RL (VLK-RL), a hybrid framework that makes LLM-derived constraint reasoning usable for RL. VLK-RL first elicits candidate constraints with an LLM and then verifies them via a dual-role crossexamination procedure to suppress hallucinations and cross-turn inconsistencies. The verified constraints are mapped into ontologyaligned slot-value representations, yielding a structured, constraint-aware state for RL policy optimization. Experiments across multiple benchmarks demonstrate that VLK-RL significantly improves generalization and robustness, outperforming strong single-model baselines on long-horizon tasks.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions cites GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-supervised Learning and Explicit Policy Injection
  • Zero-shot Cross-domain Dialogue State Tracking via Context-aware Auto-prompting and Instruction-following Contrastive Decoding cites Towards LLM-driven Dialogue State Tracking
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on7

  1. GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-supervised Learning and Explicit Policy InjectionWanwei He, Yinpei Dai, Yinhe Zheng, Yuchuan Wu et al.AAAI 2022 · 181 citations
  2. LM vs LM: Detecting Factual Errors via Cross ExaminationRoi Cohen, May Hamri, Mor Geva, Amir GlobersonEMNLP 2023 · 41 citations
  3. Towards LLM-driven Dialogue State TrackingYujie Feng, Zexin Lu, Bo Liu, Liming Zhan et al.EMNLP 2023 · 25 citations
  4. End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future DirectionsLibo Qin, Wenbo Pan, Qiguang Chen, Lizi Liao et al.EMNLP 2023 · 12 citations
  5. Zero-shot Cross-domain Dialogue State Tracking via Context-aware Auto-prompting and Instruction-following Contrastive DecodingXiaoyu Dong, Yujie Feng, Zexin Lu, Guangyuan Shi et al.EMNLP 2024 · 3 citations

Related papers

  1. KnowRL: Exploring Knowledgeable Reinforcement Learning for FactualityBaochang Ren, Shuofei Qiao, Ningyu Zhang, Da Zheng et al.ACL 2026 · 12 citations
  2. OntoGuard: Enforcing Action Admissibility for LLM Agents in Complex Interactive EnvironmentsPengxiang Liu, Tao Ren, Wei Xiong, Tingrui Yang et al.ACL Findings 2026 · 1 citation
  3. CARL: Constraint-Aware Reinforcement Learning for Planning with LLMsQiuyi Qi, Jinjian Zhang, Mutian Bao, Tian Liang et al.ACL Findings 2026
  4. PlanLLM: Video Procedure Planning with Refinable Large Language ModelsDejie Yang, Zijing Zhao, Yang LiuAAAI 2025 · 8 citations
  5. Theory of Mind for Multi-Agent Collaboration via Large Language ModelsHuao Li, Yu Quan Chong, Simon Stepputtis, Joseph Campbell et al.EMNLP 2023 · 57 citations
Dusk over the sea between two cliffs drawn in fine vertical lines