From Query to Counsel: Structured Reasoning with a Multi-Agent Framework and Dataset for Legal Consultation
Mingfei Lu, Yi Zhang, Mengjia Wu, Yue Feng
Abstract
Legal consultation question answering (Legal CQA) presents unique challenges compared to traditional legal QA tasks, including the scarcity of high-quality training data, complex task composition, and strong contextual dependencies. To address these, we construct JURISCQAD, a large-scale dataset of over 43,000 real-world Chinese legal queries annotated with expert-validated positive and negative responses, and design a structured task decomposition that converts each query into a legal element graph integrating entities, events, intents, and legal issues. We further propose JU-RISMA, a modular multi-agent framework supporting dynamic routing, statutory grounding, and stylistic optimization. Combined with the element graph, the framework enables strong context-aware reasoning, effectively capturing dependencies across legal facts, norms, and procedural logic. Trained on JURISCQAD and evaluated on a refined LawBench, our system significantly outperforms both general-purpose and legal-domain LLMs across multiple lexical and semantic metrics, demonstrating the benefits of interpretable decomposition and modular collaboration in Legal CQA.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 148cd0b1-cea7-484b-acc0-efa4e8aa0e62Cited by top-tier papers1
Ask how each one uses itBuilds on13
- BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
- PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive SummarizationJingqing Zhang, Yao Zhao, Mohammad Saleh, Peter J. LiuICML 2020 · 2,453 citations
- How Does NLP Benefit Legal System: A Summary of Legal Artificial IntelligenceHaoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang et al.ACL 2020 · 316 citations
- JEC-QA: A Legal-Domain Question Answering DatasetHaoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang et al.AAAI 2020 · 212 citations
- LawBench: Benchmarking Legal Knowledge of Large Language ModelsZhiwei Fei, Xiaoyu Shen, Dawei Zhu, Fengzhe Zhou et al.EMNLP 2024 · 59 citations
Related papers
- JurisBench: A Deep Benchmark for Assessing Large Language Models in Professional Legal PracticeZiang Chen, Guannan Li, Fanlin Ji, Yipeng Kang et al.ACL 2026
- LegalSearchLM: Rethinking Legal Case Retrieval as Legal Elements GenerationChaeeun Kim, Jinu Lee, Wonseok HwangEMNLP 2025 · 4 citations
- LegalAgentBench: Evaluating LLM Agents in Legal DomainHaitao Li, Junjie Chen, Jingli Yang, Qingyao Ai et al.ACL 2025
- CompKBQA: Component-wise Task Decomposition for Knowledge Base Question AnsweringYuhang Tian, Dandan Song, Zhijing Wu, Pan Yang et al.EMNLP 2025 · 1 citation
- HM-RAG: Hierarchical Multi-Agent Multimodal Retrieval Augmented GenerationPei Liu, Xin Liu, Ruoyu Yao, Junming Liu et al.ACM MM 2025 · 27 citations
