ACL Findings2024
Benchmarking Large Language Models on Communicative Medical Coaching: A Dataset and a Novel System
Hengguan Huang, Songtao Wang, Hongfu Liu, Hao Wang, Ye Wang
Abstract
Traditional applications of natural language processing (NLP) in healthcare have predominantly focused on patient-centered services, enhancing patient interactions and care delivery, such as through medical dialogue systems. However, the potential of NLP to benefit inexperienced doctors, particularly in areas such as communicative medical coaching, remains largely unexplored. We introduce "ChatCoach," a human-AI cooperative framework designed to assist medical learners in practicing their communication skills during patient consultations. ChatCoach 1 differentiates itself from conventional dialogue systems by offering a simulated environment where medical learners can practice dialogues with a patient agent, while a coach agent provides immediate, structured feedback. This is facilitated by our proposed Generalized Chain-of-Thought (GCoT) approach, which fosters the generation of structured feedback and enhances the utilization of external knowledge sources. Additionally, we have developed a dataset specifically for evaluating Large Language Models (LLMs) within the ChatCoach framework on communicative medical coaching tasks. Our empirical results validate the effectiveness of ChatCoach.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- The Flan Collection: Designing Data and Methods for Effective Instruction Tuning cites Language Models are Few-Shot Learners
- The Flan Collection: Designing Data and Methods for Effective Instruction Tuning cites Training language models to follow instructions with human feedback
- Large Language Models are Zero-Shot Reasoners cites Language Models are Few-Shot Learners
- Large Language Models are Zero-Shot Reasoners cites Training language models to follow instructions with human feedback
- Training language models to follow instructions with human feedback cites Language Models are Few-Shot Learners
Builds on11
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
- The Flan Collection: Designing Data and Methods for Effective Instruction TuningShayne Longpre, Le Hou, Tu Vu, Albert Webson et al.ICML 2023 · 908 citations
- CBLUE: A Chinese Biomedical Language Understanding Evaluation BenchmarkNingyu Zhang, Mosha Chen, Zhen Bi, Xiaozhuan Liang et al.ACL 2022 · 242 citations
Related papers
- MedCoach: Enhancing Medical Reasoning in LLMs via Knowledge Graph-Augmented Chain-of-Thought DistillationChuan Li, Ye Lyu, Chengyu Wang, Mingyuan Fan et al.ACL Findings 2026
- Grounded in Reality: Learning and Deploying Proactive LLM from Offline LogsFei Wei, Daoyuan Chen, Ce Wang, Yilun Huang et al.ICML 2026 · 2 citations
- NoteChat: A Dataset of Synthetic Patient-Physician Conversations Conditioned on Clinical NotesJunda Wang, Zonghai Yao, Zhichao Yang, Huixue Zhou et al.ACL Findings 2024 · 14 citations
- Chatbot To Help Patients Understand Their HealthWon Seok Jang, Hieu Tran, Manav Mistry, SaiKiran Gandluri et al.EMNLP Findings 2025 · 2 citations
- "I Wouldn't Really Use It as a Practice Tool": Understanding Medical Students' Perspectives and Needs on LLM-Enhanced Clinical Skills TrainingYuru Huang, Chao Liu, Yunna Cai, Lina Xu et al.CHI 2026 · 1 citation
