Lune

Abstract

Patients must possess the knowledge necessary to actively participate in their care. We present NoteAid-Chatbot, a conversational AI that promotes patient understanding via a novel 'learning as conversation' framework, built on a multi-agent large language model (LLM) and reinforcement learning (RL) setup without human-labeled data. NoteAid-Chatbot was built on a lightweight 3B-parameter LLaMA 3.2 model trained in two stages: initial supervised fine-tuning on conversational data synthetically generated using medical conversation strategies, followed by RL with rewards derived from patient understanding assessments in simulated hospital discharge scenarios. Our evaluation, which includes comprehensive human-aligned assessments and case studies, demonstrates that NoteAid-Chatbot exhibits key emergent behaviors critical for patient education-such as clarity, relevance, and structured dialogue-even though it received no explicit supervision for these attributes. Our results show that even simple Proximal Policy Optimization (PPO)-based reward modeling can successfully train lightweight, domain-specific chatbots to handle multi-turn interactions, incorporate diverse educational strategies, and meet nuanced communication objectives. Our Turing test demonstrates that NoteAid-Chatbot surpasses non-expert human. Although our current focus is on healthcare, the framework we present illustrates the feasibility and promise of applying low-cost, PPO-based RL to realistic, open-ended conversational domains-broadening the applicability of RL-based alignment methods.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study cites Training language models to follow instructions with human feedback
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. LLM-Based Multi-Agent Systems for Clinical Workflows: A Survey of AI HospitalsZonghai Yao, Hong YuACL 2026

Builds on9

  1. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
  2. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  3. BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
  4. Is DPO Superior to PPO for LLM Alignment? A Comprehensive StudyShusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye et al.ICML 2024 · 274 citations
  5. BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical DomainsYanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud et al.ACL Findings 2024 · 271 citations

Related papers

  1. Back to Basics: Revisiting REINFORCE-Style Optimization for Learning from Human Feedback in LLMsArash Ahmadian, Chris Cremer, Matthias Gallé, Marzieh Fadaee et al.ACL 2024 · 20 citations
  2. Grounded in Reality: Learning and Deploying Proactive LLM from Offline LogsFei Wei, Daoyuan Chen, Ce Wang, Yilun Huang et al.ICML 2026 · 2 citations
  3. Improving the Language Understanding Capabilities of Large Language Models Using Reinforcement LearningSai Ashish Somayajula, Bokai Hu, Qi Cao, Xin Pan et al.EMNLP Findings 2025 · 1 citation
  4. Consistently Simulating Human Personas with Multi-Turn Reinforcement LearningMarwa Abdulhai, Ryan Cheng, Donovan Clay, Tim Althoff et al.NeurIPS 2025 · 51 citations
  5. AGILE: A Novel Reinforcement Learning Framework of LLM AgentsPeiyuan Feng, Yichen He, Guanhua Huang, Yuan Lin et al.NeurIPS 2024 · 61 citations
Dusk over the sea between two cliffs drawn in fine vertical lines