Experience Retrieval-Augmentation with Electronic Health Records Enables Accurate Discharge QA

To improve the reliability of Large Language Models (LLMs) in clinical applications, retrieval-augmented generation (RAG) is extensively applied to provide factual medical knowledge. Beyond general medical knowledge, clinical case-based knowledge is also critical for effective medical reasoning, as it provides context grounded in real-world patient experiences. Motivated by this, we propose Experience Retrieval-Augmentation (EXPRAG) framework based on Electronic Health Record (EHR), aiming to offer the relevant context from other patients' discharge reports. EXPRAG performs retrieval through a coarse-to-fine process: it first applies an EHR-based report ranker to efficiently identify similar patients as experience, and then utilizes a context retriever to extract task-relevant content for enhanced medical reasoning. To evaluate RAG systems on EHR data including EXPRAG and medical agents, we introduce DISCHARGEQA, a clinical QA dataset with 1,280 discharge-related questions across diagnosis, medication, and instruction tasks. Each problem is generated using historical EHR data to ensure realistic and challenging scenarios. Experimental results demonstrate that EXPRAG consistently outperforms traditional text-based rankers, achieving an average relative improvement of 5.2%, highlighting the importance of case-based knowledge for medical reasoning.