Integrating Physician Diagnostic Logic into Large Language Models: Preference Learning from Process Feedback

The utilization of large language models for medical dialogue generation has attracted considerable attention due to its potential to enhance response richness and coherence. While previous studies have made strides in optimizing model performance, there is a pressing need to bolster the model's capacity for diagnostic logic to ensure patient safety. In response to this need, we propose an approach termed preference learning from process feedback (PLPF), which involves integrating the doctor's diagnostic logic into LLMs. PLPF encompasses three key components: rule modeling, preference data generation, and preference alignment. These components collectively serve to train the model to adhere to the diagnostic process. Our experimental results, utilizing Standardized Patient Testing, demonstrate that PLPF enhances the diagnostic accuracy of the baseline model in medical conversations by 17.6%, surpassing the performance of traditional approaches. Moreover, PLPF exhibits effectiveness in both multi-round and single-round dialogue tasks, thereby highlighting its potential in improving medical dialogue generation. Our dataset is available at https://github. com/Chengfeng-Dou/SpTesting D. Doctors are obliged to give a reasonable explanation of their diagnosis and proposed treatment. E. Doctors should not evade a patient's questions without providing a reasonable explanation. F. Doctors should actively seek relevant information from patients for their diagnosis. A. Before diagnosing and guiding the patient, doctors needs to carefully verify the patient's condition. B. Doctors should inform patients about their disease or the tests needed for diagnosis. C. Doctors should inform patients of treatment options for the disease. Goal-oriented rules Constraint-oriented rules F. Be proactive in collecting patient information E. Answer patient questions in a positive way A.Collecting patient information B. Making a diagnosis C. Informing about treatment B. Recommending for medical testing Yes No B. Can a diagnosis be made? Start End D. Explaining D. Explaining D. Explaining