Leveraging Knowledge Graph-Enhanced LLMs for Context-Aware Medical Consultation

Recent advancements in large language models have significantly influenced the field of online medical consultations. However, critical challenges remain, such as the generation of hallucinated information and the integration of upto-date medical knowledge. To address these issues, we propose Informatics Llama (ILlama), a novel framework that combines retrievalaugmented generation (RAG) with a structured medical knowledge graph. ILlama incorporates relevant medical knowledge by transforming subgraphs from a structured medical knowledge graph into text for RAG. By generating subgraphs from the medical knowledge graph in advance for RAG, specifically focusing on diseases and symptoms, ILlama enhances the accuracy and relevance of its medical reasoning. This framework enables effective incorporation of causal relationships between symptoms and diseases. Also, it delivers context-aware consultations aligned with user queries. Experimental results on the two medical consultation datasets demonstrate that ILlama outperforms strong baselines, achieving a semantic similarity F1 score of 0.884 when compared to ground-truth consultation answers. Furthermore, qualitative analysis of ILlama's responses reveals significant improvements in hallucination reduction and clinical usefulness. These results suggest that ILlama has strong potential as a reliable tool for real-world medical consultation environments. Our implementation is available at: https://github.com/suhyeong10/ILlama