A Unified Framework for Rule Learning: Integrating Commonsense Knowledge from LLMs with Structured Knowledge from Knowledge Graphs

Unlike many black-box machine learning models, logical rules offer human-understandable explanations for decision-making processes, which is especially important in transparency-critical domains like finance and healthcare. Traditional rule learning methods primarily rely on structured knowledge from Knowledge Graphs (KGs), which cannot align the learned rules with commonsense reasoning, leading to potentially incorrect rules. While Large Language Models (LLMs) offer rich commonsense knowledge, they are prone to hallucinations, which hinder their reliability in learning logical rules. The structured knowledge in KGs can serve as an external reference to mitigate these hallucinations. To leverage the strengths of both approaches, we propose a unified framework CSRL to integrate the commonsense knowledge of LLMs with the structured knowledge from KGs for logical rule learning. CSRL achieves a seamless integration of these two types of knowledge. On one hand, it samples multiple instances of each rule based on the KG structure and exploits LLMs to check the reliability of a rule based on such multiple cases, thereby effectively reducing the effect of hallucinations. On the other hand, CSRL utilizes commonsense knowledge from LLMs to guide dynamic and efficient sampling of useful path instances within KGs. Extensive results from quantitative KG completion experiments and qualitative LLM/human-based semantic assessments demonstrate that our algorithm not only performs well on reasoning tasks but also offers greater reliability and alignment with the real world. Our source code is available at: https://github.com/PerseidsMeteorShower/CSRL .