Debate-of-Thoughts: Resolving Knowledge Conflicts in LLMs Through Internal Deliberation

Large Language Models enhanced with Retrieval Augmented Generation show strong potential in knowledge intensive tasks. However, they often encounter knowledge conflicts, where retrieved information contradicts the model's internal knowledge or exhibits internal inconsistencies. Existing methods force models into a binary choice between context and memory, leading to unreliable predictions. We argue that a more principled approach is to embrace contradictions as opportunities for deeper reasoning. To this end, we introduce Debate-of-Thoughts (DoT), a framework that transforms conflict resolution into an active deliberation process. DoT guides a single model through three phases: 1) hypothesis generation, which forms competing perspectives; 2) internal debate, where the model acts as both a proponent and a critic to stress test each view; and 3) adjudication, where the model acts as a judge to evaluate arguments based on evidence and logical consistency. We implement DoT via two complementary strategies: inference time prompt chaining and supervised fine tuning. Experiments across multiple conflict benchmarks show that DoT consistently outperforms state-of-the-art methods, while generating transparent debate transcripts that explain its decisions. By improving both accuracy and interpretability under knowledge conflicts, DoT establishes a more reliable paradigm for retrieval augmented generation systems. 1 * Corresponding authors. 1 Code are availabe at: https://github.com/cong03/ DoT . Contextual Conflict My internal data clearly indicates that ENIAC was completed in 1945 and is not the first commercial computer. User Query: Who is the current official men's marathon world record holder, and what is the time?