Training Large Language Models for Retrieval-Augmented Question Answering through Backtracking Correction

Despite recent progress in Retrieval-Augmented Generation (RAG) achieved by large language models (LLMs), retrievers often recall uncorrelated documents, regarded as "noise" during subsequent text generation. To address this, some methods train LLMs to distinguish between relevant and irrelevant documents using labeled data, enabling them to select the most likely relevant ones as context. However, they are susceptible to disturbances, as LLMs can easily make mistakes when the chosen document contains irrelevant information. Some approaches increase the number of referenced documents and train LLMs to perform stepwise reasoning when presented with multiple documents. Unfortunately, these methods rely on extensive and diverse annotations to ensure generalization, which is both challenging and costly. In this paper, we propose Backtracking Correction to address these limitations. Specifically, we reformulate stepwise RAG into a multi-step decisionmaking process. Starting from the final step, we optimize the model through error sampling and self-correction, and then backtrack to the previous state iteratively. In this way, the model's learning scheme follows an easy-to-hard progression: as the target state moves forward, the context space decreases while the decision space increases. Experimental results demonstrate that Backtracking Correction enhances LLMs' ability to make complex multi-step assessments, improving the robustness of RAG in dealing with noisy documents. Our code and data are available at https://github.com/201736621051/BacktrackingCorrection.