RPDR: A Round-trip Prediction-Based Data Augmentation Framework for Long-Tail Question Answering

Long-tail question answering presents significant challenges for large language models (LLMs) due to their limited ability to acquire and accurately recall less common knowledge. Retrieval-augmented generation (RAG) systems have shown great promise in mitigating this limitation by integrating external retrieval mechanisms. However, dense retrieval models often face the same difficulties when generalizing to rare or niche knowledge. In this study, we introduce RPDR, a novel data augmentation framework that selects high-quality easy-to-learn training data, to enhance dense retrievers. Our approach is built around three core components: synthetic data generation, data selection with Round-Trip prediction to identify easy-to-learn instances, and retriever training with these instances. We evaluate RPDR on two longtail retrieval benchmarks, POPQA and ENTI-TYQUESTIONS, demonstrating substantial improvements over existing retrievers like BM25 and Contriver, especially on extremely longtail categories. We identify the strengths and limitations of RPDR through detailed human analysis and propose a dynamic routing mechanism to dynamically route queries to specialized retrieval modules to further improve retrieval performance. 1 * Work done when Yiming Zhang was visiting NYU Shanghai.