Can we Retrieve Everything All at Once? ARM: An Alignment-Oriented LLM-based Retrieval Method

Real-world open-domain questions can be complex, especially when answering them requires integrating information from multiple sources. Effectively identifying the necessary information involves aligning it with the available data and its organization. However, existing RAG solutions address the alignment problem in a limited manner. Using off-the-shelf LLMs for question decomposition lacks awareness of the available data and its structure, often resulting in suboptimal retrieval performance. Alternatively, iteratively generating follow-up queries and interacting with the data collection, as explored in agentic RAG approaches, shows potential but is often inefficient since each successive query depends on previous results rather than being guided by the overall organization of the available data. To address the alignment problem, we introduce an LLM-based retrieval method -ARM, designed to better align questions with the organization of the data collection. Instead of solely matching query utterance, ARM explores relationships among data objects, enabling a retrieve-all-atonce solution for complex queries. Experimental results demonstrate that ARM significantly outperforms existing RAG methods on various complex open-domain QA tasks across multiple modalities, achieving superior retrieval performance and downstream accuracy while significantly lowering monetary costs. 1