LongRanker: Efficient One-Pass Document Reranking with Long-Context Large Language Models

Large language models (LLMs) have demonstrated significant potential in listwise document reranking. Due to their limited context length, LLM-based listwise reranking methods often rely on a sliding window strategy that only processes a small subset of documents at a time. While effective, this approach lacks interactions between documents, increases computational overhead, and results in significant API costs. It is crucial to develop long-context LLMs for enabling the full ranking of all documents in one pass. We propose a long-context listwise document reranker, Long-Ranker, and make two major contributions to enable long-context LLMs for listwise reranking: (i) To improve length extrapolation for listwise inputs, we introduce an intra-inter hierarchical positional encoding approach that combines intra-document encoding to identify token locations within a document with inter-document encoding to specify the document index. (ii) To efficiently model long contexts with reduced computational costs, we introduce a top-𝑘 attention pruning mechanism; it ensures that the model attends to the query and the most important documents while disregarding less relevant ones. Extensive empirical results demonstrate that LongRanker achieves performance comparable to state-of-the-art rerankers while significantly improving efficiency through reduced LLM calls. CCS Concepts • Information systems → Retrieval models and ranking.