EMNLP Findings2025
DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillation
Abdelrahman Abdallah, Jamshid Mozafari, Bhawna Piryani, Adam Jatowt
Abstract
Large Language Models (LLMs) have transformed listwise document reranking by enabling global reasoning over candidate sets, yet single models often struggle to balance fine-grained relevance scoring with holistic cross-document analysis. We propose DeepAgentRank (DEAR), an open-source framework that decouples these tasks through a dual-stage approach, achieving superior accuracy and interpretability. In Stage 1, we distill token-level relevance signals from a frozen 13B LLaMA teacher into a compact 3, 8B student model using a hybrid of cross-entropy, RankNet, and KL divergence losses, ensuring robust pointwise scoring. In Stage 2, we attach a second LoRA adapter and fine-tune on 20K GPT-4o-generated chain-of-thought permutations, enabling listwise reasoning with natural-language justifications. Evaluated on TREC-DL19/20, eight BEIR datasets, and NovelEval-2306, DEAR surpasses opensource baselines by +5.1 nDCG@5 on DL20 and achieves 90.97 nDCG@10 on NovelEval, outperforming GPT-4 by +3.09. Without fine-tuning on Wikipedia, DeAR also excels in open-domain QA, achieving 54.29 Top-1 accuracy on Natural Questions, surpassing baselines like MonoT5, UPR, and RankGPT. Ablations confirm that dual-loss distillation ensures stable calibration, making DEAR a highly effective and interpretable solution for modern reranking systems. 1 .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Improving Passage Retrieval with Zero-Shot Question Generation cites Document Ranking with a Pretrained Sequence-to-Sequence Model
- Improving Passage Retrieval with Zero-Shot Question Generation cites Dense Passage Retrieval for Open-Domain Question Answering
- Improving Passage Retrieval with Zero-Shot Question Generation cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Dense Passage Retrieval for Open-Domain Question Answering cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks cites Dense Passage Retrieval for Open-Domain Question Answering
- Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents cites Improving Passage Retrieval with Zero-Shot Question Generation
- Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents cites Document Ranking with a Pretrained Sequence-to-Sequence Model
- Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents cites Dense Passage Retrieval for Open-Domain Question Answering
Cited by top-tier papers1
Builds on10
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- Document Ranking with a Pretrained Sequence-to-Sequence ModelRodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, Jimmy LinEMNLP Findings 2020 · 469 citations
- Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking AgentsWeiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang et al.EMNLP 2023 · 182 citations
- Dense Passage Retrieval for Open-Domain Question AnsweringVladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis et al.EMNLP 2020 · 142 citations
- Improving Passage Retrieval with Zero-Shot Question GenerationDevendra Singh Sachan, Mike Lewis, Mandar Joshi, Armen Aghajanyan et al.EMNLP 2022 · 69 citations
Related papers
- REARANK: Reasoning Re-ranking Agent via Reinforcement LearningLe Zhang, Bo Wang, Xipeng Qiu, Siva Reddy et al.EMNLP 2025 · 14 citations
- Reason-to-Rank: Distilling Direct and Comparative Reasoning from Large Language Models for Document RerankingYuelyu Ji, Zhuochun Li, Rui Meng, Daqing HeSIGIR 2025 · 3 citations
- BracketRank: Large Language Model Document Ranking via Reasoning-based Competitive EliminationAbdelrahman Abdallah, Mohammed Ali, Bhawna Piryani, Adam JatowtACL 2026
- ReasonRank: Empowering Passage Ranking with Strong Reasoning AbilityWenhan Liu, Xinyu Ma, Weiwei Sun, Yutao Zhu et al.ACL 2026 · 43 citations
- ERank: Fusing Supervised Fine-Tuning and Reinforcement Learning for Effective and Efficient Text RerankingYuzheng Cai, Yanzhao Zhang, Dingkun Long, Mingxin Li et al.AAAI 2026 · 6 citations
