Lune

ACL2023Top-tier venue

FAA: Fine-grained Attention Alignment for Cascade Document Ranking

Zhen Li, Chongyang Tao, Jiazhan Feng, Tao Shen, Dongyan Zhao, Xiubo Geng, Daxin Jiang

Abstract

Document ranking aims at sorting a collection of documents with their relevance to a query. Contemporary methods explore more efficient transformers or divide long documents into passages to handle the long input. However, intensive query-irrelevant content may lead to harmful distraction and high query latency. Some recent works further propose cascade document ranking models that extract relevant passages with an efficient selector before ranking, however, their selection and ranking modules are almost independently optimized and deployed, leading to selecting error reinforcement and sub-optimal performance. In fact, the document ranker can provide fine-grained supervision to make the selector more generalizable and compatible, and the selector built upon a different structure can offer a distinct perspective to assist in document ranking. Inspired by this, we propose a fine-grained attention alignment approach to jointly optimize a cascade document ranking model. Specifically, we utilize the attention activations over the passages from the ranker as fine-grained attention feedback to optimize the selector. Meanwhile, we fuse the relevance scores from the passage selector into the ranker to assist in calculating the cooperative matching representation. Experiments on MS MARCO and TREC DL demonstrate the effectiveness of our method. 𝑝 ! : … Spiders are animals that have 8 legs and use their fangs to inject venom into other animals and sometimes humans … 𝑝 " : … Most species trap small insects and other spiders in their webs and eat them. A few large species of spiders prey … 𝑝 # : … I caught him and now he lives in a small fish tank, I feed him 2 flies once a day.. He is a fast hunter…

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Learning Cascade Ranking as One Network cites Adaptive Neural Ranking Framework: Toward Maximized Business Goal for Cascade Ranking Systems
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Adaptive Neural Ranking Framework: Toward Maximized Business Goal for Cascade Ranking SystemsYunli Wang, Zhiqiang Wang, Jian Yang, Shiyang Wen et al.WWW 2024 · 16 citations
  2. Learning Cascade Ranking as One NetworkYunli Wang, Zhen Zhang, Zhiqiang Wang, Zixuan Yang et al.ICML 2025

Builds on6

  1. MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained TransformersWenhui Wang, Furu Wei, Li Dong, Hangbo Bao et al.NeurIPS 2020 · 2,727 citations
  2. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
  3. Transformer-XH: Multi-Evidence Reasoning with eXtra Hop AttentionChen Zhao, Chenyan Xiong, Corby Rosset, Xia Song et al.ICLR 2020 · 120 citations
  4. Efficient Document Re-Ranking for Transformers by Precomputing Term RepresentationsSean MacAvaney, Franco Maria Nardini, Raffaele Perego, Nicola Tonellotto et al.SIGIR 2020 · 62 citations
  5. Intra-Document Cascading: Learning to Select Passages for Neural Document RankingSebastian Hofstätter, Bhaskar Mitra, Hamed Zamani, Nick Craswell et al.SIGIR 2021 · 35 citations

Related papers

  1. Long Document Ranking with Query-Directed Sparse TransformerJyun-Yu Jiang, Chenyan Xiong, Chia-Jung Lee, Wei WangEMNLP Findings 2020 · 23 citations
  2. Scalable In-context Ranking with Generative ModelsNilesh Gupta, Chong You, Srinadh Bhojanapalli, Sanjiv Kumar et al.NeurIPS 2025 · 10 citations
  3. Efficient Neural Ranking using Forward IndexesJurek Leonhardt, Koustav Rudra, Megha Khosla, Abhijit Anand et al.WWW 2022 · 16 citations
  4. Leveraging Passage-level Cumulative Gain for Document RankingZhijing Wu, Jiaxin Mao, Yiqun Liu, Jingtao Zhan et al.WWW 2020 · 40 citations
  5. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTOmar Khattab, Matei ZahariaSIGIR 2020 · 1,246 citations
Dusk over the sea between two cliffs drawn in fine vertical lines