Lune

SIGIR2025Top-tier venue

RankingSHAP - Faithful Listwise Feature Attribution Explanations for Ranking Models

Maria Heuss, Maarten de Rijke, Avishek Anand

Abstract

While SHAP (SHapley Additive exPlanations) and other feature attribution methods are commonly employed to explain model predictions, their application within information retrieval (IR), particularly for complex outputs such as ranked lists, remains limited. Existing attribution methods typically provide pointwise explanations, focusing on why a single document received a high-ranking score, rather than considering the relationships between documents in a ranked list. We present three key contributions to address this gap. First, we rigorously define listwise feature attribution for ranking models. Secondly, we introduce RankingSHAP, extending the popular SHAP framework to accommodate listwise ranking attribution, addressing a significant methodological gap in the field. Third, we propose two novel evaluation paradigms for assessing the faithfulness of attributions in learning-to-rank models, measuring the correctness and completeness of the explanation with respect to different aspects. Through experiments on standard learning-to-rank datasets, we demonstrate RankingSHAP's practical application while identifying the constraints of selection-based explanations. We further employ a simulated study with an interpretable model to showcase how listwise ranking attributions can be used to examine model decisions and conduct a qualitative evaluation of explanations. Due to the contrastive nature of the ranking task, our understanding of ranking model decisions can substantially benefit from feature attribution explanations like RankingSHAP.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • WeightedSHAP: analyzing and improving Shapley based feature attributions cites Problems with Shapley-value-based explanations as feature importance measures
  • Understanding the Black Box: A Deep Empirical Dive into Shapley Value Approximations for Tabular Data cites Efficient Sampling Approaches to Shapley Value Approximation
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Understanding the Black Box: A Deep Empirical Dive into Shapley Value Approximations for Tabular DataSuchit Gupte, John PaparrizosSIGMOD 2025 · 19 citations
  2. Reason-to-Rank: Distilling Direct and Comparative Reasoning from Large Language Models for Document RerankingYuelyu Ji, Zhuochun Li, Rui Meng, Daqing HeSIGIR 2025 · 3 citations
  3. Explaining Rankings with Hidden Group BonusesAlvin Hong Yao Yan, Suraj Shetiya, Sujoy Bhore, Priyanka Golia et al.KDD 2026

Builds on11

  1. Problems with Shapley-value-based explanations as feature importance measuresI. Elizabeth Kumar, Suresh Venkatasubramanian, Carlos Scheidegger, Sorelle A. FriedlerICML 2020 · 458 citations
  2. A Consistent and Efficient Evaluation Strategy for Attribution MethodsYao Rong, Tobias Leemann, Vadim Borisov, Gjergji Kasneci et al.ICML 2022 · 138 citations
  3. WeightedSHAP: analyzing and improving Shapley based feature attributionsYongchan Kwon, James Y. ZouNeurIPS 2022 · 60 citations
  4. On Locality of Local Explanation ModelsSahra Ghalebikesabi, Lucile Ter-Minassian, Karla DiazOrdaz, Chris C. HolmesNeurIPS 2021 · 52 citations
  5. Efficient Sampling Approaches to Shapley Value ApproximationJiayao Zhang, Qiheng Sun, Jinfei Liu, Li Xiong et al.SIGMOD 2023 · 43 citations

Related papers

  1. RankSHAP: Shapley Value Based Feature Attributions for Learning to RankTanya Chowdhury, Yair Zick, James AllanICLR 2025
  2. ShaRP: Explaining Rankings and Preferences with Shapley ValuesVenetia Pliatsika, João Fonseca, Kateryna Akhynko, Ivan Shevchenko et al.VLDB 2025 · 7 citations
  3. Context Attribution with Multi-Armed Bandit OptimizationDeng Pan, Keerthiram Murugesan, Ting Hua, Nuno Moniz et al.ACL Findings 2026
  4. Accurate Estimation of Feature Importance Faithfulness for Tree ModelsMateusz Gajewski, Adam Karczmarz, Mateusz Rapicki, Piotr SankowskiAAAI 2025
  5. From Scoring to Explanations: Evaluating SHAP and LLM Rationales for Rubric-based Teaching Quality AssessmentIvo Bueno, Babette Bühler, Philipp Stark, Tim Fütterer et al.ACL Findings 2026
Dusk over the sea between two cliffs drawn in fine vertical lines