Lune

EMNLP Findings2025

Can Out-of-Distribution Evaluations Uncover Reliance on Prediction Shortcuts? A Case Study in Question Answering

Michal Stefánik, Timothee Mickus, Michal Spiegel, Marek Kadlcík, Josef Kuchar

Abstract

A majority of recent work in AI assesses models' generalization capabilities through the lens of performance on out-of-distribution (OOD) datasets. Despite their practicality, such evaluations build upon a strong assumption: that OOD evaluations can capture and reflect upon possible failures in a real-world deployment.

In this work, we challenge this assumption and confront the results obtained from OOD evaluations with a set of specific failure modes documented in existing question-answering (QA) models, referred to as a reliance on spurious features or prediction shortcuts.

We find that different datasets used for OOD evaluations in QA provide an estimate of models' robustness to shortcuts that have a vastly different quality, some largely underperforming even a simple, in-distribution evaluation. We partially attribute this to the observation that spurious shortcuts are shared across ID+OOD datasets, but also find cases where a dataset's quality for training and evaluation is largely disconnected. Our work underlines limitations of commonly-used OOD-based evaluations of generalization, and provides methodology and recommendations for evaluating generalization within and beyond QA more robustly.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future cites Adversarial NLI: A New Benchmark for Natural Language Understanding
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on8

  1. Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
  2. Adversarial NLI: A New Benchmark for Natural Language UnderstandingYixin Nie, Adina Williams, Emily Dinan, Mohit Bansal et al.ACL 2020 · 602 citations
  3. Examining and Combating Spurious Features under Distribution ShiftChunting Zhou, Xuezhe Ma, Paul Michel, Graham NeubigICML 2021 · 78 citations
  4. Causal de Finetti: On the Identification of Invariant Causal Structure in Exchangeable DataSiyuan Guo, Viktor Tóth, Bernhard Schölkopf, Ferenc HuszarNeurIPS 2023 · 57 citations
  5. Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and FutureLinyi Yang, Yaoxian Song, Xuan Ren, Chenyang Lyu et al.EMNLP 2023 · 12 citations

Related papers

  1. ID and OOD Performance Are Sometimes Inversely Correlated on Real-world DatasetsDamien Teney, Yong Lin, Seong Joon Oh, Ehsan AbbasnejadNeurIPS 2023 · 70 citations
  2. OODREB: Benchmarking State-of-the-Art Methods for Out-Of-Distribution Generalization on Relation ExtractionHaotian Chen, Houjing Guo, Bingsheng Chen, Xiangdong ZhouWWW 2024 · 1 citation
  3. Do Generalisation Results Generalise?Matteo Boglioni, Andrea Sgobbi, Gabriel Tavernini, Francesco Rita et al.ACL Findings 2026
  4. Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQAQingyi Si, Fandong Meng, Mingyu Zheng, Zheng Lin et al.EMNLP Findings 2022 · 17 citations
  5. Aggregation Hides Out-of-Distribution Generalization Failures from Spurious CorrelationsOlawale Salaudeen, Haoran Zhang, Kumail Alhamoud, Sara Beery et al.NeurIPS 2025 · 3 citations
Dusk over the sea between two cliffs drawn in fine vertical lines