Lune

EMNLP Findings2024

Efficiently Computing Susceptibility to Context in Language Models

Tianyu Liu, Kevin Du, Mrinmaya Sachan, Ryan Cotterell

Abstract

One strength of modern language models is their ability to incorporate information from a user-input context when answering queries. However, they are not equally sensitive to the subtle changes to that context. To quantify this, Du et al. (2024) gives an informationtheoretic metric to measure such sensitivity. Their metric, susceptibility, is defined as the degree to which contexts can influence a model's response to a query at a distributional level. However, exactly computing susceptibility is difficult and, thus, Du et al. ( 2024) fall back on a Monte Carlo approximation. Due to the large number of samples required, the Monte Carlo approximation is inefficient in practice. As a faster alternative, we propose Fisher susceptibility, an efficient method to estimate the susceptibility based on Fisher information. Empirically, we validate that Fisher susceptibility is comparable to Monte Carlo estimated susceptibility across a diverse set of query domains despite its being 70× faster. Exploiting the improved efficiency, we apply Fisher susceptibility to analyze factors affecting the susceptibility of language models. We observe that larger models are as susceptible as smaller ones. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts cites Large Language Models Can Be Easily Distracted by Irrelevant Context
  • Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts cites Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidence
  • Making Retrieval-Augmented Language Models Robust to Irrelevant Context cites Large Language Models Can Be Easily Distracted by Irrelevant Context
  • Making Retrieval-Augmented Language Models Robust to Irrelevant Context cites Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidence
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on8

  1. Pythia: A Suite for Analyzing Large Language Models Across Training and ScalingStella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley et al.ICML 2023 · 1,822 citations
  2. Large Language Models Can Be Easily Distracted by Irrelevant ContextFreda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales et al.ICML 2023 · 970 citations
  3. Making Retrieval-Augmented Language Models Robust to Irrelevant ContextOri Yoran, Tomer Wolfson, Ori Ram, Jonathan BerantICLR 2024 · 361 citations
  4. Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge ConflictsJian Xie, Kai Zhang, Jiangjie Chen, Renze Lou et al.ICLR 2024 · 294 citations
  5. Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting EvidenceHung-Ting Chen, Michael J. Q. Zhang, Eunsol ChoiEMNLP 2022 · 27 citations

Related papers

  1. Context versus Prior Knowledge in Language ModelsKevin Du, Vésteinn Snæbjarnarson, Niklas Stoehr, Jennifer C. White et al.ACL 2024 · 3 citations
  2. Choices Speak Louder than QuestionsGyeongje Cho, Yeonkyoung So, Jaejin LeeICLR 2026 · 2 citations
  3. Fool Your (Vision and) Language Model with Embarrassingly Simple PermutationsYongshuo Zong, Tingyang Yu, Ruchika Chavhan, Bingchen Zhao et al.ICML 2024 · 28 citations
  4. POSIX: A Prompt Sensitivity Index For Large Language ModelsAnwoy Chatterjee, H. S. V. N. S. Kowndinya Renduchintala, Sumit Bhatia, Tanmoy ChakrabortyEMNLP Findings 2024 · 36 citations
  5. Leveraging Unlabeled Data to Track MemorizationMahsa Forouzesh, Hanie Sedghi, Patrick ThiranICLR 2023
Dusk over the sea between two cliffs drawn in fine vertical lines