Lune

EMNLP Findings2024

Can We Instruct LLMs to Compensate for Position Bias?

Meiru Zhang, Zaiqiao Meng, Nigel Collier

Abstract

Position bias in large language models (LLMs) leads to difficulty in accessing information retrieved from the retriever, thus downgrading the effectiveness of Retrieval-Augmented Generation (RAG) approaches in open-question answering. Recent studies reveal that this bias is related to disproportional attention across the context. In this work, we examine how to direct LLMs to allocate more attention towards a selected segment of the context through prompting, aiming to compensate for the shortage of attention. We find that language models do not have relative position awareness of the context but can be directed by promoting instruction with an exact document index. Our analysis contributes to a deeper understanding of position bias in LLMs and provides a pathway to mitigate this bias by instruction, thus benefiting LLMs in locating and utilizing relevant information from retrieved documents in RAG applications. The code and data in our study have been made publicly available. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Rationale-Enhanced Decoding for Multi-modal Chain-of-Thought cites Efficient Streaming Language Models with Attention Sinks
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Rationale-Enhanced Decoding for Multi-modal Chain-of-ThoughtShin'ya Yamaguchi, Kosuke Nishida, Daiki ChijiwaCVPR 2026
  2. Who is in the Spotlight: The Hidden Bias Undermining Multimodal Retrieval-Augmented GenerationJiayu Yao, Shenghua Liu, Yiwei Wang, Lingrui Mei et al.EMNLP 2025

Builds on16

  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
  2. Efficient Streaming Language Models with Attention SinksGuangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han et al.ICLR 2024 · 1,714 citations
  3. Efficient Memory Management for Large Language Model Serving with PagedAttentionWoosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng et al.SOSP 2023 · 1,016 citations
  4. Augmenting Language Models with Long-Term MemoryWeizhi Wang, Li Dong, Hao Cheng, Xiaodong Liu et al.NeurIPS 2023 · 256 citations
  5. Make Your LLM Fully Utilize the ContextShengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng et al.NeurIPS 2024 · 212 citations

Related papers

  1. Position Engineering: Boosting Large Language Models through Positional Information ManipulationZhiyuan He, Huiqiang Jiang, Zilong Wang, Yuqing Yang et al.EMNLP 2024 · 3 citations
  2. Found in the middle: Calibrating Positional Attention Bias Improves Long Context UtilizationCheng-Yu Hsieh, Yung-Sung Chuang, Chun-Liang Li, Zifeng Wang et al.ACL Findings 2024 · 31 citations
  3. GraphInsight: Unlocking Insights in Large Language Models for Graph Structure UnderstandingYukun Cao, Shuo Han, Zengyi Gao, Zezhong Ding et al.ACL 2025
  4. Superposition Prompting: Improving and Accelerating Retrieval-Augmented GenerationThomas Merth, Qichen Fu, Mohammad Rastegari, Mahyar NajibiICML 2024 · 14 citations
  5. PEAR: Position-Embedding-Agnostic Attention Re-weighting Enhances Retrieval-Augmented Generation with Zero Inference OverheadTao Tan, Yining Qian, Ang Lv, Hongzhan Lin et al.WWW 2025 · 4 citations
Dusk over the sea between two cliffs drawn in fine vertical lines