Lune

ACL Findings2026

Lost in Decomposition: Analyzing and Mitigating the Limitations of Long Context Methods via Context Dependency

Jiayuan Guo, Yueyang Su, Yuyao Ge, Saiping Guan, Lei Yu, Jiafeng Guo, Xueqi Cheng

Abstract

Long context large language models exhibit the "lost in the middle" problem, where models struggle to effectively utilize information located in the middle of long contexts. Although existing workflow-based long context methods (e.g., RAG) alleviate this problem and perform well on specific datasets, can their effectiveness generalize to all types of datasets? In this work, we systematically investigate the cross-dataset generalization of long context methods. Our evaluation reveals that these methods are not universally effective. Such substantial performance variability underscores the risks of performance degradation associated with the indiscriminate application of long context methods. We investigated the reason for the failure of long context methods. We found that the intrinsic decomposition mechanisms of long context methods hinder context dependency modeling, causing these methods to suffer performance declines on documents with strong context dependency. To address this issue, We propose CoDaR (Context Dependency-aware Routing), a training-free adaptive routing strategy. By analyzing the context dependency strength of documents, CoDaR adaptively invokes long context methods, thereby significantly enhancing their overall robustness across different types of datasets.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent cites MemoryBank: Enhancing Large Language Models with Long-Term Memory
  • A-Mem: Agentic Memory for LLM Agents cites MemoryBank: Enhancing Large Language Models with Long-Term Memory
  • A-Mem: Agentic Memory for LLM Agents cites Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on17

  1. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-ReflectionAkari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil et al.ICLR 2024 · 1,798 citations
  2. A-Mem: Agentic Memory for LLM AgentsWujiang Xu, Zujie Liang, Kai Mei, Hang Gao et al.NeurIPS 2025 · 1,138 citations
  3. M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge DistillationJianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo et al.ACL Findings 2024 · 762 citations
  4. MemoryBank: Enhancing Large Language Models with Long-Term MemoryWanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye et al.AAAI 2024 · 394 citations
  5. MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory AgentHongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen et al.ICLR 2026 · 231 citations

Related papers

  1. LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question AnsweringQingfei Zhao, Ruobing Wang, Yukuo Cen, Daren Zha et al.EMNLP 2024 · 13 citations
  2. Beyond RAG vs. Long-Context: Learning Distraction-Aware Retrieval for Efficient Knowledge GroundingSeong-Woong Shim, Myunsoo Kim, Jae Hyeon Cho, Byung-Jun LeeICLR 2026 · 1 citation
  3. LLM×MapReduce: Simplified Long-Sequence Processing using Large Language ModelsZihan Zhou, Chong Li, Xinyi Chen, Shuo Wang et al.ACL 2025 · 14 citations
  4. Found in the middle: Calibrating Positional Attention Bias Improves Long Context UtilizationCheng-Yu Hsieh, Yung-Sung Chuang, Chun-Liang Li, Zifeng Wang et al.ACL Findings 2024 · 31 citations
  5. LADM: Long-context Training Data Selection with Attention-based Dependency Measurement for LLMsJianghao Chen, Junhong Wu, Yangyifan Xu, Jiajun ZhangACL 2025
Dusk over the sea between two cliffs drawn in fine vertical lines