Lune

KDD2025Top-tier venue

Tackling the Length Barrier: Dynamic Context Browsing for Knowledge-Intensive Task

Hongjin Qian, Zheng Liu, Peitian Zhang, Kelong Mao, Yujia Zhou, Xu Chen, Zhicheng Dou

Abstract

Knowledge-intensive tasks often require complex reasoning and contextual understanding over long contexts. However, the learning and deployment of long-LLMs remains a challenging problem despite recent progresses. In this work, we propose that the short LLMs have great potentiality for solving knowledge-intensive tasks that have long context, i.e. they can be solved by purely working with oracle short-contexts within the input long-context. On top of this argument, we propose a framework called DCISO DynamiC knowledge-Intensive task S>Olver), which enables a short-LLM to address the knowledge-intensive tasks with long context via dynamic context browsing. In our framework, the short-LLM prompts itself to reason for two critical decisions: 1) how to access to the appropriate part of context within the input, 2) how to make effective use of the accessed context. By adaptively accessing and utilizing the context based on the presented tasks, DCISO can serve as a general framework to handle diversified knowledge-intensive long-context problems. We comprehensively evaluate different types of tasks from popular long-context benchmarks, where DCISO is able to achieve a substantially improved performance. Our codes will be released at this repository.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • A Survey on In-context Learning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAG cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  • Reflexion: language agents with verbal reinforcement learning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAGFengran Mo, Zhan Su, Yuchen Hui, Jinghan Zhang et al.WWW 2026 · 8 citations

Builds on13

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
  3. Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
  4. YaRN: Efficient Context Window Extension of Large Language ModelsBowen Peng, Jeffrey Quesnelle, Honglu Fan, Enrico ShippoleICLR 2024 · 508 citations
  5. A Survey on In-context LearningQingxiu Dong, Lei Li, Damai Dai, Ce Zheng et al.EMNLP 2024 · 479 citations

Related papers

  1. DocAgent: An Agentic Framework for Multi-Modal Long-Context Document UnderstandingLi Sun, Liu He, Shuyue Jia, Yangfan He et al.EMNLP 2025 · 1 citation
  2. LIFT: A Novel Framework for Enhancing Long-Context Understanding of LLMs via Long Input Fine-TuningYansheng Mao, Yufei Xu, Jiaqi Li, Fanxu Meng et al.ICML 2026 · 7 citations
  3. SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference OptimizationHuashan Sun, Shengyi Liao, Yansen Han, Yu Bai et al.ICLR 2026 · 9 citations
  4. PRISM: Efficient Long-Range Reasoning With Short-Context LLMsDulhan Jayalath, James Bradley Wendt, Nicholas Monath, Sandeep Tata et al.EMNLP 2025
  5. Synergistic Multi-Agent Framework with Trajectory Learning for Knowledge-Intensive TasksShengbin Yue, Siyuan Wang, Wei Chen, Xuanjing Huang et al.AAAI 2025 · 25 citations
Dusk over the sea between two cliffs drawn in fine vertical lines