Tackling the Length Barrier: Dynamic Context Browsing for Knowledge-Intensive Task
Hongjin Qian, Zheng Liu, Peitian Zhang, Kelong Mao, Yujia Zhou, Xu Chen, Zhicheng Dou
Abstract
Knowledge-intensive tasks often require complex reasoning and contextual understanding over long contexts. However, the learning and deployment of long-LLMs remains a challenging problem despite recent progresses. In this work, we propose that the short LLMs have great potentiality for solving knowledge-intensive tasks that have long context, i.e. they can be solved by purely working with oracle short-contexts within the input long-context. On top of this argument, we propose a framework called DCISO DynamiC knowledge-Intensive task S>Olver), which enables a short-LLM to address the knowledge-intensive tasks with long context via dynamic context browsing. In our framework, the short-LLM prompts itself to reason for two critical decisions: 1) how to access to the appropriate part of context within the input, 2) how to make effective use of the accessed context. By adaptively accessing and utilizing the context based on the presented tasks, DCISO can serve as a general framework to handle diversified knowledge-intensive long-context problems. We comprehensively evaluate different types of tasks from popular long-context benchmarks, where DCISO is able to achieve a substantially improved performance. Our codes will be released at this repository.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- A Survey on In-context Learning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAG cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Reflexion: language agents with verbal reinforcement learning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Cited by top-tier papers1
Builds on13
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
- YaRN: Efficient Context Window Extension of Large Language ModelsBowen Peng, Jeffrey Quesnelle, Honglu Fan, Enrico ShippoleICLR 2024 · 508 citations
- A Survey on In-context LearningQingxiu Dong, Lei Li, Damai Dai, Ce Zheng et al.EMNLP 2024 · 479 citations
Related papers
- DocAgent: An Agentic Framework for Multi-Modal Long-Context Document UnderstandingLi Sun, Liu He, Shuyue Jia, Yangfan He et al.EMNLP 2025 · 1 citation
- LIFT: A Novel Framework for Enhancing Long-Context Understanding of LLMs via Long Input Fine-TuningYansheng Mao, Yufei Xu, Jiaqi Li, Fanxu Meng et al.ICML 2026 · 7 citations
- SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference OptimizationHuashan Sun, Shengyi Liao, Yansen Han, Yu Bai et al.ICLR 2026 · 9 citations
- PRISM: Efficient Long-Range Reasoning With Short-Context LLMsDulhan Jayalath, James Bradley Wendt, Nicholas Monath, Sandeep Tata et al.EMNLP 2025
- Synergistic Multi-Agent Framework with Trajectory Learning for Knowledge-Intensive TasksShengbin Yue, Siyuan Wang, Wei Chen, Xuanjing Huang et al.AAAI 2025 · 25 citations
