AnyEdit++: Adaptive Long-Form Knowledge Editing via Bayesian Surprise
Bowen Tian, Caixue He, Jiemin Wu, Jingying Wang, Wenshuo Chen, Zexi Li, Yutao Yue
Abstract
Editing complex, long-form knowledge in Large Language Models remains a significant challenge due to the difficulty of maintaining generation coherence. Existing autoregressive methods like AnyEdit alleviate length constraints but rely on Fixed-window Chunking, which disregards logical structure and compromises consistency. To address this, we present AnyEdit++, a structure-aware framework incorporating Bayes-Chunk, an adaptive segmentation mechanism that dynamically identifies semantic boundaries based on Bayesian Surprise. We underpin this approach with a theoretical framework establishing two key principles: (1) Structural Independence: we prove that cross-segment interference is minimized when anchor keys are geometrically orthogonal (a condition naturally satisfied by our surprisal-based boundaries but violated by fixed windows), and (2) Causal Locality: we demonstrate that updates injected at these semantic peaks yield strictly superior control compared to arbitrary split points. Extensive experiments across mathematical reasoning, code generation, and narrative tasks demonstrate that AnyEdit++ achieves superior performance and robustness compared to state-of-the-art baselines, validating that structural awareness is critical for effective long-form knowledge editing. Our code is available at Github.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on16
- Finetuned Language Models are Zero-Shot LearnersJason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu et al.ICLR 2022 · 4,966 citations
- Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
- Memory-Based Model Editing at ScaleEric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning et al.ICML 2022 · 520 citations
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value AdaptorsTom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim et al.NeurIPS 2023 · 349 citations
- Editable Neural NetworksAnton Sinitsin, Vsevolod Plokhotnyuk, Dmitry V. Pyrkin, Sergei Popov et al.ICLR 2020 · 210 citations
Related papers
- AnyEdit: Edit Any Knowledge Encoded in Language ModelsHoucheng Jiang, Junfeng Fang, Ningyu Zhang, Mingyang Wan et al.ICML 2025
- ALEX: A Light Editing-knowledge ExtractorMinghu Wang, Shuliang Zhao, Yuanyuan Zhao, Hongxia XuAAAI 2026
- Towards Meta-Cognitive Knowledge Editing for Multimodal LLMsZhaoyu Fan, Kaihang Pan, Mingze Zhou, Bosheng Qin et al.WWW 2026
- Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language ModelsJingcheng Deng, Zihao Wei, Liang Pang, Hanxing Ding et al.ICLR 2025
- SAKR-Edit: Scene-Aware Knowledge Reasoning for Text-to-Image EditingJiawen Wang, Jianjun Li, Zhiyuan Ma, Ruixia BaiACM MM 2025
