Beyond Sequential Tools: A Unified VLM Agent System for Photographic Post-Processing via Dynamic Multi-Expert Fusion
Honglin Xiong, Chenjie Zhu, Jianbiao Ding, Zixuan Ni, Wei Li, Zhenpeng Mi, Qian Wang
摘要
Real-world photographic post-processing is a formidable challenge due to the frequent co-occurrence of multiple, coupled image degradations. Current paradigms, such as monolithic "all-in-one" models, often face generalization bottlenecks, while recent agent-based systems suffer from time-consuming, sequential tool invocation and suboptimal coordination of isolated, single-task tools. To overcome these limitations, we propose a novel and efficient framework: a vision-language agent system for universal photographic post-processing. Our system employs a powerful Vision-Language Model (VLM) as an orchestrator agent to perform nuanced user intent understanding and in-depth degradation analysis. Based on its assessment, the VLM generates a structured plan, dynamically allocating weights to a suite of specialized expert LoRA modules. These experts, which adapt only the Key (K) and Value (V) matrices for enhanced composability, are then simultaneously merged into a pretrained diffusion backbone to execute a tailored restoration. To ensure perceptually optimal weights, we introduce a lightweight allocation branch trained on the VLM's features using Direct Preference Optimization (DPO) from human feedback. This dynamic fusion paradigm enables a synergistic, context-aware restoration in a single, efficient forward pass. Our method demonstrates state-of-the-art performance across a wide range of synthetic and real-world datasets with diverse degradations. Crucially, it exhibits remarkable zero-shot generalization, achieving excellent results on real-world data. Our code and weights will be made publicly available.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper15
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu 等ICLR 2022 · 被引用 18,833 次
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning 等NeurIPS 2023 · 被引用 10,924 次
- Restormer: Efficient Transformer for High-Resolution Image RestorationSyed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat 等CVPR 2022 · 被引用 3,348 次
- Uformer: A General U-Shaped Transformer for Image RestorationZhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou 等CVPR 2022 · 被引用 1,970 次
- MUSIQ: Multi-scale Image Quality TransformerJunjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar 等ICCV 2021 · 被引用 1,325 次
相关 Paper
- An Intelligent Agentic System for Complex Image Restoration ProblemsKaiwen Zhu, Jinjin Gu, Zhiyuan You, Yu Qiao 等ICLR 2025
- Hybrid Agents for Image RestorationBingchen Li, Xin Li, Yiting Lu, Zhibo ChenCVPR 2026 · 被引用 17 次
- FAPE-IR: Frequency-Aware Planning and Execution Framework for All-in-One Image RestorationJingren Liu, Shuning Xu, Qirui Yang, Yun Wang 等CVPR 2026 · 被引用 4 次
- RestoreAgent: Autonomous Image Restoration Agent via Multimodal Large Language ModelsHaoyu Chen, Wenbo Li, Jinjin Gu, Jingjing Ren 等NeurIPS 2024 · 被引用 49 次
- Language-driven All-in-one Adverse Weather RemovalHao Yang, Liyuan Pan, Yan Yang, Wei LiangCVPR 2024 · 被引用 28 次
