VQ-Insight: Teaching VLMs for AI-Generated Video Quality Understanding via Progressive Visual Reinforcement Learning
Xuanyu Zhang, Weiqi Li, Shijie Zhao, Junlin Li, Li Zhang, Jian Zhang
摘要
Recent advances in AI-generated content (AIGC) have led to the emergence of powerful text-to-video generation models. Despite these successes, evaluating the quality of AIGC-generated videos remains challenging due to limited generalization, lack of temporal awareness, heavy reliance on large-scale annotated datasets, and the lack of effective interaction with generation models. Most current approaches rely on supervised fine-tuning of vision-language models (VLMs), which often require large-scale annotated datasets and tend to decouple understanding and generation. To address these shortcomings, we propose VQ-Insight, a novel reasoning-style VLM framework for AIGC video quality assessment. Our approach features: (1) a progressive video quality learning scheme that combines image quality warm-up, general task-specific temporal learning, and joint optimization with the video generation model; (2) the design of multi-dimension scoring rewards, preference comparison rewards, and temporal modeling rewards to enhance both generalization and specialization in video quality evaluation. Extensive experiments demonstrate that VQ-Insight consistently outperforms state-of-the-art baselines in preference comparison, multi-dimension scoring, and natural video scoring, bringing significant improvements for video generation tasks.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper7
- Reasoning as Representation: Rethinking Visual Reinforcement Learning in Image Quality AssessmentShijie Zhao, Xuanyu Zhang, Weiqi Li, Junlin Li 等ICLR 2026 · 被引用 20 次
- VQAThinker: Exploring Generalizable and Explainable Video Quality Assessment via Reinforcement LearningLinhan Cao, Wei Sun, Weixia Zhang, Xiangyang Zhu 等AAAI 2026 · 被引用 6 次
- UARE: A Unified Vision-Language Model for Image Quality Assessment, Restoration, and EnhancementWeiqi Li, Xuanyu Zhang, Bin Chen, Jingfen Xie 等CVPR 2026 · 被引用 5 次
- ReAlign: Generalizable Image Forgery Detection via Reasoning-Aligned RepresentationQing Huang, Zhipei Xu, Xuanyu Zhang, Xiangyu Yu 等CVPR 2026 · 被引用 3 次
- MDS-VQA: Model-Informed Data Selection for Video Quality AssessmentJian Zou, Xiaoyu Xu, Zhihua Wang, Yilin Wang 等CVPR 2026 · 被引用 1 次
它引用的顶会 Paper21
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning 等NeurIPS 2023 · 被引用 10,924 次
- SlowFast Networks for Video RecognitionChristoph Feichtenhofer, Haoqi Fan, Jitendra Malik, Kaiming HeICCV 2019 · 被引用 4,104 次
- Exploring CLIP for Assessing the Look and Feel of ImagesJianyi Wang, Kelvin C. K. Chan, Chen Change LoyAAAI 2023 · 被引用 1,208 次
- CLIPScore: A Reference-free Evaluation Metric for Image CaptioningJack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras 等EMNLP 2021 · 被引用 937 次
- Flow-GRPO: Training Flow Matching Models via Online RLJie Liu, Gongye Liu, Jiajun Liang, Yangguang Li 等NeurIPS 2025 · 被引用 647 次
相关 Paper
- AIGV-Assessor: Benchmarking and Evaluating the Perceptual Quality of Text-to-Video Generation with LMMJiarui Wang, Huiyu Duan, Guangtao Zhai, Juntong Wang 等CVPR 2025
- LOVE: Benchmarking and Evaluating Text-to-Video Generation and Video-to-Text InterpretationJiarui Wang, Huiyu Duan, Ziheng Jia, Zicheng Zhang 等ICML 2026 · 被引用 14 次
- Subjective-Aligned Dataset and Metric for Text-to-Video Quality AssessmentTengchuan Kou, Xiaohong Liu, Zicheng Zhang, Chunyi Li 等ACM MM 2024 · 被引用 29 次
- Q-Insight: Understanding Image Quality via Visual Reinforcement LearningWeiqi Li, Xuanyu Zhang, Shijie Zhao, Yabin Zhang 等NeurIPS 2025 · 被引用 117 次
- VF-Eval: Evaluating Multimodal LLMs for Generating Feedback on AIGC VideosTingyu Song, Tongyan Hu, Guo Gan, Yilun ZhaoACL 2025 · 被引用 1 次
