ACL Findings2025
Self-Correction is More than Refinement: A Learning Framework for Visual and Language Reasoning Tasks
Jiayi He, Hehai Lin, Qingyun Wang, Yi R. Fung, Heng Ji
Abstract
While Vision-Language Models (VLMs) have shown remarkable abilities, they invariably generate flawed responses. Self-correction that instructs models to refine their outputs presents a promising solution to this issue. Previous studies have mainly concentrated on Large Language Models (LLMs), while the self-correction abilities of VLMs, particularly concerning both visual and linguistic information, remain largely unexamined. This study investigates the self-correction capabilities of VLMs during both inference and fine-tuning stages. We introduce a Self-Correction Learning (SCL) approach that enables VLMs to learn from their self-generated self-correction data through Direct Preference Optimization (DPO) without relying on external feedback, facilitating self-improvement. Experimental results demonstrate that although VLMs struggle to self-correct effectively during iterative inference without additional fine-tuning and external feedback, they can enhance their performance and avoid previous mistakes through preference fine-tuning when their generated self-correction data are categorized into preferred and disfavored samples. This study emphasizes that self-correction is not merely a refinement process; rather, it should enhance models' reasoning ability through additional training, enabling them to generate high-quality responses directly without further refinement. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
11 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and Generation cites Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and Generation cites MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
- Reflexion: language agents with verbal reinforcement learning cites Self-Refine: Iterative Refinement with Self-Feedback
- SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning cites LoRA: Low-Rank Adaptation of Large Language Models
- SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning cites Self-Refine: Iterative Refinement with Self-Feedback
- SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning cites Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning cites Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning cites MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
Cited by top-tier papers11
- Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual ReasoningYana Wei, Liang Zhao, Jianjian Sun, Kangheng Lin et al.NeurIPS 2025 · 39 citations
- DreamPRM: Domain-reweighted Process Reward Model for Multimodal ReasoningQi Cao, Ruiyi Wang, Ruiyi Zhang, Sai Ashish Somayajula et al.NeurIPS 2025 · 17 citations
- HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and GenerationLing Yang, Xinchen Zhang, Ye Tian, Shiyi Zhang et al.NeurIPS 2025 · 16 citations
- SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement LearningLin Zhang, Xianfang Zeng, Kangcong Li, Gang Yu et al.ICCV 2025 · 3 citations
- Forward-Only Continual LearningJiao Chen, Jiayi He, Fangfang Chen, Zuohong Lv et al.ACM MM 2025 · 1 citation
Builds on24
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
- Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
- Self-Refine: Iterative Refinement with Self-FeedbackAman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan et al.NeurIPS 2023 · 4,972 citations
- MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language ModelsDeyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li et al.ICLR 2024 · 3,079 citations
Related papers
- ReCot: Reflective Self-Correction Training for Mitigating Confirmation Bias in Large Vision-Language ModelsMengxue Qu, Yibo Hu, Kunyang Han, Yunchao Wei et al.ICCV 2025 · 3 citations
- Towards Self-Refinement of Vision-Language Models with Triangular ConsistencyYunlong Deng, Guangyi Chen, Tianpei Gu, Lingjing Kong et al.NeurIPS 2025 · 3 citations
- Calibrated Self-Rewarding Vision Language ModelsYiyang Zhou, Zhiyuan Fan, Dongjie Cheng, Sihan Yang et al.NeurIPS 2024 · 77 citations
- Sherlock: Self-Correcting Reasoning in Vision-Language ModelsYi Ding, Ruqi ZhangNeurIPS 2025 · 14 citations
- Can Large Vision-Language Models Correct Semantic Grounding Errors By Themselves?Yuan-Hong Liao, Rafid Mahmood, Sanja Fidler, David AcunaCVPR 2025
