Lune

ACL Findings2025

LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs

Omkar Thawakar, Dinura Dissanayake, Ketan Pravin More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, Hisham Cholakkal, Ivan Laptev

Abstract

Step-by-step reasoning is crucial for solving complex visual tasks, yet existing approaches lack a comprehensive framework for evaluating this capability and do not emphasize stepwise problem-solving. To this end, we propose a comprehensive framework for advancing multi-step visual reasoning in large multimodal models (LMMs) through three key contributions. First, we introduce a Visual Reasoning Chain Benchmark (VRC-Bench), a comprehensive benchmark for multi-step visual reasoning, covering eight diverse categories and over 4k verified reasoning steps to rigorously evaluate LLMs' ability to reason accurately and interpretably across multiple steps. Second, we propose a fine-grained visual reasoning metric that evaluates correctness and logical coherence at each step, providing deeper insights beyond traditional accuracy metrics. Third, we introduce LlamaV-o1, a state-of-the-art multimodal step-by-step reasoning model trained using a multi-step curriculum learning approach. LlamaV-o1 is optimized for structured step-bystep reasoning Our LlamaV-o1 obtains a significant gain of around 9% averaged across six benchmarks compared to the baseline, thereby demonstrating the impact of introducing the proposed step-by-step visual reasoning. Further, it outperforms the recent Llava-CoT with an absolute gain of 3.8% averaged across six benchmarks, while being 5× faster during inference scaling. On the VRC-Bench, LlamaV-o1 achieves the best performance among all opensource reasoning LMMs in terms of both final accuracy and steps. Our benchmark, model, and code is available at https://github.com/ mbzuai-oryx/LlamaV-o1.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

63 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Self-Refine: Iterative Refinement with Self-Feedback cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Direct Preference Optimization: Your Language Model is Secretly a Reward Model cites Language Models are Few-Shot Learners
  • SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement cites Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models
  • SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement cites Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
  • NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation cites SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement
  • NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation cites Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models
  • More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models cites SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement
  • More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models cites Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models
  • More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models cites Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
  • Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models cites Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
  • Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering cites Language Models are Few-Shot Learners
  • Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers63

  1. Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language ModelsWenxuan Huang, Bohan Jia, Shaosheng Cao, Zheyu Ye et al.ICLR 2026 · 670 citations
  2. SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-ImprovementXiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu et al.NeurIPS 2025 · 158 citations
  3. NoisyRollout: Reinforcing Visual Reasoning with Data AugmentationXiangyan Liu, Jinjie Ni, Zijian Wu, Chao Du et al.NeurIPS 2025 · 104 citations
  4. More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning ModelsZhongxing Xu, Chengzhi Liu, Qingyue Wei, Juncheng Wu et al.NeurIPS 2025 · 103 citations
  5. Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited ViewsZhangquan Chen, Manyuan Zhang, Xinlei Yu, Xufang Luo et al.CVPR 2026 · 61 citations

Builds on27

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  3. Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
  4. Self-Refine: Iterative Refinement with Self-FeedbackAman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan et al.NeurIPS 2023 · 4,972 citations
  5. Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question AnsweringPan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu et al.NeurIPS 2022 · 2,727 citations

Related papers

  1. VRBench: A Benchmark for Multi-Step Reasoning in Long Narrative VideosJiashuo Yu, Yue Wu, Meng Chu, Zhifei Ren et al.ICCV 2025 · 3 citations
  2. RPTS: Tree-Structured Reasoning Process Scoring for Faithful Multimodal EvaluationHaofeng Wang, Yu ZhangAAAI 2026
  3. LlaVA-CoT: Let Vision Language Models Reason Step-By-StepGuowei Xu, Peng Jin, Ziang Wu, Hao Li et al.ICCV 2025 · 37 citations
  4. M³CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-ThoughtQiguang Chen, Libo Qin, Jin Zhang, Zhi Chen et al.ACL 2024 · 13 citations
  5. From Easy to Hard: The MIR Benchmark for Progressive Interleaved Multi-Image ReasoningHang Du, Jiayang Zhang, Guoshun Nan, Wendi Deng et al.ICCV 2025 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines