Relation-R1: Progressively Cognitive Chain-of-Thought Guided Reinforcement Learning for Unified Relation Comprehension
Lin Li, Wei Chen, Jiahui Li, Kwang-Ting Cheng, Long Chen
Abstract
Recent advances in multi-modal large language models (MLLMs) have significantly improved object-level grounding and region captioning. However, they remain limited in visual relation understanding, struggling even with binary relation detection, let alone N-ary relations involving multiple semantic roles. The core reason is the lack of modeling for structural semantic dependencies among multi-entities, leading to over-reliance on language priors (e.g., defaulting to "person drinks a milk" if a person is merely holding it). To this end, we propose Relation-R1, the first unified relation comprehension framework that explicitly integrates cognitive chain-of-thought (CoT)-guided supervised fine-tuning (SFT) and group relative policy optimization (GRPO) within a reinforcement learning (RL) paradigm. Specifically, we first establish foundational reasoning capabilities via SFT, enforcing structured outputs with thinking processes. Then, GRPO is utilized to refine these outputs via multi-rewards optimization, prioritizing visual-semantic grounding over language-induced biases, thereby improving generalization capability. Furthermore, we investigate the impact of various CoT strategies within this framework, demonstrating that a specific-to-general progressive approach in CoT guidance further improves generalization, especially in capturing synonymous N-ary relations. Extensive experiments on widely-used PSG and SWiG datasets demonstrate that Relation-R1 achieves state-of-the-art performance in both binary and N-ary relation understanding.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext c7e9f6b6-1e77-456f-b814-34f2483e9ac8Cited by top-tier papers6
- SinkTrack: Attention Sink based Context Anchoring for Large Language ModelsXu Liu, Guikun Chen, Wenguan WangICLR 2026 · 4 citations
- Compositional Zero-shot Learning via Progressive Language-based ObservationsLin Li, Guikun Chen, Zhen Wang, Jun Xiao et al.ACM MM 2025 · 2 citations
- Interaction-Centric Knowledge Infusion and Transfer for Open Vocabulary Scene Graph GenerationLin Li, Chuhan Zhang, Dong Zhang, Chong Sun et al.NeurIPS 2025 · 1 citation
- PV-Ground: Text-Guided Point-Voxel Interaction for 3D Visual GroundingJunpeng Shang, Feifei Shao, Jun Xiao, Lin Li et al.CVPR 2026
- Reality vs Counterfactual: Multi-World Contrastive Reinforcement Learning for Enhancing MLLM's Theory of Mind in Egocentric VideosGuiyang Hou, Yihui Fu, Chen Wu, Xiang Huang et al.AAAI 2026
Builds on21
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
- MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language ModelsDeyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li et al.ICLR 2024 · 3,079 citations
- Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language ModelsWenxuan Huang, Bohan Jia, Shaosheng Cao, Zheyu Ye et al.ICLR 2026 · 670 citations
- GLaMM: Pixel Grounding Large Multimodal ModelHanoona Abdul Rasheed, Muhammad Maaz, Sahal Shaji Mullappilly, Abdelrahman M. Shaker et al.CVPR 2024 · 113 citations
- SGTR: End-to-end Scene Graph Generation with TransformerRongjie Li, Songyang Zhang, Xuming HeCVPR 2022 · 108 citations
Related papers
- UniMo: Unified Motion Generation and Understanding with Chain of ThoughtGuocun Wang, Kenkun Liu, Jing Lin, Guorui Song et al.AAAI 2026
- TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMsYunheng Li, Jing Cheng, Shaoyong Jia, Hangyi Kuang et al.NeurIPS 2025 · 18 citations
- Affordance-R1: Reinforcement Learning for Generalizable Affordance Reasoning in Multimodal Large Language ModelsHanqing Wang, Shaoyang Wang, Yiming Zhong, Zemin Yang et al.AAAI 2026 · 13 citations
- EARL: Towards a Unified Analysis-Guided Reinforcement Learning Framework for Egocentric Interaction Reasoning and Pixel GroundingYuejiao Su, Xinshen ZHANG, Zhen Ye, Lei Yao et al.ICML 2026
- Reinforcing Structured Chain-of-Thought for Video UnderstandingPeiyao Wang, Haotian Xu, Noranart Vesdapunt, Rui Hou et al.CVPR 2026 · 1 citation
