MPBR: Multimodal Progressive Bidirectional Reasoning for Open-Set Fine-Grained Recognition
Junfu Tan, Peiguang Jing, Yu Zhu, Yu Liu
Abstract
Open-set fine-grained recognition (OSFGR) is the core exploration of building open-world intelligent systems. The challenge lies in the gradual semantic drift during the transition from coarse-grained to fine-grained categories. However, although existing methods leverage hierarchical representations to assist progressive reasoning, they neglect semantic consistency across hierarchies. To address this, we propose a multimodal progressive bidirectional reasoning framework: (1) In forward reasoning, the model progressively refines visual features to capture hierarchical structural representations, while (2) in backward reasoning, variational inference integrates multimodal information to constrain consistency in category-aware latent spaces. This mechanism mitigates semantic drift through bidirectional information flow and cross-hierarchical feature consistency constraints. Extensive experiments on the iNat2021-OSR dataset demonstrate that our proposed method achieves superior performance.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 9b65c9fa-0f64-4cf3-8c7f-ee4e385ea046Builds on12
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Swin Transformer: Hierarchical Vision Transformer using Shifted WindowsZe Liu, Yutong Lin, Yue Cao, Han Hu et al.ICCV 2021 · 31,683 citations
- Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space ModelLianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang et al.ICML 2024 · 1,725 citations
- Open-Set Recognition: A Good Closed-Set Classifier is All You NeedSagar Vaze, Kai Han, Andrea Vedaldi, Andrew ZissermanICLR 2022 · 594 citations
- Conditional Variational Capsule Network for Open Set RecognitionYunrui Guo, Guglielmo Camporese, Wenjing Yang, Alessandro Sperduti et al.ICCV 2021 · 58 citations
Related papers
- From Coarse to Fine-Grained Open-Set RecognitionNico Lang, Vésteinn Snæbjarnarson, Elijah Cole, Oisin Mac Aodha et al.CVPR 2024
- Seeing as Experts Do: A Knowledge-Augmented Agent for Open-Set Fine-Grained Visual UnderstandingJunhan Chen, Zilu Zhou, Yujun Tong, Dongliang Chang et al.CVPR 2026 · 2 citations
- iVGR: Internalizing Visually Grounded Reasoning for MLLMs with Reinforcement LearningChangbin Zhang, Yujie Zhong, Qiang Zhang, Kai HanICML 2026
- Consistency-aware Feature Learning for Hierarchical Fine-grained Visual ClassificationRui Wang, Cong Zou, Weizhong Zhang, Zixuan Zhu et al.ACM MM 2023 · 6 citations
- Hypergraph-guided Intra- and Inter-category Relation Modeling for Fine-grained Visual RecognitionLu Chen, Qiangchang Wang, Zhaohui Li, Yilong YinACM MM 2024 · 2 citations
