Lune

CVPR Findings2026

CADRNet: Cognitively-Inspired Active Vision for 3D Reasoning Segmentation via Differentiable Rendering

Zai Yang Yu, Changshuo Wang, Yuan Shi, Linjun Sun, Shu Wei, Tingran Wang, Wangyu Wu, Yanjie Li, Weijun Li

Abstract

Segmenting 3D objects based on language queries-referred to as 3D reasoning segmentation-requires resolving ambiguities present in both textual queries and the geometric representations of point clouds. Current approaches typically employ Multimodal Large Language Models (MLLMs) to align cross-modal representations and alleviate these ambiguities. However, as MLLMs are predominantly pretrained on image-text pairs, they inherently struggle when processing unstructured point cloud data. This domain gap leads to two observable issues: (1) erroneous spatial grounding of language queries within 3D scenes; and (2) imprecise object boundary delineation.

Consequently, MLLMs fail to spatially ground the implicit semantic intent of instructions, resulting in inaccurate segmentations. To address these challenges, this paper introduces CADRNet. Its architecture is inspired by the theory of active vision, which examines how biological systems selectively sample visual information to efficiently locate targets. CADRNet employs a text-guided differentiable renderer to establish a parametric mapping from language semantics to camera configurations, actively synthesizing task-relevant 2D projections that align with the query intent and are natively compatible with MLLMs. CADRNet comprises two primary modules: (1) the Viewpoint Position Adaptive Learning (VPAL) module, which optimizes camera parameters through end-to-end gradient updates driven by the segmentation objective; and (2) the Semantic Coherence Grouping Fusion (SCGF) module, which aggregates appearance

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • OpenMask3D: Open-Vocabulary 3D Instance Segmentation cites Learning Transferable Visual Models From Natural Language Supervision
  • Learning Transferable Visual Models From Natural Language Supervision cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on20

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
  3. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  4. ScanNet++: A High-Fidelity Dataset of 3D Indoor ScenesChandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, Angela DaiICCV 2023 · 659 citations
  5. OpenMask3D: Open-Vocabulary 3D Instance SegmentationAyça Takmaz, Elisabetta Fedele, Robert W. Sumner, Marc Pollefeys et al.NeurIPS 2023 · 389 citations

Related papers

  1. TVDRNet: Text-driven Viewpoint Optimization via Differentiable Rendering for 3D Reasoning SegmentationTingran Wang, Changshuo Wang, Pinjie Xu, ZhangHuang et al.ICML 2026
  2. MLLM-For3D: Adapting Multimodal Large Language Model for 3D Reasoning SegmentationJiaxin Huang, Runnan Chen, Ziwen Li, Zhengqing Gao et al.NeurIPS 2025 · 18 citations
  3. Enhancing Spatial Reasoning in Multimodal Large Language Models Through Reasoning-Based SegmentationZhenhua Ning, Zhuotao Tian, Shaoshuai Shi, Guangming Lu et al.ICCV 2025
  4. All in One: Visual-Description-Guided Unified Point Cloud SegmentationZongyan Han, Mohamed El Amine Boudjoghra, Jiahua Dong, Jinhong Wang et al.ICCV 2025 · 1 citation
  5. RSVP: Reasoning Segmentation via Visual Prompting and Multi-modal Chain-of-ThoughtYi Lu, Jiawang Cao, Yongliang Wu, Bozheng Li et al.ACL 2025 · 15 citations
Dusk over the sea between two cliffs drawn in fine vertical lines