LangRef3DGS: Natural Language-Guided 3D Referential Segmentation from Partial Observations via 3D Gaussian Splatting
Xulun Ye, Qin Zhang, Kun Zhou
Abstract
Language-guided 3D segmentation is crucial for linking 3D perception with semantic understanding, yet it remains vulnerable to the sparse and occluded views common in real-world RGB-D data. To overcome this, we present a real-time framework that leverages 3D Gaussian Splatting (3DGS) to build a semantically continuous and differentiable embedding field from partial observations. Our approach integrates two key components: a Dirichlet Process (DP) for the adaptive discovery of novel object categories, and a gradient low-rank mechanism that enhances class separability by reducing feature redundancy. This combination enables robust open-vocabulary segmentation guided directly by text prompts. Extensive experiments on challenging benchmarks demonstrate that our method achieves strong performance, exhibiting superior accuracy, robustness to partial inputs, and a powerful capacity for novel class discovery.Our code and models are available at https://github.com/Tap12345/LangGS.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on21
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George DrettakisSIGGRAPH 2023 · 5,687 citations
- Instant neural graphics primitives with a multiresolution hash encodingThomas Müller, Alex Evans, Christoph Schied, Alexander KellerSIGGRAPH 2022 · 4,089 citations
- FastNeRF: High-Fidelity Neural Rendering at 200FPSStephan J. Garbin, Marek Kowalski, Matthew Johnson, Jamie Shotton et al.ICCV 2021 · 778 citations
- LERF: Language Embedded Radiance FieldsJustin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa et al.ICCV 2023 · 620 citations
Related papers
- Dr. Splat: Directly Referring 3D Gaussian Splatting via Direct Language Embedding RegistrationKim Jun-Seong, GeonU Kim, Kim Yu-Ji, Yu-Chiang Frank Wang et al.CVPR 2025
- FastLGS: Speeding Up Language Embedded Gaussians with Feature Grid MappingYuzhou Ji, He Zhu, Junshu Tang, Wuyi Liu et al.AAAI 2025 · 29 citations
- VaF-LangSplat: Voxel-Aware Fusion Language Gaussian SplattingChangzhou Li, Xinyu Yang, Weiguo Yang, Xinyi LiACM MM 2025
- Segment then Splat: Unified 3D Open-Vocabulary Segmentation via Gaussian SplattingYiren Lu, Yunlai Zhou, Yiran Qiao, Chaoda Song et al.NeurIPS 2025 · 9 citations
- ReferSplat: Referring Segmentation in 3D Gaussian SplattingShuting He, Guangquan Jie, Changshuo Wang, Yun Zhou et al.ICML 2025
