iSegMan: Interactive Segment-and-Manipulate 3D Gaussians
Yian Zhao, Wanshi Xu, Ruochong Zheng, Pengchong Qiao, Chang Liu, Jie Chen
Abstract
The efficient rendering and explicit nature of 3DGS promote the advancement of 3D scene manipulation. However, existing methods typically encounter challenges in controlling the manipulation region and are unable to furnish the user with interactive feedback, which inevitably leads to unexpected results. Intuitively, incorporating interactive 3D segmentation tools can compensate for this deficiency. Nevertheless, existing segmentation frameworks impose a preprocessing step of scene-specific parameter training, which limits the efficiency and flexibility of scene manipulation. To deliver a 3D region control module that is well-suited for scene manipulation with reliable efficiency, we propose interactive Segment-and-Manipulate 3D Gaussians (iSegMan), an interactive segmentation and manipulation framework that only requires simple 2D user interactions in any view. To propagate user interactions to other views, we propose Epipolar-guided Interaction Propagation (EIP), which innovatively exploits epipolar constraint for efficient and robust interaction matching. To avoid scene-specific training to maintain efficiency, we further propose the novel Visibility-based Gaussian Voting (VGV), which obtains 2D segmentations from SAM and models the region extraction as a voting game between 2D Pixels and 3D Gaussians based on Gaussian visibility. Taking advantage of the efficient and precise region control of EIP and VGV, we put forth a Manipulation Toolbox to implement various functions on selected regions, enhancing the controllability, flexibility and practicality of scene manipulation. Extensive results on 3D scene manipulation and segmentation tasks fully demonstrate the significant advantages of iSegMan. Project page is available at https://zhao-yian.github.io/iSegMan . Corresponding author. " Turn him into Tolkien Elf." " Turn him into Tolkien Elf." " Make him smiling." " Make the desk and vase blue." " Make the vase dark red." & " Enlarge" (a) Existing 3D Gaussian Manipulation Methods. (b) Our Interactive Segment-and-Manipulate 3D Gaussians.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Semantic Foam: Unifying Spatial and Semantic Scene Decomposition cites Emerging Properties in Self-Supervised Vision Transformers
- Semantic Foam: Unifying Spatial and Semantic Scene Decomposition cites Segment Anything
- Semantic Foam: Unifying Spatial and Semantic Scene Decomposition cites 3D Gaussian Splatting for Real-Time Radiance Field Rendering
- Semantic Foam: Unifying Spatial and Semantic Scene Decomposition cites Learning Transferable Visual Models From Natural Language Supervision
- Segment Anything cites Learning Transferable Visual Models From Natural Language Supervision
- Tune-Your-Style: Intensity-Tunable 3D Style Transfer with Gaussian Splatting cites Segment Anything
- Tune-Your-Style: Intensity-Tunable 3D Style Transfer with Gaussian Splatting cites 3D Gaussian Splatting for Real-Time Radiance Field Rendering
- Tune-Your-Style: Intensity-Tunable 3D Style Transfer with Gaussian Splatting cites Learning Transferable Visual Models From Natural Language Supervision
- Visual Instruction Tuning cites Learning Transferable Visual Models From Natural Language Supervision
- B^3-Seg: Camera-Free, Training-Free 3DGS Segmentation via Analytic EIG and Beta-Bernoulli Bayesian Updates cites 3D Gaussian Splatting for Real-Time Radiance Field Rendering
- B^3-Seg: Camera-Free, Training-Free 3DGS Segmentation via Analytic EIG and Beta-Bernoulli Bayesian Updates cites Learning Transferable Visual Models From Natural Language Supervision
Cited by top-tier papers3
- Tune-Your-Style: Intensity-Tunable 3D Style Transfer with Gaussian SplattingYian Zhao, Rushi Ye, Ruochong Zheng, Zesen Cheng et al.ICCV 2025 · 3 citations
- Semantic Foam: Unifying Spatial and Semantic Scene DecompositionAmr Sharafeldin, Aryan Mikaeili, Thomas Walker, Shrisudhan Govindarajan et al.CVPR 2026 · 1 citation
- B-Seg: Camera-Free, Training-Free 3DGS Segmentation via Analytic EIG and Beta-Bernoulli Bayesian UpdatesHiromichi Kamata, Samuel Arthur Munro, Fuminori HommaCVPR 2026
Builds on26
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Segment AnythingAlexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao et al.ICCV 2023 · 13,211 citations
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
- Emerging Properties in Self-Supervised Vision TransformersMathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou et al.ICCV 2021 · 8,921 citations
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George DrettakisSIGGRAPH 2023 · 5,687 citations
Related papers
- GaussianCut: Interactive segmentation via graph cut for 3D Gaussian SplattingUmangi Jain, Ashkan Mirzaei, Igor GilitschenskiNeurIPS 2024 · 35 citations
- E3Gen: Efficient, Expressive and Editable Avatars GenerationWeitian Zhang, Yichao Yan, Yunhui Liu, Xingdong Sheng et al.ACM MM 2024 · 4 citations
- Drag Your Gaussian: Effective Drag-Based Editing with Score Distillation for 3D Gaussian SplattingYansong Qu, Dian Chen, Xinyang Li, Xiaofan Li et al.SIGGRAPH 2025 · 14 citations
- 3DitScene: Editing Any Scene via Language-guided Disentangled Gaussian SplattingQihang Zhang, Yinghao Xu, Chaoyang Wang, Hsin-Ying Lee et al.ICLR 2025
- 3D Gaussian Editing with A Single ImageGuan Luo, Tian-Xing Xu, Ying-Tian Liu, Xiaoxiong Fan et al.ACM MM 2024 · 7 citations
