Efficient Indoor Depth Completion Network Using Mask-adaptive Gated Convolution
Tingxuan Huang, Jiacheng Miao, Shizhuo Deng, Tong Jia, Dongyue Chen
Abstract
Most indoor depth completion tasks rely on convolutional auto-encoders to reconstruct depth images, especially in areas with significant missing values. While traditional convolution treats valid and missing pixels equally, Partial Convolution (PConv) has mitigated this limitation. However, PConv fails to distinguish the varying degree of invalidity across different missing areas, which highlights the need for a more refined strategy. To solve this problem, we propose a novel system for indoor depth completion tasks that leverages Mask-adaptive Gated Convolution (MagaConv). MagaConv utilizes gated signals to selectively apply convolution kernels based on the characteristics of missing depth data. These gating signals are generated using shared convolution kernels that jointly process depth features and corresponding masks, ensuring coherent weight optimization. Additionally, the mask undergoes iterative updates according to predefined rules. To improve the fusion of depth and color information, we introduce a Bidirectional Aligning Projection (Bid-AP) module, which utilizes a bi-directional projection scheme with global spatialchannel attention mechanisms to filter out depth-irrelevant features from other modalities. Extensive experiments on popular benchmarks, including NYU-Depth V2, DIML, and SUN RGB-D, demonstrate that our model outperforms stateof-the-art methods in both accuracy and efficiency. The code is available at https://github.com/htx0601/MagaConv .
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext ba767321-82f1-4659-b64b-12c203697af0Cited by top-tier papers1
Ask how each one uses itBuilds on17
- Free-Form Image Inpainting With Gated ConvolutionJiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen et al.ICCV 2019 · 1,990 citations
- Fast Fourier ConvolutionLu Chi, Borui Jiang, Yadong MuNeurIPS 2020 · 842 citations
- CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth CompletionXinjing Cheng, Peng Wang, Chenye Guan, Ruigang YangAAAI 2020 · 270 citations
- Image Inpainting With Learnable Bidirectional Attention MapsChaohao Xie, Shaohui Liu, Chao Li, Ming-Ming Cheng et al.ICCV 2019 · 255 citations
- Depth Completion From Sparse LiDAR Data With Depth-Normal ConstraintsYan Xu, Xinge Zhu, Jianping Shi, Guofeng Zhang et al.ICCV 2019 · 249 citations
Related papers
- AGG-Net: Attention Guided Gated-convolutional Network for Depth Image CompletionDongyue Chen, Tingxuan Huang, Zhimin Song, Shizhuo Deng et al.ICCV 2023 · 16 citations
- ShapeConv: Shape-aware Convolutional Layer for Indoor RGB-D Semantic SegmentationJinming Cao, Hanchao Leng, Dani Lischinski, Danny Cohen-Or et al.ICCV 2021 · 186 citations
- DeCoTR: Enhancing Depth Completion with 2D and 3D AttentionsYunxiao Shi, Manish Kumar Singh, Hong Cai, Fatih PorikliCVPR 2024 · 7 citations
- Learning Joint 2D-3D Representations for Depth CompletionYun Chen, Bin Yang, Ming Liang, Raquel UrtasunICCV 2019 · 190 citations
- Aggregating Feature Point Cloud for Depth CompletionZhu Yu, Zehua Sheng, Zili Zhou, Lun Luo et al.ICCV 2023 · 42 citations
