Lune

CVPR2025Top-tier venue

HUSH: Holistic Panoramic 3D Scene Understanding using Spherical Harmonics

Jongsung Lee, Harin Park, Byeong-Uk Lee, Kyungdon Joo

Abstract

Motivated by the efficiency of spherical harmonics (SH) in representing various physical phenomena, we propose a Holistic panoramic 3D scene Understanding framework using Spherical Harmonics, dubbed as HUSH. Our approach focuses on a unified framework adaptable to various 3D scene understanding tasks via SH bases. To achieve this, we first estimate SH coefficients, allowing for the adaptive configuration of the SH bases specific to each scene. HUSH then employs a hierarchical attention module that uses SH bases as queries to generate comprehensive scene features by integrating these scene-adaptive SH bases with image features. Additionally, we introduce an SH basis index module that adaptively emphasizes relevant SH bases to produce task-relevant features, enhancing the versatility of HUSH across different scene understanding tasks. Finally, by combining the scene features with task-relevant features in the task-specific heads, we perform various scene understanding tasks, including depth, surface normal and room layout estimation. Experiments demonstrate that HUSH achieves state-of-the-art performance on depth estimation benchmarks, highlighting the robustness and scalability of using SH in panoramic 3D scene understanding. For more information, you can visit our project page https://vision3d-lab.github.io/hush/ .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • DA2: Depth Anything in Any Direction cites OmniFusion: 360 Monocular Depth Estimation via Geometry-Aware Fusion
  • Depth Any Panoramas: A Foundation Model for Panoramic Depth Estimation cites DA2: Depth Anything in Any Direction
  • Depth Any Panoramas: A Foundation Model for Panoramic Depth Estimation cites OmniFusion: 360 Monocular Depth Estimation via Geometry-Aware Fusion
  • Swin Transformer: Hierarchical Vision Transformer using Shifted Windows cites Deformable DETR: Deformable Transformers for End-to-End Object Detection
  • 3D Gaussian Splatting for Real-Time Radiance Field Rendering cites PlenOctrees for Real-time Rendering of Neural Radiance Fields
  • MTPano: Multi-Task Panoramic Scene Understanding via Label-Free Integration of Dense Prediction Priors cites DA2: Depth Anything in Any Direction
  • MTPano: Multi-Task Panoramic Scene Understanding via Label-Free Integration of Dense Prediction Priors cites Depth Any Panoramas: A Foundation Model for Panoramic Depth Estimation
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Depth Any Panoramas: A Foundation Model for Panoramic Depth EstimationXin Lin, Meixi Song, Dizhe Zhang, Wenxuan Lu et al.CVPR 2026 · 27 citations
  2. DA2: Depth Anything in Any DirectionHaodong Li, Wangguandong Zheng, Jing He, Yuhao Liu et al.ICLR 2026 · 23 citations
  3. MTPano: Multi-Task Panoramic Scene Understanding via Label-Free Integration of Dense Prediction PriorsJingdong Zhang, Xiaohang Zhan, Lingzhi Zhang, Yizhou Wang et al.SIGGRAPH 2026

Builds on17

  1. Swin Transformer: Hierarchical Vision Transformer using Shifted WindowsZe Liu, Yutong Lin, Yue Cao, Han Hu et al.ICCV 2021 · 31,683 citations
  2. Deformable DETR: Deformable Transformers for End-to-End Object DetectionXizhou Zhu, Weijie Su, Lewei Lu, Bin Li et al.ICLR 2021 · 7,353 citations
  3. 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George DrettakisSIGGRAPH 2023 · 5,687 citations
  4. PlenOctrees for Real-time Rendering of Neural Radiance FieldsAlex Yu, Ruilong Li, Matthew Tancik, Hao Li et al.ICCV 2021 · 1,284 citations
  5. OmniFusion: 360 Monocular Depth Estimation via Geometry-Aware FusionYuyan Li, Yuliang Guo, Zhixin Yan, Xinyu Huang et al.CVPR 2022 · 79 citations

Related papers

  1. PanoContext-Former: Panoramic Total Scene Understanding with a TransformerYuan Dong, Chuan Fang, Liefeng Bo, Zilong Dong et al.CVPR 2024
  2. SPHERE: Semantic-PHysical Engaged REpresentation for 3D Semantic Scene CompletionZhiwen Yang, Yuxin PengACM MM 2025
  3. DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based OptimizationCheng Zhang, Zhaopeng Cui, Cai Chen, Shuaicheng Liu et al.ICCV 2021 · 43 citations
  4. Going Beyond Multi-Task Dense Prediction with Synergy Embedding ModelsHuimin Huang, Yawen Huang, Lanfen Lin, Ruofeng Tong et al.CVPR 2024
  5. HoHoNet: 360 Indoor Holistic Understanding With Latent Horizontal FeaturesCheng Sun, Min Sun, Hwann-Tzong ChenCVPR 2021
Dusk over the sea between two cliffs drawn in fine vertical lines