Task-Specific Scene Structure Representations

Understanding the informative structures of scenes is essential for low-level vision tasks. Unfortunately, it is difficult to obtain a concrete visual definition of the informative structures because influences of visual features are taskspecific. In this paper, we propose a single general neural network architecture for extracting task-specific structure guidance for scenes. To do this, we first analyze traditional spectral clustering methods, which computes a set of eigenvectors to model a segmented graph forming small compact structures on image domains. We then unfold the traditional graph-partitioning problem into a learnable network, named Scene Structure Guidance Network (SSGNet), to represent the task-specific informative structures. The SSGNet yields a set of coefficients of eigenvectors that produces explicit feature representations of image structures. In addition, our SSGNet is light-weight (∼ 56K parameters), and can be used as a plug-and-play module for off-the-shelf architectures. We optimize the SSGNet without any supervision by proposing two novel training losses that enforce task-specific scene structure generation during training. Our main contribution is to show that such a simple network can achieve state-of-the-art results for several low-level vision applications. We also demonstrate that our network generalizes well on unseen datasets, compared to existing methods which use structural embedding frameworks. We further propose a lighter version of SSGNet (∼ 29K parameters) for depth computation, SSGNet-D, and successfully execute it on edge computing devices like Jetson AGX Orin, improving the performance of baseline network, even in the wild, with little computational delay. Our source codes are available at https://github.com/jsshin98/SSGNet .