Hierarchical Fusion for Practical Ghost-free High Dynamic Range Imaging

Ghosting artifacts and missing content due to the over-/under-saturated regions caused by misalignments are generally considered as the two key challenges in high dynamic range (HDR) imaging for dynamic scenes. However, previous CNN-based methods directly reconstruct the HDR image from the input low dynamic range (LDR) images, with implicit ghost removal and multi-exposure image fusion in an end-to-end network structure. In this paper, we decompose HDR imaging into ghost-free image fusion and ghost-based image restoration, and propose a novel practical Hierarchical Fusion Network (HFNet), which contains three sub-networks: Mask Fusion Network, Mask Compensation Network, and Refine Network. Specifically, LDR images are linearly fused in Mask Fusion Network ignoring the misaligned regions. Then the ghost regions of fusion image are restored with mask compensation. Finally, all these results are refined in the third network. This strategy of divide and rule makes the proposed method significantly more tiny than previous methods. Experiments on different datasets show that superior performance of HFNet with 9x fewer FLOPs, 4x fewer parameters and 3x faster inference speed than the existing methods while providing comparable accuracy. And it achieves state-of-the-art quantitative and qualitative results while applied with similar FLOPs.