HFR and HDR Video from Multi-Attenuated Spikes Using a Rapidly Rotating SpokeND Filter
Yakun Chang, Zhaojun Huang, Siqi Yang, Yeliduosi Xiaokaiti, Shikui Wei, Yao Zhao, Tiejun Huang, Boxin Shi
Abstract
Capturing scenes with both high dynamic range (HDR) and high-speed motion remains challenging for conventional cameras. Existing alternating-exposure approaches exacerbate temporal resolution loss, making them unsuitable for high-speed scenes. Consequently, current solutions typically fix spatial-varying attenuation levels or employ multiple sensors to maintain temporal resolution. In this paper, we leverage an ultra-high speed spike camera to enable spatial and temporal attenuation of incident light, thereby reconstructing high-frame-rate (HFR) and HDR video with a single sensor. We achieve this by placing a rapidly rotating spoke-pattern neutral density (SpokeND) filter in front of the camera, enabling each pixel to periodically capture multi-attenuated spikes. Building on these multi-attenuated spikes, we propose ReST-Net, which comprises the ReGain and ReFine modules. The ReGain module reconstructs spatially consistent frames by learning to recover relative gain from the multi-attenuated spikes, and the ReFine module removes temporal fluctuations to produce temporally consistent HDR videos. Extensive experiments on synthetic and real-world data demonstrate that our method can reconstruct HDR video at up to 2000 FPS.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext ad8cb44d-00cb-495c-bd35-4604c48d9599Builds on12
- HDR Video Reconstruction: A Coarse-to-fine Network and A Real-world Benchmark DatasetGuanying Chen, Chaofeng Chen, Shi Guo, Zhetong Liang et al.ICCV 2021 · 70 citations
- Optical Flow Estimation for Spiking CameraLiwen Hu, Rui Zhao, Ziluo Ding, Lei Ma et al.CVPR 2022 · 48 citations
- Self-Supervised Joint Dynamic Scene Reconstruction and Optical Flow Estimation for Spiking CameraShiyan Chen, Zhaofei Yu, Tiejun HuangAAAI 2023 · 22 citations
- Learning to Super-resolve Dynamic Scenes for Neuromorphic Spike CameraJing Zhao, Ruiqin Xiong, Jian Zhang, Rui Zhao et al.AAAI 2023 · 21 citations
- Joint Demosaicing and Denoising for Spike CameraYanchen Dong, Ruiqin Xiong, Jing Zhao, Jian Zhang et al.AAAI 2024 · 18 citations
Related papers
- 1000 FPS HDR Video with a Spike-RGB Hybrid CameraYakun Chang, Chu Zhou, Yuchen Hong, Liwen Hu et al.CVPR 2023
- Towards HDR and HFR Video from Rolling-Mixed-Bit SpikingsYakun Chang, Yeliduosi Xiaokaiti, Yujia Liu, Bin Fan et al.CVPR 2024
- Super Resolve Dynamic Scene from Continuous Spike StreamsJing Zhao, Jiyu Xie, Ruiqin Xiong, Jian Zhang et al.ICCV 2021 · 42 citations
- BulletTime4D: Towards High Spatio-Temporal Resolution Dynamic Scene Rendering via Spike-Guided Stereo VisionYiqian Chang, Haoran Xu, Qinghong Ye, Jianing Li et al.AAAI 2026
- Spk2ImgNet: Learning To Reconstruct Dynamic Scene From Continuous Spike StreamJing Zhao, Ruiqin Xiong, Hangfan Liu, Jian Zhang et al.CVPR 2021
