A Bit is All You Need! Efficient Video Capture via Single Bit Imaging
Kanchana Vaishnavi Gandikota, Michael Moeller, Andreas Kolb, Bhaskar Choubey, Paramanand Chandramouli
Abstract
We introduce a fundamentally new paradigm in video sensing, 1-bit computational video, that redefines the limits of imaging efficiency and performance. Instead of the conventional high-bit-depth capture, we show that one bit measurements captured by time-varying thresholding can be used to reconstruct full-bit-depth videos, eliminating the need for power-hungry, high-precision analog-to-digital conversion (ADC) at the sensor as well as reducing the energy consumption in data transmission. We propose thresholding strategies to effectively capture spatiotemporal dependencies in video streams. Despite the significant data compression at acquisition, we recover full-bit-depth videos with high fidelity through neural video reconstruction. Our method unlocks significant gains in memory efficiency, power savings, and data throughput reduction at the sensor, making it ideal for imaging systems with ultra-low-power requirements or high-speed video capture. We validate our framework on video recovery from simulated 1-bit measurements. Our work redefines the camera pipeline, potentially paving the way for gigapixel, kilohertz imaging systems on low-power sensor hardware.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on13
- Learned Step Size quantizationSteven K. Esser, Jeffrey L. McKinstry, Deepika Bablani, Rathinakumar Appuswamy et al.ICLR 2020 · 1,037 citations
- Recurrent Video Restoration Transformer with Guided Deformable AttentionJingyun Liang, Yuchen Fan, Xiaoyu Xiang, Rakesh Ranjan et al.NeurIPS 2022 · 318 citations
- XVFI: eXtreme Video Frame InterpolationHyeonjun Sim, Jihyong Oh, Munchurl KimICCV 2021 · 207 citations
- Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through EstimationZechun Liu, Kwang-Ting Cheng, Dong Huang, Eric P. Xing et al.CVPR 2022 · 108 citations
- Quanta burst photographySizhuo Ma, Shantanu Gupta, Arin C. Ulku, Claudio Bruschini et al.SIGGRAPH 2020 · 76 citations
Related papers
- An Efficient Deep Learning Accelerator for Compressed Video AnalysisYongchen Wang, Ying Wang, Huawei Li, Yinhe Han et al.DAC 2020 · 4 citations
- LeCA: In-Sensor Learned Compressive Acquisition for Efficient Machine Vision on the EdgeTianrui Ma, Adith Jagadish Boloor, Xiangxing Yang, Weidong Cao et al.ISCA 2023 · 27 citations
- Binarized Low-Light Raw Video EnhancementGengchen Zhang, Yulun Zhang, Xin Yuan, Ying FuCVPR 2024 · 10 citations
- NeRV: Neural Representations for VideosHao Chen, Bo He, Hanyu Wang, Yixuan Ren et al.NeurIPS 2021 · 430 citations
- EfficientSCI: Densely Connected Network with Space-time Factorization for Large-scale Video Snapshot Compressive ImagingLishun Wang, Miao Cao, Xin YuanCVPR 2023
