SEAL: A Single-Event Architecture for In-Sensor Visual Localization

Image sensors have low costs and broad applications, but the large data volume they generate can result in significant energy and latency overheads during data transfer, storage, and processing.This paper explores how shifting from traditional binary encoding to delay-based codes early on can address these inefficiencies, enabling keypoint detection and tracking within digital pixel sensors.The result is SEAL, a Single-Event Architecture for In-Sensor Localization.SEAL optimizes the entire pipeline between the pixel array and the sensor-processor interface by introducing a temporal processor co-designed with analog-to-time converters, followed by a custom heavily quantized frontend processor.Its implementation is fully digital, relying on off-the-shelf CMOS cells and EDA tools, and adheres to race logic's single-wire-per-variable and single-eventper-wire policies to maximize energy and area efficiency wherever possible.Our evaluation-combining analog and digital simulations, FPGA prototyping, and an end-to-end system analysis incorporating a host processor for visual inertial odometry (VIO) backend tasks-demonstrates a 16-61× reduction in the latency of keypoint detection and tracking compared to software baselines running on the host processor, and a 7× reduction in energy consumption compared to a standard digital pixel sensor without processing capabilities.Meanwhile, SEAL preserves robust tracking accuracy: on the EuRoC dataset, the average root mean square absolute trajectory error decreases by 1.0 cm for HybVIO and increases by just 0.3 cm for VINS-Mono compared to their original implementations.