An Event-tailored State-Space Based Model for Pedestrian Detection

Event cameras, as emerging bio-inspired sensors, endow us with a unique scene perception capability with sub-millisecond latency in challenging environments, such as high-dynamic range and motion blur, as to which a plausible yet efficient exploration on spatiotemporal characteristics of the sparse, asynchronous event data remains an open problem. Event-based pedestrian detection, considered as a promising alternate for road safety in autonomous driving, is chosen as the testbed in this paper for pursuing a specific event-tailored spatiotemporal model. Note that, heterogeneous architectures are generally used in literature, such as building on a CNN/Transformer-style model for capturing the spatial features and a RNN/LSTM model for mining the temporal coherence, respectively. However, existing methods still face significant limitations, particularly as deployed in multi-rate dynamic environments, characterized by pronounced sparsity patterns in slow-motion or other scenarios. As such, a homogeneous neural network for robust pedestrian detection is proposed, with Event-tailored Recurrent Spatiotemporal State-Space Module (ERS3M) as the core innovation, for a joint meticulous modeling of spatiotemporal sparsity and dynamics over event data. On one hand, inspired by Vision Mamba, ERS3M is equipped with adaptive spatiotemporal state propagation as well as multi-directional compensatory scanning, enabling elaborate detection even as to observation intervals with extremely limited events triggered. On the other hand, ERS3 M is augmented with an additional block termed Temporal-Entropy Synergy, offering a collaborative spatiotemporal event purification mechanism, so as to enhance the probability credibility of event streams in visual semantics considering their complicated dynamics. Finally, ERS3M ends with an aliasing-alleviated S5 block to transit information between consecutive time steps, facilitating the temporal consistent pedestrian detection. Evaluations on the PEDRo dataset demonstrate that, the proposed detection method with ERS3 M as backbone has achieved a comparable or even superior performance to state-of-the-art approaches in terms of both accuracy and efficiency.