EV-CGNet: Co-visible Focused 3D-guided 2D Event Keypoint Detection Network

Event keypoint detection has garnered significant attention due to its ability to establish spatial associations through matching, which is fundamental for various computer vision tasks. However, achieving robust event keypoint detection remains challenging due to the difficulty in balancing the exploitation of event data and compatibility with established algorithms. Moreover, the limited use of co-visible information often results in excessive keypoint detection in non-matching regions, leading to incorrect matches. To address these challenges, we propose a novel Co-visible Focused 3D-guided 2D Event Keypoint Detection Network (EV-CGNet), which mainly consists of a 3Dguided 2D feature prototype learning (G2PL) module and a co-visible region-focused detector and descriptor learning (CDDL) module. The proposed method enjoys several merits. First, the proposed G2PL module can leverage finegrained spatio-temporal cues from event points to guide the learning of event frame feature prototypes. Second, the proposed CDDL module can focus keypoint detection on covisible regions, ensuring accurate matches. Comprehensive experimental evaluations on six challenging benchmarks show that our method significantly outperforms state-ofthe-art event keypoint detection methods.

w/ matching priors (ours) Retain A Part w/o matching priors (SD2Event) (b) Impact of matching priors on event keypoint detection (b) Comparison of Existing Event Stream Processing Paradigms