Seeing Motion Through Polarity for Event-based Action Recognition
Meiqi Cao, Jiachao Zhang, Xin Jiang, Rui Yan, Yazhou Yao, Zechao Li, Xiangbo Shu
Abstract
Event-based Action Recognition (EAR) provides a promising pathway for understanding dynamic behaviors under challenging conditions. Recent progress in vision-language models has introduced a cross-modal learning paradigm into EAR, enabling models to associate event streams with textual semantics for enhancing conceptual understanding. However, existing methods typically overlook the intrinsic polarity-driven motion cues that are fundamental to event data, leading to suboptimal spatiotemporal representations. To address this limitation, we propose a POlarity Knowledge Enhanced framework (POKER), which explicitly incorporates event polarity-aware motion knowledge across visual and textual modalities. POKER consists of two synergistic components: Polarity Motion Capturer (PMC) and Polarity Motion Reasoner (PMR). Specifically, PMC decouples positive and negative polarities to capture polaritysensitive motion cues, while PMR semantically analyzes polarity-induced motion dynamics via large language models. Through the polarity alignment, POKER couples semantic reasoning with visual dynamics, achieving more discriminative representations. Extensive experiments on multiple benchmarks demonstrate that POKER enhances performance across diverse event representations.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 503c9d82-ec2d-46a2-ab88-0197dbfa5821Builds on18
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- PointMamba: A Simple State Space Model for Point Cloud AnalysisDingkang Liang, Xin Zhou, Wei Xu, Xingkui Zhu et al.NeurIPS 2024 · 380 citations
- GET: Group Event Transformer for Event-Based VisionYansong Peng, Yueyi Zhang, Zhiwei Xiong, Xiaoyan Sun et al.ICCV 2023 · 86 citations
- HARDVS: Revisiting Human Activity Recognition with Dynamic Vision SensorsXiao Wang, Zongzhen Wu, Bo Jiang, Zhimin Bao et al.AAAI 2024 · 80 citations
- Delving into Multimodal Prompting for Fine-Grained Visual ClassificationXin Jiang, Hao Tang, Junyao Gao, Xiaoyu Du et al.AAAI 2024 · 71 citations
Related papers
- Domain Knowledge Enhanced Vision-Language Pretrained Model for Dynamic Facial Expression RecognitionLiupeng Li, Yuhua Zheng, Shupeng Liu, Xiaoyin Xu et al.ACM MM 2024 · 4 citations
- Semore: VLM-guided Enhanced Semantic Motion Representations for Visual Reinforcement LearningWentao Wang, Chunyang Liu, Kehua Sheng, Bo Zhang et al.AAAI 2026
- Exploiting Frequency Dynamics for Enhanced Multimodal Event-Based Action RecognitionMeiqi Cao, Xiangbo Shu, Xin Jiang, Rui Yan et al.ICCV 2025 · 1 citation
- EvdCLIP: Improving Vision-Language Retrieval with Entity Visual Descriptions from Large Language ModelsGuanghao Meng, Sunan He, Jinpeng Wang, Tao Dai et al.AAAI 2025 · 5 citations
- PeVL: Pose-Enhanced Vision-Language Model for Fine-Grained Human Action RecognitionHaosong Zhang, Mei Chee Leong, Liyuan Li, Weisi LinCVPR 2024
