Eyes on the Road, Mind Beyond Vision: Context-Aware Multi-modal Enhanced Risk Anticipation
Jiaxun Zhang, Haicheng Liao, Yumu Xie, Chengyue Wang, Yanchen Guan, Bin Rao, Zhenning Li
Abstract
Accurate accident anticipation remains challenging when driver cognition and dynamic road conditions are underrepresented in predictive models. In this paper, we propose CAMERA (Context-Aware Multi-modal Enhanced Risk Anticipation), a multi-modal framework integrating dashcam video, textual annotations, and driver attention maps for robust accident anticipation. Unlike existing methods that rely on static or environment-centric thresholds, CAMERA employs an adaptive mechanism guided by scene complexity and gaze entropy, reducing false alarms while maintaining high recall in dynamic, multi-agent traffic scenarios. A hierarchical fusion pipeline with Bi-GRU (Bidirectional GRU) captures spatio-temporal dependencies, while a Geo-Context Vision-Language module translates 3D spatial relationships into interpretable, human-centric alerts. Evaluations on the DADA-2000 and benchmarks show that CAMERA achieves state-of-the-art performance, improving accuracy and lead time. These results demonstrate the effectiveness of modeling driver attention, contextual description, and adaptive risk thresholds to enable more reliable accident anticipation.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on9
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Segment Anything in High QualityLei Ke, Mingqiao Ye, Martin Danelljan, Yifan Liu et al.NeurIPS 2023 · 709 citations
- Uncertainty-based Traffic Accident Anticipation with Spatio-Temporal Relational LearningWentao Bao, Qi Yu, Yu KongACM MM 2020 · 191 citations
- Zero-Shot Reward Specification via Grounded Natural LanguageParsa Mahmoudieh, Deepak Pathak, Trevor DarrellICML 2022 · 69 citations
- DRIVE: Deep Reinforced Accident Anticipation with Visual ExplanationWentao Bao, Qi Yu, Yu KongICCV 2021 · 64 citations
Related papers
- CRASH: Crash Recognition and Anticipation System Harnessing with Context-Aware and Temporal Focus AttentionsHaicheng Liao, Haoyu Sun, Huanming Shen, Chengyue Wang et al.ACM MM 2024 · 10 citations
- When, Where, and What? A Benchmark for Accident Anticipation and Localization with Large Language ModelsHaicheng Liao, Yongkang Li, Chengyue Wang, Yanchen Guan et al.ACM MM 2024 · 11 citations
- Accident Anticipation via Temporal Occurrence PredictionTianhao Zhao, Yiyang Zou, Zihao Mao, Peilun Xiao et al.NeurIPS 2025 · 6 citations
- Predict and Resist: Long-Term Accident Anticipation Under Sensor NoiseXingcheng Liu, Bin Rao, Yanchen Guan, Chengyue Wang et al.AAAI 2026
- RiskProp: Collision-Anchored Self-Supervised Risk Propagation For Early Accident AnticipationYiyang Zou, Tianhao Zhao, Peilun Xiao, Hongyu Jin et al.CVPR 2026 · 4 citations
