ACL Findings2022
Debiasing Event Understanding for Visual Commonsense Tasks
Minji Seo, YeonJoon Jung, Seungtaek Choi, Seung-won Hwang, Bei Liu
Abstract
We study event understanding as a critical step towards visual commonsense tasks. Meanwhile, we argue that current object-based event understanding is purely likelihood-based, leading to incorrect event prediction, due to biased correlation between events and objects. We propose to mitigate such biases with do-calculus, proposed in causality research, but overcoming its limited robustness, by an optimized aggregation with association-based prediction. We show the effectiveness of our approach, intrinsically by comparing our generated events with ground-truth event annotation, and extrinsically by downstream commonsense tasks.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Counterfactual Inference for Text Classification Debiasing cites Visual Commonsense R-CNN
- Causal Attention for Vision-Language Tasks cites Visual Commonsense R-CNN
- DeVLBert: Learning Deconfounded Visio-Linguistic Representations cites Visual Commonsense R-CNN
Builds on5
- Large-Scale Adversarial Training for Vision-and-Language Representation LearningZhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu et al.NeurIPS 2020 · 561 citations
- DeVLBert: Learning Deconfounded Visio-Linguistic RepresentationsShengyu Zhang, Tan Jiang, Tan Wang, Kun Kuang et al.ACM MM 2020 · 66 citations
- Visual Commonsense R-CNNTan Wang, Jianqiang Huang, Hanwang Zhang, Qianru SunCVPR 2020
- Counterfactual Inference for Text Classification DebiasingChen Qian, Fuli Feng, Lijie Wen, Chunping Ma et al.ACL 2021
- Causal Attention for Vision-Language TasksXu Yang, Hanwang Zhang, Guojun Qi, Jianfei CaiCVPR 2021
Related papers
- Towards Causal Deep Learning for Vulnerability DetectionMd Mahbubur Rahman, Ira Ceka, Chengzhi Mao, Saikat Chakraborty et al.ICSE 2024 · 22 citations
- ROCK: Causal Inference Principles for Reasoning about Commonsense CausalityJiayao Zhang, Hongming Zhang, Weijie J. Su, Dan RothICML 2022 · 28 citations
- Causal Intervention for Abstractive Related Work GenerationJiachang Liu, Qi Zhang, Chongyang Shi, Usman Naseem et al.EMNLP Findings 2023 · 7 citations
- De-Confounded Variational Encoder-Decoder for Logical Table-to-Text GenerationWenqing Chen, Jidong Tian, Yitian Li, Hao He et al.ACL 2021
- Deconfounded Multimodal Learning for Spatio-temporal Video GroundingJiawei Wang, Zhanchang Ma, Da Cao, Yuquan Le et al.ACM MM 2023 · 7 citations
