Lune

ACL Findings2022

Debiasing Event Understanding for Visual Commonsense Tasks

Minji Seo, YeonJoon Jung, Seungtaek Choi, Seung-won Hwang, Bei Liu

Abstract

We study event understanding as a critical step towards visual commonsense tasks. Meanwhile, we argue that current object-based event understanding is purely likelihood-based, leading to incorrect event prediction, due to biased correlation between events and objects. We propose to mitigate such biases with do-calculus, proposed in causality research, but overcoming its limited robustness, by an optimized aggregation with association-based prediction. We show the effectiveness of our approach, intrinsically by comparing our generated events with ground-truth event annotation, and extrinsically by downstream commonsense tasks.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Counterfactual Inference for Text Classification Debiasing cites Visual Commonsense R-CNN
  • Causal Attention for Vision-Language Tasks cites Visual Commonsense R-CNN
  • DeVLBert: Learning Deconfounded Visio-Linguistic Representations cites Visual Commonsense R-CNN
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on5

  1. Large-Scale Adversarial Training for Vision-and-Language Representation LearningZhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu et al.NeurIPS 2020 · 561 citations
  2. DeVLBert: Learning Deconfounded Visio-Linguistic RepresentationsShengyu Zhang, Tan Jiang, Tan Wang, Kun Kuang et al.ACM MM 2020 · 66 citations
  3. Visual Commonsense R-CNNTan Wang, Jianqiang Huang, Hanwang Zhang, Qianru SunCVPR 2020
  4. Counterfactual Inference for Text Classification DebiasingChen Qian, Fuli Feng, Lijie Wen, Chunping Ma et al.ACL 2021
  5. Causal Attention for Vision-Language TasksXu Yang, Hanwang Zhang, Guojun Qi, Jianfei CaiCVPR 2021

Related papers

  1. Towards Causal Deep Learning for Vulnerability DetectionMd Mahbubur Rahman, Ira Ceka, Chengzhi Mao, Saikat Chakraborty et al.ICSE 2024 · 22 citations
  2. ROCK: Causal Inference Principles for Reasoning about Commonsense CausalityJiayao Zhang, Hongming Zhang, Weijie J. Su, Dan RothICML 2022 · 28 citations
  3. Causal Intervention for Abstractive Related Work GenerationJiachang Liu, Qi Zhang, Chongyang Shi, Usman Naseem et al.EMNLP Findings 2023 · 7 citations
  4. De-Confounded Variational Encoder-Decoder for Logical Table-to-Text GenerationWenqing Chen, Jidong Tian, Yitian Li, Hao He et al.ACL 2021
  5. Deconfounded Multimodal Learning for Spatio-temporal Video GroundingJiawei Wang, Zhanchang Ma, Da Cao, Yuquan Le et al.ACM MM 2023 · 7 citations
Dusk over the sea between two cliffs drawn in fine vertical lines