Lune

ACM MM2020Top-tier venue

Depth Guided Adaptive Meta-Fusion Network for Few-shot Video Recognition

Yuqian Fu, Li Zhang, Junke Wang, Yanwei Fu, Yu-Gang Jiang

Abstract

Humans can easily recognize actions with only a few examples given, while the existing video recognition models still heavily rely on the large-scale labeled data inputs. This observation has motivated an increasing interest in few-shot video action recognition, which aims at learning new actions with only very few labeled samples. In this paper, we propose a depth guided Adaptive Meta-Fusion Network for few-shot video recognition which is termed as AMeFu-Net. Concretely, we tackle the few-shot recognition problem from three aspects: firstly, we alleviate this extremely data-scarce problem by introducing depth information as a carrier of the scene, which will bring extra visual information to our model; secondly, we fuse the representation of original RGB clips with multiple non-strictly corresponding depth clips sampled by our temporal asynchronization augmentation mechanism, which synthesizes new instances at feature-level; thirdly, a novel Depth Guided Adaptive Instance Normalization (DGAdaIN) fusion module is proposed to fuse the two-stream modalities efficiently. Additionally, to better mimic the few-shot recognition process, our model is trained in the meta-learning way. Extensive experiments on several action recognition benchmarks demonstrate the effectiveness of our model.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

25 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Boosting Few-shot Action Recognition with Graph-guided Hybrid Matching cites Hybrid Relation Guided Set Matching for Few-shot Action Recognition
  • Boosting Few-shot Action Recognition with Graph-guided Hybrid Matching cites Motion-modulated Temporal Fragment Alignment Network For Few-Shot Action Recognition
  • M3Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action Recognition cites Hybrid Relation Guided Set Matching for Few-shot Action Recognition
  • M3Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action Recognition cites SlowFast Networks for Video Recognition
  • Hybrid Relation Guided Set Matching for Few-shot Action Recognition cites TSM: Temporal Shift Module for Efficient Video Understanding
  • Hybrid Relation Guided Set Matching for Few-shot Action Recognition cites SlowFast Networks for Video Recognition
  • Counterfactual Debiasing Inference for Compositional Action Recognition cites TSM: Temporal Shift Module for Efficient Video Understanding
  • Counterfactual Debiasing Inference for Compositional Action Recognition cites SlowFast Networks for Video Recognition
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers25

  1. Hybrid Relation Guided Set Matching for Few-shot Action RecognitionXiang Wang, Shiwei Zhang, Zhiwu Qing, Mingqian Tang et al.CVPR 2022 · 124 citations
  2. M3Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action RecognitionHao Tang, Jun Liu, Shuanglin Yan, Rui Yan et al.ACM MM 2023 · 78 citations
  3. Motion-modulated Temporal Fragment Alignment Network For Few-Shot Action RecognitionJiamin Wu, Tianzhu Zhang, Zhe Zhang, Feng Wu et al.CVPR 2022 · 73 citations
  4. Boosting Few-shot Action Recognition with Graph-guided Hybrid MatchingJiazheng Xing, Mengmeng Wang, Yudi Ruan, Bofan Chen et al.ICCV 2023 · 41 citations
  5. Counterfactual Debiasing Inference for Compositional Action RecognitionPengzhan Sun, Bo Wu, Xunsong Li, Wen Li et al.ACM MM 2021 · 25 citations

Builds on6

  1. Random Erasing Data AugmentationZhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li et al.AAAI 2020 · 4,134 citations
  2. SlowFast Networks for Video RecognitionChristoph Feichtenhofer, Haoqi Fan, Jitendra Malik, Kaiming HeICCV 2019 · 4,104 citations
  3. Digging Into Self-Supervised Monocular Depth EstimationClément Godard, Oisin Mac Aodha, Michael Firman, Gabriel J. BrostowICCV 2019 · 2,416 citations
  4. TSM: Temporal Shift Module for Efficient Video UnderstandingJi Lin, Chuang Gan, Song HanICCV 2019 · 2,049 citations
  5. Instance Credibility Inference for Few-Shot LearningYikai Wang, Chengming Xu, Chen Liu, Li Zhang et al.CVPR 2020

Related papers

  1. Unsupervised Few-Shot Action Recognition via Action-Appearance Aligned Meta-AdaptationJay Patravali, Gaurav Mittal, Ye Yu, Fuxin Li et al.ICCV 2021 · 24 citations
  2. Annotation-Efficient Untrimmed Video Action RecognitionYixiong Zou, Shanghang Zhang, Guangyao Chen, Yonghong Tian et al.ACM MM 2021 · 7 citations
  3. Active Exploration of Multimodal Complementarity for Few-Shot Action RecognitionYuyang Wanyan, Xiaoshan Yang, Chaofan Chen, Changsheng XuCVPR 2023
  4. Lite-MKD: A Multi-modal Knowledge Distillation Framework for Lightweight Few-shot Action RecognitionBaolong Liu, Tianyi Zheng, Peng Zheng, Daizong Liu et al.ACM MM 2023 · 13 citations
  5. Semantic-Guided Relation Propagation Network for Few-shot Action RecognitionXiao Wang, Weirong Ye, Zhongang Qi, Xun Zhao et al.ACM MM 2021 · 40 citations
Dusk over the sea between two cliffs drawn in fine vertical lines