Lune

AAAI2023Top-tier venue

Intensity-Aware Loss for Dynamic Facial Expression Recognition in the Wild

Hanting Li, Hongjing Niu, Zhaoqing Zhu, Feng Zhao

Abstract

Compared with the image-based static facial expression recognition (SFER) task, the dynamic facial expression recognition (DFER) task based on video sequences is closer to the natural expression recognition scene. However, DFER is often more challenging. One of the main reasons is that video sequences often contain frames with different expression intensities, especially for the facial expressions in the real-world scenarios, while the images in SFER frequently present uniform and high expression intensities. Nevertheless, if the expressions with different intensities are treated equally, the features learned by the networks will have large intra-class and small inter-class differences, which are harmful to DFER. To tackle this problem, we propose the global convolution-attention block (GCA) to rescale the channels of the feature maps. In addition, we introduce the intensity-aware loss (IAL) in the training process to help the network distinguish the samples with relatively low expression intensities. Experiments on two in-the-wild dynamic facial expression datasets (i.e., DFEW and FERV39k) indicate that our method outperforms the state-of-the-art DFER approaches. The source code will be available at https://github.com/muse1998/IAL-for-Facial-Expression-Recognition.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

13 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Learning from Heterogeneity: Generalizing Dynamic Facial Expression Recognition via Distributionally Robust Optimization cites FineCLIPER: Multi-modal Fine-grained CLIP for Dynamic Facial Expression Recognition with AdaptERs
  • TransFER: Learning Relation-aware Facial Expression Representations with Transformers cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • VidEmo: Affective-Tree Reasoning for Emotion-Centric Video Foundation Models cites Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning
  • VidEmo: Affective-Tree Reasoning for Emotion-Centric Video Foundation Models cites MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognition
  • VidEmo: Affective-Tree Reasoning for Emotion-Centric Video Foundation Models cites Context-Aware Emotion Recognition Networks
  • Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning cites MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognition
  • MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognition cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • Robust Lightweight Facial Expression Recognition Network with Label Distribution Training cites Context-Aware Emotion Recognition Networks
  • Intriguing Properties of Vision Transformers cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • FineCLIPER: Multi-modal Fine-grained CLIP for Dynamic Facial Expression Recognition with AdaptERs cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • FineCLIPER: Multi-modal Fine-grained CLIP for Dynamic Facial Expression Recognition with AdaptERs cites MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognition
  • FineCLIPER: Multi-modal Fine-grained CLIP for Dynamic Facial Expression Recognition with AdaptERs cites Context-Aware Emotion Recognition Networks
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers13

  1. Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction TuningZebang Cheng, Zhi-Qi Cheng, Jun-Yan He, Kai Wang et al.NeurIPS 2024 · 293 citations
  2. MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression RecognitionLicai Sun, Zheng Lian, Bin Liu, Jianhua TaoACM MM 2023 · 85 citations
  3. FineCLIPER: Multi-modal Fine-grained CLIP for Dynamic Facial Expression Recognition with AdaptERsHaodong Chen, Haojian Huang, Junhao Dong, Mingzhe Zheng et al.ACM MM 2024 · 26 citations
  4. VidEmo: Affective-Tree Reasoning for Emotion-Centric Video Foundation ModelsZhicheng Zhang, Weicheng Wang, Yongjie Zhu, Wenyu Qin et al.NeurIPS 2025 · 11 citations
  5. Learning from Heterogeneity: Generalizing Dynamic Facial Expression Recognition via Distributionally Robust OptimizationFeng-Qi Cui, Anyang Tong, Jinyang Huang, Jie Zhang et al.ACM MM 2025 · 10 citations

Builds on11

  1. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
  2. Intriguing Properties of Vision TransformersMuzammal Naseer, Kanchana Ranasinghe, Salman Khan, Munawar Hayat et al.NeurIPS 2021 · 863 citations
  3. Robust Lightweight Facial Expression Recognition Network with Label Distribution TrainingZengqun Zhao, Qingshan Liu, Feng ZhouAAAI 2021 · 300 citations
  4. Context-Aware Emotion Recognition NetworksJiyoung Lee, Seungryong Kim, Sunok Kim, Jungin Park et al.ICCV 2019 · 285 citations
  5. TransFER: Learning Relation-aware Facial Expression Representations with TransformersFanglei Xue, Qiangchang Wang, Guodong GuoICCV 2021 · 276 citations

Related papers

  1. Action Unit Enhance Dynamic Facial Expression RecognitionFeng Liu, Lingna Gu, Chen Shi, Xiaolan FuACM MM 2025 · 4 citations
  2. DFEW: A Large-Scale Database for Recognizing Dynamic Facial Expressions in the WildXingxun Jiang, Yuan Zong, Wenming Zheng, Chuangao Tang et al.ACM MM 2020 · 205 citations
  3. FERV39k: A Large-Scale Multi-Scene Dataset for Facial Expression Recognition in VideosYan Wang, Yixuan Sun, Yiwen Huang, Zhongying Liu et al.CVPR 2022 · 107 citations
  4. Rethinking the Learning Paradigm for Dynamic Facial Expression RecognitionHanyang Wang, Bo Li, Shuang Wu, Siyuan Shen et al.CVPR 2023
  5. Freq-HD: An Interpretable Frequency-based High-Dynamics Affective Clip Selection Method for in-the-Wild Facial Expression Recognition in VideosZeng Tao, Yan Wang, Zhaoyu Chen, Boyang Wang et al.ACM MM 2023 · 12 citations
Dusk over the sea between two cliffs drawn in fine vertical lines