Breaking Spurious Correlations: Uncertainty-Driven Causal Transformers for AU Detection

Facial Action Unit (AU) detection suffers from limited annotated data, severe class imbalance, label noise, and confounding biases, which often lead to overfitting and degraded performance. We propose an Uncertainty-Driven Causal Transformer (UDCT) framework for robust AU detection by jointly modeling uncertainty and causal intervention. Specifically, we parameterize Transformer attention weights as Gaussian distributions to capture robust AU dependencies while explicitly modeling uncertainty in attention. We further introduce an uncertainty-aware loss reweighting strategy to alleviate the effects of class imbalance and label noise during training. In addition, we incorporate a causal intervention module to suppress confounder-dependent AU associations and encourage the model to focus on more stable and less biased AU relationships. Experiments on BP4D and DISFA demonstrate that UDCT achieves competitive performance with stronger robustness under noisy, imbalanced, and distribution-shifted settings.