PairCFR: Enhancing Model Training on Paired Counterfactually Augmented Data through Contrastive Learning

Counterfactually Augmented Data (CAD) in-001 volves creating new data samples by apply-002 ing minimal yet sufficient modifications to flip 003 the label of existing data samples to the other 004 classes. Training with CAD enhances model ro-005 bustness against spurious features that happen 006 to correlate with labels by spreading the ca-007 sual relationships across different classes. Yet, 008 recent research reveals that CAD may lead 009 models to overly focus on modified features 010 while ignoring other important contextual in-011 formation, inadvertently introducing biases that 012 may impair performance on out-of-distribution 013 (OOD) datasets. To mitigate this issue, we 014 employ contrastive learning to promote global 015 feature alignment in addition to counterfac-016 tual clues. We theoretically prove that con-017 trastive loss can encourage models to leverage 018 a broader range of features beyond those modi-019 fied ones. Comprehensive experiments on two 020 human-edited CAD datasets demonstrate that 021 our propose method outperformed the state-of-022 the-art on OOD datasets.