SAFLEX: Self-Adaptive Augmentation via Feature Label Extrapolation

Data augmentation, a cornerstone technique in deep learning, is crucial in enhancing model performance, especially with scarce labeled data. While traditional methods, such as handcrafted augmentations, are effective but limited in scope, modern, adaptable techniques often come at the cost of computational complexity and are hard to fit into existing processes. In this work, we unveil an efficient approach that universally enhances existing data augmentation techniques by enabling their adaptation and refinement, thereby providing a significant and comprehensive improvement across all existing methods. We present SAflex (Self-Adaptive Augmentation via Feature Label EXtrapolation), an approach that utilizes an efficient bilevel optimization to learn the sample weights and soft labels of augmented samples. This is applicable to augmentations from any source, seamlessly integrating with existing upstream augmentation pipelines. Remarkably, SAflex effectively reduces the noise and label errors of the upstream augmentation pipeline with a marginal computational cost. As a versatile module, SAflex excels across diverse datasets, including natural, medical images, and tabular data, showcasing its prowess in few-shot learning and out-of-distribution generalization. SAflex seamlessly integrates with common augmentation strategies like RandAug and CutMix, as well as augmentations from large pre-trained generative models like stable diffusion. It is also compatible with contrastive learning frameworks, such as fine-tuning CLIP. Our findings highlight the potential to adapt existing augmentation pipelines for new data types and tasks, signaling a move towards more adaptable and resilient training frameworks.