DivAug: Plug-in Automated Data Augmentation with Explicit Diversity Maximization
Zirui Liu, Haifeng Jin, Ting-Hsiang Wang, Kaixiong Zhou, Xia Hu
Abstract
Human-designed data augmentation strategies have been replaced by automatically learned augmentation policy in the past two years. Specifically, recent work has empirically shown that the superior performance of the automated data augmentation methods stems from increasing the diversity of augmented data [4], [5]. However, two factors regarding the diversity of augmented data are still missing: 1) the explicit definition (and thus measurement) of diversity and 2) the quantifiable relationship between diversity and its regularization effects. To bridge this gap, we propose a diversity measure called Variance Diversity and theoretically show that the regularization effect of data augmentation is promised by Variance Diversity. We validate in experiments that the relative gain from automated data augmentation in test accuracy is highly correlated to Variance Diversity. An unsupervised sampling-based framework, DivAug, is designed to directly maximize Variance Diversity and hence strengthen the regularization effect. Without requiring a separate search process, the performance gain from DivAug is comparable with the state-of-the-art method with better efficiency. Moreover, under the semi-supervised setting, our framework can further improve the performance of semi-supervised learning algorithms compared to RandAugment, making it highly applicable to real-world problems, where labeled data is scarce. The code is available at https://github.com/warai-0toko/DivAug.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
5 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- On the Generalization Effects of Linear Transformations in Data Augmentation cites Unsupervised Data Augmentation for Consistency Training
- Anchor Data Augmentation cites RandAugment: Practical Automated Data Augmentation with a Reduced Search Space
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence cites Unsupervised Data Augmentation for Consistency Training
Cited by top-tier papers5
- Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language ModelZirui Liu, Guanchu Wang, Shaochen Zhong, Zhaozhuo Xu et al.NeurIPS 2023 · 22 citations
- Uncertainty-aware Unsupervised Multi-Object TrackingKai Liu, Sheng Jin, Zhihang Fu, Ze Chen et al.ICCV 2023 · 21 citations
- Diversity-oriented Data Augmentation with Large Language ModelsZaitian Wang, Jinghan Zhang, Xinhao Zhang, Kunpeng Liu et al.ACL 2025 · 11 citations
- Anchor Data AugmentationNora Schneider, Shirin Goshtasbpour, Fernando Pérez-CruzNeurIPS 2023 · 8 citations
- Molecular Data Programming: Towards Molecule Pseudo-labeling with Systematic Weak SupervisionXin Juan, Kaixiong Zhou, Ninghao Liu, Tianlong Chen et al.CVPR 2024
Builds on4
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and ConfidenceKihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang et al.NeurIPS 2020 · 5,129 citations
- RandAugment: Practical Automated Data Augmentation with a Reduced Search SpaceEkin Dogus Cubuk, Barret Zoph, Jonathon Shlens, Quoc LeNeurIPS 2020 · 4,453 citations
- Unsupervised Data Augmentation for Consistency TrainingQizhe Xie, Zihang Dai, Eduard H. Hovy, Thang Luong et al.NeurIPS 2020 · 2,774 citations
- On the Generalization Effects of Linear Transformations in Data AugmentationSen Wu, Hongyang R. Zhang, Gregory Valiant, Christopher RéICML 2020 · 92 citations
Related papers
- Regularizing Graph Neural Networks via Consistency-Diversity Graph AugmentationsDeyu Bo, Binbin Hu, Xiao Wang, Zhiqiang Zhang et al.AAAI 2022 · 35 citations
- Distribution Augmentation for Generative ModelingHeewoo Jun, Rewon Child, Mark Chen, John Schulman et al.ICML 2020 · 68 citations
- A Data-Augmentation Is Worth A Thousand Samples: Analytical Moments And Sampling-Free TrainingRandall Balestriero, Ishan Misra, Yann LeCunNeurIPS 2022 · 23 citations
- Tradeoffs in Data Augmentation: An Empirical StudyRaphael Gontijo Lopes, Sylvia J. Smullin, Ekin Dogus Cubuk, Ethan DyerICLR 2021 · 72 citations
- Exploring Temporally Dynamic Data Augmentation for Video RecognitionTaeoh Kim, Jinhyung Kim, Minho Shim, Sangdoo Yun et al.ICLR 2023 · 5 citations
