On Convergence and Generalization of Dropout Training
Poorya Mianjy, Raman Arora
2020年份
34被引次数
8顶会引用
摘要
We study dropout in two-layer neural networks with rectified linear unit (ReLU) activations. Under mild overparametrization and assuming that the limiting kernel can separate the data distribution with a positive margin, we show that dropout training with logistic loss achieves -suboptimality in test error in O(1/ ) iterations. 34th Conference on Neural Information Processing Systems (NeurIPS 2020),
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式，引用它的顶会 Paper 也一样。你提问，回答直接引用原文。
引用它的顶会 Paper8
- Why Do Better Loss Functions Lead to Less Transferable Features?Simon Kornblith, Ting Chen, Honglak Lee, Mohammad NorouziNeurIPS 2021 · 被引用 113 次
- Learning from Teaching Regularization: Generalizable Correlations Should be Easy to ImitateCan Jin, Tong Che, Hongwu Peng, Yiyuan Li 等NeurIPS 2024 · 被引用 67 次
- What training reveals about neural network complexityAndreas Loukas, Marinos Poiitis, Stefanie JegelkaNeurIPS 2021 · 被引用 12 次
- Stochastic Modified Equations and Dynamics of Dropout AlgorithmZhongwang Zhang, Yuqing Li, Tao Luo, Zhi-Qin John XuICLR 2024 · 被引用 12 次
- Structured Dropout Variational Inference for Bayesian Neural NetworksSon Nguyen, Duong Nguyen, Khai Nguyen, Khoat Than 等NeurIPS 2021 · 被引用 11 次
它引用的顶会 Paper5
- Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networksZiwei Ji, Matus TelgarskyICLR 2020 · 被引用 193 次
- Simple and Effective Regularization Methods for Training on Noisily Labeled Data with Generalization GuaranteeWei Hu, Zhiyuan Li, Dingli YuICLR 2020 · 被引用 140 次
- The Implicit and Explicit Regularization Effects of DropoutColin Wei, Sham M. Kakade, Tengyu MaICML 2020 · 被引用 129 次
- Dropout: Explicit Forms and Capacity ControlRaman Arora, Peter L. Bartlett, Poorya Mianjy, Nathan SrebroICML 2021 · 被引用 43 次
- Can Implicit Bias Explain Generalization? Stochastic Convex Optimization as a Case StudyAssaf Dauber, Meir Feder, Tomer Koren, Roi LivniNeurIPS 2020 · 被引用 26 次
相关 Paper
- Benign Overfitting in Two-layer ReLU Convolutional Neural NetworksYiwen Kou, Zixiang Chen, Yuanzhou Chen, Quanquan GuICML 2023 · 被引用 32 次
- A Non-Parametric Regression Viewpoint : Generalization of Overparametrized Deep RELU Network Under Noisy ObservationsNamjoon Suh, Hyunouk Ko, Xiaoming HuoICLR 2022 · 被引用 15 次
- How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?Zixiang Chen, Yuan Cao, Difan Zou, Quanquan GuICLR 2021 · 被引用 29 次
- Bounding the Width of Neural Networks via Coupled Initialization A Worst Case AnalysisAlexander Munteanu, Simon Omlor, Zhao Song, David P. WoodruffICML 2022 · 被引用 17 次
- Benign Overfitting in Deep Neural Networks under Lazy TrainingZhenyu Zhu, Fanghui Liu, Grigorios Chrysos, Francesco Locatello 等ICML 2023 · 被引用 12 次
