Dendritic Localized Learning: Toward Biologically Plausible Algorithm
Changze Lv, Jingwen Xu, Yiyang Lu, Xiaohua Wang, Zhenghua Wang, Zhibo Xu, Di Yu, Xin Du, Xiaoqing Zheng, Xuanjing Huang
摘要
Backpropagation is the foundational algorithm for training neural networks and a key driver of deep learning's success. However, its biological plausibility has been challenged due to three primary limitations: weight symmetry, reliance on global error signals, and the dual-phase nature of training, as highlighted by the existing literature. Although various alternative learning approaches have been proposed to address these issues, most either fail to satisfy all three criteria simultaneously or yield suboptimal results. Inspired by the dynamics and plasticity of pyramidal neurons, we propose Dendritic Localized Learning (DLL), a novel learning algorithm designed to overcome these challenges. Extensive empirical experiments demonstrate that DLL satisfies all three criteria of biological plausibility while achieving state-of-the-art performance among algorithms that meet these requirements. Furthermore, DLL exhibits strong generalization across a range of architectures, including MLPs, CNNs, and RNNs. These results, benchmarked against existing biologically plausible learning algorithms, offer valuable empirical insights for future research. We hope this study can inspire the development of new biologically plausible algorithms for training multilayer networks and advancing progress in both neuroscience and machine learning. Our code is available at https://github.com/Lvchangze/ Dendritic-Localized-Learning .
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper1
问问它们各自怎么用它它引用的顶会 Paper6
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah 等NeurIPS 2020 · 被引用 64,255 次
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn 等ICLR 2021 · 被引用 21,477 次
- A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding NetworksTommaso Salvatori, Yuhang Song, Yordan Yordanov, Beren Millidge 等ICLR 2024 · 被引用 22 次
- Hebbian Deep Learning Without FeedbackAdrien Journé, Hector Garcia Rodriguez, Qinghai Guo, Timoleon MoraitisICLR 2023 · 被引用 17 次
- Counter-Current Learning: A Biologically Plausible Dual Network Approach for Deep LearningChia-Hsiang Kao, Bharath HariharanNeurIPS 2024 · 被引用 9 次
相关 Paper
- Correlative Information Maximization: A Biologically Plausible Approach to Supervised Deep Neural Networks without Weight SymmetryBariscan Bozkurt, Cengiz Pehlevan, Alper T. ErdoganNeurIPS 2023 · 被引用 6 次
- Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward PassGiorgia Dellaferrera, Gabriel KreimanICML 2022 · 被引用 80 次
- Towards Scaling Difference Target Propagation by Learning Backprop TargetsMaxence Ernoult, Fabrice Normandin, Abhinav Moudgil, Sean Spinney 等ICML 2022 · 被引用 49 次
- Can the Brain Do Backpropagation? - Exact Implementation of Backpropagation in Predictive Coding NetworksYuhang Song, Thomas Lukasiewicz, Zhenghua Xu, Rafal BogaczNeurIPS 2020 · 被引用 117 次
- A Theoretical Framework for Target PropagationAlexander Meulemans, Francesco S. Carzaniga, Johan A. K. Suykens, João Sacramento 等NeurIPS 2020 · 被引用 110 次
