Can Biologically Plausible Temporal Credit Assignment Rules Match BPTT for Neural Similarity? E-prop as an Example
Yuhan Helena Liu, Guangyu Robert Yang, Christopher J. Cueva
摘要
Understanding how the brain learns may be informed by studying biologically plausible learning rules. These rules, often approximating gradient descent learning to respect biological constraints such as locality, must meet two critical criteria to be considered an appropriate brain model: (1) good neuroscience task performance and (2) alignment with neural recordings. While extensive research has assessed the first criterion, the second remains underexamined. Employing methods such as Procrustes analysis on well-known neuroscience datasets, this study demonstrates the existence of a biologically plausible learning rule -namely e-prop, which is based on gradient truncation and has demonstrated versatility across a wide range of tasks -that can achieve neural data similarity comparable to Backpropagation Through Time (BPTT) when matched for task accuracy. Our findings also reveal that model architecture and initial conditions can play a more significant role in determining neural similarity than the specific learning rule. Furthermore, we observe that BPTT-trained models and their biologically plausible counterparts exhibit similar dynamical properties at comparable accuracies. These results underscore the substantial progress made in developing biologically plausible learning rules, highlighting their potential to achieve both competitive task performance and neural data similarity.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper1
问问它们各自怎么用它它引用的顶会 Paper26
- Transformers Learn In-Context by Gradient DescentJohannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento 等ICML 2023 · 被引用 729 次
- Generalized Shape Metrics on Neural RepresentationsAlex H. Williams, Erin Kunz, Simon Kornblith, Scott W. LindermanNeurIPS 2021 · 被引用 182 次
- The interplay between randomness and structure during learning in RNNsFriedrich Schüßler, Francesca Mastrogiuseppe, Alexis M. Dubreuil, Srdjan Ostojic 等NeurIPS 2020 · 被引用 91 次
- Exact learning dynamics of deep linear networks with prior knowledgeLukas Braun, Clémentine C. J. Dominé, James Fitzgerald, Andrew M. SaxeNeurIPS 2022 · 被引用 75 次
- Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size OscillationsAxel Laborieux, Friedemann ZenkeNeurIPS 2022 · 被引用 65 次
相关 Paper
- Can the Brain Do Backpropagation? - Exact Implementation of Backpropagation in Predictive Coding NetworksYuhang Song, Thomas Lukasiewicz, Zhenghua Xu, Rafal BogaczNeurIPS 2020 · 被引用 117 次
- Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rulesYuhan Helena Liu, Arna Ghosh, Blake A. Richards, Eric Shea-Brown 等NeurIPS 2022 · 被引用 10 次
- Learning to Learn with Feedback and Local PlasticityJack Lindsey, Ashok Litwin-KumarNeurIPS 2020 · 被引用 38 次
- Biologically-plausible backpropagation through arbitrary timespans via local neuromodulatorsYuhan Helena Liu, Stephen Smith, Stefan Mihalas, Eric Shea-Brown 等NeurIPS 2022 · 被引用 18 次
- Convergence and Alignment of Gradient Descent with Random Backpropagation WeightsGanlin Song, Ruitu Xu, John LaffertyNeurIPS 2021 · 被引用 4 次
