Researchers develop PC-ALM algorithm to train 1000-layer neural networks using layer-local dynamics instead of backpropagation, matching performance while mimicking brain-like learning.

Augmented Lagrangian Predictive Coding: training 1000-layer networks without backpropagation

We introduce PC-ALM, a local alternative to backpropagation. PC-ALM trains residual MLPs up to 1000 layers, nearly matching backprop's performance despite using only layer-local dynamics. PC-ALM equips each layer with a feedback control dynamical system that distributes and propagates supervision credit throughout a network. Standard deep learning relies on backpropagation. The brain, however, cannot implement backpropagation, at least not exactly[1, 2]. How the brain solves the multilayer credi...