Alternative learning mechanisms

tags

Machine learning

Many people, including Geoffrey Hinton, have raised concerns about the back-propagation algorithm and the fact that it’s likely not a promising way to achieve Artificial Intelligence (see this Axios blog post).

Alternative mechanisms for learning have been and are currently studied to try and approach the learning problem in a more effective way.

Direct feedback alignment

(Nøkland 2016)

Hebbian learning

The theory is sometimes summarized as “Cells that fire together wire together.”.

Predictive coding

(Whittington and Bogacz 2017; Millidge, Tschantz, and Buckley 2020)

Bibliography

1. Millidge, Beren, Alexander Tschantz, and Christopher L. Buckley. October 2020. "Predictive Coding Approximates Backprop Along Arbitrary Computation Graphs". arXiv:2006.04182 [Cs], October.

2. Whittington, James C. R., and Rafal Bogacz. May 2017. “An Approximation of the Error Backpropagation Algorithm in a Predictive Coding Network with Local Hebbian Synaptic Plasticity”. Neural Computation 29 (5):1229–62.

3. Nøkland, Arild. December 2016. “Direct Feedback Alignment Provides Learning in Deep Neural Networks”. arXiv:1609.01596 [Cs, Stat], December.