🔙

David Rumelhart

Original nameDavid Rumelhart

American Psychologist, Co-Proposer of the Backpropagation Algorithm

The Math Behind the Algorithm · Theoretical Founders

The Backpropagation AlgorithmParallel Distributed Processing (PDP) TheoryConnectionist Cognitive Science

Who they are

David Rumelhart (1942–2011) was an American cognitive psychologist long teaching at the University of California, San Diego, and Stanford University, who in 1986 co-authored with Geoffrey Hinton and Ronald Williams a paper systematically expounding the application of the backpropagation algorithm to the training of multi-layer neural networks, a method thereafter the core mechanism of the training of almost all deep neural networks. The "parallel distributed processing" (PDP) research group Rumelhart thereafter led systematically proposed "connectionism," a cognitive-science theoretical framework holding that the essence of human cognition is the parallel-distributed interaction of a great many simple neural units rather than the explicit rules and symbol manipulation of traditional symbolism, a framework that thereafter profoundly influenced the development of both cognitive science and AI for decades. In his later years Rumelhart suffered from a rare Pick’s disease (a neurodegenerative disease affecting language and cognitive function), so that this scholar who had deeply studied the mechanisms of human cognition himself underwent a gradual decline of cognitive ability, a circumstance thereafter often seen as a lamentable irony of fate.

Primary sourcesRumelhart, Hinton & Williams, "Learning representations by back-propagating errors" (1986)

Key stories

The Priority of the Backpropagation Algorithm Is Also Disputed

The priority of the backpropagation algorithm is also discussed: the mathematical principle of this method had earlier been independently proposed by other researchers (such as Linnainmaa and Werbos), and the core contribution of Rumelhart’s team’s 1986 paper was clearly showing this method could let multi-layer neural networks learn useful internal representations, thereby winning it wide attention and application in AI.

Version differences

There is also discussion over the invention priority of the backpropagation algorithm: Werbos’s 1974 doctoral thesis had already proposed a highly similar method, and this site states each side’s contribution and historical evaluation without adjudicating who the "sole inventor" is.

Relationships

Disciple

🧠Geoffrey Hinton

Nemesis

⏮️Paul Werbos　Werbos in his 1974 doctoral thesis had already proposed a method highly similar to backpropagation in principle, but did not gain wide scholarly attention at the time, and the historical evaluation of the two on this priority question has long been discussed

Echoes today

Below are how modern works borrow or reinterpret this name or story — not the original material. The two differ, so keep them apart.

The irony of fate of cognitive declineIn his later years Rumelhart suffered from Pick’s disease, which affects cognitive function, and this scholar who had once deeply studied the mechanism of human cognition himself experiencing cognitive decline was thereafter often seen as a rueful irony of fate.

Appears in

The Perceptron, the Winter, and the Revival of ConnectionismThe Rediscovery of the Backpropagation Algorithm

← Back to Figures