⏮️

Paul Werbos

Original namePaul Werbos

American Computer Scientist, Early Independent Proposer of the Backpropagation Algorithm

The Math Behind the Algorithm · Theoretical Founders

The Early Mathematical Formulation of BackpropagationThe "Backpropagation Through Time" MethodEarly Research in Neural Network Theory

Who they are

Paul Werbos (1947– ) is an American computer scientist who in his 1974 doctoral thesis at Harvard University had already proposed a mathematical method highly similar in principle to the later widely applied backpropagation algorithm for training multi-layer neural networks, a work fully twelve years before the 1986 paper of Rumelhart, Hinton, and Williams. Yet Werbos’s early work did not draw wide scholarly attention at the time, partly because neural-network research itself was in the trough of the "AI winter," and partly related to the limits of the paper’s publication channel and manner of expression. This history is thereafter often taken as a typical case of the phenomenon that "the first proposer of a major scientific discovery is not necessarily the one who gains wide scholarly recognition," and this site states this historical fact without adjudicating the question of "who is the real inventor of the backpropagation algorithm."

Primary sourcesWerbos’s doctoral thesis Beyond Regression (1974)

Key stories

Twelve Years Early, Yet Twelve Years Late to Be Recognized

Werbos’s 1974 doctoral thesis had already systematically proposed the core mathematical principle of the backpropagation algorithm, but since neural-network research was then in the trough of the "AI winter" with few scholars attending to the direction, this work went all but unnoticed for over a decade, until in 1986 the paper of Rumelhart, Hinton, and Williams independently proposed a similar method again and systematically showed its practical effect, and the backpropagation algorithm truly gained wide scholarly recognition and application, Werbos’s early contribution thereafter gradually coming to be known and acknowledged by more researchers.

Relationships

Nemesis

🔙David Rumelhart

Echoes today

Below are how modern works borrow or reinterpret this name or story — not the original material. The two differ, so keep them apart.

A "buried for twelve years" footnote of technical historyWerbos’s early contribution was only gradually made known to more researchers after the publication of Rumelhart and others’ 1986 paper, a history often cited as a typical case of the phenomenon that "the first to propose a major discovery does not necessarily win wide recognition."

Appears in

The Perceptron, the Winter, and the Revival of ConnectionismThe Rediscovery of the Backpropagation Algorithm

← Back to Figures