🎯

Richard Sutton

Original nameRichard Sutton

Canadian Computer Scientist, Founder of Reinforcement Learning Theory

The Math Behind the Algorithm · Theoretical Founders

Reinforcement Learning TheoryThe Textbook Reinforcement Learning: An IntroductionThe Essay "The Bitter Lesson"

Who they are

Richard Sutton (1957– ) is a Canadian computer scientist long teaching at the University of Alberta, whose Reinforcement Learning: An Introduction, co-authored with Andrew Barto, is acknowledged as the most classic and widely influential textbook in the field, systematically establishing the theoretical basis of the reinforcement-learning framework of "an agent adjusting its policy by a reward signal through interaction with the environment," a theory thereafter the core training mechanism of systems such as AlphaGo. Sutton’s 2019 short essay "The Bitter Lesson" proposed a view thereafter repeatedly cited and discussed in the industry: that looking over the seventy-year history of AI research, the general methods relying on piling up compute and data rather than on human-designed domain knowledge and rules always win out in the long run, an essay thereafter widely seen as one of the important theoretical bases explaining why the "scaling path" of large language models at last took the mainstream.

Primary sourcesSutton & Barto, Reinforcement Learning: An Introduction (1998)Sutton, "The Bitter Lesson" (2019)

Key stories

"The Bitter Lesson": The More General, the More Compute-Reliant Method Often Wins Last

In the essay Sutton reviewed the development of several AI subfields such as computer chess, speech recognition, and computer vision, and found a recurring pattern: researchers at first always tend to encode human expertise and intuition into the system, indeed making progress in the short term, but in the long run the methods that give up reliance on human prior knowledge and instead do general search or learning with stronger compute and more data almost always overtake and do better. This observation is seen by many researchers as an important theoretical basis explaining why the "scaling path" of later large language models could overwhelm several earlier, more "ingenious" methods.

Relationships

Mentor

🎲Arthur Samuel

Disciple

⚫David Silver

Echoes today

Below are how modern works borrow or reinterpret this name or story — not the original material. The two differ, so keep them apart.

"The Bitter Lesson" becomes an industry-consensus citationThe view this short essay proposed was thereafter widely cited, becoming one of the important theoretical bases for explaining why the "scaling path" of large language models could overwhelm more "ingenious" methods.

Appears in

AlphaGo, the Transformer, and the Race to ScaleAlphaGo Beats Lee Sedol

← Back to Figures