Google Research Team, Co-Inventors of the Transformer Self-Attention Architecture
In 2017 eight Google researchers — Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Łukasz Kaiser, and Illia Polosukhin — jointly published the paper "Attention Is All You Need," proposing a wholly new architecture, the "Transformer," that wholly abandoned the previously mainstream recurrent and convolutional neural network structures and relied only on the "self-attention mechanism" to process sequence data. This architecture let a model process in parallel the mutual relations between any two positions in the whole input sequence, rather than having to process step by step in order as a recurrent neural network does, which not only greatly raised training efficiency but, more importantly, let the model more fully capture long-distance context dependencies. The Transformer architecture thereafter became the common underlying architectural basis of almost all large-scale language models — including Google’s own BERT series and OpenAI’s GPT series — profoundly changing the technical path of natural language processing and later of computer vision, protein-structure prediction, and many other fields, one of the most far-reaching single-paper results in deep learning in the 2010s. These eight authors thereafter mostly left Google, in turn joining or founding AI startups including Anthropic, Character.AI, Cohere, and Adept, and this phenomenon of "the paper authors collectively leaving to start companies" also reflects from the side the enormous industrial value this invention itself held.
The paper title "Attention Is All You Need" directly stated the paper’s most core claim: that processing sequence data no longer needed recurrent or convolutional structure, and attention alone was enough — a claim quite bold at publication, thereafter proven highly farsighted, and the title "attention is all you need" thereby transcended the paper itself to become one of the most quoted and adapted phrases in the AI field.
The eight authors of this paper thereafter left Google in turn, respectively taking part in founding or joining AI startups including Anthropic, Character.AI, Cohere, Adept, and Inceptive, and the core team of one paper thereafter dispersing so densely to spawn a whole emerging startup ecosystem is itself a typical case for observing the talent-flow feature of this wave of the generative-AI industry.
The ordering and description of the eight authors’ specific contributions has since provoked some public anecdotes and discussion, but the paper’s historical standing as a whole team result is undisputed in scholarship.
Below are how modern works borrow or reinterpret this name or story — not the original material. The two differ, so keep them apart.
That email didn’t look right? Free · unsubscribe anytime · Privacy