"Attention Is All You Need" is a landmark[1][2] 2017 research paper authored by eight scientists working at Google, responsible for expanding 2014 attention mechanisms proposed by Bahdanau et al. into a new deep learning architecture known as the transformer. The paper is considered by some to be a founding document for modern artificial intelligence, as transformers became the main architecture of large language models.[3][4] At the time, the focus of the research was on improving Seq2seq techniques for machine translation, but even in their paper the authors saw the potential for other tasks like question answering and for what is now called multimodal Generative AI.[5]
The paper's title is a reference to the song "All You Need Is Love" by the Beatles.[6]
As of 2024,[update] the paper has been cited more than 100,000 times.[7]
- ^ Love, Julia (10 July 2023). "AI Researcher Who Helped Write Landmark Paper Is Leaving Google". Bloomberg News. Retrieved 1 April 2024.
- ^ Goldman, Sharon (20 March 2024). "'Attention is All You Need' creators look beyond Transformers for AI at Nvidia GTC: 'The world needs something better'". VentureBeat. Retrieved 1 April 2024.
- ^ Cite error: The named reference
Forbeswas invoked but never defined (see the help page). - ^ Cite error: The named reference
Financial Timeswas invoked but never defined (see the help page). - ^ Vaswani, Ashish; Shazeer, Noam; Parmar, Niki; Uszkoreit, Jakob; Jones, Llion; Gomez, Aidan N; Kaiser, Łukasz; Polosukhin, Illia (2017). "Attention is All you Need" (PDF). Advances in Neural Information Processing Systems. 30. Curran Associates, Inc.
- ^ Levy, Steven. "8 Google Employees Invented Modern AI. Here's the Inside Story". Wired. ISSN 1059-1028. Retrieved 20 March 2024.
- ^ "Meet the $4 Billion AI Superstars That Google Lost". Bloomberg. 13 July 2023 – via www.bloomberg.com.