Deep Dive: Jeff Dean on Google Brain’s Early Days

In the fifth installment of our Moonshot Podcast Deep Dive video interview series, X’s Captain of Moonshots Astro Teller sits down with Google DeepMind’s Chief Scientist Jeff Dean for a conversation about the origin of Jeff’s pioneering work scaling neural networks. They discuss the first time AI captured Jeff’s imagination, the earliest Google Brain framework, the team’s stratospheric advancements in image recognition and speech-to-text, how AI is evolving, and more. For more on X’s AI moonshots, check out Jeff on Episode 3 of The Moonshot Podcast, “Machines That Think”: https://www.youtube.com/watch?v=_jLPA8Qpc74 00:00 - Introduction 00:35 - What Jeff’s childhood was like 03:50 - The first “non-trivial” thing Jeff coded 05:02 - Jeff’s “love/hate” relationship with C++ 06:59 - The first time AI captured Jeff’s imagination 10:40 - When neural networks fell “out of vogue” 12:24 - How Jeff made a career out of “starting over” 14:23 - The origins of Google Brain 17:30 - Critical components of scaling neural networks 18:43 - The earliest Google Brain framework 19:32 - How the team first built a 100x bigger neural network 22:04 - The cat video that started it all 25:02 - Google Brain’s major progress on image recognition 26:24 - Google Brain’s major progress on speech to text 27:29 - The origins of TensorFlow and TPUs 31:56 - Three breakthroughs in understanding language models 37:41 - Where Jeff thinks the world is headed 40:07 - The shift from “making things” to designing prompts 42:38 - How Jeff uses AI to enhance his own life 43:51 - Safety, security, and privacy in AI 50:26 - The future of understanding neural network behavior 53:34 - When computers are “smarter” than humans 56:52 - Jeff’s next-five-years plan Learn more about your ad choices. Visit podcastchoices.com/adchoices