Reinventing AI From Scratch with Yaroslav Bulatov
Reinventing AI From Scratch with Yaroslav Bulatov  
Podcast: The Information Bottleneck
Published On: Mon Mar 30 2026
Description: Yaroslav Bulatov helped build the AI era from the inside, as one of the earliest researchers at both OpenAI and Google Brain. Now he wants to tear it all down and start over. Modern deep learning, he argues, is up to 100x more wasteful than it needs to be  -  a Frankenstein of hacks designed for the wrong hardware. With a power wall approaching in two years, Yaroslav is leading an open effort to reinvent AI from scratch: no backprop, no legacy assumptions, just the benefit of hindsight and AI agents that compress decades of research into months. Along the way, we dig into why AGI is a "religious question," how a sales guy with no ML background became one of his most productive contributors, and why the Muon optimizer, one of the biggest recent breakthroughs, could only have been discovered by a non-expert.Timeline00:12 — Introduction and Yaroslav's background at OpenAI and Google Brain01:16 — Why deep learning isn't such a good idea02:03 — The three definitions of AGI: religious, financial, and vibes-based07:52 — The SAI framework: do we need the term AGI at all?10:58 — What matters more than AGI: efficiency and refactoring the AI stack13:28 — Jevons paradox and the coming energy wall14:49 — The recipe: replaying 70 years of AI with hindsight17:23 — Memory, energy, and gradient checkpointing18:34 — Why you can't just optimize the current stack (the recurrent laryngeal nerve analogy)21:05 — What a redesigned AI might look like: hierarchical message passing22:31 — Can a small team replicate decades of research?24:23 — Why non-experts outperform domain specialists27:42 — The GPT-2 benchmark: what success looks like29:01 — Ian Goodfellow, Theano, and the origins of TensorFlow30:12 — The Muon optimizer origin story and beating Google on ImageNet36:16 — AI coding agents for software engineering and research40:12 — 10-year outlook and the voice-first workflow42:23 — Why start with text over multimodality45:13 — Are AI labs like SSI on the right track?48:52 — Getting rid of backprop — and maybe math itself53:57 — The state of ML academia and NeurIPS culture56:41 — The Sutra group challenge: inventing better learning algorithmsMusic:"Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0."Palms Down" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.Changes: trimmedAbout: The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.