Tiny Recursive Models Beat the Giants - Alexia Jolicoeur-Martineau (Microsoft)
Tiny Recursive Models Beat the Giants - Alexia Jolicoeur-Martineau (Microsoft)  
Podcast: The Information Bottleneck
Published On: Tue Sep 15 2026
Description: Alexia Jolicoeur-Martineau is a Principal Researcher at Microsoft and the author of "Less is More: Recursive Reasoning with Tiny Networks," the paper behind the Tiny Recursive Model that hit about 45% on ARC-AGI-1 with a fraction of the parameters of frontier systems. It won the 2025 ARC Prize paper award.She read the hierarchical reasoning paper, thought the potential was real and the explanation was not, and rebuilt it without the mouse brains: a small network that carries a hidden state and a current answer, thinks for a few steps, updates, and repeats, with the gradient truncated at each loop. We get into why puzzles suit this and autoregression doesn't, why she thinks LLMs are bad at molecules and more data won't fix it, and what she'd do with a trillion dollars.Timeline00:01 Intro01:06 Leaving biostatistics, and why the field stagnated06:47 GANs, diffusion, and research on four GPUs12:58 What was wrong with the hierarchical reasoning paper16:31 Tiny recursive models explained without the biology22:35 Why puzzles favor recursion over left to right generation24:15 Is the bitter lesson really bitter?27:28 With infinite compute, would you still want small models?32:00 Self improvement, memory, and a trillion dollars37:01 Test time compute beyond chain of thought40:41 Why chain of thought fails on molecules45:17 Is there a universal representation?48:06 What people are already building with TRM55:22 Fixed point models and DEQMusic"Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0TopicsTiny Recursive Models and the ARC-AGI resultsWhat the hierarchical reasoning model was really doingDeep supervision and truncated backpropLooping transformers and parameter efficiencyWhy puzzles favor whole-context iteration over left to right generationTest time compute beyond chain of thoughtLatent reasoning and the Coconut line of workWhy LLMs fail on chemistry and physicsRepresentation learning and whether a universal representation exists