Subvolts take AI researchers explore how AI might automate its own research and development. Apr 2, 2026 - Developers
- Learn about the technical hurdles and future directions for AI self-improvement and Continual Learning.
- Business
- Understand the implications of AI building AI for enterprise AI pipelines and future AI capabilities.
This interview provides a grounded look at AI self-improvement and its technical challenges from a leading researcher.
AI is Already Building AI | Google DeepMind’s Mostafa Dehghani
Audio streams from Matt Turck. Your place is remembered on this device.
From Subvolts
Summary and analysis by Subvolts editorial, AI-assisted.
Matt Turck interviews Google DeepMind researcher Mostafa Dehghani on AI automating its own development. They discuss "thinking in loops," Recursive Self-Improvement (RSI), and bottlenecks like evaluation limits and long-horizon reliability. The conversation also touches on the shift from pre-training to post-training, multimodal processing in Gemini Nano, and the future of Continual Learning for enterprise AI.
Imagine AI helping to design even smarter AI. This episode explains how AI might learn to improve itself, like a student studying to get better. It covers the challenges and what's next for AI that can learn continuously.
This episode offers a deep dive into the mechanics of AI self-improvement, moving beyond hype to discuss practical challenges. It is for AI researchers and engineers interested in the future of AI development and its limitations. The discussion on RSI and bottlenecks connects directly to current frontier lab efforts in advancing model capabilities and reliability, particularly relevant for Google's ongoing work with Gemini.
- HostMatt Turck
- GuestMostafa DehghaniAI researcher, Google DeepMind
- Guest: Mostafa Dehghani
- Title: AI is Already Building AI | Google DeepMind’s Mostafa Dehghani
- Show: The MAD Podcast
- Length: 65 minutes
- Guest's work: Universal Transformers and Vision Transformers (ViT)
Are we truly on the verge of AI automating its own research and development? In this deep-dive episode of the MAD Podcast, Matt Turck sits down with Mostafa Dehghani, a pioneering AI researcher at Google DeepMind whose work on Universal Transformers and Vision Transformers (ViT) helped lay the groundwork for today's frontier models.
Moving past the hype, Mostafa breaks down the actual mechanics of "thinking in loops" and Recursive Self-Improvement (RSI). He explores the critical bottlenecks holding back true AGI—from evaluation limits and formal verification to the brutal math of long-horizon reliability.
Mostafa and Matt also discuss the shift from pre-training to post-training, how Gemini's Nano Banana 2 processes pixels and text simultaneously, and why the "frozen" nature of today's models means Continual Learning is the next massive frontier for enterprise AI and data pipelines.
(00:00) Intro
(01:17) What “loops” in AI actually mean
(05:04) Self-improvement as the next chapter of machine learning
(07:32) Are Karpathy’s autoresearch agents an early form of AI self-improvement?
(08:56) AI building AI: how close are we?
(10:02) The biggest bottlenecks: evals, automation, and long horizons
(12:36) Can formal verification unlock recursive self-improvement?
(14:06) What is model collapse?
(15:33) Generalization vs specialization in AI
(18:04) What is a specialized model today?
(20:57) Could top AI researchers themselves be automated?
(24:02) If AI builds AI, does data matter less than compute?
(26:22) Post-training vs pre-training: where will progress come from?
(28:14) Why pre-training is not dead
(29:45) What is continual learning?
(31:53) How real is continual learning today?
(33:43) Mostafa Dehghani’s background and path into AI
(36:13) The story behind Universal Transformers
(39:56) How Vision Transformers changed AI
(43:47) Gemini, multimodality, and Nano Banana
(47:46) Why multimodality helps build a world model
(52:44) Why image generation is getting faster and more efficient
(54:44) Hot takes
(54:53) What the AI field is getting wrong
(56:17) Why continual learning is underrated
(57:26) Does RAG go away over time?
(58:21) What people are too confident about in AI
(59:56) If he were starting from scratch today
- Recursive Self-Improvement (RSI)
- Mostafa Dehghani
- AI systems that can improve their own capabilities over time.
- AGI
- Mostafa Dehghani
- Artificial General Intelligence, AI with human-like cognitive abilities.
- Continual Learning
- Mostafa Dehghani
- AI that can learn new information without forgetting previous knowledge.
Same guests, other shows
31 minBehind the scenes of Google's state-of-the-art "nano-banana" image model
WithNicole Brichtova, Kaushik Shivakumar, Mostafa Dehghani, Robert Riachi
Questions people ask
- What does "thinking in loops" mean for AI?
- It refers to AI systems that can repeatedly process information or refine their outputs, enabling a form of iterative self-improvement.
- What are the main obstacles to AI building AI?
- Key bottlenecks include limitations in evaluation methods, the difficulty of automating research processes, and ensuring reliability over long periods.
Page generated Oct 19, 2026. Summaries are Subvolts' own; the audio, artwork and notes belong to Matt Turck.