Subvolts take Understand AI's future by rebuilding AlphaGo with modern tools and exploring RL in LLMs. May 15, 2026 - Developers
- Learn how AlphaGo's architecture and learning methods can inform your work on LLMs and AI agents.
- Business
- Understand how foundational AI research like AlphaGo's informs the capabilities and future trajectory of AI systems.
This episode provides a detailed technical breakdown of a landmark AI achievement, offering practical insights for current research.
Eric Jang – Building AlphaGo from scratch
Audio streams from Dwarkesh Patel. Your place is remembered on this device.
From Subvolts
Summary and analysis by Subvolts editorial, AI-assisted.
Dwarkesh Patel interviews Eric Jang on the Dwarkesh Podcast to discuss building AlphaGo from scratch using modern AI tools. They explore how AlphaGo's approach to search, learning, and self-play offers insights into future general AI. The conversation also covers reinforcement learning in LLMs and the potential for AI to automate research tasks.
This episode explains how to recreate AlphaGo, an early AI that mastered the game Go, using today's AI technology. It shows how this old AI's methods for learning and decision-making can teach us about how current AI like chatbots learn and how future AI might get much smarter.
This episode offers a deep dive into the foundational principles of AI intelligence by deconstructing AlphaGo. It's for AI researchers and enthusiasts interested in how current large language models learn and how future AI might evolve. Jang's insights connect to ongoing discussions about AI capabilities and the potential for an intelligence explosion, relevant to how labs like Google Gemini and OpenAI are pushing model development.
- HostDwarkesh Patel
- GuestEric Jang
- Guest: Eric Jang
- Show: Dwarkesh Podcast
- Length: 157 minutes
- Topic: Building AlphaGo from scratch with modern AI tools
- Key AI Concepts: Search, learning from experience, self-play, reinforcement learning in LLMs, credit assignment problem
Eric Jang walks through how to build AlphaGo from scratch, but with modern AI tools.
Sometimes you understand the future better by stepping backward. AlphaGo is still the cleanest worked example of the primitives of intelligence: search, learning from experience, and self-play. You have to go back to 2017 to get insight into how the more general AIs of the future might learn.
Once he explained how AlphaGo works, it gave us the context to have a discussion about how RL works in LLMs and how it could work better – naive policy gradient RL has to figure out which of the 100k+ tokens in your trajectory actually got you the right answer, while AlphaGo’s MCTS suggests a strictly better action every single move, giving you a training target that sidesteps the credit assignment problem. The way humans learn is surely closer to the second.
Eric also kickstarted an Autoresearch loop on his project. And it was very interesting to discuss which parts of AI research LLMs can already automate pretty well (implementing and running experiments, optimizing hyperparameters) and which they still struggle with (choosing the right question to investigate next, escaping research dead ends). Informative to all the recent discussion about when we should expect an intelligence explosion, and what it would look like from the inside.
Watch on YouTube. Read the transcript.
And check out the flashcards I wrote to retain the insights.
Sponsors
* Cursor‘s agent SDK let me build a pipeline to generate flashcards for this episode. For each card, I had an agent read the transcript, ingest blackboard screenshots, generate an SVG visual, and run everything through a critic. A durable agent is much better at this kind of work than a chain of LLM calls, and Cursor’s SDK made it easy. Check out the cards at flashcards.dwarkesh.com and get started with the SDK at cursor.com/dwarkesh
* Jane Street gave me a real deep-dive tour of one of their datacenters. I got to ask a bunch of questions to Ron Minsky, who co-leads Jane Street’s tech group, and Dan Pontecorvo, who runs Jane Street’s physical engineering team. They were willing to literally pull up the floorboards and take out racks to explain how everything works. Check out the full tour at janestreet.com/dwarkesh
Timestamps
(00:00:00) – Basics of Go
(00:08:17) – Monte Carlo Tree Search
(00:32:04) – What the neural network does
(01:00:33) – Self-play
(01:25:38) – Alternative RL approaches
(01:45:47) – Why doesn't MCTS work for LLMs
(02:01:09) – Off-policy training
(02:12:02) – RL is even more information inefficient than you thought
(02:22:16) – Automated AI researchers
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com
- AlphaGo
- Eric Jang
- An early AI program that became the first to beat a professional human player at the complex game of Go.
- primitives of intelligence
- Eric Jang
- The basic building blocks or fundamental components that make up intelligence, like problem-solving and learning.
- reinforcement learning (RL)
- Eric Jang
- A type of AI learning where a program learns by trying actions and receiving rewards or penalties, similar to training a pet.
- credit assignment problem
- Eric Jang
- The difficulty in AI of figuring out which specific action in a long sequence led to a good or bad outcome.
1 h 371 h 201 h 37AI researchers debate how close we are to recursive self-improvement
WithJohn Schulman, Beren Millidge, Charlie O’Neill
2 h 21
55 min
1 h
2 h 24Questions people ask
- What is the main idea of the episode?
- The episode explains how to build AlphaGo from scratch using modern AI tools and discusses how its core principles can inform the development of future general AIs.
- How does AlphaGo relate to current AI research?
- AlphaGo's methods for search, learning from experience, and self-play offer insights into reinforcement learning in LLMs and how AI might learn more effectively, potentially sidestepping credit assignment problems.
Page generated Oct 19, 2026. Summaries are Subvolts' own; the audio, artwork and notes belong to Dwarkesh Patel.