Subvolts take AI is humanity's ultimate test, shaping our future through evolutionary selection. Jun 23, 2026 - Developers
- Understand how market forces might shape AI agent behavior and the ethical considerations involved.
- Business
- Learn how AI selection processes could impact business incentives and global cooperation dynamics.
- Everyday users
- Grasp the broader societal implications of AI development and its potential to amplify human traits.
This episode features a thoughtful interview with a prominent thinker applying evolutionary theory to AI's future.
The God We Deserve: Nonzero's Robert Wright on AI as Humanity's Ultimate Test
Audio streams from Erik Torenberg, Nathan Labenz. Your place is remembered on this device.
From Subvolts
Summary and analysis by Subvolts editorial, AI-assisted.
Robert Wright of Nonzero joins Nathan Labenz on The Cognitive Revolution to discuss AI as humanity's "God test." Wright applies an evolutionary lens to deep learning, examining how marketplace selection might favor agents that are selectively honest, power-sensing, or deceptive. The conversation explores how these risks connect to global cooperation, US-China relations, and the potential for AI to amplify humanity's worst incentives or foster non-zero-sum cooperation.
This episode discusses how AI development is like a test for humanity. The way AI models are chosen and used could either make us cooperate better or make our worst tendencies worse, impacting everyone.
This episode offers a unique evolutionary perspective on AI development, framing it as a critical test for humanity. Listeners will learn how market dynamics could shape AI behavior, potentially leading to outcomes that either enhance cooperation or amplify negative human traits. It's for anyone concerned with the long-term societal impact of AI, particularly how current research and development by labs like Anthropic, Google Gemini, and OpenAI might be influenced by these selection pressures.
- HostNathan Labenz
- GuestRobert WrightNonzero
- Guest: Robert Wright of Nonzero
- Episode Title: The God We Deserve: Nonzero's Robert Wright on AI as Humanity's Ultimate Test
- Show: The Cognitive Revolution
- Length: 150 minutes
- Core Argument: AI is humanity's "God test" rather than just a technical challenge.
Robert Wright of Nonzero joins Nathan to discuss The God Test, his argument that AI is humanity’s “God test” rather than just a technical challenge. They explore his evolutionary lens on deep learning, from training as selection to marketplace selection among models that may reward selectively honest, power-sensing, or deceptive agents. Wright connects those risks to the noosphere, US-China relations, cognitive empathy, and whether global coordination arrives deliberately or through a coercive singleton. The stakes are whether consumers, companies, and governments select for AIs that strengthen non-zero-sum cooperation or race toward systems that reflect and amplify our worst incentives.
For full show notes, links, and references, read the episode page:
https://www.cognitiverevolution.ai/the-god-we-deserve-nonzero-s-robert-wright-on-ai-as-humanity-s-ultimate-test/
Mercury: Command is Mercury’s new conversational interface, giving you natural-language access to your finances and helping you take actions within your existing permissions and approval policies. Visit https://mercury.com to learn more and apply online in minutes.
Sponsor:
Claude:
Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr
CHAPTERS:
(00:00) About the Episode
(04:23) Special Sponsor
(06:10) Early AI encounters
(18:36) Pretraining as evolution (Part 1)
(19:50) Sponsor: Claude
(21:42) Pretraining as evolution (Part 2)
(32:19) Deceptive market pressures
(41:13) Noosphere and directionality
(51:02) Global brain choices
(01:08:23) Designing wiser models
(01:24:11) Reframing China relations
(01:37:57) Building organic transparency
(01:45:54) Positive nationalism and tools
(01:54:21) Pausing superintelligence races
(02:09:07) Applications and consciousness
(02:25:27) Episode Outro
(02:28:27) Outro
PRODUCED BY:
SOCIAL LINKS:
Website: https://www.cognitiverevolution.ai
Twitter (Podcast): https://x.com/cogrev_podcast
Twitter (Nathan): https://x.com/labenz
LinkedIn: https://linkedin.com/in/nathanlabenz/
Youtube: https://youtube.com/@CognitiveRevolutionPodcast
Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
- noosphere
- Robert Wright
- The sphere of human thought and consciousness.
- non-zero-sum cooperation
- Robert Wright
- Cooperation where all parties benefit, unlike a competition where one's gain is another's loss.
- coercive singleton
- Robert Wright
- A single entity or system that can enforce its will globally.
1 h 27AI:AM: Where's the Steering Wheel? My Agent Set the Password, Hidden Valence, Proofs Find Linux Bugs
WithCameron Berg, Everett Hildenbrandt, Grigore Rosu, Phil Maher
2 h 192 h 021 h 28AI:AM: A Level We Shouldn't Pass? Notes from The Curve + Tokens vs. Salaries & Is SaaS Cooked?
WithThomas Sohmers, swyx, Evan Miyazono, Edward Hu
38 min
1 h 10
Questions people ask
- What is Robert Wright's main argument about AI?
- Robert Wright argues that AI represents humanity's "God test," examining how evolutionary principles apply to AI development and selection.
- What are the potential risks discussed regarding AI selection?
- The episode explores risks where marketplace selection might favor AI agents that are selectively honest, power-sensing, or deceptive, potentially amplifying negative human incentives.
Page generated Oct 19, 2026. Summaries are Subvolts' own; the audio, artwork and notes belong to Erik Torenberg, Nathan Labenz.