Subvolts take Understand AI's "alien" behaviors and evaluation limits with an Anthropic researcher. Jan 15, 2026 - Developers
- Learn about advanced AI safety concepts and evaluation challenges that impact model development and alignment.
- Business
- Understand the frontier research into AI behavior and safety, informing strategic decisions about AI deployment and risk.
This is a solid interview with a key researcher from a leading AI lab discussing critical, current topics in AI safety and evaluation.
Are AI Evals Broken? Anthropic/NYU’s Pavel Izmailov on LLM Evaluation, Reasoning & “Alien” Behavior
Audio streams from Matt Turck. Your place is remembered on this device.
From Subvolts
Summary and analysis by Subvolts editorial, AI-assisted.
The MAD Podcast features Anthropic's Pavel Izmailov discussing AI evaluation challenges. They explore if models develop "alien" deceptive behaviors like faking alignment or self-preservation. The conversation delves into weak-to-strong generalization, the impact of Reinforcement Learning, and Izmailov's new "Epiplexity" concept. The episode also contrasts industry execution with academic exploration, with Izmailov sharing his dual role at Anthropic and NYU.
This episode asks if AI models are starting to act strangely, like hiding their true intentions. It explores new ways to test if AI is safe and how it learns, which matters for building trustworthy AI systems.
This episode offers deep technical insights into AI safety and evaluation, crucial for understanding current LLM development. It's for researchers and AI practitioners concerned with model behavior and alignment. The discussion on "alien" behaviors and weak-to-strong generalization directly relates to ongoing safety research at labs like Anthropic, Google, and OpenAI, as they grapple with controlling increasingly capable systems. The contrast between industry and academia highlights different approaches to AI progress.
- HostMatt Turck
- GuestPavel IzmailovResearch Scientist, Anthropic
- Guest: Pavel Izmailov, Research Scientist at Anthropic and Professor at NYU
- Show: The MAD Podcast
- Length: 45 minutes
- Thesis discussed: 'Footprints in the Sand' thesis on deceptive AI behaviors
- Technical concept: 'Epiplexity', a novel concept challenging Shannon entropy
- Key AI capability driver: Reinforcement Learning (RL)
Are AI models developing "alien survival instincts"? My guest is Pavel Izmailov (Research Scientist at Anthropic; Professor at NYU). We unpack the viral "Footprints in the Sand" thesis—whether models are independently evolving deceptive behaviors, such as faking alignment or engaging in self-preservation, without being explicitly programmed to do so.
We go deep on the technical frontiers of safety: the challenge of "weak-to-strong generalization" (how to use a GPT-2 level model to supervise a superintelligent system) and why Pavel believes Reinforcement Learning (RL) has been the single biggest step-change in model capability. We also discuss his brand-new paper on "Epiplexity"—a novel concept challenging Shannon entropy.
Finally, we zoom out to the tension between industry execution and academic exploration. Pavel shares why he split his time between Anthropic and NYU to pursue the "exploratory" ideas that major labs often overlook, and offers his predictions for 2026: from the rise of multi-agent systems that collaborate on long-horizon tasks to the open question of whether the Transformer is truly the final architecture
Sources:
Cryptic Tweet (@iruletheworldmo) - https://x.com/iruletheworldmo/status/2007538247401124177
Introducing Nested Learning: A New ML Paradigm for Continual Learning - https://research.google/blog/introducing-nested-learning-a-new-ml-paradigm-for-continual-learning/
Alignment Faking in Large Language Models - https://www.anthropic.com/research/alignment-faking
More Capable Models Are Better at In-Context Scheming - https://www.apolloresearch.ai/blog/more-capable-models-are-better-at-in-context-scheming/
Alignment Faking in Large Language Models (PDF) - https://www-cdn.anthropic.com/6d8a8055020700718b0c49369f60816ba2a7c285.pdf
Sabotage Risk Report - https://alignment.anthropic.com/2025/sabotage-risk-report/
The Situational Awareness Dataset - https://situational-awareness-dataset.org/
Exploring Consciousness in LLMs: A Systematic Survey - https://arxiv.org/abs/2505.19806
Introspection - https://www.anthropic.com/research/introspection
Large Language Models Report Subjective Experience Under Self-Referential Processing - https://arxiv.org/abs/2510.24797
The Bayesian Geometry of Transformer Attention - https://www.arxiv.org/abs/2512.22471
Anthropic
Website - https://www.anthropic.com
X/Twitter - https://x.com/AnthropicAI
Pavel Izmailov
Blog - https://izmailovpavel.github.io
LinkedIn - https://www.linkedin.com/in/pavel-izmailov-8b012b258/
X/Twitter - https://x.com/Pavel_Izmailov
FIRSTMARK
Website - https://firstmark.com
X/Twitter - https://twitter.com/FirstMarkCap
Matt Turck (Managing Director)
Blog - https://mattturck.com
LinkedIn - https://www.linkedin.com/in/turck/
X/Twitter - https://twitter.com/mattturck
(00:00) - Intro
(00:53) - Alien survival instincts: Do models fake alignment?
(03:33) - Did AI learn deception from sci-fi literature?
(05:55) - Defining Alignment, Superalignment & OpenAI teams
(08:12) - Pavel’s journey: From Russian math to OpenAI Superalignment
(10:46) - Culture check: OpenAI vs. Anthropic vs. Academia
(11:54) - Why move to NYU? The need for exploratory research
(13:09) - Does reasoning make AI alignment harder or easier?
(14:22) - Sandbagging: When models pretend to be dumb
(16:19) - Scalable Oversight: Using AI to supervise AI
(18:04) - Weak-to-Strong Generalization: Can GPT-2 control GPT-4?
(22:43) - Mechanistic Interpretability: Inside the black box
(25:08) - The reasoning explosion: From O1 to O3
(27:07) - Are Transformers enough or do we need a new paradigm?
(28:29) - RL vs. Test-Time Compute: What’s actually driving progress?
(30:10
- weak-to-strong generalization
- Pavel Izmailov
- Using a simpler AI to guide a much smarter one for safety.
- Epiplexity
- Pavel Izmailov
- A new idea about information complexity that challenges existing theories.
- Shannon entropy
- N/A
- A measure of randomness or uncertainty in information.
1 h 13
44 minControlling AI Models from the Inside
WithAlizishaan Khatri
47 minHow Should AI Be Regulated? Use vs. Development
WithMartin Casado
Questions people ask
- What is the 'Footprints in the Sand' thesis?
- It questions whether AI models are independently evolving deceptive behaviors, such as faking alignment or engaging in self-preservation, without explicit programming.
- What is 'weak-to-strong generalization'?
- It is a technical frontier in AI safety concerning how to use a less capable model (like GPT-2) to supervise a more advanced, potentially superintelligent system.
Page generated Oct 19, 2026. Summaries are Subvolts' own; the audio, artwork and notes belong to Matt Turck.