Subvolts take AI evaluation is getting harder. Listen to how OpenAI tests its frontier models. Jun 16, 2026 - Developers
- Learn how advanced AI models are tested, informing your own development and evaluation strategies.
- Business
- Understand the cutting edge of AI assessment, crucial for anticipating future capabilities and market shifts.
This interview features a key researcher discussing the practical challenges of evaluating advanced AI, offering direct insight into OpenAI's research process.
Why Tejal Patwardhan stopped underestimating the models - Episode 21
Audio streams from OpenAI. Your place is remembered on this device.
From Subvolts
Summary and analysis by Subvolts editorial, AI-assisted.
In this OpenAI Podcast episode, Tejal Patwardhan, head of OpenAI's frontier evals team, discusses the challenges of measuring AI progress. She explains why old benchmarks become ineffective as models improve and what new methods are needed to assess advanced AI capabilities, including reasoning, voice, and vision.
Imagine AI models are students taking tests. As they get smarter, old tests become too easy. This episode explains how OpenAI creates new, harder tests to see how smart their AI is really becoming, especially in areas like understanding speech and images.
This episode is for anyone tracking AI development, especially those interested in how leading labs assess their own progress. Patwardhan's team faces the difficult task of creating tests that keep pace with rapidly improving AI. This offers insight into the practical research challenges at OpenAI and how they are preparing for future AI advancements, relevant to all frontier labs.
- HostAndrew Mayne
- GuestTejal Patwardhanleads OpenAI’s frontier evals team, OpenAI
- Guest: Tejal Patwardhan
- Title: Why Tejal Patwardhan stopped underestimating the models - Episode 21
- Show: The OpenAI Podcast
- Length: 44 minutes
- Guest's Role: leads OpenAI’s frontier evals team
The old tests are getting too easy. Tejal Patwardhan leads OpenAI’s frontier evals team, which is finding new ways to measure and forecast progress as models become more capable. She and host Andrew Mayne discuss why evals matter for research, how benchmarks can break or get gamed, and what models need to be judged on next.
Chapters
00:00:24 Growing up at OpenAI
00:03:10 Why reasoning changed everything
00:06:28 What made o1 surprising
00:11:20 Why old benchmarks stopped working
00:14:45 What makes a good benchmark
00:17:35 Why evals are getting harder
00:22:09 Measuring voice and vision models
00:24:48 Testing models on real science
00:33:23 How OpenAI tracks frontier progress
00:40:47 What AI means for work
Hosted on Acast. See acast.com/privacy for more information.
- frontier evals
- Tejal Patwardhan
- tests for the most advanced AI models
- benchmarks
- Tejal Patwardhan
- standard tests used to compare AI performance
41 minWhat racing reveals about working with AI - Episode 22
WithJoyce Ruffell, Chase Holden
41 minHow a reasoning model cracked an 80-year-old math problem - Episode 20
WithAlexander Wei, Hongxun Wu, Lijie Chen
29 minInside image generation’s Renaissance moment - Episode 19
WithAdele Li, Kenji Hata
38 minWhy AI needs a new kind of supercomputer network - Episode 18
WithMark Handley, Greg Steinbrecher
31 minQuestions people ask
- What is the role of Tejal Patwardhan?
- Tejal Patwardhan leads OpenAI's frontier evals team, responsible for finding new ways to measure and forecast AI model progress.
- Why are old benchmarks no longer sufficient?
- Old benchmarks are becoming too easy for current AI models, making them ineffective for measuring true progress and capability.
Page generated Oct 19, 2026. Summaries are Subvolts' own; the audio, artwork and notes belong to OpenAI.