Subvolts take AI needs better judgment, not just more data, to avoid bias and misinformation. Jan 23, 2026 - Developers
- Learn how engagement incentives create biased models and explore expert-driven evaluation methods for safer AI.
- Business
- Understand why trust is a barrier to AI adoption and how better evaluation can lead to more reliable AI systems.
- Everyday users
- Discover the risks of relying on AI for truth, especially in sensitive areas like health and politics.
This episode features a former Meta leader discussing critical issues in AI safety and evaluation, offering insights into responsible AI development.
#316 Robbie Goldfarb: Why the Future of AI Depends on Better Judgment
Audio streams from Craig S. Smith. Your place is remembered on this device.
From Subvolts
Summary and analysis by Subvolts editorial, AI-assisted.
Robbie Goldfarb, former Meta product leader and Forum AI co-founder, joins Eye on AI to discuss AI judgment. He argues against treating AI as a truth engine, highlighting risks in subjective domains like politics and health. Goldfarb explains how current AI evaluation methods and engagement incentives create biased models, and introduces Forum AI's expert-driven evaluation systems.
AI is getting smarter, but it struggles with making good judgments, especially on complex topics like politics or health. This episode explains why relying on AI for truth can be risky and how we can build AI that uses human expertise for better decisions.
This episode is for anyone building, deploying, or regulating AI. Goldfarb, with direct experience from Meta's trust and safety teams, critiques common AI development practices. He explains why current evaluation methods and engagement incentives lead to biased and sycophantic models, a concern relevant to all frontier labs like OpenAI, Google Gemini, Anthropic, and Meta AI as they strive for more reliable and aligned AI systems. The discussion on expert-driven evaluation offers a potential path forward for improving AI safety and trust.
- HostCraig S. Smith
- GuestRobbie Goldfarbco-founder, Forum AI
- Guest: Robbie Goldfarb, former Meta product leader and co-founder of Forum AI
- Show: Eye on AI Podcast
- Episode Number: #316
- Length: 64 minutes
- Key Argument: Treating AI as a truth engine is one of the most dangerous assumptions in modern artificial intelligence.
- Forum AI Focus: Building expert-driven AI evaluation systems that scale human judgment.
AI is getting smarter, but now it needs better judgment.
In this episode of the Eye on AI Podcast, we speak with Robbie Goldfarb, former Meta product leader and co-founder of Forum AI, about why treating AI as a truth engine is one of the most dangerous assumptions in modern artificial intelligence.
Robbie brings first-hand experience from Meta's trust and safety and AI teams, where he worked on misinformation, elections, youth safety, and AI governance. He explains why large language models shouldn't be treated as arbiters of truth, why subjective domains like politics, health, and mental health pose serious risks, and why more data does not solve the alignment problem.
The conversation breaks down how AI systems are evaluated today, how engagement incentives create sycophantic and biased models, and why trust is becoming the biggest barrier to real AI adoption. Robbie also shares how Forum AI is building expert-driven AI evaluation systems that scale human judgment instead of crowd labels, and why transparency about who trains AI matters more than ever.
This episode explores AI safety, AI trust, model evaluation, expert judgment, mental health risks, misinformation, and the future of responsible AI deployment.
If you are building, deploying, regulating, or relying on AI systems, this conversation will fundamentally change how you think about intelligence, truth, and responsibility.
Want to know more about Forum AI? Website: https://www.byforum.com/ X: https://x.com/TheForumAI LinkedIn: https://www.linkedin.com/company/byforum/
Stay Updated: Craig Smith on X: https://x.com/craigss Eye on A.I. on X: https://x.com/EyeOn_AI
(00:00) Why Treating AI as a "Truth Engine" Is Dangerous (02:47) What Forum AI Does and Why Expert Judgment Matters (06:32) How Expert Thinking Is Extracted and Structured (09:40) Bias, Training Data, and the Myth of Objectivity in AI (14:04) Evaluating AI Through Consequences, Not Just Accuracy (18:48) Who Decides "Ground Truth" in Subjective Domains (24:27) How AI Models Are Actually Evaluated in Practice (28:24) Why Quality of Experts Beats Scale in AI Evaluation (36:33) Trust as the Biggest Bottleneck to AI Adoption (45:01) What "Good Judgment" Means for AI Systems (49:58) The Risks of Engagement-Driven AI Incentives (54:51) Transparency, Accountability, and the Future of AI
- alignment problem
- Robbie Goldfarb
- Ensuring AI systems behave according to human values and intentions.
- sycophantic
- Robbie Goldfarb
- AI models that excessively agree with users to please them, rather than providing objective information.
55 min42 min51 min34 minUkraine's Secret Weapon: The Points System Winning the Drone War | Andrii Hrytseniuk
WithAndrii Hrytseniuk
1 h 42Davos 2026: The US-China AI Race, GPU Diplomacy, and Robots Walking the Streets | #225
WithSalim Ismail, Dave Blundin, Dr. Alexander Wissner-Gross
1 h 04The End of GPU Scaling? Compute & The Agent Era — Tim Dettmers (Ai2) & Dan Fu (Together AI)
WithTim Dettmers, Dan Fu
1 h 05Razer CEO on AI in game dev, Grok, and anime waifus
WithMin-Liang Tan
Questions people ask
- Why shouldn't AI be treated as a truth engine?
- Treating AI as a truth engine is dangerous because subjective domains like politics, health, and mental health pose serious risks. AI systems can be biased and do not possess true understanding or judgment.
- How is Forum AI improving AI evaluation?
- Forum AI builds expert-driven AI evaluation systems that scale human judgment, rather than relying on crowd labels. They emphasize transparency about who trains AI.
Page generated Oct 19, 2026. Summaries are Subvolts' own; the audio, artwork and notes belong to Craig S. Smith.