Subvolts take Understand AI agency and energy-based models with Dr. Jeff Beck's deep dive. Jan 25, 2026 - Developers
- Learn about energy-based models and their differences from standard neural networks for potential implementation.
- Business
- Understand the philosophical and technical definitions of AI intelligence and agency to guide future AI strategy.
- Everyday users
- Grasp complex AI concepts like agency and model types in simpler terms to better understand AI's potential.
This episode features a solid interview with a researcher discussing fundamental AI concepts like agency and energy-based models.
VAEs Are Energy-Based Models? [Dr. Jeff Beck]
Audio streams from Machine Learning Street Talk (MLST). Your place is remembered on this device.
From Subvolts
Summary and analysis by Subvolts editorial, AI-assisted.
Machine Learning Street Talk features Dr. Jeff Beck discussing agency and intelligence. Beck argues mathematically, agents and rocks are similar, with sophistication defining intelligence. The episode explores the black box problem of agency and explains energy-based models (EBMs), contrasting them with standard neural networks by highlighting EBMs' optimization of both weights and internal states. Beck also proposes a surprising evolutionary link between the brain and the nose.
This episode explores what makes something truly intelligent, suggesting even a rock could be considered intelligent from a simple math view. The real difference is how complex its thinking is. It also explains a type of AI model called energy-based models, which work differently than typical ones.
This episode offers a deep dive into the philosophical and mathematical underpinnings of AI agency and intelligence. It's for researchers and AI enthusiasts grappling with defining consciousness and sophisticated AI behavior. Understanding energy-based models provides insight into alternative AI architectures beyond standard neural nets. This connects to frontier labs' ongoing work in developing more capable and potentially conscious AI systems, pushing the boundaries of what machines can understand and do.
- HostTim Scarfe
- GuestJeff Beck
- Guest: Dr. Jeff Beck
- Show: Machine Learning Street Talk
- Length: 47 minutes
- Core Argument: Mathematically, there's no structural difference between an agent and a rock; sophistication defines intelligence.
- EBM Distinction: EBMs optimize weights and internal states, unlike traditional networks that only optimize weights.
What makes something truly *intelligent?* Is a rock an agent? Could a perfect simulation of your brain actually *be* you? In this fascinating conversation, Dr. Jeff Beck takes us on a journey through the philosophical and technical foundations of agency, intelligence, and the future of AI.
Jeff doesn't hold back on the big questions. He argues that from a purely mathematical perspective, there's no structural difference between an agent and a rock – both execute policies that map inputs to outputs. The real distinction lies in *sophistication* – how complex are the internal computations? Does the system engage in planning and counterfactual reasoning, or is it just a lookup table that happens to give the right answers?
*Key topics explored in this conversation:*
*The Black Box Problem of Agency* – How can we tell if something is truly planning versus just executing a pre-computed response? Jeff explains why this question is nearly impossible to answer from the outside, and why the best we can do is ask which model gives us the simplest explanation.
*Energy-Based Models Explained* – A masterclass on how EBMs differ from standard neural networks. The key insight: traditional networks only optimize weights, while energy-based models optimize *both* weights and internal states – a subtle but profound distinction that connects to Bayesian inference.
*Why Your Brain Might Have Evolved from Your Nose* – One of the most surprising moments in the conversation. Jeff proposes that the complex, non-smooth nature of olfactory space may have driven the evolution of our associative cortex and planning abilities.
*The JEPA Revolution* – A deep dive into Yann LeCun's Joint Embedding Prediction Architecture and why learning in latent space (rather than predicting every pixel) might be the key to more robust AI representations.
*AI Safety Without Skynet Fears* – Jeff takes a refreshingly grounded stance on AI risk. He's less worried about rogue superintelligences and more concerned about humans becoming "reward function selectors" – couch potatoes who just approve or reject AI outputs. His proposed solution? Use inverse reinforcement learning to derive AI goals from observed human behavior, then make *small* perturbations rather than naive commands like "end world hunger."
Whether you're interested in the philosophy of mind, the technical details of modern machine learning, or just want to understand what makes intelligence *tick,* this conversation delivers insights you won't find anywhere else.
---
TIMESTAMPS:
00:00:00 Geometric Deep Learning & Physical Symmetries
00:00:56 Defining Agency: From Rocks to Planning
00:05:25 The Black Box Problem & Counterfactuals
00:08:45 Simulated Agency vs. Physical Reality
00:12:55 Energy-Based Models & Test-Time Training
00:17:30 Bayesian Inference & Free Energy
00:20:07 JEPA, Latent Space, & Non-Contrastive Learning
00:27:07 Evolution of Intelligence & Modular Brains
00:34:00 Scientific Discovery & Automated Experimentation
00:38:04 AI Safety, Enfeeblement & The Future of Work
---
REFERENCES:
Concept:
[00:00:58] Free Energy Principle (FEP)
https://en.wikipedia.org/wiki/Free_energy_principle
[00:06:00] Monte Carlo Tree Search
https://en.wikipedia.org/wiki/Monte_Carlo_tree_search
Book:
[00:09:00] The Intentional Stance
https://mitpress.mit.edu/9780262540537/the-intentional-stance/
Paper:
[00:13:00] A Tutorial on Energy-Based Learning (LeCun 2006)
http://yann.lecun.com/exdb/publis/pdf/lecun-06.pdf
[00:15:00] Auto-Encoding Variational Bayes (VAE)
https://arxiv.org/abs/1312.6114
[00:20:15] JEPA (Joint Embedding Pred
- agency
- Dr. Jeff Beck
- the capacity of an entity to act in an environment.
- energy-based models
- Dr. Jeff Beck
- a type of AI model that learns by assigning low energy to correct configurations and high energy to incorrect ones.
- Bayesian inference
- Dr. Jeff Beck
- a statistical method that uses probability to update beliefs as new evidence becomes available.
48 minWhat Most People Get Wrong About Evolution | Akarsh Kumar
WithAkarsh Kumar
1 h 101 h 14Who Checks a Proof No Human Can Read? — Leo de Moura
WithLeo de Moura
44 minSame guests, other shows
1 h 17Questions people ask
- How does Dr. Beck define intelligence and agency?
- Dr. Beck argues that mathematically, agents and rocks are similar, both mapping inputs to outputs. The distinction lies in the sophistication of internal computations, such as planning and counterfactual reasoning.
- What is the key difference between energy-based models and standard neural networks?
- Traditional neural networks optimize only weights, while energy-based models optimize both weights and internal states, a distinction that connects to Bayesian inference.
Page generated Oct 19, 2026. Summaries are Subvolts' own; the audio, artwork and notes belong to Machine Learning Street Talk (MLST).