Subvolts take Train AI on AI: learn how existing models can become the next training data source. Jul 27, 2026 - Developers
- Learn about a novel method to train AI models that could reduce computational costs and development time.
- Business
- Understand a potential future direction for AI development that may lower the expense of creating specialized AI solutions.
This interview presents a novel research direction for AI training that could significantly impact future model development and costs.
Why Models Are AI’s Next Training Dataset with Damian Borth - #772
Audio streams from Sam Charrington. Your place is remembered on this device.
From Subvolts
Summary and analysis by Subvolts editorial, AI-assisted.
Damian Borth joins The TWIML AI Podcast to discuss a new approach to AI training. Instead of relying solely on massive datasets, Borth's research treats previously trained neural networks as data. This "weight space learning" method aims to reduce the cost and complexity of developing specialized AI models by learning from existing optimized networks.
Imagine instead of teaching a student from scratch with books, you teach them by having them learn from other already smart students. This episode discusses training AI models by having them learn from other AI models that have already been trained, making AI development cheaper and faster.
This episode explores a potential paradigm shift in AI development, moving beyond ever-larger datasets to leverage existing trained models. It is for AI researchers and developers interested in more efficient model training. This approach could offer a way for frontier labs like Google Gemini, OpenAI, and Anthropic to reduce the immense computational costs associated with training foundation models, potentially accelerating the creation of specialized AI applications.
- HostSam Charrington
- GuestDamian Borthprofessor of AI and machine learning, University of St. Gallen
- Guest: Damian Borth
- Title: Why Models Are AI’s Next Training Dataset with Damian Borth - #772
- Show: The TWIML AI Podcast
- Length: 47 minutes
- Research approach: Weight space learning treats trained neural networks as data.
For more than a decade, AI has advanced by training ever-larger models on ever-larger datasets. But as high-quality training data becomes harder to find and pretraining grows increasingly expensive, researchers are looking for new ways to keep foundation models improving.
In this episode, Damian Borth, professor of AI and machine learning at the University of St. Gallen, argues we’ve been overlooking an important source of knowledge: the models we’ve already trained. His group’s work on weight space learning treats trained neural networks themselves as data, learning from the distilled results of millions of GPU hours of optimization rather than starting from raw data each time.
We explore what it means to build foundation models of neural networks, how knowledge can be transferred across architectures and domains, why this approach could dramatically reduce the cost of developing specialized models, and whether future AI systems may be trained on collections of existing models instead of ever-growing datasets.
🗒️ Full show notes: https://twimlai.com/go/772.
- foundation models
- Damian Borth
- large, general-purpose AI models that can be adapted for many tasks
- weight space learning
- Damian Borth
- a method where trained AI models themselves are used as data to train new models
53 min1 h 30Why Jev Is Changing How We Build With AI with Diogo Almeida - #779
WithDiogo Almeida
1 h 081 h 05From Voice Agents to AI Avatars with Alexander Smola - #777
WithAlexander Smola
Questions people ask
- What is weight space learning?
- Weight space learning treats trained neural networks themselves as data, learning from the distilled results of optimization rather than starting from raw data.
- Why is this new approach to AI training needed?
- High-quality training data is becoming harder to find, and pretraining foundation models is increasingly expensive.
Page generated Oct 19, 2026. Summaries are Subvolts' own; the audio, artwork and notes belong to Sam Charrington.