Subvolts take New architecture promises massive speedups for long-context AI models. Oct 7, 2025 - Developers
- Learn about the Power Retention architecture and Vidrial for building optimized GPU kernels.
- Business
- Understand how new architectures can lead to significant speedups and cost reductions in AI model development.
This interview details a novel architecture for significantly improving AI model speed and efficiency.
Recurrence and Attention for Long-Context Transformers with Jacob Buckman - #750
Audio streams from Sam Charrington. Your place is remembered on this device.
From Subvolts
Summary and analysis by Subvolts editorial, AI-assisted.
This TWIML AI Podcast episode features Jacob Buckman, co-founder of Manifest AI, discussing methods for extending transformer context length. They cover bottlenecks in scaling, techniques like windowed and latent space attention, and the Power Retention architecture. The discussion also touches on Manifest AI's open-source projects, Vidrial and PowerCoder, and metrics for evaluating context utility.
This episode explores ways to make AI models "remember" more information at once. They discuss a new method called Power Retention that could make AI much faster at processing long texts or conversations, potentially leading to smarter and more helpful AI.
This episode is for AI researchers and engineers focused on improving transformer efficiency and capabilities. Buckman introduces the Power Retention architecture, promising significant speedups for training and inference, which directly addresses a key challenge for frontier labs like Google Gemini and OpenAI. Understanding these architectural innovations and open-source tools like Vidrial offers practical insights into pushing the boundaries of long-context models, a critical area for developing more capable AI systems.
- HostSam Charrington
- GuestJacob Buckmanco-founder and CEO, Manifest AI
- Guest: Jacob Buckman, co-founder and CEO of Manifest AI
- Episode Title: Recurrence and Attention for Long-Context Transformers with Jacob Buckman - #750
- Show: The TWIML AI Podcast
- Length: 57 minutes
- Power Retention Speedups: Promises speedups of >10x during training and >100x during inference
Today, we're joined by Jacob Buckman, co-founder and CEO of Manifest AI to discuss achieving long context in transformers. We discuss the bottlenecks of scaling context length and recent techniques to overcome them, including windowed attention, grouped query attention, and latent space attention. We explore the idea of weight-state balance and the weight-state FLOP ratio as a way of reasoning about the optimality of compute architectures, and we dig into the Power Retention architecture, which blends the parallelization of attention with the linear scaling of recurrence and promises speedups of >10x during training and >100x during inference. We review Manifest AI’s recent open source projects as well: Vidrial—a custom CUDA framework for building highly optimized GPU kernels in Python, and PowerCoder—a 3B-parameter coding model fine-tuned from StarCoder to use power retention. Our chat also covers the use of metrics like in-context learning curves and negative log likelihood to measure context utility, the implications of scaling laws, and the future of long context lengths in AI applications.
The complete show notes for this episode can be found at https://twimlai.com/go/750.
- Transformers
- Jacob Buckman
- A type of AI model architecture that excels at processing sequential data like text.
- Attention
- Jacob Buckman
- A mechanism in AI models that allows them to focus on the most relevant parts of input data.
- Recurrence
- Jacob Buckman
- A method in AI models where outputs from previous steps are fed back as inputs to the current step, useful for sequences.
- CUDA
- Jacob Buckman
- A parallel computing platform and programming model created by Nvidia for general computing on its graphics processing units (GPUs).
53 min1 h 30Why Jev Is Changing How We Build With AI with Diogo Almeida - #779
WithDiogo Almeida
1 h 081 h 05From Voice Agents to AI Avatars with Alexander Smola - #777
WithAlexander Smola
Same guests, other shows
57 min43 min
1 h 20
47 min45 min#292 Peeyush Ranjan: How Nurix Is Redefining Voice AI for the Enterprise
WithPeeyush Ranjan
Questions people ask
- What is the Power Retention architecture?
- Power Retention blends the parallelization of attention with the linear scaling of recurrence, promising significant speedups for training and inference.
- What open-source projects did Manifest AI release?
- Manifest AI released Vidrial, a custom CUDA framework for GPU kernels in Python, and PowerCoder, a coding model fine-tuned from StarCoder.
Page generated Oct 19, 2026. Summaries are Subvolts' own; the audio, artwork and notes belong to Sam Charrington.