Subvolts take Optimize AI agent compute costs by mixing hardware, not just using expensive GPUs. Dec 2, 2025 - Developers
- Learn how to disaggregate AI workloads across diverse hardware for better performance and cost efficiency.
- Business
- Understand strategies to optimize AI inference costs for agents, moving beyond reliance on expensive GPUs.
This interview offers practical insights into optimizing AI inference costs, a growing concern for companies deploying agents.
Scaling Agentic Inference Across Heterogeneous Compute with Zain Asgar - #757
Audio streams from Sam Charrington. Your place is remembered on this device.
From Subvolts
Summary and analysis by Subvolts editorial, AI-assisted.
Zain Asgar of Gimlet Labs discusses scaling AI agent inference across diverse hardware. He argues against using only high-end GPUs for agents, which consume more tokens. Gimlet Labs disaggregates workloads across GPUs and CPUs, using a three-layer architecture for optimization. The episode covers their approach to mapping models to hardware and optimizing compute kernels.
This episode explains how to run AI agents more cheaply. Instead of using only the most powerful computers, it suggests using a mix of different machines, like older ones and even regular computer processors. This helps save money while still making the AI work well.
This episode is for AI engineers and infrastructure leads grappling with the cost and performance of running large AI agents. As frontier labs like Google Gemini and OpenAI develop more complex, token-intensive agents, efficient inference becomes critical. Gimlet's approach offers a practical strategy for optimizing compute economics by leveraging a mix of hardware, a key challenge as AI adoption grows.
- HostSam Charrington
- GuestZain Asgarco-founder and CEO, Gimlet Labs
- Guest: Zain Asgar, co-founder and CEO of Gimlet Labs
- Show: The TWIML AI Podcast
- Length: 49 minutes
- Core Argument: Running all AI agent workloads on high-end GPUs is unsustainable due to high token consumption.
- Gimlet's Architecture: Three-layer cake: workload disaggregation, compilation layer, LLM-driven kernel optimization.
In this episode, Zain Asgar, co-founder and CEO of Gimlet Labs, joins us to discuss the heterogeneous AI inference across diverse hardware. Zain argues that the current industry standard of running all AI workloads on high-end GPUs is unsustainable for agents, which consume significantly more tokens than traditional LLM applications. We explore Gimlet’s approach to heterogeneous inference, which involves disaggregating workloads across a mix of hardware—from H100s to older GPUs and CPUs—to optimize unit economics without sacrificing performance. We dive into their "three-layer cake" architecture: workload disaggregation, a compilation layer that maps models to specific hardware targets, and a novel system that uses LLMs to autonomously rewrite and optimize compute kernels. Finally, we discuss the complexities of networking in heterogeneous environments, the trade-offs between numerical precision and application accuracy, and the future of hardware-aware scheduling.
The complete show notes for this episode can be found at https://twimlai.com/go/757.
- Agentic Inference
- Zain Asgar
- Running AI systems that can make decisions and take actions on their own.
- Heterogeneous Compute
- Zain Asgar
- Using a mix of different types of computer hardware (like various GPUs and CPUs) for tasks.
- LLM
- Zain Asgar
- Large Language Model, a type of AI that understands and generates human-like text.
- Compute Kernels
- Zain Asgar
- Small, optimized pieces of code that perform specific calculations on computer hardware.
53 min1 h 30Why Jev Is Changing How We Build With AI with Diogo Almeida - #779
WithDiogo Almeida
1 h 081 h 05From Voice Agents to AI Avatars with Alexander Smola - #777
WithAlexander Smola
1 h 04
37 min
1 h 28Pedro Domingos: Tensor Logic Unifies AI Paradigms
WithPedro Domingos
32 minUnlocking AGI: How Life Changes for Everyone w/ Jack Hidary, Salim Ismail & Dave Blundin | EP #213
WithJack Hidary, Salim Ismail, Dave Blundin
Questions people ask
- Why is running AI agents on GPUs alone unsustainable?
- Agents consume significantly more tokens than traditional LLM applications, making the current industry standard of running all AI workloads on high-end GPUs unsustainable for them.
- What is Gimlet's approach to heterogeneous inference?
- Gimlet disaggregates workloads across a mix of hardware, including H100s, older GPUs, and CPUs, to optimize unit economics without sacrificing performance.
Page generated Oct 19, 2026. Summaries are Subvolts' own; the audio, artwork and notes belong to Sam Charrington.