Subvolts take Master AI inference engineering to build faster, more efficient AI products. Apr 30, 2026 - Developers
- Learn practical techniques for optimizing AI model performance and efficiency in production environments.
- Business
- Understand how efficient inference engineering leads to better AI product performance and cost savings.
This episode offers a deep dive into inference engineering, a vital but often complex area for AI deployment.
How to Engineer AI Inference Systems with Philip Kiely - #766
Audio streams from Sam Charrington. Your place is remembered on this device.
From Subvolts
Summary and analysis by Subvolts editorial, AI-assisted.
Philip Kiely, head of AI education at Baseten, joins The TWIML AI Podcast to discuss engineering AI inference systems. The episode unpacks why inference is a critical AI workload, blending GPU programming, research, and distributed systems. Kiely explains how understanding inference "knobs" like batching and quantization enables better products and service level agreements, and traces the journey from APIs to dedicated deployments.
This episode explains how to make AI models run quickly and efficiently after they are built. It covers techniques that help AI systems respond faster, which is important for many new AI applications people are starting to use.
This episode is for engineers and product managers building and deploying AI models. It demystifies inference engineering, a crucial but often overlooked part of making AI work efficiently in production. Understanding concepts like batching, quantization, and specialized runtimes like vLLM can help teams optimize performance and cost. This knowledge is directly relevant as frontier labs like Google Gemini and OpenAI focus on making their models more accessible and efficient for real-world applications, impacting enterprise and consumer use cases.
- HostSam Charrington
- GuestPhilip Kielyhead of AI education, Baseten
- Episode Title: How to Engineer AI Inference Systems with Philip Kiely - #766
- Show: The TWIML AI Podcast
- Length: 55 minutes
- Guest: Philip Kiely, head of AI education at Baseten
- Research to Production Time: Can move in hours, not months
In this episode, Philip Kiely, head of AI education at Baseten, joins us to unpack the fast-evolving discipline of inference engineering. We explore why inference has become the stickiest and most critical workload in AI, how it blends GPU programming, applied research, and large-scale distributed systems, and where the line sits between inference and model serving. Philip shares how research-to-production can move in hours, not months, and why understanding “the knobs” of inference—batching, quantization, speculation, and KV cache reuse—lets teams design better products and SLAs. We trace the inference maturity journey from closed APIs to dedicated deployments and in-house platforms, discuss GPU lifecycles, and survey today’s runtime landscape, including vLLM, SGLang, and TensorRT LLM. Finally, we look ahead to agents and multimodality, making the case for specialized, workload-specific runtimes when performance and efficiency matter most.
The complete show notes for this episode can be found at https://twimlai.com/go/766.
- inference
- Philip Kiely
- running a trained AI model to get a result
- batching
- Philip Kiely
- processing multiple requests together for efficiency
- quantization
- Philip Kiely
- reducing the precision of numbers in a model to make it smaller and faster
- KV cache reuse
- Philip Kiely
- reusing previously computed information to speed up AI model responses
53 min1 h 30Why Jev Is Changing How We Build With AI with Diogo Almeida - #779
WithDiogo Almeida
1 h 081 h 05From Voice Agents to AI Avatars with Alexander Smola - #777
WithAlexander Smola
Questions people ask
- What is inference engineering?
- Inference engineering focuses on making AI models run efficiently and effectively in production, blending GPU programming, applied research, and distributed systems.
- What are key techniques for optimizing AI inference?
- Key techniques include batching, quantization, speculation, and KV cache reuse, which help teams design better products and meet service level agreements.
Page generated Oct 19, 2026. Summaries are Subvolts' own; the audio, artwork and notes belong to Sam Charrington.