Test-time compute
The computer power an AI model uses when it is actively working on a task and generating a response.
Test-time compute is the processing power an AI model uses when it's actively working on a task. Think of it like the electricity a calculator uses when you type in numbers and it shows you the answer. The more complex the calculation, the more power it needs.
For AI, this means the resources needed to understand your question and generate a helpful answer. More advanced AI models or more complex requests often require more test-time compute.
Test-time compute refers to the computational resources, primarily floating-point operations (FLOPs) and memory bandwidth, consumed by a trained AI model during the inference phase. This is distinct from training compute, which is used to update model weights. The efficiency of test-time compute is crucial for real-time applications, impacting latency, throughput, and operational costs.
Frontier AI labs are increasingly focused on optimizing AI performance for real-world applications. Innovations in model architecture and efficient inference techniques aim to reduce test-time compute, making powerful AI more accessible and cost-effective. This is essential for deploying models in products like Microsoft Copilot.
- EvoLib: Turning experience into evolving knowledgeMicrosoft · Microsoft Research Blog ·
- Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI Research Scientist Noam BrownOpenAI · No Priors ·
- OpenAI's Dan Roberts: Why AI Can Now Make DiscoveriesOpenAI · The MAD Podcast ·
- Microsoft Copilot
- Microsoft Copilot leverages various AI models, and optimizing their test-time compute is key to providing fast and efficient user experiences.
- Training compute
- The computational resources used to train an AI model, which i
- Model size
- The number of parameters in a model, which influences but does not d
- Hardware acceleration
- Specialized hardware like GPUs or TPUs designed to speed
The term 'compute' in the context of AI has been used since the early days of machine learning to refer to the processing power required. 'Test-time' specifically distinguishes this from the compute used during model training.
Ask a complex question to an AI chatbot like ChatGPT or Google Gemini, then ask a very simple question. Notice if there's a difference in how quickly you get an answer. The more complex question likely required more test-time compute.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
- HeuristicDouble coding agent task completion with docs linksProviding coding agents with a documentation link significantly increases their task completion rate. · 2 min
Questions people ask
- What is test-time compute?
- It's the processing power an AI model uses when it's actively working to understand a prompt and generate a response.
- Why is test-time compute important?
- Lower test-time compute means faster responses, lower costs, and the ability to run AI on less powerful devices.
First seen on Subvolts Jul 30, 2026. Updated Sep 15, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.