Top news
Investing in the public benefit OfficialAnthropic NewsOct 18Popular Musk Pledges SpaceX Shares to ‘Every Kid in America’ WebBloomberg TechnologyOct 19Popular Websites are closing their doors to AI agents WebThe Verge AIOct 17 This is the new ChatGPT Voice, powered by GPT Live OfficialOpenAI on YouTubeOct 18 How AI giants will reshape global philanthropy WebAxios AIOct 19 Gemini 4 Argon Really Surprised Me WebTwo Minute PapersOct 19 Google Gemini 4 agents just beat Minecraft with no human intervention WebEngadget AIOct 19 Who could it be? 🧐 OfficialGoogle on YouTubeOct 18 Anthropic gives outside AI watchdog “employee-like” access WebArs Technica AIOct 18 What are self-replicating ‘AI worms’, and why is Anthropic worried about them? WebThe Guardian AIOct 18
GlossaryInferenceNew to AI

Inference

Also written: prediction, Inference-time compute, Inference compute, Inference cost, Inference time, Inference latency, Inference optimizations, Inference deflation

Using a trained AI model to make predictions or generate outputs based on new input data.

In plain terms

Inference is what happens when you use a finished AI model to do something. Think of it like asking a student who has studied for a test to answer a new question. The student uses their knowledge to give you an answer.

For example, when you ask ChatGPT to write a poem, it's using its trained model to perform inference. It takes your request (the input) and generates the poem (the output). This is different from training, which is the process of teaching the AI model in the first place.

The expert version

Inference is the process of deploying a trained machine learning model to generate predictions or outputs on new, unseen data. It involves feeding input data into the model and obtaining a result, such as a classification, a generated text, or an image. This stage is distinct from training, where the model's parameters are adjusted based on a dataset. Inference is often computationally less intensive than training but requires efficient execution for real-time applications.

Why it matters now

AI inference is a major focus as companies work to make AI models more efficient and accessible. Recent news highlights efforts in custom silicon, software optimizations, and edge computing to reduce inference costs and latency. This push aims to enable faster, more powerful AI applications across various devices and services.

Where it shows up 54 items, newest first
How each company uses it
Google Gemini
Google Gemini is developing faster voice interactions and edge inference frameworks to bring powerful AI directly to phones and computers.
Microsoft Copilot
Microsoft is investing in custom silicon like Maia chips and infrastructure to improve AI efficiency, speed, and cost for its services, and is also managing risks associated with open-weight model inference.
OpenAI ChatGPT
OpenAI's discussions touch on how post-training methods are crucial for unlocking AI value, particularly for inference.
Related terms

Model TrainingLLMAI HardwareLatencyThroughputDeployment

Often confused with
Model Training
Training is the process of teaching an AI model using data, whil
Fine-tuning
Fine-tuning is a form of training that adapts a pre-trained model t
Prompt Engineering
Prompt engineering is about crafting inputs for an AI model,
Origin

The term 'inference' in the context of machine learning originates from statistical inference, the process of drawing conclusions from data. In AI, it specifically refers to the model's act of inferring an output from given input.

Try this about 5 minutes

Ask ChatGPT, Google Gemini, or Claude to write a short story about a robot who loves gardening. Then, ask it to summarize the story in one sentence. This shows the model performing inference to generate and then condense information based on your prompts.

Try it: Playbook entries that use thisPlaybook →

The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.

Questions people ask

Is inference the same as AI?
No, inference is a specific process within AI. It's how a trained AI model is used to perform tasks, but it's not the entire field of AI.
Why is inference important?
Inference is crucial because it's how AI models deliver value to users. Fast and efficient inference makes AI applications practical for everyday use.

First seen on Subvolts Jun 20, 2024. Updated Oct 18, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.