Tokens per watt
Also written: T/W
Measures how many AI processing tasks an AI model can complete using one watt of electricity.
Tokens per watt is a way to measure how efficient an AI model is. Think of it like miles per gallon for a car. A car that gets more miles per gallon is more efficient. Similarly, an AI model with more tokens per watt can do more work using less energy.
For example, if Model A can process 100 tokens with one watt of power, and Model B can process 200 tokens with one watt, Model B is twice as efficient. This matters because running AI models, especially large ones, uses a lot of electricity.
Tokens per watt quantifies the energy efficiency of AI models, specifically during inference. It represents the number of tokens a model can process (generate or understand) per unit of electrical power consumed (typically measured in watts). Higher tokens per watt indicate greater computational efficiency, meaning less energy is required to perform a given amount of AI work. This metric is crucial for optimizing deployment costs and reducing the environmental impact of large-scale AI operations.
AI models are becoming more powerful and complex, leading to increased energy consumption. Companies are focusing on improving efficiency to manage costs and environmental impact, making metrics like tokens per watt crucial for tracking progress.
- Making AI more efficient, from the chip to the power gridMicrosoft · Microsoft Source ·
- Microsoft to expand Maia chip for the AI eraMicrosoft · Microsoft Source ·
- OpenAI’s Compute Chief: We Can’t Build Fast Enough | Sachin KattiOpenAI · The MAD Podcast ·
- Microsoft Copilot
- Microsoft is developing custom silicon and infrastructure aimed at improving AI efficiency, which directly relates to tokens per watt.
- OpenAI ChatGPT
- OpenAI's compute chief highlighted the immense hardware and physical scale required for AI, implying a focus on optimizing energy use for processing tokens.
- Tokens
- The basic units of text (words or parts of words) that AI models process
- Watts
- A unit of power, representing the rate of energy transfer.
The concept of measuring computational efficiency in terms of output per unit of energy is fundamental to computer engineering. The specific "tokens per watt" metric gained prominence as large language models became more prevalent and their energy demands became a significant factor.
While you cannot directly measure tokens per watt yourself, you can observe the impact of efficiency by comparing how quickly different AI models respond to the same prompt. Faster responses with less perceived lag can indicate more efficient processing, which is often linked to better energy usage.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
- HeuristicDouble coding agent task completion with docs linksProviding coding agents with a documentation link significantly increases their task completion rate. · 2 min
Questions people ask
- Why is tokens per watt important?
- It's important because it directly impacts the cost and environmental footprint of running AI models. More efficient models use less electricity, making AI cheaper to operate and more sustainable.
- Does a higher tokens per watt mean a better AI model?
- Not necessarily. Tokens per watt measures efficiency, not necessarily the quality or capability of the AI's output. A highly efficient model might still not be as good at complex tasks as a less efficient but more powerful one.
First seen on Subvolts Jul 16, 2026. Updated Oct 14, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.