Tokens per second
Also written: t/s, Tokens per second per user, Tokens/sec
Tokens per second measures how fast an AI language model generates text, with tokens as units of text.
Tokens per second is a way to measure how quickly an AI can produce text. Think of it like the speed of a printer, but for words and sentences. The AI breaks down text into small pieces called tokens. These tokens can be whole words, parts of words, or even punctuation.
So, tokens per second tells you how many of these pieces the AI can generate in one second. A higher number means the AI is faster at writing or responding. For example, an AI that generates 100 tokens per second is faster than one that generates 50 tokens per second.
Tokens per second (t/s) quantifies the output generation rate of a large language model (LLM). It measures the number of tokens, which are discrete units of text (e.g., sub-words, words, punctuation) that the model processes or generates, within a one-second interval. This metric is crucial for evaluating the responsiveness and throughput of LLMs, particularly in interactive applications like chatbots or real-time code generation. Higher t/s generally correlates with a better user experience and increased efficiency for tasks requiring rapid text production.
Frontier AI labs are racing to improve inference speed, the rate at which models generate output. This focus on tokens per second is driven by the need for real-time applications like coding assistance and in-car AI agents. Companies are investing in specialized hardware and new model tiers to achieve these speed gains.
- Coding for 40 tokens/sec on a Pixel 10 📱Google · Google Cloud Tech on YouTube ·
- OpenAI feels the frontier need for speedOpenAI · The Rundown AI ·
- Previewing Ultrafast mode: GPT-5.6 Sol at up to 14X the speedOpenAI · OpenAI News ·
- The Biggest Chip Ever Built — Why OpenAI Runs On It | Cerebras CEO Andrew FeldmanOpenAI · The MAD Podcast ·
- Introducing GPT-5.3-Codex-SparkOpenAI · OpenAI News ·
- Google Gemini
- Google Gemini is exploring edge AI applications that require fast token generation for real-time driver coaching.
- OpenAI ChatGPT
- OpenAI is actively working to increase tokens per second for its models, introducing new tiers like Ultrafast and optimizing for speed with specialized hardware.
- Latency
- Latency measures the delay before output begins, while tokens per secon
- Throughput
- Throughput is a broader measure of work done over time, of which tok
- Words per minute
- Words per minute is a simpler measure of text speed, but token
The concept of tokens as fundamental units for language processing originated with early natural language processing research. The measurement of tokens per second became prominent with the rise of large language models and the need to quantify their generation speed.
When using a chatbot like ChatGPT, observe how quickly it responds to your prompts. Try asking it to write a short story or explain a complex topic. The speed at which the text appears on your screen is a direct reflection of its tokens per second capability.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
- HeuristicDouble coding agent task completion with docs linksProviding coding agents with a documentation link significantly increases their task completion rate. · 2 min
Questions people ask
- What is a token?
- A token is a piece of text, like a word, part of a word, or punctuation, that an AI model uses to understand and generate language.
- Why is tokens per second important?
- It tells you how fast an AI can generate text, which affects how responsive and useful it is for tasks that require quick answers or content creation.
First seen on Subvolts Feb 12, 2026. Updated Oct 13, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.