FLOPs
Floating Point Operations, a measure of computer performance for calculations, especially in AI.
FLOPs stands for Floating Point Operations. It is a way to measure how much math a computer can do. Think of it like measuring how many steps a runner can take in a minute. For AI, especially training large models, computers need to do a lot of math very quickly.
This is important because bigger and more complex AI models require more calculations. Companies building these models need powerful computers that can perform trillions of FLOPs to train them effectively. More FLOPs generally means faster training and the ability to build more advanced AI.
FLOPs (Floating Point Operations) is a standard metric for measuring computational performance, particularly for scientific and AI workloads. It quantifies the number of floating-point arithmetic operations (addition, subtraction, multiplication, division) a processor can perform per second. Common units are FLOPS (per second), GFLOPS (gigaFLOPS, 10^9), TFLOPS (teraFLOPS, 10^12), PFLOPS (petaFLOPS, 10^15), and EFLOPS (exaFLOPS, 10^18).
In the context of AI, FLOPs are crucial for both training and inference. Training deep learning models, especially large language models (LLMs), is computationally intensive and often measured in PetaFLOP-days or ExaFLOP-days. Inference, while less demanding per query, still requires significant FLOPs for real-time performance. The efficiency of hardware in delivering FLOPs, along with algorithmic improvements, drives progress in AI capabilities.
FLOPs, a measure of computing performance, is relevant as frontier AI labs like Google and Anthropic discuss the immense computational power needed for training and running advanced AI models. Discussions around compute consolidation and efficiency, such as Google's focus on 'goodput' over raw FLOPS, highlight the ongoing importance of understanding and optimizing computational resources in the AI race.
- Google's AI Infrastructure Chief, Amin Vahdat, on the Physics & Economics of Frontier AIGoogle · Training Data ·
- Dylan Patel – Two labs will soon control most of the world's workforceAnthropic · Dwarkesh Patel ·
- Dylan Patel – Anthropic & OpenAI will have most of the world’s compute by 2028Anthropic · Dwarkesh Podcast ·
- Anthropic Claude
- Anthropic is mentioned in relation to the potential consolidation of AI compute power, implying their role in the large-scale computational demands of frontier AI.
- Google Gemini
- Google's AI infrastructure chief discussed 'goodput' as being more important than FLOPS for frontier AI, indicating a focus on efficient computation rather than just raw processing power.
- MACs
- Multiply-Accumulate operations, a related but distinct measure of computat
- TOPS
- Tera Operations Per Second, a measure often used for integer operations, c
- Bandwidth
- The rate at which data can be transferred, which is also critical for
The concept of measuring computational performance using floating-point operations has been standard in high-performance computing for decades. The term FLOPs became particularly prominent in AI with the rise of deep learning and the need to quantify the computational demands of training large neural networks. Source ↗
You can observe the impact of FLOPs by comparing the speed of different AI models performing the same task. For example, try asking ChatGPT and another AI chatbot the same complex question and note any differences in response time. Faster responses often indicate more efficient use of computational resources (FLOPs).
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
- HeuristicDouble coding agent task completion with docs linksProviding coding agents with a documentation link significantly increases their task completion rate. · 2 min
Questions people ask
- What is a floating point operation?
- A floating point operation is a mathematical calculation involving numbers that can have a decimal point, like 3.14 or 0.001. These are fundamental to the complex math used in AI.
- Why are FLOPs important for AI?
- AI models, especially large ones, require billions or trillions of calculations to learn from data and generate responses. FLOPs measure the speed at which computers can perform these calculations, directly impacting how quickly AI can be trained and how complex it can become.
- Are more FLOPs always better?
- More FLOPs generally mean more computational power, which is needed for larger, more capable AI models. However, efficiency in using those FLOPs through better algorithms and hardware is also crucial.
First seen on Subvolts Aug 25, 2026. Updated Oct 13, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.