Top news
Investing in the public benefit OfficialAnthropic NewsOct 18Popular Musk Pledges SpaceX Shares to ‘Every Kid in America’ WebBloomberg TechnologyOct 19Popular Websites are closing their doors to AI agents WebThe Verge AIOct 17 This is the new ChatGPT Voice, powered by GPT Live OfficialOpenAI on YouTubeOct 18 How AI giants will reshape global philanthropy WebAxios AIOct 19 Gemini 4 Argon Really Surprised Me WebTwo Minute PapersOct 19 Google Gemini 4 agents just beat Minecraft with no human intervention WebEngadget AIOct 19 Who could it be? 🧐 OfficialGoogle on YouTubeOct 18 Anthropic gives outside AI watchdog “employee-like” access WebArs Technica AIOct 18 What are self-replicating ‘AI worms’, and why is Anthropic worried about them? WebThe Guardian AIOct 18
GlossaryModelsIntermediate

Elo score

Also written: Elo points

A score measuring AI model performance relative to others, adapted from chess rankings.

In plain terms

An Elo score is a way to rank players in competitive games like chess. It shows how likely a player is to win against another player. The higher the score, the better the player.

In AI, this system is used to compare different language models. Instead of chess players, it ranks AI models based on how well they perform on various tasks or how much people prefer their outputs. For example, one AI model might be given an Elo score of 1200, while another gets 1350, meaning the second model is considered better based on the tests.

The expert version

The Elo rating system, originally developed for chess, is adapted to measure the relative performance of AI models. It's a statistical method that estimates the skill level of players (or models) in a zero-sum game, where a win against a higher-rated opponent yields more points than a win against a lower-rated one. This approach is used to benchmark models against each other based on head-to-head comparisons, often through human preference studies or automated evaluations on specific benchmarks.

Why it matters now

Elo scores are appearing more frequently as AI labs focus on head-to-head comparisons of their models. Google Gemini, for instance, uses Elo to demonstrate the performance gains of its newer models like Gemini 3.7 Flash over previous versions or competitors. This reflects a trend towards quantifiable, comparative performance metrics in the competitive AI landscape.

Where it shows up 5 items, newest first
How each company uses it
Google Gemini
Google Gemini uses Elo scores to benchmark and compare the performance of its various models against each other and potentially against external benchmarks.
Related terms

LLM benchmarksHuman preferenceModel comparisonWin rateHead-to-head evaluationPerformance metrics

Often confused with
Accuracy
Accuracy measures the percentage of correct predictions, while Elo mea
BLEU score
BLEU is a metric for evaluating machine translation quality based on
MMLU
MMLU is a broad benchmark measuring knowledge across many subjects, wherea
Origin

The Elo rating system was invented by Arpad Elo, a Hungarian-American physics professor, for the chess community. It was later adapted for other competitive games and now for AI model evaluation. Source ↗

Try this about 5 minutes

You can see Elo scores in action by looking at leaderboards from AI research groups or companies that publish them. For example, search for 'Chatbot Arena Leaderboard' to see how different models are ranked using Elo based on user votes.

Try it: Playbook entries that use thisPlaybook →

The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.

Questions people ask

How is an Elo score calculated for AI?
It's calculated based on the outcomes of pairwise comparisons between models. When one model 'wins' against another (e.g., a human prefers its response), its score increases, and the loser's score decreases, with the magnitude of change depending on the initial score difference.
Is a higher Elo score always better?
Yes, in the context of AI model performance, a higher Elo score indicates that the model is rated as performing better relative to other models in the comparison pool.

First seen on Subvolts Mar 3, 2026. Updated Oct 5, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.